Anyone using LTFS with LTO-5?

[D Fuller]

Hopefully, with the new feature set in BRU PE, LTFS will become a useful addition to our customers' archival needs.

Tim, does LTFS work with LTO-4 or only with LTO-5?

 

[Tim Jones]

LTFS is LTO-5 ONLY

LTFS is LTO-5 ONLY

Unfortunately, LTFS is LTO-5 and later only. It requires that the drive and media support partitioning and LTO drives prior to LTO-5 don't offer that feature.

 

[Mark L. Pederson]

Several LTO 5 LTFS options to look at this year at NAB.

 

[Tai Wah Lim]

Several LTO 5 LTFS options to look at this year at NAB.

Where Mark? Lim

 

[Tim Jones]

Hi Lim,

At NAB, we (TOLIS Group), Cache-A, SGL, and a few others will be demonstrating products at their booths. Our offering incorporates the LTFS volume support directly into our existing BRU PE workflow.

If you're going to be at NAB, be sure to stop by and say high (SL11909).

 

[Chris Parker]

any update on LTFS here? has BRU released their update to incorporate this into their system?

 

[M Most]

Just FYI.....

LTFS was developed and released under the GNU GPL, and code that implements it is freely available from a number of vendors, including Quantum, HP, IBM, and others. You can download precompiled binaries for all three major OS's directly from Quantum's website, as well as other places. You don't need to wait for support in commercial backup software unless you want that.

More FYI......

The company I work for has already implemented LTFS for backup of a Sony feature shooting on Alexa using ArriRaw, using the publicly available software. It has been working quite well, all tapes are being verified and so far, so good.

 

[Joe Carney]

Just FYI.....

LTFS was developed and released under the GNU GPL, and code that implements it is freely available from a number of vendors, including Quantum, HP, IBM, and others. You can download precompiled binaries for all three major OS's directly from Quantum's website, as well as other places. You don't need to wait for support in commercial backup software unless you want that.

More FYI......

The company I work for has already implemented LTFS for backup of a Sony feature shooting on Alexa using ArriRaw, using the publicly available software. It has been working quite well, all tapes are being verified and so far, so good.

Thanks Mike, I'm getting ready to purchase a Mac Tower for feature work. This info will help.

 

[Tim Jones]

Can you share more details of your success?

Can you share more details of your success?

The company I work for has already implemented LTFS for backup of a Sony feature shooting on Alexa using ArriRaw, using the publicly available software. It has been working quite well, all tapes are being verified and so far, so good.

Would you mind sharing your verification process? We've tried a number of methods and the results have always been spotty at best. Simply performing an "ls -l" or a "DIR /S" does not read the file data, only the cached index entries. Everything else (diff, xcopy, etc.) are taking a very long time compared to non-LTFS tape operations using tar or bru.

Also, how do you handle the timeouts involved when you open a folder on the tape. This is the main thing that we are focusing on resolving for customers. We created one tape with 28K DXF files and simply opening the folder on the tape on the Mac took over 18 minutes :Yawn:.

Also, have you tried mounting the tapes on another platform? I'm really curious as we're getting scattered reports from the field of failures and 'ltfsck' runs that are taking multiple hours with unsuccessful completions.

Tim

 

[M Most]

Would you mind sharing your verification process? We've tried a number of methods and the results have always been spotty at best. Simply performing an "ls -l" or a "DIR /S" does not read the file data, only the cached index entries. Everything else (diff, xcopy, etc.) are taking a very long time compared to non-LTFS tape operations using tar or bru.

Every frame is being restored and checksums are being done for additional confirmation. I've been told that since the picture started (its been shooting for over a month now) the only error that was discovered was one pixel on one frame. And as I recall that shot was B-negative. as far as operating systems go, we are running on CentOS and the Filmlight variation of that distro (FLOS). it does not require Red Hat Enterprise. I don't know what Sony is using for restoration, but since they're a Baselight centric facility, it's a pretty good bet they're using what we're using. We've now implemented LTFS on a second show, a rather large television series shooting in Australia, also with no known problems. I don't know what they're using over at the studio (it's not Sony) for restoration, but so far, so good. As for speed, we seem to be averaging about 130-140MB/sec, writing from a reasonably robust SAN through a Linux host via SAS. I don't really feel comfortable giving more exact details as to the show involved or the studios, but hopefully what I've already said will serve to let people here know that LTFS is alive, well, working, on real projects, and being deployed very successfully. Now, this should not in any way imply that commercial products that support it are not worthwhile. To the contrary, most people in the film industry are not Linux jocks, are probably not comfortable with command line tools, aren't very good at both hardware and software troubleshooting when something does go wrong, and for these and other reasons greatly benefit from GUI driven tools that abstract all of the underlying geekiness. So if it sounds like I am not supporting Bru and other software vendors who are planning to implement LTFS within their programs, that's not the case. I applaud their efforts in bringing good tools to a wider audience and for simplifying things so that they can be used by everyone.

 

[Tim Jones]

Every frame is being restored and checksums are being done for additional confirmation. I've been told that since the picture started (its been shooting for over a month now) the only error that was discovered was one pixel on one frame. And as I recall that shot was B-negative.

Aha, that's overhead most of our users wouldn't accept. With BSDtar and BRU, the verify pass doesn't require additional disk space (in your case, basically doubling the storage requirement for any given job).

As far as operating systems go, we are running on CentOS and the Filmlight variation of that distro (FLOS). it does not require Red Hat Enterprise.

We've seem similar success on Ubuntu, Fedora, and OpenSuSE. We did need to recompile and install the kernels, but that's old news to my team. However, that might not be an option for non-guru users in the field.

I don't know what Sony is using for restoration, but since they're a Baselight centric facility, it's a pretty good bet they're using what we're using.

I'm not surprised in that case. We've not seen issues with like to like (i.e.: Linux to Linux), the reports we've received have been Mac to Windows or Windows to Linux.

We've now implemented LTFS on a second show, a rather large television series shooting in Australia, also with no known problems. I don't know what they're using over at the studio (it's not Sony) for restoration, but so far, so good. As for speed, we seem to be averaging about 130-140MB/sec, writing from a reasonably robust SAN through a Linux host via SAS.

We see the same on Linux, it's the Mac and Windows platforms where the write speeds fall off depending on the number of write processes that you use to fill the tape (a single large write is fast, multiple small writes take more and more time as you add data to an existing tape).

I don't really feel comfortable giving more exact details as to the show involved or the studios, but hopefully what I've already said will serve to let people here know that LTFS is alive, well, working, on real projects, and being deployed very successfully.

Thanks, you've answered my questions well. The entities involved don't really make a difference in how the technology works :smile5:

Now, this should not in any way imply that commercial products that support it are not worthwhile. To the contrary, most people in the film industry are not Linux jocks, are probably not comfortable with command line tools, aren't very good at both hardware and software troubleshooting when something does go wrong, and for these and other reasons greatly benefit from GUI driven tools that abstract all of the underlying geekiness.

And there-in lies the rub for the vast majority of users. Even on the Mac, you still currently need to understand the Terminal and the typing of sometimes very long commands to create, mount, and manage LTFS volumes. That's why the commercial vendors are stepping up to provide non-guru tools for LTFS use.

So if it sounds like I am not supporting Bru and other software vendors who are planning to implement LTFS within their programs, that's not the case. I applaud their efforts in bringing good tools to a wider audience and for simplifying things so that they can be used by everyone.

Thanks for the additional input on your current processes. Hopefully our, and others', work will further simplify what you're doing in the future.

 

[Michael Lindsay]

LTFS support is a no-cost update for all BRU PE users.

TOLIS Group will be showing it off at our booth at NAB (SL11909) and shipping BRU PE 2.3.0 either the week of or the week after NAB.

.

Any news?

Michael Lindsay

 

[Tim Jones]

BRU PE 2.3.0 Now Available

BRU PE 2.3.0 Now Available

Hi Folks,

We've beaten the latest Lion into submission (10.7.2) and there are no unexpected gotchas :banghead:. Therefore, we released 2.3.0 as of this morning.

If you're currently using BRU PE 2.2.0, please be sure to update to 2.3.0 as there are some important fixes and a few new features (in addition to LTFS, that is). Additionally, the environment has been relocated from /private/var/lib/bru (which is now a dangerous place after 7 years) into /Library/Application Support/BRU PE/. This adjustment is made automatically during the upgrade install and a full backup of your BRU PE environment (settings, catalogs, etc.) is created and placed into the /bru folder on your boot volume. Finally, an alias is created from the new location back to /private/var/lib/bru so that any external scripts will continue to function without requiring changes.

An important note about LTFS - all of the issues that we've spoken about in the forums still exist and until such time as Apple, Microsoft and the various Linux wranglers take on ownership of the low level stuff (FUSE, file manager integration, etc.), we don't see that changing any time soon.

One aside - because each vendor is doing their own little different thing on the LTFS layer for each platform, TOLIS Group can only provide the drivers for our hardware customers. For other vendors' LTO-5 drives, you will need to contact the vendor to gain access to their LTFS drivers - sorry :blush:.

 

[fabrizio carraro]

Happy I've found a forum on LTFS, which is taking me many hours/day of
attempts to make it work properly. Until now I'm not very happy about it.
I would like to share my experiences on it.

I installed SLES11-Sp1 on a PC, and spent some time to re-compile the
mpt2sas driver for the LSI9212-4i4e board (the driver inside SLES11 is
version 4, while LSI releases the sources for version 12). The LTO5 drive
is a Quantum. I need to back-up on tape two kind of things: long sequences
of 10MB frames, and DCPs (Digital FIlm Packages) with big files ranging from
20GB to 200GB each. Also uncompressed audio files: this is interesting
since they may be big files full of silence (zeros) and this seems to be
unpleasant for the LTFS compression algorithm.

Anyway: transferring from a disk to LTFS is fast, around 100MB/sec. From
LTFS to disk varies a lot: if the tape is formatted without options (that is
compression enabled and standard block size) it can be reasonably fast when
reading sequence of frames, or deadly slow while reading big files
containing just zeros. Sometimes in the last case it can also crash, leaving
the tape half unmounted. The only solution is umount it and ltfs/mount it
again. The crash only happens using the mpt2sas driver (the new one but also
the original ver.4 that comes with sles11-sp1). Using a 3GB LSI SAS board
instead of a 6GB SAS board makes Linux to load the mptsas driver (instead of
the mpt2sas) and even if slow in reading the tape never disconnects.
Noticed that reading old LTO3 tapes written using tar, there are no problems
at all, and the speed is >50MB/sec.

Formatting tapes with no compression and smaller block size (32000) makes
everything a bit more reliable: reading files full of zeros never goes down
to ridicolous speeds (a few KB/sec). Writing speed is still excellent.
Very odd.

Any experience anybody with some other SAS board (for example the Atto H380
or H680 on a Mac?) I see that the platform and boards being used can change
the result a lot.

 

[Mark Pierce]

Fabrizio,

My name is Mark Pierce of Quantum Corporation Devices Division. I am the Applications Engineer responsible for supporting Quantum LTO-5 drives. Please contact me @ mark.pierce@quantum.com regarding your issues with LTFS.

Thank you,

Mark

 

[Ces T]

LTFS Cannot mount volume

LTFS Cannot mount volume

Hi there,

I'm new to this whole Terminal and command line stuff and I've come a bit stuck, I hope someone can help here.

My LTO won't mount giving me the error LTFS11027E Cannot mount volume: medium consistency check failed
HP's instructions suggest running the command 'ltfsadmintool' with '-c -- -f'. But when I type ltfsadmintool -c -- -f into the Terminal it gives me an error: Command not Found

Am I doing something wrong?

 

[M Most]

Hi there,

I'm new to this whole Terminal and command line stuff and I've come a bit stuck, I hope someone can help here.

My LTO won't mount giving me the error LTFS11027E Cannot mount volume: medium consistency check failed
HP's instructions suggest running the command 'ltfsadmintool' with '-c -- -f'. But when I type ltfsadmintool -c -- -f into the Terminal it gives me an error: Command not Found

Am I doing something wrong?

Download HP's HP StoreOpen Standalone software. It's a GUI front end for LTFS on HP LTO drives for the Mac that makes LTO management and operation very easy.

 

[Ces T]

So I tried HP StoreOpen and it's great to know there is a GUI for this so thanks very much for that. Problem is, it still doesn't mount my tape and it doesn't give any options or solutions. Apparently I need to run the command 'ltfsadmintool' with the option '-c -- -f' which will fix this and allow me to retrieve the non-currupt data. So I'm certain it's just a question of syntax, I reckon I'm just typing the command in wrong. Can someone tell me exactly how I should type it into Terminal please?

 

[Jeff Kilgroe]

Have you installed the LTFS software package? Seems like you have an improper installation or you are not referencing the location of the ltfsadmintool command. You will have to run it from where it's installed or reference the location. I don't know where it defaults to, not in front of a system with LTO-5 at the moment to check. The command not found error means it can't find ltfsadmintool. You need to help the system know where it's at.

Which tape drive, host controller and media are you trying to use? Not sure why it wouldn't want to mount, I guess admin tool thing needs to be sorted first to see if there's something wrong.

If your LTFS stuff is in a folder called LTFS inside your applications folder, then you would type:

/Applications/LTFS/ltfsadmintool

 

[M Most]

So I tried HP StoreOpen and it's great to know there is a GUI for this so thanks very much for that. Problem is, it still doesn't mount my tape and it doesn't give any options or solutions. Apparently I need to run the command 'ltfsadmintool' with the option '-c -- -f' which will fix this and allow me to retrieve the non-currupt data. So I'm certain it's just a question of syntax, I reckon I'm just typing the command in wrong. Can someone tell me exactly how I should type it into Terminal please?

Do you have OSXFuse installed? You won't be able to mount any LTFS drive unless you do. Once you've installed it, try the GUI again. It works very well, provided you have an HP drive.