Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

LTO tape drives for MacPro?

If LTO-3 is 400Gb Uncompressed / 800Gb Compressed, what does that mean for storage of video files? I can't find any information on what they mean by "compressed." Is the 2:1 compression an average, or do they litterally mean that every data tape is good for 800Gb?
 
Cinewalt said:
If LTO-3 is 400Gb Uncompressed / 800Gb Compressed, what does that mean for storage of video files? I can't find any information on what they mean by "compressed." Is the 2:1 compression an average, or do they litterally mean that every data tape is good for 800Gb?
Click to expand...

It's an average. Trying to compress compressed data a second time doesn't accomplish anything, so for backing up REDCODE footage the backup system's compression doesn't do anything of any use.
 
Ok so an interesting question I have. I have never worked with tape before and have alway done manual back ups. I guess my question would be if I just wanted to back up project by project for just that project not everything that is on my Raid. can I use LTO like a drive. Can I just drag files over and off. or do I have to use a back up program?
 
IMGentertainment said:
Ok so an interesting question I have. I have never worked with tape before and have alway done manual back ups. I guess my question would be if I just wanted to back up project by project for just that project not everything that is on my Raid. can I use LTO like a drive. Can I just drag files over and off. or do I have to use a back up program?
Click to expand...

This depends on what software you use with your tape system, but yes you can do it this way. Ideally, when you run a tape library, you have an online backup or mirror of your data on a RAID or something. The tape does an automated back-up of that mirror or online backup at a scheduled interval or time. That way, it's mostly transparent.

With larger tape autoloader systems, which don't cost much more than a single slot, desktop tape unit, you can do incremental backups every evening automatically. And then swap out the set of tapes every several days/weeks/etc..
 
Thanks Jeff, So I guess the best idea for my situation would be to make a mirror of just the data for that project or client and have the tape drive back up that mirror only. What software do you recommend for this type of option. I think one day when we are doing more of our own data back up the autoloader would be a must and we would prob do a mirror of our whole raid. But for right now I am looking at more of a client by client back up that we would keep stored in a secure location for a period of time as a reassurance to our clients. and offer a more permanent storage solution for the clients that can't or don't want to do long term back ups of their raw negatives them selfs
 
Just thought I'd chime In as I'm averaging 100 LTO-3 tapes a month now while working with the Viper/Stwo. Really you do want to do a dual tape write out, I got a LTO-3 vendor to admit to me a .1 % chance failure rate, compared to hard drives thats pretty damned good but still you have such a better chance to recover those same missing frames off of a secondary by using standard linux tarmap block sizes. Instead of a mechanical hard drive factor.

Also the ASC archival commitee are evaluating systems like this for vaulting (still very preliminary stuff) but the general concensus is to stick away from closed source (retrospect) for long term archival of a digital negative. Quite the same argument can be used for/against compression but I'll stick out of that for now.

Also a couple other bits of info for uncompressed DPX's that need to be archived such as a final render we've been seeing around 600 GB fit onto an LTO-3 tape.
 
Thanks guys this is very informative I really think this is the best bet for me. And honestly the price for what your getting is not that bad.

Stuart I am really looking at your selection but my question would be would my client be able to find post house that would be able to use this or would they have more luck with LTO
 
I'd also like to echo what Jeff Brue brought up about closed source for long term backups. This is something that I've personally run into myself and it can be a real problem. 5 years from now if you're restoring an old backup and you suddenly don't have the software used to create it, you could be royally screwed. Always store your data in a standard format either as directly transferred files or use a conventional container format like ZIP that every major OS understands.

Beyond that Retrospect is a good software and I still would recommend it for some uses. In my current backup workflow, I just copy over to VXA2 tape. With larger tape systems I've run in the past, I would create the necessary scripts/macros for the OS to run at the desired times. A bit more setup, but no third-party software and oddball data formats that force you to keep using their software. And also keep hoping that they're still in business and you can still get a copy of the software to run on a current computer in 20 years... Then again, as reliable as tape is if a good backup practices are followed, I still would think that in 20 years the archives would have been re-checked and consolidated onto newer media at least once or twice.
 
Jeff Brue said:
Just thought I'd chime In as I'm averaging 100 LTO-3 tapes a month now while working with the Viper/Stwo. Really you do want to do a dual tape write out, I got a LTO-3 vendor to admit to me a .1 % chance failure rate, compared to hard drives thats pretty damned good but still you have such a better chance to recover those same missing frames off of a secondary by using standard linux tarmap block sizes. Instead of a mechanical hard drive factor.

(cut)...
Click to expand...

Thanks Jeff for this useful info, however I must admit I'm a bit lost here. Do you have two tape drives in parallel to do a dual tape write out? Do you do your backups off of a Linux system, or can you use linux tarmap blocks with a Mac or PC platform?

Cheers,
Damien
 
Jeff Brue said:
…for uncompressed DPX's that need to be archived such as a final render we've been seeing around 600 GB fit onto an LTO-3 tape.
Click to expand...

But that is uncompressed media. REDCODE will be compressed, and probably fairly efficiently packaged already to keep the datarate down. That probably means that compression during archiving to LTO would have little impact on the data size. I would allow one 400GB LTO tape per RED-DRIVE, and use the remaining 80 odd GB (probably a bit less in reality) for eg an offline codec copy of the same media. A one to one RED-DRIVE to LTO relationship seems a good idea to me to keep things simple.
 
Earthling,
The scalar 24 has an automated dual tape backup. Also a couple programs can be used for archival in standard linux tarmaps. The tools themselves are built into most flavors of linux, and Totalus offers totalus tape tools for the mac. I'm sure there are windows softwares out there as well that offer this.

Nick,
As far as initial archival of content a tarmap with archive for redcode files will be fine. But the idea of adding a second layer of compression for archival of a final "digital negative" is really unnerving. That should be in a non company specific image wrapper (ie DPX) I believe in graeme for the initial pass but any additional compression passes and you will be losing quality (and believe me bonding agents once this gets in their ear arn't going to like the idea of that). In any case what you're ultimatly going to be doing is either a 2k or 4k conform and color grading to get it onto a quvis dci package for distribution.

Now here would be an interesting challenge for graeme and the scratch boys. A dci package is in XYZ color space using a similiar (to my knowledge) compression. To avoid the inherent loss of quality by converting to RGB for traditional grading and effects, how about a compression transform from the captured XYZ to a dci compatible projectable image XYZ jpeg 2000. I'd still say write out to 16 bit RGB for long term archival, but it would be interesting to see a shake difference map of two such transformed images.


Jeff Brue
Digital Film Company
 
earthling said:
Thanks Jeff for this useful info, however I must admit I'm a bit lost here. Do you have two tape drives in parallel to do a dual tape write out? Do you do your backups off of a Linux system, or can you use linux tarmap blocks with a Mac or PC platform?

Cheers,
Damien
Click to expand...

OS X is unix, so you can use tar, it's in the commandline.

In fact it would be better to tar the tapes you send out since tar is considered an industry standard for archiving.

If you use proprietary software like Retrospect, don't expect anyone else to support it.

If you only plan to do internal archives, then use what you feel works for you and provide media to whomever on firewire drives.
 
OSX is unix but it doesn't have the binaries for writing out to tape. So it can TAR but you can't write out.
 
Jeff Brue said:
OSX is unix but it doesn't have the binaries for writing out to tape. So it can TAR but you can't write out.
Click to expand...

Says who?

If you can mount the tape drive, you can write to it. You just need to know what the driver is called.

It's not hard for anyone who uses unix or linux systems.

There are even free apps like Xtar for those who aren't into using command line.
 
Hrm, I'd love to be wrong with this but I don't believe I am.

"Mac OS X includes tar, cpio, and pax utilities, but no native tape device
support mechanisms."

A direct line from Tolis's website, and I spent a couple hours bashing my head trying to get it to work. Again though If I'm wrong I'd love to be corrected.
 
Backup software for all OS

Backup software for all OS

Another really cool softwarepackage for crossplatform archival strategy is Archiware from Germany. It is rather pricey but writes tarmap. Basically 3 parts: Backup, Synchronize & Archive.

It's browser based and takes a bit of getting used to but they've got great support. Check them out at http://www.archiware.com :greedy:


Cheers from

Steen Dongo

---------------
PICTUREWISE FILM & TV
 
Wow, there's quite more to this than I thought...

So assuming I've got a good fast RAID 2 or 3 TB setup for editing, and shoot around 5 Redrives per show. I can just copy them to my RAID after shooting, and Back them up to LTO using finder, or some simple application right? I mean, if I just want LTO to have a master tape somewhere of my raw material, probably never to use it, should I care about back up software, data management or hiring computer geniuses from oracle?
 
You must log in or register to reply here.
Back
Top