Long-Term Storage of RED-One footage "other than hard drives" ??

[Johnny Friday]

my personal thing is, i get tired of having drives pile up, i really wanna start recycling them for editorial work or short term storage. the matter of dumping to new drives will leave me buried in drives like a hoarder.

I don't like throwing out working drives either, I have hard time letting to

but LTO is nice that I can just throw it on a tape, and shelve it in a cool semi dry closet, and not have to look at it until I really need it, even if it's 10 years from now and I wanna dig up some old stuff.

smaller, compact, LESS worries, not void of worries.

PROVES the point---different strokes for different folks. Mine works for me; Yours works for you....

 

[Elsie N]

Just curious about the LTO tape drives. Can you go immediately to footage in the middle of the tape or do you have to search through all the footage until you find what you are looking for? I really like having folders I can retrieve instantaneously from HDDs.

 

[Jay Gannon]

PROVES the point---different strokes for different folks. Mine works for me; Yours works for you....

from this I assume you don't work for clients then only for yourself and sell your own footage?

Because anyone who has worked with storage tech will tell you that HDD's are not a long term storage option, thats not a matter of opinion its a matter of fact.

What your system is is a online or possibly a near line storage system it is not a long term backup system which is what George was asking about.

Using HDD's for your system is perfect of course, but if your talking about archiving footage for clients it needs to be a long term offline remote location compatible system and the only option for that at the moment is tape.

 

[Jay Gannon]

Just curious about the LTO tape drives. Can you go immediately to footage in the middle of the tape or do you have to search through all the footage until you find what you are looking for? I really like having folders I can retrieve instantaneously from HDDs.

Yep remember we are storing data to the tapes not the actual video, its not like making a film-out to stock, its more akin to making a DVD backup onto a very very secure backup medium.

 

[Johnny Friday]

from this I assume you don't work for clients then only for yourself and sell your own footage?

Because anyone who has worked with storage tech will tell you that HDD's are not a long term storage option, thats not a matter of opinion its a matter of fact.

What your system is is a online or possibly a near line storage system it is not a long term backup system which is what George was asking about.

Using HDD's for your system is perfect of course, but if your talking about archiving footage for clients it needs to be a long term offline remote location compatible system and the only option for that at the moment is tape.

Jay....agreed. thanks for the notes.

 

[Elsie N]

Yep remember we are storing data to the tapes not the actual video, its not like making a film-out to stock, its more akin to making a DVD backup onto a very very secure backup medium.

Yes, understand that this is different from Mini DV, and I assume the OP was asking about a way to store edited footage that would be one continuous stream of data.

But even archived footage has the potential to be looked at someday. I'm just thinking that if the data to be stored is more akin to acquired footage, that is, individual takes, it might be better to go the HDD route.

I have probably a couple hundred hours of Mni DV tape that I haven't a clue what's on them. I tried to make a note of generally what I was shooting, but invariable a shot I remember will elude me to the point that I just quit searching tape after tape. Can only imagine the hours it would take to describe all the individual shots on an LTO tape.

 

[Jay Gannon]

No as in your storing the files not the footage, I can put pictures, scripts, takes everything onto an LTO tape, it acts like an external hard drive.
Its a digital storage method, I can store any files that my Mac/Pc can access onto an LTO tape.

 

[Johnny Friday]

Elsie, what you are talking about is more of a FCP Server solution.....i use that for storing my footage and use small proxies to call up footage, give metadata to said footage....and many of those clips are mini dv that i've digitized. So, i call up humboldt squid....then appears a data list of all data with humbold squid as metadata, then click on any of those.....proxi clip plays.... can then pull original hi-rez clip to edit or send to client.

 

[Elsie N]

No as in your storing the files not the footage, I can put pictures, scripts, takes everything onto an LTO tape, it acts like an external hard drive.
Its a digital storage method, I can store any files that my Mac/Pc can access onto an LTO tape.

I understand that you could... but what I'm not learning is how easy or difficult it is to go to one particular file in the middle of the tape? Instantaneous? Seconds? Minutes?

If it takes very long, then LTO would be a good second backup, just in case. But if that data is needed occasionally, well... maybe a combination of HDD/LTO.

 

[Jay Gannon]

Elsie, what you are talking about is more of a FCP Server solution.....i use that for storing my footage and use small proxies to call up footage, give metadata to said footage....and many of those clips are mini dv that i've digitized. So, i call up humboldt squid....then appears a data list of all data with humbold squid as metadata, then click on any of those.....proxi clip plays.... can then pull original hi-rez clip to edit or send to client.

Yup big FCServer fan here too.

@Elsie: Instant, less than 2 seconds I would say. No longer than it takes a hard drive to spin up anyway.

EDIT: I lie, maybe 10-12 seconds from initial tape insert to file read..

 

[Elsie N]

Yup big FCServer fan here too.

@Elsie: Instant, less than 2 seconds I would say. No longer than it takes a hard drive to spin up anyway.

EDIT: I lie, maybe 10-12 seconds from initial tape insert to file read..

Wow, had no idea you could go to the middle of a tape that fast. I was thinking more in terms of how a Mini DV tape works and how you have to unwind tape to get to a certain spot.

By all means, with this ease of access to individual files the LTO would be the method of choice

 

[Jay Gannon]

Yep the more modern the drive the quicker the read and rewind times. I think the new LTO-5 Drives are very very quick. And once you have the tape in and being accessed the times drop down to less than a few seconds (What i was thinking with my 1st answer)

 

[Adam Eden]

I have been using DLT for over 10 years now and I have just upgraded to an LTO 5 drive with BRU PE.

It takes about 2 hours to backup 1 terrabyte from my AVID Fibre Media Engine.
That is lightning fast!!! The BRU PE software is also pretty easy to use. The LTO tapes are about $100 for 1.5 -3 Terrabytes so why would anyone bother backing up to a Hard drive?

 

[Johnny Friday]

so why would anyone bother backing up to a Hard drive?

do you really want me to answer that?

 

[Jeff Kilgroe]

The big issue with HDD's for long-term storage is they are simply not designed for that use. Not using an HDD is almost as hard on it as using it every day. LTO tape is actually designed for long-term shelf storage.

I disagree that HDDs are the cheapest solution. They are for small to moderate archival loads, but as the archival sizes increase, LTO tape rapidly becomes a clear price winner. LTO-4 tapes still pack 800GB into less than $25. It's the up-front cost of the tape drive system and software that raises the total cost up and it takes several terabytes of data archived to bury that initial cost.

People often think that tape is slow... It's not. Current LTO tape systems can typically write tapes faster than most HDDs, even as fast as many 3~5 drive RAID arrays can serve up the data. As Adam points out a couple posts up, it takes about 2 hours to write 1TB of data to an LTO-5 tape. LTO-4 takes between 2 and 3 hours to write a tape on the slimmer desktop units (750~800GB after overhead), even faster on the larger full-height units. LTO-5 is twice that fast as it takes about the same amount of time per tape, but stores double the amount of data. The current full-height HP LTO-5 system can write in excess of 280MB/s, or more than fast enough to archive 2K DPX files at 24fps real-time... Just sayin'

The big thing that everyone needs to consider, or even realize, about data backup and archival is that it's not about any particular technology. It's a complete strategy that must fit the operational parameters of each individual or business. I use LTO tapes as archival, I almost exclusively use LTO-4 as they are currently the best $$/GB ratio of any media. But I also backup to hard drives and occasionally LTO-5 tapes. It's a strategy, not a dedication to one specific technology. I'm starting to shift to LTO-5 from LTO-4 now that tapes are getting cheaper. They're still 20% more per GB than LTO-4, but they put twice as much data into the same amount of shelf space.

My backup strategy usually involves multiple online copies of my data via SAN or localized RAID. Occasionally to external / removable HDD for intermittent backup and transport. Archival goes to LTO tape. Always 2 or 3 copies of the archival data stored in at least two different locations. I tend to have more than that as regular back-up sets often include data that has been backed up before as it may still be on my online storage as I haven't had a need to move it off yet.

Tape will someday be replaced by another technology for archival storage. But that technology hasn't materialized yet. Some hopeful candidates have reared their heads, only to fall short of promises and fizzle out into nothingness. ...How many more times do we have to see the "holographic" storage promises that seem to pop up every 7 years or so, ever since the early '80s???

 

[Johnny Friday]

So what this thread come to is what is best for your CLIENT & what is best for you as a USER. And, this all depends on many other finite aspects.....I'll only go with what i know.....the up front costs as Jeff mentions....and new software and another PCIE slot lost on my machine--i have no extra space with a red rocket, MXO2 card, Esata, and my monitors drivers...So, would need another dedicated machine to run an LTO......now, that said, if your in the business of storing data for clients....Hey, LTO's the way to go. Or....if you are needing storage of a project here and there then maybe it's best to hire out that service....Great, ten different ways to go on all of this. I'm mentioning my way, because i'm not alone with stock shooters that sell stock footage and i know other natural history documentary shooters that store everything on HDD. So, what's best for you is up to you to make that decision based on all the cards on the table.

....My gut feeling TODAY 2011. is HDD at 2TB for YES---$125 + tax is that solution for me....upgrade next year to 3 or 4TB for $149.....and then await what we ALL know is COMING.....SSD or some magic bullet but it's not far off....I"m not ready to drop $3k or $4k for an LTO i know I'll trash next year for something better or newer technology. Ok...now were going to 5k, Jim's thkning 10k or 28k ......is 2TB storage going to be enough...NO. We all know what's coming. 800gb is NOTHING for me....since 2007 i've been shooting RED....and only part time.....and i have just near 20TB of stock select footage. What will it be like when i'm shooting 5k THIS YEAR and 28k NEXT YEAR.......ok, maybe an exaggeration, but tell Jannard that. I think he'll agree that 5k is last year......he's probably on 28k or more....

 

[Alexander Ibrahim]

How many more times do we have to see the "holographic" storage promises that seem to pop up every 7 years or so, ever since the early '80s???

That's because the technology is AMAZING.

You can basically shatter a holographic storage device and still read out the data.

data rate and density is theoretically limited by quantum effects, tehrmodynamics and the uncertainty principle.

The problem isn't the science, its the technology.

Holography won't cross over with hard drives in performance per dollar until sometime after 2020.

With luck we can get holographic archival media before then- it only has to outperform LTO after all, and that's a much easier target than trying to replace hard drives. I think SSD has that in the bag based on performance.

By 2020 we should have 2.5" spinning disk drives with 14TB for about $40.

Assuming that SSD follows the same curves, we should see a 14TB SSD for about $1500

http://www.physorg.com/news175505861.html

 

[shashbugu]

Even for medium term storage, LTO tape backup is simply the best and cheapest Period! The really is no debating this.

 

[George D.]

Guys.....a gang rape if feels like. I can only speak from experience as of 1997 on....been with online HDD's and NEVER a problem. In fact in the field it's how we work....deliverables etc....and same with the server that we uploaded to. Now... I won't disagree that there can be downs to consumer HDD's. but if anyone has been to the test....i certainly have. It's in fact how i've run my stock footage for EVER. NOT ONCE since 2001 have i had a drive fail from anything OTHER than dropping it or WINDOWS use. My storage drives have been 100% safe. Yep.....bring on the gang rape again. Sure, anything can happen, and one day a tape drive might eat up a tape....had that happen A LOT...or 120f heat where i live basically melts tapes when left in the heat (inside my home-when i'm gone for a month on a shoot--had that happen as well). But always came back to my HDD's.....i guess we have different experiences. So wont' argue....anything can happen to tape or hdd under the right circumstances.

You wont convince me nor will i convince you. But i also don't put them on a shelf for 5 years as you may have suggested. I originally started with 120gb hdd's, then went to 250gb; 320; 500, 750; 1tb 2tb etc....so i'm continually on top of moving to new drives and AWAITING SSD at an affordable price and size.

**i'm not typically ARCHIVING footage to review in 20 years...i'm using it weekly for sale or use on other documentaries etc....so i NEED ONLINE and instant access.

PRICE: don't be silly....you can get these Hitachi 2TB 3.5" SATA 7200rpm drives anyplace for $169 retail and I have bought 20 or so at Fry's on sale at $129 + tax....so unless you live on mars then those prices are REAL. Just do a simple search before you bash me on pricing:
http://www.google.com/search?source...0CCsQrQQ&biw=1039&bih=943&fp=7ed1b761d79a69af

Whoops: looks like you can get them on sale for $119.....my mistake

Thank you Johnny.

I am going with your suggestion over all the others for three main reasons:

First, I cannot afford "another" $7,000.oo+ for even more equipment just for storage.

Second, I have no desire to set up yet another computer and learn another software program, just to handle the storage. I have 4 PCs already.

Three, tape is "dead", and I do not trust tape. Too much can happen to it. As you say, a faulty hard drive can be saved, a melted or stretched tape, cannot.

I realize you are speaking of "internal" hard drives. That is totally impractical for me. What "EXTERNAL" hard drive would you recommend?

Thanks.

.

 

[George D.]

Another option using HDDs might be to archive on the VERY inexpensive green drives (5400 rpm), get a docking station and then spin up the drives every few months.

What's a "green" drive?

.