Cheap eSATA RAID solution

[Andrew M.]

go to my last link TDK news.
200GB per side gives you 400GB for two sides.
I agree with you that we need maximum 10 years.
If you take all the tapes you have that are 10 years old and you put it on the BD probably 10 BDs will hold all your 10 year old library.

 

[Mark B.]

More important than the storage space, in my mind, is that the holographic storage is going to be designed with an exterior casing around the disc surface (like the old 3 1/2 discs from the past). Scratches and dust on the data surface won't be much of a problem that way.

 

[Jeff Kilgroe]

go to my last link TDK news.
200GB per side gives you 400GB for two sides.
I agree with you that we need maximum 10 years.
If you take all the tapes you have that are 10 years old and you put it on the BD probably 10 BDs will hold all your 10 year old library.

Er... I did click the link and I've already read that a few weeks ago anyway. Mentions nothing in there about 400GB discs or layer configurations. 200GB discs (like these from TDK -- TDK is the primary maker of Sony branded BD media too) were demonstrated at CES with Sony's new quad-layer BD writer. They can put 200GB on a BD by using 4 layers on each side, so you still have to flip the disc after you record the first 100GB. 8-layer readers/writers, or what will be needed to put 200GB on a single side have not even been constructed in a test lab scenario yet. Sony has even stated that they doubt the ability of BD writers to expand beyond 4 layers at this time with the current diode lasers they're using, they will probably need a more powerful laser system.

BluRay DOES NOT allow for 200GB per side and never will unless Sony, Pioneer, Philips, Samsung and Matsushita all agree to revamp the standard and allow for a thicker disc. 12 layers is the maximum number of layers in a disc. So any way you slice it, that's 300GB per disc, not per side. Once all 12 layers are there, you can't magically bond another 12 layers to the other side of the disc. Now you can make double sided discs, but with the physical space constraints and substrate thicknesses, you're still limited to 12 layers... Divide them up however you want, 8 on one side, 4 on the other or 6 and 6, 2 and 10, etc.. So if you have a BluRay disc with 200GB on one side, the most you can put on the opposite side would be another 100GB.

Anyway that said, it's still early in the game and Sony and friends could decide to add another 0.475mm of thickness to the disc spec and jam another 4 layers in there to give you a 400GB disc... All accessible from one side or as 200GB on each side or whatever.

 

[Jeff Kilgroe]

More important than the storage space, in my mind, is that the holographic storage is going to be designed with an exterior casing around the disc surface (like the old 3 1/2 discs from the past). Scratches and dust on the data surface won't be much of a problem that way.

I don't know about the casing/caddy... Sony dropped the caddy design for BluRay (they kept it for XD media, BD was the same, just larger) because it would require more parts to support both BD media and current DVD/CD media on the same device and because of manufacturing costs.

Of the holographic media types demonstrated publicly thus far, some have a case/caddy some do not. Some aren't even based on the rotating disc paradigm.

 

[Andrew M.]

Yes, in the article TDK do not mentioned if it is single side or dual sided 200GB disc.
I presumed single side 6 layers at 36.7 GB per layers.

 

[Stokestack]

Why you will not consider Blue-ray Disc as a good long term archive.

Because the organic dyes in writable optical discs don't last. I would never archive anything on these things. I had some name-brand CD-Rs that were totally worthless after about eight years. And who knows when they actually went bad? Probably long before that.

With the ever-falling cost of hard-drive storage and the low capacity, poor reliability, and slow speed of optical discs, I don't see that there's really any contest currently.

The optical-disc market has been riddled with blunders from the beginning. Multi-session CDs never really worked. CD-RW never worked as promised. After all that, the release of DVD without a working recording strategy was inexcusable, compounded even further by this asinine DVD+/-R fiasco. And I don't expect recordable "HD" discs to be any different. And now we hear talk of adding layers, but where is the discussion of how current players will support this? If they don't, then it's worthless.

 

[Andrew M.]

I didn’t know that!
But I have to tell you that I moved to optical storage only after I couldn’t access some of my old hard disks that I never touch for 5 years or so.
I was told that for hard disks you have to re-write every bit, including fill in bits and sector marks every now and then to make it last.
I guess I could design refresh/read/write program to do it on the magnetic storage.
For tapes though, rewinding it from end to end used to help but I don’t think it is the case with the current tape technology.
We better do some serious research on this subject.

 

[Jeff Kilgroe]

Hard drives are not made to sit on a shelf. The data will go bad, it's just a matter of time 1 year, maybe 5... Optical media is far more vulnerable error. Scratches if improperly handled, the dyes degrade over time - especially if exposed to light. That laser that writes to the discs isn't that powerful, leave a CDR on the dashboard of your car for a day or two and you won't have any data left.

For long-term shelf storage, tape is the best format... But it's far from perfect as well. Most archival tapes like LTO, DLT, AIT, etc.. Are well shielded and constructed very well. That's why they're $60+ They do hold up very well long term and can handle some decent temperature and humidity variations. You still have to watch out for moisture with tape, but this is still less of a concern as it would be with hard drives.

Anyway, whatever format we pick, I think it wise to not assume that it will sit on a shelf indefinitely and for anything truly important, more than one backup copy would be a wise decision.

I plan to keep all current projects as well as everything I've worked on for the last year in online SAN/RAID storage. Regular backups will be made to tape -- and I will probably make duplicate archival backups to store off-site for a portion of the data, or all the stuff that I really don't want to lose. If I had to buy an archival solution today, I would just add more RAID storage to the SAN and I would get an LTO-3 autoloader drive for archival. But I will re-evaluate this situation continuously and I'll see what comes of all this by July/August when my RED is close to shipping. Hopefully LTO-4 and DLT-S5 will be available by then giving tape a much higher capacity. Perhaps BluRay will be commonly available in 4-layer (200GB per disc side) for cheap. I wouldn't trust optical media for a primary backup, but as a secondary it wouldn't be bad. Besides, if BD+R media can drop in price to $5 for a 100GB disc, then that would be a great option.

 

[Andrew M.]

Yes. Looking back, the tape was serving us well, so we trust it.
Tapes though should be re-winded once a year, but as I said check it with the tape handling procedure since my information is 10 years old when I was dealing with university networks. We did use tapes for backups in the computing center. And we had fair share of mechanical or other soft error experience often. I will not recommend tape backup though.
Now I tend to trust optical storage since I didn’t have any bad experience with it yet. It is just my personal experiance.

 

[Jeff Kilgroe]

I will not recommend tape backup though.
Now I tend to trust optical storage since I didn’t have any bad experience with it yet. It is just my personal experiance.

Understandable. I personally can't recommend hard drives. I've had dozens of drives fail on me over the last 10 years, even lost a 500GB SATA-II drive last week. Motor froze up. Yes, dozens sounds like a lot, but not really. I'd put my failure rate with HDDs probably at about 5% if I had to guess. I've had far more defective pieces when ordering memory sticks and other components. With tape, I have had the best luck. Not to say that I have had no failures. I went through data archive hell with an HP 10GB tape drive about 8 years back.

 

[Andrew M.]

What I like about tapes is that if the header of the whole tape has error you just loose the volume info of the tape and direct access data information, however all the data is in tact.
Also any error on the tape will just affect the sector with the error and all other data is safe. Wit the hard drives or optical storage, if the header info of the disk has error you can’t continue, you have to use special utility to recover the data and then try to assemble it from the pieces, real puzzle job. Also if the media is not removable, any problem with the motor or electronics makes the whole unit dead. Again you can replace PCB or even the motor but it is data recovery job. Tape has the nice ability to recover from all these entire problems but in turn it is easy to mangle it in a bad tape drive.

BTW what problems do we have when storing rolls of films for long time in the archives?

 

[zak forrest]

Understandable. I personally can't recommend hard drives. I've had dozens of drives fail on me over the last 10 years, even lost a 500GB SATA-II drive last week. Motor froze up. Yes, dozens sounds like a lot, but not really. I'd put my failure rate with HDDs probably at about 5% if I had to guess. I've had far more defective pieces when ordering memory sticks and other components. With tape, I have had the best luck. Not to say that I have had no failures. I went through data archive hell with an HP 10GB tape drive about 8 years back.

i can relate jeff, because i have had about a half dozen drives fail on me in the past year and im currently in the middle of trying to figure out the best way to archive when i have 2 red cameras soon. i am forever scared of hard drives, even though i think the problems i had were mainly horrible parts in lacie enclosures (anyone else have this problem?), and might not be so bad with other drives. i hope you continue to keep us posted with whatever you decide on upgrading to when you get your red camera.

could you tell me: do you think the same situation you had with the tape/tape drive of 8 years ago could happen today, or has this problem(or whatever happened) been fixed for good in the past 8 years of tape technology development?

only one hellish experience with tape (and so long ago) seems like a pretty sweet track record to me. i think im going to go the tape route. this is a new thing for me, i am not sure what format, or brand drive etc to get, and i dont know what an autoloader drive is either. any suggestions from the crowd?

the archival solution i come up with will be used on a feature film with 2 red cameras and a high shooting ratio, i do not have any clients, i wouldnt be shooting 12 hours straight every day or anything like that. just the film.

 

[Andrew M.]

From my experience, most long term archives are stored on the tapes.
We are talking large institutions here….hmmm….
I am tempted to use Optical storage though. I use two copies anyway for the archives storage location diversity. On site and off site copy.
If you check the copies every two years, it is very unlikely that both copies will go bad at the same time. If they will go bad, just single one, then we do scrap the idea of archiving on optical media all together.
Alternatively we can move to the optical media with the proven track record then.

Anybody has personal bad experience with optical storage going bad after one year or so?

 

[Jeff Kilgroe]

I've had CD-Rs go bad after 8+ years... As I mentioned in another related thread around here. They dyes just don't last that long. Manufactured discs and even some types of CD-Rs with metallic compounds in them can last longer. Be wary of those with metal foil layers (manufactured or writable) though as moisture can get inside and corrode that layer.

The only optical media I've had go bad after a year or so are discs that I've mistreated. Like direct sunlight on the dashboard of a car can easily wipe a CD-RW if left for too long. DVD-/+R/RW|DL, etc.. are basically the same thing as CD-R, mostly the same dyes and manufacture processes, just developed for and written with higher density data. This in turn means more chance for error with small scratches and whatnot. Compound that even further for BluRay and HD-DVD.

My big concern right now over optical media is that it's expensive and time consuming to use. BluRay is showing some promise with capacity -- double-sided 100GB discs are available. But even single-sided 50GB (dual layer) discs are still $35 each. And write times are very slow compared to just about everything else out there.

 

[feb31films]

eSATA raid

eSATA raid

i'm looking into raids atm and at the low end, i'm looking at this: http://www.macgurus.com/productpages/sata/Burly8RHS.php filled with either 500 or 750gb HDs.

has anyone used these, i've never heard of them before but it looks ok.

I am actually going to be ordering the 2 Bay version of this next week (with a couple of Seagate drives) for some external editing/storage. Got a short project shot in DVCPRO HD that I'll be editing and I will be more than happy to post my results on this forum. I've been looking into this for a few weeks and if you are thinking of going the same route, one word of caution. There are SEVERAL manufacturers of these types of enclosures (Sonnet, Otherworld Computing, MacAlly, etc..) and though nearly all of them will mount the new SATA II drives (3.0 Gb/s) MOST of the enclosures limit the data transfer across the cable to 1.5 Gb/s. That will create a serious bottleneck in your workflow. These BURLY enclosures will allow up to 3 Gb/s transfer speed which is why I am going with them. Make sure you do your research. Happy editing!