Jim, how about data backup?

[Peter Majtan]

Jim - You guys should spend some R&D resources on developing some kind of medium-to-long-term backup system for all the 3K, 4K and 5K footage people will shoot on their "redheads"...

Tape might be dead as a recording medium, but (for now) it can't be beaten in archiving situations. Shelf life of well over 12 years, affordable and minimum storage requirements (both space and ambient)...

It will be great if You could develop a "RED VAULT" system that would record RED RAW onto a MiniDV tapes - this should not be too hard...

RED RAY is out of question as an archival medium.
CD's are able to handle most common scratches relatively well.
DVD's are already quite sensitive to any scratching.
Any optical format that will use recording density (track width) equal or higher then DVD will suffer from scratching greatly.
I can still (but barely) access 5-year old CD's, but having troubles even with 2-year old DVD's...

With new digital camera's coming out from RED next year, You expect to revolutionize the art of storytelling through cinematography. You do have a little responsibility to look into archival solution... :innocent:

I can see the proud Scarlet (and RED ONE / EPIC) owners pulling their hair out after the HDD with all "original" footage ("digital negative") have died on them... :waaa:

RED ONE and EPIC owners will probably have TB's of RAID-protected storage as well as DLT, LTO, or some other form of tape backup for their data. but what about Scarlet owners? It sure will be nice to have some affordable backup solution and MiniDV is perfect for this!!!

Everyone who would like JIM and his team to develop some kind of affordable RED-RAW data backup solution (at least medium-term: 10+ Years) please vote in the poll...

 

[Peter Majtan]

Offer

Offer

If your R&D resources are spread too thin already (which is easy to imagine) I would love to develop this for RED myself!!! :w00t:

PS:
I have single-handedly designed from scratch an Apple/ARRI/Foundry/SONY based film restoration system in Tokyo for Hisao Kurosawa (Akira's son).

I have also designed an optical archival system for digital data (as a part of research for the above project), that allows one to record digital image data onto 35mm film (with 100+ year shelf life) for archival purposes...

Drop me an e-mail if You are interested, I'm certainly capable of doing this and can provide You with references...

 

[Jason Wingrove]

Probably wht Red Ray will be, they've already said you can fit hours of red data on it imagine for about a grand having the redray on your desktop, plug in your cards like a card reader, copy it to your computer or backup to Red Ray.. Anyway we've got a year ... all will be revealed. Red is working this stuff out as they go along too.

 

[Chris Parker]

what about lto-3a data tape?

 

[Peter Majtan]

Probably wht Red Ray will be, they've already said you can fit hours of red data on it imagine for about a grand having the redray on your desktop, plug in your cards like a card reader, copy it to your computer or backup to Red Ray.. Anyway we've got a year ... all will be revealed. Red is working this stuff out as they go along too.

In the main post above I have specifically mentioned why RED RAY is out of question for backup purposes. Dailies, presentation, even distribution - sure...
Backup? Count me out...

 

[Peter Majtan]

what about lto-3a data tape?

Sure one of the best options around (for now), but quite pricey for "home" user (drives starting at around $2K) and 400GB might be too much (mind You there is never too much) if You prefer to backup on shooting basis, but that is personal preference. MiniDV RED-CODE-RAW data recorder could cost bellow $500,- or even less...

 

[Allbird]

cash test

cash test

the cash per GB in hard disks are becoming cheaper.
what do u think guys, about 4TB 100$ in two or three years?

cash test: DVD_taiyo, DV_Exellence, HDD_Seagate, BR_MCC

DVD - 0.10 $/GB
MiniDV - 0.25 $/GB
HDD - 0.13 $/GB
BR - 0.60 $/GB

 

[michael zaletel]

Secondary backup (off-site) is crucial as well. Even if we had cheap 4TB drives, it just takes too long to make duplicate copies and they can't easily or quickly be made over the Internet to a remote location for Disaster Recovery.

Any ideas along these lines would be greatly appreciated. BTW, what is currently the fastest possible technology for transferring data from 1TB drive to another? What type of drive? What type of bus, connection, etc?

-shooter

 

[Allbird]

Any ideas along these lines would be greatly appreciated. BTW, what is currently the fastest possible technology for transferring data from 1TB drive to another? What type of drive? What type of bus, connection, etc?-shooter

e-SATA II runs at 2Gbit/sec. the fastest, just the half of dual link 4Gb fiber chanel.

and e-sata II is standard & afforadable.
only need a dual-built-in-harddrive in a RAID1 configuration (like WD) with e-sata connection and u´ll transfer at 100 MBytes/s sustained, from CF to 1TB e-sata RAID1 drives, & faster in a future 2TB drives.

if REDCODE-36 RAW data rate is 36MBytes/s, then, with a 100 MBytes/s cannection, u´ll transfer about 3 seconds of RAW data in a single second.

----- it means that u´ll back up your data in a 1/3 of the recorded footage, in a RAID1 mirror secure HDD. -----

 

[kmikami]

The benefit of a data based workflow is that you can use existing data backup technology. There's no need for a Red-specific solution. Long term archival of digital information is the billion dollar question that affects the future of every information based industry. There are plenty of other people working on the issue and while the team at Red is obviously brilliant I think they're busy enough making cutting edge cameras. We can't expect them to solve this problem as well!

 

[Dylan Reeve]

MiniDV archival is unrealistic. DV is 25mbit/s or less than a 10th of REDCODE. Which means at best you'd be looking at about 4 minutes of RED footage on a single MiniDV tape.

Worse than that, the solution would be entirely proprietary - there are no existing DV-data drives.

There are plenty of data backup systems around, and they should be the ones we look to. They have advantages of scale (ie. not limited to a small market of video users) and have substantial uptake (meaning there are lots of drives around if you need to get the data back later).

I don't think it's really in RED's interest to get into that area too much when there are plenty of providers who can provide options.

ShotWrangler will develop and market a backup system once the SolidStore is shipping (assuming there is a sufficient demand for such a product). And other data backup solutions are already being adapted/adopted to suit RED workflows.

 

[Peter Majtan]

e-SATA II runs at 2Gbit/sec. the fastest, just the half of dual link 4Gb fiber chanel.

and e-sata II is standard & afforadable.
only need a dual-built-in-harddrive in a RAID1 configuration (like WD) with e-sata connection and u´ll transfer at 100 MBytes/s sustained, from CF to 1TB e-sata RAID1 drives, & faster in a future 2TB drives.

if REDCODE-36 RAW data rate is 36MBytes/s, then, with a 100 MBytes/s cannection, u´ll transfer about 3 seconds of RAW data in a single second.

----- it means that u´ll back up your data in a 1/3 of the recorded footage, in a RAID1 mirror secure HDD. -----

Kind off...

A) In RAID-1 You don't benefit with speed - You get 100% security. In fact the speed may slightly decrease depending whether the RAID-1 is HW or SW-based. You most probably meant RAID-0 where the total transfer speed is almost double of a single drive...

B) E-SATA would be the best choice (especially on budget), but mind You - most interfaces are many times faster than the maximum transfer rate of the HDD itself. USB is a definitive "no-no" due to the large "overhead". Both FW-400 and FW-800 would be my second recommendation after E-SATA...

 

[Dylan Reeve]

Also, looking at that - I doubt you'll even come close to 100MB/s read from CF cards.

The limitation with these connections is more often the drive. Transferring from a single 1TB drive to another, the best option is SATA (or eSATA) however the drive won't achieve anywhere near 1.5Gb/s (or 3Gb/s) read speeds, and the target drive certainly won't reach those write speeds. And the OS is likely to slow things down too.

Copying ~400GB from a RAID-5 set to a RAID-1 the other day took something like an hour.

 

[Peter Majtan]

MiniDV archival is unrealistic. DV is 25mbit/s or less than a 10th of REDCODE. Which means at best you'd be looking at about 4 minutes of RED footage on a single MiniDV tape.

Which is about the equivalent of 400ft magazine...
Unless You shoot "video-style" (read hours per day), You don't shoot that many... Being able to backup day worth of filming on $25 worth of MiniDV tapes is definitely worth it. The 25mbit/s is in this case referring to density, rather then the speed of recording, which would have to be improved of course.
But You do have a good point.

Worse than that, the solution would be entirely proprietary - there are no existing DV-data drives.

So is going to be RED RAY. But at least You could use standard media...

There are plenty of data backup systems around, and they should be the ones we look to. They have advantages of scale (ie. not limited to a small market of video users) and have substantial uptake (meaning there are lots of drives around if you need to get the data back later).

Yes, however majority is relatively expensive and You might find it hard to get the specific tape on location, or at 2AM. MiniDV are sold in 7-11's...

I don't think it's really in RED's interest to get into that area too much when there are plenty of providers who can provide options.

So there is Blu-Ray and yet here comes RED RAY...
Looks like they are trying to offer most common tools needed by filmmakers. I can see coming RED LIGHT (4K projector), RED VIEW (4K display) - why not backup solution?

ShotWrangler will develop and market a backup system once the SolidStore is shipping (assuming there is a sufficient demand for such a product). And other data backup solutions are already being adapted/adopted to suit RED workflows.

The Kiwi solution looks interesting, but there is yet to be any proper study as to the longevity of solid media. Also I can see the cost of such a solution to be aimed at postproduction houses - hardly at home users...

 

[Peter Majtan]

The benefit of a data based workflow is that you can use existing data backup technology. There's no need for a Red-specific solution. Long term archival of digital information is the billion dollar question that affects the future of every information based industry. There are plenty of other people working on the issue and while the team at Red is obviously brilliant I think they're busy enough making cutting edge cameras. We can't expect them to solve this problem as well!

Yet they had (and have) the time to develop RED RAY...

 

[Allbird]

Kind off...

A) In RAID-1 You don't benefit with speed - You get 100% security. In fact the speed may slightly decrease depending whether the RAID-1 is HW or SW-based. You most probably meant RAID-0 where the total transfer speed is almost double of a single drive...

B) E-SATA would be the best choice (especially on budget), but mind You - most interfaces are many times faster than the maximum transfer rate of the HDD itself. USB is a definitive "no-no" due to the large "overhead". Both FW-400 and FW-800 would be my second recommendation after E-SATA...

B) yeahh! sure.

A) RAID 0 you have 30% more speed, not double.
RAID 1 you may have the same speed if there is a well designed controller.
Is in RAID 5 where you decrease the performance, because the calculation of parity bits. RAID 1 hasn't parity bits, is just a mirror.
and of course, it should be HW controlled because is an external drive connected to an eSATA bus.
either you can go with a diferent sector sizes when formating the HD, increasing from 128kB to 2048kB sector size for increase performance with large files.
its possible, of course, to get 100MBytes transfer from a 1TB drive, & increase the speed, increasing the density per rpm in a 2TB 7200rpm drive with 2 plates.

then, actually:

cash test: DVD_taiyo, DV_Exellence, HDD_Seagate, BR_MCC

DVD - 0.10 $/GB
MiniDV - 0.25 $/GB
HDD - 0.13 $/GB
BR - 0.60 $/GB

 

[Dylan Reeve]

Which is about the equivalent of 400ft magazine...
Unless You shoot "video-style" (read hours per day), You don't shoot that many... Being able to backup day worth of filming on $25 worth of MiniDV tapes is definitely worth it. The 25mbit/s is in this case referring to density, rather then the speed of recording, which would have to be improved of course.
But You do have a good point.

It's would be a painful and frustrating process to archive or recover with such small loads.

The density of the tape could probably be improved, but look at standard DV with a 15micron track pitch, it's pretty prone to drop outs from even minor mistreatment. Increasing the track pitch to 18micron (DVCAM) improves the reliability somewhat, but it's still a fairly fragile format.

So is going to be RED RAY. But at least You could use standard media...

Storing 2 Hours in 9GB doesn't seem much like a backup system. RED RAY looks likely to serve a different purpose. I don't think it's a backup soluton, and the investment in it's proprietary nature therefore is quite a different issue.

Yes, however majority is relatively expensive and You might find it hard to get the specific tape on location, or at 2AM. MiniDV are sold in 7-11's...

The drives (thinking LTO specifically) are expensive. But the tapes are very cost effective, and pretty widely available. While MiniDVs are readily available now, that may not be the case in a few years. Tried to buy a Hi8 tape recently?

Looks like they are trying to offer most common tools needed by filmmakers. I can see coming RED LIGHT (4K projector), RED VIEW (4K display) - why not backup solution?

No reason necessarily, but to some extent RED seems to be creating products to fit places where none exist already. There are many data storage options, which are, in general, suitable for RED footage and varied in budget.

The Kiwi solution looks interesting, but there is yet to be any proper study as to the longevity of solid media. Also I can see the cost of such a solution to be aimed at postproduction houses - hardly at home users...

The solution we're [in the very early stages of] planning will be an open-architecture system, able to be deployed at a range of different price levels with a range of different technologies (Blu-Ray, LTO, etc). We're not currently considering storage on solid state memory - to my mind the physical size of most options presents a significant problem, no point archiving to something that is really easy to misplace or loose.

I'm not a huge fan of reinventing wheels, so I think it's very much worth exploring the options with existing technologies before creating something new (especially something with no current precedent). Also, if RED (or someone) were to experiment with a new product and then abandon it there's not much chance of it continuing one - people risk having their backups on an orphaned format. One advantage with existing systems (especially with LTO) is that the large market penetration and multiple vendors pretty much assures that the technology will survive.

 

[WesVasher]

I wonder if you can re-code your Redcode RAW footage with a heavier pass of compression or even downscale it also? Get the datarate down to an acceptable level, say 25Mbps and back that up to DV tape using one of these DV tape backup programs for Mac and PC? At the lower datarate you could also burn 30+ minutes to a DVD DL burnable.

Sure you aren't going to have a backup of the absolute best quality source but you'll have something on a pretty good and cheap backup.

And then just keep your full res files on Drobo or a comparable product.

 

[JoakimZiegler]

We use LTO-4, 800GB tapes, 120MB/sec transfer rate, and tapes are about 95 dollars in the US now (around 12 cents per GB).

 

[Peter Majtan]

We use LTO-4, 800GB tapes, 120MB/sec transfer rate, and tapes are about 95 dollars in the US now (around 12 cents per GB).

But the price of the drive unit is way more then that of Scarlet...

There are plenty of options for good quality storage, but we need something in terms what Scarlet is doing for cinematography - something affordable that all indies (targeted Scarlet demography) can and will buy. It should be bellow $500,-