Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Backing up data "on the road"...

Raid & Sata

Raid & Sata

Great Thread thanks all!
I was looking around because as RED teach me single drives are too slow for REDwork and I start reading about eSATA & RAID.

For example at Addonics they have this Mini Storage Tower that allow to fit 4 "as you like" internal 3.5" SATA HDD connected individually or via "Port Multiplier". Every configuration is under 200$.
Really interesting for us could be their allumium rugged anti-shock case to protect HDDs while they have important datas in, for 89$ looks good.

They are open to "Storage Tower Build to Order" may be they want to produce a rugged MiniTower for 400$??

Any way... my idea is have 4 HDDs in double RAID 0, to make 2 copies at the same time splitted in 2 HDD each. Using the data rate of SATA & Raid 0 I think it could be really faster than with FW800. Is it possible?

I was looking to eSATA EXPRESS CARD & MacBookPro, I've got an unanswered question:
Wikipedia tells me that there's a huge difference:
"The ExpressCard has a maximum throughput of 2.5 Gbit/s through PCI Express or 480 Mbit/s through USB 2.0 dedicated for each slot"
The EXPRESS CARD slot on a new MacBook Pro is PCI-E or USB based?

Ciao Carlo
 
The ExpressCard slot in the Macbook Pro systems sits on a 4-lane PCI-Express interface. So no worries there. Be sure you get an eSATA-II (eSATA 300MB/s) capable card and one that has onboard RAID functionality. The software RAID within OSX is OK for some things, but can slow you down when you start running RAID-1 mirrors of large volumes with large data. Get a controller that will handle this for you. This way, you can create one RAID-0 stripe volume out of the first two drives and one out of the second two drives. Then create a RAID-1 mirror set where both volumes contain the same data in cae one fails.

You may also want to consider a RAID-5 volume. If any drive fails, it can be replaced and the RAID volume will rebuild itself with no data loss. RAID-5 will give you more usable space than a RAID-1 mirror.

RAID volumes like this are the best solution for everyday workspace and short-term backups, etc.. I would not recommend hard drive based solutions for long-term archival. Hard drives are not designed to sit on shelves and they are not as economical as tape once capacities increase into the tens of terrabytes.
 
for these documentarians, how about a third party thing that sits on the back of the camera holding (or connected to) the red drive and two small removeable standard usb drives, when not shooting it is quietly doing a double backup and verifying it
 
How 'bout a SATA port splitter that can operate two RED DRIVES simultaneously. When done, you have two copies right there. Although, the camera would probably need to be aware of the device. Or the device would have to be intelligent and buffered to handle running its own RAID-1 style mirror. Should be possible though.
 
JohnF said:
I'll throw my thoughts in at this point!

2. For higher budget shoots I was considering shooting 4k to RED magazines whilst feeding HD-SDI out to HDCAM SR as a lower but acceptable rez back up format. This will have the added benefit of reassuring producers worried about tapeless work flow. Though the data BU tape formats are also a possibility (re costs)
JohnF
Click to expand...

Eek, I'd argue against that because:

1.) 5000 & 5500 HDCAM SR decks are a $100,000 deck with the 4:4:4 board/s
2.) it weighs something like 60-90 pounds if I recall correctly
3.) and it STILL requires AC power
4.) SRW-1 is smaller but still pricey - about a $80K package with the SRPC module (required)

For that kind of size/money/weight, you could get a roadcased server with twin tape drives (IF there is software that can write 2 copies simultaneously and compare them both to source).

Again, depending on how much footage to be shot, 500-750GB on a 3.5" drive (padded and packed of course) is pretty good GB/$/cubic inch of storage

Get a hot-swap drive shell and an ExpressCard34 eSATA card and you're off. Tape backup would be nice, but as has been pointed out on here, problematic to connect to a laptop. All of the go-between bits could either not work or hinder throughput as well.

And there's always 2K RAW to do better than 4K@24p's 100GB/hr...

-mike
 
MikeCurtis said:
Eek, I'd argue against that because:

1.) 5000 & 5500 HDCAM SR decks are a $100,000 deck with the 4:4:4 board/s
2.) it weighs something like 60-90 pounds if I recall correctly
3.) and it STILL requires AC power
Click to expand...

This really baffles me. What exactly could be going on inside that case? As far as I can tell, the whole device should basically be a few custom ASICs and a tape transport mechanism. I can't figure out why this would need to be much larger than a clock radio, or why it shouldn't be able to run for a couple of hours off a laptop-sized battery. Is there something I'm missing here?

REDCODE RAW 4K will almost certainly require much more processing than HDCAM SR, and Red is managing to fit that into an 8 pound battery-powered device, which also just happens to be a camera as well.
 
Shoot smart, shoot less? Just thought I'd throw that out. Are hard drives not the new tape stock? I just Fedex'd a Maxtor TB drive, it arrived in fine shape. It was the main backup drive on an international project.
 
MikeCurtis said:
Eek, I'd argue against that because:

1.) 5000 & 5500 HDCAM SR decks are a $100,000 deck with the 4:4:4 board/s
2.) it weighs something like 60-90 pounds if I recall correctly
3.) and it STILL requires AC power
4.) SRW-1 is smaller but still pricey - about a $80K package with the SRPC module (required)
Click to expand...

Mike, I completely agree with you but producers are a pretty conservative lot and frankly I think that a camera system like RED is, at first, going to scare the hell out of them in terms of format reliablity so I think you're likely to see RED adopted for shoots with someform of "traditional" tape back up whilst producers become comfortable with hard-disk-only shooting.

I don't know if you've read through the whole thread but I am more interested in running data back up to tape using a Sony AIT-5, which is small enough to fit inside some ruggedised laptops. This would also have the added benefit of being a simple method of storing footage over the long term...

The HD deck would be more about giving clients a feeling of security.

JohnF
 
Here’s what I’d do regarding backup if I’m traveling round the jungles of Borneo, Antarctic (sans jungles), outer space etc.

I must plan on everything going wrong!!

I’ve got 1 RED camera, let’s hope it doesn’t break. Either I’ve got a spare camera or I wait for another to arrive.
Hopefully my backup regime will fit spare camera too, particularly if it’s not RED.

Hopefully my REDDRIVE doesn’t break. But hey I’ve got a spare REDDRIVE/REDFLASH or two. I might be even able to change the faulty drive and not the whole REDDRIVE unit?

Say REDDRIVE’s average backup transfer rate is 40MB/s.
Nothing in my backup system needs to be faster than this.
Really only need usb2 (or firewire) here.

I’d go for the laptop /usb/ raid 1 solution.
Here’s why?

If my laptop dies (yes, I’ve got a backup of OS/Raid s/w & REDCINE on DVD) I’m gonna be in a much better position trying to source another simple pc/laptop with usb port and external sata box.

My Raid 1 unit would be an usb2 device i.e. Thecus etc, which would take 2 drives at whatever size I choose. Probably go for Seagate 320GB,500GB or 750GB’s depending on shooting rate, ease of transporting backups or need for footage back at ranch/ mothership/ edit facility or just plain paranoia/safety. Nothing new here!!

I’d also have a single usb/sata drive cradle. This would be my B plan if the Thecus or equivalent dies and would let me review backups on the road.

Ok, as I’ve filled up the REDDRIVE, it’d be also backed up to my RAID 1 device.

Nice as I’m shooting to have 3 copies of my footage.

As the REDDRIVE is filled (or before), I’d ship off or separate (hide in a tree/cave) one of the drives of my Raid 1 pair. Nice if it was going back to production house that would then be able to be properly backed up and start editing. They’d also tell me the disk arrived safely, but anyway I’ve got the matched pair and if the production was worth billions I’d probably have another copy on my single usb drive until I knew the transported copy arrived safely.
Note: As REDDRIVE is full, is it FIFO or do I have to erase and start shooting again on a blank REDDRIVE?

But wait, I’ve got another REDDRIVE (or not) and cycle them so i've still got 3 copies of my footage using the cheapest, easiest to replace components there are!!
One copy is always going back to production house (or secure location) and the other copy I can always keep with me for all sorts of uses. Sure I can use tape but it sounds like the tape equipment is more specialized and harder to replace and I’d ALWAYS have to make two tape backups anyway and that's slower.

So I’d start my trip with a laptop, raid 1 device, usb/sata cradle and as many blank drives as I needed.
To have 2 continuous backup copies of 40 hours footage, (hopefully 3 with an extra REDDRIVE @ $1K and production house backup), I’d need to buy 30 x Seagate 320GB drives (~$100 ea = $3,000) or 16 x 750GB drives (~$300ea=$5,000). Place them in two small pelican cases.
That’s it! And for what, an extra small pelican case and $2,500 less!!!
This solution is cheaper, easier to replace parts, takes less backup time, easier to review/edit footage on road and a bucket load of other silly stuff I’m too tired to mention as it’s 3am.
DF

Stop Press: You'll be needing 2 tape drives unless you can wait till your 'on the road' drive comes home! Good night....
 
This is a great thread and addresses many of the issues tapeless/filmless production is now going through. Of course there are no secure answers yet.

It's interesting how many of those who work with tape just cannot fathom the jump to digital media. I've got a doc coming up soon and the company I'm working with really wants to make the jump into HD. I've done docs with them for almost 15 years. I've suggested going tapeless. They can't wrap their heads around it. It's very frustrating. They want to record DVCPro HD Varicam on tape...because it makes them feel better. They know and trust tape. Getting people to change their thinking is going to be quite a battle. I think I've convinced the head of the company (and a good bud of mine) to come to NAB this year and see the change that is taking place for himself.

I've been playing with an HVX for my own enjoyment and brought it on vacation to Hawaii last week. I also brought a firewire on-the-go drive with an 80GB (laptop size) HD to dump the P2 card. No laptop. Put the camera in 1394 Host mode, attach a firewire cable to the drive, turn it all on, camera sees the drive and with a press of the button transfers the material to the drive...pretty quickly (of course this is 720 24pN and only 8GB). Worked like a charm. Will RED camera have this ability to transfer directly to HD?

For most of the work I'll be doing with RED, it will be HD doc orientated, 1080 most likely for broadcast. I think at this point HD backup is going to be the way to go. I'm sure we'll see many solutions at NAB with much more to follow.

Our big mission as photographers will be getting the producers to accept the media digitally just as they accepted tape. I've seen lots of things go wrong with tape in its day, from head clogs to tapes eaten in the camera or a deck. I was on a doc once where we finished about 10 days of production out in the middle of nowhere Churchill Manatoba. Packed up 20 betaSPs nicely (with a handle and everything!) for the producer to hand carry back home (he left separately as his home was different from the crew). The sound guy and I make it through customs and security at Toronto and start heading toward the gate. The sound guy says "Hey look!" and sure enough sitting on top of a rubbish can is the nicely packaged 20 betacamSP tapes! The producer, in a rush to catch the flight, got through customs and sat the tapes down to arrange his stuff, and forgot the tapes (he actually didn't remember them until he was in the air for about an hour, then freaked out trying to get the flight attendants to get the pilot to call back to Toronto to find the tapes and not destroy them...they didn't do it.)

Of course we picked up the tapes and brought them with us. Had the tapes been lost, the cost of a reshoot would have been incredible. With digital, making backups in the field is easy if not a little time consuming. But if the producer wants to do it...I say have at it!
 
getken said:
Will RED camera have this ability to transfer directly to HD?
Click to expand...

I suspect not. I think it may even have been said by somebody from RED in another thread. Shame, but as I understand it the camera only has a single e-SATA host port for connection to the RED-DRIVE. No firewire, and USB only as control/comms, not for connecting a hard drive. The RED DRIVE has eSATA, USB-2 and Firewire 800/400, but I believe is only a 'dumb' drive, and cannot act as a host to attach a backup drive. I could of course be wrong, and "all specs are subject to change".

So your stuck with taking a laptop on location for backup.

I intend to use LTO-3 for archiving my footage, but I see that as long term storage, to be done at the post facility. I see short term backup on location being done with multiple redundant hard drives and a laptop. In extremis I would consider taking a load of RED DRIVEs on location if backing up as we went along was impractical.
 
I feel that until more practical solutions come along, or rather production methods that producers can feel confident in, shooting to hard-drives is going to be more like shooting film than video in terms of work flow/practise.

This will be an advantage for RED as film shooters already understand that they have to protect the master footage with their lives! (or at least their professional careers)

Television shooters have had decades of a surprisingly rugged tape based shooting formats to work with so the change back to film type workflows might well be a little jarring for them. But this shouldn't be too difficult as RED's offered pic quality is likely to persuade alot of producers of the potential for footage and production resale.

Personally I'm looking to use RED for it's pic quality and lens options giving me the option of obtaining fantastic images. Carrying a laptop and a few spare hard-drives, which I'll probably be doing anyway, is not much of a price to pay for the potential results...

JohnF
 
Recording to tape

Recording to tape

Yes, recording 1080p to a VTR is one option, but as we have also discussed, so is recording REDCODE to RED-DRIVE and then cloning that to an appropriate data tape. As long as the transfer rate is at least as fast as REDCODE, you should be able to play back the rushes just like a "videotape".
 
Off the shelf decision time

Off the shelf decision time

Nick Shaw said:
So your stuck with taking a laptop on location for backup.

I intend to use LTO-3 for archiving my footage, but I see that as long term storage, to be done at the post facility. I see short term backup on location being done with multiple redundant hard drives and a laptop. In extremis I would consider taking a load of RED DRIVEs on location if backing up as we went along was impractical.
Click to expand...

We used the G-Safe 500 GB RAID 1 mirror drives to back up the hundreds of 8GB card we shot on location recently. Worked well. I understand these fabulous drives are now "end of life cycle" - bummer really liked the solution.

The work flow we have adopted currently seems ideal with today's off the shelf products.

So here is my present day plan pending newer technologies in the future:
We shoot CF cards (and RED DRIVES soon) then dump to RAID 1 mirror drives on location. Then when at the studio back up to LTO3 or DLT4 or which ever one chooses. Obviously make 3 copies and store in different locations (General rule of backing up critical data). In our case, valuable .r3d files.

Here are the readily available RAID 1 HDD compatible drives systems I have found and plan to use to replace the phased out G-Safes (a damn good product).

There are three field back up solution options for RAID mirror drives that duplicate .r3d files to two independent 500GB drives simultaneously that I have found.

http://www.cooldrives.com/dusahddfiusb.html

http://www.cooldrives.com/2usb20andfi8.html

http://www.cooldrives.com/blackicelcd2x.html

I think that owning two of the selected models to travel in the field is key for redundancy. Also, the 500 GB hard drive trays run $20-30 each. With the multiple options from firewire to USB it isreasonable to supply one drive system and one set of the HDD's to the client while retaining the redundant back up for security.

The seagate 500 GB drives are currently $119 each plus the model specific tray ($20)

http://www.tigerdirect.com/applications/SearchTools/item-details.asp?EdpNo=2273393&Sku=TSD-500AS2

My idea for shows I am producing or shooting for clients and for stock footage is to use the HDD's as digital mags. Specifically, each drive stores 250 minutes of 4k redcode raw. Making two copies at the same time is prudent sense if dumping cf cards or red drives to these hard drives in the field. That works out to $280 per 250 minutes or $1/minute for duplicate drives.

Simple labeling A1/A2, B1/B2, etc works well. The drives are light and will fit into a pelican or soft case for carry on. Back at the studio back up to LTO3 or DLT4 and retain one HDD as a short term working drive for transfer to the editing HDD and on site back up storage.

This is the best solution I have developed thus far.

Let me know your thoughts.
 
sharkguy said:
There are three field back up solution options for RAID mirror drives that duplicate .r3d files to two independent 500GB drives simultaneously that I have found.

http://www.cooldrives.com/dusahddfiusb.html

http://www.cooldrives.com/2usb20andfi8.html

http://www.cooldrives.com/blackicelcd2x.html
Click to expand...

Just to let you know, I've seen people on other forums that have not had good things to say about cooldrives.com--I believe both about products and customer support.

Are you basically the G-Safe as a swappable system? AKA, copy the CF to two drives, and then take one out and stick in a new drive and have that duped onto? That way the client can walk away with a drive, or you just have three copies?

Also, what LTO drive(s) are you using?

Matthew
 
Security Vs Cost

Security Vs Cost

it seems to me you can have security for your footage or low costs but you cant have both. if you want a solution thats rock solid its going to cost you something either in money or weight carrying around backup devices
 
I think a case study is in order, resulting in white papers that could benefit the community. I have several solutions in mind, but without a camera in hand yet I haven't been able to try them out. I see the three challenges as follows and in the order of importance.

1. Data Integrity. Whatever we backup to there needs to be very little chance of data corruption. Must avoid any solution that is a kludge of interfaces that could result in data being compromised.

2. Speed. In situations where we are on the move a lot, speed will become an issue. Must have a solution that is fast, but must stay true to the first rule about data integrity. What is the fastest solution that maintains that integrity.

3. Cost. Always important, but in light of the first two issues, needs to be third in the equation. It will cost you more in the end to have a day's worth of shooting go corrupt then it will to spend some money up front to offer a reliable, relatively bulletproof backup system. I also think this backup system is a justifiable rental item to our clients, so we should be able to see a ROI.

Are there any plans to do testing at LART for backup systems? I know you guys have a lot on your plate, but was curious if this was one of the things on the agenda.

Steve
 
macville said:
Just to let you know, I've seen people on other forums that have not had good things to say about cooldrives.com--I believe both about products and customer support.

Are you basically the G-Safe as a swappable system? AKA, copy the CF to two drives, and then take one out and stick in a new drive and have that duped onto? That way the client can walk away with a drive, or you just have three copies?

Also, what LTO drive(s) are you using?

Matthew
Click to expand...

Thanks for the feedback on cooldrives - I can see that their customer support sucks based on their sales policy "People are stupid - if you buy it and don't know how to use it or set it up don't bother us." Not finding a lot of other dual drive RAID 1 500 GB options though. The G-Safes are out of stock at Gtech and many other vendors. Too bad, a great product.

Your triple disk idea works also. My practice thus far is to make the RAID 1 dual drive copy of the cf cards on set. When the 500 GB drives are near capacity I swap them both out, label and save the dual drives for LTO 3 back up at the studio (or in the field if feasible). You can then pass one drive to client and keep second as a back up. I am sticking with Seagate 500 GB drives for now due to their reliability and track record - the 1 TB drives are a bit new in comparison.

For the future of 4K footage and amazing stock footage, I am practicing the 3 back up system with the LTO 3, you know, one copy that will be corrupt, one copy that will be damaged and the third one prevents ulcers. All stored in different location, rotated and backed up on a regular basis (6 months/12 months?)

LTO3 choices
BEST:
Quantum scsi 160 800 GB LTO3 drive of choice:
http://www.quantum.com/Products/TapeDrives/LTOUltrium/LTO-3HH/Index.aspx
http://triointernational.com/all.cfm/partno/TCL32BXEY/stat/froogle

Good:
Dell option if you don't mind their lousy customer service from India:
http://www.dell.com/content/products/productdetails.aspx/pvaul_110tlto3?c=us&l=en&s=bsd&cs=04
 
You must log in or register to reply here.
Back
Top