Storage/Backup Workflow on the Road

[Alexis Hanawalt]

I plan on shooting a documentary this spring on the Red - and I'll be traveling around the country visiting many different cities. Because of the nature of hte project, I will need a system that is mobile and has a very large storage/backup capacity.

I keep coming back to LTO or similar tape backup solutions, but can't find a decent way to make that work with a Macbook Pro - they're almost all SCSI interface. I know there's an expresscard34 to PCI converter that could fit a SCSI card - but that all seems error-prone.

The few firewire and gigabit ethernet solutions out there for tape drives are a bit more expensive than what I'd like to pay. Raid enclosures don't seem like a suitable backup if they're traveling around with me...

Anyone have a similar dilema or any solutions?

 

[Kevin Halverson]

The solution that I use is physically small,fairly inexpensive but does require mains power. I use a Shuttle XPC (a very small form factor PC) with a SCSI III interface and a DLT drive. While I don't shoot docs, I still want on set back up capability and DLTs have proven themselves over time to be highly reliable devices.

I agree about the express card to PCI slot to SCSI card... seems like a long path and one that might be problematic.

 

[John Moores]

Hey cinewalt.
Same boat here. We're leaning toward just having a pelican full of the largest firewire drives we can get at the time, backing up to them, but also having some tape based option as well. We haven't got to the specifics of hardware interfaces etc yet, but there's got to be a way to do it.

Also, keep in mind, you won't need a raid, unless you want a backup to your backup. We're thinking a drive, and a tape is good enough. I mean, prior to this all you ever had was your master anyway, so this way you're already 100% better than every other shoot you've ever done before in terms of safeties.

I'd love to hear other's thought's about this as well.
Jm

 

[Fergus Meiklejohn]

exactamundo dude

I think that unless you are super scared about a particular very expensive impossible to repeat setup then dumping the footage onto a properly looked after g-tech drive on the day then duplicating the footage on separate drives while editing (and keeping them in different locations), is as backed up as we've been ever been so far. Remember Kim Longinotto lost entire reels of documentary footage on 16mm because the airport guys screwed up the search and xray'd the lot. If you can afford Raid 5 for the edit and tape backup and blah then total respect, but it's not suddenly essential surely. Or at least we don't know that it is yet..
I think my workflow will be: dump to g-tech qdrive in the field (on a SATA link because the RED drive is firewire 800 and I've only got one firewire 800 port on my macbook pro), REDCINE and transcode to prores422 for editing, and keep the REDCODE files as backup, and to go back to if necessary for an "Online"

JOY :biggrin:

 

[Adrian Correia]

exactamundo dude

I think that unless you are super scared about a particular very expensive impossible to repeat setup then dumping the footage onto a properly looked after g-tech drive on the day then duplicating the footage on separate drives while editing (and keeping them in different locations), is as backed up as we've been ever been so far. Remember Kim Longinotto lost entire reels of documentary footage on 16mm because the airport guys screwed up the search and xray'd the lot. If you can afford Raid 5 for the edit and tape backup and blah then total respect, but it's not suddenly essential surely. Or at least we don't know that it is yet..
I think my workflow will be: dump to g-tech qdrive in the field (on a SATA link because the RED drive is firewire 800 and I've only got one firewire 800 port on my macbook pro), REDCINE and transcode to prores422 for editing, and keep the REDCODE files as backup, and to go back to if necessary for an "Online"

JOY :biggrin:

This is exactly what i will be doing. Best in terms of cost, ease and security for my situation.

 

[John Moores]

I should probably just do a search but I'm feeling kind of lazy at the moment due to the sweltering temp here...;
In the field>
Dump the data via FW800, or eSata to Macbook (is eSata an output option from the red drive? Input option on the Mac?)

For output to the portable BU drive again eSata or FW800. Are there eSata portable enclosures? How much?

How long is it going to take me to copy a full red drive to the laptop, and then a)copy to the portable drive, and b)copy to a tape back up.

Math has never been my strong suit.

Thanks
Jm

 

[Stuart English]

LTO-3 to desktop for Mac / Win XP

LTO-3 to desktop for Mac / Win XP

Quantum have a model that does just what you are describing - i.e. with a Gigabit ethernet connection to any Mac OSX or Win XP computer - including laptops.

 

[Fergus Meiklejohn]

sweltering temperature..? You should come to London mate.. :sad:
I think I'll use g-tech qdrives, because they are quite small, come in 1 TB size, and have a SATA port. I think you are right to worry about how long it will take to dump 320GB from the REDDRIVE to anything else (when you are exhausted and it's late and you're getting up for dawn the next day..). The Macbook Pro only has one Firewire 800 port, so I though it would be best to bring the footage in Firewire 800 from the REDDRIVE and transfer it onto another external hard drive SATA via the 3/4 expresscard slot. It seems sensible I guess, and I can imagine sometimes having to dump 2 or 3 drives at the end of the day; which will take HOURS eh...:help:

 

[John Moores]

having a bit of a cold spell over there eh? nice

2 or 3 drives at the end of a day? Sounds like alot of footage. Multiple cameras?
Hoping to have a PA along who can transfer stuff, either at night or, with 2 or more red drives, during the day.

The other problem is, and I think it's been covered before, if you're really out in the middle of nowhere, where are you getting power for all these drives and laptops etc. Dragging a generator around isn't terribly appealing, and you'd need a truckload of batteries to keep all this stuff running.

 

[Alexis Hanawalt]

Here's why I'm not comfortable with regular drives as backup:

I have worked with nothing but external firewire 800 drives in my past 3 years professional editing experience. In that time I've had 2 1Tb Lacies die (both a year old, both died within a week of eachother) and we lost everything on them. At that time, we switched over to G-Raid 500's. Generally they've been fine - except a month ago when 1 of the 2 drives I'd archived a $500K project on died on us... G-Raid got it fixed for us and no data was lost... But - that's a few drives in a few years with substantial problems. A hard drive is guaranteed to fail at some point - and according to the people at Drive Savers who I talked to at NAB, there's no way to know - we all have superstitions about makes, brands, enclosures, etc... but they said they see it all from everything.

That coupled with the fact that I will be shooting a LOT - and I'll be traveling around by car with the camera and media - makes me wary of drives. Ideally, I'd be able to backup to 2 data tapes and fedex batches of them to a vault somewhere, keeping a copy of each with me. I'll probably have at least 300Gb of footage to backup every day for a couple months.

The last film I worked on shot on P2 cards and we used a G-Raid Pro with Raid-3 enabled. It was very reassuring knowing that we could lose a drive and not have to sacrifice 1/4 of a feature film.

For me it's like traffic tickets and speeding. I drove as fast as I could for years when I got my license because I never got a ticket. Then, 1 month I got two speeding tickets. Now I drive the speed limit. It doesn't take things going sour more than a couple times to learn to be very safe.

 

[Ken K]

I too have been looking for a portable backup solution and I'm getting frustrated. I'm not comfortable with backing up to hard drives, because I've had too many crash on me. G-Tech has a 2 year warranty on their drives, which isn't very comforting. I'd also like to know which drives they use. I'm tempted to do a custom RAID 5 solution with Seagate ES drives, but that's pretty darn bulky. Best for mobility would probably be a single eSATA 750GB drive (custom - no prebuilt solutions I've seen are impressing me) and some type of backup like tape. I wanted to do Magneto-Optical for rock solid reliability, but they're limited to 2.3GB per cartridge. Most tape solutions (LTO, SLR, VXA) require SCSI, but there some older VXA drives that do USB/Firewire. Some place MUST have an option for people like us. If not, I see a business opportunity here.

 

[Alexis Hanawalt]

G-Tech has a 2 year warranty on their drives, which isn't very comforting. I'd also like to know which drives they use.

They use the top of the line Hitachi drives - they are pretty good about taking care of you if there's ever a defect, regardless of the warranty.

 

[Curran Giddens]

They use the top of the line Hitachi drives - they are pretty good about taking care of you if there's ever a defect, regardless of the warranty.

While I don't have any experience with G-Tech or the G-RAID, I have had a lot of experience dealing with Medea's support staff (G-Tech is a spinoff of Medea). Medea has the best support staff I have ever had the pleasure of dealing with. They even fixed my old VideoRAID RT that was past it's 5 year warranty!

 

[Alexis Hanawalt]

I haven't ruled out the expresscard to scsi option by any means - I've just never heard of anyone using this. So people know what I'm talking about, here's the product: http://www.magma.com/products/pciexpress/index.html

They seem to indicate that it seemlessly works with any PCI card - I can't find any user or pro reviews of the product. If it does work - It's basically $700 for the device and a scsi card to be able to use any scsi tape drive with a laptop - still coming in way cheaper than the fibre, firewire, or ethernet tape drives out there.

An additional note - there's a cheap cheap cheap firewire to scsi converter out there. It looked like a miracle solution, but then I read a review posted by someone who tried to use it for tape backup and it kept freezing their computer - turns out the device slows down the data flow so much, any typically fast tape drive is going to take a million years shuttling tape back and forth...

 

[David Cox]

Here are some ideas I've been researching:

2-Port SATA2 Express Card/34mm MAC and Windows eSATA-EC
http://fwdepot.com/thestore/product_info.php/products_id/1753
They say that they are multiport able. That means that you can connect up to 10 drives from multiport external enclosures.

I've also found some really interesting products that I believe would be a nice fit for heavy achieving.

The new technology (quick swap) allows for popping in hard drives without having to mount the drive to a tray first. Remember the good ole days when we used floppy drives. Well this is the HDD version. Pop bare drives in, raid them if you please, or use single drive configuration, fill them up, pop them out.
See this link for specs:
http://www.cooldrives.com/sixhadresusb.html

What to do with all bare drives?:
Pelican cases sells this with up to 24 drive storage. They use a specific anti-static foam. Pop the drives in a case and ship to client or back to home base.
http://www.casesbypelican.com/hdrives.htm

I believe this process can be modified by mirroring the sata raid when initializing the drives. Just my two cents. I haven't ordered the equipment yet.I will do further research and report back.

_____________
Dave Cox
www.highwaydefinition.com

 

[Alexis Hanawalt]

By the end of the year it's reasonable to expect 500Gb drives to hit $100 a piece. That's only twice the expense of a similar data tape - Or, by my brilliant math, about 100 hours of 4K redcode raw before using drives surpasses the expense of a mobile data tape solution.

I'm starting to like this idea more - particularly from a workflow standpoint... As long as those drives are treated like gold it's actually pretty unlikely that there would be a failure - and if the material is truely precious, one could manually duplicate the drives.

I wonder if there's insurance that can pay for data rescue (usually $10K for a 500Gb drive) - that would be another way to put fears at ease.

---If some genious wants to get rich, invent a data tape module for the red - ironic, yes... but If one could record 4K to tape in the field, I think the "coolness" of tapeless workflow would subside.---

 

[Curran Giddens]

Thanks for the link. I just ordered the Pelican case with 24-drive storage since I have a bunch of bare drives laying around.

I'm currently using this device to connect the bare drives.

http://www.wiebetech.com/products/satadock.php

My old Sonnet Tempo-X 4+4 PCI-X card and FirmTek SeriTek/1EN2 combo is too unreliable (drives sometimes don't mount). I don't want to get another PCI-X solution since I will be getting a Mac Pro with PCIe by the end of the year....

 

[Ken K]

I wonder if there's insurance that can pay for data rescue (usually $10K for a 500Gb drive) - that would be another way to put fears at ease.

Wow, that sounds a little steep. I've done data rescue on dead hard drives and have been quite successful. Same tools. So... if anyone needs my services, I'll just charge you an hourly rate for my time. You're looking at 3 figures instead of 5. But if insurance is paying for it, it might be 4 figures.

 

[Alexis Hanawalt]

Wow, that sounds a little steep. I've done data rescue on dead hard drives and have been quite successful. Same tools. So... if anyone needs my services, I'll just charge you an hourly rate for my time. You're looking at 3 figures instead of 5. But if insurance is paying for it, it might be 4 figures.

The quote I got was from Drive Savers - who every drive manufacturer refers you to when your drive has physical damage. They're a "class 100" facility, one of only a handful in the country, that will crack open the drive itself to attempt to rescue data. The problem with their steep quote is that they don't guarantee a result.

 

[Ken K]

Ahh... okay. That makes a little more sense. I don't go that far since I don't have a clean room. I just do recovery from crashed drives or accidental deletions, using forensics tools.