Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Backing up data "on the road"...

stevesherrick said:
Are there any plans to do testing at LART for backup systems? I know you guys have a lot on your plate, but was curious if this was one of the things on the agenda.

Steve
Click to expand...

Great suggestion - I'll pass the request along to the LART workflow team for possibilities.
 
Why not just use a bunch of 500GB or 1TB external harddrives? They work well enough, and they are cheap now. Make multiple copies on externals and split them up among your crew. Maybe even mail copied drives back to your home or office every now and then using Fedex or whatever?
 
sharkguy said:
Thanks for the feedback on cooldrives - I can see that their customer support sucks based on their sales policy "People are stupid - if you buy it and don't know how to use it or set it up don't bother us." Not finding a lot of other dual drive RAID 1 500 GB options though. The G-Safes are out of stock at Gtech and many other vendors. Too bad, a great product.

Your triple disk idea works also. My practice thus far is to make the RAID 1 dual drive copy of the cf cards on set. When the 500 GB drives are near capacity I swap them both out, label and save the dual drives for LTO 3 back up at the studio (or in the field if feasible). You can then pass one drive to client and keep second as a back up. I am sticking with Seagate 500 GB drives for now due to their reliability and track record - the 1 TB drives are a bit new in comparison.

For the future of 4K footage and amazing stock footage, I am practicing the 3 back up system with the LTO 3, you know, one copy that will be corrupt, one copy that will be damaged and the third one prevents ulcers. All stored in different location, rotated and backed up on a regular basis (6 months/12 months?)
Click to expand...

Have you tried looking at this: http://eshop.macsales.com/shop/firewire/usb/raid_1/Gmax It supports Raid 1! I've actually got two OCW Mercury cases on my desk that support RAID 0+1--that's what I am probably going to use on set.

Thanks for the tape drive suggestions. As much as I don't want to spend the money on that, it's gotta happen since I won't have any other tape to go back to for shoots. At $35-$40 a tape it's still cheaper than shooting DVCPROHD tapes! Of course, if you are giving a HD to clients, then you've got to add the cost of a 500 GB drive also. Then again, both of those together is cheaper than a few feet a film.

Matthew
 
Tom said:
Why not just use a bunch of 500GB or 1TB external harddrives? They work well enough, and they are cheap now. Make multiple copies on externals and split them up among your crew. Maybe even mail copied drives back to your home or office every now and then using Fedex or whatever?
Click to expand...

I think the problem with this is whether or not you data is getting verified properly. Won't do you any good if the data that is copied onto three drives all have the same inherent corruption. Not saying that the workflow isn't valid, I just feel that we need to have established protocols for high level productions so that there is a confidence level amongst producers and financers that the data files are safe. I know there are current standards on movie sets that are shooting with the Genesis and other cameras where they are backing up to RAID and LTO or DLT. I guess what I'm looking for is to establish a whitepaper specific to the Red camera that outlines best practices for mission critical projects and perhaps others that detail solutions for other types of productions as well.

Cheap solutions are all well and good, but in my opinion the cheap part has to come into play after you have established the first two pieces of the pie. If budgets are very tight, then you do what you can, I completely understand that. But some projects will have very little room for error.

Steve
 
LEON said:
it seems OK but time consuming if you want to swap the disks because they are not fixed in a tray. It is rather a fixed installation, no ?
Click to expand...

It is fixed, but if you were going to give the HD to a client, you would have to remove the drive from the tray anyway, so why not from a case? I can understand if you are just swapping out for your own use...

Matthew
 
No clone on location?

No clone on location?

This may sound ridiculous but as an Aussi based in the UK I know how much excess baggage can eat into a Aussi production budget.
BA now meant to charge a flat rate of £75 per case per sector but I was quoted £120 from South Africa to London last week.
Other airlines charge +$30 per kilo.

So how many CF cards do we need for 40 hours!

Is there a CF rental opportunity here?
Producer can fedex them back or hang onto them as he would tape.

When the RED drives becomes available this will be even cheaper way to go.

You'de only need 15 RED raids. Dont let the munbers scare you, why do we think 15 drives is a high number and 40 tapes isn't?


Savings in excess baggage would be significant compared to taking firewire drives and computers and enough incentive to rent a 15 RED drives?

Think of the savings in time and not putting doc crew under pressure to clone stuff late at night ect.


Renting a piece of kit worth $15k (15 x RED drives) is no big deal for a production. 15 red drives take up less space and excess baggage than 40 HDCAM tapes.
$15k doesn't even buy a HD 2/3 inch lens.....



Mike Brennan
 
For mobile hard drive solutions, take a look at the caldigit ones. www.caldigit.com. They have dual-drive RAID units with USB2/Firewire800 interfaces or dual-drive and 5-drive units with an eSATA connection.

I will be bringing one of their dual-drive S2VR eSATA RAID units to LART. It measures roughly 4.5" x 5" x 9" and weighs about 7lbs. Will be connecting it to a Macbook Pro via the Caldigit ExpressCard34 eSATA adapter.

The one I'm bringing has 2x750GB HDDs in it and I'm planning on using it for RAID-0 workspace while there. But maybe I'll do some preliminary testing with it configured as a RAID-1 for offload and backup purposes first. I'll be bringing it empty, I guess we'll see what the schedule allows.
 
Jeff Kilgroe said:
For mobile hard drive solutions, take a look at the caldigit ones. www.caldigit.com. They have dual-drive RAID units with USB2/Firewire800 interfaces or dual-drive and 5-drive units with an eSATA connection.

I will be bringing one of their dual-drive S2VR eSATA RAID units to LART. It measures roughly 4.5" x 5" x 9" and weighs about 7lbs. Will be connecting it to a Macbook Pro via the Caldigit ExpressCard34 eSATA adapter.

The one I'm bringing has 2x750GB HDDs in it and I'm planning on using it for RAID-0 workspace while there. But maybe I'll do some preliminary testing with it configured as a RAID-1 for offload and backup purposes first. I'll be bringing it empty, I guess we'll see what the schedule allows.
Click to expand...

I just got two of these exact drives. They are great. Just today, I was on set with an HD Camera, my MacBook Pro, this Caldigit drive, and an AJA IO HD. I am testing a new system. I recorded in prores 422 at 1080i23.98psf, and it worked like a dream. I loved the fact that my rough 'video assist' footage was of higher quality than the actual DVCPRO HD tape in the camera (their actual master).

Anyways, I can vouch for these Caldigit hard drives. They are good.

But still, for me, I am going to avoid the whole raid 1 thing, and go with two separate drives. Why? Because I want to be able to peel one off, give it to production, and keep one as the backup. If someone happens to drop it that night and break it, I will still have the backup. If they dropped my CalDigit unit, the whole thing might break.
 
I'd like to chime in on a few things mentioned here.

At Silverado, we've been thinking A LOT about these details since Mike Curtis asked me my opinion about this a while back. There are some considerations that I don't think have been mentioned yet.

The AIT-5 tape drive is awesome for size and capacity, but since its SCSI based its a non-starter for taking onsite with a MacBook Pro.

Further, ANY tape type device that is SCSI or SAS based is not really practical for a laptop in the field.

You need to look for single points of failure in your data chain. Your first SPOF is the CF card itself. If that goes bad or gets damaged, your data is hosed.

We all know that, so lets move on to the next stage.

Backing up to a single drive in the field is dangerous since the drive will most likely be used in a field-situation that is dusty, hot, and subject to bumps and bruises while getting dragged along the way through your shots.

Talking about drives in the field. I think it is critical to have at least a 2-drive RAID-1 setup for transfer. With 1 Tb drives as the largest available, you are looking at two drives for a 1.0 Tb volume in RAID-1, or four drives for a 2.0 Tb RAID-1 volume--basically 10-20 hours of footage.

How do you carry all this stuff without resorting to baggage handlers who toss your stuff like a discus thrower?

From an insurance standpoint, what about insurance cos. that mandate that copies be made and kept offsite? This would preclude the drives-only approach.

We are working on the following package:

1. LTO-3 tape (400 MB native)
2. 2-4 Tb of storage on HD
3. Small enough to bring on a plane as a carry-on so you DON'T have to check it.

You would copy to your RAID-1, QC the footage, then backup to tape when you have aggregated enough to warrant a backup. Then, review off the RAID-1 while shooting a safe copy (or two!) on LTO-3 to an independent location via Fedex.

Further, we'll build into the system the value of having loaners that we can ship anywhere in the world overnight if something goes down on one of your components.

Give us a week or so and we'll have more information on the website. We'll explain everything probably via a video Podcast or a PDF.

This is a great thread and everyone has put in a lot of solid thinking.
 
In theory, I'd like my workflow to be...

On set, backup to a RAID-1 two drive system. Make a third copy to either one of my firewire drives or the client's drive and have someone else go home with that copy. When I get home, make a copy of the all that data to a LTO-3 tape and copy the data from the RAID-1 to a RAID-5 system for editing.

The only issue I am worried about with that workflow is making sure I have enough time between shoots to handle/copy all of that data safely. I don't want to have three different shoots day after day after day. Of course, that workflow is already far more safe than just shooting a tape and there only being one copy of that tape!

I've got to write this whole system up so I can show producers how easy and safe it is (heck, I should just make a video!)

Matthew
 
Other considerations. Laptops will need to have enough battery power to last a 10 hour or more day. If you are providing AC to the laptop and are using external drives that need power then you have to factor in a UPS system. There's no way I would take the chance of running this off a power strip only. As for LTO 3 and a laptop, the LTO-3A has gigabit ethernet and could work in that arrangement. Personally, I'm waiting for LTO-4 with it's increased capacity and speed.

I'm thinking of having a cart engineered that can house all of this gear in a compact, robust form factor with built in shock absorption for relatively easy maneuvering around set. It will have batttery backup built in and can even hold a desktop mac.

I agree, let's keep this thread going. A lot of details to be worked out.

Steve
 
sharkguy said:
For the very paranoid producers not comfortable with tapeless field acquisition I wonder what it would take to utilize the Sony HDW-S280 to make 1080/60-50i tape back ups in the field from the red drive media in 4K/2K?

I would presume the deck could be gotten for $1500 a week?

A pain in the butt, however, might be a good intermediary step for producers?
Click to expand...

this, as well as a lot of other suggestions, are getting a bit over the top dont you think? if you are in a situation where you have to record to 1080, why dont you take a 1080 camera with a good lens that you can carry in one hand and be done with it!!!

I would love to see a "run & gun" crew hauling around a red kit with lenses, accs, support, HD monitors, HDDs, laptops, VTR decks... perhaps you might need a spare camera body just in case (which would then get used as a second camera), also a 4K projector for dailies viewing would be nice as well!!!
 
There will be different levels of production with Red cameras just as there are with any other camera on the planet. I can assure you that a feature film set and even a high level commercial set will have all of the accessories you mentioned. Other productions that are going to be "run and gun" will have to equip themselves with whatever makes sense for their given situation. They may have to take more risks because there just isn't enough time, manpower, or gear available to do the high level of data protection and tape backup that the film setup can afford.

I just wouldn't underestimate the importance of data backup. I'm fairly confident that the first shoot you do where you lose a day's worth of shooting, you'll have trouble getting work in the future if it's a big enough production. This is a professional camera format, not a DVX (although I still love that camera).

Having said all that, people can use the camera any way they want. I think some of us are just trying to implement the best backup strategy we can, so we're tossing around ideas.

Steve
 
*****************************************************
4. The other and most simple choice is, untill a workable solution comes along, keep your shooting ratio low and treat RED footage like you would a 35/S16mm and have enough RED magazines.

I think it's important to remember that for the best part of 100 years the world has been shooting on a format that could not be backed up. So try and learn how the hell they managed it with so much sucess.
*****************************************************

I think that the above quote is the best post on this tread so far, and not because it has anything to do with film.

I know this is about backup on the road, but the reality is that there will always be compromises depending on your work situation, and in the context of doco, perhaps you have to ask a few objective questions such as "do I ACTUALLY have to back up on the road" or the big one.... "is Red ACTUALLY the best tool of the job?"

If want to shoot doco for cinema with a non-red camera capable of high resolution we are lookng at a 35mm kit right? If you made this decision, you would be buying into the fact that there will be more, heavier gear, you would have to carry around with you enough film rolls to cover what you want to shoot, you dont have backup so if you post any rushes you accept the risk off loss or damage and you probably get clever on your coverage (like people had to be before tape)

If this is not an option, then perhaps you consider 16mm, cause although it's lower resolution, you can blow it up to 35mm for release, less gear, lighter gear, longer run-times on mags, etc, etc.

Lets not forget that both of these camera formats are very robust and experienced camerapeople/ ACs can fix them in the field. A loaded film mag can potentially recover from being submerged in salt water!!

In the case were you might want to roll for excessive amounts of time and film was not within the budget, you would pretty much have to go with an acceptable tape based option, and live with the quality loss, perhaps grade in post or just intercut a different look into the film footage.

Applying this to Red:

4K: If you made this decision, you would be buying into the fact that you will be carrying a significant amount of gear (lenses, support, etc. Add to this a backup workstation of some sort and it could easily equal a 35mm kit). OR, you would have to carry around with you enough Red drives to cover what you want to shoot, you dont have backup so if you post any rushes you accept the risk off loss or damage and you probably get clever on your coverage (like people had to be before tape)

If this is not an option, then perhaps you consider 2K, cause although it's lower resolution, you can blow it up to 35mm for release (digital projection at this stage is mostly 2K anyway), less gear, lighter gear, longer run-times on HDDs, etc, etc.

Lets not forget that both of these camera formats are a lens attached to a computer and experienced camerapeople/ ACs can't fix them in the field. A HDD would never recover from being submerged in salt water!!

In the case were you might want to roll for excessive amounts of time and storing so much data was not within the budget, you would pretty much have to go with an acceptable tape based option, and live with the quality loss, perhaps grade in post or just intercut a different look into the RED footage.

also consider:

If backup is not feasible (and it certainly isn't practical in the field yet), but reliability is key, Ditch HDDs altogether and use CF cards. They are small, light & reliable. Suddenly you are a lot more portable, you don't have to consider hours of backup time, etc, etc. Sure record time is limited, but no more so than a film magazine. Sure they are expensive, but no more than a roll of film.

I have had to deal with portable backup already and it is enough of a hassel working out of the back of a van in the middle of a major city, let alone in the jungle halfway up a mountain. It's slow too. It takes longer to download a CF card to a HDD than to download a magazine of film, and you need power.

Don't kill your crew by making them stay up all night after a long day shooting, sitting by a blueray burner swapping discs every 40GB!!!!

Accept what is most practical might not be the cheapest option, then decide to do it or not. I'm sure before long this whole dilema will be a non-issue with high capacity low cost solid sate media (or something equivalent that we havent thought of yet), but until then it's all about compromising to achieve what you want to.

SPRINGER'S FINAL THOUGHT:

"Remember, it's not what you shoot with, it's what you shoot & how well you shoot it that will dictate how good your film is."
 
Interesting points. I think a lot of doc work will be shot in 2K for exactly the reasons you mentioned. To me, this is all scalable in terms of backup solutions. When time and budget allow, there will be extensive backup done, a lot of it right there on set. Shooting in the jungle with little or no power, then that changes the strategy. But I still wouldn't shoot 40 CF cards and hope they get back to post safely. Too risky. Somehow, some way you need to make a backup, even if it's hiring someone to be doing this with a laptop and a portable drive. When all else fails and the only option is to shoot CF or Red Drives with no backup, then that is what you have to do and as you mentioned, this has been done with film and videotape for so many years.

Each project will carry its own set of requirements and will be tied to contracts, insurance, producer's comfort levels, etc.

Steve
 
torster said:
Backing up to a single drive in the field is dangerous since the drive will most likely be used in a field-situation that is dusty, hot, and subject to bumps and bruises while getting dragged along the way through your shots.

Talking about drives in the field. I think it is critical to have at least a 2-drive RAID-1 setup for transfer. With 1 Tb drives as the largest available, you are looking at two drives for a 1.0 Tb volume in RAID-1, or four drives for a 2.0 Tb RAID-1 volume--basically 10-20 hours of footage.
Click to expand...

I'd like to mention LaCie Rugged drives again here.

They're practically pocket-sized, they're FW800, they're bus-powered, they're shock-resistant, and they're nearly sealed (I suppose dust could get into the ports, but they have no vents). I feel a lot more comfortable traveling with these than with 3.5" drives in full-sized external enclosures, which are much less likely to survive a fall, need external power, and typically have vents for dust to get into (and often even have fans to helpfully pull dust in and make noise on set).

And yes, I too have grown leery of LaCie's desktop drives. But we've had a bunch of these kicking around for awhile, and we haven't had any problems. Despite the fact that they've probably all been dropped a few times by now.
 
stevesherrick said:
There will be different levels of production with Red cameras just as there are with any other camera on the planet. I can assure you that a feature film set and even a high level commercial set will have all of the accessories you mentioned. Other productions that are going to be "run and gun" will have to equip themselves with whatever makes sense for their given situation. They may have to take more risks because there just isn't enough time, manpower, or gear available to do the high level of data protection and tape backup that the film setup can afford.

I just wouldn't underestimate the importance of data backup. I'm fairly confident that the first shoot you do where you lose a day's worth of shooting, you'll have trouble getting work in the future if it's a big enough production. This is a professional camera format, not a DVX (although I still love that camera).

Having said all that, people can use the camera any way they want. I think some of us are just trying to implement the best backup strategy we can, so we're tossing around ideas.

Steve
Click to expand...

Hi Steve

yes I agree with you. I was really refering back to the origin of this post with was about practical backup solutions in the field for a doco. i don't see lugging around VTR decks & computer work stations as practical for a doco.

Yes I have worked on high end commercials and feature films where they have this gear, and oh so much more. They also have had a shooting crew of up to 700 people. not exactly doco.

I have also worked on low budget docos where we had 4 people knee deep in a muddy swamp, so kind of looking at it from the two extremes here.....
 
I have been thinking about this for a few months. I am about to start on a large project that will go coast to coast. I am currently shooting P2 media so it's relevant to that workflow as well.

1. Have enough P2/CF cards for a days work, in my present situation enough to cover 4 hours of shooting. I have worked on docs that I would shoot all day and only walk away 2-4 hrs of footage a day.

2. In an emergency be able to copy off to a laptop or a Hyperdrive unit.

3. At the end of the day copy the P2/CF media to a Raid 1 setup, I think 1 TB Raid 1 would work for most situations.

4. Go through the shots to do a one light grade as time allows, also set a few looks in camera. Most situations are totally uncontrolled so a generic look would be a useful starting point. Being able to do a quick grade and storing the metadata with the clips is important.

5. When the section is done or weekly, whichever works, make 2 single drive copies and send one to editorial and another to a holding area. The drive can be copied onto the producers Raid 5 storage and the drive sent back to the crew. The second holding area could keep the drive as longer term storage. I also think I will keep a copy of the material for myself, and secondary backup. When ready to start the next section, of course after verification from editorial, erase the Raid 1 and start all over again.

One thing that a 3rd party might develop for this new tapeless workflow would be a firewire (to use any kind of reader you want) in to SATA drive (or maybe raid 1), removable, copying device that incorporates data verification for on the road crews who need the minimum amount of gear. Battery powered might be good or at least able to use AB batteries or the same Vmount batteries that the camera uses. Some CF backup devices work this way now. I have a SmartDisk FlashTrax that works pretty well, I don't think it has any data verification built in though. It's small enough to fit in my backpack with my camera so when I run out of CF card space I can free up some cards. I shoot Hi res panoramas so I do run out of space occasionally.
 
I can second the recommendation for LaCie Rugged drives. Ours has been back and forth through Fed Ex almost 50 times without a problem. They can take a beating. We use them to send our finished shows off for captioning and output to HDCAM.
 
You must log in or register to reply here.
Back
Top