RAID or not?

[David McDonald]

I've recently been hired as editor for a low budget feature film and I'm trying to decide what kind of storage we'll need. We're shooting on the Epic, and I'm predicting we will probably need around 10 TB for the R3D files & the ProRes proxies that I'll be editing in.

I know everyone uses RAID set ups but I don't really know if it's necessary for this.....basically all we want is to have a set of drives to edit with, and a set of back up drives to put somewhere else for safe keeping. And as far as I know, all RAID is good for is just backing itself up, so if we plan to have separate backups, is there any point in getting RAID drives? Am I wrong?

My plan was just to buy 2 TB or 3TB Lacie or G Drives (maybe someone could persuade me which is better), and get a few of those adding up to around 10 TB....and then double that for back up. This is my first time editing a feature so I've never dealt with anything at this scale before so any advice would be appreciated.

 

[Barry Bishop]

Check out Drobo, cant say enough good things about ours.

 

[Harrison Diamond]

The Drobo might be serviceable for the backup, but IMHO would be a mistake for the primary drives.

Even for the backups it's pretty slow.

Of course, if you want one, I've got one you can buy

 

[joe hedge]

Who's transcoding, you in editorial or the DIT on-set? You or he, or maybe the post house, will need a MacPro/RedRocket for transcoding

Will there be shuttle drives to get dailies from set to editorial every day, so you can start asst editing and rough cutting during production, or are you going to wait till the shoot is done?

You don't really need RAID for either on-set backup, transcoding or editing, it speeds up the back-up and transcoding (especially if the DIT is doing both at the same time), but the transcodes are low-bandwidth enough that RAID isn't needed for editing. I'd make sure the GPU of your editing system can handle multiple tracks of 1080p, if not, maybe transcode to 720p

I usually go with G-Drives, LaCies have a pretty bad reputation...quite a few other caveats involving basic hard drive practices, make sure you hire an experienced DIT, preferably with MacPro/RedRocket

 

[David McDonald]

Honestly it's a pretty shitty situation right now.....it's such a low budget film that we currently don't have a DIT or Assistant Editor yet who is willing to work for little or no money, and the shoot begins next week. They want me to begin rough cutting as we shoot, but we currently have no Red Rocket, or the DIT and Assistant Editor, so right now there's no way that I'm going to be able to cut while we shoot. We're working on getting all those things but there's almost no money left in the budget for this stuff, which is ridiculous, but it's the way it is.....so I may just end up being a DIT/Assistant editor during the shoot, and then begin cutting when the shoot is over. We're located in Vancouver if anyone is available to fill any of those positions

I'm probably just going to get a bunch of G-Drives, and back up the R3Ds on one set of drives, and edit on another set of drives.

 

[joe hedge]

They want me to begin rough cutting as we shoot, but we currently have no Red Rocket, or the DIT and Assistant Editor, so right now there's no way that I'm going to be able to cut while we shoot.

Have you tried to edit Raw 5k R3D's yet?

 

[D Fuller]

My plan was just to buy 2 TB or 3TB Lacie or G Drives (maybe someone could persuade me which is better), and get a few of those adding up to around 10 TB....and then double that for back up. This is my first time editing a feature so I've never dealt with anything at this scale before so any advice would be appreciated.

I'd recommend a raid, but you can do it like this if it's all you can afford. Just remember that you almost always have to trade time for money.

I'd recommend G-Raid drives. I find them to be very reliable, and I think they ae a bit faster (no quantitive tests for that, just a gut impression). But make sure you're connected to them over SATA. Don't even think about NAS or USB connections; even Firewire 800 is too slow for the project you're undertaking. And don't ever skimp on the additional drive for backup, the render drives are going to get a real workout,

I'd recommend transcoding to a fairly low-bitrate version of your editing codec. (ProRes 8-bit 4:2:2 or DNxHD-36 8-bit) The trick is going to be transcoding all the footage without a Red Rocket.

For that, try to set the in-camera look as accurately as you can, and just render the footage out using that setting for the roughcut. I'd recommend selecting 1/2 good debayer, and I'd batch render in Redcine-X 506, not RedCine-X Pro. 506 is more stable for long renders than RCX Pro, and you're going to be doing a lot of long renders. The Alchemy group won't be available, but it will slow down your renders more than you can afford anyway.

Find as fast a machine as you can that you can dedicate to this task for the duration. 10TB of footage is going to take a lot of time to render, but it'll get done.

Many of us were doing real work before there was a Red Rocket, so you can too.

 

[M Most]

I'd recommend transcoding to a fairly low-bitrate version of your editing codec. (ProRes 8-bit 4:2:2 or DNxHD-36 8-bit) The trick is going to be transcoding all the footage without a Red Rocket.

For that, try to set the in-camera look as accurately as you can, and just render the footage out using that setting for the roughcut. I'd recommend selecting 1/2 good debayer, and I'd batch render in Redcine-X 506, not RedCine-X Pro.

Assuming there will be sufficient time and/or resources to conform the final version from the original R3D files, I'd recommend no more than 1/4 debayer - actually 1/8 will work also - and rendering either to 1280x720 if you're using Final Cut (ProRes LT or Proxy will work just fine) or either 1280x720 or 1920x1080 using DNxHD36 (for 1920) or 60 (for 720) if you're cutting on Avid. In either case, using 1/8 debayer will give you a very good image and transcode in faster than real time, without a Red Rocket. And even using 1/4 debayer you should be able to get close to real time performance with today's hardware. Many of us who have been dealing with Red footage for a few years now used these settings with Red One footage prior to the arrival of the Rocket card with good results, and some of us even continued to do it after the card was released.

 

[Barry Bishop]

Sorry, yes drobo for backup. Kingston hyperX SSD's is what I use for my raid.

 

[joe hedge]

Many of us who have been dealing with Red footage for a few years now used these settings with Red One footage prior to the arrival of the Rocket card with good results, and some of us even continued to do it after the card was released.

Except we're talking about 5k Epic footage here. I've tried to transcode Epic 5k without a rocket, using 1/4 res debayer to DNxHD 36, it took overnight to get about half-way through 1.5 hours of footage on a Nehalem 8-core writing from a G-Raid to a separate, empty transcode drive...

 

[Zeb B]

I'd recommend G-Raid drives.

Just had a newer 6T G-Tech die on me : - (

 

[Von Thomas]

Hold your horses group. For those not in the know, Lacie is crap!! Except for the Rugged Drives (the Orange covered small ones), Lacie has not been a reliable storage solution. If yours has not died, consider yourself lucky, and move on. G-Tech, well this is a love story gone bad. I endorsed G-Tech for years, and they were reliable for years, but since Hitachi bought them a couple of years ago, the product has changed. I've had more G-Tech's die (right out of the box) in the past year than Lacie over 5 years combined.

I recommend Maxx Digital drives, why, because they are a small company, and they buy premium product. They stand behind their goods, and that works for me. To date, all their drives, have done nothing but work, and work well. Bad thing is, you can't get their product everywhere, hopefully that will change soon.

As far as drives for your project, a good solution (but a bit pricey) is to go RAID, it is safe. I know many of you, want to save a buck, but at what cost? I'd rather be safe, than sorry down the line.

Von

 

[D Fuller]

Just had a newer 6T G-Tech die on me : - (

Sorry to hear that. I've always had good luck with them.

But if it's a 6-Gig drive, it must be a raid, and a raid 0 at that. So the chances of the drive failing are doubled. I would never recommend a raid 0 for anything I'd consider a backup. And never for anything I don't have a separate backup of either. I've mostly used the G-Tech G-Drives that are single drives. I've got a fast, 8TB raid 0 on my workstation, but it gets frequent backups. It's not to be trusted with the preservation of data.

 

[joe hedge]

But if it's a 6-Gig drive, it must be a raid, and a raid 0 at that. So the chances of the drive failing are doubled. I would never recommend a raid 0 for anything I'd consider a backup.

Except that if it's being used for back-up, by definition that means there's 2 of those G-RAID's, and both of them are extremely unlikely to fail at the same time - kind of like 2 tires blowing out at the same time. Especially if you format, test and treat them correctly. Also, if you buy the G-Raid's at a local retail outlet, and one of them fails within 30 days, you simply take it back to the store, they give you another one, you copy everything to that, and you're back in business with 2 back-up drives the next day.

 

[Wes Howell]

I don't understand the logic behind all the Lacie vs Gtech vs other retailer, etc.

My experience has been that it's the drive inside the box that matters. i.e. WD, Samsung, Seagate, etc. These manufacturers all have bad runs or drives... and the retailers often use specific drives so you can find out what's under the hood. You can research this and make your purchase based on specific drives. IMO, I don't think the enclosure makes a whole lotta difference. And FWIW, I've used Lacie for the past 5 years and I don't have one dead drive.

 

[Peter Stelzhammer]

it's not all about the drives inside (or let me better say "not only") but also about the housings/enclosures, the power supply, and so on... sure - a harddisc can die anytime in any case (proper cooling can help to stay inside specs for eta. lifetime)...

and I'd also state that I've witnessed more failures on LaCie units then on G-Tech units over the time of a few years.
doesn't mean thats the case for everyone else... I wouldn't call it "crap" - but in my experience I'd say LaCie is a bit more prone to failure...
(again: not only the drives - also the electronics of the enclosures itself)

I'm having good experience with WD (not with the Green-Series disks trough), G-Tech, CalDigit...
Needless to say that having a (real) backup and/or redundancy is much better then having full confidence in 1 RAID-unit only
(shit can happen - e.g. during a rebild process)...

 

[Tony Lorentzen]

I totally agree that you should avoid the LaCie drives. Also - the quality of the G-Tech has gone down over the last couple of years. I've heard of dealers stopped selling them because of too many problems and a shitty support. A guy I know has waited 6 months (!) for a new power supply for his drive. He ended up buying a new external enclosure and swapped the drives over to that successfully. In my book there's only CalDigit, but I've heard lots of good things about Maxx Digital too. Drobo is not for high speed editing - only back-up (though I've heard quite a few horror stories about failing Drobos and lost backups too).

 

[Blair S. Paulsen]

I would echo the warnings about Drobo and LaCie. I had 3 G-Tech drives fail on two consecutive projects about a year ago which kind of shocked me as like many others they had been the go-to choice since late 2007.

In any case, for archive I prefer single drives to any form of RAID. I always recommend that multiple copies be made in separate passes since if one read error gets written twice you have two copies with the same exact corruption. On a related note, when tight on budget I prefer to get a decent quality enclosure (decent chipset, good cooling) and populate it with bare drives which I park in anti-stat cases for storage. If you buy a bunch of ready made drive/enclosure combinations you end up buying a bunch of chipsets, fans, ports, etc that spend most of their life on a shelf which is wasteful.

Several of the popular enclosures (Towerstor, OWC, Stardom, Icy Dock, etc) can run either as RAIDs or JBOD so if you decide to go RAID at some point in the process you can load in fresh drives and re- format.

Cheers - #19

 

[Joel Arvidsson]

I totally agree that you should avoid the LaCie drives. Also - the quality of the G-Tech has gone down over the last couple of years. I've heard of dealers stopped selling them because of too many problems and a shitty support. A guy I know has waited 6 months (!) for a new power supply for his drive. He ended up buying a new external enclosure and swapped the drives over to that successfully. In my book there's only CalDigit, but I've heard lots of good things about Maxx Digital too. Drobo is not for high speed editing - only back-up (though I've heard quite a few horror stories about failing Drobos and lost backups too).

What do you do about the noise of the Caldigit? I never turn my on since it sounds to much. And the cable is to short to move it to another room. I have the S2VR with 4 drives.

What is the max length of an Esata cable?