RED Scarlet Set Up - RAID Question

[Patrick O'Sullivan]

Hi,

We've got a Scarlet X package coming our way and we are taking the time to boost our post production workflow.

When I say we I mean me. In the past we have only been working with H.264 stuff in Premiere Pro CS5 and 5.5. Sending it over Dynamic Link and coloring in AE with Colorista II.

We are going to add Resolve once they make it available Q1 2012 for PC.

So far we have been very happy with our system performance. The only reason we are changing or thinking of changing is because of the increased file size with the .R3D files.

My question is about RAID set ups and speeding up post by best utilizing one.

Our main editing station is running as follows:

i7 950 @ 3.07
12 GB RAM
Windows 7 64-Bit
1 GeForce 470 GTX
Sharkoon SATA QuickPort Duo USB3.0 -v.2-

So basically ever few weeks I get a new 1TB Samsung hard drive and throw it in the Sharkoon and I just start offloading footage to it. I then use another 1TB Samsung to output finished comps and project files to.
When the drive fills up I replace it.

Surely this isn't the most effficient way to do it. I have no idea how to properly use a RAID array to get the most performance out of my system. I've looked around the forums and elsewhere but I can't grasp how an external RAID thing like the G Speed es (http://www.g-technology.com/products/g-speed-es.cfm) works.

I don't get it. I mean I understand that the read and write speed increases by spreading the data over the different drives but what happens when the capacity is reached. Do you have to replace all 4 drives? What if you want/need to access that footage once you've swapped out all the drives. Do you have to put all 4 drives back in or just one or what?

How much does it actually improve performance having one of these External Raid devices?

Any advice or knowledge would be much appreciated.

Thanks

 

[Brian Merlen]

there are various raid settings which will effect your answer. raid 0 means you have no safety net but all your drives can write at once. if one drive dies you lost it all. but you get a massive speed increase since data is shared between drives, so your working at double the speed with 2 bays or quad speed with 4 bays. you should try to use a fast port if you have a few bays in a raid 0, thunderbolt, usb3, esata, something along those lines.

raid one and 2 bay drive makes both your drives have the same stuff on them. it runs at a capacity like using single drives, but has safety in the since if one drive dies you got all the data.

raid 5 is the best of both worlds, it lets you get a slight speed increase but you need i think 2 drives to fail to be screwed.

there are also various other settings, each having a benefit comparable to the three above. its sort of a trade off for speed vs reliability, vs total storage size available. you still will need to buy drives all the time when your out of room, you still need to manage media and be aware of where things are if you need a file you can't find.

the more bays the more pricey it'll get, try using 3tb or 2tb drives to gain a little extra wiggle room for data wrangling. data recovery costs a lot more then drives, raid 0 has its risks without you having another raid to back it up to...

having a raid with a few bays can help you out with not needing to fill them up quite as often but it will still happen eventually. the setting you pick will play a significant factor in this answer. get one with a few bays and use 3tb drives, it will bide you some time between swap outs. they sell models with massive amounts of bays, but they get expensive quickly.

http://en.wikipedia.org/wiki/RAID

 

[Patrick O'Sullivan]

OK

I am on to it. Now I understand all the aspects of it. I am looking to get a competent RAID 3 system set up.

From my research it looks as though I'll need an External RAID enclosure, a RAID controller, and the drives.

Any suggestions?

I'm interested to hear what others are using and the read write speeds they are getting.

My budget is $3,000.

Right now I am looking at Areca ARC-1220 (http://www.pc-pitstop.com/sata_raid_controllers/arc1220.asp) for the Controller, and http://www.pc-pitstop.com/sata_enclosures/scsat84xt.asp this enclosure.

The drives will just be Samsung SpinPoint F3 Desktop Class 1 TB.

Looking at RAID 3 at the moment as a bunch of people over on the Adobe forums recommend that set up.

Any thoughts or suggestions?

 

[Mark George]

Hi Patrick,

I found this thread to be very helpful. http://www.reduser.net/forum/showthread.php?64176-G-Technology-s-G-SPEED-eS-PRO.-Thoughts

And this review: http://www.tomshardware.com/reviews/sas-6gb-raid-controller,3028.html

I have decided to get an Areca 1080ix and an ARC-4036 enclosure (hard to locate, but CDW carries them). I will most likely buy 5 or 6 Western Digital Caviar Black HDDs (the 6 Gbps versions). I don't own any of this equipment yet, so I cannot speak to how it actually performs.

I use Win7 and PPro 5.5.

Mark

 

[Rudy Lehfeldt-Ehlinger]

Patrick,

I would STRONGLY advise against a RAID 3. There are several factors that are undesirable about it, i would Recommend a raid 10 (if your primarily worried about speed, or a raid 5 if data integrity and speed are of concern.

among these reasons are it is very hard to find a raid controller that will easily support raid 3.

the enclosure that your link goes to as far as i can tell is not compatible with your controller card one of them (controller card) is sata, and the enclosure looks to be SAS (serial attached SCSI), which are not cross compatible....

My reccomended setup would be something along these lines for harddrive Hitachi deskstar 3TB 7200rpm they run 399 a pop but are 3tb at 7200rpm, i would get 4 to start with, that brings us to ~$1500

For the enclosure i would look along the lines of SOHOTANK ST8-U5

For a controller card its hard to beat ATTO ATTO ExpressSAS R680 for 889

our grand total is $3,132 I would also reccomend getting a 5th drive to keep on hand you can actually have it in the encloseure as a "hotspare" that means when one dies it will automatically start rebuilding to the new drive.

This config will give you 8.4TB of useable space, and be able to scale up if you ever need to add drives to the enclosure...

 

[Patrick O'Sullivan]

Thanks Mark and Rudy.

That is exactly the kind of info I am looking for. Talked to a guy at PC Pit Stop and he got me thinking that it may be best to go with an SAS expander so we can upgrade as we go. We pump out massive amounts of 5D stuff so I am just calculating on doing much the same with the RED. That turns out to be a whole lot of data pretty quick.

What does everyone else do for storage? How long do you guys keep your work easily accessible. I have seen Chase Jarvis talk about Cloud back ups for all his photo work but I'm trying to figure out what we are going to do. How long will a hard drive stay good without being worked?

How do they store? I seem to recall in other forums seeing that they don't last long without being spun. Is that the case?

Also how much of a difference will 600MB/sec - 900MB/sec average make when editing in CS5.5? I see Tom Lowe can get playback without a RAID on just a gamer PC with a pretty high end CUDA card.

I'm not looking for Hollywood levels. I just need it to work smoothly at a decent res level. Say 1/2 on a 4k timeline.

Thanks again guys

 

[Martin Widerberg]

I run two raid systems in raid0.
Second is as a back-up of the first.
I use CopyCloner to do the back-up.
Once I grow out of space I swap the discs out for bigger ones. Hitachi has announced 4TB drives
4TB x 5 slots = 20TB

 

[Ian Edwards]

Use a RAID 0 or a RAID 5 as a 'workspace.' Keep active projects on this drive. Make sure it's backed up (if RAID 0) daily. Treat it just like you treat your desk -- if something is done, sweep it away to an organized place where you can retrieve it should you need to.

When you're done, archive them off to tape or if you think you'll need them in the next 3-6 months, off to tape + single hard disks/another raid set. Just remember that hard drives are NOT archival tools. If you let the drives sit for too long, they will most likely fail. Tape is the most reliable method for archival.

Keeping everything on one big spinning disk set indefinitely is asking to lose data.

Doing this ensures you always have disk space available on a high speed array and ensures the safety of your work.

 

[emil tanem]

I'm running RAID5 on a rocket raid622.
Using premiere on PC
have a SSD for OS (obviously)
Have a dedicated HDD for cache (supposedly good for lower export times)
PC: Quad intel i7 2,66GHz, Geforce GTX470

This can playback red one mx 4,5K footage at 1/4 resolution. It can almost playback smooth at 1/2.

I just re-initialized the raid. I had to do wild guessing at block sizes, sector sizes and file allocation size throughout the process.
I don't really know anything about raid settings, so if anybody has anything to suggest to make it faster, it's appreciated.