Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

March 2013 Reccomendations for RAID ARRAY editing R3D footage in CS6

How do I make sure this isn't happening? When to Energy Saver and "Put hard disks to sleep when possible" is NOT checked.

Thought it seems that my SAS 15,000 RPM drives with the Apple Raid card in RAID 0, take a good 10 seconds to come to speed when I've been away or have put hte computer to sleep.

Jeff Kilgroe said:
If you run a RAID-0 in a Mac Pro, be sure that your system DOES NOT POWER DOWN HARD DRIVES WHEN INACTIVE!!!
Click to expand...
 
Will Keir said:
How do I make sure this isn't happening? When to Energy Saver and "Put hard disks to sleep when possible" is NOT checked.
Click to expand...

That's all you need to do there.

Thought it seems that my SAS 15,000 RPM drives with the Apple Raid card in RAID 0, take a good 10 seconds to come to speed when I've been away or have put hte computer to sleep.
Click to expand...

Apple RAID card has its own management. Last time I worked with the Apple RAID card, it didn't have any ability to control this, nor did it have the ability to schedule battery recondition or other things that affect performance. I have absolutely zero love for their RAID card. It's crap.

If you're running a RAID-0 on your Apple RAID card, you're doing nothing more than wasting a slot, IMO.
 
Yeah I know what you mean. When I purchased the card I got it because it was the only way to use SAS 15,000 RPM drives.... Knowledge is power.

So I've been hanging on to it and getting some use because it cost me 3-4k. I'm looking to dump it. I need to open a slot for the RR card.

Mac Pro towers (4,1) have four slots not five?

16x for GFX Card (double)
16X for RR Card
4x for CalDigit USB3/ESATA Card
4x for AJA Kona 3G

That's my plan. Either RAID 0 with the spinning media your recommended in 4x 3TB for Apps and System or 2x 1TB SSDs in RAID 0 for the same.

Now, can I add more SSDs to bay 3 and 4 when I can afford it. Without a complete reinstall?


Jeff Kilgroe said:
If you're running a RAID-0 on your Apple RAID card, you're doing nothing more than wasting a slot, IMO.
Click to expand...
 
Keep in mind that in the Mac Pro you're subject to the speed limit on the southbridge chip when doing soft RAID 0 with the internal bays so the max you'll ever see is about 600 MB/s. PCI-E Raid controllers do not suffer this limit.
 
Will,

All Mac Pros have 4 slots. It's 5 slot spaces because the bottom slot (slot #1) is double-width. Your card arrangement looks fine, you may want to swap the SATA card with the Kona card as that top slot's air flow is a bit more restricted. Although, I don't know if it matters either way, I would put the smaller of the two cards in slot-4. Kona 3G is sometimes easier in slot 3 for getting to cable connectors on the back of the system.

If you add a couple drives/SSDs as a RAID-0 and then add more, you have to rebuild the RAID, which wipes the data, if you wish to add them into the same stripe set. I don't recommend running the system/ apps off of the RAID. I do that off a single drive or SSD in the lower optical bay, keep the 4 bays just for the RAID-0 used for fast workspace storage. If it's smaller, 4TB or less, you'll be purging it all the time anyway because there isn't enough space for all your projects.

Before you invest in all the $$ for fast SSDs there, I would make sure you have a good backup setup in place. Whether you use SSDs or HDDs, you're going to want backups of all your data. FWIW, most of the times I have to go to backups and retrieve data, it's due to operator error or that I just need to pull something from an old project. Has nothing to do with hardware failures.

DJ Meyer said:
Keep in mind that in the Mac Pro you're subject to the speed limit on the southbridge chip when doing soft RAID 0 with the internal bays so the max you'll ever see is about 600 MB/s. PCI-E Raid controllers do not suffer this limit.
Click to expand...

This is true for the Mac Pro 3,1 (2008). In the Mac Pro 4,1 (2009), the limit is still there, but higher, and there's plenty of bandwidth to support all 4 drive bays at their maximum 3Gbps SATA bandwidth. Even with that, if using SSDs in the system, it's still going to top out at about 900MB/s of real-world, interface limited, speed. That limit also applies to the 2010-currnet Mac Pro 5,1. The bandwidth of the 2008 Mac Pro is increased when using the Apple RAID card. The 2009+ version of the Apple RAID card doesn't increase bandwidth as it is still based around 3Gbps interfaces, so we're still interface limited.

If using HDDs, a 4-drive RAID-0 isn't going to sustain better than 450MB/s for reads anyway, not across the entire RAID volume. I wouldn't be concerned about this limitation.

PCIe RAID controllers still suffer from the limitations of their interfaces and onboard RAID chipsets. In experimenting with SSD RAID setups, I've had to resort to using multiple RAID controllers to get the desired speed, even though I had not used up all the ports on the RAID controllers.

At any rate, he's not going to be exceeding any of these limits with that '09 Mac Pro. :)
 
Thanks Jeff.

I was thinking of making the internal RAID 0 as my OS and Apps drive. My media and projects, renders, etc... will go on a CalDigit HD One 16 TB RAID 5 array (14 TB in actual storage space).
If I use a single SSD for OS and Apps, I'm only getting 250ish MB/S.

Why use the single SSD in the optical bay instead of a much faster RAID 0 array with 4-9 GB of storage?

These days, my goal is to use fast enough external drives to do all my editing on them, leaving nothing on the system OS Drive. This makes backups smooth and simple. The OS is for apps, which I can easily time machine often to avoid all the reinstall. The external HD One is for all my media, projects and renders using CS6.

I use the lower optical bay for a SSD for bootcamp. Can I RAID 4x SSDs in the optical bay? Or at least two to double the speed?
 
Got my list or recommendations for SSDs for my Mac Pro (4,1)

SAMSUNG 840 Pro - $520
OCZ Vertex 4 - $570
OWC Mercury EXTREME Pro 6G - $560 (SandForce technology on this drive = avoid?)

Any drives I'm leaving out? Recommendations and feedback most welcome.

Looking to buy two for now, creating a RAID 0 for my OS & Apps. Storage is handled by my external CalDigit HDONE.
 
Last edited:
Just wondering which of these SSDs people recommend around here. Anyone?
 
Samsung 840 pro's are just about the best drives out there right now. The samsung 843 ssd is the enterprise version of the same drive.
The intels have a sandforce controller but the 520 series is bloody reliable too ! Another good drive with sandforce is the kingston hyper x series.
The plextor m5p xtreme series of drives is also very reliable and uses a marvel controller which has none of the compression issues of the sandforce controller.
Also check out the crucial m4 drives.. slightly older and slower but super reliable too..
Crucial just came out with their M500 series of drives that are priced ridiculously low... 960GB drives at under 600 $ but they are new and wsit till anvil or someone does a proper long term reliability test on it before i look at that drive.
AS for Raid controllers..
Have you looked at the areca 188xx series raid card for mac use ? its an awesome controller.. Areca also has a external 8bay box Areca Arc 4036 i think that will deliver over 2.5GB/sec with 8x ssd's. Want to use conventional drives and still get insane speed. just strap multiple units together :)

Found a very nice review here .. http://thessdreview.com/raid-enterp...sure-patriot-wildfire-raid-0-sf-2281-results/
 
For SSD's , The new Micron 1TB ones are very good.
They are rated 2 Peta Bytes write as far as wear out. Target price $600 each.
 
Very interesting guys. I'm also looking heavily into storage.
My first question:
- How fast do the drives really need to be?
Myself I edit all the time from ProRess Proxies so the speed there is not the bottleneck. Also without a rocket the cpu is always the bottleneck and not the drive speed. Of course I can imagine if you have a powerful Xeon or rocket then it would make more sense to edit from Red Raw or color grade in Red Raw. I could also imagine if you read from dpx files it will be needed but in every situation where I have heavy data I just create proxies.

What I'm having some problem with though is
- having the data accessible on multiple computers. Me and my lady work together one everything, editing, color grading, after effects, lightroom, sound.
- doing backups safely
- fairly portable so I can take at least some of the data with me on the trip.

After much thought the best option seem to be to have several external cases.
- 20TB in each. Preferably Thunderbolt and USB3. Probably I will avoid Raid since speed doesn't seem to be so important to me yet but the increased risk is a problem. I would maybe have Raid 0 one of them, but of course only if it's really needed.
- I have a feeling that the network will be to slow so usb3 or thunderbolt may be better, (at least with 1Gbit ethernet). I of course enjoy speed but I'm not sure the Risk of Raid 0 is really worth it. The 4TB Hitashi has a speed of about 150MB/sec which is pretty good.
- I'm considering having a syncing program that syncs both harddrives back and forward. Need to do more tests about this.
- an offsite backup harddrive that I move from site once in a while.
- currently I edit in an Apple Laptop and I really like Final Cut. So our 2 powerful 24" PCs are only used for effects and some grading currently. If Apple releases more powerful stationary computers we may get 2 of those.

Any thoughts are of course welcome.
/Andreas
 
On another note I can mention that the G Tech 8TB Raid drives actually has 330MB/sec read and 310 write according to Davincis own benchmark program. I would expect that to be it's maximum speed however. The G Tech has only 2 4TB drives inside them. One good point with 2 drives is that it reduces the risk of damage since it's not a 4 raid setup for example.

I'm considering using 2x 8TB thunderboolt drives to each laptop we have for editing and then sync them in the end of the day or at need. That way we should be able to sync event databases in Final Cut Pro and even Lightroom cataloges.

/Andreas
 
In my experience the minimum for an editing drive with R3D footage is 300MS/s. Shoot for higher with the Dragon footage.


You'll also appreciate the speed in an external RAID. USB3/eSATA/TB.


AndreasOberg said:
- How fast do the drives really need to be?
Click to expand...
 
Last edited:
Read through this whole thread and not understanding the need to work with 5k raw files...ProRes HQ transcodes are 1/3 of the size/bitrate of the raws, and 422 files are 1/4...meaning no need for fast storage, or dozens of terabytes of it...

If you don't have access to a RR card for transcoding, Davinci Resolve with a Quadro 4000 or GTX 680 will transcode at about half real time at half-res debayer, and those cards are only $750

Just seems like if you don't already have access to fast storage (like at a post house or editorial shop with networked SAN), then setting all that up just so one person can edit raw files is a waste. Just transcode.
 
Thanks for the reply Will.
From the tests I have done so far I wouldn't want to edit in raw even if it was fast. For example in Final Cut Pro X I really like that you can skim through the footage mega faster (when they are proxies). I often press the L or J key several times increasing playback. Can a faster computer really do that with RAW? If the skimming would be as fast then it would of course be better with raw. I'm not saying that there isn't a need, I just don't quite understand it yet :)

/Andreas
 
Umm the Micron M500 drives i would totally stay away from.. 72TB of Total Bytes Written is the rated endurance.!!
On the 960GB drive that is only 72-75 total erase/write cycles.. In a Raid array this could be devastating !
 
What does that mean exactly?

Shatul Durlabhji said:
Umm the Micron M500 drives i would totally stay away from.. 72TB of Total Bytes Written is the rated endurance.!!
On the 960GB drive that is only 72-75 total erase/write cycles.. In a Raid array this could be devastating !
Click to expand...
 
Incorrect.

Resolve does not use GPU to transcode R3D files. It uses CPU (words from a conversation with the guys over at BM). The bottleneck is still the debayer of R3D clips.

On a 2009 8-core Mac Pro tower, 48GB Ram with a new GTX 680, a 48 second clip shot at 5k HD takes 15 minutes 53 seconds to transcode to Avid DnXHD175X.

A Mac Book Pro can do it in 11 minutes 58 seconds.

Where are you getting the the results on a Quadro 4000 or GTX 680 at "half real time?"



joe hedge said:
If you don't have access to a RR card for transcoding, Davinci Resolve with a Quadro 4000 or GTX 680 will transcode at about half real time at half-res debayer, and those cards are only $750
Click to expand...
 
Last edited:
You must log in or register to reply here.
Back
Top