Pegasus3 R8 setup tips / expected performance

[Robino_J]

I should be getting my Pegasus3 R8 this week and reading reviews online I see mixed results. Some report fast read results 1000mbps+ and others around 650mbps.

I'll be using the unit with my new iMac Pro 10Cores, 64GB, Vega64 - and will work with 6k and 5K R3Ds at low compression 2:1 to 5:1 and also 4K DPXs (51mb per frame).

I'm not expecting realtime 24fps playback from the DPXs but enough speed to work comfortably with R3D at better than 1/4th.

I never worked with a drive array like this before, any tips from Pegasus owners on how to get the best performance with my setup? (should I keep Raid 5, what strip size do you recommend etc.)

Thanks !

 

[Michael Tiemann]

You will find that R3D decode, not the drive, is your bottleneck. RR-X solves thst problem, but 10.13.2 update and 10.13.3 both break it. Expecting fix soon from RED.

 

[Jeff Kilgroe]

Speed of your Pegasus3 will depend a lot on which HDDs are installed and your RAID configuration. The 8-bay unit should be able to get you >1600MB/s read and write under most conditions. I run RAID-6 on my Pegasus units. It’s a tad more robust and does impact speed just a bit, but I’ve had better luck with it.

The drive is not going to be a bottleneck for your R3Ds. In fact, you will be able to play the 4K DPX stream off of it just fine too as that’s about 1250MB/s and quite possibly write them to disk that fast as well.

RR-X can speed things up a lot, but it can also be a hindrance. It does not support several processing features that have come along since its release and it’s already obsolete in the face of multiple GPUs and some of the larger dual Xeon configurations out there. The iMac Pro sits in a spot where the RR-X will probably still be useful for a lot of the decode processing/ playback in editing and such, but you should not feel that it’s necessary. I don’t know if you’ll be using a RR-X or not, that’s just my current thoughts on it, since Michael brought it up. And yes, seems there are some compatibility issues at the moment.

Back to the RAID config. A 128KB stripe size might work well for you. It really does not matter all that much with the current hardware. In years past, we could see a noticeable speed boost for large file I/O by using a larger stripe size. With the current buffering on the drives and performance of the host controllers it seems to interleave and push the data about the same either way. Larger stripe sizes can, in theory, make indexing and access/ reads of some types of data a lot faster, but you also end up reading and caching larger chunks. I would advise against going to 256KB or larger unless you *only* store large files like DPX and R3D, etc... As you can see a detrimental effect of 256KB+ when you’re accessing thousands of config files, small PDFs and images.

I’ve seen some reports of slow performance on some of these RAIDs. Of the ones I’ve helped assist with, it seems they’re all too often chaining too many TB devices together or intermixing TB generations on the device chain. Or they have other issues with their RAID config or even the RAID operating int a failed state where performance is cut way down. Generally speaking, these RAIDs are pretty fast. Or as fast as you’re going to get from a handful of 3.5” HDDs.

 

[Robino_J]

Thanks Jeff! Really appreciate you explaining all this.

I won’t be using a RRX, not sure what drives they put in there I ordered a built to order drive directly from Promise.

I’m planning on using Raid 6 as well for more security - The drive is set for delivery tomorrow, I’ll report back here.

 

[Marc Wielage]

I'm using 8 x WD Red 8TB drives (with a 256GB cache) in a RAID5, and I'm getting about 1100 MB/s read:

This was about a 10% increase over the previous 3TB drives that were originally installed in this RAID (a G-Tech). I was actually expecting it to be slower since the drives were almost 3 times bigger, but the opposite happened... which is a good thing.

 

[Robino_J]

I'm using 8 x WD Red 8TB drives (with a 256GB cache) in a RAID5, and I'm getting about 1100 MB/s read:

This was about a 10% increase over the previous 3TB drives that were originally installed in this RAID (a G-Tech). I was actually expecting it to be slower since the drives were almost 3 times bigger, but the opposite happened... which is a good thing.

That's nice performance - Is your enclosure using Thunderbolt 3 or another interface?

 

[Jeff Kilgroe]

FWIW, I get about 1300MB/s read and 1070MB/s write on one of my Pegasus2 (previous Thunderbolt 2 models) with 8 x HDDs inside. I would have to look, but I think it has Toshiba 3TB HDDs in them. I don’t recall if that one is set up as RAID6 or I left it as RAID5. My other Pegasus2 R8 is definitely set up as RAID6 and also has 3TB HDDs in it (would have to look ti see which ones, HGST I think). It runs noticeably slower, similar to Marc’s numbers. The drives have a lot to do with it. Also have an Areca 8-bay unit with 4TB WD Black RAID Edition drives inside and that unit screams. The TB2 interface is the bottleneck for reads.

Have an iMac Pro coming in next week it seems and I plan to pick up a Pegusus3 or Areca TB3 RAID to go with it. I kinda like the QNAP unit with both TB3 and 10GE that has 8 x 2.5” bays. But it’s expensive as I can get one of the Promise or Areca units with 8 x 4TB HDDs for the same price as that QNAP with no drives.

 

[Robino_J]

Ok got the unit and setup a Raid 6 logical drive.

Running a Blackmagic disk speed test while the drives are synchronizing I get 1500mb/s write and 1100mb/s read

I assume it's going to be a little faster when synchronization is completed, but if it stays like that I'm happy, never had that speed before that's pretty amazing.

Synchronization is super slow I think it's going to take long time to finish the process.

 

[Kyle Jones]

FWIW, I get about 1300MB/s read and 1070MB/s write on one of my Pegasus2 (previous Thunderbolt 2 models) with 8 x HDDs inside. I would have to look, but I think it has Toshiba 3TB HDDs in them. I don’t recall if that one is set up as RAID6 or I left it as RAID5. My other Pegasus2 R8 is definitely set up as RAID6 and also has 3TB HDDs in it (would have to look ti see which ones, HGST I think). It runs noticeably slower, similar to Marc’s numbers. The drives have a lot to do with it. Also have an Areca 8-bay unit with 4TB WD Black RAID Edition drives inside and that unit screams. The TB2 interface is the bottleneck for reads.

Have an iMac Pro coming in next week it seems and I plan to pick up a Pegusus3 or Areca TB3 RAID to go with it. I kinda like the QNAP unit with both TB3 and 10GE that has 8 x 2.5” bays. But it’s expensive as I can get one of the Promise or Areca units with 8 x 4TB HDDs for the same price as that QNAP with no drives.

My Pegasus2 R6 in RAID 5 gets about 800 MB/s write and 815 MB/s read. Is that what you would expect for the R6?

 

[Robino_J]

Ok after sync (had to do it twice because I changed my mind on the logical drives organization) I get 1557mb/s write and around 1200mb/s read

Happy with that but wondering why Read is that much slower.

 

[Robino_J]

Had some time to test.

Jeff you were 100% correct - In Resolve it's crunching through 4K 10bit DPXs (4096x3112) like butter but R3Ds at "Full Res" are playing at 17fps with or without a bunch of effects / nodes.

The strange thing is that playback in "Half Res Premium" and "Half Res Good" yield the same exact result as "Full Res" with 17fps (with or without a bunch of effects / nodes.)

Going to 1/4th Resolution fixes that and I'm ok with that but seems like a software issue..

Very impressed by this combo of Pegasus3 R8 + iMac Pro. Can't believe I'm playing back my DPXs like this - I'm used to less than 1fps with 4k DPXs

 

[Solomon Nero]

Speed of your Pegasus3 will depend a lot on which HDDs are installed and your RAID configuration. The 8-bay unit should be able to get you >1600MB/s read and write under most conditions. I run RAID-6 on my Pegasus units. It’s a tad more robust and does impact speed just a bit, but I’ve had better luck with it.

The drive is not going to be a bottleneck for your R3Ds. In fact, you will be able to play the 4K DPX stream off of it just fine too as that’s about 1250MB/s and quite possibly write them to disk that fast as well.

RR-X can speed things up a lot, but it can also be a hindrance. It does not support several processing features that have come along since its release and it’s already obsolete in the face of multiple GPUs and some of the larger dual Xeon configurations out there. The iMac Pro sits in a spot where the RR-X will probably still be useful for a lot of the decode processing/ playback in editing and such, but you should not feel that it’s necessary. I don’t know if you’ll be using a RR-X or not, that’s just my current thoughts on it, since Michael brought it up. And yes, seems there are some compatibility issues at the moment.

Back to the RAID config. A 128KB stripe size might work well for you. It really does not matter all that much with the current hardware. In years past, we could see a noticeable speed boost for large file I/O by using a larger stripe size. With the current buffering on the drives and performance of the host controllers it seems to interleave and push the data about the same either way. Larger stripe sizes can, in theory, make indexing and access/ reads of some types of data a lot faster, but you also end up reading and caching larger chunks. I would advise against going to 256KB or larger unless you *only* store large files like DPX and R3D, etc... As you can see a detrimental effect of 256KB+ when you’re accessing thousands of config files, small PDFs and images.

I’ve seen some reports of slow performance on some of these RAIDs. Of the ones I’ve helped assist with, it seems they’re all too often chaining too many TB devices together or intermixing TB generations on the device chain. Or they have other issues with their RAID config or even the RAID operating int a failed state where performance is cut way down. Generally speaking, these RAIDs are pretty fast. Or as fast as you’re going to get from a handful of 3.5” HDDs.

Hey Jeff!
Do you have any experience with the newer pegasus32 r8 units? I am seriously considering it, and I dont know anyone personally who uses it. Or is there something better around a similar price point that you would recommend?

 

[Jeff Kilgroe]

Have not bought any Pegasus units recently. I do own one Pegasus3 R8 that I bought when they first came out a few years ago. Speeds were comparable to the Pegasus2 because HDDs only go so fast. I have 4TB HDDs in it -- I think it shipped with Toshiba HDDs and that's what it still runs. The last Thunderbolt RAID purchase I've made was a 12-bay Areca. I like it and have 6TB IronWolf drives in it. These RAIDs, especially when paired with newer HDDs with large caches like to favor write times in the benchmarks. IIRC, I'm running RAID-60 on that 12-bay unit and I think it gives me 1600~1700MB/s all around.

@Robino_J -- the R3D performance seems a bit slow, but I've encountered that before. It's not an issue with your storage speed, it's software all the way and CPU limitations. What iMac configuration do you have?.

 

[Robino_J]

Have not bought any Pegasus units recently. I do own one Pegasus3 R8 that I bought when they first came out a few years ago. Speeds were comparable to the Pegasus2 because HDDs only go so fast. I have 4TB HDDs in it -- I think it shipped with Toshiba HDDs and that's what it still runs. The last Thunderbolt RAID purchase I've made was a 12-bay Areca. I like it and have 6TB IronWolf drives in it. These RAIDs, especially when paired with newer HDDs with large caches like to favor write times in the benchmarks. IIRC, I'm running RAID-60 on that 12-bay unit and I think it gives me 1600~1700MB/s all around.

@Robino_J -- the R3D performance seems a bit slow, but I've encountered that before. It's not an issue with your storage speed, it's software all the way and CPU limitations. What iMac configuration do you have?.

Oh that's an old post, I'm now on the new mac pro (2019) - when i wrote the post I was on iMac Pro 10Cores 64GB.

R3D playback on my new mac pro is ok, I expected more. (16cores,96gb, Pro Vega II) - Waiting patiently for BMD to add the new SDK to Resolve and see how much it improves.

 

[Jeff Kilgroe]

Yeah, I suppose it is old... I thought it was just a couple weeks and didn't notice it was more like 2 years. Ooops... Probably because this thread got resurrected from the dead.

Anyway, also want to add that Pegasus2 RAIDs are NOT playing nicely with macOS Catalina 10.15. Not sure about Pegasus3 RAIDs. I hear they work better with Catalina, but still have some issues with disconnects or the Promise utility causing spontaneous crash/reboots. I'm sure Promise will get it sorted, but not sure what's taking so long. Uninstalling the Promise Utility software seems to have alleviated most of the issues with the Pegasus2 on my iMac Pro for my Catalina boot partition.

The Areca RAIDs are not having such issues.

 

[Robino_J]

hey no worries - thx for your reply - I experienced many random reboots on Catalina, good to know it's the pegasus. Haven't got any reboots since updating to last version 10.15.3