DIT - Speed test results eSATA vs Firewire800 on Macbook pro

[Jonathan B.]

Hi there!

When unloading the memory card on set from our Red Epic, it's very important to be able to give a feedback on the video material as fast as possible. For a while, we moved around our Macpro and used eSATA connectivity. But that was quite some hassle in term of amount of gears to move. I wanted to find a solution based on a Macbook pro. I had a 15'' available (no Express card slot, and not eSATA). So I gave a shot to Sonnet's solution through a Echo Expresscard thunderbolt adapter, and a Tempo SATA Pro ExpressCard.

Now I have both Firewire 800 and eSATA available on the Macbook pro. I ran an easy speed test transferring a 4,29GB file of RED footage from the RED Mag over to : Internal SSD hard drive, a 4 discs RAID5 from RAIDON, and a Lacie Rugged drive. Here are the results switching between eSATA and Firewire.

RED -> FW800 -> internal
52.083s


RED -> eSATA -> internal
40.928s


RED -> FW -> eSATA -> RAID5
1m1.090s


RED -> eSATA -> eSATA -> RAID5
1m4.555s


Internal -> eSATA -> RAID5
55.685s


RED -> FW800 -> FW800 -> RAID5
2m44.691s


RED -> eSATA -> FW800 -> Lacie Rugged
56.636s

Not much surprise, we notice clearly that transferring in and out of Firewire gives a very significant speed drop.

To check the integrity of our files, we use the MD5 algorithm. I measured how that algorithm performed with FW and eSATA on the different discs. Again, this test is performed with the 4.29GB RED footage.

MD5 FW800 RED
53.470s


MD5 eSATA RED
23.950s


MD5 FW800 RAID5
1m2.472s


MD5 eSATA RAID5
22.855s


MD5 FW800 Lacie Rugged
57.615s

Interesting to notice that the MD5 check takes about as long as the copying with Firewire. But here, eSATA makes a real difference, it's more than twice as fast! On a copy+verify operation, that would be around a 30% increase in speed.

Last but not least, I checked how the whole system performed using Silverstack to copy 44GB of footage on two discs, and verify it. I used my RAID5, and a Lacie Quadra (so that I have eSATA available as well). With my system from Sonnet, I get only two eSATA ports. It's theoretically possible to use a port multiplier to get more ports, but I didn't have any available. So I used the RED Mag on Firewire (conscious of the slower MD5 processing time, which might take about 3 extra minutes on the whole process).

Test1:
Lacie quadra eSATA
RAID5 eSATA
Result: 22mn 35s


Test2:
Lacie quadra FW800
RAID5 FW800
Result: 54mn 23s

Conclusion
By using this setup, my DIT process is more than two times faster! That's a significant improvement, worth the investment. Eventually, we'll rack all the hard discs, adapters, card readers,... into one box with just a Firewire cable and a Thunderbolt cable to hook into the Macbook pro.

One important note, with Mac OS X 10.7, memory cards formatted by the Red camera are not recognized by the Mac when used through Sonnet's adapter with eSATA. Sonnet is aware of the issue, and currently working with Apple on a fix. So for now I'll stick to Firewire for using the Red Mag.

Do you have some similar experience, tips, insight, into speeding up the transfer with a Macbook Pro?

[Quentin Devillers]

Hi !
I'm new in DIT stuffs but I try to understand well everything.
Your test are really complete and interesting !
One thing I didn't get :
When you use silverstack the copy take 22mn instead of 23s using MD5 ? Is this the same process and amount of data ?
Thanks !

[Jonathan B.]

Hi Quentin,

when you transfer file, there's a chance that something goes wrong and your system might not notify you. Particularly if the media you're copying to is physically corrupted in some way. Footage are to precious to gamble, you need to make sure after the copy that everything has been copied properly. One way to do that, would be for a system to read and compare each bit between the original and the copy. An other way, maybe more commonly used is to use a tool like MD5. It's an algorithm which will generate a code out of each bit of your file. What you would want to do, is to generate such a code for the original, and each copy, and then compare everything. MD5 is designed so that if any bit in the file is not like the original, the code will also not match.

This would be a quite some hassle to process by hand for each file you copy. That's where software like Silverstack (among others) come in the picture. What they do is that they first copy each file (often to several discs, at least two for safety), and then compare each copy with the original, and eventually gives you a thumb up if everything went well. In addition you get a log of everything you have copied.

With the two first tests, I wanted to see how eSATA would impact each process separately, first the copy, then the verification. And that was for one file only. The last test is a full process where I unload AND verify one card with more data (I forgot to mention that, I'll correct it, it was 44GB for the last test).

I hope that'll be helpful. So in conclusion, if you have the chance, try to get a hold of a machine that has eSATA, or adapters to get it. I didn't mention it here, but you could also consider using Thunderbold for your discs, and Firewire 800 for the Red Mag. It would probably go about as fast but save you some hassle with the adapters. I guess I would do that if I didn't have a bunch of eSATA discs available from before.

Take care

[Stephen Townsend]

How would using an Echo Expresscard thunderbolt adapter straight in to a newer model MacPro with a thunderbolt connection compare I wonder?

[Jonathan B.]

I think we're quite close to what the RED Mag can deliver. The transfer RED->InternalSSD ran around 104MB/sec, 66MB/s with the RAID, the bottle neck in my case in the RAID, not the connectivity. With a Macpro instead of the Macbook pro, you might save some seconds on the processing of the MD5, but I don't think it'll make a big difference.

What might improve the performance is using a faster RAID (for instance with more discs), or even some SSD RAID solution. The ideal setup I think would be something like :
- RED Mag with Sonnet and eSATA to the Macbook pro
- 2 Thunderbolt RAIDS with SSD drives

That would give a significant speed increase as well as a significant cost increase ;-)

[joe hedge]

[I]One important note, with Mac OS X 10.7, memory cards formatted by the Red camera are not recognized by the Mac when used through Sonnet's adapter with eSATA.

Actually I've done 3 or 4 jobs recently, on a 17' MBP, plugging the Red Mag card reader into the Thunderbolt adapter/Express34 combo, works fine. A little bit faster than FW 800, but not as fast as eSata, because it's only PCIe 1.

[Dustin Cross]

Jonathan,

Your numbers seem pretty slow. 104MB/sec for Red SSD is pretty slow and 66MB/s for a RAID is really slow. Some people are getting closer to 200MB/s from Red SSD.

Another thing that confused me was the speed of our transfers with esata:
RED -> eSATA -> eSATA -> RAID5
1m4.555s

Internal -> eSATA -> RAID5
55.685s

They should be much faster than Firewire unless you are using really slow drives and/or bad esata cards.

I get speeds on drives from 120MB/s - 200MB/s for esata drives. Anything less than 100MB/s is too slow. Raids should be closer to 200MB/s or much faster. Anything less is not good.

If you are not seeing drastic speed increase with esata over firewire something is wrong.

Run AJA System Test on your drives and SSD to get a baseline and see how fast each device is firewire vs esata.
http://www.aja.com/products/software/


Dusty

[Gary CTY]

Dusty

What hardware you are using to transfer speed from 120MB/s - 200 MB/s? Mac Pro or Macbook Pro system?

Gary

[Shapour Daneshmand]

Thank you Jonathan for the effort.

[Björn Benckert]

We use a maxi digital 8 disc PC express / red rocket raid where we also inserted a esata card in the raid. This gives what I think is the fastest macbookpro solution. Or?