Welcome to our community

Be a part of something great, join today!

Register Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

DIT - Speed test results eSATA vs Firewire800 on Macbook pro

J

Jonathan B.

Member
Joined
Jan 25, 2012
Messages
9
Reaction score
0
Points
0
Hi there!

When unloading the memory card on set from our Red Epic, it's very important to be able to give a feedback on the video material as fast as possible. For a while, we moved around our Macpro and used eSATA connectivity. But that was quite some hassle in term of amount of gears to move. I wanted to find a solution based on a Macbook pro. I had a 15'' available (no Express card slot, and not eSATA). So I gave a shot to Sonnet's solution through a Echo Expresscard thunderbolt adapter, and a Tempo SATA Pro ExpressCard.

Now I have both Firewire 800 and eSATA available on the Macbook pro. I ran an easy speed test transferring a 4,29GB file of RED footage from the RED Mag over to : Internal SSD hard drive, a 4 discs RAID5 from RAIDON, and a Lacie Rugged drive. Here are the results switching between eSATA and Firewire.

RED -> FW800 -> internal
52.083s


RED -> eSATA -> internal
40.928s


RED -> FW -> eSATA -> RAID5
1m1.090s


RED -> eSATA -> eSATA -> RAID5
1m4.555s


Internal -> eSATA -> RAID5
55.685s


RED -> FW800 -> FW800 -> RAID5
2m44.691s


RED -> eSATA -> FW800 -> Lacie Rugged
56.636s
Click to expand...

Not much surprise, we notice clearly that transferring in and out of Firewire gives a very significant speed drop.

To check the integrity of our files, we use the MD5 algorithm. I measured how that algorithm performed with FW and eSATA on the different discs. Again, this test is performed with the 4.29GB RED footage.


MD5 FW800 RED
53.470s


MD5 eSATA RED
23.950s


MD5 FW800 RAID5
1m2.472s


MD5 eSATA RAID5
22.855s


MD5 FW800 Lacie Rugged
57.615s
Click to expand...

Interesting to notice that the MD5 check takes about as long as the copying with Firewire. But here, eSATA makes a real difference, it's more than twice as fast! On a copy+verify operation, that would be around a 30% increase in speed.

Last but not least, I checked how the whole system performed using Silverstack to copy 44GB of footage on two discs, and verify it. I used my RAID5, and a Lacie Quadra (so that I have eSATA available as well). With my system from Sonnet, I get only two eSATA ports. It's theoretically possible to use a port multiplier to get more ports, but I didn't have any available. So I used the RED Mag on Firewire (conscious of the slower MD5 processing time, which might take about 3 extra minutes on the whole process).

Test1:
Lacie quadra eSATA
RAID5 eSATA
Result: 22mn 35s


Test2:
Lacie quadra FW800
RAID5 FW800
Result: 54mn 23s
Click to expand...

Conclusion
By using this setup, my DIT process is more than two times faster! That's a significant improvement, worth the investment. Eventually, we'll rack all the hard discs, adapters, card readers,... into one box with just a Firewire cable and a Thunderbolt cable to hook into the Macbook pro.

One important note, with Mac OS X 10.7, memory cards formatted by the Red camera are not recognized by the Mac when used through Sonnet's adapter with eSATA. Sonnet is aware of the issue, and currently working with Apple on a fix. So for now I'll stick to Firewire for using the Red Mag.

Do you have some similar experience, tips, insight, into speeding up the transfer with a Macbook Pro?
 
Last edited:
Hi !
I'm new in DIT stuffs but I try to understand well everything.
Your test are really complete and interesting !
One thing I didn't get :
When you use silverstack the copy take 22mn instead of 23s using MD5 ? Is this the same process and amount of data ?
Thanks !
 
Hi Quentin,

when you transfer file, there's a chance that something goes wrong and your system might not notify you. Particularly if the media you're copying to is physically corrupted in some way. Footage are to precious to gamble, you need to make sure after the copy that everything has been copied properly. One way to do that, would be for a system to read and compare each bit between the original and the copy. An other way, maybe more commonly used is to use a tool like MD5. It's an algorithm which will generate a code out of each bit of your file. What you would want to do, is to generate such a code for the original, and each copy, and then compare everything. MD5 is designed so that if any bit in the file is not like the original, the code will also not match.

This would be a quite some hassle to process by hand for each file you copy. That's where software like Silverstack (among others) come in the picture. What they do is that they first copy each file (often to several discs, at least two for safety), and then compare each copy with the original, and eventually gives you a thumb up if everything went well. In addition you get a log of everything you have copied.

With the two first tests, I wanted to see how eSATA would impact each process separately, first the copy, then the verification. And that was for one file only. The last test is a full process where I unload AND verify one card with more data (I forgot to mention that, I'll correct it, it was 44GB for the last test).

I hope that'll be helpful. So in conclusion, if you have the chance, try to get a hold of a machine that has eSATA, or adapters to get it. I didn't mention it here, but you could also consider using Thunderbold for your discs, and Firewire 800 for the Red Mag. It would probably go about as fast but save you some hassle with the adapters. I guess I would do that if I didn't have a bunch of eSATA discs available from before.

Take care
 
I think we're quite close to what the RED Mag can deliver. The transfer RED->InternalSSD ran around 104MB/sec, 66MB/s with the RAID, the bottle neck in my case in the RAID, not the connectivity. With a Macpro instead of the Macbook pro, you might save some seconds on the processing of the MD5, but I don't think it'll make a big difference.

What might improve the performance is using a faster RAID (for instance with more discs), or even some SSD RAID solution. The ideal setup I think would be something like :
- RED Mag with Sonnet and eSATA to the Macbook pro
- 2 Thunderbolt RAIDS with SSD drives

That would give a significant speed increase as well as a significant cost increase ;-)
 
Jonathan B. said:
One important note, with Mac OS X 10.7, memory cards formatted by the Red camera are not recognized by the Mac when used through Sonnet's adapter with eSATA.
Click to expand...


Actually I've done 3 or 4 jobs recently, on a 17' MBP, plugging the Red Mag card reader into the Thunderbolt adapter/Express34 combo, works fine. A little bit faster than FW 800, but not as fast as eSata, because it's only PCIe 1.
 
Jonathan,

Your numbers seem pretty slow. 104MB/sec for Red SSD is pretty slow and 66MB/s for a RAID is really slow. Some people are getting closer to 200MB/s from Red SSD.

Another thing that confused me was the speed of our transfers with esata:
RED -> eSATA -> eSATA -> RAID5
1m4.555s

Internal -> eSATA -> RAID5
55.685s

They should be much faster than Firewire unless you are using really slow drives and/or bad esata cards.

I get speeds on drives from 120MB/s - 200MB/s for esata drives. Anything less than 100MB/s is too slow. Raids should be closer to 200MB/s or much faster. Anything less is not good.

If you are not seeing drastic speed increase with esata over firewire something is wrong.

Run AJA System Test on your drives and SSD to get a baseline and see how fast each device is firewire vs esata.
http://www.aja.com/products/software/


Dusty
 
Dusty

What hardware you are using to transfer speed from 120MB/s - 200 MB/s? Mac Pro or Macbook Pro system?

Gary
 
We use a maxi digital 8 disc PC express / red rocket raid where we also inserted a esata card in the raid. This gives what I think is the fastest macbookpro solution. Or?
 
Gary,

Just did a quick test of a raid to get benchmarks.

100MB/s Macbook Pro w/ Sonnet Tempo Sata
150MB/s Macbook Pro w/ Sonnet Tempo Sata Pro
250MB/s Mac Pro w/ Atto H380

Macbook Pros will transfer pretty fast with a good Expresscard Sata card. As you can see the Expresscard used has a pretty big speed difference and tower is needed to see the full potential of the drive. The Expresscard slot on a Macbook pro should be able to in theory push around 250MB/s.



Dusty
 
Jonathan B. said:
Hi there!

...a Macbook pro. I had a 15'' available (no Express card slot, and not eSATA).
Click to expand...



Jonathan. what's the model of your Macbook Pro? I have the same of yours but i'm not sure mine is compatible with Thunderbolt as a port for connecting hard drives, neither other stuff (like the Echo Express...) Do you know is there any difference between Macbook pros buyed in 2010 and the ones in 2011-2012 ?
Thanks.
 
hello,
i am transfering RED SSD via FW800 -> Macbook Pro -> two Esata Destinations simultaniously (with MD5 checksums) and getting the following results:
65GB in 41minutes - do you think another esata expresscard would get me faster results?
best wishes and kind regards,
martin
 
firewire and usb2 take wayyyy too long. esata is the only thing i can tolerate...and even that is slow. im waiting for the new mag reader with usb3 and thunderbolt, i can't wait until they eventually come to market. that and i cant wait for usb3 to be native on all computers without getting additional cards installed... i can't even imagine trying to use firewire to transfer cards, that must take forever man! here i am whining about esatas on set!
 
yes if your putting it on a single optical drive that is a bottle neck, but these days people use all kinds of stuff 36tb raids, 6g ssd's, stuff that can really utilize some serious bandwidth depending how its raided or if its an ssd. time is money so paying extra to use fast stuff can be essential depending on what your doing. right now im using graids set to raid 0 with 7200rpm drives on set, next week im gonna start using a 36tb raid array i am told, i don't know what its raided to but i am sure i can bottleneck my esata port with that thing if i raid it right! then i am using 6g ssd's to get the stuff to the post guys... with stuff this high end you really gotta try to keep things rolling smoothly and quickly if you can, i am talking fastest possible workflows here, cuz i dont wanna sit on set copying cards after a 10 hour day, know what i'm saying?
 
Last edited:
what if your copying to 6g ssd's? or raid 0 6g ssd's (like say an r6 promise raid with ssds is your storage drive)? perhaps i am mistaken but i'd like to think having a usb3 or thunderbolt port on a mag reader could hypothetically speed things up if i have some crazy ssd array. or do you mean the saturation is in the redmag itself?
 
Here is a benchmark done by Barefeats:
http://www.barefeats.com/tbolt01.html

See that SAS is much faster than Thunderbolt is a real world test. SAS is capable of 2400MB/s today and there are lots of solutions out today. Or a SAS port can be broken out to four 6G Sata ports. The 36TB raid you will be using will either be SAS or multiple Sata ports.

The main thing to look at is the raid 5 speeds. We only want to use raid 5 or 6 for professional work and the quality of the raid processor really effects performance. Raid 0 is a nightmare waiting to happen.

The 600MB/s you can get out of a Sata port is plenty fast, plus you can easily get 16 sata ports in one card. That is 9600MB/s of potential throughput. Almost 10 times faster than what Thunderbolt can do. Plus it is tested and available today. SAS will double its speed to 12G before Thunderbolt makes any speed increases.

Thunderbolt will be great for connecting devices like QIO, MXO, decklink, etc. Until Thunderbolt drives are multiport with Thunderbolt, Esata, Firewire, and USB they are useless. Right now they are only Thunderbolt and if a system doesn't have Thunderbolt you can't use the drive. So right now Thunderbolt drives only work on laptops and iMac, not any pro systems.

USB3 has had almost four years and really done nothing. I don't see anything special coming from it.



Dusty
 
Adrian Hernandez said:
Jonathan. what's the model of your Macbook Pro? I have the same of yours but i'm not sure mine is compatible with Thunderbolt as a port for connecting hard drives, neither other stuff (like the Echo Express...) Do you know is there any difference between Macbook pros buyed in 2010 and the ones in 2011-2012 ?
Thanks.
Click to expand...

You should check the tech specs of your Macbook Pro. I'm not sure when they introduced the Thunderbolt ports on the Macbook pros. Before that it was just a "Universal display port", it looks exactly the same, but it only works for screens.
 
You must log in or register to reply here.
Back
Top