Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

redmag transfer time via esata

Philipe Ratton

Philipe Ratton

Well-known member
Joined
Jan 30, 2009
Messages
709
Reaction score
0
Points
0
Getting 17 minutes for a 64gb (actually 59gb) redmag transfer via esata to a laptop 7200rpm drive.
Is this about the same everyone is getting ?
 
The bottleneck is the hard drive. the SATA interface for the REDMAGs gives over 220MB/s consistently if you have a proper connection and fast enough media to offload your data to. Top speed on most 7200rpm laptop HDDs, even the "fast" ones, is usually no more than about 70MB/s. So offloading to such a drive via eSATA or FW800 is going to be the same speed, as the interface is not the bottleneck. And 17 minutes or so for a 64GB SSD pretty much fits this scenario.

Via ESATA to a fast disk array, I'm unloading [almost] full 256GB REDMAGs in about 18 minutes.
 
Jeff Kilgroe said:
The bottleneck is the hard drive. the SATA interface for the REDMAGs gives over 220MB/s consistently if you have a proper connection and fast enough media to offload your data to. Top speed on most 7200rpm laptop HDDs, even the "fast" ones, is usually no more than about 70MB/s. So offloading to such a drive via eSATA or FW800 is going to be the same speed, as the interface is not the bottleneck. And 17 minutes or so for a 64GB SSD pretty much fits this scenario.

Via ESATA to a fast disk array, I'm unloading [almost] full 256GB REDMAGs in about 18 minutes.
Click to expand...

Jeff have you had any experience with USB3 for transferring - I'm shooting a feature in a month or so and I'm being told they want to offload via a USB3 docking station and am not sure how the speeds would hold up against eSATA
 
im curious of that . too Jason
im putting together a card /ssd dumping and monitoring station together.
with esata stations and a late 2011 macbook pro 17 inch with 16 gig ram and maxx digital mobile red rocket enclosure going into the express 34 slot
and trying to figure out how to best use the single thunderbolt port i have .
that lacie hub has caught my attention , but i do not know what is best for me
 
Chris McKechnie said:
Jeff will know more, but you can get roughly 625mb/sec via USB 3.0, but I believe it peaks there, and that's writing to a very fast drive/raid.
Click to expand...

USB3 in "super speed" mode is 4.8Gbps -- some advertise 5Gbps, but that's stretching the truth and it seems most manufacturers are pulling away from that after criticism from end users. Your 625MB/s number is what you get when you take 5 billion bits and convert to decimal megabytes with no accounting for the actual interface speed and byte packing overhead. :(

Unfortunately, that's about double what you should expect from USB3 under ideal real-world circumstances. Absolute peak bandwidth of 4.8Gbps translates to 527MB/s, or if you're old-school and like your MB's to have 1024KB each, that makes it 514MB/s. Those are absolute peak rates taking the byte structure into account and the 4.8Gbps. With my better USB3 disk systems or single enclosures with SSD's, I'm getting 180~300MB/s writes and about 220~400MB/s reads, depending on which devices are in use.

That said, those rates are faster than the SATA interface (3Gbps) used in the current REDMAG SSD's and RED STATION modules. RED USB3 SSD reader is giving me about 180~190MB/s reads. I've been borrowing one since getting my retina MBP, but I need to order one or two of my own... So far have only connected it to the retina Macbook Pro and an HP Z820 workstation. The SATA reader is giving me about 235MB/s average and peaking a bit higher at times. I have a CalDigit 6GU3 (USB3 and 6Gbps eSATA host) in a Sonnet Thunderbolt expander on an iMac. I'll do some tests with that setup later today or tomorrow.

One thing you'll find with USB3 is that it's kinda flakey. Not all USB3 host chips and devices match up as we would like them to. Sometimes the connection is slower than we want, actually that is the case most of the time. USB3 super speed mode operates as an additional layer over the top of a USB2.0 interface and it comes at a heavy price with high latency and high overhead. it's a terribly inefficient transport that many in the industry were hoping would die. Primary example being Intel... The creators of USB and primary funding source for USB2.0 initially refused to support USB3.0 because they felt it was crap. It has progressed and is more mature now, most devices at least work when connected these days, a huge improvement over just a year ago. Intel has incorporated support into their new platforms since the industry has effectively demanded it, but when you get down to analyzing its technical merits, it's a crap format.

Anyway, not here to rip on USB3.0. It just is what it is and everyone is using it. Can't complain about it being all bad because other than Thunderbolt, it's still the best thing out there for what it does. 10G ethernet would make a far superior interface topology and it should be cheap and commonplace by now. Unfortunately a few companies that hold the patents and licensing on 10Ge have a stranglehold on the industry and are keeping the technology out of reach and prices high. ...Which has in turn led to the development of Thunderbolt / LightPeak and Intel's acquisition of Infiniband.

Theoretically, the USB3 RED reader should be as fast as the eSATA one. In practice, it seems to be about 15~20% slower.
 
Are there any large capacity (~1TB+) bus powered external drives (thunderbolt or USB3) that can actually read at ~180mb/s? I'm thinking the only way to sustain those write speeds would be either SSDs (which, in my opinion is not cost effective for 512GB, and not large enough for a backup) or RAID (which you probably couldn't find bus powered.)

Not sure what would be more cost effective (or convenient); setting up a portable/speedy DIT-lite (mainly just offloading) or getting 6 hours worth of REDmags (2x512s?) and offload at end of day. Something tells me it'd be way faster in production to just offload all at once at EOD, but that most producers/directors would shit down your throat... Plus if one of the REDmags accidentally has a date with a magnet before you get a chance to offload you'd be boned.

Even then, it still takes too long; 256GB in ~24mins at ~175mb/s... twice that with checksum (and twice that again if you duplicate at the DIT). That's not cool. Does anyone know if you can record footage to both the DSMC Rear SSD and Side SSD module simultaneously?
 
esata to esata

esata to esata

hi guys
using 2ports esata xpress card on macbookpro i can offload connected via esata
64 redmag to a lacie little disk (2x500gb - 5400rpm) in raid in 7/8 minutes
and 15 minutes to the same drive connected to fw800 and redmag via esata
cheers
 
Ok, I'm really pissed now.
I'm getting 100MB/s via esata to my internal 7200rpm laptop drive and 40MB/s via esata to a external USB3 drive.
WTF am I doing wrong here ?
 
Philipe Ratton said:
Ok, I'm really pissed now.
I'm getting 100MB/s via esata to my internal 7200rpm laptop drive
Click to expand...

That sounds about right... In fact, that's pretty good. The internal drive is the bottleneck.

and 40MB/s via esata to a external USB3 drive.
WTF am I doing wrong here ?
Click to expand...

Which USB3 drive? What system are you using? Any other devices connected? 40MB/s suggests that USB3 SuperSpeed mode isn't being engaged. Or the cabling / some component is junk.
 
Through my Thunderbolt Lacie eSata hub, I'm getting 220MB/s dumping my 64GB REDMAG to a 8tb G-raid.

I'm content. :biggrin:
 
Hey Jeff,
it's a hitachi touro 1TB, no devices other than a mouse and I'm running windows 7 on a MSI laptop.
How do I check to see if the superspeed thing is engaged ?
Thanks Jeff, kind and helpful as usual.
 
Philipe,

If your mouse is in a USB3 port, move it. It may be on the same header and that will keep your Touro drive from engaging into SuperSpeed mode. Otherwise, it could just be the drive and how it performs. I've never used one of those Hitachi Touro units, but pulling up reviews seems to indicate it tops out around 65MB/s or so at peak rates, which is really sad, but for large files and sustained transfers, the reviews are less than stellar. A lot of people think it's fast and they're amazed at the 45-60MB/s speeds they're getting, so it makes the good reviews a bit hard to separate from the bad and you have to look for the numbers. :(

Hopefully there's a way to squeeze a bit more speed out of it, but when it comes down to single-HDDs in a box, the best write speeds you're going to see will hover around 75 to 100 MB/s. If you want faster, you need a SSD or a RAID and beware of the SSD options... Other than the latest generation of 6Gbps models, most of them don't have write times any better, or little better, than what the better hard drives can do.

I'm trying to think off the top of my head if there's a utility that can show what link mode a USB3 port is in.... Not thinking of one. Google isn't helping me either, it seems the most common suggestion out there for this question is to just copy a bunch of data and see if it moves faster than it should via USB2.0. Not real helpful.


In regards to my previous note about the CalDigit 6GU3 card in a Sonnet ThunderBolt expander... Here's what I get unloading about 200GB from a 256GB REDMAG.

eSATA offload to Pegasus R6 ThunderBolt RAID - ~220MB/s

USB3.0 offload to Pegasus R6 - ~165MB/s


The Sonnet Echo Express is on one TB port of the iMac, the Pegasus R6 12TB RAID is on the other, didn't test them daisy-chained.
 
Jeff, thanks a lot for your answer, I think I will just have to live with those speeds for now.
If by any chance you find the utility you talked about, please let me know.
Thanks a lot brother, gotta love reduser.
 
You must log in or register to reply here.
Back
Top