How long does it take to empty a 64gb red ssd card?

[Brian Iannone]

You Guys remember, that when you copy on a single dasychained FW800 Bus from RED SSD (a) via fw800 to external disk (b) you pipe the whole data through the controller in your MacBook (c) so
doubling the traffic ;-)
Thus halving the realworld speed.. and suddenly 27min makes sense...

40 MB/s would make sense if you were daisy-chaining the RED Station to an external drive to a MBP via FW800. However, the way I understood Joshua's post was that he stated FireWire 800 is the bottleneck. I wasn't aware that he (or you) were going to be daisy-chaining devices.

When you start daisy-chaining over FW800, you can just completely forget about speed/performance because at that point, it doesn't exist. Expect 20-30 MB/s when two devices are connected and simultaneously transferring data.

 

[Brian Iannone]

I believe you, I just want to know the specifics.

The specifics of what exactly? Now I'm not sure about what you meant in your previous post...

If anyone is copying 64 gigs of data from one place to another over FW800 and doing it in (significantly) less than 27 minutes, I would very much like to know how.

Are you referring to a daisy-chain scenario, or do you just not think that a single FireWire cable can transfer 64GB in 27 minutes?

Which of these situations are you referring to?
1. RED Station ---- FW800 ---- External Drive ---- FW800* ---- Computer
2. RED Station ---- FW800* ---- Computer ---- Any Interface (eSATA, Thunderbolt, etc.) ---- External Drive
*Bottleneck

Or something else?

I was under the impression that you didn't think the FireWire 800 interface could support speeds of more than 40 MB/s.

 

[Daniel P Rudin]

I wasn't aware that he (or you) were going to be daisy-chaining devices.

That was my impression after i read 27min.. lets see what Joshua writes..
i personally use eSata, so no problem here, but i can see where peoples problems coming from if they have only USB and 1 x FW800 available at their Macbooks on the road..

 

[Kevin Marshall]

Ah okay, I see.

How long does it take you to offload a full 256 GB REDMAG with your current system?

Not entirely sure. I just took over data management on this shoot the other day, and we haven't filled a whole 256GB since then - we usually try and get a good rotation going with the cards to keep things moving.

 

[joshua csehak]

Sorry for my terse response. What I meant was, yes, I don't believe FW800 can, in a real life scenario, copy 64 gigs of data in much less than 27 minutes. My reasoning is that if I take a 7200 rpm external drive and plug it in, via a single FW800 cable, directly into my Mac Pro, that's how long it takes. When it takes half that time to copy the same data over sata, logic tells me that FW800 is the bottleneck. If I'm wrong, I would be very grateful to be set straight. It would mean faster copying times for me!

So yes, I'm skeptical. But show me some numbers and tell me exactly what you're plugging in to where and how, and time it with a stopwatch (not gb/second math), and I'll be a convert.

 

[Brian Iannone]

Sorry for my terse response. What I meant was, yes, I don't believe FW800 can, in a real life scenario, copy 64 gigs of data in much less than 27 minutes. My reasoning is that if I take a 7200 rpm external drive and plug it in, via a single FW800 cable, directly into my Mac Pro, that's how long it takes. When it takes half that time to copy the same data over sata, logic tells me that FW800 is the bottleneck. If I'm wrong, I would be very grateful to be set straight. It would mean faster copying times for me!

So yes, I'm skeptical. But show me some numbers and tell me exactly what you're plugging in to where and how, and time it with a stopwatch (not gb/second math), and I'll be a convert.

Will do!

 

[Jeff Kilgroe]

Brian, I applaud your helpfulness, but theoretical speeds are only useful for people in hard drive marketing departments. Real-world times are what people need to see here. It takes me 14 minutes to copy a folder full of 64 gigs of footage from one 7200 internal drive to another. If you're going from a Red -- or any -- SSD over eSata to a 7200 drive, I can't imagine that number being much different -- the bottlenecks are sata and drive speed. It takes me 27 minutes to transfer the same folder to an external 7200 drive via FW800, which fits with Michael's estimation of 2 gigs a minute. FW800 is the bottleneck here. If anyone is copying 64 gigs of data from one place to another over FW800 and doing it in (significantly) less than 27 minutes, I would very much like to know how.

You have a serious bottleneck somewhere. Most of the numbers we have been throwing about are "real-world" speeds, provided your drives are fast enough. In terms of FW800, it tops out at 76MB/s. That is it's real-world limit. Which also happens to be the theoretical limit... As it's slower than most decent multi-drive external storage systems or SSD-based units these days. 64GB across FW800 takes about 14 minutes. If it's taking you 27 minutes to move 64GB between internal drives within the same system, you have an issue with that system that needs real attention soon. Without more info, I can only speculate as to what that issue might be.

I would be happy to shoot a demo video of real-world transfer performance if anyone wants to see something specific. But I can assure you that most of the numbers I'm throwing around are what I'm getting. My storage that I'm offloading to is not the bottleneck in my setups -- it's the interfaces themselves. I make sure of that where I can. Nothing more frustrating than having a 286MB/s SATA interface and only having a 65MB/s single external HDD connected to it. That would drive me mad...

 

[joshua csehak]

You have a serious bottleneck somewhere. Most of the numbers we have been throwing about are "real-world" speeds, provided your drives are fast enough. In terms of FW800, it tops out at 76MB/s. That is it's real-world limit. Which also happens to be the theoretical limit... As it's slower than most decent multi-drive external storage systems or SSD-based units these days. 64GB across FW800 takes about 14 minutes. If it's taking you 27 minutes to move 64GB between internal drives within the same system, you have an issue with that system that needs real attention soon. Without more info, I can only speculate as to what that issue might be.

I would be happy to shoot a demo video of real-world transfer performance if anyone wants to see something specific. But I can assure you that most of the numbers I'm throwing around are what I'm getting. My storage that I'm offloading to is not the bottleneck in my setups -- it's the interfaces themselves. I make sure of that where I can. Nothing more frustrating than having a 286MB/s SATA interface and only having a 65MB/s single external HDD connected to it. That would drive me mad...

Thanks Jeff, it would be awesome to know how I can make my system faster. It takes me about 14 minutes to copy 64 gigs of footage from one WD Black 2TB drive in an internal bay in my Mac Pro to another WD Black 2TB drive in an internal bay in my Mac Pro, and 27 minutes to copy the same data from one of the former drives over FW800 to an external 7200 rpm drive. What devices are you using? How long does it take you? I definitely don't need anyone to shoot a video for me, just say, "I have X plugged into X via X, and it took me X to copy X gigs of footage."

 

[Brian Iannone]

Thanks Jeff, it would be awesome to know how I can make my system faster. It takes me about 14 minutes to copy 64 gigs of footage from one WD Black 2TB drive in an internal bay in my Mac Pro to another WD Black 2TB drive in an internal bay in my Mac Pro, and 27 minutes to copy the same data from one of the former drives over FW800 to an external 7200 rpm drive. What devices are you using? How long does it take you? I definitely don't need anyone to shoot a video for me, just say, "I have X plugged into X via X, and it took me X to copy X gigs of footage."

Jeff is absolutely right; you do have a serious bottleneck somewhere...

In my opinion, the speeds that your system is transferring data at from your one of your WD Black 2TB disks to the other and from one of those disks to an external FW800 drive are both anomalous.

Regarding your two internal disks, I know you said earlier that you don't like to hear numbers, only times in minutes, but you say that it takes 14 minutes to transfer 64GB from one 2TB disk to another. That's a rate of 76 MB/s (which coincidentally, is the max speed of FW800). While a single drive could possibly transfer at that speed, it seems a little slow... What model Mac Pro are you using (i.e. MacPro4,1)? How many individual files are in this 64GB of data that your transferring? Also, how are you timing this?

As for your external FW800 drive, what drive is it? (manufacturer, model, etc.)

One interesting test you can do is to download AJA's System Test utility and run a speed test on those two internal WD Black 2TB disks and your external FW800 drive. Just for the sake of continuity, set the settings to this:

Test: Disk Read/Write
Video Frame Size: 1920x1080 10-bit
File Size: 16.0 GB
Make sure that "Disable file system cache" is checked.

Just like this...

 

[joshua csehak]

Thanks for the link Brian. I tried the software, and it's giving me 88 MB/s for my two 2TB WD Black drives, and 96-102 MB/s for my two 1TB WD Blacks. Which, if I do the math, is about 68 gigs in 14 minutes for the slower ones, so that sounds about right. I also copied some footage from a (labeled as 60 MB/s) CF card via a SanDisk FW800 reader, this time through the FW800 ports in the back, and same thing -- just under 2 gigs a minute (48 seconds). AJA System Test tells me it's reading from the CF card at 38 MB/s, and I get the same thing when I test my external drive over FW800, which is one of these: http://eshop.macsales.com/item/Other World Computing/MS8U7320GB16/

One thing that might be going on is that my drives are all about 9/10 full. I know that makes them slower, but I can't imagine it'd be by that much. And my external is only 5% full, so there's no excuse there. I'd really like to know how you guys are getting these blazing fast speeds!

 

[Brian Iannone]

Thanks for the link Brian. I tried the software, and it's giving me 88 MB/s for my two 2TB WD Black drives, and 96-102 MB/s for my two 1TB WD Blacks. Which, if I do the math, is about 68 gigs in 14 minutes for the slower ones, so that sounds about right. I also copied some footage from a (labeled as 60 MB/s) CF card via a SanDisk FW800 reader, this time through the FW800 ports in the back, and same thing -- just under 2 gigs a minute (48 seconds). AJA System Test tells me it's reading from the CF card at 38 MB/s, and I get the same thing when I test my external drive over FW800, which is one of these: http://eshop.macsales.com/item/Other World Computing/MS8U7320GB16/

One thing that might be going on is that my drives are all about 9/10 full. I know that makes them slower, but I can't imagine it'd be by that much. And my external is only 5% full, so there's no excuse there. I'd really like to know how you guys are getting these blazing fast speeds!

Okay, so... Just to be sure, is this correct?

Mac Pro Internal:
- Two 2TB WD Blacks (88 MB/s - Estimate: 12 minutes to transfer 64GB)
- Two 1TB WD Blacks (96-102 MB/s - Estimate: 11 minutes to transfer 64GB)
All 90% full

External FW800 Drive:
- OWC On-The-Go Pro (Your speed: 38 MB/s, OWC benchmarks: 73-80 MB/s)
5% full

SanDisk FW800 CF Card Reader:
- CF Card (Your speed: 38 MB/s, Should be: 60 MB/s)

 

[joshua csehak]

Okay, so... Just to be sure, is this correct?

Mac Pro Internal:
- Two 2TB WD Blacks (88 MB/s - Estimate: 12 minutes to transfer 64GB)
- Two 1TB WD Blacks (96-102 MB/s - Estimate: 11 minutes to transfer 64GB)
All 90% full

External FW800 Drive:
- OWC On-The-Go Pro (Your speed: 38 MB/s, OWC benchmarks: 73-80 MB/s)
5% full

SanDisk FW800 CF Card Reader:
- CF Card (Your speed: 38 MB/s, Should be: 60 MB/s)

Yup. Except it took 14 minutes to transfer from one 2TB black to the other, timed with a stopwatch. I'd hazard a guess of 12 minutes for the 1TB ones. And the CF cards are, while not Red, still pretty good -- SanDisk Extreme UDMA 60 MB/s.

What the for crying out loud? OWC is benchmarking 75 MB/s? What the freak is wrong with my computer?

 

[Brian Iannone]

Yup. Except it took 14 minutes to transfer from one 2TB black to the other, timed with a stopwatch. I'd hazard a guess of 12 minutes for the 1TB ones. And the CF cards are, while not Red, still pretty good -- SanDisk Extreme UDMA 60 MB/s.

What the for crying out loud? OWC is benchmarking 75 MB/s? What the freak is wrong with my computer?

Based on those speeds you got (88 MB/s and 96-102 MB/s), 14 minutes for the 2TB and 12 minutes for the 1TB are probably accurate guesses. Those estimates I posted are based on continuous rates, which as we all know, don't happen in real life.

As for what's wrong here, I don't know. I must admit that it's a little tricky troubleshooting issues over forum posts. While I have a few guesses, I'll wait to see what Jeff says first.

I do have one unrelated question, though. Just out of curiosity, why didn't you stripe your Mac Pro's four internal disks? You can use Disk Utility to create a software RAID striped array (equivalent to RAID 0) and get speeds much faster than what you're getting now.

 

[Jeff Kilgroe]

OK, I'm going to address this using Brian's numbers. From this posting of his, and your confirmation, it's a lot clearer now. I'm still not sure what configuration you were talking about that was giving you 27 minutes to copy 64GB over FW800, there is definitely a bottleneck or some sort of speed obstruction there. Or are you talking about the SanDisk card being read?

Okay, so... Just to be sure, is this correct?

Mac Pro Internal:
- Two 2TB WD Blacks (88 MB/s - Estimate: 12 minutes to transfer 64GB)
- Two 1TB WD Blacks (96-102 MB/s - Estimate: 11 minutes to transfer 64GB)
All 90% full

These numbers are right about where they should be. Perhaps a bit slow, but with the drives running full that can make all the difference. As you fill up hard drives, they don't run slower because they are full. What happens is you tend to delete files and then add new ones regularly over time as the drives fill up. This leads to fragmentation of your data and your read/write heads have to jump around and seek to lots of different locations to read or write data. That slows things down. That said, optimal speed for the 2TB WD Black drives is 95MB/s or so. You can get to about 100~105MB/s if you read large chunks of data that are contiguous and under ideal conditions.

The 1TB drives are actually a bit faster and show based on your reported numbers. But once again, still suffer from the same issues. For reference, the WD Black-X 7200rpm 1TB 6Gbps model with 64MB cache is the fastest 1TB HDD on the market. Under ideal conditions, the fastest sustained speeds you will see on contiguous data with that drive would be about 125MB/s read and 95MB/s write. Of course, bursts can go higher, but those are sustainable numbers. The latest Hitachi UltraStar and Seagate Barracuda XT 1TB drives are pretty much the same in terms of performance. Not sure which model of WD Black drives you have, but your numbers seem to be in line with typical 1TB HDD numbers.

External FW800 Drive:
- OWC On-The-Go Pro (Your speed: 38 MB/s, OWC benchmarks: 73-80 MB/s)
5% full

Er... Those OWC benchmarks can be summed up in one word: CRAP. First of all, they don't specify what drive they are benchmarking. The capacity numbers indicate it's a 320GB unit and we'll assume it's the 7200rpm model. Well, the fastest 7200rpm 320GB HDDs out there can only post sustainable numbers in the neighborhood of 55MB/s for writes. I'm going to go out on a very short limb here and assume that these benchmarks are showing the FW800 model -- the ~75MB/s rates suggest that and with the small data sizes, the numbers they're posting are indicative of being cache-loaded. The largest test number they show is for transferring 100MB of data. And like most contemporary HDD's on the market, the unit they're testing probably has 32MB of cache. So, with the cache active, you won't see any overrun or real shift in registered speed unless they copy a few hundred MB or half a GB or more...

To be perfectly frank here, for a 320GB 2.5" HDD, that 38MB/s is not out of line. I would think it's a bit low if it's the 7200rpm model, but I would have to see it to really determine if there is a problem and what that problem might be. If it's the 5400rpm model, I'd say the speed looks about right. If it's the 7200rpm model, I'd say it's running about 8 to 10 MB/s slow...

SanDisk FW800 CF Card Reader:
- CF Card (Your speed: 38 MB/s, Should be: 60 MB/s)

LOL... Nope. This is a prime example of why RED has not certified any third-party CF cards or other flash memory to work with their cameras. In fact, if you're getting over 35MB/s sustained from a SanDisk "60MB/s ", then you have a good one! I have yet to see any CF or SDxx or SSD on the market, other than one from RED, where the manufacturers don't flat out LIE about the performance. That 60MB/s stamped on the front of the SanDisk card is their advertised peak burst rate for reads. That's it. You wanna know a little secret? The 90MB/s SanDisk cards are actually slower! They have higher burst speeds, but their sustained performance is even more erratic and often clocks much slower on large transfers. I've tested these cards quite a bit, BTW...


OK, so then how do we get all these fast speeds we're talking about above??? We use faster storage solutions. In most situations, the interface you're connecting with does little to determine how fast your drives are. It is only the bottleneck when you start using really fast drives or multiple drives in RAID configurations. As Brian asked above, how come you don't have your 2TB HDD's in a RAID-0 stripe configuration? That will effectively convert them into a 4TB volume (which you can partition into smaller pieces if you desire) and will effectively double the speed by accessing both drives and writing interleaved bits of data to them simultaneously. Want to triple the speed? Add a third drive. Quadruple? Yep, you get the idea.

The '09 Mac Pro I'm sitting at right now has a WD Black-X 1TB drive in the lower optical bay for system and apps. It has 4 of the 2TB WD Black-X RE4 drives in the internal drive trays. All in a RAID-0 stripe. I can read and write multiple streams of 2K ProRes in real time, can play dual streams of 4K ProRes from that array. Can write 2K DPX or TIFF sequences in real-time with bandwidth to spare.

OK that's fine and great, but there are some trade-offs to be made. Such as data security and the ability to manage these larger volumes of data. You will want a good backup solution in place because if one drive fails, all the others in that RAID volume are essentially rendered as good as blank. RAID level 5 is similar, but writes additional parity and redundancy data across the drives, so if one drive fails, you can recover and rebuild it. Theoretically. In many of my personal experiences, I haven't been so lucky. Typically one drive fails due to a controller failure or power issue and takes out another drive or two with it. You know, Murphy's Law and all that. Have a backup.

Anyway, we're getting into an entirely different discussion here that has been heavily discussed on these forums many times before. But this is all food for thought. I'm still a little fuzzy on some of your numbers, like the 27 minutes to copy the 64GB, as that is quite slow -- half as fast as it should be over a FW800 connection. One of those WD HDD's can saturate a FW800 connection, even full and fragmented as your's seem to be. Although, that OWC unit seems to be the culprit, but a single 2.5" HDD in an external enclosure like that is never going to be very fast...

 

[Brian Iannone]

SanDisk FW800 CF Card Reader:
- CF Card (Your speed: 38 MB/s, Should be: 60 MB/s)

By the way, when I wrote "Should be: 60 MB/s," I meant "SanDisk says it should be: 60 MB/s."

 

[Brian Iannone]

The 90MB/s SanDisk cards are actually slower! They have higher burst speeds, but their sustained performance is even more erratic and often clocks much slower on large transfers.

I know this question isn't really related to this thread, but now I'm curious... Jeff, have you done speed tests on SanDisk's Extreme Pro "90 MB/s" CompactFlash Cards (these)? If so, what speeds have you found them to produce?

 

[Jeff Kilgroe]

I know this question isn't really related to this thread, but now I'm curious... Jeff, have you done speed tests on SanDisk's Extreme Pro "90 MB/s" CompactFlash Cards (these)? If so, what speeds have you found them to produce?

Yep, tested several of those. Best sustained write times I could muster out of them were just over 40MB/s. I could actually get them to peak close to 95MB/s on reads, with sustained hovering around 53MB/s, but a performance graph looked like a cardiograph of a rabid squirrel on meth. They are probably the most erratic flash devices I have ever tested. I have not tested the 100 and 120 MB/s variants, but I've heard nothing but bad things about them. In all the testing I had done with CF cards over the past few years, the most solid-performing cards I ever ran across were the Transcend 333X 32GB cards. They're out of production now, but they were solid cards and actually perform the best in my DSLRs compared to the other "fast" cards from SanDisk, Lexar, etc.. They are very solid and regular in performance, not erratic. They sustain an even 42MB/s or thereabouts speed for writes and close to 55MB/s sustained for reads, which is higher than their advertised speed. Not quite fast enough for use with the RED One, almost comparable to the RED and Lexar 8GB cards. They're consistent and don't tend to spike or fall off and have been my favorite cards for use in the 7D and similar cameras.

 

[Jon Thomasberg]

..... but a performance graph looked like a cardiograph of a rabid squirrel on meth.

Love the analogy, Jeff

 

[Brian Iannone]

Yep, tested several of those. Best sustained write times I could muster out of them were just over 40MB/s. I could actually get them to peak close to 95MB/s on reads, with sustained hovering around 53MB/s, but a performance graph looked like a cardiograph of a rabid squirrel on meth. They are probably the most erratic flash devices I have ever tested. I have not tested the 100 and 120 MB/s variants, but I've heard nothing but bad things about them. In all the testing I had done with CF cards over the past few years, the most solid-performing cards I ever ran across were the Transcend 333X 32GB cards. They're out of production now, but they were solid cards and actually perform the best in my DSLRs compared to the other "fast" cards from SanDisk, Lexar, etc.. They are very solid and regular in performance, not erratic. They sustain an even 42MB/s or thereabouts speed for writes and close to 55MB/s sustained for reads, which is higher than their advertised speed. Not quite fast enough for use with the RED One, almost comparable to the RED and Lexar 8GB cards. They're consistent and don't tend to spike or fall off and have been my favorite cards for use in the 7D and similar cameras.

I see. When you tested the card and got read speeds of 53 MB/s sustained and 95 MB/s burst, what speed test utility did you use? Have you ever tried transferring data from a full card over to your computer and clicking on the 'More info' dropdown check in the Windows file transfer dialog box? It shows a speed estimate along with remaining time, number of files left, etc. What sort of speeds does that typically show you? Also, what interface are you using between the CF card reader and your computer?

The reason I ask about CompactFlash card speeds is that I just (quite randomly, actually) performed a speed test on one of those "90 MB/s" SanDisk Extreme Pro cards about a month ago on a Lexar USB 3.0 CF card reader. The results were quite interesting. I'll run another test tonight and post some screenshots.

 

[Jon Jones]

It took me 20 minutes tonight to empty my 64GB SSD over eSATA. It was totally full from shooting for less than an hour at 4k; RC 6.1.