Fastest Thunderbolt DIT solution

[David Baumber]

I'm currently using Mac Air with RED SSD USB3 reader to a Thunderbolt Seagate Goflex. I backup each SSD to 2 drives. It's fast but is there a faster solution? I am considering a single RAID instead of the 2 x Goflex drives. Would you think this was crazy? Are there better Thunderbolt drives available you have tested?

 

[Nick Burridge]

You're using the GoFlex Thunderbolt adaptor? You can attach 2.5" SSDs to those and get some pretty impressive speeds. Capcity vs cost pretty limiting though.
Aside from that LaCie do thunderbolt versions of most of their drives (Rugged, Little Big Disk, d2, etc) - they should all be faster than a stock GoFlex drive. Promise do some big thunderbolt raid units too.
Or get a thunderbolt->esata hub and go out to fast esata storage

 

[Kaku Ito]

What is the transferring rate you getting now?

 

[AndreasOberg]

Your limiting factore will be the Red SSD. My harddrive read about 200MB/sec which your external SSD drives will have no problems keeping up with. So upgrading the external drives will not help I'm afraid. Simple answer is I don't think you can do something to make it faster.

Use Black Magics HDD benchmark program to test how fast the different drives are.
- If the Apple could somehow write to 2 disks at the same time the speed could actually be double, but I don't know how to do that, on pc you can do that and it would be faster.
- I'm beginning to think that Red should build a DIT station themselves where you can connect with USB3 and thunderbold harddrives and red drives and just copy directly from there. This would be faster and easier to manage.

Best
/Andreas

 

[Brian Merlen]

I don't use a usb3 to thunderbolt solution but I would imagine since your going from a proprietary solid state device to a single optical drive that the bottle neck would be the optical drive. As a test go get a thunderbolt raid (promise r6 and maybe something else, that ssd lacie, whatever) and do a test...


Just a guess on my end, for all i know the usb3 is the limiting factor, but i don't think it is. the only way to know for sure is to test it. I'd like to think the guys throwing around more coin for there raid 5 or 6 capture scratch's would out perform single optical hdd performance...

ps doesn't r3d data manager do single to double stream copies, I remember reading about it being an advantage vs drag drop copies. But then if you add checksums in you go backwards on any system supposedly...

 

[Michael Romano]

since your going from a proprietary solid state device to a single optical drive that the bottle neck would be the optical drive.

What optical drive?

With SSD over thunderbolt, bottleneck is always the REDMAG

 

[Brian Merlen]

are both ends ssd? the redmag obviously is, but is the seagate an ssd or a 7200rpm or 5400 rpm hdd?

I would imagine going to a raid 6 would be faster then copying to a single hdd drive solution... I mean whats the actual r/w of a single hdd vs a redmag, the mag has to be faster then that I'd imagine...

but going from mag to ssd or raid 5/6 solutions surely must be faster then a single hdd?

 

[AndreasOberg]

These are the speeds I have measured
RED 512GB SSD 195mb/sec read write
RED 128GB SSD read 177 - write 170

Normal spinning 7200 harddrives if they are new and big will normally be around 140-200mb/sec. Speed depends on brand and how full they are. As I remember the segate barracude are the fastest with near 200mb/sec.
USB3 and Thunderbolt are both fast enough to handle these speeds.

Myself I use a GTech Raid with 2 drives in. They have 320MB/sec read 310 write when they are empty so more than enough to here the RED SSD are the bottleneck.
The WD 2TB Passport can write about 110mb/sec when they are empty. They are small and need no external power, but are a bit too slow for the job at hand.


For really fast offloading SSD drives seems like the best option since they are so fast I would myself get two Samsung 540 SSD 512GB and purchase a thunderbolt enclosure for each.
But like I said the main way of making this faster would be if we could read the Red SSD and write to 2 disks simultaneously. Hardware wise it's possible, just need software to make it happen.
Another way would be to have 1 raid enclosure that has Raid backup, something like Raid 1. This will save to 2 disks at the same time. Still not as safe as having to physically different disk.

/Andreas

 

[AndreasOberg]

Just a guess on my end, for all i know the usb3 is the limiting factor, but i don't think it is. the only way to know for sure is to test it. I'd like to think the guys throwing around more coin for there raid 5 or 6 capture scratch's would out perform single optical hdd performance...

USB 3 is not the bottleneck here. It can handle about 500MB/sec while thunderbolt is double. No normal ssd drives can write faster than the usb3 standard. Only if you use something like the Revo drives which you connect to PCIE will you get faste ssd speeds.
/Andreas

 

[Mathieu Ghekiere]

Have you thought about the Pegasus Promise R6? You get 500MB write speeds. Great thing.

 

[Peter Majtan]

If the Apple could somehow write to 2 disks at the same time the speed could actually be double, but I don't know how to do that

My standard DIT procedure is to use identical drives - 2 or 3 depending on the budget / seriousness of the production company.
At the start of the project the drives are setup as RAID-1 (mirror), this basically means that they all appear as just a single drive.
Copying to this "drive" has the same exact speed as coping to an individual drive before they were RAID-ed, but the data is being written to ALL the drives in the RAID at the same exact time.
So if your RAID-1 has 3 drives - you are essentially cutting down the time needed to 1/3 (1/2 in 2-drive setup).
When project is done - or the drive(s) are full - you simply "destroy" the RAID from within the Disk Utility (where you have setup the RAID in the first place).
This will result in individual drives with identical set of data. Works like a charm and I have used this on over 50 projects and counting...

:sifone: Peter

 

[AndreasOberg]

Have you thought about the Pegasus Promise R6? You get 500MB write speeds. Great thing.

Hiya Mathieu. Can you elaborate a bit more Mathieu. What raid more would you use then? A few problems I see
- normally it's used in Raid 0 which is super risky, that is when you get 500mb write speeds
- 500mb write speeds doesn't matter since red ssd cannot read faster than about 200mb/s
- it's also darn big and expensive

Best wishes

 

[AndreasOberg]

My standard DIT procedure is to use identical drives - 2 or 3 depending on the budget / seriousness of the production company.
At the start of the project the drives are setup as RAID-1 (mirror), this basically means that they all appear as just a single drive.
Copying to this "drive" has the same exact speed as coping to an individual drive before they were RAID-ed, but the data is being written to ALL the drives in the RAID at the same exact time.
So if your RAID-1 has 3 drives - you are essentially cutting down the time needed to 1/3 (1/2 in 2-drive setup).
When project is done - or the drive(s) are full - you simply "destroy" the RAID from within the Disk Utility (where you have setup the RAID in the first place).
This will result in individual drives with identical set of data. Works like a charm and I have used this on over 50 projects and counting...

:sifone: Peter

That sounds like the perfect workflow Peter. What harddrive enclosure do you use for this? And what harddrives?

I'm just thinking, but I think I have a perfect way of doing it.
- 3 ssd drives in raid 0. So you have the backup on 3 disks. This will be very fast, lightweight AND less sensitive to vibrations and impacts than magnetical disks. Shouldn't be too expensive either. Samsung 840 512GB are quite cheap.
- spinning drives could be used as well but will cut transfer times in nearly half
- Ideally we would have thunderbolt. I just trust it more than USB3, I had problems with USB3 but never with thunderbolt. Also remember that the raid station uses 2 usb3 so on an apple retina you have no usb3 left.
- When the copy is done I would remove the 3rd drive from the enclosure. It's always risky to have the same data in one place. What if you drop it, or someone steals it, or forgets it. I always splits the data. I have carried data worth tens of million dollars and it really makes you rethink your security.

Finding a good enclosure could be a challenge however.
/Andreas

 

[AndreasOberg]

Wow, our "prayers" have been heard. Check this out.
It's can have 4 2.5" drives, either ssd or spinning drives.
It also has 2 thunderbolt and 1 usb 3 which is PERFECT, we can only hope that more devices begin adding thunderbolt. Or eh, if Apple adds more usb3 ports on their laptops.
Its also made of carbon fiber which seems nice.
It also has a battery protecting from power failure. This is very important since I just lost a red 512 drive because of power problems.
http://www.drobo.com/products/professionals/drobo-mini/index.php#!prettyPhoto

Have to ask if their Raid 1 supports multiple disk. They also support using 2 disks for storage and 2 disks for backup which will give you double the size for storage but not quite as safe.

One of the most important reasons for me to be fast in offloading footage is to be able to offload data at lunch for example so one can continue shooting. Another option is of course to buy more red mags, but then you dont have the same backup and they are considerably more costly.
/Andreas

 

[Kwan Khan]

Mbp (retina) + OCZ Vertex 4 512GB with Seagate Goflex Thunderbolt adapter

Below is my MBP (r) added Just two seagate backup Plus (New version, 7200 rpm, Raid 0 via Seagate thunderbolt adopter,
Four seagate backup Plus, Raid 0 you will get up to above 700 Mb/s read &write

 

[Patrick Grossien]

Wow, our "prayers" have been heard. Check this out.
It's can have 4 2.5" drives, either ssd or spinning drives.
It also has 2 thunderbolt and 1 usb 3 which is PERFECT, we can only hope that more devices begin adding thunderbolt. Or eh, if Apple adds more usb3 ports on their laptops.
Its also made of carbon fiber which seems nice.
It also has a battery protecting from power failure. This is very important since I just lost a red 512 drive because of power problems.
http://www.drobo.com/products/professionals/drobo-mini/index.php#!prettyPhoto

Have to ask if their Raid 1 supports multiple disk. They also support using 2 disks for storage and 2 disks for backup which will give you double the size for storage but not quite as safe.

One of the most important reasons for me to be fast in offloading footage is to be able to offload data at lunch for example so one can continue shooting. Another option is of course to buy more red mags, but then you dont have the same backup and they are considerably more costly.
/Andreas

Andreas, do you have any experience with drobo enclosures? Because of reliability, service, etc?

 

[AndreasOberg]

Sorry Patric, I have not tested them only read about them on the internet. Compared to their previous cases the new ones seem to be a strong step up in quality and features. I always have reliability in mind and from reading the reviews and looking at how it's constructed they seem safe. I read a test where they unplugged harddrives and even the power of the case while in operation and everything worked fine afterword. That is quite impressive.
I looked at their Drobo 5D which also is very interesting. It supports 5 3.5" harddrives and 1 ssd. Interesting for home work, but not sure I would want to take it on a plane since it would be about 9kg heavy. It's still light for what it can do but weigh adds up. I just found out that the Drobo Mini also supports a 5th drive which is an ssd. Interesting.

/Andreas

 

[Patrick Grossien]

Andreas, that whole setup really sounds promising. Especially if you're trying to pack light! Maybe Jeff knows more about it... he usually does

 

[Peter Majtan]

You don't want to be using HW RAID-1. In general these enclosures will partition the drives in a way that if you delete the RAID - you end up loosing all the data. On the contrary - the Apple Disk Utility when you select to delete the RAID-1 will actually prompt you and tell you that "after deleting the RAID-1 volume you will end up with two (or 3) identical volumes and each will contain an identical copy of the data from the RAID-1 volume. So how you connect the drives really does not matter - it can be FW, USB, Thunderbolt, or any combination of these - as long as the Disk Utility can see the individual volumes and you are able to use them to create the RAID-1 volume. For best / optimal performance you want all the individual volumes to be an identical drives connected via the same interface...

:sifone: Peter

 

[Tom Gleeson]

What about programmes like Double Data that can copy onto multiple drives simultaneously ? I tested the demo of version 1 and got some good speeds. Only marginally slower than finder copies. Did not test with multiple destination drives as it was buggy so I dropped it. I see there is now a v1.1 so hopefully stable now.