Released: R3D Data Manager for Mac v6.1

[Conrad Hunziker]

R3D Data Manager for Mac version 6.1 is released!

Updates since v5.4:

• Support for single source read to multi-destinations We completely re-wrote the copy process from top to bottom. The program now does a copy pass first, so that at least a copy of your data is stored should something happen to the original. In addition, its not possible for our program to write anything to the source at any time. We also re-wrote the code to only read once from the source and write to multiple destinations in one pass. This is a significant time savings.
• Unified processing code for Mac and Windows We also now unified our code between the Mac and Windows releases for the copy process. Though each version will have slight differences, the heavy lifting is done by the same code on all platforms.
• Expanded code for new Epic released details We have supported the announced Epic details for some time, but this version now aligns our software with newer Epic updates.
• Updated Quicktime Render for RedRocket support We now support RedRocket rendering. To enable you must have a RedRocket and the beta version of REDline installed.
• Updated Quicktime Render for new color science We now support the new color science. To enable, you must have the beta REDline installed.
• Added support for new gamma and color spaces These options only appear if the beta REDline is installed.
• Added checking to ensure output codec is supported
• Changed user interface so the buttons and icon display overall status for all copies We changed this because the previous way may have been misleading to some users.
• View Results button now shows all results for all copied folders, not just the one selected This aligns ourselves with the user interface goals.
• Retry All Errors now retries all errors for all sources This allows for a single click to retry all errors
• Fixed a bug that caused extremely slow copies in Mac OS 10.5 This bug only affected certain processors, even though the same code worked in Windows and 10.6
• Now shows in the activity log if writing to a destination was skipped by user preference
• Fixed a display bug in R3D File Recovery
• Fixed a bug with verify checksums button not finding original checksums
• Removed alternate checksum folder option, no longer needed It is not possible to write to the source, so this option is no longer needed.
• Fixed an error when sources would be read even if the copy is to be skipped to all destinations

Upgrade Notes

This version requires a new registration code. The registration code for 6.x can only be obtained if you have a lifetime upgrades packages or purchase an upgrade for USD$16.99. You can check on your current license level by visiting our website order form.

If you are upgrading from version 5.1 or earlier: Due to an error in a 3rd party installer, you must install this upgrade manually.

 

[Conrad Hunziker]

The following announcement deserves its own post: When copying to multiple destinations, R3D Data Manager is faster than Finder!

During our complete re-write of our copy process we also decided to see how we could optimize the process. We spent a great deal of time breaking down the process to its elements, then subdividing those and running them in parallel. What we ended up with is a system that when copying to 4 destinations is faster than a finder drag-n-drop copy!

We know theres going to be a lot of doubters out there. Theres people who refuse to use R3D Data Manager because "its too slow". So we prepared a demo of our software in action, and compared it to finder. We took one of our test systems, a Mac Pro with an Atto R380 and 8 drive RAID 5, added a couple of internal drives and eSATA drives. We shot some material, and hooked the Red Media up to the system via Firewire 800. The clock starts once the drive is hooked up to the system.

It took finder 15m 45s to copy all the footage to 4 destinations simultaneously. That includes the 7 seconds it took the drive to mount, and the 38 seconds it took me to setup all the concurrent file transfers.

It took R3D Data Manager v6.1 15m 15s to copy to the exact same destinations, with full source checksum verification and checking each destination independently.

Im sure theres still a few holdouts, so how about a side-by-side shootout. Ive uploaded a couple of videos to our site and put them on a page. Theres a brief overview video and a 15m 45s video of the full copy process, for those that like to watch progress bars. http://www.r3ddata.com/fasterthanfinder/

One thing to note about the R3D Data Manager side is how much the mouse does NOT move! Once I start the clock, the mouse doesnt move until I stop it. Also not shown was the text message it sent me telling me its done!

Another thing for you eagle eyed people out there: R3D Data Manager actually starts to copy before finder even notices the drive is attached! If you look frame by frame on the R3D Data Manager side at around 7 seconds into the video, you'll notice that R3D Data Manager detected the Red Media and started the copy, but the Red Media hasnt even shown up in the finder window right next to it. I know its only a tenth of a second, but I found that to be quite interesting.

So, if a copy and full source and destination verification is faster than a simple copy via finder for multi-destinations, why would you not use it?

 

[Nick Shaw]

So, if a copy and full source and destination verification is faster than a simple copy via finder for multi-destinations, why would you not use it?

Fair enough. But we know that Finder is not efficient for multiple parallel copies. How does the speed compare if copying to a single destination?

I often make my first copy onto one destination (RAID-5 protected) and then start working with that for on-set compositing etc, while the RED Drive is passed to a second copy station to make another copy to a transfer drive to go to the post facility.

 

[Dustin Cross]

I too am interested in how fast it is in series. I normally copy from Red Drive to destination A, then from A to B, etc... The firewire on the Red Drive is typically the slow point.

Then i compare the last copy to the original Red Drive.


Dusty

 

[Conrad Hunziker]

Fair enough. But we know that Finder is not efficient for multiple parallel copies. How does the speed compare if copying to a single destination?

You are never going to get around the bandwidth limitation on the source or destinations, and R3D Data Manager will never compromise on being a three-read process.

The point is that in the time it takes finder to do multiple copies, you can have R3D Data Manager do a copy, verify the source and verify the destination. Most people will copy to multiple destinations at once, because a) its easier and b) its generally faster.

 

[joe hedge]

The following announcement deserves its own post: When copying to multiple destinations, R3D Data Manager is faster than Finder!

Using ShotPut Pro I have consistently (past 1.5 years) pulled raw camera files off the Red Drive, via FW800, to 2 eSata drives plugged into the express 34 slot on a MBP, in almost the same speed that it would copy to one using the finder. 50 gigs from the camera drive copies in 25 to 35 mins depending how full the eSata drives are; if they're empty, it's 25, if near-full (close to 75%), 35 mins. No other programs running, of course. SPP has always had multi-threading but I leave it on single-thread because the i/o of the laptop doesn't seem to be able to handle multi-threading, so on a MacPro it would presumably go even faster. And a new version of SBP came out last week, which is noticeably faster than the one I've been using.

I'm running empty drives today on-set and I'll post the logs if anyone's interested...

 

[Conrad Hunziker]

Using ShotPut Pro I have consistently (past 1.5 years) pulled raw camera files off the Red Drive, via FW800, to 2 eSata drives plugged into the express 34 slot on a MBP, in almost the same speed that it would copy to one using the finder.

Thanks for trying to rain on the parade, but its not going to work

The simple fact is that ShotPut's verification method is shockingly bad, at best. It doesnt verify that the source actually read correctly. It has a history of not reporting that a destination may be invalid. And its feature set is considerably smaller for more money.

Should we compare Shotput again?

 

[Gunleik Groven]

Thanks for trying to rain on the parade, but its not going to work

The simple fact is that ShotPut's verification method is shockingly bad, at best. It doesnt verify that the source actually read correctly. It has a history of not reporting that a destination may be invalid. And its feature set is considerably smaller for more money.

Should we compare Shotput again?

Can confirm Shotputs verification issues.
I am also not too impressed with ShotPuts copyong skills if you copy from only one media (to 2 destinations)

One thing which is nice, though, is that you can simultainiously copy from more than one destination to more than one destination. (ex one USB + one FW copy), and then it becomes rather rational copy-wise.

Is it possible with simultanious copy-jobs in r3d manager?

Gunleik
(who'll get it tomorrow...)

 

[David Battistella]

This is the most important tool you can own for your RED camera.

David

 

[Gunleik Groven]

I have used it on different productions.
The copies have belonged to the productions, though. As it will now...

 

[Dave Weber]

Hey Conrad!

I've been patiently waiting for this since you told me about this version in another thread. I had questions about transfer speeds.

I just loaded this new version and used it to transfer some footage we shot last week.

MacBook Pro, 53GB, MD5 checksum to 2 sata drives from RED Drive via R2E cable ...... 25 minutes ...... this is way faster than the older version of R3D Data Manger.

Thank you, Thank you, Thank you.

You have made my life a little easier!

 

[JosephArthur]

Thanks!

 

[JosephArthur]

Guys are you testing with the RED readers and how would it test or work with the Red Tape Backup?

 

[Brandon Kraemer]

This is the most important tool you can own for your RED camera.

David

Second that... or for a post facility to use when adding Red footage to a RAID. We don't move .R3D files in our facility with our using R3D Data Manager.

 

[Justin Barham]

The following announcement deserves its own post: When copying to multiple destinations, R3D Data Manager is faster than Finder!

That is exciting news. Great work Conrad!

 

[Chris Parker]

AWESOME work man!

 

[joe hedge]

From just a few minutes ago:

ShotPut Pro by Imagine Products (www.imagineproducts.com)
Offload Began at 2010-05-18 11:52:02 -0700
Offloading files from /Volumes/A026_05189N to /Volumes/PF4/A026
Total files at source:133
Total bytes at source:62786951226

ShotPut Pro Offload Summary
Total Files at source:133
Total files copied:133
Total bytes at source:62786951226
Total bytes copied:62786951226
Total offload duration (HH:MM:SS):00:25:54
Verification Status:Offload Verified

ShotPut Pro offload ended at 2010-05-18 12:18:01 -0700

From RedDrive via FW800 to 2 never-used eSata drives via MBP express34 card

62 gigs (actually 124 gigs)

26 mins

Dragged through all files on both eSata drives, all files match camera drive

 

[Gunleik Groven]

BTW. Is it possible to use r3d manager on network-volumes.

G

 

[Conrad Hunziker]

Guys are you testing with the RED readers and how would it test or work with the Red Tape Backup?

We are waiting for the final released version of Red Readers for testing. In theory as long as it mounts on your filesystem, it should work. And if it doesnt mount of your filesystem - theres larger issues!

 

[Conrad Hunziker]

Dragged through all files on both eSata drives, all files match camera drive

Good luck with that. Because you had to drag through all sets of copies:

1) It took you longer to complete the entire process. Because we use checksums, you know that both files are the same on both copies, and therefore only need to check on copy.

2) You may have missed frames or partial frames that have errors, even if these errors were not caused by the copy process. For proper data management you must watch the footage in real time, regardless of what software you use to copy it.