Ballooning Data Rates and Crippling Workflow

[I Bloom]

Is this true if you are using dedicated software, or does this just apply for dragging and dropping? Because ShotPut Pro, at least, copies faster to 2 drives simultaneously than if you did one at a time, and it's only a little slower than it would copy to 1 without it. 60 gigs takes about 40 mins, for instance...

Basically any program will do this if it is accessing the same drive at the same time for two different reasons, even if you are doing the same exact copy at the same exact time the operating system treats them as two distinct actions. It's not really a software issue, it's a hardware issue. However software can overcome the problem easily just by waiting to make the second copy after the first is done.

This is only really and issue with mechanical hard-drives such as Red drives. Solid state drives might add some overhead by switching but even if they do it's muuuch less.

IBloom

 

[Blair S. Paulsen]

FWIW my best benchmark times when working on a MBP with the eSATA Express/34 adapter was to hook up the RedDrive via R2E and the catch drive via the eSATA as well, make one copy, swap the catch drives and make another copy. Faster than any daisy chain. Faster than FW800 off the RedDrive and eSATA to 2 catch drives. Obviously if you have an upgraded drive the R2E cable is out. The option to have software that forces the copies to be sequential sounds like a winner to me. Anyone know of a current methodology for that, or are we waiting for the Mac port of R3D Manager?

Cheers - #19

 

[Conrad Hunziker]

The option to have software that forces the copies to be sequential sounds like a winner to me. Anyone know of a current methodology for that, or are we waiting for the Mac port of R3D Manager?

Cheers - #19

You can do this today with the current version of R3D Data Manager. Open multiple copies of R3D Data Manager:

Copy 1: Red Media to destination 1
Copy 2: Destination 1 to Destination 2
Copy 3: Destination 2 to Destination 3
...

You still cant do that simultaneously. However, by just making Destination 1 an eSATA RAID setup, and destination 2 a eSATA device, you can do this round robin with zero downtime.

The next version of R3D Data Manager is days away.

 

[Conrad Hunziker]

However software can overcome the problem easily just by waiting to make the second copy after the first is done.

Theres many ways to overcome this issue, the suggestion above is the slowest, and not used by R3D Data Manager.

This is only really and issue with mechanical hard-drives such as Red drives. Solid state drives might add some overhead by switching but even if they do it's muuuch less.

SSDs and CFs still run into the issue of data throughput on the bus. If your destination is faster than your source (especially as you add multiple destinations), then this will always be an issue no matter what the source technology. The current windows version and next mac version of R3D Data Manager eliminates this issue - the single slowest device is the fastest you will copy at. Of course, you will still need to avoid doubling up on a single firewire bus.

 

[Noah Yuan-Vogel]

I didn't realize R3D Data manager for Mac couldn't do one read for many writes, but then I haven't used it. I was always impressed that the Win version supported this but then I don't use it because I found the software to be too buggy and provide two few details of copies in progress.

I recently built a portable workstation with pci-e2.0, Sata6G, and USB3 and an internal RAID for just this reason, esata or sata is mandatory in my mind, as are backup destinations that can handle writing 100MBps+ sustained, FW800 is a backup if necessary. I did a couple low budget jobs last year that only had 4-5 8GB cards and required copying RC36 footage on a laptop to FW400/USB hard drives (thats what was provided by production) so I know the ridiculousness of dealing with this kind of data without proper equipment and media. What's worst is doing that in the front of a van or on top of a car. One bumped cable and the whole production is held up waiting for a new card to shoot to. With the Red Drive, I am often forced to do many partial copies throughout the day and/or forgo one backup drive by not formatting the 640GB Red Drive until after the post house has received and backed up a transport drive to keep up with data and avoid coping for hours at the end of the day. From now on I will insist on the portable workstation for RED shoots. It's so new I haven't brought it on a RED shoot yet, but I did a few SxS shoots and copying a 16GB card in under 3 minutes (80+MBps) is a beautiful thing, that's twice as fast as my old laptop did to its internal drive and about 4 times as fast as it did on average with external drives.

I think software will really help this, I know there are a few solutions that do it but it should be standard to be able to write to all destinations from a common buffer and a single read and have stable well-laid-out software that allows canceling without freezing the software or leaving partial files, allows resuming transfers, and gives full real-time details on file copy speed, progress, and verification.

 

[Conrad Hunziker]

I didn't realize R3D Data manager for Mac couldn't do one read for many writes, but then I haven't used it. I was always impressed that the Win version supported this but then I don't use it because I found the software to be too buggy and provide two few details of copies in progress.

Too buggy? To date we have not received a single bug report of the windows version, from anyone. If you have some, please submit them.

We will bring the windows version up to the same reporting features as the Mac version, as soon as this mac port is finished.

 

[Noah Yuan-Vogel]

I will admit my experience with R3D Data Manager for windows is rather limited. I really only tried a trial version once and when I saw it gave me so little information about the copy (I expect to see speed, progress, current file, etc) that was in progress I canceled the copy and the program froze up and I had to eventually end the process after it didn't unfreeze for a few minutes. I apologize as calling the program buggy could be a significant mischaracterization since my experience was very limited and anecdotal. It didnt give me a great first impression of the software, though. I was also off-put by the fact that the versioning and feature set was not unified between the mac and windows version as I use Windows far more often than Mac due to the greater quantity and stability of software tools and drivers in windows in my experience (Macdrive, Teracopy, expresscard-pcie bridge hotswap support, bad sector remap in chkdsk, trim function for SSDs, etc).

 

[Rob Ruffo]

Here's a really sleazy solution I used one time, being my own DT on an 18 hour shoot: Go to an anonymous big box store, the kind that put mom and pop out of business, buy another laptop, then, after that one job where more speed was needed, return it on the 20 day no questions. I know, it's ghetto, but it might save your sanity, and it only costs gas and/or shoe leather.

 

[Jay Gannon]

I think what it comes down to is if your paying for a DIT he should have a MacPro based station.
I wouldnt DIT anything short of a free gig on a Macbook Pro their just not up to the job.

Saying that theirs a 15" and a new 13" in the Jeep at all times as emergency devices.