Data Management - drop frames generated during download.

[Pasha Patriki]

I am using Shotput Pro software and a 15" MacBook Pro for file transfer, Lexar FW800 Card reader, external drives are USB and/or Esata (through Expresscard 34/Esata adapter).

This system worked flawlessly for me, however now problems started happening, like some clips get corrupted, and some have green drop frames.

When I check the original CF card (16GB REdFlash), the footage is clean.

The surprising detail is that the drop frames appear in different places on two or three backup copies, so that puzzles me. Since Shotput Pro creates multiple copies simultaneously, so if there was an error, it would be identical on all copies?

The other surprise is that ShotPut Pro doesn"t actually detect, and shows "OK" message upon completing of the transfer. Only when I visually verify the files, I discover the corruption.

I tried changing card readers (I have 3 of them), cables, everything....

Can't figure out a pattern of why this is happening.

And why Shotput Pro byte verification does not detect the problem...

 

[Conrad Hunziker]

I developed R3D Data Manager for this purpose, and I believe it overcomes the limitations of Shotput's byte verification.

R3D Data Manager reads the source material twice during the copy, and checks them against each other. This ensures that your media was able to reproduce the same data twice, so very likely the data is valid. Shotput's method doesnt do this - it assumes the read is valid, which it may not be. So any errors in the read will propagate to the destinations.

In addition, I dont know how their verification works under the hood, but the general process is to read the source and read the destination, and see if the zeros and ones match. The issue here is that after the source is deleted, there is no way to re-verify the destination. R3D Data Manager works differently. It adds up all the zeros and ones, then comes up with a unique number that represents that file. If any of the zeros and ones are switched, the number is vastly different. Since we save those numbers, we can verify destinations even after the source is written over. I have footage from build 8 that I know is exactly as the camera wrote it, because I used R3D Data Manager.

By the way, with all verification programs you will still want to visually verify the footage. You need to ensure that all the zeros and ones add up to form an image. The only way to do that is for a human to watch it.

 

[Pasha Patriki]

thanks Conrad!
I tried a trial of Data Manager a while back, just when I started getting into RED workflow... It kept crashing my laptop... Going to try it again.

Regarding the human factor. If you can"t trust the software, why bother using it? The copy and paste is more than twice as fast, and then you have lots of time left to visually check all the footage.

Just for the sake of philosophy..... Thoughts on that?

 

[Conrad Hunziker]

R3D Data Manager offers signifigant improvements over finder. Finder doesnt do any type of verification at all. It just dumps footage to the drive. With R3D Data Manager you just have to visually verify one copy of the footage. Since the checksums match between copies, you know that if one copy is valid ALL copies are valid. With finder, you would need to check each copy.

In addition, finder doesnt always copy all the material, and doesnt have any checks to ensure that it does. R3D Data Manager does.

In addition, R3D Data Manager will pass bond company regulations, if your show is bonded. Finder doesnt.

I could go on, but its already on our website all the differences.

Some references:
http://www.reduser.net/forum/showpost.php?p=595060&postcount=29

http://www.r3ddata.com/compare/