Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Data verification software

double data hangs up too much, i think shot-put is the way to go
 
Brian Merlen said:
double data hangs up too much, i think shot-put is the way to go
Click to expand...

+1

Only minor issue I've encountered (I run shotput on both Mac and Win machines) is that for some reason the time remaining indicator on Windows is completely inaccurate (I gather this is something of a known issue). But as long as you just know this in advance it's no big deal. Also, just make sure you take some time setting up your export presets the way you like them, otherwise it's easy to end up with some pretty cumbersome nested folder structures for your footage...
 
ShutPut is great at just transferring files, without caring what those files are. I recently stepped up to Silverstack from Pomfort, for the ability to generate reports at the end of the day, as well as viewing footage and metadata all within the same program. As of version 11 (which came out a few months ago) DaVinci Resolve has transfer/checksums built in, and the Lite version is free. I've been meaning to evaluate it, and haven't gotten around to it yet.
 
Terry VerHaar said:
You can look at Red Giant Offload.
Click to expand...

I tried offload. The interface is great. But it doesn't allow more than two backups. It doesn't create a verification check file at the end. On windows, it doesn't deactivate the sleep function while copying. It doesn't copy all the files. For example, it'll copy the r3d files but not copy the rmd files. This threw me off for a bit because when I did a folder compare after, the size was always different.

I've run into issues with Davinci Resolve Clone tool. I think it's something to do with copying between different file systems (exfat, fat32, ntfs). I believe Blackmagic confirmed this issue from another forum. This is referring only to the Windows version.

One program to try that's been good is Setcopy.

https://www.sandust.com/setcopy-software/

It's free at the moment.
 
Last edited:
Roger Singh said:
I tried offload. The interface is great. But it doesn't allow more than two backups. It doesn't create a verification check file at the end. On windows, it doesn't deactivate the sleep function while copying. It doesn't copy all the files. For example, it'll copy the r3d files but not copy the rmd files. This threw me off for a bit because when I did a folder compare after, the size was always different.
Click to expand...

Wait - are there actually RMD files created on the recording media? I didn't think there was.
 
Terry VerHaar said:
Wait - are there actually RMD files created on the recording media? I didn't think there was.
Click to expand...


If you have your Redmags set to mount Read/Write and adjust them in RCX before performing your copy, there will be RMDs. Whether or not this is the right way to do it is up for debate; personally, I'd rather my raw media mount Read-only, so that there's no chance to delete anything accidentally. But done the former way, you don't have to, say, set your RMDs twice, or perform an RSync after already copying and checksumming once.
 
I absolutely don't mess with my mags before copying. The very first thing I do is copy them to multiple destinations.
 
ShotPut is probably the best tool to simplify things. If you're running a Mac or Linux system, you actually have all the tools you need right in the OS, if you know what you're doing -- it's not instantly apparent unless you're a unix geek.

I'm old-school and started doing it the geeky way long ago and haven't changed as it all serves me just fine. On OSX, I use an automator widget that creates a block-level disk image of my REDMAG with a secondary verification pass and it generates a MD5 checksum.

No matter which tool you use, always do a visual verification of the footage. Because there is no error correction or checksum generated by the camera, so if data gets corrupted being written to the card by the camera or offloading the media to a computer, you'll never know unless you check the media with your own eyes. Once the media is offloaded and a checksum is there, then if it passes visual verification, you're good to go.

Being a small operation consisting of myself and an occasional assistant or two, I got into the habit of offloading media, but never erasing until all was verified later. Having enough REDMAGs for that gets expensive, though. And I can't work that way at the moment as I recently transitioned to mini-mags and don't quite have enough to cover a single day of busy or excessive shooting with Dragon.

So it takes longer to offload a card as I have it do the process twice and create images to two separate locations, but then I feel more confident about erasing a mag. I still only erase a mag if I absolutely have no other option. Plan to pick up more mini-mags early next year...
 
Hey Jeff, would you be willing to share your OSX widget? I know how to run everything I need through terminal (it's the backup to the backup of my backup method), but I'm curious to see how you lay things out.

Also, I can't remember the last job I was on that didn't need to erase media same-day. Aside from the largest budget movie I worked on, but even that, at the rate they were shooting, our original plan of "not clearing media until editorial gave the go-ahead" went out the window on day 2. Maybe it's just an east coast thing, but every single commercial, regardless of camera type, erases and reuses mags same-day. And most features.
 
Jeff Kilgroe said:
ShotPut is probably the best tool to simplify things. If you're running a Mac or Linux system, you actually have all the tools you need right in the OS, if you know what you're doing -- it's not instantly apparent unless you're a unix geek.

I'm old-school and started doing it the geeky way long ago and haven't changed as it all serves me just fine. On OSX, I use an automator widget that creates a block-level disk image of my REDMAG with a secondary verification pass and it generates a MD5 checksum.

No matter which tool you use, always do a visual verification of the footage. Because there is no error correction or checksum generated by the camera, so if data gets corrupted being written to the card by the camera or offloading the media to a computer, you'll never know unless you check the media with your own eyes. Once the media is offloaded and a checksum is there, then if it passes visual verification, you're good to go.

Being a small operation consisting of myself and an occasional assistant or two, I got into the habit of offloading media, but never erasing until all was verified later. Having enough REDMAGs for that gets expensive, though. And I can't work that way at the moment as I recently transitioned to mini-mags and don't quite have enough to cover a single day of busy or excessive shooting with Dragon.

So it takes longer to offload a card as I have it do the process twice and create images to two separate locations, but then I feel more confident about erasing a mag. I still only erase a mag if I absolutely have no other option. Plan to pick up more mini-mags early next year...
Click to expand...


Jeff just curious about the disk image technique you're using... I'm sure I could learn something from your approach.
I understand a block level disk image would possibly allow for REDundead operations on the image and checksum/verify you did an exact copy of the everything on the REDmag but...

as far as I understand a disk image is more dangerous than a classic disk/folder copy in the sense that if for some reason a single byte of data on that image goes wrong down the road you loose not just one clip but the whole REDMag as you cannot mount it... of course having several copies will let you retrieve your clips but I usually never recommend the 'disk image' method... altho I'm sure you have a good reason for doing this...
 
I've been using the latest version of Pomfort SilverStack on the last two features. While its a bit pricey it does a great job with plenty of options for the types of offloading you want to do. It recognizes most camera media and creates some petty handy reports.

I've been writing to four drives at once on the first pass and it chugs right along.
 
So does Offload still not copy the RMD? Anymore else have experience with Setcopy?
 
The software I like best is Yoyotta
 
You must log in or register to reply here.
Back
Top