Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Copy & md5 Shell Script

hi dustin, i dont see a link for the command line version. if i try to extract the script from the app version and run it it fails. is there a link to the command line version?

thanks, great script!
 
David,

In the first post is a link to SandustTools which is just a simple GUI that launches the command line app. Inside that little app you can find the script.

Or at the bottom of the first post you can download just the script. CopyScript_20110701.zip‎

Hope that helps.


Dusty
 
thanks, i missed that somehow...is it as up to date as the script that runs in the beta2 of the gui link that's posted?
 
David,

The version in the CopyVerify app is very different because it is written to use Pashua. Pashua is what is used for the GUI.

The version in SandustTools app is the most up-to-date. There might be a couple minor changes in the copyscript in SandustTools and the stand alone CopyScript.


Dusty
 
Dustin,

I have been looking for a fast/simple/reliable/free, copy and checksum app for a little while now. I am new to the idea of checksuming copies rather than just simple drag and drop for backing up media. This tool you (and everyone who has giving input and changes) have created is great! I can't wait to put it through the paces. Thanks for the app and all the knowledge I gained reading this thread.

-Sean
 
Hi there.
I don't know if you are really still working on this, but I wondered if I could ask something about it.

I have just been trying this out on some memory cards, to see how feasible it would be to use it at work, and while I imagine it works well with r3d files where each mag has its own unique name, I have noticed that if I drag and drop a volume such as EOS_DIGITAL, then it would overwrite anything previously copied from a card with the same name, without any warning. On the face of it, this seems a little too risky for certain types of media.

Is there a way for it to be adapted so that I can enter a new "reel" name for each transferred card, and possibly ask it to refuse to overwrite any existing transferred media.

Thanks so much for making this, and I hope I can get it to fit into the things I need to do at work.

Kind regards

- Ian
 
I've been using this app for a while now and love the simplicity. For DSLR shoots I create a new destination folder for each card.
 
Hi Dustin.
Thanks for the response. I have messed around with some scripting (a long time ago), and I was wondering how difficult it would be to implement an incremental folder creation function (just say from "0001" onwards, based on what is already in the destination folder) into what is already there? I obviously don't really expect you to do this, especially if it would not really be of any use to you for your own work.

I think perhaps the renaming of the cards before transfer might well be the best solution for this.

Could I also ask you one question about how the script/app handles existing data at the same path? If I had already transferred a volume "001" and then attempted the same volume name again with different media contained on it, what would the script be set to do? Would it merge the new media into the existing folder name? Would any files with the same filename be overwritten, ignored or warned against?

Sorry for all the questions. Thanks so much for your generosity with this. Is there any donation scheme set up for your work?

Kind regards,
- Ian
 
I think numbering the volume name before transfer is a good idea. But Ian has an excellent point — if you accidentally name a volume a number that's already been used, and the script overwrites the previous one, that could be catastrophic. Hopefully it will add a unique name to the new one.
 
Right now, the script would overwrite the existing files. I should add -n to the copy command so it doesn't overwrite.

Telling the script to add something to the fold name if the folder already exists would be easy.

This is something I have never had to deal with. I always know if I have duplicate mag names and deal with that before I ever download the mag. I would prefer the script to error out and leave me to fix the problem than do something automatically. Thats just me though.

When I get some free time, I will look into this.


Dusty
 
It's hard to believe, with all the big companies out there, but this is still my favorite way of downloading footage on set, although it's not perfect. So a big thank you to Dusty for writing this and leaving it free. You might want to consider making some more improvements and selling it for $80 or so.
Here are my thoughts on some other apps.

R3D Data Manager $80. Very stable, fast, recognizable name. No longer available. RED only.
Double Data $75. Unstable, sometimes crashes. Unusually slow. Confusing interface. RED and Alexa only. First destination copied first.
YoYotta ID $250. Stable and fast. Some nifty features, active development.
ShotPut Pro $100. Seems ok, have not tested much.
Silverstack $600. Great organization features. Unusually slow.
Light Iron Sentinel. Not available for purchase. Sounds great, but haven't tested.
CopyVerified. Very stable. Vastly inaccurate progress bar... First destination copied first. That last feature is the number one reason I prefer CopyVerified over everything else. I typically have a fast RAID on my cart, and 2 slow client drives (FW800 or USB 3 maxing out around 100MBps). CopyVerified lets me get the first copy done ASAP, so I can start syncing sound, checking footage, and transcoding while the slower copies are happening.
Express Dailies $15K. Obviously the cream of the crop, does everything, including setting to copy first destination first.
 
Yes same here, love this script and use it for all my copying needs, be it RED, audio files or anything else in fact.

Thought several times in the past about writing my own app for copying + md5 creation / verification but this tool works well and I really see no need at the moment to spend time writing anything else !

Thanks Dusty
 
Thanks guys!

I have a bunch of features I want to add that people have requested. I have just been too busy. I started a complete rewrite with better progress bar and even how much time is left. Then I got sidetracked looking at tools to write a cross platform version so it would run on OSX, Windows, and Linux. And working.

The features I am working on:

- better progress bar
- all copies at once option. Some situations the first copy first is not good
- RSYNC to make copies with no MD5. A little faster, does internal checksums, but does not save checksums. People ask for it a lot.
- ability to turn off MD5. People ask for it.
- Cross Platform so it runs on OSX, Windows, and Linux

I just need to not be so busy and I have gotten a little lazy about this since it works.


Dusty
 
Elliott,

I like command line, but I understand most people prefer GUI, so that is what I am working on.

What do you mean "the ability to create dynamic destinations based on shoot day and camera letter"?



Dusty
 
I like to have a folder structure like this:

RawFootage
-Day01
--A_Camera
---A001
---A002
--B_Camera
---B001
---B002
-Day02
--A_Camera ...

So then you can setup the destinations in the app like Drive1/RawFootage, Drive2/RawFootage, etc., and it will automatically place the files in the right place. To do it relatively simply, maybe just look at the first letter of the volume name.
 
I see people add the extra directories for each mag all the time and I don't get it? It actually makes more work to have extra directories.

My normal directory structure is similar, but does not contain the extra camera directory.

ProductionName
--Day_01
----RAW_FOOTAGE
------A001
------B001
------C001

----SOUND
------Roll_01

----LOOKS
------LUTs_CDLs_Files

----TRANSCODES
------all_the_days_transcoded_footage

----DAILIES
------all_the_days_h264_files

Just go in the RAW_FOOTAGE folder and select all the A cam footage if that is all you want. You operating system automatically sorts everything alphabetically. When I get mags that do not have a proper naming convention like A001_ I rename the mag to something useful like GoPro_01...GoPro_99.

I have thought about adding a few destination presets. So you can set up where Footage goes and Sound goes and quickly jump back and forth.

I have also thought about adding a feature to rename cards, so it is easy to change GoPro mags to GoPro_03, but I go back and forth. Some things should be a manual operation so the automagic doesn't screw something up.


Dusty
 
Dusty, I have been using and recommending your tool for mac - now I am searching for something similar on windows 7 - any tips?
best wishes and kind regards,
martin
 
You must log in or register to reply here.
Back
Top