Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Archive/Backup DIT question

T

Tommy Boulding

New member
Joined
Nov 23, 2012
Messages
2
Reaction score
0
Points
0
Hi All. First time post - hope it's in the right forum.

I'm after some friendly advice with regards to backing up/copying/DMG's of red mag rushes. I am currently a week into my first DIT job (normally an FCP editor but doing long time friends a favour). We are shooting a feature on Epic at 5k. I am working on a MacPro with RedRocket and a 2 port eSata card. I have two 4tb G-raid drives (labelled Raid_01A and Raid_01B) - more drives are standing by.

My workflow so far has been to make a copy of the entire red mag file directory into a corresponding folder on both Raid_01A and 01B. On top of that I have been making a DMG of the mag using Disk Utility onto Raid_01B as I wanted to be super cautious. I have then been opening the DMG (including the verify) and double checking the file sizes and that all the clips playback in RedCine X Pro. So currently I have 3 backups of the rushes over 2 drives. I have been transcoding to the internal HD.

My question: is it redundant to have a copy of the card folder and a DMG on the same drive (01B)? Can i delete the folder copy and keep the DMG safe in the knowledge that I have another copy of the card on the 01A drive? I am now getting to the point where my 01B drive is low on space and I would like to know if I can free some up or should move onto the next 4tb drive. I'm reluctant to do this if i can as there is still over 1tb spare on the 01A drive and I would like to keep them in pairs.

Just so you know - I do not and will not be getting R3D data manager at any point as the budget doesn't allow it.

Any feedback would be very much appreciated.
 
feature film. first job.. as a friendly favour... no budget for R3D data manager....

i really wanna help you, but it seems you've gone off the deep end already :( regardless, this forum offers tons of information so search and read as much as you can.

good luck.
 
Only time I typically ever do a DMG file is on alexa jobs, I got in the habit and don't know why really.

I just do a copy of the RDC foler via red data manager to 2 destination drives from my own Raid setup.
 
In cases where r3d data manager isn't available (budget is a pretty lame one - it's cheap and isn't there a free trial you can use for the duration of the show?), usually due to a new file format or something like that, there are md5 scripts you can run.

All R3D DM does - and well, I might add, it is worth the 75$ or whatever - is run an md5 checksum on every file in the folder after it copies it. Frankly, in many cases you are just as safe doing a get info on both root folders and comparing the byte counts and then taking a screenshot of the matching folders. I did this for a year until software came along to do an md5 for me.

I don't remember the name of the script I used to use but basically you can drop your folder onto it and it will spit out a list of md5 checksums. Do this for the copy and the original and store it with the footage for later accountability, and you're set.

Hopefully someone can give the name of a script to do this. Best of luck, it does sound like you're pretty far down the rabbit hole if you can't extract the cash for r3d DM but you've got a full mac pro setup with a rocket card.
 
Tommy Boulding said:
Hi All. First time post - hope it's in the right forum.

I'm after some friendly advice with regards to backing up/copying/DMG's of red mag rushes. I am currently a week into my first DIT job (normally an FCP editor but doing long time friends a favour). We are shooting a feature on Epic at 5k. I am working on a MacPro with RedRocket and a 2 port eSata card. I have two 4tb G-raid drives (labelled Raid_01A and Raid_01B) - more drives are standing by.

My workflow so far has been to make a copy of the entire red mag file directory into a corresponding folder on both Raid_01A and 01B. On top of that I have been making a DMG of the mag using Disk Utility onto Raid_01B as I wanted to be super cautious. I have then been opening the DMG (including the verify) and double checking the file sizes and that all the clips playback in RedCine X Pro. So currently I have 3 backups of the rushes over 2 drives. I have been transcoding to the internal HD.

My question: is it redundant to have a copy of the card folder and a DMG on the same drive (01B)? Can i delete the folder copy and keep the DMG safe in the knowledge that I have another copy of the card on the 01A drive? I am now getting to the point where my 01B drive is low on space and I would like to know if I can free some up or should move onto the next 4tb drive. I'm reluctant to do this if i can as there is still over 1tb spare on the 01A drive and I would like to keep them in pairs.

Just so you know - I do not and will not be getting R3D data manager at any point as the budget doesn't allow it.

Any feedback would be very much appreciated.
Click to expand...

For what's worth (and I'm new to the DIT game), I would put the DMG's on another drive entirely if your going to incorporate it into that workflow. That in my mind would be the safest and best use and spread the risk a bit more evenly.

Question - is the DMG part of his workflow something other's do?
 
Great stuff.

So i have downloaded the above script and R3D Data Manager trial and will have a play with them before using them proper. I have been doing what Dave said up until now - comparing byte counts on the get info panels and then i have been viewing all versions of the backup in RedCine pro and checking the clips all look ok.

I still would like to know what people think about whether or not I can delete one of the copies on my B drive? - either the DMG or the straight forward copy. I kind of think having both versions on one drive is redundant isn't it?

Thanks for the feedback. I realise I have dived in a bit quick and going forward I can adapt my workflow a bit to be more in line with the suggestions on here. I'll be honest i was reluctant to purchase Data Manager as this ideally will be my one and only DIT job. I was always pencilled into the edit but when the original DIT let them down at late notice I offered to help out.
 
Having both the DMG and the normal cop is probably redundant. Especially if your budget is so tight you can't afford R3D Datamanager!
I'd probably delete the DMG.

Generally good practice to only fill drives to 90% of capacity.

Although checking byte counts is better than nothing, it IS possible to have apparently correct byte count and still have a bad file. MD5 is slower to copy, but gives very good confidence that the copy is byte for byte identical to the camera media, and that both backup copies are also identical. Then you only need to playback check one of the backups for good picture and sound, to know both copies are good recordings.
 
RedMag and disk folder size different

RedMag and disk folder size different

Since someone suggested, that one does a finder copy and then compares the Red Epic Mag size vs the size of the copied folder on hard disk, I have an observation. On a shoot I was recently, I found the Epic mag size and copied folder varied by just a very small number of bytes or maybe kilobytes. But the copy was perfect. Is this a filesystem thing? Has anyone else observed this?

Or maybe one should not compare Epic mag volume size with the copied folder size. But compare .RDM folder on mag with .RDM folder on hard drive?

Neil Sadwelkar
 
Having two copies (an RDC folder and DMG) on the same drive is redundant, wastes space, and time consuming. If the drive with two copies fails on the hardware side, you've just lost 2 of your 3 backups. If my data isn't on three drives, it isn't safe.

Rant:
For the money production spends on two 4TB g-raids ($800 from apple), you could buy four 3TB Seagte Backup Plus drives ($760). Plus, with the extra money they can buy you Red Data Manager (not that you need it with the scripts or free trial, but I consider it a must have if you plan on doing more DIT work.

G-Raids are slightly faster and a couple of percentage points more reliable, but Raid-0 is inherently unstable, doubling your chance of drive failure. I just finished a feature where we had 15x 3TB Seagate Backup plus drives and 6x Seagate Barracuda internal drives with no failure or issue.
 
Last edited:
Do you have time to scrub clips before wiping mags? do you have time for full playbacks?

red data manager or shotput pro are cheap ways to copy that are better then doing a drag and drop...

disk image backs up, when corrupted, lose it ALL. as opposed to copying the file structure and potentially only corrupting a single clip. therefore i don't bother with disk image copies...only backups to 2 raids, and take one off site constantly. have red tape and green tape to put on the mags to be methodical...

if you check youtube theres some good DIT how to videos
 
Neil Sadwelkar said:
Since someone suggested, that one does a finder copy and then compares the Red Epic Mag size vs the size of the copied folder on hard disk, I have an observation. On a shoot I was recently, I found the Epic mag size and copied folder varied by just a very small number of bytes or maybe kilobytes. But the copy was perfect. Is this a filesystem thing? Has anyone else observed this?

Or maybe one should not compare Epic mag volume size with the copied folder size. But compare .RDM folder on mag with .RDM folder on hard drive?

Neil Sadwelkar
Click to expand...

Number of BYTES should always be identical, but kilobytes/megabytes/gigabytes/etc may vary due to file system. Also, if you don't compare straight away after the copying, other programmes you open the files with may append files (sometimes hidden files) or alter FOLDER sizes.
 
You must log in or register to reply here.
Back
Top