Project Management Issues

[Eric Wescott]

I have a completed project I'm archiving and I'm attempting to organize it a bit because it's quite a mess...

1) I have multiple .rcx files. Is there a simple way I can merge them into a single file?

2) I want to change where files are located on the hard drive itself so more sanely organized as right now it's a god awful mess. I can easily add a search to find moved footage in RedCine-X but it doesn't do it for the attached audio clips. How can I properly reconnect the audio that is no longer located in the same place on the drive? If I can't do it in RedCine-X itself... I noticed the .rcx is just a zipped up tar file and contains an XML. Is it safe to just modify the path of the XML file and zip/tar the folder up again or is it more complicated than that?

3) I have three backup copies for the entire project. I wanted to ensure all the footage matched 100% bit for bit. However, to save time I decideded to do md5sums instead of binary diff. Then I can just archive a text file will all the md5s anyway... I only did this on 2 of the 3 drives so far for all the original raw .R3D files but some of the md5s do not all match but the file sizes are identical according to finder. I know Adobe After Effects likes to update meta data when you import a clip.. I also know for a fact that some of the visual effects were done via importing the .r3ds into AE directly and not ProRes footage. So could this mismatches be just meta data? Any suggestions on how I can test this theory?

Thanks
Eric

 

[Matthew Yaeger]

1) No there is no RCX based simple way to do this.

2) I don't advise changing the contents of the XML or RMD files due to the likely hood of accidents, however it is possible and currently is the only way to "re-link" external audio clips which have had manually syncing done. Note if your audio clips have not had manually syncing done then you can just re-link them in RCX one at a time.

3) Nothing overwrites R3D files and all metadata changes are stored in sidecar files like the RMD file. An MD5 difference on an R3D file would then mean that some data was corrupted in the copies or an issue with the MD5 checking occurred.

 

[Eric Wescott]

Just to make a few things clear... This was a student production and our film program is very experimental so the majority of projects are very small crews and sometimes just one person. I was a member of a student organization that tries to do larger scale projects and provide students with more real world experience in film making. We've done some larger scale projects in the past but this is the first time we raised any real for and use "professional" equipment. It was pretty much everybody's first experience working with the RED and any "real world" workflows. So, attempt and building a workflow had it share of problems.

1) Thanks and bummer Guess I'll just have to rebuild the projects myself. Since this project will be passed down to future members of the organization to play with I think I'll be worth the time to properly organize it into a single RCX file.

2) I'm curious... How do you handle archiving projects on multiple drives. The reason I'm in this mess is because each drive was named exactly the same and finder showed up as the same drive but the OS has to mount it to a unique path. On our shoot the DIT was given a card of footage and he copied it onto our two backup drives. As he was doing this he was also manually syncing audio (as we didn't have the ability to jam timecode until almost half way into our shoot) and he accidentally linked some audio from different drives but in finder they were the same "name"

So
/Volumes/ProjectNameDrive/ProjectName/Audio/Day1/S01T01.wav
and
/Volumes/ProjectNameDrive1/ProjectName/Audio/Day/1/S01T02.wav

While both drives had the same contents when attaching the audio in the RCX project it was accidently flipped between both drives throughout the shoot.

So to use the current RCX we have to plug in multiple archive drives so the OS renames one.. I suppose we could mount the same drive twice just to different places as well but that's not exactly ideal either...
For this project I think I'm going to take the time to rebuild the RCX in a single file anyway so modifying the XML for the path isn't worth it.

Why doesn't it work on relative paths based on the location of the RCX file instead a path? It would be so much nicer! Also, in the future if we wanted to consolidate projects onto larger backup drives this could be a nightmare. I guess that just never happens in the real world? I have this same compliant in FCP XML files as well. Makes sharing projects with other people tricking if they name their hard drive different and put the root project folder in a different location...

3) This concerns me greater... We have two drives we originally stored the raw footage on and since then it's been transferred two several other work drives. I only did md5s on one original backup and one workdrive. I'd say like 50%-75% MD5s matched. I just hope when I do it on both of our original backups they match. If they don't.... Do you have any suggestions as how I should pick one file as "good" over another? Is there any internal CRC check I can perform on a R3D file to ensure it's not corrupt?

I should have pushed for our DIT to do md5s (or something like it) on location... Is this something done in the real world? If so can you maybe elaborate on workflow or point me in the direction of some resources?

Thanks again for your help!
Eric

 

[Marc Wielage]

It's important to come up with a workflow methodology before you start shooting, not afterwards. Using tools like R3D Data Manager and DoubleData (both made by by the R3Data company in California) is very important and helpful to avoid disasters like losing links to footage and not having good backups.

To me, a better use of RedCineX Pro is just to create viewable files that work for dailies, create low-res files for your editing program, sync up the sound within the editing program, edit the show, and then relink and conform the highres R3D files (or DPX files if that works better) for the final color-correction and output. If it's a student project, I wouldn't be consumed with keeping the whole thing in 4K; it'll work fine in HD, assuming the exposure, lighting, and creative content are good.

 

[Eric Wescott]

The project itself is completed and we did develop a workflow prior to shooting. This project was are first attempt working with the RED workflow so some things could have been done better with more experience. Some problems you just don't run into until they happen and you learn to deal with them. In addition to a BFA in Film I have a BS in Computer Science so a lot of this workflow stuff just comes naturally to me. However, most other film students I work with just don't like spending time thinking about it and don't have the skill set to really fully understand it's importance and subtleties. I do my best to teach what I can but it really is complicated and more involved than just having good organizational skills.

I'm attempting to personally correct some of mistakes we made now. The reason for doing this is because the film is not owned by individuals it's owned by a student organization. So while I was one of the four students who produced the film, ownership and the media itself will remain with the student organization and not me. I personally could plug in the drive and rebuild the project from raw footage to final render quite easily. However, for somebody else it might not be obvious how to do it even if they have experience with RED workflow. Which most students do not... even those involved with on this project only a small few had direct contact with all the subtleties of the RED workflow.

The film itself is in the festival circuit now but I don't know what the future holds for it. It might just sit on a hard drive and never touched again or it might be made public to all students to just play with as they see fit. It will be up to future officers of the club to decide and will change over time. So, my goal is just to make it as easy as possible for them to do what they want with it. So, I'm attempting to simplify everything I can for them and document the process. So if they decide to shoot another project with a more complicated workflow in the future they have something to reference.

While I don't expect them to ever have a need to return to working from the raw footage I would prefer to give them the option and make it as easy as possible to do so. In doing this I've also hope to gain more experience about the finer points of the workflow myself. Not only because it's valuable to know but I am just naturally interested in workflow and the choices people make in developing theirs and the limitations of the software used.

Long story short is I think this film is unique in that it probably won't ever be "complete" and will be revisited by students and used as a tool to learn from. So the workflow needs to constantly evolve based on how this project will be used in the future. I'm just trying to make that process as simple as possible for future students as in my experience workflow is not something most film makers enjoy thinking about. So, if I can give them some good "best practices" on this reference project it will be well worth it.

 

[Eric Wescott]

To me, a better use of RedCineX Pro is just to create viewable files that work for dailies, create low-res files for your editing program, sync up the sound within the editing program, edit the show, and then relink and conform the highres R3D files (or DPX files if that works better) for the final color-correction and output. If it's a student project, I wouldn't be consumed with keeping the whole thing in 4K; it'll work fine in HD, assuming the exposure, lighting, and creative content are good.

We used RCX to sync the footage and the reason why is because while you can do it in FCP it's actually faster and easy to do in RCX. Especially once we started jamming timecode to the camera itself and then it became completely automated. FCP has no ability that I can find to automate syncing and even if they have the same time code you manually have to select each clip pair one by one and perform a "join" which is just tedious. Since we had to use RCX to produce ProRes and ProRes-Proxy footage so we could even start working on it in FCP it didn't make sense to sync outside of RCX because it was actually faster and easier than doing it in FCP.

Had I known that RCX has it's own set of issues reconnecting media I would have probably just bit the bullet and synced it all in FCP since it's much easier/safer to reconnect media in FCP. I can also safely export an XML and modify it manually in FCP where I haven't been successful (and discouraged from doing so) in RCX.

Thanks for pointing out the software but we were unaware of this specific product at the time we shot this film (a year ago) and probably couldn't afford to purchase it anyway... However, reading the description we essentially did everything it does (just manually) with the exception of running a bit for bit comparison after every copy. It was done at random but not after every single card was transferred as time constraints didn't allow it and the computer hardware we had access to on set was our own personal computers.

We could not afford to hire a professional DIT with faster hardware that would allow us to perform bit for bit comparison/error detection every time. The person we had in charge of this process is VERY tech savvy and did a fine job. We just didn't have all the resources a professional crew probably should have. It was a risk we were willing to take on this production but I would never allow it to happen if we were working for a client or something of that nature.

 

[Marc Wielage]

We used RCX to sync the footage and the reason why is because while you can do it in FCP it's actually faster and easy to do in RCX. ...Had I known that RCX has it's own set of issues reconnecting media I would have probably just bit the bullet and synced it all in FCP since it's much easier/safer to reconnect media in FCP.

Yep. Just about every Hollywood movie and TV show ever made is done exactly as I describe, either by the assistant editor or in a dailies process (onset or right afterwards).

Thanks for pointing out the software but we were unaware of this specific product at the time we shot this film (a year ago)... We could not afford to hire a professional DIT with faster hardware that would allow us to perform bit for bit comparison/error detection every time... We just didn't have all the resources a professional crew probably should have.

A lot of this kind of boils down to: "we didn't know what we were doing, we faked it, and made lots of mistakes because nobody told us how to do it in the first place."

Most of that could have been answered -- for free -- by simply reading the messages here, on Final Cut Pro user groups, and other vast resources on the net. It saddens me to see the amount of misinformation and sheer lack of knowledge, especially by students, who are so ignorant that they don't know what they don't know. Good resources, like the post workshop held by Createasphere, or any number of seminars all over the place, get into this stuff in detail. I have seen crews who at least make an effort to do things the right way, even when budgets are tight and schedules are very stressful, just by asking the right questions and being prepared long before they shoot, and weeks before they edit.

I've also seen very sad post disasters over the past few decades where people who should've known better didn't ask the right questions, leading to very costly mistakes and bad decisions. It's sad when that happens. In my years at post houses, none of us liked having to make those phone calls having to explain to the client why their $2000 job was now going to actually cost $10,000 or even $20,000, all because they didn't know, they didn't ask, and everything was done wrong. Or worse, their footage was lost or corrupted. I could write a book (and I might just do that, someday).