Welcome to our community

Be a part of something great, join today!

Register Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

avid mxf red workflow

J

jamie parry

Well-known member
Joined
Mar 7, 2009
Messages
158
Reaction score
0
Points
0
Website
www.sumners.co.uk
www.jamieparry.demon.co.uk

any thoughts?
this is mac only but it does what i wanted metafuze to do on a mac!
i've got some work to do on it but it basically does what the editors wanted ie
mimic an offline-online workflow they already know
set and forget encoding for offline dnx36
metadata tracked in avid mxf (tape name= file name and clip timecode = original file timecode and project name is user definable)
online pull easy peasy from EDL of decomposed sequence and
re encode directly to higher res (DnxHD 185 in our case) of only bits required and handles
relink easy to higher res
done
any requests?
i'm working on doing a bit of a gui thing but dont know if its really necessary
and it now tidies up after itself (getting rid of mov files and raw dnx files) and will, if i can figure out a tiny bit more, copy/move the mxf files directly into the Avid Mediafiles/mxf/1 folder
cheers
jamie
 
very cool. a couple of questions.

how is the color of the clips determined? can your 'app' access the rlx files for each clip?

also, how do you decide on framing of the transcoded mxf files? what if you wanted them to be 16:9 letterbox vs. 4:3 centre cut, etc. etc.?
 
really nice workflow, well done. question? why did you need to make a 2nd sequence in the decomposed bin? the decomposed clips are already attached to that. just curious. otherwise well done. any place to download those scripts? :)
 
-chris the colour is determined in a redline command ( i believe that can access the metadata in a file) not sure about rlx-i'll have to look it up (or someone can tell me!)
framing similar- i've just made it output 1920x1080 "fit height" because that's what we usually want.
Jason the 2nd sequence is just me doing as i'm told from the editors! apparently if you do mixes and such, the EDL you make from the original sequence confuses the issue whereas making it from the new sequence with decomposed clips in guarantees an easy EDL for parsing that will get you all the media you want transcoded.
the scripts may be available soon after i've done a bit more work but they're no good without the unix command line apps listed. one of which I've written with a friend so i need his permission to post it and the others are open source so i'm not sure where i stand about putting them all together.
My other thought was just to give all i've done to the "monkey extract" boys and get them to send me $1 for every sale!!
cheers
j
 
It's definitely similar to doing the online using monkey extract, but I'm much more interested in the part where you created 'dailies' using... what was it ffmpeg? Huh? The problem with using redrushes, or compressor, or whatever, when generating DNX-codec inside a QT, is that you then have to import the QT into Avid, which converts it to mxf. Even if you get the 'fast import' to work, you're still making a second copy of the DNX material, which is only slightly faster than real time, so now you're up to about 5x realtime just to get dailies. That's where the advantage is with metafuze- (plus whatever they've added to it now) you don't have to import the dnx into avid.

Does this ffmpeg re-wrap the QT as MXF, or is it copying the media? Re-wrapping seems like it would be faster.

The EDL of the decomposed works really well - I can't imagine doing it the other way, even with a simple EDL.
 
Greg-there are 3 stages to the process
1 redline makes the quicktimes (dnxhd 36 in the offline stage).
2 ffmpeg unwraps the "essence" from the quicktime making a dnxhd file with no wrapper around it.
3 writeavidmxf wraps the dnxhd essence in an avid mxf container and inserts metadata.
The "ffmpeg" and "writeavidmxf" parts are "unwrap/rewrap" processes so they take hardly any time at all
and now the scripts tidy up after themselves by getting rid of the movs and dnxhd essence files so your storage requirements don't go crazy!

basically it means that as long as redline can write it and ffmpeg and writeavidmxf undestand it there are a few flavours to go back into the symphony/media composer with.
dvcproHD is one dnxhd is one and uncompressed is one.
There may be more combos that work but those 3 are the only ones my company would realistically be involved in.
Any 4K DPX style stuff and they'll have to buy me a resolve/baselight/nucoda which is about 50-100 times as expensive as a mac pro.....but that's another story

Chris -I agree it would be cool to have a GUI for it but you'll have to wait while i figure out how to do it!! (or persuade the monkey extract guys that what i have is of interest to them)
cheers
j
 
hey.....monkey extract guys! give this guy something to make this part of your program......would be very cool.

there.....i tried.

seriously. the more options to get to native MXF files for AVID, the better. i would have to say i like how metafuze is coming along, but a Mac version of the same would be welcome of course.

monkey boyz??
 
Chris Parker said:
hey.....monkey extract guys! give this guy something to make this part of your program......would be very cool.
Click to expand...

While the way Jamie has done this is quite effective and clever, the fact is that ffmpeg and the libMXF library (which contains writeavidmxf) are both open source that is freely available to anyone who wants to use them under the GNU GPL. ffmpeg is used by a number of vendors (including SpectSoft and others) to read and write Quicktime files (and do numerous other file format conversion operations) on Linux systems, and libMXF was written by the folks at BBC Research specifically to do what Jamie is doing - create Avid compatible MXF OP-Atom files.

Along those lines, though, I would ask Jamie: What do you do with audio? Do you split it out of the Quicktime you created and separately wrap the tracks? I say this because Avid only accepts OP-Atom files, which can only contain one track at a time. As a result, when you import into an Avid, separate MXF files are written for the video track and each audio track. I didn't see anything in the video regarding audio...
 
you could do split audio imports. as long as you add audio tracks to your files via "modify" (and after unlinking/change/relinking). as long as you audio and video files are the exact same length you can select your clips in the bin and go to "batch import" and match the audio to the video files and it will put the audio into the clips. its a nice sneaky way to do it.
also might need to make sure the audio is 24bit although im pretty sure AVID now accepts the 32bit pack and strips it down.
 
Jason Diamond said:
you could do split audio imports. as long as you add audio tracks to your files via "modify" (and after unlinking/change/relinking). as long as you audio and video files are the exact same length you can select your clips in the bin and go to "batch import" and match the audio to the video files and it will put the audio into the clips. its a nice sneaky way to do it.
also might need to make sure the audio is 24bit although im pretty sure AVID now accepts the 32bit pack and strips it down.
Click to expand...

All that is, of course, true, but the very reason to make MXF files directly is to avoid all of that in the first place. That's why I was curious as to what Jamie is doing for that. If you're going to use batch import, it's just as simple - if not simpler - to use the Red tools (like Rushes) to make DNxHD Quicktimes, with sound, make an ALE, and do a fast import from the Quicktime files - which is, frankly, the way I've done it. And in the end, it's probably just as fast, too. The nice part of Jamie's workflow is to use the scripts to to generate new clips for the high res version.
 
mmost- first of all i have to thank you for all I've learned from you over the years lurking on TIG and CML (I'm a colourist really not a coder...)
that out of the way
auuuudiooo hmmmmm..... like i say i'm a colourist so i hadn't thought about it!!!
the project i'm doing this for will be shooting separate sound that will be synced up in the avid by timecode so i haven't really had to worry about it.
I also haven't got any R3D files with audio in them!!
my plan would be....drum roll please....
make the redline command render a small QT wrapper with avid compatible audio in it
ffmpeg the audio out of that into a wav or raw pcm files (not sure about extracting audio from QT REFERENCE files- its easy to do with self contained ones)
writeavidmxf them into avid mxf audio with the commands that are in there (I know that works i've done that bit before)
put them into the avid/mxf/1 folder
i'm really sorry i can't say one way or the other until i get a file with audio!
if someone has one you could put it on a ftp site and i'll get it at work (we have a fast connection )
the video ends up with_v1.mxf at the end the audio with _a1.mxf at the the end and it links up in the avid.Like i say i've done that bit before OK.
doesn't redline do audio directly? that would help a lot!!

i've also got some bad news for you american types. Because i am rubbish and i've been writing it for me and a UK project it only works at 25fps so far.
this is due to to the bit that i've written with my friend (aditc) only being set to 25 so far.
by the way all that discussion about open source and stuff making my "application" worthless (in the monetary/monkey extract sense) is probably correct (although aditc is NOT available anywhere cos i looked on the internet for something that did what i wanted and...nothing which is why i rolled up my sleeves and entered the murky world of C++ with my friend who probably dreams in that language.)
so I'll just keep on working on it here and post my results as a science experiment!
cheers
j
 
Can you explain why the 185 material did not match the 36 in color space? It seemed the steps were the same, but the outcome was different.

Michael
 
MichaelP hello sir!
different redline command for the online transcode -I made it redlog and redspace in just so i could see whether i could grade it OK. that's all.
 
yes of course you are right however batch importing audio into a clip and via jamie's way seems waaaaay faster than doing the ALE fast import way REDTools.

either way we all win and thats the best part.
 
This is really cool but just an FYI to everyone, this is basically what metafuse does and it has a GUI already. :)
 
metafuze don't work (natively) on a mac and at the moment is non batchable (is that a word????)
i mean at the moment you can only do one clip at a time. that's better! and it doesn't do the pull script at the other end for making hi resolution versions of the material you need...but apart from that its the same, kinda, sorta....
 
You must log in or register to reply here.
Back
Top