QT Encode times

[David Battistella]

So far I am noticing that the new QT codec takes far longer to encode a prores file than it did in the previous version.

Is this the new speed of the QT codec or can we expect some of this to be optimized soon?

I am encoding footage shot on B15 and it is taking a VERY long time to encode 5 minutes of footage to Prorez through compressor.


David

 

[Christian Berg]

Jim said they are going to optimize the code in the coming weeks. So it should be faster.

 

[David Battistella]

Is it the REDCODE QT component that needs to optimized?

David

 

[Christian Berg]

I think it is the debayer process in the QT component. Some one from Red might fill in...

 

[David Battistella]

I think it is the debayer process in the QT component. Some one from Red might fill in...

I hope so because there is a serious slowdown with the new REDCODE QT component. I wonder if I can revert?

Any timeline on when the new Log and Transfer tool will be enabled? That was a very quick and efficient way to transcode footage. (which we need to do because editing with proxies is not quite realistic yet)

David

 

[Concave]

FYI, my homemade converter did 17fps converting R3D files to AVID XMF Wrapped DNxHD36 (1920x1080) It processed on the GPU, and did scaling and color correction (primary and secondary)

Now, after the file format changes in build 16 (RED have deliberately changed the file format to be unreadable by 3rd parties) I can't run it anymore, and I'm stuck with the close-to-unusable slowness and instability of REDCine.

Now, if RED would only release the SDK, one would have a chance to make usable RED tools, but they will probably be as slow as REDCine, so all is lost. I fear the only way to get back up to speed is to downgrade your camera to build 15 (Which they have made impossible, for no reason whatsoever) or for RED to rethink their strategy on the matter, and open up the file format.

Other vendors (AVID, Quantel, Discreet etc) have realized that they need a open platform for their products to be successful. Seems RED have the opposite opinion. (

Damn shame. I really LOVE the RED camera, it's just so perfect in every way. It makes my heart bleed to not be allowed to use it to its max potential.

 

[reality]

AVID, Quantel, and Discreet are open platforms?
Whoa, did I miss the bus?

 

[Deanan]

Concave, a few things to consider before you assume your tools are intentionally not working:

1. We made changes to the codec to improve quality and proceed with planned changes that have been in progress for 8 months.

2. The format has been in flux since build 1. Expecting your tools to remain functional is not realistic for tools based on hacking the R3D format.

3. Increasing boot times and general camera performance for build16 and later builds meant making fundamental driver changes to build 16 that made downgrading irreversible.

 

[Mike Prevette]

I have also noticed the speed drop in the new QT component. However I can still process and color correct footage (REDCINE) in half the time it takes me to do the same with 16mm or 35mm. So in my opinion it's still a net gain.

 

[Shawn Booth]

and I'm stuck with the close-to-unusable slowness and instability of REDCine.

I couldn't disagree more with this statement. I absolutely love this little app. I have not yet experienced "problems" with it. I wish it was developed more and actually cost to have/use it.

 

[Concave]

Concave, a few things to consider before you assume your tools are intentionally not working:

I don't say your tools are not working, just that after the changes in the file format, it's impossible for 3rd parties to decode r3d files. It was easy before, as the statements from various 3rd party developers testify. (Cineform among others) I belive you are deliberately making the format harder to read to stop 3rd party developers from reading the files, rather then for any practically reasons. I can see why you removed the jpeg2000 header from each frame, it should never have been there in the first place, but to change the codes in the codestream of the still jpeg2000 encoded data serves no purpose other then to stop 3rd party developers from reading the files.

1. We made changes to the codec to improve quality and proceed with planned changes that have been in progress for 8 months.

I believe you could have made the same changes, quality improvements and so on, and still use a readable standard format for the files. The differences lies not in the way each frame is compressed, but rather in the stuff you do with the files before doing the wavelet compression.

2. The format has been in flux since build 1. Expecting your tools to remain functional is not realistic for tools based on hacking the R3D format.

True, and I understand that. If stopping 3rd party developers from reading the files, why don't you expose the format, rather then releasing a sdk?
When does the sdk come out, and what will it allow us to do? I fear it'll be in the line of openr3d(); Getframe(Timecode); close(); Which will be a valid sdk, but will not allow the kind of flexibility 3rd party developers need.

3. Increasing boot times and general camera performance for build16 and later builds meant making fundamental driver changes to build 16 that made downgrading irreversible.

Ok, I can't argue with that.

Will the sdk be available to everybody or just a small set of companies? Will there be a source-code version of the or is it going to be win32/OSX only?

Do you have a sdk out in beta? - Can I join it ? :greedy:

Anyway, thanks for taking this discussion, I really want to get back to programming my red converter asap, so we can integrate RED properly into our workflow.

 

[Concave]

AVID, Quantel, and Discreet are open platforms?
Whoa, did I miss the bus?

Avid released the source code for the DNxHD codec on their webside for everybody to download.

Discreet released a sdk that allows you to access a framestore from your own software. The newer versions of flame can use data stored on a XFS disk.

I don't have access to any Quantel machines, but I believe they share data over network too.

All of the above used to be closed platforms.

 

[Concave]

I couldn't disagree more with this statement. I absolutely love this little app. I have not yet experienced "problems" with it. I wish it was developed more and actually cost to have/use it.

Here's some of the problem I have seen, just on top of my head:

Exporting 4K dpx from redcine, it crashes after 1000 frames, because it's out of memory.

There's problems with exporting stuff with a 25fps framerate, they usually comes out as 24fps.

Timecodes in exported Dnxhd files are the TOD timecode, not the usable one.

DPX files exported from redcine has wrong timecodes (The timecode is from the timebar in redcine, not from the clip)

Loading a EDL, exporting only needed material don't work on windows, and only using all kinds of hacked workflows on MAC

PC version don't load XML files.

DNxHD files has wrong timecodes, and are not XMF wrapped, so they need to be imported into AVID. XMF wrapped files just needs to be copied into the media disk of your AVID, and you can start editing it.

--- end of bug list


RedCine is fine if you need to export one clip to PAL targa, but to convert all the raw material for a feature film (20 hours+) to Dnxhd, or converting the 90 minutes needed for conforming a feature film to 4K DPX is next to impossible.

 

[Deanan]

I believe you could have made the same changes, quality improvements and so on, and still use a readable standard format for the files. The differences lies not in the way each frame is compressed, but rather in the stuff you do with the files before doing the wavelet compression.

Not True in the slightest. We have some incredible wavelet guys here and they're doing some excellent work to improve quality, efficency, and reliability of the encoding which is of course coupled to decoding. They are also further tied to future products and performance goals. We simply could not have made the improvements to quality and in camera performance/reliability without making the changes.

Will the sdk be available to everybody or just a small set of companies? Will there be a source-code version of the or is it going to be win32/OSX only?

Win/OSX/Linux for the SDK. It will be available to everyone. The SDK is not available in beta yet but when it's close to beta we'll announce more details.

 

[Graeme Nattress]

Deanan's right. We've not just changed things, but radically re-written how the compression works, right from the ground up, and as he says, got in some superb wavelet gurus to seriously re-write stuff. The B16 quality difference over B15 is not window dressing, but serious improvements that necessitated serious changes. Indeed, that's the "why" for the SDK, so that everyone can benefit these improvements we've made.

Graeme

 

[Concave]

Ok, maybe I was a bit too fast in judging you, but please, please release the SDK soon - my development has come to a complete hold with build 16, and I want to get back in business.

I understand it id daunting to write documentation, so just release it with a few samples, to get us started, and release the docs when they are ready.

 

[Anthony S. Pratt]

Ok, maybe I was a bit too fast in judging you, but please, please release the SDK soon - my development has come to a complete hold with build 16, and I want to get back in business.

I understand it id daunting to write documentation, so just release it with a few samples, to get us started, and release the docs when they are ready.

Feel your pain mate, and Build 16 has really made some serious changes to the world (most of them really REALLY good...). This is sort of like the discussion about Apple 'not supporting' RED users. It's a daunting task in a small market (sorry, but it's true - think of us as Ducatis, not so many of us but we're freakin' class - and we look pretty). I have seriously hacked workflows using both RED, 3rd party and custom tools all tied together with (insert proprietary magic here).

The SDK will come. Politics and commercial realities will bring some pressure to bear (and these will bring benefits to the code unseen by most of us), and it will be later than we think (but better because of it). This codec now looks like the foundation of an ecosytem I want to play in...

...bring it on, when it's ready, I can stand the wait.

Better tools to enable creative vision. Sweet.

Arrrrgh, too much red wine (no pun intended) and codeine, time for sleep...