Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

How to get Scarlet 4K HDRX at 24 fps ... my proposed method

Les Dittert

Les Dittert

Well-known member
Joined
Sep 29, 2011
Messages
1,543
Reaction score
0
Points
0
Location
Los Angeles
I'm a big fan of high dynamic range capture and processing. As we all know, right now the 4K HDRX is not there on Scarlet.
I think I have a solution that is a slight modification of the scheme that Red currently is using.
This method relieves the compression overhead of the X track by only compressing the actual highlights that are
useful in the subsequent processing ( blending or toning ) of the two tracks.

It has been indicated that the processing limits the HDRX frame rates. Understandable as it has to compress double the number of frames!
If the code is allowed to skip over a threshold for localized brightness, much of a frame does not have to be compressed at all. In a crude sense this is a form of compression in itself ... a rather harsh quantizer, as it were...

Complications:
The scene turns into almost all 'bright stuff' that overload the X track compression anyway. Possible solution is to reduce compression of the A track as well, via a similar RLE or skipping strategy. After all, you only really need image from one track or another. Overlap a little for transition areas.

The simple 'blend' RCX HDR fails. Yes, that would have to be updated for this new mode. Perhaps to true high bit depth HDR that still guys use.



I worked up a little video to help show this. The above may make more sense after seeing the video. The result at the end was actually processed from the reduced compression requirement X track with a normal A track. You really don't need those dark X track pixels anyway! So why compress them ?

Video: http://dl.dropbox.com/u/61349012/demo_HDR_method.mp4

Comments are welcome!
 
Dood, you'z smart! Maybe Red will hire you as a freelance programmer to make this work. What a great idea. I'm sure there's some sort of reason why this hasn't been done. But who knows, not everyone can have every idea all at once!
 
Red was to have 150fps 3k on the fixed, how come they can't manage a 24fps hdr mode (please nite this is not getting into why there leading sensor can't do normal, simpler and superior, multi-slope or other from the 1990's, they use in $70 HD video camera). I give merit to your scheme les, but why can't Red store a 48fps 4k stream and integrate in post, a lot less data than 3k at 120fps even. You can even use the overlap between the track to form 48fps new hdr track in post. The software will have to fill in missing 24fps frame data in the highlights and dark areas, which should be very possible with modern software. Now we are close to 50p broadcast, another bonus if it records at that frame rate.

On what you say Les, yes it should easily be able to clip off the top or bottom useful data and integrate it with the other track on the fly to have just one 24fps track compressed and recorded (if need be) with clip point per frame if need be. It will still have to have the offsetted clip data realigned through software. Looking at this, and the way comoressiin works, a seperate track maybe preferable, but compression still gets stuffed around with table top curves, you could just define the curve to be conveniently rounded, and replace during integration, but loss some of the efficiency, but if it is maxed out anyway you would still be lossing it. The issue is, is red code capable of a 16-24bit pixel? If so that is the way to go, as 16bit is the extra precision of the curve which is seperate from the amount of compressed data.
 
Get a proper home :smilielol5:
 
Wayne,
I'm proposing this as a short term solution. Probably less than 500 lines of code.
Your other ideas are interesting, but I have a 5 micron sensor in my cameras that has no well capacity to do heavy HDR images without multi exposures.
Let's keep this thread for more 'easy to get in 8 weeks' & 'with current hardware' type of discussions.
Thanks.
 
Great idea! A question about RED integrating these inputs: Is RED usually responding to that kind of proposals/ideas and if so, how long can it take to integrate (i know that kind of a "when-wil-my-scarlet-ship" question ;-) ... but still. In the Epic Forum i see a lot of feature requests-posts and i've never seen a single one of these integrated into the camera build yet.
 
Wayne Morellini said:
Red was to have 150fps 3k on the fixed, how come they can't manage a 24fps hdr mode (please nite this is not getting into why there leading sensor can't do normal, simpler and superior, multi-slope or other from the 1990's, they use in $70 HD video camera). I give merit to your scheme les, but why can't Red store a 48fps 4k stream and integrate in post, a lot less data than 3k at 120fps even. You can even use the overlap between the track to form 48fps new hdr track in post. The software will have to fill in missing 24fps frame data in the highlights and dark areas, which should be very possible with modern software. Now we are close to 50p broadcast, another bonus if it records at that frame rate.

On what you say Les, yes it should easily be able to clip off the top or bottom useful data and integrate it with the other track on the fly to have just one 24fps track compressed and recorded (if need be) with clip point per frame if need be. It will still have to have the offsetted clip data realigned through software. Looking at this, and the way comoressiin works, a seperate track maybe preferable, but compression still gets stuffed around with table too curves, you could just define the curve to be conveniently rounded, and replace during integration, but loss some of the efficiency, but if it is maxed out anyway you would still be lossing it. The issue is, is red code capable of a 16-24bit pixel? If so that is the way to go, as 16bit is the extra precision of the curve which is seperate from the amount of compressed data.
Click to expand...

25 fps instead of the 24 and 50 fps instead of the 48 would be enough for me for the beginnning ;-))) but your idea sound's great...
 
Sascha Rossier said:
Great idea! A question about RED integrating these inputs: Is RED usually responding to that kind of proposals/ideas and if so, how long can it take to integrate (i know that kind of a "when-wil-my-scarlet-ship" question ;-) ... but still. In the Epic Forum i see a lot of feature requests-posts and i've never seen a single one of these integrated into the camera build yet.
Click to expand...

We are talking about one of them, it took a few years, and I have had several integrated into Red products, except the fixed. Look through the thread history, I was one of the first to independently suggest the multi-exposure her scheme as a simple firmware fix due to even cheap picket cameras having her with 16 stop plus latitude. But my suggestion to back to back the exposures, leaving the only sizable shutter gap between sets of exposures, so that movement difference would be miminised (instead of this compromise scheme that adds sizable gaps that have to be integrated). They are not gods you know. The fixed redone dynamic is very much like a market segregation strategy I suggested around 2004 either in the original anonymous Red camera suggestion thread or the original DIY digital cinema project thread Jim followed, and the original scarlet seems very much like the compromised camera I put forward there. About the pocket camera idea that became the scarlet, I was on Jim about the pocket camera around 2006, maybe earlier, I had an idea that I referred to as The Director's Veiwfinder. They claim big things shortly, I hope so, they are light years behind so far. Without the feedback of many other people aswell, Red would not be the success it is today.
 
Norbert, in the spirit of co-operation, thanks :-)

Les, they are simple, even 8 weeks update simple, using current hardware.
 
Wayne Morellini said:
We are talking about one of them, it took a few years, and I have had several integrated into Red products, except the fixed. Look through the thread history, I was one of the first to independently suggest the multi-exposure her scheme as a simple firmware fix due to even cheap picket cameras having her with 16 stop plus latitude. But my suggestion to back to back the exposures, leaving the only sizable shutter gap between sets of exposures, so that movement difference would be miminised (instead of this compromise scheme that adds sizable gaps that have to be integrated). They are not gods you know. The fixed redone dynamic is very much like a market segregation strategy I suggested around 2004 either in the original anonymous Red camera suggestion thread or the original DIY digital cinema project thread Jim followed, and the original scarlet seems very much like the compromised camera I put forward there. About the pocket camera idea that became the scarlet, I was on Jim about the pocket camera around 2006, maybe earlier, I had an idea that I referred to as The Director's Veiwfinder. They claim big things shortly, I hope so, they are light years behind so far. Without the feedback of many other people aswell, Red would not be the success it is today.
Click to expand...

Hej Wayne ... thanx for the clarification. This makes my Scralet shine even brighter. To know, that inputs from users (an geniuses like you ;-) get implemented makes me look forward to the future of our cams. Thanx again for inputing stuff, that makes my everyday-shooting
so much easier and my footage look so much better !
 
Last edited:
The funny thing is that it was a silicon imaging suggestion that their chief engineer asked me about around 2004, which I immediately pointed out the movement problem. I had forgotten (because it was a crappy idea at the time) and think I came up with the same solution after analyzing how to give Red hdr, but at that stage I factored down the movement problem with back to back exposures, which is the best cheap solution (if the readout delay is short enough, but any significant reductions would help the software reconstruction anyway but maybe negligibly). Software and computing power has moved on since then allowing red patch up the images. I love that my meta information track idea also took, allowing you to adjust post, and suggest at filming (I forget if Red is using it like that, I suggested around 2004 originally, before Red).


Don't worry, they are not listening to me much now either. Look at Kodak, the only good products lately have been products series I contributed suggestions into on another forum, far away (I've done this a fair bit), actually Sanyo and aiptek in that sector at a time too. This feedback from users is why those companies had the best cameras in that sector, and beautiful products at that. Look what happened to the rest of it (Kodak). They are wasting their time closing the video camera business, hopefully their manufacturer is where the design expertise lays, or they sell their unit on, and stick down to three product lines.
 
I didn't have time to read through the entire thread, but I think you've posted a similar proposal in the past...

With that in mind, the only problem I see is that you're assuming it's computationally easier to 'delete' those bits/pixels below a specific IRE than it is to just record them. In actuality, with the way RAW capture works, this method could potentially require MORE horsepower to do (even though all you're doing is telling the camera "below 50ire = null"). Again, bare in mind I'm looking at it as simplistically as possible, but the Scarlet would have to read/check each and every one of it's 4k pixels to make sure they're below the given IRE before discarding it... With the way RAW works, it'd probably be as easy (maybe easier) to just record it, without having to do said IRE check of each pixel.

Also, the CPU may not be geared for this kind of stuff; RAW is actually easier to process/record than a compressed format like h264 (which actually requires CPU cycles for the compression process.)

Lastly, I think it's safer to assume that RED engineers are fairly smart and have already considered every option (including this one, if it is indeed easier on the camera). At this point, 4k HDRx on the Scarlet is still in-the-works, but RED obviously has bigger fish to fry (like Dragon, getting out of back-order, RCX updates, enabling phantom-power, etc.)
 
Mike,

You make some valid points, but to say that any engineer has considered every option for anything is a little much, IMHO. There are certainly ideas that haven't been considered. And of course, I wager that almost all of the ideas floated here on Reduser are not practical.

Here's my $.02. I think the unanswered ? is "Is it the speed at which the sensor can be read or the speed at which the data can be compressed that causes the bottleneck?" If it's the speed at which the sensor can be read, then you're ? of if on-the-fly dynamically ignoring certain pixels is possible and will it speed up the read time is spot on.

If, on the other hand, it's caused by the ability to compress two 4K streams fast enough, then Les' idea is probably a lot easier to implement if Redcode is a constant quality codec as opposed to a constant bit rate one. I don't know the answer to this. The easiest way to find out would be to record just black with the lens cap on and see if the file size is less than it would be when shooting a normal scene. If that's the case then, it seems that creating as much black in the scene has a benefit.
 
Peter Moretti said:
Mike,

You make some valid points, but to say that any engineer has considered every option for anything is a little much, IMHO. There are certainly ideas that haven't been considered. And of course, I wager that almost all of the ideas floated here on Reduser are not practical.

Here's my $.02. I think unanswered ? is "Is it the speed at which the sensor can be read or the speed at which the data can be compressed that causes the bottleneck?" If it's the speed at which the sensor can be read, then you're ? of if on-the-fly dynamically ignoring certain pixels is possible and will it speed up the read time is spot on.

If, on the other hand, it's caused by the ability to compress two 4K streams fast enough, then Les' idea is probably a lot easier to implement if Redcode is a constant quality codec as opposed to a constant bit rate one. I don't know the answer to this. The easiest way to find out would be to record just black with the lens cap on and see if the file size is less than it would be when shooting a normal scene. If that's the case then, it seems that creating as much black in the scene has a benefit.
Click to expand...

Touche... In a round about way that was kind of my point, too: we don't know... RED obviously knows more about the camera's hardware/capabilities than we do, so they'd be in a better position to know why it can/can't be done (using a method like this or another). We, on the other hand, are just spitballin', and I'd be more inclined to assume they know how to get the most out of the camera then assume dark data can be discarded on the fly. Also, with this idea, I can't imagine it not being thought of by RED... Even for Epic (which has horsepower to spare), as it'd save substantially on REDmag space.

As mentioned, I'm trying to dumb it down to a really basic flow and comparing from there... for example, 'sensor sees light and sends data to CPU, data is converted to .r3d by CPU, data/.r3d is stored on REDmag' (in other words, three basic steps) instead of, "sensor sees light and sends data to CPU, data checked by CPU to see if it's below a given IRE, data is converted to .r3d by CPU, data/.r3d is stored to REDmag (which is four basic steps.) That additional step, even if it's easy, causes latency and an additional layer of complexity even if it's a relatively simple computational process...

Let me re-phase again, just so I don't look like a complete tool - in my opinion, I don't think the dark data can be thrown away "on-the-fly." More over, from my understanding, having an additional layer in chain, even an easy one, would actually be harder on the CPU than just dumping the RAW data to storage.

OH and I think solid black frames (with lens cap or otherwise) are the same size... If they're not, then I agree, it makes implementing this method more plausible (although, I'm not familiar with r3d raw compression). Actually, if a black frame is in fact smaller, than it essentially already does what this method suggests (throws away similar looking data).
 
Les, I haven't the vaguest idea if RED can do what you are suggesting, but it does give me hope that they can come up with some way to get HDRX out of the Scarlet at better frame rates!!
 
Mike: Skipping past pixels darker than a certain value is easy. You are basically doing a simple 'if' statement during compression and skipping the whole wavelet encoding complexity entirely ( the hard part that takes many many cycles ).
In my simulation video I did this in a block manner which may be closer to the compression engines data chunk architecture. I'm only guessing on that.
It is possible that the way compression is implemented would make such a branch hard to insert in the data pipeline. Without knowing this, I just wanted to throw out this method as a possibility.
Even if the compression core is hard to change, it certainly would be a shortcut for it to compress 'zero black' instead of the usual noisy image data.
 
You must log in or register to reply here.
Back
Top