2K Scaling - A thought about datarates

[Seth Larney]

Ok, this is somehing I've been thinking of for some time now.

I am wondering, what benifit compression-wise we might get from the added datarate per frame of 2k scaled considering the write limit of RED's current onboard recording media (28-ish Mb/s) vs the loss (if any) in quality of downscaling in-camera as opposed to in post.

Or, in other words :

Considering that we are compressing less pixels per frame for the same bandwidth with 2k (we know this works out with bayer compression because 2k cropped @ 24fps is less bandwidth than 4k), will that benefit in picture quality outweigh any possible LOSS is quality from capturing in 2k vs 4k ?

If we capture in 2k scaled, are we getting the same oversampling benefits as if we captured in 4k and then downscaled ? I'm hoping yes..

I find this as an EXTREMELY important question as while REDCODE is a phenomenal compression algorithm by all accounts, and while it is indeed 'magic' I'm sure there wil be situations where extra bandwidth will help (as has been discussed already). Also, we will perform a 2k uncompressed DI in our pipeline, and I will be wanting (as will many people) the BEST possible picture to feed that pipeline for a 2k finish.

Lets ignore the benefits of cropping in/pan'n scan for now ..

<EDIT> Case in point : You are shooting a film and you somehow are totally confident that your framing is perfect, you then record in 2k scaled. Providing that the onboard scaling provides you a picture identical to if you had a 4k image and scaled in post. Then by the time the image hits the compression algorithm it has less pixels to compress but already has the benifits of oversampling applied, theoretically you could get less compression artifacts due to the higher datarate vs resolution, right ?

Could it be possible that 2k scaled could yield a better picture when all is said and done for SOME projects ?

This is not a hassle for 2k scaled etc, I'm just interested in this from an academic standpoint as I know there are people on here that know much more about bayer compression than myself. If this feature eventually comes from RED, then it will be in due time. I have heard people argue for 2k scaled because of the benefits of lower recording space required, but for cinema application where storage is not so much of an issue, I would personally rather the best image possible from the hardware.

<EDIT> Disclaimer : I think the current image that the RED is providing, rocks the proverbial CASBAR. This is not an argument that this feature is NEEDED, its just theory. RED is already providing us with a tool that is enabling us to tell stories with a production value that has until now, been out of reach for many of us. Thanks RED !

Cheers,
Seth.

 

[ChrisLyon]

It will produce a more 35mm like image if 2k scaled were an option. Looking at the previous posts, the benefits are clear. It all depends on what you intend to yield from your project.

 

[Seth Larney]

Sorry Chris, I think you misunderstood.

I realize that 2k scaled will give you identical DOF and all other characteristics that shooting in 4k gives due to the s35 format size, which is a huge part of why many of us will choose to shoot 4k vs 2k CROPPED currently.

What I'm asking is purely about compression, and whether the lower resolution going through the compression will result in cleaner images vs recording in 4k.

Cheers,
Seth.

 

[ChrisLyon]

I would think that the answer would be no. But what do you mean by "Cleaner images." Certainly 4k would have more resolution, but I'm not clear on what "cleaner" is to you.

 

[Seth Larney]

Sorry, by 'cleaner' I'm purely talking about compression artifacts (if any are present by shooting 4k for instance).

Cheers,
Seth.

 

[ChrisLyon]

Right now there are some questions about artifacts- particularly on the blue channel. According to the Red Team, things are still changing. Maybe one will chime in.

 

[sander kamp]

You have to remember that 2k scaled is still RAW. That means the theoretical sharpness of the image is about 70% of a scaled 2k RGB image. So it seams unlikely to me that even with a higher data rate, the 2k scaled RAW image would be of higher quality than a 4k image that is scaled in post to 2k.

There has been talks in the past about offering in-camera scaled 2k RGB option. However, the datarate of RGB images is 3 times that of RAW (3 colors per pixel instead of 1) so the data rate is going to be much higher as well. But since that is a scaled version of the 4k image it should be of higher quality than the 2k RAW scaled picture, which might come close to what you are asking.

 

[Stokestack]

Seth, I've been wondering the same thing. I've asked exactly the same question, but lots of people misinterpret it. I think it comes down to: Nobody knows yet. Because while there may be fewer artifacts if you use the same data rate at 2K, the artifacts at 4K could be smaller and thus blended away with the resize down to 2K.

Sander's point is excellent, though: If you scale the full image down to "2K", you're really scaling it down quite a bit more than that, because you're storing it as Bayered raw. To really get quality equivalent to the full sensor scaled to 2K RGB in post, they'd need to scale to something bigger than 2K raw.

Another issue is that so far, no one from Red has talked about what the true hardware datarate limitations are, and whether we could hope for adjustable rates that we can maximize for a given combo of FPS and storage medium.

The question is: If the recording medium has limitless speed, what's the highest bitrate at which the camera can encode 4K Redcode at 24 FPS?

 

[Miguel "Macgregor" De Olaso]

Maybe you could try.

Compress a 4K image into a 1.2 MB JPEG2000.

Compress a 2K image into a 1.2 MB JPEG2000.

Resize the 4K JPEG2000 to 2K. Compare to the 2K JPEG2000.

Tell us.

Note: it won´t be the same as compressing RAW data, but you can spend 5 minutes enjoying the experiment. XD

 

[Clint Johnson]

I would think that if you used the full datarate of the 4k with the scaled 2k you would go from a (numbers pulled from a hat strictly for example purposes) 24:1 compression to a 6:1 compression and couldn't help but get fewer compression artifacts.

Capturing at 2.25k scaled would probably give a good compromise for the bayered raw to resolve at 2k while still using far less compression at the maximum datarate.

I don't even have my Red yet (maybe on January 14th for my birthday?) and
I'm just speculating, but this would probably be so small an increase as compared to 4k scaled that it would probably have limited practical benefits. That said, there are those who would be happy with any improvement no matter how small... and if you put ten small improvements together they just might add up to a significant improvement eh?

 

[Kevin Halverson]

As long as the rescale is an integer resize, I would think that any compression artifacts would be fractionally reduced when maintaining the same data rates. I agree, 2k scaled at the current data rate sounds like an interesting idea, but I would also be interested in 2k scaled at the same data rate as 2k windowed.

 

[Darren Orange]

I am pretty sure that Jim has said that this was being addressed/fixed with the Red One update to hardware. The Red One "X" model if you will, they changed/improved a lot such as the DSP if I recall correctly. In regards to Channel Artifacts...

 

[Tom Lowe]

I think the main benefit of resampling to 2K would be less noise.

 

[Seth Larney]

I would think that if you used the full datarate of the 4k with the scaled 2k you would go from a (numbers pulled from a hat strictly for example purposes) 24:1 compression to a 6:1 compression and couldn't help but get fewer compression artifacts.

These are my thoughts as well.

Capturing at 2.25k scaled would probably give a good compromise for the bayered raw to resolve at 2k while still using far less compression at the maximum datarate.

This is an interesting idea, although I'm not sure how the scaling would go from 2.4 to 2k ? Whether that would introduce softening that would counter any benifits or not ?

I don't even have my Red yet (maybe on January 14th for my birthday?) and
I'm just speculating, but this would probably be so small an increase as compared to 4k scaled that it would probably have limited practical benefits.

I guess this depends on how much of an improvent Red can get out of the blue channel compression we have been seeing ? They have surprised us so many times before, I would not be surprised if they still make improvements that made this a moot point.

That said, there are those who would be happy with any improvement no matter how small... and if you put ten small improvements together they just might add up to a significant improvement eh?

Absolutely. I think the point is that for Red to be applied as a viable high-end cinema camera (which it 100% is), people are going to want to squeeze absolutely every single ounce of quality for a given pipeline out of it (for certain applications), I know that's what I will be doing. This may seem like nitpicking for some people, but for others it is a huge deal.

Seth.

 

[Seth Larney]

I am pretty sure that Jim has said that this was being addressed/fixed with the Red One update to hardware. The Red One "X" model if you will, they changed/improved a lot such as the DSP if I recall correctly. In regards to Channel Artifacts...

I am VERY interested to see what benifits in image quality we see from this.

Red continue to amaze me at every turn

 

[Seth Larney]

As long as the rescale is an integer resize, I would think that any compression artifacts would be fractionally reduced when maintaining the same data rates. I agree, 2k scaled at the current data rate sounds like an interesting idea, but I would also be interested in 2k scaled at the same data rate as 2k windowed.

I guess maybe user definable datarates for 2k would be the best option ? When shooting >24fps your effective datarate would have to compensate anyway..

 

[Axel Mertes]

Seth,

I see the requirement of a 2K scaled from 4K as you do, as most other do. The question remains what the physics of the camera can really do, ie. if the electronic board is capable of reading the full sensor area at 4k at full speed (60 Hz) in the standart version.

Next thing would be scaling down to 2K. Greetings from Nyquist theorem it makes no practical sense to go inbetween 2K and 4K, so 2K is OK, 2.4K would be NOT OK. That's the bad that Sony & Panasonic do on their HD gear with subsampling from 1920 to 1440 or 1280 to 960, to reduce pixel data and smooth out any high contrast data rate peaks. We all went for RED for NOT seeing this.

So you can either melt down each quadrant of the RGBG Bayer image to 1/4 (half height, half width) and store it as a new "virtual" Bayer image. Positive: You gain some dB PSNR as this 4:1 will reduce gain artefacts and give you effectively 2 bits more color depth precision.

BIG problem: You need to mark that new virtual Bayer image as being from the virtual downscale, BECAUSE its not that simple. You need a modified decoding scheme to handle the underlaying pixelshifts introduced by the scaling. But given the expected limitations of the recording electronics you may not be able to do this better during recording, so you need to fix for that during playback (which should be not an impossible task).

The other option is to combine the two green channels quadrants from 4K RGBG and thereby create three 2K channels R, G and B, which would be full color and potentially better than HD 3 chip systems, especially for the increase PSNR on green.

The 2K RAW will be lower data rate than 2K RGB. The question is what is the clearer image to store for post production. If I need to do some keying stuff, I'd for sure vote for the RGB. The RAW will benefit mainly for less compression artefacts (just deBayer artefacts, which can not be prohibited that way) as the datarate could be set fairly high. But you need to check the codec what its minimum compression leads to. Certain codecs will not allow for lesser compression ratio than a given minimum, so increasing the datarate is not a linear process that can end up anywhere between nothing and original uncompressed size. I do not argue on REDCODE here, its just WE users don't know, its up to the RED team to check this out and let us know.

My main concerns go for the fact that 4K 60Hz is never spoken about as REDCODE but only uncompressed via the 6,500 US$ module thats not yet available. We all don't know if it comes with additional electronics that only make the 4K readout at 60Hz possible, or if that is already standart possible in the base cam. And we don't know yet if there is enough processing power in the updated version so that a 4K 60Hz may get downscaled to 2K 60Hz and stored as REDCODE in realtime.

We can just hope for it, which I surely do.

Poeple want to use the full sensor area. Another method may be to read out only every second line and second row, but from what I know about CMOS sensor design THIS is not quite easy - and must be in the design from the chip or its almost impossible. We are on speculation here, unfortunately.

I'd wish we all would know a bit more on the port to which the CF or RAW Port are mounted, that whould shed some light into this.

Cheers,
Axel

 

[Seth Larney]

Axel,

You make a very good point about the 60Hz readout at full sensor. I think for me though, I'd be happy (enough) if we could only get 24fps at full sensor 2k .. providing this gave us a better image after REDCODE than 4k downscaled.

Very nice post ..
Cheers,
Seth.

 

[Clint Johnson]

Seth,

Next thing would be scaling down to 2K. Greetings from Nyquist theorem it makes no practical sense to go inbetween 2K and 4K, so 2K is OK, 2.4K would be NOT OK. That's the bad that Sony & Panasonic do on their HD gear with subsampling from 1920 to 1440 or 1280 to 960, to reduce pixel data and smooth out any high contrast data rate peaks. We all went for RED for NOT seeing this.

So you can either melt down each quadrant of the RGBG Bayer image to 1/4 (half height, half width) and store it as a new "virtual" Bayer image. Positive: You gain some dB PSNR as this 4:1 will reduce gain artefacts and give you effectively 2 bits more color depth precision.

The problem with what Sony & Panasonic do is that their downscaling for capture is meant to be upscaled back from 1440 to 1920 and 960 to 1280 for display- an ugly compromise. But with the Red this would be is scaling down in camera to a resolution that is actually higher than the resolution that it will be edited, FXed and displayed in (2k or 1920).

Using the full resolution of the sensor, scaling down to 1/4 would give you 2.26K capture from the full 4.52K available. This would in turn be scaled down in the computer to get the 2K that would be the working resolution.

Maybe the best 2K image out of the Red One won't be captured at 4K and scaled in REDCINE- but scaled in camera to 2.26K and then in REDCINE to 2K?

 

[Axel Mertes]

Clint,

ah, now I got your point, you went down from the full sensor area. The question here is if they will ever allow storing the "underscan" area. Right now it seems they not allow for this, but I bet this can be enabled with a new firmware.

Anyhow, as long as the downscale from Bayer to Bayer is fixed 1/2 step, it should be both fast and of high quality. I like the idea of storing away the underscan, so you can still pan around if required.

I think in most situations you'll actually crop out a 2K, as your framing on the camera is done via the displayed guides. The extra scan area is for the DOP or AC to see if they are going to hit undesired objects during shooting, like micros, lamps etc. Anyhow, technically it might be possible to do what you explained.

I just don't believe in a big gain of a 2.26K to 2K downscale, because due to the Nyquist theorem this yields to smoothed out images, much like what you get from the Panasonic and Sony cameras AFTER decoding from tape. IMHO its better to crop than to scale, but for many people its not directly obvious visible, and thereby may not be sooooo important after all.

You can make a simple test in Photoshop, Fusion, Shake, AE or whatever you like:

1. Make a checkerboard pattern 1 pixel black, 1 pixel white (just 1 pixel, not bigger).

2. Then fill your source size image with this pattern (so 2.26k).

3. Scale down to 2K using any algorythm you like, even the best and most expensive ones (CPU wise).

Look at the visible "waves" that you'll see then.

Repeat the steps, but for 2nd point change the resolution not to 2.26K but 4.52K instead as you mentioned, so that you scale from 4.52K to 2K directly in step 3. Compare the results.

Cheers,
Axel