Understanding REDCODE RAW SCALED...

[Jim Arthurs]

I was thinking about the potential of Redcode Raw scaled, and I am excited about the possibility, but confused as to how it could be done... when we were told that it was "impossible", that seemed right on first inspection... and yet now, well, I can guess two possible methods from a layperson's point of view...

1.) Simply skip over every other set of colored pixels from the rows. The easiest method on hardware... but I'm not sure if there would be unintended color issues when de-bayering.

2.) Probably the highest quality method would be too hardware intensive to do in camera... you'd read out all the green, red, and blue into individual buffers... scale each buffer indendently to the correct size, and then write each color site back into the now 2K raw stream in the proper bayer order. This would give better S/N that method 1.) and also reduce aliasing.

None of these approaches will give you a better color to luma ratio than the windowed sensor extract, because that's the nature of raw, but at least we'd have the promise of higher frame rates and the full sensor DOF from scaled Redcode Raw, no matter how it's done.

Any other ideas?

 

[brandon herman]

sRAW

sRAW

sorry, i don't know how they are doing it...

But I wanted to point out that something similar is being done on the new Canon DSLRs. In the description of the new 40D, it discusses "sRAW" as approx 1/4 of the full resolution.

So the 10.1 MP camera can be set to shoot 2.5 MP RAW images.

I'm not sure why you would take 2.5 MP stills now, but it's available....


---------------------------------------------

from the AMAZON product description.....

sRAW
In addition to retaining the RAW image capture capabilities of its predecessors, the EOS 40D SLR now offers a more manageable "sRAW" recording format. In sRAW mode, the number of pixels is reduced to one-fourth that of a standard RAW image and the file size is cut in half, while retaining all of the flexibility and creative possibilities associated with full-size, conventional RAW images.

 

[Steve Freebairn]

Is Graeme working for Canon too? Or did they still some Pixel Fairies from RED?:ranting2:

 

[Michael Lindsay]

Jim

I've puzzled over the possibility, and then impact, of scaled for some time..

The potential extremely low fill factor of the skipping pixels approach is made better, in my mind, by the fact I rarely ever shoot wide high detail shots at high frame rates... The option to either accept low fill factor or crop the chip is, despite the compromise, still exciting..

I doubt there will be extra colour compromises post a few tweeks to their de-bayer algorithms...

Red has an amazing drama camera (I know it can do much more than just drama) with Red 1... maybe they would consider a 435 like really extreme Red 2..

regards

Michael Lindsay

PS I just finished a 2 (long) day shoot, Hdcam PS 35mm adapter and some superspeeds. Shooting 50i/60i again as fudged slowmo rates depressed me and helped me appreciate Red and all its options/compromises..

 

[I Bloom]

I was thinking about the potential of Redcode Raw scaled, and I am excited about the possibility, but confused as to how it could be done... when we were told that it was "impossible", that seemed right on first inspection... and yet now, well, I can guess two possible methods from a layperson's point of view...

1.) Simply skip over every other set of colored pixels from the rows. The easiest method on hardware... but I'm not sure if there would be unintended color issues when de-bayering.

2.) Probably the highest quality method would be too hardware intensive to do in camera... you'd read out all the green, red, and blue into individual buffers... scale each buffer indendently to the correct size, and then write each color site back into the now 2K raw stream in the proper bayer order. This would give better S/N that method 1.) and also reduce aliasing.

None of these approaches will give you a better color to luma ratio than the windowed sensor extract, because that's the nature of raw, but at least we'd have the promise of higher frame rates and the full sensor DOF from scaled Redcode Raw, no matter how it's done.

Any other ideas?

Well, I proposed a method a while back in this thread .

My method was this:
1. Capture 4K Data from the sensor.
2. Demosaic to 4K RGB.
3. Scale to 2K RGB
4. Remove 2/3 of the samples, creating a "soft" 2K Bayer Pattern.
5. Compress using REDCODE.

I don't know how well my solution would work. I think both of the solutions you proposed might not work very well. You can't scale a bayer pattern directly, undersampling would yeald poor quality, and treating the three channels independantly might lead to all manor of color artifacts.

I think if their is a solution that will work, Graeme is the guy, who is going to find it. But its fun to brainstorm. :blush:

sRAW
In addition to retaining the RAW image capture capabilities of its predecessors, the EOS 40D SLR now offers a more manageable "sRAW" recording format. In sRAW mode, the number of pixels is reduced to one-fourth that of a standard RAW image and the file size is cut in half, while retaining all of the flexibility and creative possibilities associated with full-size, conventional RAW images.

I'd be interested to know if by RAW they mean Bayer Pattern. It might be that they mean, "no compression" and/or "full bit depth" but actually RGB. Seems like RAW could mean a lot of things.

IBloom

 

[Kevin Halverson]

I'd be interested to know if by RAW they mean Bayer Pattern. It might be that they mean, "no compression" and/or "full bit depth" but actually RGB. Seems like RAW could mean a lot of things.

IBloom

I would think that RAW is certainly Bayer, not RGB. I would also assume that RAW is full bit depth (but possible not padded to 2 bytes per sample).

 

[Bruce Allen]

I don't know how well my solution would work. I think both of the solutions you proposed might not work very well. You can't scale a bayer pattern directly, undersampling would yeald poor quality, and treating the three channels independantly might lead to all manor of color artifacts.

You're absolutely right. That's why anything with a Bayer sensor needs a reasonably strong low pass filter - to avoid weird color dots on the bride's lace, etc...

So you'd have to do a low-pass filter on the color channels (and a little on the luma) between steps 3 & 4.

Or else in step 4, when generating the Bayer data, just take into account neighboring pixels somewhat (essentially similar to a low-pass filter)... you could be clever about it though.

The problem is that all of this would involve quite a bit of FPGA heavy lifting.

I'd be interested to know if by RAW they mean Bayer Pattern. It might be that they mean, "no compression" and/or "full bit depth" but actually RGB. Seems like RAW could mean a lot of things.

I think RAW equals "data as straight from the sensor as possible" (after the on-chip A/D and noise reduction process but before the most damaging stages). If you're compressing the RAW data (losslessly or lossily), it makes sense to do a bit of noise reduction first though.

As for scaled (compressed?) RAW, buddy, your guess is as good as mine. They could do tons of different things, choose different color sampling / subsampling methods (4:4:4 RGB vs 4:2:2 vs Bayer, etc), compression levels, quantization, etc. Whether it's better to take the channel info down to 10-bit log and do 4:4:4 RGB or whether you should go to another Bayer system and try to have higher bits per channel is really something only they can figure out with good old trial and error Somewhere you need to figure out the right tradeoff between chroma range (in terms of both the number of steps and where to clip), luma range, luma resolution, chroma resolution, compression ratio, etc.

Maybe it's not worth the effort, though? Spend resources on just getting the one kind of lossy compression working right?

Bruce Allen
www.boacinema.com

 

[Jim Arthurs]

The problem is that all of this would involve quite a bit of FPGA heavy lifting.www.boacinema.com

Yes indeed... but I don't think I explained my #2 very well... think of it this way (helps to have a little bayer pattern image in front of you while I go through this)...

1.) Read out ALL of the green sites to a single full size buffer (OLP is of course taken into account at this stage). Do the same for the red and blue sites.

2.) Now you have 3 B&W buffers, each uncontaminated by the other colors.

3.) Scale each buffer down to your final size... still uncontaminated, but now supersampled down to the new size taking advantage of the full sensor's signal to noise ratio and scaling out any noise in the process.

4). Into the final 2K raw buffer... write one sample of the green, then red, then green... on the next line... a sample of blue, then green, then blue... and on and on until you've made a "new" bayer image. You're basically making a final image created with the same bayer pattern as the original. No color strangeness that I can foresee... and no passing through an RGB stage.

If it's still unclear how I'm describing this, think of it this way... when you scale an interlaced image, you don't simply scale the whole image... you de-interlace, then scale each field independently and then re-interlace them... I'm talking the same thing, but not only across lines but from pixel to pixel as well...

As for scaled (compressed?) RAW, buddy, your guess is as good as mine.....
....Maybe it's not worth the effort, though? Spend resources on just getting the one kind of lossy compression working right?
Bruce Allen
www.boacinema.com

I only put this out as a brain exercise, and only because the team mentioned that they were looking into it in the first place!

... I think you can have scaled raw and have it look exactly the same as presented to the REDCODE RAW compressor, but as you say... is it worth it?

 

[Dominic Jones]

... I think you can have scaled raw and have it look exactly the same as presented to the REDCODE RAW compressor, but as you say... is it worth it?

It certainly would be in my books if it allowed you to take advantage of the higher 2k framerates (to a degree at least - I imagine the overhead of the scaling would reduce the max fps somewhat) from the full sensor - other than that I can't really think of any great advantages, unless for some reason you really needed the lower datarate...

 

[I Bloom]

2.) Now you have 3 B&W buffers, each uncontaminated by the other colors.

3.) Scale each buffer down to your final size... still uncontaminated, but now supersampled down to the new size taking advantage of the full sensor's signal to noise ratio and scaling out any noise in the process.

In the spirit of thought experiments, I think you are missing something here and that is that the beauty of the bayer pattern demosaicing is that it can capture a lot of detail in the luma aspect of an image by looking at color ratios among neighboring pixels rather than seperating color channels blindly. When you start scaling you need to preserve these ratios first and foremost. I don't think the method you are proposing is going to do so.

In comparison to wavelet compression, demosaicing and scaling are not nearly as computationally intensive.

IBloom

 

[MikeCurtis]

Guys - AFAIK the way 2K goes to higher frame rates is by chopping down the active area, processing fewer photosites at a higher clock rate - metaphorically, if you can do 4.5K @ 60p in theory, the lower the live area, the higher the potential frame rate possible, until some gating/limiting factors are encountered that I'm not engineery enough to know what those would be.

Only advantage of 2K scaled that I'm aware of would be to save space - shoot full aperture to get desired DoF, then scale those results down. With the efficiency of Redcode and a Red Drive, I don't see it as too much of an issue - should RED team bend over backwards to try and get this to work, or just get better compression and/or bigger Red Drives & other storage media to market?

My $0.02.

-mike

 

[Jason Murphy]

Guys - AFAIK the way 2K goes to higher frame rates is by chopping down the active area, processing fewer photosites at a higher clock rate - metaphorically, if you can do 4.5K @ 60p in theory, the lower the live area, the higher the potential frame rate possible, until some gating/limiting factors are encountered that I'm not engineery enough to know what those would be.

Only advantage of 2K scaled that I'm aware of would be to save space - shoot full aperture to get desired DoF, then scale those results down. With the efficiency of Redcode and a Red Drive, I don't see it as too much of an issue - should RED team bend over backwards to try and get this to work, or just get better compression and/or bigger Red Drives & other storage media to market?

My $0.02.

-mike

I think, however, that because of the lower data rate there was talk of potentially being able to record 2K Scaled Redcode RAW at 60 FPS to CF/RedDrive, retaining the 35mm DoF. So, without 2K Scaled Redcode RAW, your only other option for overcranking and retaining the DoF characteristics of the full sensor would be to go with the RAW port and a RAID.

If the above is true (and with the speed that things change around here, I could well be wrong), then I think it's something that is definitely worth the RED team's time and energy. Being able to overcrank with the full sensor's DoF characteristics, and record it onboard - that would be a pretty major coup.

 

[Dominic Jones]

Hi Jason - that's my understanding too... Mike - is that opinion from better info from inside Red, or an assumption (no offense intended, of course!)?

My understanding was that the overcrank limitation was a bottleneck in the Redcode compression engine, so reducing the datarate to the compression engine (by scaling the RAW image from 35mm sensor area) should allow framerates up to the full rate the sensor is capable of - i.e. 60fps in full-frame 35mm.

That would be well worthwhile, in my opinion...

However, if you're right Mike, then I agree - lower datarate 2k with the same overcrank options as 4k would not be very worthwhile.