General Demosaicing Questions

[Carsten Fenstermacher]

The basic idea is that when you want to reduce the number of colors you are doing dimensionality reduction, in this case to a single dimension, which actually means that you are projecting colors along a certain direction in linear terminology. In the color space representation the luminance is just a vector - pointing in a certain fixed direction. Similarly, CIE lightness is also a "vector" if you allow non-linear coordinates in the color space. There are infinitely many directions to pick. I picked a "middle ground" direction that gives importance to color saturation.

Sincerely,

Joofa

Your explanation is very obtuse.
The alternate image has had it's black point, white point, and contrast so it's no longer a conversion comparison. Luminance also has the same adjustments it appears.

I got almost the same image as alternate in photoshop by amplifying the CMY in the B&W conv and adjusting the blackpoint.

 

[Dj Joofa]

Your explanation is very obtuse.
The alternate image has had it's black point, white point, and contrast so it's no longer a conversion comparison. Luminance also has the same adjustments it appears.

I got almost the same image as alternate in photoshop by amplifying the CMY in the B&W conv and adjusting the blackpoint.

I don't understand what you mean here. It may be possible to generate a similar image using Photoshop. The explanation is correct for those who want to program this thing, similar to what I did, as opposed to using a package such as Photoshop and keep tweaking the parameters, so that they have an automated conversion without even having a need for visual inspection, in one go.

Such an automated process is helpful if one has to convert a large number of images.

And, there is a broader issue here regarding the nature of color space and the role of CIE luminance whose surface we have just scratched.

EDIT: Peter, thanks for the link. I shall check it out.

Sincerely,

Joofa

 

[Bill Anderson]

Agreed Carsten, As a long time B&W guy, I'm occasionally compelled to coax any color to B&W conversion towards emulating a Panchromatic response, even with the typical filters of the day (yellow, red, orange) qualities, and occasionally towards an orthochromatic emulation. The tools at our disposal allow for a very fine tuned black and white image.




I don't think that the "realization that standard notions of B/W or grayscale are not always very attractive" is "new", at least not to the general practitioner. A few years ago perhaps, when color stock was being used to shoot for a B&W finish. But that "notion" is no longer, due to today's technology. There is always room for new, and more effective aproaches, I'm sure.

 

[Eric Lange]

The basic idea is that when you want to reduce the number of colors you are doing dimensionality reduction, in this case to a single dimension, which actually means that you are projecting colors along a certain direction in linear terminology. In the color space representation the luminance is just a vector - pointing in a certain fixed direction. Similarly, CIE lightness is also a "vector" if you allow non-linear coordinates in the color space. There are infinitely many directions to pick. I picked a "middle ground" direction that gives importance to color saturation.

Sincerely,

Joofa

DJ JOOFA


I take my hat off to you as it takes a lot of balls to post imagery from industrial systems as a talking point for various de-bayer type machinations on REDUSER. Awsome!


My color science is a little old, out of date and foggy.


However, theoretically if you are mapping from one space to another space then things can suddenly get a little more complicated for what seems like a really straight forward “linear” problem.


For example if you map from one 3d (vector) space (as you describe) into another 3d space, or 3d space into a 2d space then one needs to use more advanced techniques. It doesn’t matter if it’s XYZ into xyz or Lab, or CMY or elephants, zebras and orangutans. As I understand it, the problem is “framed” as you set out, by an input vector and an output vector. In between there is a transformation matrix that has to be computed or solved for. In most cases I suspect that one color space is not going to be orthogonal to the other and as you mention there is the complication of non-linear output and response curves.


So even in the seemingly “linear” case with a vectorial system you have a direction component equivalent to a rotation matrix. So you have in effect, scale (in xyz), translation/offset (in xyz) and rotation about (xyz). Unfortunately the rotation bit for non-orthogonal systems throws a spanner in the works right there, and has to be treated in a non-linear way to compute the transformation matrix from one space to the other. Typically a perverse form of linear regression has to be used to get the right answers. There is a direct solution but it is a total bastard and is very unstable. In the general case, what is used instead, is an iterative least squares solution that uses linearized versions of these equations (formed from their partial derivatives) that within three or four iterations is able to converge on the most probable solution (which in practical terms IS far more accurate than any other method that exists). Things can get a little more challenging when you have truly non-linear factors that have to be thrown into the mix as well, so for example a simple sigmoidal input/output curve. These also need equations built for them and again a set of partial derivatives are stacked into matracies to represent a momentarily “linearized” approximated version of these polynomials. So you can throw that into the mix too, so that you build a monster system of equations that require many sets of input and output values to even derive a unique ‘Transform” or solution for all of that mess. The one advantage of the iterative least squares approach is that you can use redundant data. I.e. you can throw in a couple of hundred of your favorite input and output values to derive a pretty decent and refined set of computed transforms (that is statistically much more accurate than just averaging multiple unique single sets of solutions) . Probably nowadays (using sparse matrix and better numeric methods etc.) you could probably throw every pixel into the mix, and have it crunch for less than an hour or so without the computer system chocking or keeling over.


I think it’s these types of approaches that start to help build “nice” color and even then mathematically speaking it’s a fine art to know how to massage or correctly weight a set co-factors to produce a simplified or coherent set of transformations without things going “pear shaped”. I think beyond that, what might be usefully derived is a three dimensional manifold (polynomial/nurbs surface) in 3d space that coverts your input color to your output color.


To my way of thinking L,a,b type systems are useful for applications like spectro-photometry, but I sense that when flipping/transforming from one color space to another, even if you are collapsing a couple of dimensions, intuitively I seem to think that there is a much more complex interplay between chroma and luminance and that these are somewhat “artificially” discreet concepts; at least if you are trying to get really super color conversions OR if you are trying to get really good B&W from a color/Bayer chip.

By extension I am guessing that it is this special understanding of the interplay between a “notional” luminance, human response curves and the color filters on the Bayer array that all start to build and yield acceptable color.

I don’t fully know as I’m not a color guy, but I think the things you present raise a lot of really useful questions.

Cheers,

Eric

PS; If I remember rightly, about 20 years ago, that the National Gallery in London built a 2d digital recording system to accurately record their paintings in terms of accurate color and resolution to monitor the rates of deterioration and color change of these works of art, [called the Vasari system]. So they were doing a lot of very accurate color transformations but I think the key was NOT to use standard color targets but instead to use ceramic color chips that were selected to be, by eye as close as possible to the colors in the paintings to begin with. All of these color tiles had very accurate spectrophotometer measurements made of them. The idea was that you could carry out very accurate color transformations and calibrations if the color target tiles were closely matched to the subject, so that ‘vectorially” in color space the distances are relatively small and not prone to being muddied up by more “distant” or unstable (non-linear) computed transforms.

I don’t know if on set a similar approach is used , i.e. you cheat with color standards that match closely the chromaticity of the subject as lit for good calibrations and color grading/matching, (not just grey card and a Macbeth)?

 

[Dj Joofa]

DJ JOOFA

My color science is a little old, out of date and foggy.

Agreed.

Sincerely,

Joofa

 

[Jannard]

Hope you guys are having fun with this stuff...

If you change one thing in the discussion, like a different way to debayer... all the results change significantly.

In the end, the image speaks volumes.

Jim

 

[Eric Lange]

Hope you guys are having fun with this stuff...

If you change one thing in the discussion, like a different way to debayer... all the results change significantly.

In the end, the image speaks volumes.

Jim

Ditto!

This is why I try to steer clear of “fiddling” with color. It can take the greater part of a life time to get it good and right, I don’t even try, as I have other very demanding fish to fry. It took the National Gallery’s research group of a specially assembled large imaging team, from Imperial College London about a decade to get a decent handle on the problem of “accurate” color.


So if someone else has this problem working well, that’s a huge relief to me.


We do things that are more of a “spatial” nature, so when I look at imagery I ask a much more basic question. Does it look like a sucky industrial sensor? If the answer is no, then I am happy. Anything up form there is huge bonus. It does take a hell of a lot (in terms of engineering) to make some of these sensors produce imagery that is worth looking at.



Cheers,

Eric