RED Rushes Resample Filters Comparison

[D'Arcy Foley-Dawson]

I just want to say at the outset: yes, I have tried searching the forum to see if this has already been addressed. So if it has, I would very much appreciate being re-directed.

My question is: has anyone done a comparison test of the various Resample filter algorithms available in Red rushes - eg. Cubic, Mitchell, Gauss, Quadratic etc. etc. - and posted their results ie. some screencaps? Any sort of general, 'in my opinion, this filter is more suited for blah blah blah type footage'....?

Whilst I know many people will have a particular algorithm they always favour, I'm assuming none are necessarily 'better' than another, merely a matter of taste and/or what suits the particular material being transcoded. Otherwise why would Red rushes offer so many different ones? (I'm also assuming it's not because some are faster to transcode than others, because the various debayering options are far more relevant to speed issues).

I know I could also do the comparison myself, and I might when I get the time. But I was just wondering if anyone has done some real comprehensive tests that might be of value to others. Cos that'd be frickin awesome.

 

[Nicholas Shields]

Great question. I too look forward to a response.

Nick.

 

[SF Geek]

good luck

 

[Gunleik Groven]

Not in the way you suggest.

Problem is that is quite a huge task, as I tend to favor different filters based on:
What the footy looks like
Which deliveryformat(s) is/are relevant

Interpolate footagelooks (individual in some extreme cases) with deliveryformats and the choice of filters, you'll end up with a diagram so huge, that it basically goes back to:
Test, test, test and make up your opinion yourself.

Generally, though, I'd say, stay away from sharpening filters if you're going to SD of any kind. I tend to use gauss more and more for those applications. Sharpening should happen at the end of the signal chain (if at all...). Just makes problems on the way there.

But that's just me...

 

[D'Arcy Foley-Dawson]

Yeah I know it's not something that you could get easy straightforward answers for, but I think some sort of basic comparison could enable each individual to draw their own conclusions and/or guide them in their choices.

If I were able to see a bunch of high res 1920x1080 screencaps from a small sample of source footage: just for example, say a bright outdoors high contrast scene, a low lit indoors scene and a focus test chart, all transcoded with each filter (painstaking, I know); then I might be able to pick say three filters I feel would be best suited to my current project, and then test those three myself with the relevant footage. Or if I decide that I can't see any difference between a set of filters, that could also be valuable knowledge.

Of course it's something that requires trial and error and guesswork, but that doesn't mean we can't be helped along the way by something that takes out a little bit of the guesswork. Do other people feel the same way?

I say this as someone who has just started his first project with Red Rushes, and like many people I must confess I didn't do much research into it beforehand and now I'm working to a tight deadline :blush:.Hell, it's the first time I'm doing the down-scaling myself, and whilst I am very picky about my visuals, I don't have much personal experience playing around with different scaling algorithms. Nor do I have the time, cos I gots me a video ta edit, finish and deliver!

 

[Nook Kim]

I'd appreciate if people can just give their opinions like what filers they normally use and why those filters.

Actually, a short explanation from Graeme would help a lot.

 

[Florian Stadler]

gauss and Mitchell seem to be a good compromise for a smooth filmic 1080p finish

 

[Deanan]

gauss and Mitchell seem to be a good compromise for a smooth filmic 1080p finish

Agreed. They are my favorites. However if you prefer sharper, catmul-rom is nice. Lanczos is sharp to but tends to be a bit harder.

On the other hand, alot of film outs are being sharpened these days. I was really turned off at the amount of artificial sharpening on Body of Lies. It was almost like looking at F900 footage at times.

 

[GlennChan]

Another approach would be to use gaussian to prevent ringing + aliasing, and then enhance perceived sharpness via contrast enhancing tricks. e.g. unsharp mask with larger radii. A S-curve on the RGB channels helps too, but presumably you're already doing that.
(Of course, this is if you have a lot of time to burn. USM may take a while to render... but may be fine if you like to finish in After Effects and are ok with its speed.)

Somewhat related article:
http://www.cambridgeincolour.com/tutorials/sharpness.htm

- My prediction of the future: Presumably when Epic rolls around, Red will start looking at higher quality downsampling algorithms (e.g. being adaptive around edges to prevent ringing). Or if they don't, somebody else will (or should).

 

[Dj Joofa]

I'm also assuming it's not because some are faster to transcode than others

Different flavors of resampling filters have different complexity for performance measure. In hardware implementation some of the computational saving are achieved by moving the downsampler/upsampler through the filtering operation to achieve a configuration what is known as "polyphase filterbank."

Such filters typically have an inherent low-pass operation built-in. However, realtime hardware face another issue, and that is the ideal low pass filter does not exist in nature -- if somebody tells you it does, then politely ask them to go forth and multiply. Software solutions don't face this problem, as "future" data is already there. Hardware solutions counteract that on the expense of delay.

Additionally, many of these filters were conceived as pure resampling filters. They don't discriminate between what is noise and what is signal, beyond what low-pass filtering built into downsampler/upsampler might accidentally remove -- accidentally, because, the low-pass was there not for noise removal, but for adjusting to signal spectrum expansion/contraction after downsampling/upsampling.

As an example, the filter below, is the "best" least square estimator of the "true" signal in the presence of white noise, and an image correlation model that is typical, (using an AR (1) approach.) Note the filter is also low pass, however, it is not yet ready for downsampling after filter operation, as it was not designed for signal decimation (spectrum expansion).

Note that the above filter has a twisted phase response.

- My prediction of the future: Presumably when Epic rolls around, Red will start looking at higher quality downsampling algorithms (e.g. being adaptive around edges to prevent ringing). Or if they don't, somebody else will (or should).

Some more complex solutions exist right now. The future is here.

 

[D'Arcy Foley-Dawson]

wow. :blink: i understood very little of that Joofa, but thank you nonetheless!

i have to agree with what others have been saying, i personally don't like a really sharp look, cos it can make it look a bit like video.

so far, for my first project doing the rendering myself, i've rendered all my footage out in cubic, simply because the dp recommended it. he also said it would give it a softer less video-y look.

won't really have time to run some tests with other algorithms before i have to deliver this project - more because of the time involved in re-rendering everything and doing a conform, rather than the actual testing itself. but there's always the next job.

I was really turned off at the amount of artificial sharpening on Body of Lies. It was almost like looking at F900 footage at times.

i haven't seen body of lies yet. should i make the effort?

i saw the new(ish) mike leigh film 'happy go lucky' the other night. i was amazed at how well some of the scenes that were shot on video - all inside a car, so obviously for practical reasons - cut together with the 35mm. these driving scenes recur about five or six times throughout the film and it wasn't until the second or third such scene that i realised it was video. it's amazing how much can be achieved simply by getting the colours (especially skin tones) right. on the other hand, you see so much stuff these days (eg. 21, get smart) shot on the genesis, that just looks, in my opinion, crap.... and as far as i can tell because they don't know how to shoot/grade... or for some reason want it to look crap. but it's got very little to do with the format - consider what dion beebe/michael mann did on collateral... although it could be argued beebe used that format's inherent characteristics to create a unique look, rather than trying to mimic film. i think that should be the real goal of anyone shooting on video, altho exactly how to do it i'm not entirely sure yet.

i am btw completely rambling off topic now...

 

[GlennChan]

Some more complex solutions exist right now.

What practical software solutions are there? A lot of the software resampling now is uh... lacking to say the least.

e.g. look at the HD to SD resampling thread(s).

 

[Vincent Thomas]

Depend of the look you wank and the shoot... Gaussian is the smoother one

v*

 

[Dj Joofa]

Depend of the look you wank and the shoot... Gaussian is the smoother one

There are two important parameters: (1) when the resampling filter falls off in the frequency response, and, (2) once it does, how steeply it goes. Good pass band performance is exhibited by Catmull-Rom. Nearest neighbor is good in pass band, but falls off very slowly. Linear interpolation has less desirable pass band performance. Quadratic and approximating B-splines are also not great in this respect. Good stop band performance (staying close to zero) is exhibited by approximating cubic B-spline, followed by Catmull-Rom.

The positivity of certain filters is desirable sometimes, as unlike many other functions that have negative lobes, the resampled signal may not be clamped.

What practical software solutions are there? A lot of the software resampling now is uh... lacking to say the least.

e.g. look at the HD to SD resampling thread(s).

Hi Glenn, I shall get back at you soon.

 

[Stacey Spears]

Catmull-Rom is one of my favorites, though it is really an interpolation filter vs. a low pass filter. Personally I use Catmull-row for resizing down.

One problem I see today with a sinc function, like Lanczos, is the number of lobes chosen. Many people really like to use more than two, which I believe is a bad idea. Every time you cross over the zero-crossing point, you introduce more ripple along edges. I once tried a Kaiser with a lot of lobes and, while the MatLab graph was pretty, the visual results were not.

 

[Dj Joofa]

What practical software solutions are there? A lot of the software resampling now is uh... lacking to say the least.

Many resampling solutions provided by manufacturers are limited by two things among others, (1) noise is not taken into account, (2) filter coefficients are not always adaptive.

And, importantly, there is little appreciation of when mean square error, the darling of physicists and engineers, is not a good estimator (or objective function) when trying to determine resampling coefficients -- many manufacturers will just use static coefficients, or choose from a family of static coefficients, instead of trying to determine "optimal" coefficients that may be time/space dependent. (Physicists love mean square error as it is differentiable and they can derive results on paper and feel important. Engineers take it a step farther and implement filters, both adaptive and non-adaptive, based upon that. But, we the people, can't let that go on.)

A quantitative example will clear that up. Suppose you wanted to estimate the value of two variables, A and B, and the only knowledge you have is that A * B = 1. Assuming that error is distributed as a Gaussian (engineers love to assume that, don't they?), such as exp (-|A*B - 1|), with equal possibility of any values of A and B. Now what is the value of A and B?

One can calculate that using mean square error the value of A = B = 1.3. But that is not satisfactory, since we knew that A*B = 1, using A = B = 1.3, we get A*B = 1.69.

Many of the analytical solutions derived in signal processing for minimum mean square estimation are actually special cases of what is known as Bayesian estimation in pattern recognition. Using another very frequently used special case of Bayesian estimation, the maximum a posteriori estimator (MAP), we get an infinite number of answers -- i.e., all A's and B's which product is equal to 1.

However, if instead of using a quadratic cost function (mean square error) and delta functional cost function (MAP estimator), if we choose something in between, we can get unique solution(s), which product is equal to one, and hence are more satisfactory than mean square error or MAP. Though, we may have to pay some price, such as lack of analytical solution, but we may get a "better" solution than what typical signal processing will give on the cost of more computational power.

 

[GlennChan]

To look at it from a different perspective...
I don't think that look at downconversion from a frequency-based point of view will give the best visual results. Simply put, it's the wrong model for human vision... we don't see the world like that.

I think engineering needs to be more bottom up, about what looks good (and trying to figure out how our visual system works), instead of sticking to particular models about how it "should" work.

2- If I wasn't lazy I'd try and figure out something in Shake. Do sinc downsampling, except make a matte to catch the edges and use a different algorithm for edges to avoid ringing artifacts/halos on edges.
(Well, that's the kind of adaptiveness I had in mind anyways!)

If you really wanted to get picky you would handle color differently, because ringing artifacts look TERRIBLE whereas blurriness doesn't (and I don't think we'd notice color aliasing if gamma correction is accounted for).

*Shake's sinc filter really has 1 lobe from what I remember of testing it.

 

[Dj Joofa]

To look at it from a different perspective...
I don't think that look at downconversion from a frequency-based point of view will give the best visual results. Simply put, it's the wrong model for human vision... we don't see the world like that.

I think engineering needs to be more bottom up, about what looks good (and trying to figure out how our visual system works), instead of sticking to particular models about how it "should" work.

Hi Glenn, I can appreciate what you are saying, that's why I gave you the example of Bayesian estimation, because it is quite rooted into human way of perception (I can go in details if you want to).

 

[D'Arcy Foley-Dawson]

To look at it from a different perspective...
I don't think that look at downconversion from a frequency-based point of view will give the best visual results. Simply put, it's the wrong model for human vision... we don't see the world like that.

in all honesty, and not to sound ungrateful to the people who've contributed to this discussion, i'm glad you pointed this out. this thread was created with the intention of exploring what different algorithms translate into visually. all manner of graphs and equations, whilst maybe very interesting, don't have any relevance to my original post.

what i think there is a need to understand is how different algorithms are going to translate into visual results, not their inner mathematical architecture.

 

[Stacey Spears]

I don't think that look at downconversion from a frequency-based point of view will give the best visual results.

I strongly agree with that.

(Well, that's the kind of adaptiveness I had in mind anyways!)

That is one type of adaptive, another that you will see more of is edge adaptive. Companies have done it for deinterlacing video sources since Jack Campbell (Faroudja) introduced DCDi. MSU had the Smart-Edge demo a while ago.