Help me understand oversampling / downsampling etc.

[Hugo Noir]

Hi everyone!

I've been browsing the REDUser forum for a while to learn about the tech in our industry.

With the recent Sony FX9 announcement, I would use the word downscaling to describe a camera using a 6K sensor to record a 4K image. However Sony call it oversampling.

This has made me confused! So may I please ask if anyone here could help me understand what each term below has to be used for?

- Oversampling
- Overscaling
- Downsampling
- Downscalling

Thank you!

 

[Hrvoje Simic]

Sampling typically refers to bit depth (or frequency), scaling to pixel count.

Overscaling - recording a larger frame than you will be using
Oversampling - signal quantization at higher bit depth/frequency to increase precision and reduce aliasing
Downscaling - remapping the whole image to a lower resolution raster
Downsampling - reducing sampling bit depth/frequency

In case of FX9, it both downscales from 6K to 4K and downsamples to 10 bit from 16, they are just using the term broadly and generally to pass the notion of image improvement by more precise/dense quantization.

 

[Phil Holland]

I'll go for the general definition as it pertains to resolution in our industry rather than I guess the standard actual definition of these terms, which are correct as well.

Oversampling = Having more captured resolution than your output resolution, i.e. having an 8K camera used on a production that requires a 4K finish.

Downsampling = Describes taking a higher resolution image to a lower resolution without mentioning the actual method in doing so.

Downscaling = References the concept of taking a higher resolution image and scaling it down to your output resolution. Generally the benefits are increasing image detail, minimizing image noise, and also minimizing potential image artifacts. There are many different scaling algorithms that can be used to effect the level of detail, some are computationally more complicated than others.

Overscaling = not a term used in this industry. Upsampling or Upscaling is however used when describing scaling up a lower resolution image to a higher resolution output. This method is generally frowned down upon as it increases visibility of image noise and potential image artifacts related to the codec.


In the case of the Sony FX9 which you reference, it is a 6K sensor that can Downsample/Downscale to a 4K or 2K image from the full sensor readout of 6K resolution. This describes a scaling process rather than a Pixel Binning or Line Skipping method. Scaling is generally more processing intensive. Generally speaking it's prefered to Downsample/Downscale over Pixel Binning or Line Skipping as they can lead to particularly aliasing artifacts that are not something anybody wants in their final image.


Sony is correct calling it Oversampling because they are sampling from a higher resolution image of 6K in some of the filming formats the camera provides to create the 4K or 2K output. They don't mention what specific techniques they are using in this process yet, but it will be either a Downscaled or Downsampled in some way. I think most would prefer Downscaled in a camera at this price point.


As Hrvoje mentions this also can be applied to the bit-depth via sensor technology and the analog to digital converter towards the recoding codec's final output.

 

[John David Pope]

Phil, not to be greedy with your generosity, but could you also explain the differences between the C500MkII's "6K" and the Fx9's 6K downscaled to 4K? Or maybe what I am looking for is explanation of the differences between the Fx9's 4k downsampled from 6k vs the C500MkII's 4k.

Is it accurate to say the C500MkII can record in 6k but the Fx9 cannot?

 

[Phil Holland]

Phil, not to be greedy with your generosity, but could you also explain the differences between the C500MkII's "6K" and the Fx9's 6K downscaled to 4K? Or maybe what I am looking for is explanation of the differences between the Fx9's 4k downsampled from 6k vs the C500MkII's 4k.

Is it accurate to say the C500MkII can record in 6k but the Fx9 cannot?

Sure thing John.

Sony: FX9 clearly states "4K" on the camera body. The strategy with this camera is to use the good stuff that comes with the FF 6K format size to downsample within the camera for a 4K output. What they haven't announced nor hinted at is if the ability to ever output or record the full 6K resolution of the camera. In fact Sony has already planned their two major firmware updates for December 2019 and Summer 2020 and nothing is mentioning anything other than 4K Linear RAW or Higher Supported Frame Rates.

Canon: The C500 MK II does record at the full resolution of the sensor as well as providing a Compressed RAW Format in the form of Cinema RAW Lite.


My personal take on this, Sony is going for the market that really wants a straight deliverable image. Canon is providing the whole image for you to do what you want with like reframing, cropping, stabilization, etc. The benefit with the Canon method particularly if you are outputing to 4K is you still have a lot of resolution to work with if you are going into the image that way. The Sony FX9 however, the second you are cropping, reframing, etc. you will then need to upscale back out to 4K. This will be somewhere between a big deal for some and perahps not a big deal at all to others.

To your point, yes. The C500 MK II can actually record 5.9K internally as well as in Compressed RAW. The FX9 cannot record 6K internally or with the Extension Kit for RAW Output, but does downsample from the full 6K image resolution to produce it's 4K and 2K varieties.

If i was a betting man, I'd think that by Summer 2020 through Fall 2020 Sony will devise some sort of strategy for internal and external 6K likely via a paid firmware update. Just a hunch. At the moment they truly can't step on the Venice market at all. Canon's strategy is moderately like the Alexa LF/Alexa LF Mini with the C700 and C500 MK II using the same sensor. But truthfully the C500 MK II will likely be a far more popular camera than the C700 was/is.


Also, if you'd like to compare the two maximum format sizes between the Canon C500 MK II and the Sony FX9, here's a quick visual reference:
http://phfx.com/tools/formatCompare/formatCompare.cgi?makeA=Canon&modelA=C500+MK+II&formatA=5.9K+17%3A9&makeB=Sony&modelB=FX9&formatB=FF+6K&focalLengths=

 

[John David Pope]

Thank you, Phil. That explanation helps greatly with my understanding.

 

[Blair S. Paulsen]

Oversampling is of particular importance with Bayer pattern sensors due to their design. The short version is that shooting 6K for 4K provides enough discrete values that a decent algorithm can construct an image that resolves 4K cleanly. Doing that downscale in camera is convenient, but not ideal for stabilization, reframes, etc.

Cheers - #19

 

[Mark Farag]

Sure thing John.

Sony: FX9 clearly states "4K" on the camera body. The strategy with this camera is to use the good stuff that comes with the FF 6K format size to downsample within the camera for a 4K output. What they haven't announced nor hinted at is if the ability to ever output or record the full 6K resolution of the camera. In fact Sony has already planned their two major firmware updates for December 2019 and Summer 2020 and nothing is mentioning anything other than 4K Linear RAW or Higher Supported Frame Rates.

Canon: The C500 MK II does record at the full resolution of the sensor as well as providing a Compressed RAW Format in the form of Cinema RAW Lite.


My personal take on this, Sony is going for the market that really wants a straight deliverable image. Canon is providing the whole image for you to do what you want with like reframing, cropping, stabilization, etc. The benefit with the Canon method particularly if you are outputing to 4K is you still have a lot of resolution to work with if you are going into the image that way. The Sony FX9 however, the second you are cropping, reframing, etc. you will then need to upscale back out to 4K. This will be somewhere between a big deal for some and perahps not a big deal at all to others.

To your point, yes. The C500 MK II can actually record 5.9K internally as well as in Compressed RAW. The FX9 cannot record 6K internally or with the Extension Kit for RAW Output, but does downsample from the full 6K image resolution to produce it's 4K and 2K varieties.

If i was a betting man, I'd think that by Summer 2020 through Fall 2020 Sony will devise some sort of strategy for internal and external 6K likely via a paid firmware update. Just a hunch. At the moment they truly can't step on the Venice market at all. Canon's strategy is moderately like the Alexa LF/Alexa LF Mini with the C700 and C500 MK II using the same sensor. But truthfully the C500 MK II will likely be a far more popular camera than the C700 was/is.


Also, if you'd like to compare the two maximum format sizes between the Canon C500 MK II and the Sony FX9, here's a quick visual reference:
http://phfx.com/tools/formatCompare/formatCompare.cgi?makeA=Canon&modelA=C500+MK+II&formatA=5.9K+17%3A9&makeB=Sony&modelB=FX9&formatB=FF+6K&focalLengths=

Phil, do you know how the downsample process works (if at all) with the C500 II in its 4K Super35 mode? Haven't been able to find any solid info about this. My understanding is that when shooting 4K FF the image is being downscaled for a very rich 4K result. However, is the image simply being cropped in from the 5.9K when shooting 4K S35?

Thanks very much.

Mark

 

[Björn Benckert]

Phil, do you know how the downsample process works (if at all) with the C500 II in its 4K Super35 mode? Haven't been able to find any solid info about this. My understanding is that when shooting 4K FF the image is being downscaled for a very rich 4K result. However, is the image simply being cropped in from the 5.9K when shooting 4K S35?

Thanks very much.

Mark

If the 5.9k sensor is bigger than s35 then likly thats whats going on.

 

[Phil Holland]

Phil, do you know how the downsample process works (if at all) with the C500 II in its 4K Super35 mode? Haven't been able to find any solid info about this. My understanding is that when shooting 4K FF the image is being downscaled for a very rich 4K result. However, is the image simply being cropped in from the 5.9K when shooting 4K S35?

Two filming codecs are available on the C500 MK II and that effects things.

For Canon's Cinema RAW Lite codec:
- 5.9K = FF 35 - 1-60fps
- DCI 4K = Super 35mm - 1-60fps
- DCI 2K = Super 16mm - 1-120fps
* I also believe 16x9 variants will be available at launch

I'm having a difficult time digesting the information about the 12-bit and 10-bit variation in Cinema RAW Lite in their published specs, previously it was 12-bit 1-30fps and 10-bit anything beyond 30fps.

For Canon's XF-AVC codec:
- DCI and UHD 4K - Oversampled from the 5.9K FF35 - 1-60fps
- DCI and UHD 4K - Cropped to the Super 35mm format - 1-60fps
- DCI and HD 2K - Oversampled from the FF35 or Super 35mm format - 1-60fps
- DCI and HD 2K - Super 16mm crop - 1-120fps

The AVC codec is 10-bit and there is an 8-bit proxy mode which I believe you can send to the SD Card.

For the curious in comparing the max format size to RED Monstro from the C500 MK II you would be looking at the 7.5K FF resolution/format to approximately match the FOV:
http://phfx.com/tools/formatCompare/formatCompare.cgi?makeA=Canon&modelA=C500+MK+II&formatA=5.9K+17%3A9&makeB=RED&modelB=Monstro+8K+VV&formatB=7.5K+FF&focalLengths=

About 5K on Monstro to match Canon's 4K S35. About 2.5K on Monstro to match Canon's 2K S16.