ML Super Resolution in Pixelmator Pro - very effective upsampling

[Karim D. Ghantous]

Are you the owner of a S35 Alexa, and thinking of buying that 4K license? Well, don't! There is now a pretty effective upsampling algorithm in Pixelmator Pro, called ML Super Resolution. It's the most effective upsampling algorithm I've yet seen. If you have to upsample an image, you can't get better than this, as far as I can tell.

https://www.pixelmator.com/blog/201...l-super-resolution-feature-in-pixelmator-pro/

It is not a substitute for actual resolution, but when you have only a low res file, it's better than nothing. I wonder how well this would work for motion? I guess it won't be long until someone gives it a try. Those old TV shows recorded on broadcast cameras can now be better viewed on larger displays. At least, in theory.

I'm fairly sure this kind of thing could be done without machine learning, but I don't have the know-how or inclination to try and build something like this using simpler algorithms. The question has to be asked: if you can do this without ML, then why hasn't it been done already?

 

[Andy Jarosz]

The reason it can't be done without machine learning is because machine learning is "context aware." Obviously it's not consciously making decisions, but it's pushing and adding pixels from comparing the input image to similar known images from training data.

i think it would be a lot of fun to feed in an image sequence and watch the crazy Lovecrafitian output.

 

[Hrvoje Simic]

Are you the owner of a S35 Alexa, and thinking of buying that 4K license? Well, don't!

Did you notice render times ?

300,000 pixel image, MBP 16" = 2.4 sec

Only ~ 57 times slower then RT video playback.

Arri OG image before upsampling = 7,539,648/7,503,972 pixels
~ 25x 300,000 pixel image

57 x 25 ..... ?

 

[David Rasberry]

Did you notice render times ?

300.000 pixel image, MBP 16" = 2.4 sec

Only ~ 57 times slower then RT.

Arri OG image before upsampling = 7,539,648/7,503,972 pixels
~ 25x 300,000 pixel image

57 x 25 ..... ?

I’ve used Raw Therapee and Capture One Pro to process 2k CDNG footage before Resolve debayer quality improved enough to make it not worth the time. Typical 2.3 Mp frame processing time was in the 2 to 5 second range depending on settings on a loaded i7 desktop setup.
Still produces superior results, but it really is like sending film to the lab. Come back in a couple of days.

 

[Hrvoje Simic]

I’ve used Raw Therapee and Capture One Pro to process 2k CDNG footage before Resolve debayer quality improved enough to make it not worth the time. Typical 2.3 Mp frame processing time was in the 2 to 5 second range depending on settings on a loaded i7 desktop setup.
Still produces superior results, but it really is like sending film to the lab. Come back in a couple of days.

Superior results and in developing what you captured.

This is neither superior result in general, nor developing what you captured.

This is computational image alteration. Typical upsampling does not change the image, it just remaps it to a higher pixel count. The point of it is larger raster with no detail loss. Not gaining something in the image and affecting the look by computation.

If a goal was more image detail/higher detail sharpness, camera with a higher pixel count would be the choice. Or sharper lens. Or both.

Pretty sure many DoP's would be pissed for a computer altered look of what they get with Alexa 35 and preferred glass, ending up with re-processed image and different aesthetic. Even if we disregard a day of rendering for one minute of footage.


Also....a note for younger Creatives:

The term thrown on the internet attemping to promote pre-programmed image enhancement for the masses - "computational photography" = "digestive cooking" = oxymoron.
Ergo/therefore/hence - nonsense.

 

[Misha Engel]

Is looks a lot like this https://videoai.topazlabs.com/

We use Topaz Gigapixel AI to upscale some of our SD footage to 1080p (SD-video file > convert to JPEG > upscale > convert to videofile)

 

[Karim D. Ghantous]

The reason it can't be done without machine learning is because machine learning is "context aware." Obviously it's not consciously making decisions, but it's pushing and adding pixels from comparing the input image to similar known images from training data.

i think it would be a lot of fun to feed in an image sequence and watch the crazy Lovecrafitian output.

1. Well, let's try and think of at least one way in which we can approach this problem without using ML. We start by vectorizing the edges. That's a start, right? Once that can be done well, we then move on to the more difficult parts.

2. Yes I think that would be quite fun. ;-)

 

[Zack Birlew]

I tried to fix a blurry out of focus shot with Topaz Sharpen AI, the result wasn't very good at all, it completely distorts eyes and didn't help skin so much, even uprezzing with a batched image sequence using the standard Gigapixel AI didn't help much for a VFX shot we tried to upscale from 4K to 8K as it didn't add much to the image that couldn't have been done with basic sharpening and denoise plugins when scaled back down to 4K. I do think it's helpful for photography as there are some older images done with 3mp and 5mp digital cameras that couldn't be scaled properly otherwise, however, for video use, I think there's a good deal of development left to do for promised results. Now, the service option for Gigapixel AI for video is a good way to develop things but I would think Topaz Labs should be on the lookout as the competition could simply offer a plugin option at any time.