Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Realtime GPU accelerated debayer

L

lordtangent

Well-known member
Joined
Oct 30, 2007
Messages
62
Reaction score
0
Points
0
I saw a GPU accelerated debayer software at Cine Gear Expo in the Dalsa booth. It works today. Realtime 4k debayer. The quality was exactly the same as the much slower software only decode. They are using a "hybrid" debayer approach, meaning the debayer more than one way and somehow adaptively combine the output of each to get the best results possible. Did I mention it's realtime? Yeah, I was quite impressed.

A lot of people liked the idea of some sort of accelerated RAW processor when I did a survey a while back. But it didn't occur to me most people already have a really good co-processor for the purpose in their machine. Red should do something similar to what Dalsa was showing. It would satisfy the need for an accelerator without requiring people to buy special hardware (beyond perhaps a compatible commodity graphics card.)

If Red built the GPU acceleration right into the SDK any developer who used the libs to read the R3D files would get the acceleration.

Comments?
 
but if fcs3 would read r3d files natively, and fcs has accelerated h.264 export, it would be even better.

also, future cameras can have video cards doing all the hard work and keeping the camera cooler and faster the process that would allow a better more stable camera and maybe make more fpss.
 
REDCINE and SCRATCH are GPU accelerated today. But the debayering is not done on the GPU. It's done on the CPU.

There are many, many ways to debayer and demosaic an image. Unless there is dedicated hardware involved, I seriously doubt they are doing a full debayer/demosaic from 4K in realtime.

Also, for debayering/demosaicing, GPU acceleration won't help much. All that work is typically done on the CPU. GPU threads are great for color grading, which is why when you adjust a control on REDCINE and/or SCRATCH, the feedback is instantaneous.

Lucas
-----
ASSIMILATE, inc.
LA, CA, USA
 
I hate everything realtime. If a debayer is labelled "realtime", I don`t want to use it for final output, period. Lets make RC stable and reliable, not realtime. In my head I usually swap the word "realtime" with "compromised". Such compromises can be useful (and quality good) for editing, but not for final output (unless you`re up against a time deadline).
I`ve personally designed a few demosaic algorithms in the past, and the numbers of calculations can get pretty large, especially if you want a robust and adaptive demosaic that can deal with everything you throw at it.
 
Ronx, "robust" is the key word there :-)
I've had demosaic running on GPU since before RED started, but the current CPU one is better. Also, pulling 4k data back off the GPU is still a bottleneck.

Graeme
 
with some radical changes to how the debayering works, u can take serious advantages from a gpu accelerated hardware. u would store the data in a completely different way, and maybe add more data to be stored.

then u can use gpu accelerated hardware to process this data when transforming the data into something nle friendly.

that would make the most out of everything and faster the process and make a faster camera and maybe, u could add more fpss in full frame cameras in the future.

but, gpu accelerated in export would be nice in redcine(if it's not implemented)
 
Am I correct in thinking that the amount of RAM is the least important thing here [I'm talking about a system with 4GBs of RAM].
 
Graeme Nattress said:
Ronx, "robust" is the key word there :-)
I've had demosaic running on GPU since before RED started, but the current CPU one is better. Also, pulling 4k data back off the GPU is still a bottleneck.

Graeme
Click to expand...

Agreed. I had a similar 4k debayer to 4k monitor working at another unnamed company I used to work at about 2 years ago. To get realtime you are limited in the number of operations you can do and that limits the debayer quality. Beyond a somewhat simple debayer you start to lose the benefits once you factor in texture uploads and downloads and an increasing number of memory accesses and branches which aren't super fast on gpus like matrix multiplies are.

Also, going from uncompressed to gpu is alot different than decompressing and then going to gpu.

On top of that, supporting more than a couple of graphics cards is a nightmare as no two models really function the same.
 
Well guys. You can keep preaching to the choir that your way is the best. I'm just telling you what I saw. To my well trained eyes (I've been a professional CG lighter and compositor on TV and features for 12+ years) there were no compromises on quality. As I mentioned, they even claim to be doing some sort of hybrid debayer... it is supposed to actually be better than a simple single algorithm debayer. To be honest decoding wasn't "real-real-time" more like 12 FPS. But 12 FPS 4K debayering is damn good in my book. (If Adobe Camera RAW is any indication of the speed of software only debayering) The Dalsa guys told me the speed drop to 12 FPS was only the result of some improvements they recently made to the quality of the algorithm that had yet to be optimized. It ran real-time with the older less good algorithm. They anticipated they would be able to get it back up to realtime with some re-factoring.

Now I realize RC needs to decompress the wavelet stuff before you even get to the debayer stage. I don't know how CPU bound that is making the process. Maybe speeding up the debayer in the case of RC is just pointless. It's your call I guess.

I understand that supporting more than one family of graphics card is a pain. But a requiring a user to buy a specific graphics card is to get real-time is nothing compared to the cost of the number of render nodes they would have to buy to get similar performance software only.

2 years in "GPU years" is like 100 years in "Human Years". GPUs are still advancing faster than even CPUs.

I just learned last night that Iradas is ALSO doing GPU accelerated de-bayering. So this isn't some dead end thing that some clueless company is chasing after. And people certainly want more speed. I know that when I'm doing wholesale de-bayering on my Scarlet footage I'm certainly going to want it.

A while back I proposed the idea of a RC accelerator based on the Red Ray guts. Lot's of people thought it was a good idea. That might also be another possibility for speeding up final renders that doesn't require tons of additional R&D on RDC part. In fact, if you engineered Red Ray correctly, the "Red daughter Card" in the Red Ray could be a component that is identical to a "Red Card" for a PC. For example, a PCIe daughter card in Red Ray that could just as easily be installed in a regular PC or Mac and driven though the RedCode API if available.
 
Graeme Nattress said:
Ronx, "robust" is the key word there :-)
I've had demosaic running on GPU since before RED started, but the current CPU one is better. Also, pulling 4k data back off the GPU is still a bottleneck.

Graeme
Click to expand...

Thats why you`ll never hear me asking for real-time anything, and why I have four machines. Keep up the good work :)

Also (Graeme/Deanan), off debayer side of topic, but on the "realtime one".. what do you make of the downsampling problems some users claim to be having converting 4k->SD?

Is it another case of speed over quality in products aimed at consumers? I find it hard (if not impossible, but I haven`t got it running yet!) to believe RC doesn`t do a good job at such a trivial task (compared to debayering), especially with the other stuff going on at Red, as some have suggested. I actually wrote a little freeware downsampling tool a few weeks ago in response (actually just the user interface, and crude 48bit TIFF import was added to old code ready for my R1), and created this page for it (for others):

http://helloconradgaunt.googlepages.com.

Its all floating-point batch image processing, and I`ve thrown all kinds of zone charts and zig zags at it to remind myself (modestly) how robust it is ;) Anyway..

I ask because the blur resize blur approach some users say you`ve suggested seems like a real hack to me, and I`m sure meant as a temporary solution until software catches up with the 4k idea, but the threads keep popping up on the subject? The gaussian/circular approach to dealing with discrete samples is sound, but the blurring should surely be an integral part of the downsample function?
My approach simply fires every pixel from the source image at the destination image and keeps track of how many times each destination pixel is hit by a source pixel (and calculates and stores the intensity of any hits), then averages out the result for each new pixel at the end of processing, which avoids users having to use a two stage blur approach (especially since most gaussian blur functions in realtime NLEs don`t allow users to input an accurate floating-point radius value anyway), and is much closer optimal. Its fairly fast and works great on big images that need to shrink. I wrote my code to deal with 3k DSLR timelapse footage some years ago because I didn`t like what premiere 6 was doing at the time, although I tested Prem.Pro2 recently and that seemed much better (although Adobe seem to like changing contrast levels and other subjective stuff?)
Anyway, I don`t remember ever seeing a post from anyone at Red on any of the downsampling threads, and I can`t test RC myself yet, but whats your take on this matter? Can RC do a HQ downsample (now or when out of alpha), or are some people just convinced for whatever reason that only expensive hardware, designed mainly to be fast, can produce high quality output? :detective2:
 
Downsampling is always a trade between:

aliasing,
sharpness,
ringing.

Pick any two. And there's nothing myself, RED, nor anyone else can do to change the basic math of how downsampling works, and the above trade-offs.

Graeme
 
http://graphics.cs.williams.edu/papers/BayerJGT09/

This is a GPU debayer implementation, written in way back in 2009.
From the abstract:
"can filter 40 simultaneous HD 1080p video streams at 30 fps (2728 Mpix/s) on current hardware."

Of course, this doesn't factor R3D decryption and JPG2000 decompression (are they one step?), but I'm sure it's way more than possible.

My observation is RED's biggest problem with producers not wanting to shoot Epic is real-time playback without a Rocket. If GPU accelerated debayer was an option, this would be a non-issue.
It's my feeling that RED is losing marketshare to cameras that are perceived to be easier to work with in post, for whatever reason. Is it to sell Rockets? I dunno.
 
Hi Graeme - correct me if I'm wrong, but I believe the issue with GPU decoding is not "debayering" but decoding wavelet compression - the back and forth required in transferring that amount of data into the memory buffer, having the card parallelize the routine and pass back data - The efficiency is not there because of all the data "handholding" or compartmentalization going on in laymen's terms.

I'm curious if the Xeon Phi implementation offers any promise over shader-optimized chips? It is interesting to see how they've implemented a "network" style path for transferring data through the card driver - I've been wondering if that will still be a bottleneck...

Thanks for all your hard work BTW - I am currently a very happy FCP X/Scarlet user... Anxious to see what next years crop of multicore hardware will bring.
 
Last edited:
I just saw the real real time debayering of the cinema dng files from the Aaton Delta (awesome camera btw), this is done with the gpu of a quadro 5000, it's a full debayer of 4k image at 24fps (or 25fps), doesn't use cpu. Of course the files are not compressed and so it's only debayering, but this is the way to go in my book, real real time debayer with gpu or cpu (i don't really care) and real real time decompression with gpu or cpu, but not with additionnal proprietary card (for 5K 2x redrocket).
 
You must log in or register to reply here.
Back
Top