Fredrik Callinggard
Well-known member
Good for you
Welcome to our community
Be a part of something great, join today!
Uncompressed vs REDCODE
- Thread starter Jonas Drehn-Knudsen
- Start date
Good for you
stephan orlandic
Well-known member
- Joined
- Aug 4, 2008
- Messages
- 63
- Reaction score
- 0
- Points
- 0
Dear mr. Jarred,
first of all let me again clearly state, that I don't mean any offense to You, mr. Jannard or RED company.
The RED ONE has made a lot of money for us directly through camera rent, and indirectly through postproduction, and in general we are very happy with it. And we will continue to purchase more red Epic cameras in the near future.
I fully understand that You are being suspicious about people complaining about camera all the time.
And this has become a major issue on this forum, because too many people, non-owners, owners with very small experience, and many others, complain too much about great many things with questionable reasons.
The only reason I posted massage in this thread is that I have been working for a very long time in postproduction (15 years), and handling high res film images for 10 years now.
It seems that there are people here, who are confusing very often camera/sensor noise with compression artifacts.
And also there are people who say that they haven't ever seen any compression issues.
Although we are working with our red camera for 8 months, we are working with RED files for almost 12 months. And I think that we have been proficient with handling the RED files.
We had worked with RED on more than 50 high profile commercials (SWATCH, TELENOR, TETRAPAK, US STEEL, VODAPHONE/MOBILKOM AUSTRIA, ...),
10+ music videos (UNIVERSAL MUSIC, ...), three 30min feature film, ...
Our clients, have very big experience with 35mm film telecine or DI, and also with Phantom HD, Viper, and D20/D21 cameras.
Therefore, many of them have the ability to see when there is something in the images which they don't like (Noise, Compression, ...).
Also, our reference monitoring is 45.000 us$ SONY BVM 24" monitor, and 50.000 us$ DLP 3chip projector, which both provide more than a good insight into any image issues.
For REDCINE viewing we use calibrated Eizo CG241W, which is currently the best computer display money can buy.
Now back to Your reply,
You have mentioned few things:
1.Old builds with less efficient codec implementation.
This is not the case since we have been working with stable builds 15,16 and 17, and still the same things are visible.
2. H264 compression or any other type of compression
We don't work with compressed images. We are working at shots native resolution (4k,3k,2k) in fully uncompressed 16bit workflow.
Our RED files have been finished with tools like Autodesk Lustre, Smoke, Flame, Avid DS, and we keep it uncompressed all the time.
Sometimes our clients want REAL TIME 4k grading, with 10+ layers, and than we send them to BASELIGHT 8 facility.
After we finish the complete post, we deliver hdcam sr, hdcam or digibeta to
the clients.
3. User error
As I said previously we have some very good experience with RED files.
First of all,
We see the compression in the REDCINE, and we see it exactly the same in the 16bit tiffs or 10bit dpxs later in post.
Second of all,
we don't make any compromises when it is about image quality.
We only do FULL debayer, with no downscale. if it's 4k, than it's the 4k all the way.
When we started to encounter the compression issues, in some cases very heavy ones, we consulted with a number of leading european post houses, and we have learned that all of them had same or very similiar issues as we had.
For many cases, this issue can be solved, but there are cases when it's realy very complicated to resolve the problem.
In several jobs, we had to go 35mm and do a DI for commercials, because after tests, for those commercials and those clients the RED wasn't satisfying enough.
Kindest Regards
Stephan
stephan orlandic
Well-known member
- Joined
- Aug 4, 2008
- Messages
- 63
- Reaction score
- 0
- Points
- 0
As I said previously, neither me or other members here have any detailed info about setup, and what was going on at the projection.
So every comment made about it is nothing more than a speculation, which in general I realy dislike.
However, since there is a reference that nobody could tell a difference between the REDRAW and RAWCODE, I feel the need to try to explain some things.
As I said in my first post on page 3, myself and a great deal of people I know and who are from the same film bussiness, can tell You with no problem a difference between high res uncomressed images from different film scanners in both a digital file or on a projection.
And this is by all means a much more subtle difference than between RAW files and very heavy compressed 36MB/s REDCODE.
Also many of us were envolved in many digital (cinema) camera test shootings and their image quality comparisions.
I don't question technical knowledge or visual capabilities of Peter Jackson and his crew, moreover I have nothing but highest regards for them.
Not seeing the difference between those two files REDRAW and REDCODE, could mean three things from my point of view:
1. The hole crew can't see it because they lack the visual capabilities
(It is almost impossible)
2. REDRAW is not performing as all of us are expecting it to perform, meaning that REDCODE isn't so bad, but REDRAW isn't so good
(I realy do not want to believe this)
3a. Something went wrong in the preparations for projection.
If the projection was 4k uncompressed for both REDRAW and REDCODE, at that moment I know only of DVS Clipster 4K and Baselight 8 to be capable to playback 4k uncompressed. Neither of those two can't make any degradation issues which would make the difference between files not to be visible.
If the 4k playback system was jpeg2000 based, or something like that,
than conversion from REDRAW to this playback format caused it to be just like the REDCODE and of course the difference can't be easily seen on some projection.
3b. Original conversion of REDRAW files by Your side caused REDRAW's underperformance.
Kindest Regards
Stephan
Miguel "Macgregor" De Olaso
Well-known member
Some JPGs directly from redcine at Full quality (yes highly compressed JPGs, but the artifacts you see are there). Shot in 4K 16:9 RC36 5000ºK 80C filter with Master Prime 65mm:
It´s true that when watching RED footage in RGB we´ve never experienced any comrpession artifacts. Images look good and in the worst case scenario they get soft when there is a lot of detail in the image.
But when you need VFX...
It´s true that when watching RED footage in RGB we´ve never experienced any comrpession artifacts. Images look good and in the worst case scenario they get soft when there is a lot of detail in the image.
But when you need VFX...
Axel Mertes
Well-known member
Hi Macgregor,
we have seen similar issues, many times. Its not only specific to RED ONE, other cameras can show similar artefacts. But RED ONE does too.
It has a lot to do with the different sensitivity for R, G and B on sensors, plus how the specific channel is being processed. As the human eye has a different perception of R, G and B too, it is NOT really wise NOR fair to watch single color channels as grayscale images, because you switch your perception of that color then. You actually visually increase the artefacts presence this way.
The human eye can distinguish many more green levels than blue ones. So when watching a blue channel in anything but blue you force to show up the artefacts way more than watching them in blue.
That being said, the blue of the RED ONE is generally weaker than e.g. the green. This is much related to the sensor itself I suspect (maybe someone from RED can chime in hear). The new upcoming sensors are mainly addressing these issues by increasing sensitivity and bit depth (well, within physical limits).
On the processing chain: Maybe the three color channels get compressed with even different compression ratios, to gain quality on green, while loosing some (mostly not percepted) blue - I don't really know, but it could be the case. Its thinkable.
Its very hard to complain about the REDCODE compression to be an issue if having no chance of storing uncompressed for real comparison.
So my question would be if RED could implement kind of an uncompressed still frame mode to shoot a single frame in totally uncompressed RAW. This way it would be very easy to compare "lossless uncompressed" and "compressed" to find out the real difference. The best would be to store a single uncompressed RAW image and all along the very same image with the R3D REDCODE compression applied.
I would understand if RED is not interested in the above, because it would open up the door for many nay-sayers that RED CODE is doing something bad here and there, and that is apparently not always "visually lossless". But on the other hand it would put this whole discussion to a more objective and productive level of analysing images side by side and seeing the real loss, to improve from real world issues.
Any compression system is a compromise - we all have to have that in our minds, all time.
The compromise I see with RED is that we are enabled to work 4K in days of HD at an affordable price level for both acquisition and post production due to a compression being applied that melt down the massive data to something we can afford to handle.
At the same time I see the real interest of RED on improving RED CODE in terms of e.g increased data rates (not likely to happen on RED ONE, but the upcoming systems) and therefor less lossy compression. Depending on the way the compression works, there might be a minimum loss that can not be avoided at any data rate, but I am not a RED CODE programmer and therefor can only make assumption here. But the idea is to get less artefacts, whatsoever. Visually lossless, as much as possible. Its pure nonsense to not think that RED isn't concerned about this (no offense to anybody).
We also have to think about how many of the artefacts may be related to noise issues due to sensor "issues". The more perfect a sensor becomes, the less the compression artefacts will be that relate to noise. As the blue channel is weaker than the green channel, its just normal that blue noise will cause more compression artefacts.
I do believe that RED is aware of this. And I also agree on keeping this kind of discussions going on, so I really thank you for sharing the example images here and pointing our "there is something".
My intention is to give a little balanced comment on this to say "its not as bad as it looks in first place, but bad enough to think about addressing it" and "its not RED only".
If we get footage from a HD-CAM it may look even worse, shot under similar conditions. It'll be far away from RED ONE footage in many terms (except the F35 for instance, which is extremely good in low light conditions). In that context the RED ONE is a compromise.
When someone does not want a compromise at all then there are potentially other options. If money doesn't play a role, go and use an D21 or F35 to get potentially better color and less compression (but less resolution - another compromise in that context) or go for film.
I do not think that RED is owing us a CODEC with less artefacts, but they try to give that to us - isn't that a nice attitude?
RED says they are constantly improving on RED CODE. When was the last time I heard something like that from Pana & Sony? Did I ever? I don't think so...
They just released new ones with new systems.
Just my thoughts on this,
Axel
we have seen similar issues, many times. Its not only specific to RED ONE, other cameras can show similar artefacts. But RED ONE does too.
It has a lot to do with the different sensitivity for R, G and B on sensors, plus how the specific channel is being processed. As the human eye has a different perception of R, G and B too, it is NOT really wise NOR fair to watch single color channels as grayscale images, because you switch your perception of that color then. You actually visually increase the artefacts presence this way.
The human eye can distinguish many more green levels than blue ones. So when watching a blue channel in anything but blue you force to show up the artefacts way more than watching them in blue.
That being said, the blue of the RED ONE is generally weaker than e.g. the green. This is much related to the sensor itself I suspect (maybe someone from RED can chime in hear). The new upcoming sensors are mainly addressing these issues by increasing sensitivity and bit depth (well, within physical limits).
On the processing chain: Maybe the three color channels get compressed with even different compression ratios, to gain quality on green, while loosing some (mostly not percepted) blue - I don't really know, but it could be the case. Its thinkable.
Its very hard to complain about the REDCODE compression to be an issue if having no chance of storing uncompressed for real comparison.
So my question would be if RED could implement kind of an uncompressed still frame mode to shoot a single frame in totally uncompressed RAW. This way it would be very easy to compare "lossless uncompressed" and "compressed" to find out the real difference. The best would be to store a single uncompressed RAW image and all along the very same image with the R3D REDCODE compression applied.
I would understand if RED is not interested in the above, because it would open up the door for many nay-sayers that RED CODE is doing something bad here and there, and that is apparently not always "visually lossless". But on the other hand it would put this whole discussion to a more objective and productive level of analysing images side by side and seeing the real loss, to improve from real world issues.
Any compression system is a compromise - we all have to have that in our minds, all time.
The compromise I see with RED is that we are enabled to work 4K in days of HD at an affordable price level for both acquisition and post production due to a compression being applied that melt down the massive data to something we can afford to handle.
At the same time I see the real interest of RED on improving RED CODE in terms of e.g increased data rates (not likely to happen on RED ONE, but the upcoming systems) and therefor less lossy compression. Depending on the way the compression works, there might be a minimum loss that can not be avoided at any data rate, but I am not a RED CODE programmer and therefor can only make assumption here. But the idea is to get less artefacts, whatsoever. Visually lossless, as much as possible. Its pure nonsense to not think that RED isn't concerned about this (no offense to anybody).
We also have to think about how many of the artefacts may be related to noise issues due to sensor "issues". The more perfect a sensor becomes, the less the compression artefacts will be that relate to noise. As the blue channel is weaker than the green channel, its just normal that blue noise will cause more compression artefacts.
I do believe that RED is aware of this. And I also agree on keeping this kind of discussions going on, so I really thank you for sharing the example images here and pointing our "there is something".
My intention is to give a little balanced comment on this to say "its not as bad as it looks in first place, but bad enough to think about addressing it" and "its not RED only".
If we get footage from a HD-CAM it may look even worse, shot under similar conditions. It'll be far away from RED ONE footage in many terms (except the F35 for instance, which is extremely good in low light conditions). In that context the RED ONE is a compromise.
When someone does not want a compromise at all then there are potentially other options. If money doesn't play a role, go and use an D21 or F35 to get potentially better color and less compression (but less resolution - another compromise in that context) or go for film.
I do not think that RED is owing us a CODEC with less artefacts, but they try to give that to us - isn't that a nice attitude?
RED says they are constantly improving on RED CODE. When was the last time I heard something like that from Pana & Sony? Did I ever? I don't think so...
They just released new ones with new systems.
Just my thoughts on this,
Axel
Axel Mertes
Well-known member
Hi,
just to show everybody what I tried to explain above about viewing color channels not in their native color and what that means to visual perception, here the three channels in their native color.
The artefacts are there, unchanged, but way less visible.
Axel
red channel in red color:
versus
red channel in gray color:
just to show everybody what I tried to explain above about viewing color channels not in their native color and what that means to visual perception, here the three channels in their native color.
The artefacts are there, unchanged, but way less visible.
Axel
red channel in red color:
versus
red channel in gray color:
Axel Mertes
Well-known member
green channel in green color:
versus
green channel in gray color:
versus
green channel in gray color:
Axel Mertes
Well-known member
blue channel in blue color:
versus
blue channel in gray color:
versus
blue channel in gray color:
Eric Lange
Well-known member
AXEL, from a psycho-visual point of view I 100% agree!
However from an algorithmic point of view I don’t think the grey scale images go far enough in explaining why such artifacts trip up chromakey and other processes for compositing. The math (apart from the occasional tone curve) does not “look” at things from a human point of view. If you were to “look” at these images from the image processing algorithms point of view you would have to imagine each tone value represented as a Z height. I.e. a 3d landscape. Imagine you are the hapless convolution filter (a committee of automatons armed with slide rules sitting in a rickety raft trying to navigate through a giant landscape and seascape). To a spatially based image processing algorithm an exotic compression artifact looks much more like a 100 foot maze of canyon walls that are almost impossible to navigate directly through. You have to look at these things from the machine’s point of view. Unfortunately machines and their algorithms are not very smart (right now) and do not understand what is intended, I.e. what is obvious to a human being. Most algorithms still munch through individual channels and then compare notes later with their complimentary color channel (I.e. there is not a truly unified color representation.) Unfortunately automated processes in machine vision and image processing are still pretty dumb. In our software designs for 3d extraction from stereo pairs we first built the best manual 3d stereoscopic editing interface (we believe) for geometry extraction first as we know that automated processes live or die by how well you can edit and correct them when they go wrong or are not “seeing” what is intended. We read just as many neurology papers and books as we do programming and signal processing books, to look for clues as to how we could be doing these things better.
The fact that these artifacts are present in the RED one images seems to indicate to me that the RED team are truly brilliant engineers. They have taken everything to the max with the existing RED ONE sensor. Simply put NO COMPRESSION, NO CAMERA or you can shoot at 8fps for 4k. It seems that the on the fly compression is absolutely integral to the camera’s performance, and it seems that RED has steered a very brave and well considered course to deliver a 4k cinematic camera. Regardless of various implementations of REDCODE etc. I don’t think anyone has ever squeezed more out of a sensor and it would be very hard to improve upon without new hardware (I.e. the new line of products).
On a practical note, if some "shaggy" composites some times crop up, how long does it take in (man hours) to correct 30 seconds of 4k footage for chromakey? [Very interested to understand how much manual or hand correction most post houses can tolerate or handle?] (any answers to this question would be greatly appreciated)
Cheers,
Eric
However from an algorithmic point of view I don’t think the grey scale images go far enough in explaining why such artifacts trip up chromakey and other processes for compositing. The math (apart from the occasional tone curve) does not “look” at things from a human point of view. If you were to “look” at these images from the image processing algorithms point of view you would have to imagine each tone value represented as a Z height. I.e. a 3d landscape. Imagine you are the hapless convolution filter (a committee of automatons armed with slide rules sitting in a rickety raft trying to navigate through a giant landscape and seascape). To a spatially based image processing algorithm an exotic compression artifact looks much more like a 100 foot maze of canyon walls that are almost impossible to navigate directly through. You have to look at these things from the machine’s point of view. Unfortunately machines and their algorithms are not very smart (right now) and do not understand what is intended, I.e. what is obvious to a human being. Most algorithms still munch through individual channels and then compare notes later with their complimentary color channel (I.e. there is not a truly unified color representation.) Unfortunately automated processes in machine vision and image processing are still pretty dumb. In our software designs for 3d extraction from stereo pairs we first built the best manual 3d stereoscopic editing interface (we believe) for geometry extraction first as we know that automated processes live or die by how well you can edit and correct them when they go wrong or are not “seeing” what is intended. We read just as many neurology papers and books as we do programming and signal processing books, to look for clues as to how we could be doing these things better.
The fact that these artifacts are present in the RED one images seems to indicate to me that the RED team are truly brilliant engineers. They have taken everything to the max with the existing RED ONE sensor. Simply put NO COMPRESSION, NO CAMERA or you can shoot at 8fps for 4k. It seems that the on the fly compression is absolutely integral to the camera’s performance, and it seems that RED has steered a very brave and well considered course to deliver a 4k cinematic camera. Regardless of various implementations of REDCODE etc. I don’t think anyone has ever squeezed more out of a sensor and it would be very hard to improve upon without new hardware (I.e. the new line of products).
On a practical note, if some "shaggy" composites some times crop up, how long does it take in (man hours) to correct 30 seconds of 4k footage for chromakey? [Very interested to understand how much manual or hand correction most post houses can tolerate or handle?] (any answers to this question would be greatly appreciated)
Cheers,
Eric
Axel Mertes
Well-known member
Hi Eric,
thank you very much for your post, was a fun ride to read through
You should write some book, you have such a "visual" wording here and there, highly appreciated!
I agree completely on the mathematical view of such "noise artefacted" image data. Having studied computer science in the past my knowledge of programming image processing pipelines is not without fundaments. We see it every now and then when doing keying / blue box / green box stuff with the RED ONE. In fact we use RED ONE for the even the smallest projects (web, PAL, HD) as keying went EASIER with it than with the other traditionally used image sources (DigiBeta, HDV, Varicam, HD CAM) in most cases, because the extra in resolution does lead to an easier keying and downscaling compensates for most of the rest issues.
But if you want to do brilliant theater release work in 2K or 4K you clearly hit an edge of what can be done, as there is mostly no downscaling anymore that smooth out some issues. Then you had to deal with artefacts 1:1. That is the moment where I point people to the phrase "compromise".
Even when shooting on film people seem to ignore that it is not really a perfect image. When scanned you reveal the film grain. Its somehow like a random pattern (a random bayer pattern whould be a wrong description) that changes every frame. Grain would cause artefacts on compression as well. So what is the real alternative to RED ONE in this moment? If you put in "budget" into the equation most if not all productions will prefer RED over film IMHO. Having to rotoscope is most time an stupid unfriendly job for the newbies around, the hard school, but often not avoidable. If true uncompressed would prevent this - we can not tell until we have seen it.
So if one shoots film, the grain can be an as unglorious pattern as the compression artefacts are. And it changes every frame. I have no idea how your algorythms react on this compared to RED footage of similar resolution. Can you shed some light on this? It would be interesting for all of us to understand if there is an underlying problem that has not yet been discussed. In case film is "as bad as RED" I have no real problem with it. In case its way better, well, we can only hope it catches up at some point.
My previous message is just to demonstrate that the weaker red and blue does not mean to have a weak overall RGB experience in terms of an image perception. When having not VFX at all, just grading, its often simply very good footage. Better than most HD/2K scans we were used to before...
I suggest that RED is adding a mode to shoot a still frame that is going to be saved in true uncompressed RAW and in compressed R3D at the same time.
This would bring many discussions to an end or to a new level, less speculative, more analytic, more targeted.
This would allow us to analyze. Maybe all that it reveals would be that we need a new sensor with better blue and red response? Or that RED CODE has to be a higher data rate? Or that we need tech from outer space? We can't say that, its pure speculation. But I do see that RED put it on its roadmap to make better sensors (better than then potentially best sensor we can get hands on right now out there) for an upcoming promising product line. To me thats at least saying its being addressed.
When shooting green screen with an HD CAM or Varicam, we used to have an uncompressed editing system on-set and capture directly from the camera to bypass its compression chain. The quality was WAY better than the tape stuff. Unfortunately such an option does not exist for RED cameras as of now, and it feels unlikely to happen, nor to happen at reasonable cost (budgetwise). Jim clearly said "we do it if 20 put down money" - but the league is small where people can hand out this amounts.
Btw, it sounds pretty interesting what you noted here and there about your 3D stuff. We have a project in pipeline that is aimed to be shot in 3D for theatrical release. Maybe you can give me some infos offlist if you like, so I can hand this over to the producer and see if its interesting for us. We are very early in planning, so there is no tight schedule on the horizon.
Best regards,
Axel
thank you very much for your post, was a fun ride to read through
You should write some book, you have such a "visual" wording here and there, highly appreciated!
I agree completely on the mathematical view of such "noise artefacted" image data. Having studied computer science in the past my knowledge of programming image processing pipelines is not without fundaments. We see it every now and then when doing keying / blue box / green box stuff with the RED ONE. In fact we use RED ONE for the even the smallest projects (web, PAL, HD) as keying went EASIER with it than with the other traditionally used image sources (DigiBeta, HDV, Varicam, HD CAM) in most cases, because the extra in resolution does lead to an easier keying and downscaling compensates for most of the rest issues.
But if you want to do brilliant theater release work in 2K or 4K you clearly hit an edge of what can be done, as there is mostly no downscaling anymore that smooth out some issues. Then you had to deal with artefacts 1:1. That is the moment where I point people to the phrase "compromise".
Even when shooting on film people seem to ignore that it is not really a perfect image. When scanned you reveal the film grain. Its somehow like a random pattern (a random bayer pattern whould be a wrong description) that changes every frame. Grain would cause artefacts on compression as well. So what is the real alternative to RED ONE in this moment? If you put in "budget" into the equation most if not all productions will prefer RED over film IMHO. Having to rotoscope is most time an stupid unfriendly job for the newbies around, the hard school, but often not avoidable. If true uncompressed would prevent this - we can not tell until we have seen it.
So if one shoots film, the grain can be an as unglorious pattern as the compression artefacts are. And it changes every frame. I have no idea how your algorythms react on this compared to RED footage of similar resolution. Can you shed some light on this? It would be interesting for all of us to understand if there is an underlying problem that has not yet been discussed. In case film is "as bad as RED" I have no real problem with it. In case its way better, well, we can only hope it catches up at some point.
My previous message is just to demonstrate that the weaker red and blue does not mean to have a weak overall RGB experience in terms of an image perception. When having not VFX at all, just grading, its often simply very good footage. Better than most HD/2K scans we were used to before...
I suggest that RED is adding a mode to shoot a still frame that is going to be saved in true uncompressed RAW and in compressed R3D at the same time.
This would bring many discussions to an end or to a new level, less speculative, more analytic, more targeted.
This would allow us to analyze. Maybe all that it reveals would be that we need a new sensor with better blue and red response? Or that RED CODE has to be a higher data rate? Or that we need tech from outer space? We can't say that, its pure speculation. But I do see that RED put it on its roadmap to make better sensors (better than then potentially best sensor we can get hands on right now out there) for an upcoming promising product line. To me thats at least saying its being addressed.
When shooting green screen with an HD CAM or Varicam, we used to have an uncompressed editing system on-set and capture directly from the camera to bypass its compression chain. The quality was WAY better than the tape stuff. Unfortunately such an option does not exist for RED cameras as of now, and it feels unlikely to happen, nor to happen at reasonable cost (budgetwise). Jim clearly said "we do it if 20 put down money" - but the league is small where people can hand out this amounts.
Btw, it sounds pretty interesting what you noted here and there about your 3D stuff. We have a project in pipeline that is aimed to be shot in 3D for theatrical release. Maybe you can give me some infos offlist if you like, so I can hand this over to the producer and see if its interesting for us. We are very early in planning, so there is no tight schedule on the horizon.
Best regards,
Axel
Eric Lange
Well-known member
Axel, thanks for the input and kind words! I’ll try to answer as best I can…
Our first applications were in the recording of cultural heritage, I.e. fantastically complex surfaces over a very large area that have to be accurately recorded in 3d. So about ten years ago we were using a variety of very large plate photogrammetric cameras (like ZEISS UMK 130 x 180 mm transparency etc.) to shoot many stereograms of a site [the RED 617 might be nice for this]. For some smaller film formats like 4’ by 5” we would scan at 15 micron resolution and then down sample (just like what you were saying for RED at 2k). Back then very small film grain was about 7.5 micron, but it was certainly possible to see larger grain artifacts. For conventional pattern matching algorithms for 3d extraction, film grain is not such a problem as compression artifacts. In precision 3d work it is mandatory to use lossless(ly) compressed imagery/raw as even totally invisible compression artifacts can cause a lot of false matches. Conventional 3d/pattern matching algorithms unfortunately get fooled into believing that there are many more plausible matches on an image than there really are. So you pick an area on the left image and tell the algorithm to hunt around on the right image to find the unique corresponding match. So the algorithm hunts around the right image a bit like a blind man in an alley way with a stick prodding things to find some thing that seems similar to what he has been told to “look” for. Because compression schemes largely rely on substituting the actual image with a set of pre-prepared pattern blocks (that are much more efficient to describe computationally) what happens is that there is a much higher incidence of similarity between one part of the image to another part of the image. So what happens is that algorithm find not one unique match for corresponding scan lines it might find fifteen or twenty. This translates into false parallax measurements which then translates into noisy spurious 3d data points.
However the RED ONE camera has provided a boon for the stereo 3d movie business as digital recording devices are a hell of lot more stable than film. When film travels and stops in the film gate for successive frames you get a lot of jitter, and this translates to fatiguing stereo. So for dynamic applications I much prefer/reccomend digital especially as it is possible to calibrate for focal length distortion and sensor position far more accurately with digital sensors.
Our software however is designed to circumvent conventional approaches and problems to 3d extraction. Even with the best stereo imagery you cannot have perfect AUTOMATED 3d extraction as there is simply missing data that computer vision cannot correctly infer the missing parts. Our approach was two fold. Firstly we implemented a fully immersive 3d plotting environment where an operator can plot all of the polygons by hand, you just literally trace around in immersive 3d at the object you are looking at. It is very fast at producing clean models. The second thing we do is that we have a patented process for "baking" in stereo textures into 3d polygonal substrates acording to a special relationship. What this means is that you don’t need to extract millions of points, instead you make an invisible polygonal substrate in the BASIC shape of the stereo imaged objects (through our process) and then embed the stereo textures to the model. The stereo texture does all the work to represent high frequency three dimensional features and the substrate correctly disposes the model correctly in 3d explicit space. In other words it looks visually flawless and you can get away with absolute murder, however for some applications green screen stereo alpha channel in needed. This new type of spatial representation we call a Coherently Stereo Textured Model. It is a new spatial paradigm and data format. So we add auto assist algorithms to help speed things up. However we think manually plotted CSTMs are faster and cleaner than trying to extract millions of 3d points and then try to edit them into sensible surfaces. The human visual system is so much better at finding 3d matches in stereo than any automated algorithm right now. Our goal is to enable the user to get the highest quality 3d texture mapped model from stereo sequnces. Conventional systems tend to create a bunch of unusable noisy 3d data that is nearly unusable for cinematic applications. The CSTM has many applications and would be very good for 3d composites of complex surfaces and dynamic phenomena such as fluids, plasmas, explosions etc. The CSTM is also 400x more efficient rendering time over the same conventional complex representation and therefore has many applications in stereo gaming.
So for 3d extraction with RED (stereo) I would go manual with auto assist, and would prefer it to film. RAW/lossless would make automated 3d much smoother and less error prone. Never the less we are of course looking to build systems around the new Scarlet and EPIC.
We have mainly worked with 360 degree set ups but STATIC, and have been experimenting with dynamic applications for almost two years, (test data sets etc.). We are trying to seek out some representative stereo footage to test our software for the DI and post market, so we are eager to partner with folks that can shoot good stereograms. Right now we are looking to make sure our software can work with a wide range of stereographic systems.
Will P.M. some details,
Cheers V. much
Eric
Our first applications were in the recording of cultural heritage, I.e. fantastically complex surfaces over a very large area that have to be accurately recorded in 3d. So about ten years ago we were using a variety of very large plate photogrammetric cameras (like ZEISS UMK 130 x 180 mm transparency etc.) to shoot many stereograms of a site [the RED 617 might be nice for this]. For some smaller film formats like 4’ by 5” we would scan at 15 micron resolution and then down sample (just like what you were saying for RED at 2k). Back then very small film grain was about 7.5 micron, but it was certainly possible to see larger grain artifacts. For conventional pattern matching algorithms for 3d extraction, film grain is not such a problem as compression artifacts. In precision 3d work it is mandatory to use lossless(ly) compressed imagery/raw as even totally invisible compression artifacts can cause a lot of false matches. Conventional 3d/pattern matching algorithms unfortunately get fooled into believing that there are many more plausible matches on an image than there really are. So you pick an area on the left image and tell the algorithm to hunt around on the right image to find the unique corresponding match. So the algorithm hunts around the right image a bit like a blind man in an alley way with a stick prodding things to find some thing that seems similar to what he has been told to “look” for. Because compression schemes largely rely on substituting the actual image with a set of pre-prepared pattern blocks (that are much more efficient to describe computationally) what happens is that there is a much higher incidence of similarity between one part of the image to another part of the image. So what happens is that algorithm find not one unique match for corresponding scan lines it might find fifteen or twenty. This translates into false parallax measurements which then translates into noisy spurious 3d data points.
However the RED ONE camera has provided a boon for the stereo 3d movie business as digital recording devices are a hell of lot more stable than film. When film travels and stops in the film gate for successive frames you get a lot of jitter, and this translates to fatiguing stereo. So for dynamic applications I much prefer/reccomend digital especially as it is possible to calibrate for focal length distortion and sensor position far more accurately with digital sensors.
Our software however is designed to circumvent conventional approaches and problems to 3d extraction. Even with the best stereo imagery you cannot have perfect AUTOMATED 3d extraction as there is simply missing data that computer vision cannot correctly infer the missing parts. Our approach was two fold. Firstly we implemented a fully immersive 3d plotting environment where an operator can plot all of the polygons by hand, you just literally trace around in immersive 3d at the object you are looking at. It is very fast at producing clean models. The second thing we do is that we have a patented process for "baking" in stereo textures into 3d polygonal substrates acording to a special relationship. What this means is that you don’t need to extract millions of points, instead you make an invisible polygonal substrate in the BASIC shape of the stereo imaged objects (through our process) and then embed the stereo textures to the model. The stereo texture does all the work to represent high frequency three dimensional features and the substrate correctly disposes the model correctly in 3d explicit space. In other words it looks visually flawless and you can get away with absolute murder, however for some applications green screen stereo alpha channel in needed. This new type of spatial representation we call a Coherently Stereo Textured Model. It is a new spatial paradigm and data format. So we add auto assist algorithms to help speed things up. However we think manually plotted CSTMs are faster and cleaner than trying to extract millions of 3d points and then try to edit them into sensible surfaces. The human visual system is so much better at finding 3d matches in stereo than any automated algorithm right now. Our goal is to enable the user to get the highest quality 3d texture mapped model from stereo sequnces. Conventional systems tend to create a bunch of unusable noisy 3d data that is nearly unusable for cinematic applications. The CSTM has many applications and would be very good for 3d composites of complex surfaces and dynamic phenomena such as fluids, plasmas, explosions etc. The CSTM is also 400x more efficient rendering time over the same conventional complex representation and therefore has many applications in stereo gaming.
So for 3d extraction with RED (stereo) I would go manual with auto assist, and would prefer it to film. RAW/lossless would make automated 3d much smoother and less error prone. Never the less we are of course looking to build systems around the new Scarlet and EPIC.
We have mainly worked with 360 degree set ups but STATIC, and have been experimenting with dynamic applications for almost two years, (test data sets etc.). We are trying to seek out some representative stereo footage to test our software for the DI and post market, so we are eager to partner with folks that can shoot good stereograms. Right now we are looking to make sure our software can work with a wide range of stereographic systems.
Will P.M. some details,
Cheers V. much
Eric
the point is that it is eaiser to deal with the grain when keying than it is the compression. Also the thing with budget is an interesting one and as the person who started the thread mentions, clients choosing the kit, at the highend, when you consider all the expense of the whole production, the saving that red gives you is not that big a precentage. ie you still have to pay for all the crew/post etc. mainly you save in the transfer costs, but to these clients, they are not going to worry over saving a few thousands of dollars and risk the many thousands of dollars spent on the whole production if the results are compromised so they will tend to go for film...
and just to reiterate here is a very simple colour diff key math shot on green:
green - max(red,blue).
so you can see how it splits the channels and it is absolutely essential and fair to view them as above to look at the compression
Axel Mertes
Well-known member
Hi Paul,
I agree on most points, especially in high end productions. Controversy, in mid and low end productions the RED turns out to be a big improvement over traditional aquisition systems, due to often lower cost (if calculated fair) in post.
We shot nearly every project this year on RED ONE. Even small jobs for web or TV. Many green screen projects etc. It turned out that the extra costs for a RED compared to an HDV or Varicam or HD CAM are a good investment, because it was so much easier to key the RED footage in the end and downscale to the desired resolution. It was pretty much forgiving in many situations (if done right).
So it really depend on a specific situation. I do see it being addressed with the announced camera lineup. Maybe it can be improved somewhat on the RED ONE too, we would all be glad to see that, clearly.
About viewing the color channels as grayscales:
Its clearly fair to view them to exaggerate the artefacts and issues you have to deal with, no question. I just wanted to explain that the impact of those noise/artefact "errors" in the final RGB image is less visible when viewed in the native color - and that its probably part of the compromise equation. If RED would try to apply a much less strong compression to red and blue channels to prevent compression artefacts there will be two new issues: First of all, data rate will grow dramatically, and the results gained will potentially not always justify that data rate nor that the current equipment will handle it. Second, it might turn out that the sensor data is still noisy enough to not really solve the issue with keying.
Again, as long as we have not seen it on our own, its all speculation. My wish would be to have RED implement a still frame shooting mode that stores an uncompressed RAW and the same image as RED CODE. THEN we are in business of analyzing that (and I do see real world applications for this too, beside artificial discussions, e.g. stop motion) and making suggestions or comments from there.
I have no real idea of the physical limits, ie. I do not know how "clean" a blue or red channel can become at some point with advancing technology. Even if we capture EVERY photon, I still wonder how clean the blue channel would be?
Does anybody know the efficiency of the current sensors, in terms of x% of hitting photons get counted?
And then again, how much noise is added during readout and black noise etc.?
As a rule of thumb, good lighting is "key" for good keying results. We have seen bad and good lighted stuff, and it can be a big difference.
The bottom line is for me that the RED ONE gives me the best keying results of any camera in that price range if treated right. I am happy with that, anything that makes it better is just adding up to this, comfortably.
If poeple decide that for their projects, the RED ONE is not good enough, well, then its absolutely fair to use e.g. film or another camera, add additional acquisition cost but potentially equal or lower cost in post than for a RED ONE. Its an very economical and not so much ideological question IMHO. It all becomes a problem if poeple think that RED must be better for everything, that is a good wish, but not a reality, not even from REDs own perspective (just think about super high speed etc.).
But I do like that this discussion brings up some good points for others to know about. So if someone has a project that stands or falls with the keying issue, it needs to be considered carefully which camera to use and under what conditions. Some will just consider to have a potentially better lighting and stay with RED ONE, others may choose a different camera. We've seen many projects where poeple went to a shooting and returned with bad exposed blue/green screen stuff. They thought "we have so much bit depth - isn't it there? Its simply not that simple...
Axel
I agree on most points, especially in high end productions. Controversy, in mid and low end productions the RED turns out to be a big improvement over traditional aquisition systems, due to often lower cost (if calculated fair) in post.
We shot nearly every project this year on RED ONE. Even small jobs for web or TV. Many green screen projects etc. It turned out that the extra costs for a RED compared to an HDV or Varicam or HD CAM are a good investment, because it was so much easier to key the RED footage in the end and downscale to the desired resolution. It was pretty much forgiving in many situations (if done right).
So it really depend on a specific situation. I do see it being addressed with the announced camera lineup. Maybe it can be improved somewhat on the RED ONE too, we would all be glad to see that, clearly.
About viewing the color channels as grayscales:
Its clearly fair to view them to exaggerate the artefacts and issues you have to deal with, no question. I just wanted to explain that the impact of those noise/artefact "errors" in the final RGB image is less visible when viewed in the native color - and that its probably part of the compromise equation. If RED would try to apply a much less strong compression to red and blue channels to prevent compression artefacts there will be two new issues: First of all, data rate will grow dramatically, and the results gained will potentially not always justify that data rate nor that the current equipment will handle it. Second, it might turn out that the sensor data is still noisy enough to not really solve the issue with keying.
Again, as long as we have not seen it on our own, its all speculation. My wish would be to have RED implement a still frame shooting mode that stores an uncompressed RAW and the same image as RED CODE. THEN we are in business of analyzing that (and I do see real world applications for this too, beside artificial discussions, e.g. stop motion) and making suggestions or comments from there.
I have no real idea of the physical limits, ie. I do not know how "clean" a blue or red channel can become at some point with advancing technology. Even if we capture EVERY photon, I still wonder how clean the blue channel would be?
Does anybody know the efficiency of the current sensors, in terms of x% of hitting photons get counted?
And then again, how much noise is added during readout and black noise etc.?
As a rule of thumb, good lighting is "key" for good keying results. We have seen bad and good lighted stuff, and it can be a big difference.
The bottom line is for me that the RED ONE gives me the best keying results of any camera in that price range if treated right. I am happy with that, anything that makes it better is just adding up to this, comfortably.
If poeple decide that for their projects, the RED ONE is not good enough, well, then its absolutely fair to use e.g. film or another camera, add additional acquisition cost but potentially equal or lower cost in post than for a RED ONE. Its an very economical and not so much ideological question IMHO. It all becomes a problem if poeple think that RED must be better for everything, that is a good wish, but not a reality, not even from REDs own perspective (just think about super high speed etc.).
But I do like that this discussion brings up some good points for others to know about. So if someone has a project that stands or falls with the keying issue, it needs to be considered carefully which camera to use and under what conditions. Some will just consider to have a potentially better lighting and stay with RED ONE, others may choose a different camera. We've seen many projects where poeple went to a shooting and returned with bad exposed blue/green screen stuff. They thought "we have so much bit depth - isn't it there? Its simply not that simple...
Axel
Fredrik Callinggard
Well-known member
There's a big difference between that and arguing about RED vs FILM, which this thread were more intended for.
Axel Mertes
Well-known member
Fredrik,
thats true what you say about it beeing a big difference.
But the thread was primarily about Uncompressed vs. RED CODE. And in that context there is the assumption / hope that keying an uncompressed RED RAW would be so much easier than keying a RED CODE RAW.
We all know that it is very unlikely to see a RED RAW uncompressed recording in near future, and we brought up arguments that let us at least leave with some doubt if uncompressed will be the solution or not for the issues seen. But we all seem to agree that any improvement on this is cool, but there are situations where poeple prefer film - from a professional non-ideological stand point, putting other variables into the equation.
I made a wish to RED here that would finally allow us - the users - to make comparisons on this subject on our own. What do you think about THAT?
My interest for this wish to be implemented would not really be to make such comparisons, but to "abuse" the RED ONE for stop motion work. The reason is simple: It does less flicker than typical DSLR cameras used nowadays. We did a lot of stop motion in the past, and the RED would be a cool camera to do so. And as there is no time issue, I could do long exposures (many photons get counted) and could store the results uncompressed, as there is also no data rate issue.
I have no idea about the internal design of the RED ONE in that particular case, but for still frames I simply hope they can do it. And the side effect would be that we all get an idea if and how much RED RAW uncompressed would change the world.
Hey, THAT would make the decision easier for those who want that uncompressed recorder to put down their deposits as Jim originally proposed. They would be able to see what they get. No cat in a bag anymore!
Imagine, the unexpeced uncompressed recorder would finally be build and you can rent one if you like. That would be a satisfying solution IMHO. But I can live without it for now.
What about that?
Axel
thats true what you say about it beeing a big difference.
But the thread was primarily about Uncompressed vs. RED CODE. And in that context there is the assumption / hope that keying an uncompressed RED RAW would be so much easier than keying a RED CODE RAW.
We all know that it is very unlikely to see a RED RAW uncompressed recording in near future, and we brought up arguments that let us at least leave with some doubt if uncompressed will be the solution or not for the issues seen. But we all seem to agree that any improvement on this is cool, but there are situations where poeple prefer film - from a professional non-ideological stand point, putting other variables into the equation.
I made a wish to RED here that would finally allow us - the users - to make comparisons on this subject on our own. What do you think about THAT?
My interest for this wish to be implemented would not really be to make such comparisons, but to "abuse" the RED ONE for stop motion work. The reason is simple: It does less flicker than typical DSLR cameras used nowadays. We did a lot of stop motion in the past, and the RED would be a cool camera to do so. And as there is no time issue, I could do long exposures (many photons get counted) and could store the results uncompressed, as there is also no data rate issue.
I have no idea about the internal design of the RED ONE in that particular case, but for still frames I simply hope they can do it. And the side effect would be that we all get an idea if and how much RED RAW uncompressed would change the world.
Hey, THAT would make the decision easier for those who want that uncompressed recorder to put down their deposits as Jim originally proposed. They would be able to see what they get. No cat in a bag anymore!
Imagine, the unexpeced uncompressed recorder would finally be build and you can rent one if you like. That would be a satisfying solution IMHO. But I can live without it for now.
What about that?
Axel
Rudi Herbert
Well-known member
- Joined
- Jun 1, 2007
- Messages
- 1,433
- Reaction score
- 0
- Points
- 0
That sounds VERY interesting indeed. I suspect that a lot of the major rental houses would not hesitate to buy the uncompressed recorder and have it available for those high end productions that wish it. However, as you say, the determining factor in making that decision would be to have incontrovertible proof that uncompressed is that much better than REDCODE, which nobody knows, except perhaps for some people at RED. Another more important point is actually the fact that we are discussing REDCODE 36 vs. UNCOMPRESSED, and why? I don't want fully uncompressed, but something with as little compression as possible that still can be viably recorded and handled in post. I've said that I'd be happy with CF cards recording 2-3 minutes instead of 7-8 or to work with color accurate proxies and do a long render at the end of the project, since I don't really need to work with original 4K sources at real time, who honestly needs that? Anyway, uncompressed may be too much, but REDCODE as it stands now, may be too compressed and its flaws show under certain conditions, that's all. Less compression would be the ideal solution to this small but undeniable problem.
Eric Lange
Well-known member
Everybody has their own applications worries and agendas. I think the scope of the thread is O.K. I am thoroughly embarrassed by the length, agenda and boringness of my last post; I think its very difficult to discuss “communally” complex subjects with many related branches. Things can get a bit retarded/autistic as quite often people glom onto tenuous parallels to a particular topic as people are so desperate for useful information especially to avoid costly or risky mistakes. A thread that says UNCOMPRESSED VS REDCODE seems to get a lot viewings and it seems to be something that people are not clear on apart from RED.
However I do think Jonas and Anders message has gotten through loud and clear, and I have no doubt that RED is absolutely aware of these situations.
One of the things that AXEL you seem to want is a RAW/ lossless image for stop motion animation and for easy workflow high res composites. I also hear you when you say a single frame RAW and uncompressed image would be great as a basis of comparison. However, I get the impression that the compression scheme in RED is almost absolutely integral to the architecture of the system. I.e. its not possible to output an uncompressed RAW. Nobody outside of RED has every seen a processing schematic for the camera, but it seems to me that the camera does not wait for a whole image to be buffered somewhere and then wait for it to be compressed. Some other type of first stage compression may be happening at the very early acquisition stage of perhaps groups of scan lines; (I maybe completely wrong and out to lunch here?). However it does seem probable that uncompressed RAW even outputting one frame is not possible without screwing up the whole architecture of the camera. I feel sure that if it was possible then this would have been a feature on the camera, rather than just an oversight or an application of the 80:20 rule.
So this thread has been very helpful to me as it tells me that I need to carry out very specific test shoots before any major production or project (I.e. part of research budget/pre production planning). This thread also seems to indicate that 3:1 compression may indeed cure most of the chromakey issues that people sometimes experience (when the new line of cameras comes out). The only thing that is left is whether there is a preference for film or the D21. Would be interested to learn what the work flow of a RAW D21 is like from a practical standpoint compared to RED, and is it any different from scanned film?
[AXEL would be interested to hear of some of the problems you have with standard digital stills cameras, we re-engineer different types of stills cameras for stereo work, would be interested to see if there was something we could solve?, Feel free to P.M. me off list etc.]
Cheers,
Eric