Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

What is 4k REDCODE RAW @27MB/sec?

Patrick Tresch

Patrick Tresch

Well-known member
Joined
Aug 2, 2007
Messages
5,808
Reaction score
86
Points
48
Age
55
Location
Switzerland, Lausanne
Is the REDCODE codec for 4k, what HDV would be for HD?
How can 4k 24p be for less than 27MB/sec?
4k 24/25/30p on SATA drives or flash cards?
This means huge compression!!!

Up today Spiderman has been made on a Da Vinci optionally streched to 4k... (Da Vinci Resolve 4K)

http://www.studiodaily.com/filmandvideo/currentissue/8139.html

Do you think FCP/Color could handle 4k?:help:
How do you color correct something you can't see in realtime?

If compared to HDV compression, would you finalise a project in HDV..? I think no. The codec is just enough to shoot with, but not to make CC, VFX, Keying, compositing....

There are codec wich are very efficient and virtually don't show any loss (like avid's DNxHD 10bit) but has anyone put REDCODE codec under pressure? How does it handle mutiple generations? Extreme color correction? Keying...

Thank you.

Patrick.
 
REDCODE RAW Not your father's compression. Compression is huge, but cutting edge. 10 bit log -full raster -wavelet based . Read up on wavelet here...

http://en.wikipedia.org/wiki/Wavelets

Like Cineform, it is considered to be visually lossless. Assuming that's true, the trade off is...well...no trade off. The advantage is you can store an entire movie's worth of rushes (say, a 2 hour movie, @ 10:1 ratio) on two 1TB hard drives. Uncompressed 4k? Don't even wanna think about it.
 
Tresch said:
Is the REDCODE codec for 4k, what HDV would be for HD?
How can 4k 24p be for less than 27MB/sec?
4k 24/25/30p on SATA drives or flash cards?
This means huge compression!!!

Up today Spiderman has been made on a Da Vinci optionally streched to 4k... (Da Vinci Resolve 4K)

http://www.studiodaily.com/filmandvideo/currentissue/8139.html

Do you think FCP/Color could handle 4k?:help:
How do you color correct something you can't see in realtime?

If compared to HDV compression, would you finalise a project in HDV..? I think no. The codec is just enough to shoot with, but not to make CC, VFX, Keying, compositing....

There are codec wich are very efficient and virtually don't show any loss (like avid's DNxHD 10bit) but has anyone put REDCODE codec under pressure? How does it handle mutiple generations? Extreme color correction? Keying...

Thank you.

Patrick.
Click to expand...

Do you really think that your the first person to think about these things and want to discuss them? Or are you just trying to stir up trouble?

Please do a search before posting. There has been plenty of discussion about all of your concerns.
 
HDV??? Oh no you di'n't

HDV??? Oh no you di'n't

No compression tech is perfect, but state of the art wavelet is a different beast. I would suggest you avoid comparisons to HDV until you actually work with RedCode RAW material. The compression ratio may look similar numerically but even if RedCode doesn't perform as well as we expect when pushed hard it is still a completely different animal.
 
Tresch said:
Is the REDCODE codec for 4k, what HDV would be for HD?
Click to expand...
no. don´t be afraid.

hdv is a terrible gop-codec on stoneage datareduction. colorgrading is catastrophic on hdv.

redcode is single frame datareduction, waveletbased, singleframe, raw data basing. much better to reduce, excellent results. all the files we used so far showed nice behaviour in colograding.

Just to give you the idea - DCI 4k, the digital delivery codec to the cinemas, is 250Mbit.

Tresch said:
Do you think FCP/Color could handle 4k?:help:
How do you color correct something you can't see in realtime?
Click to expand...
FCP is limited to 2K in 8bit RGB.
Some people recommend a hack to run FCP at 4k. I am not one of them.

Color is limited to 2k, but it has a solid bitdeptht of 10bit and in RGB.

If you want to edit in FCP, and then online 4k@10bit RGB, you need to have Adobe Premiere - this can do 4K@RGB@10bit.


Tresch said:
There are codec wich are very efficient and virtually don't show any loss (like avid's DNxHD 10bit) but has anyone put REDCODE codec under pressure? How does it handle mutiple generations? Extreme color correction? Keying...
Click to expand...
In our (few) tests redcode behaved much better than DNX. Next week, possibly, there should be lots of testimages and sequences around.
 
Sorry... I have not seen a lot of answers from RED.com:spidy:

http://www.red.com/workflow

But a lot of people talking for them!

Anyway, thanks a lot Dale for the Wavelet answer. And all the others who contributed with answers.

See you in post!

Patrick
 
Keep in mind the difference between mega-bits and mega-bytes. HDV runs at 25 megabits, which is a 60:1 compression ratio (correct me if I'm wrong - 1920x1080 pixels * 8 bits per pixel * 3 colors * 30 frames per second = 1.5gigabits per second compared to 25 megabits).

27 mega-bytes (which is what Redcode raw is...) is more than eight times the data rate for less than four times the pixels, and its bayer pattern, and I'll bet dollars to doughnuts that redcode is more effective than that horrific long-gop HDV.
 
ONLY FOR MATH LOVERS...!!!

Poets, please skip this quote!

Thanks Angus for your Bits/Bytes remark. I shurely never know when it is bits or bytes... But I wonder how can you put 27x8=216MB sustained on SATA drives without using raid? I think that I am still missing a point...

But however here the promised maths:

4520X2540 = 11480800 pixels
1440X1080 = 1555200 pixels
11480800 / 2073600 = 7,38 more pixels

HDV = 8bit YC (4:2:0) PAL (I'm not sure about signal incoding, YC or YUV)
4K = 10bit RGB

To account for 10 bit color, you can multiply 8 bit numbers by 1.25...
I don't have the figures for RGB/YC comparisons...

So 7,38*1,25= 9.2 so 25megabits / or bytes ...

I'm sorry to tell that we are not far far away from HDV compression (compared to uncompressed HD).

Now, I have not yet met mister Bayer nor mister Wavelet in post. Hope our dreams come true.

Sorry to be so sceptical. I truely would love to work with efficient tools but I've been long enough in this industry (as you all) to have been despointed from the gap between what was sold and the reality.

I really admire all your enthousiasm about this new hope.

Best regards.

Patrick
 
Tresch,

You math is slighty flawed by the pixel count. While Red has one sample per pixel, as all bayer cameras do, the HDV equivalent is YUV 4:2:0 (i.e. more samples per pixel.) The number of primary samples (pixel) compressed is 1.5 times the number you show. Yes, REDCODE is heavily compressed, but not quite as heavy as you show. But the numbers game is not a good predictor of quality, as it is easier to compress a larger image than a smaller one.
 
Patrick,

Take a look at the TIFFs at
http://www.cinematography.net/Red/comp-matrix.html

You can see an early version of Redcode compression. It looks very good in my opinion (e.g. visually lossless, better than uncompressed 4:2:2).
*The final version of Redcode will likely yield different results. (Probably better.)

There are also some reasons why it compresses better than HDV: (in no particular order)
A- Compressors vary in quality. MPEG-2 is getting old and is designed for lower-cost devices. Even in MPEG-2 codecs, some compressors do a better job than others (e.g. cinemacraft comapred to other codecs yield different results).
B- The Mysterium sensor has very little noise.
C- Redcode RAW operates on the RAW data. Operating on RAW means 1/3 the pixels to start off with. It also means that no image processing has been applied- image processing adds noise to the signal and makes it a little harder to compress.
D- There is a lot of optical low pass filtering on the sensor compared to HDV cameras. This makes the image easier to compress (less high frequency information; no/little aliasing).

E- HDV implements temporal compression, which does help it compress a lot better.
 
As I recall, nobody at NAB could tell the difference between redcode raw and raw footage even on the 4k projector.
 
Hence the term, "RED Revolution".
 
Doc... wasn't all the footage redcode RAW? (The crossing the line screening at NAB.)
 
I've seen this on a FAQ

Uncompressed RAW 4.5K at 24p comes in at roughly 323 MB/sec. This number is actually pretty low for 4.5K data [remember, RAW vs. RGB], but it's still a very big number. Such data rates would require snaking a large cable from the camera to a large [think refrigerator] RAID. Nobody wants a fridge connected to their camera.

So what is the solution? Scroll down to the REDCODE section below and see. I promise it'll make your day.

...it was not yet online...

But as for uncompressed formats? The only official number so far is roughly 323 MB/sec for 4.5K uncompressed RAW at 24p. A little speculation and math reveals the approximate data rates of other formats.
 
Patrick...

1- The practical thing to do is to look at the results of Redcode compression. You can download the TIFFs and key them and play around with them yourself.

You could get into theory, but theory is sometimes/often wrong. I'd take a look at the compressed images... they look great.

2- If you want to know the numbers and theory, here goes...

The sensor has an analog->digital convertor that takes the analog voltage off the sensor and converts it to 1s and 0s.
Cameras have different bit depths in the A->D convertors. Red's A->D is 12-bit. Some other cameras have 14-bit A->Ds, others 16-bit (Dalsa). Higher bit depth isn't necessarily better, as the extra bits can get swamped with noise.

The values coming off the A->D are linear to light. Most of the brightest values are visually indistinguishable from each other. After the A->D, the camera may apply gamma correction or some sort of gamma or log curve. This discards many values that are (mostly) indistinguishable from each other.

So this is kind of like compression... but is generally not referred to as such.

Redcode records the linear light values without gamma compression. So 12 bits of data per pixel are being recorded.

2B- In HDV, the camera will apply image processing. A color matrix is used to correct the color. This will generate values that lie outside the original range of values... this expands the amount of data that has to be compressed. Some of it may be clipped/discarded.

The R'G'B' values (gamma-corrected RGB values) are also converted to 8-bit Y'CbCr values, with headroom and footroom. Y'CbCr color space contains many illegal values, most of which are never used. This is all wasted space. Converting to Y'CbCr generally de-correlates the image (real world colors aren't that saturated) so this improves the efficiency of entropy coding (lossless compression is applied on top of the final values).

2C- Once the values are in Y'CbCr space, chroma subsampling is applied. Filtering is applied on the chroma values, and then 3/4 of them values are discarded (assuming 4:2:0).
If you started with 4 Y' 4 Cb and 4 Cr' values, then you end up with 4 Y', 1 Cb and 1 Cb value.

2D- Before chroma subsampling, the overall image (BOTH luma and chroma) is subsampled beforehand. 1080i HDV is subsampled to 1440x1080 from 1920x1080. And then chroma subsampling is derived from there... 4:2:0 on the 1440x1080 image yields 720x540 chroma samples.

2E- HDV may also be interlaced (though there is HDV that is not interlaced). If interlaced 4:2:0 chroma subsampling is used, then the effective chroma performance is not very good. The effective chroma performance is about a quarter vertically (and a half horizontally). Even though the numbers on the surface suggest that chroma performance should be halved vertically... effective performance is a quarter vertically.

2F- HDV uses temporal compression, which greatly increases compression efficiency.

2G- HDV only operates on (co-sited) RGB data. If the HDV camera's sensor is Bayer pattern, then the camera has to demosaic it and therefore increase the amount of data that needs to be compressed.


So in conclusion, there are different ways of counting the compression ratio. Do you consider gamma correction to be compression? It's up to you.

Or.... just look at (and/or test) the end results.
 
"Uncompressed RAW 4.5K at 24p comes in at roughly 323 MB/sec.... such data rates would require snaking a large cable from the camera to a large [think refrigerator] RAID. "

Actually, you can load up a MacPro with 8 SATA drives in a RAID 0 and get 323MBs pretty easily.
 
You must log in or register to reply here.
Back
Top