Will data rate vary with image size?

[Stokestack]

Hi all.

I was reading the Cinematography mailing list and someone asked why you'd shoot 2K on the Red.

It remains to be seen what kind of quality Red'll get out of 25 MBps. Also, there's the view that the "real" resolution of a single Bayer chip is less than 4K. So why bloat the data payload if you're really only getting 3K's worth of resolution, especially if you only need 2K?

The question is, which will look better:

4K --> 2K @ 25MBps
4K @ 25MBps --> 2K

I guess that depends on the kind of artifacts wavelet compression produces, and whether they'll be more obscured by downrezzing than they would be reduced by simply recording a smaller frame at the same bitrate. Does anyone from Red have an idea on this? And that's assuming that the camera always encodes at the same bitrate, regardless of frame size. Is that true?

I guess this trade-off is hard to evaluate because there's no downrezzed 2K raw currently.

 

[I Bloom]

Hi all.

I was reading the Cinematography mailing list and someone asked why you'd shoot 2K on the Red.

It remains to be seen what kind of quality Red'll get out of 25 MBps. Also, there's the view that the "real" resolution of a single Bayer chip is less than 4K. So why bloat the data payload if you're really only getting 3K's worth of resolution, especially if you only need 2K?

The question is, which will look better:

4K --> 2K @ 25MBps
4K @ 25MBps --> 2K

I guess that depends on the kind of artifacts wavelet compression produces, and whether they'll be more obscured by downrezzing than they would be reduced by simply recording a smaller frame at the same bitrate. Does anyone from Red have an idea on this? And that's assuming that the camera always encodes at the same bitrate, regardless of frame size. Is that true?

I guess this trade-off is hard to evaluate because there's no downrezzed 2K raw currently.

Just put "I think" at the beggining of each sentence below:

I'm assuming you mean 2K RGB but also 12 bit linear (which I don't think is in the current specs, its only 10 bit 1080p, almost the same). Because as far as I know 2K RAW (and by RAW we mean Bayer pattern) only comes from windowing the chip. Scaling RAW to smaller RAW isn't something that I think anyone does.

There is compression on the scaled RGB. You can get it with no compression only from the optical output. So I don't think that the scaled RGB is 25MBps, I think it must be less. So its not a tradeoff, it's really just less, but not exactly one quarter of the data as would be the case with 2K windowed. Does this make any sense?

I'm hoping Mr. Lohman can chime in with the scoop, but the thing is, I don't think they are even working on RGB yet, so I doubt the answer is even available.

IBloom

 

[Brook Willard]

First, scaling RAW data is impossible [AFAIK]... hence the lack of a 2K scaled RAW option.

As for your worries about the "actual" resolution of the 4K sensor regarding the Bayer pattern... I'd recommend a search. It's been discussed at length. If not, I'm sure Graeme will be here shortly to teach us all a little more.

If you're worried about the kind of quality they can squeeze into ~25MB/s... well... just ask anybody who's been to any of the screenings.

I think you're asking about compression before or after scaling to achieve a 2K resolution... correct me if I'm wrong.

I think these are the workflows you outlined:

• Shoot 4K REDCODE RAW @ ~25MB/s... then scale to 2K in post.
• Shoot 4K uncompressed RAW... then scale to 2K and compress using REDCODE.

One important thing to understand is that the 2K end result for either of these equations is not RAW data... as RAW data cannot be scaled [again, AFAIK]. So either way you'll be losing some data in the transformation [beyond the data you'd lose by dividing the resolution by four].

I think the determining factor of which will look better after the scale [assuming an identical scaling algorithm] is the nature of the compression in the former example. If you scale to 2K in post [from 4K REDCODE RAW] and then compress to REDCODE [RGB?] afterwards... yes, that will technically be lower quality than shooting uncompressed RAW 4K and scaling in post. That's just the nature of the beast. If you compress twice, you're bound to lose more than if you just compress once. Here's a more specific workflow to outline what I'm saying:

• Camera shoots 4K ------ compress with REDCODE ------- scale to 2K in post ------ compress with REDCODE
• Camera shoots 4K ------ scale to 2K in camera ------- compress with REDCODE

As you can see, the first option has two generations of compression and the second option only has one. Higher quality, right?

I can see where the logic of adding uncompressed 2K scaling in camera to avoid this second generation of scaling comes from. Unfortunately, since RAW scaling isn't possible [one more time, AFAIK]... I don't think that would work in any reasonable way. You'd have to scale to a new format [lets say REDCODE RGB] and you'd lose all of the benefits of RAW... as well as roughly tripling your data rate.

So you accept that you have to go to RGB... because we're talking about scaling. So why isn't the scaling possible in camera? All of the techno mumbo-jumbo I don't understand aside... I see more control in the "scale in post" route. Here's another vague workflow:

• Shoot 4K REDCODE RAW ------ grade to your heart's content in post using all tools available to you ------- scale to 2K ------- compress [or don't compress] as you see fit
• Shoot 4K ------- set RGB settings in camera ------- bake in RGB settings and scale to 2K ------ compress 2K in camera

The first option here seems a lot nicer. You have a higher quality "digital negative" to work with, you have all of your favorite tools available for grading [with all of the RAW data intact], you get to scale using any algorithm available to you and you get to choose whether or not to add a second generation of compression.

The second option seems to suck. You have to scale in camera [using whatever scaling algorithm this theoretical feature would use], you have to compress to REDCODE RGB in camera [based on whatever settings you enter into the camera... no more grading using the tools you're used to] and you still end up compressed. If you decide to grade this footage again in post, you won't even be working with all of the data originally available... because you baked it into an RGB format. It just doesn't seem desirable at all.

Lets say that the 4K REDCODE RAW data rate is 25MB/s. That would mean that the 2K REDCODE RAW data rate would be roughly 6-7MB/s [this data rate would only apply to "windowing" or "cropping" the sensor... not what we're discussing here]. It would also mean that the 2K REDCODE RGB data rate would be roughly 18-22MB/s [give or take a lot depending on how the compression really works]. That means that my wild speculation 2K RGB format is only 10-15% less data than the 4K RAW format... a format with four times the resolution and all of the benefits of RAW data. So if the goal is to have a lower data rate, well... it's not really there.

This is all speculation, I could be wildly off-base on everything I've written here. Did I come close to what you were asking?

 

[Gavin Greenwalt]

Couple points to elaborate on what brook said.

If 4k Bayer is actually 3k RGB then...
2k Bayer is actually 1.5k RGB so if you want a 2k deliverable, you've dropped in this theoretical simplification below 2k.


2K RAW is 1/4 the data of 4k RAW however, if you want to down-rez in camera you have to multiply by 3 since you have to de-bayer (otherwise you're at 1.5k again and not really 2k). (technically you can rescale a RAW image, but beside it dropping you below the 'true' 2k standard you set out to achieve in the first place, it would very processor intensive on the camera side.

So the *real* difference between shooting 4k RAW and 'true' 2k is:

25MBs(1/4)(3) = 18.75 MBs (2k RGB)

So yes, it is true, you are saving space by going straight to 2k RGB. However going with our made up values of resolution loss from a bayer capture you've lost 33% of your resolution (3k -> 2k) while only saving 25% of your bandwidth (25MB -> 18.75MB)

And the moment you start to add any sort of stabilization, reframing or scaling you're going to drop below 2k. The '3k' image gives you a lot of room for manipulate and resize while still sampling at a full 2k resolution.

If you're a purist and don't to do any reframing/stabilization... etc then 2k RGB definitely the way to go.

---------------------------

That was the mathematical explanation of why 2k RGB really isn't a very cost-effective shooting format. Also factor in that with wavelet compression the compression only throws data at the parts that need it. So if you shoot a large low-frequency white wall and compress it into 4k RGB and 2k RGB you won't see a 1/4 drop in bandwidth because if it's sufficiently flat both the 4k and the 2k might only be using a '1k' resolution level to encode. If parts of the image don't need 4k worth of data; wavelets will discard the 4k 'layer'. You can almost think of a 4k wavelet encoded image as a composite of a .5k, 1k, 2k and 4k image stacked on top of each other. In areas with no high-frequency detail the compressor cuts out the layers until it gets down to a level that's sufficient to represent the image. Technically speaking that's innacurate, but from a conceptual level it's an easy way to understand what's going on.

 

[I Bloom]

One other thing I'd mention is that for each RAW pixel we are storing just 12 bits of information. For each RGB pixel we are storing 12x3 bits (or 10x3 depending). So a 2K RGB image is actually 1/4 the total info in a 4K RAW image x 3, only 3/4 of the total information.

Now according to the specs, we are really talking about 1080p, RGB, so its a little less. But at this point it doesn't make much sense to me to shoot this format. Since again its REDCODE. The 720p option does make a little more sense, though only for increasing the amount of footage on a drive.

But what does make some sense to me. Is to have the option, not compress these formats with REDCODE, but instead with standard HD compression, to standard formats and color spaces that we know and love (?) and possibly even to an Avid native format as well. The point being to save labor when the delivery format cannot be wavelet, i.e. with clients who have established post productions pipelines, who aren't ready or wiling to change.

Please just remember that in the world of broadcast, producers hate color correction, THEY WANT IT BAKED IN! They also don't want lots of data, they don't want to upgrade their post gear. You could roll your HD-SDI output onto a deck, but that is alot of expense and adds a capture step to post.

I think REDCODE is an incredibly smart and beautifully concieved idea, and I will shoot that format as much as I can. But I know I'll be in very good shape if I can sometimes just deliver exactly what people need or want, straight off the camera.

I think I'm repeating myself on two different threads now. Apologies.

IBloom

 

[Gavin Greenwalt]

Does anybody have any information or knowledge on 'tape' format licensing fees?

Would it cost anything to use a DVC-Pro HD codec? Is there an HDCAM (or SR) codec at all?

RED + DNxHD could be a very attractive option for fast turn arounds.

 

[Brook Willard]

Does anybody have any information or knowledge on 'tape' format licensing fees?

Would it cost anything to use a DVC-Pro HD codec? Is there an HDCAM (or SR) codec at all?

RED + DNxHD could be a very attractive option for fast turn arounds.

I think that REDCODE is the only supported compression format in-camera. You can recompress in post if you require some other codec.

As for Avid codecs, I don't know if they're supported out of REDCINE... I bet Avid wanted to charge for it.

 

[Cail Young]

As for Avid codecs, I don't know if they're supported out of REDCINE... I bet Avid wanted to charge for it.

If you can get the DNxHD codec as a Quicktime component for free like the Meridien codecs, then it shouldn't be an issue.

 

[Brook Willard]

Hence the "I don't know" on my part... good catch. I imagine that REDCINE uses a normal QT export engine, so you're probably right.

 

[combatentropy]

that's assuming that the camera always encodes at the same bitrate, regardless of frame size. Is that true?

I severely doubt it.

 

[Jim Arthurs]

Would it cost anything to use a DVC-Pro HD codec? Is there an HDCAM (or SR) codec at all?

There is indeed a software HDCAM codec, not in common usage. Outside of Sony proper, I'm only aware of its use in a discontinued product from Harris/Leitch/DPS called "RealityHD".

It was their SDI standard def boardset that used could take the HDCAM decks dub outs and digitize the signal. You'd then use the software codec in a version of Digital Fusion for rendering. The whole concept was clever, as you were using HDCAM the way you'd work with miniDV... the downside is that for realtime playback monitoring you'd need not only an HDCAM deck, but one fitted with the optional dub card.

However, it was a terrific way to clone HDCAM material without loss if doing straight cuts, and the bandwidth was about equal to uncompressed SD.

The guys at Eyeon re-wrote the HDCAM codec for speed, and I believe the cost per license accounted for $4500 of the product cost/unit.

They made only a handful of these, less than 20, of which I still have one in working order...

 

[I Bloom]

I think that REDCODE is the only supported compression format in-camera. You can recompress in post if you require some other codec.

I guess what is being suggested by myself and others, is that the format options such as 1080p RGB REDCODE and 720p RGB REDCODE, seem to have limited usefulness, because they are not that much smaller and still foreign to most post production pipelines.

I would be more useful if RED could create footage that didn't need to go through REDCine, that would fit into existing HD pipelines, because many of your potential clients will want that. They will be less concerned with flexability in post and more concerned with turnaround time and learning curves.

If they say to you, "We use Varicam, that's all we need", you should be able to say to them, "No problem. We'll put our cameras in Varicam mode and deliver the same footage you've been working with, just on a disk so you don't have to capture it." And then deliver to them, DVCProHD footage with all the advantages of Red's dynamic range, chip size, etc.

You say, right but, you can already do that with REDCine. I think that is true, but it ignores the reality of the logistics of many productions. It means I have to have a MacPro in my office, take the footage to my office, convert all of the footage and then deliver it. I don't think that people will understand that need, and or want to pay for that extra labor.

Of course this discussion is about EFP/ENG type productions. But I think it really matters. The reason I'm buying a RED is because I want to use it for high end dramatic work. The reason it's going to pay for itself is because it can adapt to almost any type of production. Maybe I'm shooting too low.

IBloom

 

[David Mullen ASC]

What if making the camera do all that internal processing into different video codecs made it larger and more expensive? Would you still want that feature?

Maybe it would make more sense to build the external recorders with that feature, so you could buy a REDDRIVE, let's say, dedicated to processing the signal into a few common HD codecs, like going to an DVCPRO HD VTR except that it's not a VTR but a hard drive.

 

[Stuart English]

On-board compression.

On-board compression.

RED ONE has one flavor of on-board compression - REDCODE RAW / RGB.

We don't and won't offer DV / DVCAM, DVCPRO 50, DVCPRO HD, HDCAM, ProRes or DNxHD video codecs in the camera body.

It would require considerable additional development time, cost, complexity and power to do so, and frankly there is no point - all of these codecs leave image quality on the table v's REDCODE RGB and especially REDCODE RAW.

As soon as REDCODE is editable on an NLE's timeline this issue becomes moot and just an interesting footnote for the video history books.

In summary, REDCODE via REDCINE to any of the above (and more) codecs, and REDCODE directly editable on a NLE's timeline are the twin paths we are focused on promoting, developing and optimizing.

 

[I Bloom]

As soon as REDCODE is editable on an NLE's timeline this issue becomes moot and just an interesting footnote for the video history books.

In five years, maybe. Does Avid have plans to run native REDCODE?

Sometimes we need to leave image quality on the floor. We don't want to, but in the age of reality television, and web distribution content is king. I don't like it but I recognize it. I recognize that the people who make business decisions care about quality less than they care about how quickly easily and cheaply they deliver. Outside of the dramatic/feature film market, producers are not crying out for something better than HD they seem pretty happy.

What I've learned about the camera body leads me to believe, that it is in fact extremely flexable inside, with the ability for example to switch from wavelet to another type of transform via soft rewiring. So the idea that the camera would be larger etc. doesn't make sense to me. The additional developement time for RED, and the cost versus benefits for them to implement some older codecs... that I understand, I can't make that calculation.

Initially I thought that you had implemented your own encoders to run in REDCine, meaning that you already had some code that would just need to be migrated and optimized. With further thought it seems more likely that Quicktime conversion is the last step in the export process. Increasing the scope of an in-camera implementation.

Not everyone cuts on FCP. I think they probably should, I also think everyone should buy a Mac, but I know that they won't. Its frustrating to me because I feel strongly that HD isn't going to go away any time soon, just having the ability in Final Cut does not in reality open up all doors. I think constantly putting image quality above integration with the current market is a mistake. A camera that has the option to deliver sharp, high dynamic range, selective focus, DVCProHD on a drive is still an awesome camera. That would be a Varicam killer. On the other hand camera that requires adopting a new format or an extra step in post, is a harder sell at least in the immediate future.

I'm all for early adoption, that's why I'm here. But I feel like their are some limitations here that scare me. I don't see RED as a "footage at the end of the day" camera. I see it as a "get involved in post" camera. I know there is a certain type of job that I can't get with this camera.

I feel like you guys don't recognize that the people you are really selling this camera to are not the camera guys....we're sold. It's really the people we are working for that I'm worried about.

Anyways, I go on and on about this. I'm still going to buy it regardless. I'm just hoping that after I buy one RED ONE, I can buy two RED TWO's.

IBloom

 

[Gavin Greenwalt]

I think the comparison is slightly unfair. The Varicam for instance still has to digitize the DVCAM tapes. So it's actually *the same* amount of steps in post.

 

[Stuart English]

Codecs

Codecs

The history of Varicam was Apple's FCP demonstrated support for it first (almost two years after the camera first shipped BTW), the market responded by buying lots of Varicams and DVCPRO HD Firewire based tape decks and Avid added DVCPROHD support across its product line pretty quickly after that. So unlike HDCAM, DVCPRO HD is a pretty universally accepted / adopted video recording and editing codec.

Now RED: we haven't even shipped yet, but at NAB we were able to demonstrate support for REDCODE native playback on the FCP timeline.... and we are able to transcode to pretty much any codec that already exists.

So why would it be 5 years before RED is appropriately supported?

 

[Evin Grant]

Ibloom, it seems to me that a DVCPro HD or HD-CAM deck rental is the best option for your concern. This is the exact reason that dual link HD-SDI outs are included on the camera. Panasonic also has thier new P2 drive (AG-HPG10) with HD-SDI in, battery powered. Although it seems a bit silly you could ostensibly turn your Red into a UberHVX by hooking up this drive and recording the HD out to P2 in the DVCPro-HD codec your clients are demanding, keeping a 4K Redcode Raw master for yourself.

 

[I Bloom]

So why would it be 5 years before RED is appropriately supported?

As far as I can tell, it is already appropriately supported, between FCP6 and REDCine, no one should have a problem working with the footage.

Unfortunately, I think that is not the point. Instead its about what people want and their capacity for change. That is what your not recognizing, this human reality. That's what I'm giving 5 years to change.

Its obvious to me that RED is presenting video "the way it should be". I'm excited. But its still a camera that has to make a return in 2008 and 2009 and I think that means being optionally downward compatible.

So yes, DVCProHD and HDCam in camera. I'd like to see it. It seems possible that the current hardware could do it. It seems like a more useful feature than RGB Redcode in the grand scheme of things.

Is it just that the developement time is prohibitive, or is it a decision based on something else?

IBloom

 

[David Mullen ASC]

As mentioned, you could get a DVCPRO HD deck to go with the RED, run single-link HD-SDI out to it. Here is a $12,000 product:
http://catalog2.panasonic.com/webap...d=103037&catGroupId=34402&surfModel=AJ-HPM100