Question for Jim about Line-Skipping

[Tom Lowe]

Jim,

It seems that these prosumer, larger-sensor cameras are continuing down the path of line-skipping.

So here's my question: What would it take for companies like Panasonic or Sony to fully downsample the sensor image in camera (with no line-skipping) to 1920x1080, instead of using line-skipping techniques?

For example, could Panasonic simply add a very powerful, purpose-built processor to the AF100, whose sole job was to downsmaple the full sensor image to 1080p? If so, what would such a processor cost? How much would it add to the price of the camera?

How do cameras like the Sony EX1 and Panasonic AF100 create their images now? These are CMOS sensors, right? Are they working from a Bayer pattern now?

 

[Stephen Williams]

Jim,

It seems that these prosumer, larger-sensor cameras are continuing down the path of line-skipping.

So here's my question: What would it take for companies like Panasonic or Sony to fully downsample the sensor image in camera (with no line-skipping) to 1920x1080, instead of using line-skipping techniques?

For example, could Panasonic simply add a very powerful, purpose-built processor to the AF100, whose sole job was to downsmaple the full sensor image to 1080p? If so, what would such a processor cost? How much would it add to the price of the camera?

How do cameras like the Sony EX1 and Panasonic AF100 create their images now? These are CMOS sensors, right? Are they working from a Bayer pattern now?

Hi Tom,

I was told at IBC that there was no line skipping when I asked on the Panasonic stand, no idea if that is accurate or not.

Sony EX3 uses 3 chips of 1920 x 1080, so no Bayer involved.

Stephen

 

[Tom Lowe]

Hi Tom,

I was told at IBC that there was no line skipping when I asked on the Panasonic stand, no idea if that is accurate or not.

Sony EX3 uses 3 chips of 1920 x 1080, so no Bayer involved.

Stephen

Thanks, Stephen.

My impression from watching the AF100 interview with their rep was that the AF100 was in fact not sampling the entire sensor. She said something about severely degrading/softening the image in order to avoid aliasing/moire.

 

[Mike Gross]

I heard it's using a very strong low pass also. Anyone know about the sensor size? There's speculation that it's 17.30mm wide x 9.73mm high for the 1080P sampling.

 

[Michael Olsen]

I heard it's using a very strong low pass also. Anyone know about the sensor size? There's speculation that it's 17.30mm wide x 9.73mm high for the 1080P sampling.

Those "in the know" over at DVXuser seem to be quite sure its 18.9mm wide.

 

[Matt W.]

The 7D/5D don't line skip; they bin pixels together, like with sRAW. The aliasing is roughly the same with this technique as it would be with line skipping (2MP sampling from a high megapixel chip) but sensitivity isn't sacrificed. If these cameras really used line skipping, they'd be a lot slower for video than for stills--on the order of three stops.

Even the Red can't deliver 1080p bayer in real time (for monitoring OR recording). The Genesis had to use an alternative sensor design (stripes) to allow real-time recording. The ALEXA has a very low-res chip. Debayering full res in real time is hard and the AF100 probably doesn't do it.

But claiming that the AF100 line skips? That's unsubstantiated. Chances are it bins pixels and uses an appropriately stronger anti-aliasing filter, so all it's sacrificing is resolution, which doesn't really matter with output to a 1080p codec.

This "line skipping" talk is largely FUD. You just have to look at this site to see Red's good at marketing and that's all this is. Not that RAW isn't obviously better, but there's nothing inherently terrible about what the AF100 is likely doing.

 

[David Mullen ASC]

Line-skipping and pixel-binning are just two ways of reducing resolution / information before debayering, I don't see one process as being necessarily superior to the other, it's creating the same problem, it's the same compromised way of getting the sensor information down to 1080P / 2MP.

ARRI D21 and Alexa can debayer in real-time to 1080P and apparently the Epic will be able to as well. I think Tom's question is a good one, which is exactly what sort of processing power is needed to make this possible -- and how much would it add to the cost of a Canon DSLR, etc. compared to their current processors. The flipside would be what would it take to allow a DSLR (or the Panasonic) to record RAW for video so that the real-time debayering wouldn't necessarily have to be high-quality.

 

[Mike Gross]

The 7D/5D don't line skip; they bin pixels together, like with sRAW. The aliasing is roughly the same with this technique as it would be with line skipping (2MP sampling from a high megapixel chip) but sensitivity isn't sacrificed. If these cameras really used line skipping, they'd be a lot slower for video than for stills--on the order of three stops.

Even the Red can't deliver 1080p bayer in real time (for monitoring OR recording). The Genesis had to use an alternative sensor design (stripes) to allow real-time recording. The ALEXA has a very low-res chip. Debayering full res in real time is hard and the AF100 probably doesn't do it.

But claiming that the AF100 line skips? That's unsubstantiated. Chances are it bins pixels and uses an appropriately stronger anti-aliasing filter, so all it's sacrificing is resolution, which doesn't really matter with output to a 1080p codec.

This "line skipping" talk is largely FUD. You just have to look at this site to see Red's good at marketing and that's all this is. Not that RAW isn't obviously better, but there's nothing inherently terrible about what the AF100 is likely doing.

Binning horizontally probably but not vertically. If 5D/7D did binning vertically, then it wouldn't have a huge difference that it does between horizontal and vertical. 5D/7D is definitely line skipping and lineskipping wouldn't affect sensitivity.

 

[Deanan]

I think Tom's question is a good one, which is exactly what sort of processing power is needed to make this possible -- and how much would it add to the cost of a Canon DSLR, etc. compared to their current processors. The flipside would be what would it take to allow a DSLR (or the Panasonic) to record RAW for video so that the real-time debayering wouldn't necessarily have to be high-quality.

Hopefully we have a good idea about what it takes

The main pieces are:
a. Sensor design so it can run at a speed fast enough to read/reset ALL pixels. The higher the speed, the lower the skew/jello.

b. Sensor processing fast enough to keep up with the max sensor speed. This one depends mainly memory and processing speeds.

c. processing/memory bandwidth to deal with raw or compressed raw out to media or compressing and out to media.

Note: estimated numbers for the 5D and Alexa numbers below...

5D@12.7mpix@3fps = ~38 megapixels/sec @ 58MB/s

Alexa raw @4.6mpix@60fps = 279 megapixels/sec @ ~420MB/s (12bit numbers after compressing the sensor 16 bit to 12bit, 558 before)

Epic@13.8mpix@96fps = 1327 megapixels/sec @ ~2.7 GBytes/sec (not including the extra sensor speed and processing needed for HDRx, )

That's only part of it though as debayer, monitor outputs, compression engines, etc all add alot to how much processing and memory bandwidth is needed.

 

[Peter Chang]

Binning horizontally probably but not vertically. If 5D/7D did binning vertically, then it wouldn't have a huge difference that it does between horizontal and vertical. 5D/7D is definitely line skipping and lineskipping wouldn't affect sensitivity.

Straight from the upper echelons at Canon: "We don't line skip. We bin. When the chip is decimated to 1920x1080 each pixel is used."

 

[Jannard]

Straight from the upper echelons at Canon: "We don't line skip. We bin. When the chip is decimated to 1920x1080 each pixel is used."

Was he proud of that?

I guess you can line-skip, pixel bin or use other short cuts... why not just read every pixel and do a proper scale if you want 1080P from a higher resolution sensor?

Important fact: 1080P is a file size, not a measured resolution. It is 1.92K (horizontal). We have never seen actual measured 1.9K from any camera that records that format. The 5D measures 1.2K (and is pretty scary after 1K). An F35 measures about 1.7K. An Alexa about 1.6K. Think of 1080P as a drawer size. Not all drawers are full.

Jim

 

[Mike Gross]

Straight from the upper echelons at Canon: "We don't line skip. We bin. When the chip is decimated to 1920x1080 each pixel is used."

They might as well line skip then because their binning looks pretty bad or it wouldn't be the problem it is.

 

[Gunleik Groven]

Looking as a resolution-chart, binning looks a fair bit like skipping... -

 

[Tom Lowe]

sRAW bins, so I am familiar with what that looks like. You can get hot pixels burned in to an sRAW, but you really don't see aliasing, at least not in any sRAW motion timelapses I have shot. The HD video on most DSLRs doesn't look like simple sRAW style binning. The moire is too horrendous.

So Jim, Deanan... getting back to my original question. If Panasonic wanted to sample the full sensor (it's something called a live MOS 4/3 sensor... is this Bayer?)... assuming the sensor can read/reset all pixles at the claimed framerate, how much money would it cost them to add a processor to the AF100 capable of proper, realtime downsampling to a pristine 1920x1080 RGB image ready for their stupid codec to compress? Are we talking 5 grand? 10 grand?

 

[Deanan]

So Jim, Deanan... getting back to my original question. If Panasonic wanted to sample the full sensor (it's something called a live MOS 4/3 sensor... is this Bayer?)... assuming the sensor can read/reset all pixles at the claimed framerate, how much money would it cost them to add a processor to the AF100 capable of proper, realtime downsampling to a pristine 1920x1080 RGB image ready for their stupid codec to compress? Are we talking 5 grand? 10 grand?

Tom, I thought you didn't think 1080P was enough?
Or 8bit to a long GOP codec...

 

[Peter Chang]

Was he proud of that?

Not proud, but defensive... He was refuting my assumption/accusation of line skipping. He followed that up with "RAW would be nice."

 

[Jannard]

sRAW bins, so I am familiar with what that looks like. You can get hot pixels burned in to an sRAW, but you really don't see aliasing, at least not in any sRAW motion timelapses I have shot. The HD video on most DSLRs doesn't look like simple sRAW style binning. The moire is too horrendous.

So Jim, Deanan... getting back to my original question. If Panasonic wanted to sample the full sensor (it's something called a live MOS 4/3 sensor... is this Bayer?)... assuming the sensor can read/reset all pixles at the claimed framerate, how much money would it cost them to add a processor to the AF100 capable of proper, realtime downsampling to a pristine 1920x1080 RGB image ready for their stupid codec to compress? Are we talking 5 grand? 10 grand?

Tom... I think the problem is your 1st assumption. I'd be very surprised if their sensors have a clock speed and A/Ds capable of read/resetting all pixels at 24fps. I don't know for sure... but if the sensor was fast enough, why would they not deliver it's full capability?

Jim

 

[Gavin Greenwalt]

...but if the sensor was fast enough, why would they not deliver it's full capability?

Jim

They just don't care? That's my theory.

If there is a way to sell extra cameras without any extra work they'll do it. Why else would it have taken 18 months or whatever it was to deliver 24p? They don't care.

The not caring strategy seems to be working for them still. Lots of shooters are being asked to also not care-and an unfortunate number are listening. They probably looked at it from the perspective of: "Nikon/Canon would skewer us for the reduced battery life, increased weight and cost." Entry level cameras cost $600. Mid levels either $1200 or $3500 and Top of the line $6k. Those are just kind of "The Prices" now for DSLRs. I imagine using the full sensor would add cost.

Whenever presented with a problem there are always two solutions: Fix it or Fix expectations. Just convince people they don't want anything better.

 

[Gunleik Groven]

There are allways trade-offs.

It is easier to get "the most" out of a 5D vs a RED at the moment, especially considering how the on-set preview works and how people seem to use the ISO settings and expecting a finnished look off set.

The 5D does these things out of the box.

You need less everything (from knowledge to equipment) to get the best possible performance out of the camera.

Problem - in my opinion - is that "best performance" is sub par in so many shooting circumstances that it is really not worth the tradeoff to spend time with it.

A 5D can look great under a few shooting conditions, but really sucks for a most other "normal" uses.

Systems like RED has much more flexibility in use, and the potential for looks are much bigger. It is not in any way "hard" to use it, but like with a lot of "pro" tools, you are left on your own: Knowledge is king. That is knowledge about photography and image workflows and signalchains. That knowledge can be applied in a 5D workflow, too - but the potentialgains aren't really that remarkeable.

I think it is good that more people get access to shallow DOF and 24p, as that has been a thing indies have been dreaming of.

Comparing that to professional camera-systems OTOH is not really relevant.

As a comment to the "pro" thread, it may be measured in potential, reliability and flexibility + (very often) the ability to make manual adjustments to most everything - when it comes to pro equipment.

That doesn't mean that everyone is better off doing manual adjustments and choices, but that the potential when you know how to do those manual adjustments are miles ahead of "factory pre engineered" choices.

 

[Tom Lowe]

Maybe Panasonic doesn't care.... or maybe they are trying to protect their $60,000 2/3" 3-CCD 1080p P2 Varicams? Think about it.