Jim, what about a custom mode?

[Axel Mertes]

"Specs can change at any time". But anything over 4520x2540 would have to be a different model. This is the largest size we could ever go to on the RED ONE.

This comment is not to encourage rampant speculation. Just pointing out a fact that the sensor and system architecture of the RED ONE tops out at 4520.

Jim

Jim,

as we experience some flood of new formats these days (no worry, I like them) but so many applications having different requirements, what about implementing a real custom resolution mode?

Right now we all know that there is a fixed maximum pixel bandwidth of the Mysterium sensor, which I'd calculate as roughly:

4520x2540y60Hz as physical limit, ie.:

4520 * 2540 * 60 = 688,848,000 pixel/second.


The next limit is the REDCODE encoder limit, which currently peaks at 3K 2:1 60Hz as it appears to me:

3072 * 1536 * 60 = 283,115,520 pixel/second

which apparently equals 4K 16:9 30Hz:

4096 * 2304 * 30 = 283,115,520 pixels/second

while 2K 2:1 120Hz is less data, as we can see here:

2048 * 1024 * 120 = 251,658,240 pixel/second


Finally we have the line speed limit, which is:

2540 lines @ 60 Hz = 152,400 lines/second from theory,

or:

1024 lines @ 120 Hz = 122,880 lines/second from practice.


I'd summarize:

1. Limitation: Sensor pixel speed:
4520 * 2540 * 60 = 688,848,000 pixel/second.

2. Limitation: Codec pixel speed:
3072 * 1536 * 60 = 283,115,520 pixel/second (hopefully not the max)

3. Limitation: Line speed:
1024 lines @ 120 Hz = 122,880 lines/second from practice.

The above 2. and 3. are real world proved values, while 1. is an given value from specs. Clearly other factors may influence actual speeds. Values can be even higher (which I hope for the codec limit) or lower.

So as we clearly hope for 4K 60p at some point (inspite of a 2K scaled as you requested us some time ago, but recently stated "[surely] not until NAB"), we can at least identify that within above constrains certain custom resolutions could be possible, maybe with some nurdish increments of 4, 8 or 16 lines as size multiplicators, but at least possible.


Having worked with lots of industrial cameras, I see technically no big reasons to not have user defined custom modes within these constraints.

What could be an issue is how a custom mode is displayed / refreshed on the LCD/EVF/HD-SDI/HDMI path. But if one would agree that a 2K 2:1 120Hz is already far out of question to be in any way compatible to any of these output ports, its already the solution: Treat it as if you were treating this high speed 2k 120Hz format.

So given the above facts, slightly including some speculation (but minimal as I think), we should be able to do some interesting custom modes, such as:

High Speed thoughts...:

2048y @ 60 Hz maximum, maximum width then ~2304x (nearly 1:1 mode)
1536y @ 80 Hz maximum, maximum width then ~2304x (a 3:2 mode)
1200y QXGA @ 102 Hz maximum, maximum width then ~2312x
1080y HD @ 113 Hz maximum, maximum width then ~2316x
1024y SXGA @ 120 Hz maximum, maximum width then ~2304x (hey, more than 2:1...)
800y XGA @ 153 Hz maximum, maximum width then ~2312x
720y HD @ 170 Hz maximum, maximum width then ~2312x
576y PAL @ 213 Hz maximum, maximum width then ~2304x
480y NTSC/VGA @ 256 Hz maximum, maximum width then ~2304x
288y PAL one field @ 426 Hz maximum, maximum width then ~2304x
240y NTSC one field @ 512 Hz maximum, maximum width then ~2304x
128y YouTube style @ 960 Hz maximum, maximum width then ~2304x


NEW:

Other format wishes collected from other threads:

4520x1884y 2.40:1 cinemascope
4520x2443y 1.85:1 academy widescreen
3048x2540y native 2.40:1 anamorphic 2:1 cinemascope (ie. 1.20:1)
1920x1080y native HD
3072x1850y 1.66:1 widescreen

and the list goes on and on and on...


Comments welcome!

Cheers,
Axel

 

[Eryc Tramonn]

Didn't Jim just write, "No 2K scaled" somewhere. I could have sworn that he's said that about a dozen times over the last week alone.

 

[Kenn Christenson]

If I'm not mistaken, the poster, above is asking for different "windowed" modes.

Variable "windowed" modes would sure be great! You could balance the frame rate you want to the resolution hit you're willing to take. You could also "window" to different aspect ratios... but, hey, this would all be gravy!:wink:

 

[jbeale]

I'd bet that the custom modes are indeed possible, given the right firmware. But given that these are "windowed" modes, anything below 1080p framesize seems rather special purpose/limited interest, because any normal lenses will have such a small field of view, due to cropping out a small area of the full sensor.

 

[Alexis Vanier]

I personally would very much enjoy such a feature. A lot of my family and friends are engineers and I could easily see application there. It would also be very useful in SD. SD's supposed to keep rocking until 2011 here in Canada (even if the national broadcaster is pretty much converted to HD already). Also, speciality shots for web distribution under SD is something I can see a market for. Finally there's educational purposes, for individuals and organizations. Schools could give their students an opportunity to fiddle with more extreme off-speed and I could test all my household bulbs to figure the decay rate of their tungsten filaments just for the kick of it.

I say Robin Jannard should steal from the Sheriff of Vision Research and give to the Poor. :whistling:

Please, bring it on!

 

[Axel Mertes]

Custom modes help on having less skew?

Custom modes help on having less skew?

Didn't Jim just write, "No 2K scaled" somewhere. I could have sworn that he's said that about a dozen times over the last week alone.

Hey, this is not asking 2K scaled. If you read it right, its based exactly on Jims answer that he don't want to bring 2K scaled at all.

The idea is to use the CMOS feature of using a subwindow of the sensor (cropping) to do alternative image sizes & aspect ratios and clearly different speeds. Most interesting is the high speed feature here, which is a market of its own.

I have worked with many industrial cameras, mainly high speed cameras. Almost every CMOS based camera can do windowing, binning or region of interest or how you'd ever call it (certain binning modes being a very special feature here, not available in all sensor designs). We've shot HD with 1000 fps and PAL with 3000-4000 fps. RED is not up to these special cameras, but for many applications it has enough potential to do things up to 200, 300, 400 frames in reasonable resolutions (for a SD use for instance).

I do understand that Jim's intention is mainly high res imaging. But we have a reasonable high speed camera in our hands and no additional cost other than implementing a custom mode (which might be so not much more work than any other "new" mode). The benefit is the resulting universality of the camera. I do believe that the codec will fit many sizes, as does the post production if required.

Jim mentioned a while back that Steven Soderbergh is using something they call "Soderscope". Why not have him adjust the custom mode to do exactly this.

One more thing:

The skew may be reduced as the line speed can go up. If you shoot with double frequency on a lower resolution, your skew should be half. If you imagine that we can have custom modes and one might use a "subsize" anyway for his specific purpose, it may be wise to just read out that subsize only, instead of cropping post production. The result would be less skew, which seems to be desired by many users out there.

Jim, Jarred, Stuart, the whole RED team:

Sorry for begging you (but wasn't that forum for this?), but this custom mode thing would be a really big hit to be done, IMHO, and I believe many others out there would love it.

You define the constraints. My calculations in the first post here are based on the constraints we have seen that work already in other modes. I believe the modes calculated are more or less possible. The hope is you encoder can do faster and reach at some point the pixel speed of the Mysterium sensor.

Best wishes,

Axel

 

[Lewis-M Soucy]

I totally support that! Also, having custom modes and/or pre ratio or scaled modes would be usefull for a return to film transfer or any inserts/pick ups to be put back in a specific film format... It would ease the work for back to film and lab work... 4K is still really expensive to deal with in labs, and they often require specific sizes for the TK...

 

[laguun]

I totally support that!

If this is doable without to many efforts, it would really make sense and add another nifty feature to reds already impressive functions.

 

[Andrew Antoniewicz]

While I understand the desire for complete freedom on the framesize front, my guesses on the underlying hardware design would make this very hard to accomplish.

I expect the hardware pipe is roughly as follows:
(1) Mysterium chip; 4520 X 2540 max at 60fps connects to
(2) FPGA Based Wavelet Compression Engine connects to
(3) Storage system interface

For those who dont know, FPGA stands for Field Programmable Gate Array and is a form of "soft" hardware. It is hardware which is configured at a very low level via a downloaded configuration bit stream. The inputs and outputs of the chips are fixed because that is the pattern of copper on the boards, but the internal logic of each chip is downloaded to the part in a matter of a few seconds.

The reason I bring this up is because I believe that Red must seperately design these bit streams for each crop size and compression level. So for example they would have one configuration file for 4096_2048_Redcode36 and a different stream for 4096_2304_Redcode36 or 4096_2048_Redcode28 or for any of the other setups. Once a user picks a format via the user interface, the camera downloads the appropriate stream to the hardware, and then the camera is ready to shoot.

Given that my above guesses are true, you can see why they would not be able to support any random crop size. It would require that they have a nearly infinite number of configuration streams ready to be downloaded to the camera FPGA system. Depending on the FPGA the streams are not tiny either, as some of the higher end FPGAs need a 1MBit or more of configuration data for a setup.

So my suggestion is that folks decide on the formats they really want and ask for them. Given that Red has some engineer time, they may be able to add a favorite size to the already growing list. Cheers.

 

[Axel Mertes]

While I understand the desire for complete freedom on the framesize front, my guesses on the underlying hardware design would make this very hard to accomplish.

I expect the hardware pipe is roughly as follows:
(1) Mysterium chip; 4520 X 2540 max at 60fps connects to
(2) FPGA Based Wavelet Compression Engine connects to
(3) Storage system interface

For those who dont know, FPGA stands for Field Programmable Gate Array and is a form of "soft" hardware. It is hardware which is configured at a very low level via a downloaded configuration bit stream. The inputs and outputs of the chips are fixed because that is the pattern of copper on the boards, but the internal logic of each chip is downloaded to the part in a matter of a few seconds.

The reason I bring this up is because I believe that Red must seperately design these bit streams for each crop size and compression level. So for example they would have one configuration file for 4096_2048_Redcode36 and a different stream for 4096_2304_Redcode36 or 4096_2048_Redcode28 or for any of the other setups. Once a user picks a format via the user interface, the camera downloads the appropriate stream to the hardware, and then the camera is ready to shoot.

Given that my above guesses are true, you can see why they would not be able to support any random crop size. It would require that they have a nearly infinite number of configuration streams ready to be downloaded to the camera FPGA system. Depending on the FPGA the streams are not tiny either, as some of the higher end FPGAs need a 1MBit or more of configuration data for a setup.

So my suggestion is that folks decide on the formats they really want and ask for them. Given that Red has some engineer time, they may be able to add a favorite size to the already growing list. Cheers.

Nerdicus,

I've used tons of camera head that do exactly what I'd described, especially high speed cameras. They do not need any additonal FPGA design other than supporting a flexible design approach, which they all do because of their requirements (such as high speed cameras). Its possible and I really doubt that all modes we see in RED are "hard wired" in the end in the FPGA. I don't think RED has made a design that would make it harder for them to have new modes, rather than making it easier. If you were right and I am not, that would be a real design pitty IMHO.

I think the bigger question is if they see problems on support side arising, when people use custom modes. In such case I'd say "no", because in worsed case they (RED) could simply deny on officially supporting the custom modes, but still implement them for us to let us play how we want. The same way we get beta firmware, but should not use it (other than testing). As a result cool formats could get back to RED after "field test" and come along as presets. Imagine loadable presets you can change via files and CF cards!

The VHDL & FPGA designers I know clearly deny on the assumptions you make here. My own technical background tells me the same. But I may be wrong, I clearly don't know everything. But I have hope.

My point is clearly:

We (Users) beg them (RED) ONCE for ONE new mode - a CUSTOM mode. Then all other begging for special modes will be obsolete - forever. THEN RED can concentrate on other stuff and leave us doing what we want: Shooting in whatever way we like to...´

Axel

 

[Andrew Antoniewicz]

Well Axel I guess we will just have to see what Red says.

Which high speed cameras did you use that allowed you to set the frame size directly? I suspect these cameras are so flexible because they do not perform the same level of compression in the camera or the compression scheme is smaller block friendly. Its much easier to allow the user flexibility if they don’t have to morph their compression blocks to allow for lots of weird sizes. Wavelet compression (unlike say JPEG's DCT) with anything more complex than the most simple Haar wavelet is hard to subdivide into the nice separable parallel paths you need for efficient high speed compression.

<quote>The VHDL & FPGA designers I know clearly deny on the assumptions you make here.</quote>

This comment surprises me a bit. I expect the FPGA hardware is indeed synthesized from VHDL or Verilog source into the bit streams needed for Red's hardware. This flexibility is what allows Red to build a new format in only a few days instead of weeks. I very much doubt that this synthesis is running in the camera *grins* as I have had less complex synthesis/routing runs that lasted overnight.

Now I agree that with a different form of compression you might build an engine around a core of an 8x8 pixel block (like DCT). But from my somewhat dated experience with Wavelet compression, the best subdivision you usually get is based on a known horizontal length (like 4096 or 2048) and some small number of lines (like 4-16) depending on the size of your wavelet filter. This allows you to perform your initial horizontal filter pass with all its overlapping components in parallel and properly update/shift the line cache as new line data is read from the sensor.

Now all the above is complete speculation of course, Red may have found a much better way to subdivide the compression problem. If so, they may be able to give you what you want. I will let someone else talk about why it might be a headache for the post tools if you have a soft frame size.

 

[Axel Mertes]

JPEG 2000 for instance is a quite flexible wavelet algorithm, isn't it?
Cineform, also wavelet based, is again flexible.
There are others, that are flexible too. In fact I know none which is not.

So why should post production tools be in any way affected by a "soft" frame size. We do post production in many different resolutions, and there is no need to worry about specific sizes, except sometimes codecs want to stay in certain 4, 8 or 16 pixel multipliers. But thats about the biggest headache I can imagine.

The cameras for high speed recording are indeed ultra-flexible. Some of them allow windows / ROI of 1 pixel steps actually. And then no new firmware gets uploaded, just parameters get set in the camera and thats it. I hope RED has similar abilities lying underneath.

My point is I hope for this kind of features until RED is telling us it won't work. Until then I can only find it frustrating if someone chimes in and speculates it would not work, without having REDs infos on hand. Sorry, this is no offense, just saying I find it frustrating being negative poled here. So I am the optimistic guy here and wish they can do it. They might want to do it for themselves.

A while back Jim said we should stop begging him for 2K scaled, instead we should ask for 4K 60Hz RED CODE to RED DRIVE. So I stand and be optimistic to ask for RED CODE 4K 60 Hz - until Jim or RED officially denies on it.

My glass of RED wine is still half full, yours is currently half empty See the difference...


Thanks listening,
Axel

 

[Axel Mertes]

Nerdicus,

to answer you question about which cameras we've been working with in High Speed, here is a list of what I can recall (not necessarily complete):

- Microtron MC1311 or the like (don't know exact name)
- PCO (same as in WeissCam)
- Weinberger CineSpeed Cam, its the same as Visario G2, but with some small - IMHO useless - add ons from band pro, selected for "better quality" samples.
- Weinberger Visario G2 (which is not Bayer and only 1536x512y Pixels, anamorphic resulting in 3:2 images)
- Phantom
- Photron
- Basler
- and some more

Most of them are flexible to let you adjust the image window in at least 2 pixel steps (due to color Bayer pattern).

Axel

 

[Andrew Antoniewicz]

Well I wouldn't say I am a glass half empty person, we just disagree on what are the underlying capabilities of the system. The only person who can really answer the question is someone who knows the actual system. I question the NEED for this capability, but would be just as happy as you if Red can provide it without compromise and without delaying some other more important feature.

From what I can gather those cameras all seem to write uncompressed data to storage, so they aren't great examples for this problem.

After some poking around, I found a hardware compressor part for JPEG2000 (Analog Devices ADV212) which looks like it would work just fine for a system like Red. They would need to have probably eight of the parts on a board to get the 4K throughput required but the ADV212 looks like it would be able to handle the needs of Redcode. As a bonus the parts can support up to 4K line width and have the ability to crop to a software defined rectangle. So if Red is using this part or something they designed like it, a crop to an arbitrary window doesn't seem as involved as I imagined.

Good luck with your crusade, I am just happy to have learned about a new piece of tech.

 

[Axel Mertes]

Hi Nerdicus,

yep, the ADV chips are the state of the art for JPEG2000 encoders right now, but I think RED uses a Xlinix FPGA to do the job (I think the Virtex line, according to some error messages we have seen on occasion from the camera).

Anyhow, I think it can be done.

Btw, we had once used a high speed cam (can't remember the actual manufacturer), which did JPEG encoding at 500fps. It was not best image quality, but it was capable of compressing any adjusted image size. So not a real prove of concept, but at least a "symbol" of what I (and so many others) want to see at some point.

Cheers,
Axel

 

[Axel Mertes]

I've update some new formats in the first post here and would invite everybody interested in "new" or "custom" formats to explain her/his interest here, because when we show enough interest, somebody at RED will hopefully respond and we may see an exiting new feature!

Thanks,

Axel