Welcome to our community

Be a part of something great, join today!

Register Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

why 5k? go after uncompressed instead please

Losseless compression can get you around 2.5:1 - not enough, really to get data rates manageable.

Graeme
 
BTW, the data rape i was talking about was data recording on set, not post.
 
There should be multiple options ideally. An uncompressed output option for people who want it (like for efx work) and a couple of different compression levels, some milder than the current REDCODE options.

As for 5K, I'm all for it, but it's not like 5K is radically bigger than 4K, so I don't get the objection to it.

You'd think that an uncompressed output would actually be less processing work for the camera anyway, other than the issue of what cable would carry 5K uncompressed RAW data -- that would probably add to the cost of the camera.
 
Well, back when Red was talking about an uncompressed 4.5K data port on Red One, it was going to be capable of 60fps 4.5K shooting.
 
RAW uncompressed

RAW uncompressed

Well, back when Red was talking about an uncompressed 4.5K data port on Red One, it was going to be capable of 60fps 4.5K shooting.
Click to expand...

Its the device on the other end of the data port that's the problem, it kind of kills your portability, and costs a few more $ than a CF card :red_bandana:
 
so what would lossless compression 2.5:1 be at 5K, what is the data rate and how many of those acam style cartridges would you need in a raid configuration to run it, would it fit on a battery style belt, how much is advanced flash storage going to cost a year from now
 
Cost per minute... where is the cost problem?

Cost per minute... where is the cost problem?

agwah said:
so what would lossless compression 2.5:1 be at 5K, what is the data rate and how many of those acam style cartridges would you need in a raid configuration to run it, would it fit on a battery style belt, how much is advanced flash storage going to cost a year from now
Click to expand...

A 5K 12bit Bayer 16:9 frame is 5120x2880x1.5=22.2MB/frame.

(22.2MB if you pack the data 2pixels in 3 bytes 12+12bits)

If you archive on 750GB disks (about $130 each) you get,

750000/22.2=33783 frames/24=1407sec/60=23minutes.

With 2.5:1 lossless compression you get 23*2.5=57.5minutes for $130, I hardly think that would put anyone in the poor house?

It is much cheeper than 35mm film with processing.

What I am thinking now since I got my program running faster is that you do not need to archive everything, just the source RAW Bayer files, and the color correction files for the key frames and edit lists. You can then make the output files for the film recorder and delete the work files along the way so long as you keep any macros and such so that you can regenerate the output files if you need to, that can cut the archive costs by 80% maybe. The RAW Bayer files are smaller than the TIF output files, so you can save a larger ratio of camera shots in the same number of disks.

If you can archive the "negative" RAW files for about $150 per hour shot (using lossless compression), then if you have a 5:1 ratio of files that need to be saved for an 90 minute feature your cost is $150*1.5*5= $1125, compare that to 35mm film!

I don't think you can get 2.5:1 lossless on all shots, but you should be able to average 1.5:1 maybe, even so being able to shoot a feature with only about $2000 for the "digital negative" cost RAW uncompressed 5K is not such a problem as far as I can see.

If you do lossless compression before you output from the camera you get,

22.2MB/~2.5=8.88MB/f*24=213MB/sec, which is lower than the A-cam dII memory can run at, so for 24fps the A-cam dII could shoot 5K if they used this 2.5:1 lossless compression. Without the compression three A-cam memory blocks in parallel would cover the bandwidth, that would be 80x3=240GB, so

240000/22.2=10810/24=450/60=7.5minutes record time,

Anyone who has used a movie camera can shoot with a 400' mag, so this is longer, 90x7.5=675 foot 35mm mag . With lossless compression you might get 1687 foot from three A-cam memory blocks, now who cannot shoot with a 1000'+ mag? And it would be smaller than a 400' film mag and be re-usable!

Please point out any math errors...
 
Some quick numbers (feel free to correct, based on some files on my HD):

1920x1080 RGB 10bit UC 24p: 174MB/s

5k RGB 16 bit TIFF 24f: 2.7 GB/s

5k RAW 16 bit 24f: 900 MB/s (based on the above)

That means an average master two shot of a scene running, say, 6 minutes, would come in at 330 GB.

The fastest 2.5" hard drive's performance is in the 80mb/s range (SG Savvio). It would take, say, 15 of these in RAID 0 to hit the speed required. This RAID would have capacity of about 900 GB. 15 drives is a little silly, but totally doable. Keep in mind how big a 1000' 35mm film mag is.

So far, this seems reasonable, until you realize that eSata can, at best, support only 300 MB/s. Whoops, now we're into fibre optic territory or other "exotic" connections. But even dual-link 2Gb/s fibre channel is only 400mb/s.

Uncompressed 5k? Would be nice. Not going to happen.



But what about the next best thing? Graeme says that lossless compression can get you 2.5:1 (I assume this depends heavily on the scene, exporting a one-color 5k TIF with deflate compression yields a file that weighs all of 72kb). So let's say 2:1 from lossless compression. That's still 450mb/s, still too much.

Basically, the data rate has to be low enough to go through eSata, so that off the shelf components can be used. So, to allow some overhead, it would be great to have a 200MB/s or even a 250 MB/s, which would be bringing us into the 4:1 or 5:1 range, similar to HD-SR.

I think everyone will agree that HD-SR looks great - nearly indistinguishable from uncompressed, especially in 444 (which is actually more compressed).

I think RED could deliver some pretty amazing images at a 5:1 ratio, and I think asking for the least compression possible is more reasonable than asking for uncompressed.

So, my question - EPIC 5k at 5:1? Can it be done?
 
Unbaked...

Unbaked...

AngusChandler said:
5k RAW 16 bit 24f: 900 MB/s (based on the above)

I think RED could deliver some pretty amazing images at a 5:1 ratio, and I think asking for the least compression possible is more reasonable than asking for uncompressed.
Click to expand...

Its probably 12bit data not 16bit data, so the size and bandwidth is smaller. You can store two pixels in three bytes without lossless compression, and more with lossless compression since a long string of bytes is used to hold more pixels using bits in bytes for more than one pixel.

To do compression you need to do some things to the image data, so there are issues baked into the frames.

If you have FULL RAW data then there are no baked features or compression artifacts, and no assumptions, 20 years later you can pull more out of the RAW data, just like going back to a film negative.

You do not need fiber optics, a parallel wire cable (14+ conductors like a printer cable) can take the bandwidth, and anyway the A-cam memory block is solid state and does 250MB/sec apparently.

With RAW camera builds would not affect the resulting images, the RAW data would only be due to the limits of the sensor and would not change with firmware revisions.
 
What was the RAW Port going to have as a connection?
 
thank you for doing the numbers dancad3d

i think in images and just saw this little camera with a thick cable to my belt pack - this system would have:

a large enough sensor to get rid of the olpf blurring for a 4k output - a high enough data rate to take care of all the details in the image no matter what forest you are standing in - then sensitivity and dynamic range to cover toms timelapses from dusk til dawn and back to dusk again with one exposure setting - kidding aside - it seems the data rate is not the problem
 
David Mullen ASC said:
Sounds like something Batman would wear -- the BAT-RAID belt... I've been storing too much around the waistline lately but unfortunately, it's not in bytes of data that can be easily downloaded...
Click to expand...

"BAT-RAID" = exactly! If you don't want to wear it like a belt.. you could go for the Rambo-style.. cross-ways over the shoulder.. :turned: Might be kinda cool fashion statement.. add a metallic Red logo.. wear Oakley sunglasses.. I'm on it!

Oh wait.. don't own a Red yet. hrmmm....

~S
 
Uncompressed RAW works the same, or one extra step...

Uncompressed RAW works the same, or one extra step...

Ayoji said:
How would the uncompressed RAW work?
Click to expand...

Uncompressed RAW works the same for steps after the de-Bayer, you run the true raw files through a de-Bayer program to make DPX or TIF files, then use those.

You get more options for using different de-Bayer software, and can go back and reporcess the RAW files later without having artifacts or other issues burned into the pseudo RAW data.

What I am talking about is true RAW sensor data, in DSLR some processing of the sensor data can go on before the pseudo RAW data is saved.

In a compressed camera it seems the camera buffers a 1/4 size image in monochrome for the one red and blue image and for the two green images giving four 1/4 size monochrome images, which are then compressed using JPEG2000 like compression. The problems with that is in the artifacts that get into the image before the de-Bayer, that sort of compression was conceived for use with video gamma RGB images, not high range linear pre-de-Bayer sensor data. Because they are basicly compressing a four view quadrant monochrome image they do not need to reduce the bandwidth for chroma until after the de-Bayer, when shuch filtering is needed to remove additional chroma moire that the wavelets aritfacts can contribute. The camera sensor makes four 2K images, these are interoplated into a single 4K image. Since the Bayer image is 1/3 as big as a RGB image, 10:1 compression of the Bayer image is like 30:1 compression over the RGB image, hence the use of compression before the de-Bayer. Since the data is 12bit rather than 16bit it is more like 45:1 compression over a 16 bit TIF file of the same image.

The assertion that you cannot see any difference between true RAW data and something that has been low pass filtered in several ways is probably not going to hold up when you can process the true RAW data in many different ways. There are many reasons that seeing degraded versions of RAW data vs. compressed data might not be very noticable, the primary one is that the projection you use does not really reach 4K resolution uncompressed, and the way the 4K uncompressed RAW files were processed before display.

I do not know of any display that can show a whole 4K single pixel checker board pattern in white, red, green, and blue on a screen where you can distinctly see the checker board pattern, in that case you are looking at a 2K display or less at 80%MTF, so the affect of the low pass filtering on the images would be less noticeable. In the future there may be better displays, so if you have true 4K RAW sensor data you can then see the datail that would have been lost.

If you have true RAW sensor data to work with you can get better Black and White images since much of the low pass filtering would not be needed.

The workflow for true RAW data is already in place, I work with "RAW" files from our film scanner without problems, you just need to do some extra adjustments for the white balance and chroma masking, but those give you more control over the final look of the images. There are "free" programs to de-Bayer DNG files of "raw" sensor data.

The DNG file type that A-cam dII will use seems to store the RAW data along with meta data that is to help reduce the number of adjustments to the de-Bayer program. Such meta data is not always a good idea since you can pick the white and black clip points better yourself by doing it manually (looking at the RGB histograms from the RAW file), and there by avoid off color highlights or shadows, if meta data is used it can reduce the dynamic range of the sensor data more than is needed if the light is a bit off the K value assumed when the meta data was formulated.

Being able to see the histograms of the true raw sensor data also helps you filter the camera to balance the sensors to get the maximum dynamic range, something you cannot do if you can only look at de-Bayer data that has been white balanced by using meta data clip points.

So there does not need to be any more steps to the workflow if you have metadata like in a DNG file, but if you want the highest quality the sensor can give it is better to add an extra step where you review the histograms of the raw data for each shot and make small adjustments to get the maximum dynamic range, chroma range, and detail vs. aliasing. The detail vs. aliasing issues comes up since the each lens will be soft edge because of the OLPF to a different amount at each f/stop and each lens at a given f/ stop will get extra negative spherical aberration from the sensor OLPF and cover glass.
 
You must log in or register to reply here.
Back
Top