Onboard downconverting

[Christoffer Glans]

I need to get my head around something so please help me out.
If, or rather when I get my own RED, I will most likely shoot 4K or 2K, then downconverting it to 1080p - edit and make the final copy a 1080p master.
For me, I cannot work with higher resolution, because I don't have the money to buy that kind of editing system.

So, If I shoot 4k or 2k, can I convert it to 1080p onboard the camera before it is stored on the CompactFlash (which I find a very good storage option)?
That way I can get much more footage onto one 8gb CF card.
The reason I don't want to shoot 1080p directly is the same as why a dvd made from 35mm footage looks better then something shot in SD resolution.

So, is onboard converting in "realtime" possible? Or is the workflow I need to go with: Shooting 4k/2K - transfer data from CF card onto a laptop with REDCINE - converting it to 1080p - store on harddrive.


For me, the only thing I miss with HD and high resolution is the tape. CompactFlash is really something that could be the "future tape", so I'm trying to figure out the best way to use RED as cheap as possible, but still get as much footage as possible. I shoot aprox 5 hour of footage for a 12 minute film with my old DVX and it's just my way of making movies, I want many takes and variations.

Any sheds of light?

 

[Gavin Greenwalt]

If you haven't ordered a RED yet. Yes. Onboard downconverted RGB will be made available through a software update at some point in the months following the release.

However... you won't see as big of increase you might hope in your recording time.

Here's why:
4k Raw is one color per pixel.
1080p is 3 colors per pixel.

1080p is roughly 1/4 the pixels of 4k. But 3 times as many colors.

So that means you're seeing: 1/4 (3) = 3/4 or 75% the datarate of 4k.

If 4k is 24MB a second, 1080p is about 17 or 18MBs. Is it worth a 75% loss in resolution for a 25% drop in bandwidth? Your choice.

Or the converse is: Is a 33% increase in record time worth losing 75% of the resolution?

 

[Christoffer Glans]

Sometimes it is, but then again, what will I edit it on? How much do you think a 4k editing system costs? I will not be able to edit movies on my system at home over 1080p. If I manage to get a budget for a feature film, I can edit in 1080p, saving the 4K material, then batching it at some post-prod company, but for my usual shortfilms, music videos etc etc, it's better to go 1080p.
It's not so much about storage space but more about what I have avalible and if I put $24700 on a RED kit, I will not have much money left to buy an expensive post-prod machine.

Sometimes I see people in here saying: "If you can afford this camera, you should be able to afford this n that". Well, money is spent on the camera and as long as you don't own a big corp, it's not a thing you can buy like a coffee at a café.

Anyway, 1080p is enough for many things. I don't know if Fincher batched his movie, but Zodiac was edited in 1080p and most times 1080p is enough resolution to show your movie in. Have you seen a blu-ray movie at 1080p on a 1080p projector? I think that is enough for some time until cheaper storage and faster system comes around. 2K is still something that the usual consumer knows nothing about. It's not geeks and techteams that are going to see our movies, it's the audience outside of this "art-industry"

It's the workflow, rather then storage space, but 33% increase is better then nothing.

Do you mean that later serials of the camera will have onboard realtime conversion to 1080p? That means I will get aprox 6-7 minutes of 1080p footage on a 8gb CF card?
That mean I could shoot 6-7 minutes, switch cards, transfering the full card onto a harddrive, while shooting with my second card?

That is enough for me at this moment. I think the real deal with this camera for me personally is that I will never look at another camera again for many years to come. I am future proof in that when cheaper storage and editing system arrives, I will not have to upgrade to a new expensive camera, I will just step up my output from my RED.
Then I can finally focus on my art, rather then tech.

 

[GlennChan]

I don't believe the Red team has nailed down the exact data rates for Redcode RGB??

2- You can edit the 4k RAW Red footage at low quality and then conform to HD or 4K quality. Different ways of doing it:

A- Run the 4K RAW material through Redcine to convert it to something your NLE will take. Once you have edited your material, you export an EDL/XML to Redcine and it will convert the footage to HD (or 4K), with high-quality de-Bayering.

B- Each of the 4K RAW files will come with a Quicktime proxy. The Quicktime proxy will extract a lower-resolution version of the image... e.g. 1K. You edit the 1K proxies.

Then spit out an EDL/XML for Redcine. It will convert the 4K RAW files to 4K or HD.

For HD output, you ?might? get a slight advantage in shooting 4K RAW and downconverting in Redcine. Since Red uses a Bayer pattern, you have to debayer the image to get a RGB image. A high quality debayer takes a lot of computing. If you shoot Redcode RGB, the debayering has to happen in the camera. This is speculation, but the in-camera debayer may or may not be the highest quality. By doing it in post, you aren't limited by needing to be real-time so you can use a slow, high-quality algorithm. You can also use improved algorithms if the Red team comes up with them. With the camera, the debayering will be programmed into a FPGA and it's not as easily upgradeable.

So for high-quality, you might get slightly better results from shooting 4K RAW instead of RGB. Workflow-wise, it shouldn't be much additional hassle to shoot 4K RAW. The only added time is waiting on Redcine to debayer + downconvert. That's what sleep is for.

If Jannard's words are any indication...

Don't use REDCODE RGB... it will be the last thing we enable (it sucks). Shoot REDCODE RAW (it ROCKS).

BTW, REDCODE RAW is native in FCP 6.

Jim

3- Reddrive should give you a lot of recording time. There are bunch of threads about this... Redflash is more reliable, Reddrive will give you more recording time.

 

[Jeff Kilgroe]

Sometimes it is, but then again, what will I edit it on? How much do you think a 4k editing system costs? I will not be able to edit movies on my system at home over 1080p. If I manage to get a budget for a feature film, I can edit in 1080p, saving the 4K material, then batching it at some post-prod company, but for my usual shortfilms, music videos etc etc, it's better to go 1080p.

If you can edit 1080p, then you'll be able to edit 4K by way of 1K proxy pretty easily. Or at least that's the way it looks. What type of system do you have or will you have to edit with?

As has been pointed out, there's several ways to do this. And you're probably not going to want to shoot in any of the on-camera RGB modes unless there's a specific reason to do so. You lose a lot more than just resolution... Once you convert to RGB, you lose the ability to work with your imagery in a RAW workflow and you lose a lot of the flexibility for correcting exposure, latitude, coloring, etc..

At 24fps in 4K REDCODE, you can get a bit over 3 hours of record time on a single RED DRIVE.

 

[Christoffer Glans]

I will shoot 4k RAW, but downconverting it to a 4:4:4 1080p output isn't too bad is it?

Anyway, my system is a 3,4ghz dualcore pentium machine with 3gb of RAM.
The editing system on it is Avid Xpress PRO HD 5,5 (or whichever version it is).
So, as I said, I'm only able to work with 1080p when editing and I can't play 4k files on my system.

 

[Jeff Kilgroe]

RED should be usable with AVID Xpress Pro. I would recommend you wait and see what happens when RED starts shipping and which systems can do what. I think you'll be able to edit the 4K. A dual-core 3.4GHz P4 isn't a shabby system... Not when many of us are thinking we'll be able to edit 4K on a 2.3GHz C2D notebook. And like I and others have said, you should be able to edit 4K by way of proxy and then conform to 4K, 2K, HD or whatever when ready. Why downrez and edit in 1080p when you can edit a 1K or 2K proxy and still retain the 4K benefits? I think that you may have to beef-up your hard drive(s) possibly or update your AVID to the latest Xpress Pro version, but that should be minor things in the grand scheme of being able to shoot and edit 4K.

Anyway, right now we're making a lot of assumptions. But from what was shown at NAB and based on how REDCODE allows low-res proxies generated right from the 4K footage, it seems counter-productive to plan on a down-convert just so you can edit in a format equivalent to a proxy that you will have direct access to.

 

[Christoffer Glans]

What will I use as a master then? Will I have a master stored on a harddrive? It just feels too unsafe. Best way I think, is to make a 35mm master, because it will make the frames physical... though, I will never have the money to do that. So how do we store our 4K masters?

 

[Craig Schober]

storage

storage

What will I use as a master then? Will I have a master stored on a harddrive? It just feels too unsafe. Best way I think, is to make a 35mm master, because it will make the frames physical... though, I will never have the money to do that. So how do we store our 4K masters?

build a cheap raid or even better, build a few cheap raids for redundancy. buy a few drobos. the chances that all drives in all of them fail in the next few years is statistically "impossible". if you like tape then use that. no matter what you choose, it beats 35mm in terms of cost, time spent and future flexibility. once you print to 35mm (probably only a 2k scan anyway), you're going to alter your 4k data forever. some would say that the introduction of grain and downrez to affordable 2k scan is great-more like film. but once you do that, you're stuck in film without a ticket to get back to pure 4k.

 

[Christoffer Glans]

You forget that a 35mm master can last up to 100 years...
The digital age is fascinating, but still, the lifetime of what we store digitally is just plain scary. That's why.
If I do a feature film, I will choose to do a 35mm master, maybe storing it also as a digital, but the 35mm master will inprint my movie as physical art in this world. Remember that a physical frame that you can hold in your hand is going to last much longer then hoping a raid put in on the same shelf will.

 

[Craig Schober]

film won't exist in 100 years

film won't exist in 100 years

You forget that a 35mm master can last up to 100 years...
The digital age is fascinating, but still, the lifetime of what we store digitally is just plain scary. That's why.
If I do a feature film, I will choose to do a 35mm master, maybe storing it also as a digital, but the 35mm master will inprint my movie as physical art in this world. Remember that a physical frame that you can hold in your hand is going to last much longer then hoping a raid put in on the same shelf will.

35mm is fine if you have the money but you also don't want to sacrifice the ability to afford and produce future work just to save one film for 100 years...and that brings up other issues. film must be stored in a certain position in a temp/humidity controlled space. and unless you're using a fireproof room, it won't survive any contact with flame.

but that brings us to the bigger issue. if you want your work to outlast you, it won't on film unless you consider "lasting" some reels of film sitting in storage. 100 years from now, there will be millions of film reels and no way to view or scan/transfer them. like now, 99% of viewing will be done on tv or dvd or vcr or on a computer. that last 1% will disappear in about 50 years. sure you'll be able to go to a museum and fire up a projector or film scanner but no one will have equipment or knowledge of old film systems in 50-100 years from now. i know, i know, people have been saying this for years now but we've never had a quality replacement for film before, only a more convenient digital replacement. now with systems like red we have no reason to pause before taking the digital leap. the film industry has evolved over the past 100 years at an analog pace. the digital evolutionary pace is 1000 times faster so even 10 years from now, red 4k will probably already be obsolete but just think of how many less people will be using film. no one will know and only a few historians will care.

i like film and it will be just fine for the next decade or so but consider it's rare silver nitrate base, the harmful processing chemicals involved, the fact that kodak and fuji have both made official announcements that digital is the future. it becomes clear that the countdown on film's life has begun. maybe my timing is off but i fully expect to outlast my film prints that i have stored. i only keep them now for nostalgic reasons.

 

[Christoffer Glans]

There is one major thing you forget. 35mm or any film is made out of physical frames. You can take a part of it, hold it to the sky and see the frame.
And a HDD isn't fireproof either. The thing is that film can be stored without me ever needing to worry about it. The films I made that are stored on a DV master are decaying and I need to make copies after copies to hold them at a fresh state. With film, you might just need to make one copy every hundred years or so.
Yes, the digital age is coming and it will take over, I'm not stupid enough to think otherwise, but there is still not a safe way of storing digital data.
Before the digital age is as safe as artwork painted on a stonewall, I don't see any good sides of the storing part.

People get so clouded in how easy it is to work with digital media.
We have already experienced this with digital photography.

When people started to buy digital cameras instead of regular, they got photocrazy and photographed everything. After a couple of years some did get to feel the dark side of it all. Some lost their harddrives or discs, some lost data in other ways. The whole thing started to go down, because people realized that digital photography meant that all their precious memories could be wiped out in the blink of an eye.

So, what did the companies do? They started selling printers at the size of the good old photocopies. People could now print their photos, put them in albums and save them as physical photographs.
This is what it's all about, there isn't a safe way of storing digital data.
The digital age is as dark as the middle age, not because of terror and suffering, but because this is an era that is actually facing the possibility to be a gap in human history. We have records and history from several thousands of years behind us, but in the future, some people might not be able to find anything of use from this period.

Think about this:
Archeologists from the future world of "new earth", a follow up to our planet, some lightyears from here some lightyears from here, comes across a hidden stash in the basement of the old destroyed parts of lets say, Stockholm, on their expedition to the old earth. The stash contains four things, one is a roll of a thin, but wide black and shiny plastic material.
One is a grey block of some plastic material with a long, very thin, grey band.
One is a circular, very shiny disc with a hole in the middle.
The last is thing is a metallic block with a text written in some latin ancient language. The letters spells "LaCie".
Now, the archeologists discuss what these things are and one of them opens up the metallic block. He finds a golden disc and they speculate that this was a storage for valuable discs like this other one, which the other puts a thread through and hangs around his neck. They put forward the theory that these people used these light shiny objects as jewelry and that Lacie is someone that selled these or maybe even someone they might have worshipped, thus the robust way of keeping the golden disc inside.
Then they take out the grey and plastic block. They rip out the band and figure out that it was used as a small strong thread, maybe even the one to hang the jewelry. The grey block was just the way of storing the thread.
But the last item had something on them, so one of the archeologists put a light behind it and they all saw an image. Each frame progressed what was happening so they built a machine that could move this long roll of shiny material. Now they could see moving pictures and discover what the other three things was really used for.

We are not there yet. The digital age is the age when books are gone and everything is stored digital. This is why I still love to make a 35mm master or copy if you like, because it's physical. I love to work digital, but I have never been fond of storing my things digitally. I have lost some precious things and
that is what haunts my opinion.



Anyway, I want to ask about the CF cards
If I have two 16gb Extreme IV (let's say they exist in the future) and a port to unload them onto a laptop. Would that be enought to shoot 4K REDCODE RAW for an entire movie? (If the main storage is harddrives connected to the laptop. Would the workflow then be:
1. Shoot onto one card
2. Unload the first card in the CF slot connected to the computer
3. While unloading, shoot with the other card.
4. When finished shooting on that, switch cards and do the same.
5. Continue throughout the day of shooting.

If so, that would be great. Is that how Sodenbergh will shoot his movies?
Maybe with more cards, but anyway.

 

[Charles Angus]

Film lasting for 100 years is not entirely accurate, as far as I know. It's not like it doesn't degrade over that time. Ever seen a print from the 50's/60's/70's that's all magenta?

Just saying, if it's archival you need, look into it and make sure that's what you're getting.

I may be wrong, but I think the best way to store film very long term is to separate into RGB and make a black and white print of each channel.

Charles

 

[Jeff Kilgroe]

You forget that a 35mm master can last up to 100 years...

This topic is a dead horse and has been beaten to a pulp already on this site and elsewhere.

The fact of the matter is, the 100 year claim or whatever number you want to designate to a film transfer is just a number. There is no guaranty that a 35mm film master will last 100 years. None of the current stocks (or any color stock for that matter) has been around 100 years to test that theory. Film, like any other form of media, degrades over time. Even in a controlled environment. How much it degrades will depend on how long it sits, what type of film stock, environmental conditions, etc.. There is also a threshold of acceptable loss. How much degradation will occur before it's noticeable? How much is acceptable? How long will all that take?

The digital age is fascinating, but still, the lifetime of what we store digitally is just plain scary. That's why.

One thing to keep in mind is that many digital storage methods, primarily optical disc, use the same photochemical processes that are used with film processing. So on a chemical level, they're just as reliable as film. So that can be good or bad depending on your point of view. Remember, Kodak was one of the original creators of CD and CD-R media and claimed a 100 year shelf life for many years.. Really until the mid-90's when discs started to fail is when they kinda started to admit the 100 years was a bit optimistic.

But like film, photochemical media degrades over time. And when we're talking about very small binary dots, that's a problem. A slight shift or change in a couple of those dots may not even be noticeable to the human eye under a magnifying glass, or even to all reading equipment, but could be enough to get erroneous data in many CD drives. With film, what may be degradation in the form of a slight color shift or fading obviously doesn't destroy any data, it just exhibits altered optical properties. ...Which may or may not be fixable depending on the extent of decay.

If I do a feature film, I will choose to do a 35mm master, maybe storing it also as a digital, but the 35mm master will inprint my movie as physical art in this world. Remember that a physical frame that you can hold in your hand is going to last much longer then hoping a raid put in on the same shelf will.

I think a lot of people approach digital archival with the wrong mentality. With film masters and negs, it makes sense to preserve the original to the best of your abilities. Long shelf-life is valid concern. With digital archival, you have to look at the benefits. There is no decay or generational loss of what is being stored as long as the data is preserved and/or perpetuated. 1000 years from now, it will be identical down to every last pixel as when it was originally created. Film masters will not do this, no way, no how... The methodology behind preserving digital information is obviously different. With film, there's obviously the huge expense of creating the film master and then you need to provide a proper environment for storage of that film and that costs money too. In some cases, you may even be renting storage space in a climate-controlled room to store a few film cans. With digital archives, the environment isn't as touchy... It still needs to be fairly dry with a very consistent humidity level and within a decent temperature range. But in reality, a shelf in a closet is probably good enough if you keep the dust off the drives/tapes themselves. Now where the methodology change kicks in is that with digital archives you will need to revisit, condense and transfer to new media every so many years. 5 years is probably a bit more often than necessary, but also affords some benefits in the process. For example, if an original is made on LTO-3 and duplicate copies are made, in 5 years, duplicate copies can be made to a new format like LTO-5. In another 5 years, make two new copies to solid-state media, and so on and so forth. It's best to store them in various locations and not all in the same place too. When you start running out of storage space, you can begin to toss the oldest archive form.

Anyway, with all that said, the archive needs can and will vary with each project. If you're going to create a film digitally, you're going to have a final digital master... Makes perfect sense to perpetuate that as long as possible. If you're going to distribute that same feature on 35mm film, then you'll have a film master as well. May as well store that... Then you have the best of both worlds. Another way to put it, as far as RED is concerned, you're shooting digital. You would be a fool to not keep a full-quality digital archive of any important work. If you can do a film transfer and provide adequate storage, that's just gravy.. However, I think you'll find the digital archival process isn't as dreary or unreliable as people make it out to be. Huge government institutions, large companies, etc.. have been making data archives for years. The US Library of Congress archives multiple Terrabytes of data each week. A small production company with a handful of employees and one or two RED cameras has data requirements that are both small and easy to manage by comparison.