We will always tell you the truth about our development.

[Gavin Greenwalt]

I think mmost's criticism would be valid in some situations and not in others. I know some applications have a habit of feeling the need to load an entire file (r3d file) even if only 2 frames are needed from it which can suck up a lot of memory very quickly.

It's the classic framesequence/mov performance debate.

To see this behavior load an uncompressed quicktime and a tiff sequence into a bunch of compositing applications. Some will handle it ok some will handle it terribly.

 

[M Most]

Using RAW files from RED (.r3d) - keeping all the files "on line" for entire feature (in 4K!!) is under 5T - that is well within the management scope of even a small shop like ours - and yeah, bigger budgets have bigger shooting ratios - but even 8 to 10 T is no big deal these days. We are keeping ALL camera master .r3d on-line on a san connected to our Scratch system. This way at any time during the edit - we can drop in a edl and screen or export or test grades -

It is still a big deal if you're doing work for clients that actually has a deadline. First, it takes a long, long time to copy 8 TB of material, no matter what drives you're using. Second, it requires copious amounts of storage for an extended period of time on a SAN that the client is not likely to be willing to pay you for - and ultimately, if you have to park 8 TB or more (it will be more when you start rendering the output files) of material on your SAN for any length of time, that's a lot of space you can't use for anything else. If you want to have two projects going at the same time, you're boned. Now, if you're doing only your own projects, you can set this up any way you want. But if you're working with clients, it's very difficult to justify - and worth avoiding if there's a way to avoid it (like maybe exporting a Raw frame sequence - or truncated R3d files - from Redcine - something I've suggested on another thread).

And we haven't even mentioned the performance issues (how long do you think it's going to take Scratch to load a project that involves 8 TB worth of source files that have to be reconciled every time you load the project??).

 

[Nick Shaw]

I think it is also worth bearing in mind that since the camera generates a separate .r3d file for each take, only the takes that have been used need to be kept available during the finishing process. The rest could be deleted from the SAN (having been archived of course) once the cut is locked.

Documentary type work could still be an issue of course. I hope it won't be necessary to keep the entire .r3d file of a long interview permanently available just because one short sound-bite was used. I'm certainly hoping it will be possible to consolidate .r3d files. Since they are based on intra-frame compression, there seems to be no reason that should not be relatively easy.

 

[M Most]

I think it is also worth bearing in mind that since the camera generates a separate .r3d file for each take, only the takes that have been used need to be kept available during the finishing process. The rest could be deleted from the SAN (having been archived of course) once the cut is locked.

While that's true, it is also true that if all the original material is given to editorial, the chances are very, very good that at least some part of almost every take will be used. The notion that you can cut down the required footage by any significant amount seems somewhat naive to those of us who are quite experienced at finishing longform material.

The notion of what a SAN is used for also seems to be at odds with what finishing facilities do. A SAN is used for working storage in a finishing facility, not as a permanent storage vault. The purpose of a SAN is to allow shared access to material as well as reliable high bandwidth access and reliable high speed data retrieval and real time playout. This allows for the conform process, and later the color correction process, to proceed while other steps such as visual effects, dirt cleanup in the case of film, and other things are also taking place. It also allows for a central "drop off" point for rendered material that must then be accessed elsewhere (such as a film recorder). Material is only loaded on the SAN when it is needed for the final conform, not during basic editorial, which is done elsewhere. Hence my comment about how long it takes to restore or copy 8 TB of material for one conform.

 

[Mark L. Pederson]

The notion of what a SAN is used for also seems to be at odds with what finishing facilities do. A SAN is used for working storage in a finishing facility, not as a permanent storage vault. The purpose of a SAN is to allow shared access to material as well as reliable high bandwidth access and reliable high speed data retrieval and real time playout. This allows for the conform process, and later the color correction process, to proceed while other steps such as visual effects, dirt cleanup in the case of film, and other things are also taking place. It also allows for a central "drop off" point for rendered material that must then be accessed elsewhere (such as a film recorder). Material is only loaded on the SAN when it is needed for the final conform, not during basic editorial, which is done elsewhere. Hence my comment about how long it takes to restore or copy 8 TB of material for one conform.

Mike -

That's all true -

but as I am sure you know - there is more than one way to skin a cat -

as for time to restore - well ... where there's a will ... there's a way -

if I told you how fast I can restore/conform a 100 minutes of Redcode Raw .r3d files from an edl ... well ... you would not believe me -

 

[M Most]

Mike -

That's all true -

but as I am sure you know - there is more than one way to skin a cat -

as for time to restore - well ... where there's a will ... there's a way -

if I told you how fast I can restore/conform a 100 minutes of Redcode Raw .r3d files from an edl ... well ... you would not believe me -

Well, if you're using Scratch (as we do) there's no real secret to that - but if you want to use things like scaffolds (and I don't know how one can do a final color correction without them), staying in Raw is not necessarily the best option, at least until some other things within Scratch change. But on a full length feature, you're going to have to restore a hell of a lot more than 100 minutes. In most cases these days, that's only one day's worth of dailies. However, there are the generally held beliefs and then there is the reality, as I'm sure you know. Nothing is finished until it's completely finished and deliverables are created. That's the part that many here fail to understand.

 

[laguun]

Well, if you're using Scratch (as we do) there's no real secret to that - but if you want to use things like scaffolds (and I don't know how one can do a final color correction without them), staying in Raw is not necessarily the best option, at least until some other things within Scratch change.
But on a full length feature, you're going to have to restore a hell of a lot more than 100 minutes. In most cases these days, that's only one day's worth of dailies. However, there are the generally held beliefs and then there is the reality, as I'm sure you know. Nothing is finished until it's completely finished and deliverables are created. That's the part that many here fail to understand.

we recommend:
after extensive testing for the full features shot on red, and the shooutouts between the di systems:

to buy an areca (or else) 24 Terrabytes (or 48) Terrabytes controller.
That will give you 4k 24p realtime.

> 1200 Mbytes sustained throughput, at least on the Win 64 system i am writing this.

More than enough for real 4k uncompressed for -any- , VFX, 3D, editing, plugin etc.

Working in 1K redcode realtime in scratch isn´t a good alternative for those ho don´t want to afford a good raid.

a good idea, if you intend to spot sharpness or the 1st acs focus pulling competence or to see noise in 4k, it would be stupid to work with any redcode realtime system as assimialte or fcp.

you need bigger guns for this.


Working below realtimein scratch in 2K is ok, but our directors and dps complain "why they can´t have 2k realtime as before".

Therefore - we decided to go the high quality route.
4K realtime.
2K realtime

and this isn´t available with -any- redcode debayering application yet. be it win/scratch or osx/FCP - and won´t probably be available until Q1/2 2008 with any offering. 4k/2K realtime is however avaiable from any manufacturer -now. if you wirk uncompressed.

we use uncompressed, high power 4k.
In a secure Raid 6 or 5.
if someone wants informations about what kind of drives and cotrollers: WD & areca.


btw, 24 Terrabytes are ~$$$$ today, not $$$$$.

P.S. its fun to see discreet combustion rotating with 25p with color correction and blur in realtime as well.

 

[M Most]

to buy an areca (or else) 24 Terrabytes (or 48) Terrabytes controller.
That will give you 4k 24p realtime.

On one host. This doesn't give you facility wide sharing of material, it gives you very fast storage on one unit - an island, if you will. This might work for some. It doesn't work for many.

a good idea, if you intend to spot sharpness or the 1st acs focus pulling competence or to see noise in 4k, it would be stupid to work with any redcode realtime system as assimialte or fcp.

You know, I respect the fact that you clearly have some expertise in this area. However, you are not the only one who's smart or right, and the rest of us are not dumb and stupid. Show a little respect for others using other approaches. You might find that it gets returned.

...and this isn´t available with -any- redcode debayering application yet. be it win/scratch or osx/FCP - and won´t probably be available until Q1/2 2008 with any offering. 4k/2K realtime is however avaiable from any manufacturer -now. if you wirk uncompressed.

we use uncompressed, high power 4k.

And that's fine. But for many - if not most - facilities, concurrent workflows and shared storage are considerably more important and more useful than 4K real time, particularly as 2K is the predominant distribution - and delivery - medium in the first place. FYI, none of the largest DI vendors in the world project anything larger than 2K while performing DI color correction work - because it just isn't necessary. You're entitled to your opinion. But what you consider important just isn't that important to others. And they're entitled to their opinion.

 

[laguun]

On one host. This doesn't give you facility wide sharing of material, it gives you very fast storage on one unit - an island, if you will. This might work for some. It doesn't work for many.

if you want to san this - add some $$$$ and you are good.
indeed, i think it is completly necessary to have >32 render CPUs munching through the footage as renderslaves for longform. for this, you dont need a san, multiple gigia ethernet swiztched will do.

You know, I respect the fact that you clearly have some expertise in this area. However, you are not the only one who's smart or right, and the rest of us are not dumb and stupid. Show a little respect for others using other approaches. You might find that it gets returned.

honestly - i am not attacking anyone.
i simply have to layout postproduction workflows and production methods for 3 fullfeatures now.

therefore, i am allergic to any "will do", "should/could work", "you don´t need to see the full resolution" approach. As i exactly understand the possible consequences.

For red->hd or 2k or pal/ntsc, this is different. for red in mastering, di etc environment i simply describe --facts-- and i can´t seew who should feel attacked by this, in the contrary.

the workflow requirements i want to have:
4k
realtime
>10 bit
solid proxy 2k system, realtime as well
mid price
networkable
top quality open access to -maya/3ds max/xsi- in 3d, -fusion/ae/combustion&shake- in VFX.

and after many tests i found our solid good, available right now way for this. fcp/shake and redcode aren´t the solution for us, nothing wrong with that, no pun intended.
that can change soon, and once redcine is released things will be different anyway.

And that's fine. But for many - if not most - facilities, concurrent workflows and shared storage are considerably more important and more useful than 4K real time, particularly as 2K is the predominant distribution - and delivery - medium in the first place.

i would say 1080p, pal and ntsc are the dominant delivery formats. on digibeta, hdcam, blueray/hdvd.

FYI, none of the largest DI vendors in the world project anything larger than 2K while performing DI color correction work - because it just isn't necessary. You're entitled to your opinion. But what you consider important just isn't that important to others. And they're entitled to their opinion.

i am talking facts.
not opinion.

i didn´t say that anyone should want/need/require a 4k workflow.
i just described a $$$$ 4k discsubsystem and there are several $$$$ softwares to allow 4K post and benefit of this.

therefore don´t rant at me.

 

[M Most]

i would say 1080p, pal and ntsc are the dominant delivery formats. on digibeta, hdcam, blueray/hdvd.

I was referring to delivery for theatrical distribution, be it film or digital cinema.

i am talking facts.
not opinion.

i didn´t say that anyone should want/need/require a 4k workflow.
i just described a $$$$ 4k discsubsystem and there are several $$$$ softwares to allow 4K post and benefit of this.

therefore don´t rant at me.

Then don't say things like "it would be stupid" to do what others are doing.

 

[Allan Stallard]

I have been reading this discussion enthusiastically from the first post. I just want to point out, politely, to you laguun that you do say some inflammatory things. I believe it's part internet, part first/second language, part frustration. You certainly are entitled to say whatever you want, but if you want people to take you seriously, (perhaps you don't care) then take more time to phrase what you're saying.

So now you are saying that only an uncompressed 4K working/monitoring solution is what will make you happy? I would really question why you wouldn't want to concentrate on a 2K uncompressed workflow/monitoring...especially considering that you are trying to do this "mid price".

 

[I Bloom]

I just want to say, that I've just started to try posting RED footage myself, working off a Macbook Pro. Laguun I have to say that after reading this whole thread, you have nothing to worry about mate, the dream is real! Give it another couple of months and you will be loving this stuff, and swimming in RedCode like Scrooge McDuck in his money vault.

And yes it seems clear to me that you'll be able to grade on any system, whether they support R3D natively or not. It's simply a matter of achieving realtime and how much space is neccessary to contain the conform.

IBloom