RAID I/O questions

[J.D. Frey]

I'm looking at various FC arrays (HP, Apple, G-tech) out there as an archive/edit solution to use when the RED and all it's goodies show up at my doorstep.

Most of the high capacity FC raids (in the $12,000.00 range) look like they get around 1600 MBps. Is that enough I/O to work with 4k? 2k?

Will I want the RAW port option as well? I'm assuming that this will allow me to tether the camera to an FC RAID in the studio? If not what is the Raw port?

Thanks!
J.D.

 

[J.D. Frey]

Sorry- I just saw the pipeline diagram so that answers question two.

So any recommendation on the type of big drive RAID I should look into?

The Apple Xserve raid looks like the best price I've seen so far, However HP has a FC kit that comes with head unit and an array unit (no drives) plus two FC cards. Anyone used either system?

 

[david farland]

Not sure what all the real specs are on the Xserve. Saw one version using 2Gbit fibre which is probably not backwards compatible with Red's 10GBit ethernet fibre (although Red is still to tell us the actual spec of the fibre channel. 4.5K/60fps uncompressed is 975MB/sec and 2K@120fps is 385GB/sec. I saw figures of 200MB & 375MB for the Xserve.

Cheers,

 

[Jeff Brue]

woah lot of misinformation.

4k Recode Raw is 28 MB/s at 23.98 it can play back off a firewire 800 drive.

4k Uncompressed DPX's 1,152 MB/s add 20% for overhead and ya need around 1400 MB/s

A single Fibre channel 4 GB device (ie a gspeed or half a ciprico 4210) can put out between 200-300 MB/s in protected mode (ie raid3 or raid5) thats for writes reads are higher. So at minnimum you would need 7 4 GB fibre arrays to be able to handle 4k or 4 ciprico 4210's. They also offer another laptop drive thing that has 4 fibre arrays in it, but still...

Either way if you're working in redcode raw you don't need it, unless you're planning on doing the final conform and grading.

 

[david farland]

woah lot of misinformation.

4k Recode Raw is 28 MB/s at 23.98 it can play back off a firewire 800 drive.

4k Uncompressed DPX's 1,152 MB/s add 20% for overhead and ya need around 1400 MB/s

A single Fibre channel 4 GB device (ie a gspeed or half a ciprico 4210) can put out between 200-300 MB/s in protected mode (ie raid3 or raid5) thats for writes reads are higher. So at minnimum you would need 7 4 GB fibre arrays to be able to handle 4k or 4 ciprico 4210's. They also offer another laptop drive thing that has 4 fibre arrays in it, but still...

Either way if you're working in redcode raw you don't need it, unless you're planning on doing the final conform and grading.

Was obviously answering queries on uncompressed rates gathered at 10Gb here, not processed footage. Missed the archive bit!

Cheers,

 

[Jeff Brue]

Was obviously answering queries on uncompressed rates gathered at 10Gb here, not processed footage. Missed the archive bit!

Cheers,

Ahh cool. Also don't think the optical port is 10 Gb take a look over at Dalsa's page at their 4x infiniband optical port. Conjecture on my part granted. But I've been talking to codex quite a bit and they've already got a realtime debayering algorithm and they can more than enough handle the 2k@120 fps bayer data.

 

[david farland]

Jeff,
Red have said a few times it'll be 10Gb fibre port & mentioned single mode, ethernet. Interested in why you think they're gone infiband?

Cheers

 

[J.D. Frey]

Thanks for the feedback. looks like what i'm reading in other threads is that the dual channel might be ok for archive but not anywhere near the level as an editing stream. I may just have to wait for 10GB to come down in price and look at some of the eSATA recommendations I've been seeing.

 

[Jeff Kilgroe]

Ahh cool. Also don't think the optical port is 10 Gb take a look over at Dalsa's page at their 4x infiniband optical port. Conjecture on my part granted. But I've been talking to codex quite a bit and they've already got a realtime debayering algorithm and they can more than enough handle the 2k@120 fps bayer data.

I really doubt they'll go Infiniband. First of all, they have not said Infiniband, Stuart has specifically mentioned single mode fiber. Second, 4x Infiniband can't deliver 10Gbps with it's 10bit/byte implementation. For Infiniband to deliver 10G, you would have to go to 5 * single-links (not going to happen), dual quad link (2 cables) or to 12x Infiniband. 12x is total overkill and I don't even know if anyone is offering it yet.

Edit:> OTOH, 4x Infiniband could actually work... It's not clear yet if the full 10G is really necessary. We keep seeing specs claiming about 900~950MB/s for 4.5K uncompressed @ 60fps. If that's the case, you can theoretically pack 1024MB/s through a 4x Infiniband connection. Just not the full 10Gbps 1280MB/s, which may not be needed.

 

[J.D. Frey]

So I'm guessing if you want to edit 4k you need something like the ExtremeStor DI? All the articles I could find said the array was around 30 grand. They claim 1030MB/sec for raid 5 though.

 

[Jeff Kilgroe]

So I'm guessing if you want to edit 4k you need something like the ExtremeStor DI? All the articles I could find said the array was around 30 grand. They claim 1030MB/sec for raid 5 though.

Hardly. 4K REDCODE RAW @ 24fps is supposed to be < 28MB/s. Most any desktop hard drive will handle that these days. It doesn't look like REDCODE RAW will be an editing codec and we're not sure about the rates for REDCODE RGB yet. So it will be interesting to see what is presented at NAB. The larger data rates we're looking at (approaching 1GB/s) have to do with capture of uncompressed RAW data from the camera. In most situations you wouldn't edit that natively, but would compress to one of the REDCODE formats or other desired formats first. The reason for capturing uncompressed from the camera is to gain access to the full 4.5K frame and/or higher frame rates than what the camera can compress onboard.

 

[Lucas Wilson]

So I'm guessing if you want to edit 4k you need something like the ExtremeStor DI? All the articles I could find said the array was around 30 grand. They claim 1030MB/sec for raid 5 though.

They claim it, and they achieve it. We have a lot of customers out there with Globalstor sytems. They make very good, solid hardware.

But let's be real about this - uncompressed 4K RGB workflow is still very much a science project and the almost exclusive realm of the high end, mainly because of the cost associated with implementing a 4K RGB workflow.

It is non-trivial and very expensive.

Hence the beauty of REDCODE...

Lucas Wilson
------------
ASSIMIATE, Inc.
Los Angeles

 

[J.D. Frey]

I think i'm finally catching on. So for instance I might shoot raw for say a slow motion FX shot then redcode compress it and work with the compressed version and keep the raw archived for whatever reason- a geroge lucas special addition maybe?

 

[Rob Lohman]

The camera compresses the RAW on-board to REDCODE (RAW) and lays that to disk / flash etc.

The only way to avoid this compression is go out through the RAW port where you get a bit higher resolution (4.5K), higher frame rates (up to 60 fps) and no quality loss.

We haven't announced / talked about the rest of this workflow. More on that later.