Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Primer for HDD speeds?

Shawn Nelson

Shawn Nelson

Well-known member
Joined
Dec 28, 2006
Messages
4,595
Reaction score
1
Points
0
Age
43
Location
Portland, OR
I am trying to understand HDD speeds better and how they work together. I am looking for a primer, good msg posting (here on Reduser or elsewhere) that would tell me...

-MBs rate needed for good 2k work in Color
-MBs rate needed for 4k
-MBs rate people are getting from various HDD solutions (internal, Raid 0, 1, 3, 5, etc).

Any links are appreciated!
 
4K (4096x2304) RGB 10bit @24fps:
850MB/s

4K (4096x2304) RGB 16bit @24fps:
1360MB/s

2K (2048x1152) RGB 10bit @24fps:
212MB/s

About color I have no idea if the gpu acceleration is able to do RT in 2K or even 1080 even if you have the hdd bandwith.

Tipically you get around 300MB/s with a pciexpress raid card with 4 internal drives put in raid 0.
More bandwith will ask more drives, like 8 and most probably ask an external box for those drives, then you need to look at gtech, caldigit or a raid card with external sata connection and buy a box with port multlipliers (and the drives of course).

One of the new elegant (IMHO) solution coming is ssd on an pci express slot, and it achieves very fast i/o BUT with a high price MB/$

cheers,
antoine.
 
Color can play at 16bits 2K real time from DPX files in the secondary monitor with disabled video out. I can say that the playback is perfect especially when the "synchronize refresh" is on...

But when video out is on then its dead... 12Fps or less...
 
Shawn Nelson said:
I am trying to understand HDD speeds better and how they work together. I am looking for a primer, good msg posting (here on Reduser or elsewhere) that would tell me...

-MBs rate needed for good 2k work in Color
-MBs rate needed for 4k
-MBs rate people are getting from various HDD solutions (internal, Raid 0, 1, 3, 5, etc).

Any links are appreciated!
Click to expand...

Hi Shawn- the bottleneck is often on the bus... even if you have the thruput...

not to mention the need/screaming desire for a FCS/Snow Leopard update on the GPU/Debayer side!

but one has to wonder what the high end boys will do if someone can achieve this on a desktop? its getting closer every year... but one also wonders if patents come in to play at some point... i.e quantel... who
owns a ton of them... and is proprietary

but then again, a patent is only worth the $ you have to defend and/or prosecute it...
 
any links to this solution antoine?
Click to expand...

he probably means this

http://www.fusionio.com/Products.aspx

about $30 a GB

but there is also this
http://www.tomshardware.com/news/OCZ-Z-Drive-RAID,7181.html

about $15 a GB

also all the data rates posted here are for uncompressed....

cineform 4k can run 70-90mb per sec....and 45/sec as cineform Raw...
could do it with one single hard drive.....and different from Redcode...cineform can also be the render codec

also with a single core i7 it can real time debayer 4k....
http://4.bp.blogspot.com/_bkp96efBnsQ/SSSk1zCSagI/AAAAAAAABAE/YF7pgPtDbIU/s1600-h/Corei7chart3.png

I don't think color can take it in natively though

to do real time without cineform...you'll need a raid 0 array....ideally with 4 drives, or a single one of the ones listed above....only will get you real time 2k mind you
 
yeah fusion i/o, not really a solution when you need TBs, but for media cache like for a compositing package it is really ideal. And next year I guess we will see major price down...hopefully.

And yeah sure I was talking about uncompressed rgb, not compressed raw (redcode) which is very easly handled with one sata drives, but is cpu intensive.

What's the frame rate of cineform 4K rgb (4:4:4)?

Thanks,
antoine.
 
Is "presentation" a better debayer than "simple debayer"? And "finishing" a full debayer?

Which would be, out of the rates on the chart, the ones that concern people working at 2k for 1080 finishing using FC + Color + AE, or Premier + AE?
 
laboprod said:
4K (4096x2304) RGB 10bit @24fps:
850MB/s

4K (4096x2304) RGB 16bit @24fps:
1360MB/s

2K (2048x1152) RGB 10bit @24fps:
212MB/s
Click to expand...

Also be sure your RAID can sustain a good 15-20% higher than those rates in order to continuously guaranty the performance.

Apple Color will grade 2K in real-time on my 3.2GHz OctoMac and RAID-5 pushing around 325MB/s. However, when you start adding additional masks, layers, etc.. it can bog down quick. Color isn't the most efficient software by any means and on a system with plenty of fast storage it will still choke.

Like Tim said, you can run into bottlenecks on the peripheral bus. And elsewhere too. Most host controllers have a bandwidth limit of some sort. For example, the CalDigit RAID card can make use of all your internal drive bays, plus 4 external RAID enclosures. However the card tops out around 500-600MB/s, even though you could easily build RAIDs that operate faster than that. Most real high-performance storage systems (500MB/s and faster) will use multiple host controllers in order to achieve the necessary bandwidth.
 
I am sorry Shawn if I am sort of hijacking your thread, but I am trying to understand the same sort of thing you are asking, only with much lesser of an understanding about the Red workflow.

Is this the way it works in Color?: You get your masks etc rendered to the RAID, and then the CPUs use those to create the final composite in RT, then send it out through your Kona or other card to a monitor? (real basic question, but I want to understand what does what)
 
Roberto,

Everything in Color is done on-the-fly by the GPU (the GFX card). Nothing is written to disk until your are done and export your finished files from Color.


-A
 
Jeff Kilgroe said:
Also be sure your RAID can sustain a good 15-20% higher than those rates in order to continuously guaranty the performance.

Apple Color will grade 2K in real-time on my 3.2GHz OctoMac and RAID-5 pushing around 325MB/s. However, when you start adding additional masks, layers, etc.. it can bog down quick. Color isn't the most efficient software by any means and on a system with plenty of fast storage it will still choke.

Like Tim said, you can run into bottlenecks on the peripheral bus. And elsewhere too. Most host controllers have a bandwidth limit of some sort. For example, the CalDigit RAID card can make use of all your internal drive bays, plus 4 external RAID enclosures. However the card tops out around 500-600MB/s, even though you could easily build RAIDs that operate faster than that. Most real high-performance storage systems (500MB/s and faster) will use multiple host controllers in order to achieve the necessary bandwidth.
Click to expand...

this is where an item like this gets more and more attractive
http://www.tomshardware.com/news/OCZ...RAID,7181.html

it is more expensive, until you factor in Raid controllers....and the some times neccesity of multiple Raid controllers....still limited in size...but in a year will be the clear choice.

interestingly the Fusionio just got twice as fast....now 1.5 gig a second....you'll pay for that of course
http://www.fusionio.com/PressDetails.php?id=69
http://www.tomshardware.com/news/ioDrive-Duo-Worlds-Fastest-SSD,7254.html
 
I've heard that even if you have the storage bandwidth for RED 2k (~400mb/s) in Color.. because of the cpu/gpu work needed process red material and the fact that Color is not the most efficient software you can not grade RED material in Color in RT.

If anyone knows of a Color setup that can do this... tell!
 
So when using color with a setup that has a Kona Card (or other), the GPU still does the processing and the Kona card only output it or does it handle the processing for the grading?
 
Yes, the GPU does all the processing and the video I/O card (Kona, Blackmagic, etc.,) just sends out the video signal.


-A
 
So Apples really suck right now in that sense... cause the best card supported isn;t even in the lest of the faster cards right now. Correct? Isn't it the ATI Radeon HD 4870 with only 512MB... pathetic doesn't even begin to describe that decision from Apple.

What the heck can we do about it? No SLI right? So WTF... seriously... there has to be something right? Cause 2k grading in 10-bit ain't gonna happen without a new, perfectly optimized card, right?
 
Good thread.

I'd also add to keep in mind that SSD's decrease in write performance by up to ~15% (for good drives) after heavy use because they can't delete individual pages of information, only blocks. It's a little complicated, but explained well at Anandtech (long, but good editorial):

http://anandtech.com/storage/showdoc.aspx?i=3531

Depending on the types of files you're dealing with and their sizes, YMMV. A clean delete command (not just a format) is the only thing that can restore full speed. Improvements are coming soon however with the implementation of TRIM commands, hopefully with Windows 7 and maybe Apple OS updates. Even TRIM, however, won't eliminate SSD's fundamental write problem.

I suppose a good solution if you had the cash and needed 4k throughput would be to use an Intel X25 SSD as your boot / application drive, then get a pricier SSD RAID or the FusionIO / OCZ solutions for your temporary video workflow, and then offload the files to your HDD RAID medium-term storage. That way, you could clean delete your pricey SSD RAID for long-term performance. Just my $.02.

Also, it's not that the Radeon 4870 is a bad graphics card. Its chipset is the foundation for the Firepro V8700, which is a great workstation card. The optimized drivers are the key. And I'm just not sure if anyone is going to optimize something like color for a consumer GPU. Check out these benchmarks with the 4870, the V8700 and a FireGL card. Keep in mind that the 4870 and V8700 have virtually identical raw GPU power.

http://www.phoronix.com/scan.php?page=article&item=amd_firepro_v8700&num=7

Hope you find it useful.

Eric
 
Aren't those cards not supported? If 4870 512 gets spanked by $1,000+ VFX cards we can't use then I that makes my point.

If so, then knowing that Mac Pros are "pro", and knowing that FCS is so popular and respected now, and knowing that Apple talks about how awesome Color is and how intuitive it is and how blah blah blah... you can't help but think that someone deserves a kick in the privates over at Apple.

THEN AGAIN... I believe they knew that their contract for dual Nehalems was going to sell enough computers for a while, and they are saving some of the goods till later... goods like more compatibility options, with consumer Nvidias that will humiliate the 4870 and save you a bit of money. Hopefully much more on the soft side.
 
Mike N said:
http://www.reduser.net/forum/showthread.php?t=27474&page=2

in this post here the poster says that the RAID setup really doesn't change anything when debayering 4k in Redcine and another poster on THIS thread said that with a 4core i7 you could debayer 4k in almost realtime.

I really don't understand anything anymore...:cute:
Click to expand...

debayering cineform....not redcode....much faster

and at an input of 45 MB per second why would raid help debayering.....a single hard drive can feed that...it's CPU (or GPU) power that is needed, and that is where the bottleneck is.

hence why an i7 can debayer 4k cineform real time.

your other option is to debayer to a 4:4:4 format...which on playback needs little cpu power...but it's filesize has jumped at least 10 times in size...hence then the need for a raid setup for real time playback.

with cineform or redcode you can debayer on the fly when editing....which needs way more cpu power....but less hard drive bandwith
 
You must log in or register to reply here.
Back
Top