Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Rocket or fast storage ?

M

Mike N

Well-known member
Joined
Nov 4, 2008
Messages
232
Reaction score
0
Points
0
Hello,

I want your opinion about which is best to buy, Red rocket or a fast RAID storage in a situation in which I don't have money for both.

Maybe a silly question, why would you need a Rocket if you can't grade (don't need to) in native 4k resolution, since as far as I know at the moment only Scratch can take advantave of both native R3D grading accelerated by Rocket.

Wouldn't I be better of buying a fast RAID storage which gives me 600-700mb/s + read write speeds ? Wouldn't this also allow me to view full 4K playback in Redcine-X or other programs which work natively with red files ?

Would using a Rocket and outputting DPXs or TIFFS be slower if I don't have a fast storage which can handle all that amount of data ?

Ultimaltely what makes this whole process so speed hungry regarding storage speed ? I thought the codec is around 42 MB/s for the highest setting. A decent SATA drive easily tops that ? Is that during the debayering process file sizes increase exponentially ?

Would using an alternative codec like Cineform also spare me from owning a Rocket ? As far as i know Cineform preservs the RAW data but keeps the low data rates.

Thanks
 
You will need fast storage to make full use of a RED Rocket, but you don't need 600-700MB/s. With refurbished Rocket cards available for $3K and relatively affordable RAID solutions out there, perhaps there is a compromise solution that would allow you to own both? What is your budget?

The RED Rocket accelerates the debayer process as well as provides hardware scaling and transform of your full resolution REDCODE images. So in that regard, it's much more than just for grading. It's really a power-house for transcoding and processing REDCODE.

You should also consider what application tool sets and which workflows you will most commonly use. Many will benefit from a RED Rocket, some will not as there is no support for the Rocket in some applications -- Adobe CS5, for example.

The REDCODE codec can be more than 42MB/s, it really tops out a bit over 50MB/s in some cases. But it seems that you will want a drive system capable of 100MB/s or more, SUSTAINED, to ensure reliable playback of R3D media. Typically, any two or more decent hard drives in a proper RAID-0 set or 3 or more drives in RAID-5 will handle this with ease. Many individual drives are up to the task these days, but a good host controller or RAID controller running that drive is also important.

Your speed requirements for storage will grow considerably, depending on your needs for transcoding or rendering your footage. In many peoples' workflows, their storage is often a bigger bottleneck than they realize. If you are reading your source data from the same volume that you are writing to, then you will need that 500MB/s or faster RAID when writing 2K, and especially 4K DPX files or other formats that take up a lot of space.

The Rocket is capable of a full resolution and quality debayer / demosaic of your RED footage at real-time or faster. But we find that the speed of transcodes is limited by a couple factors -- primarily, the speed of the storage, as we have been discussing. And also by whatever codec you are using to encode your footage. For example, the Apple ProRes codec is still bound mostly to a single CPU and is itself a real bottleneck.
 
Good question, not a fun decision because everything is a compromise. Since there was no Rocket card available when I purchased my system, I spent the money on a good RAID at the expense of not getting an LTO also. Plus, I wasn't convinced which LTO was the right choice, so I demurred. Now, there are better options and I am considering which route to take.

If it were me, I would still get a strong RAID first, then get the Rocket, if it has to be one versus the other.
 
You NEED fast storage for every major step in the workflow. You don't NEED a rocket. I did just fine for 18 months posting RED doing renders overnight without a rocket, but I could not have done much without fast storage in any part of the workflow, not well. You don't need a rocket to render, you need a rocket to render FAST. You don't need a rocket to view redcode, you need a rocket to view redcode at high res full framerate. Finally, a rocket without fast storage is problematic, the storage will limit the render speed even with a rocket. As Jeff notes you don't need crazy fast, just fast. But from where I sit, this is not even a debate.
 
To add to my above, since I didn't come right out and say it, I would also go for the storage before the Rocket. Can't fully use Rocket without it anyway. Concentrate on adequate storage for your needs, both capacity and performance, also make sure you have a good backup strategy in place. Then you can augment your workflow(s) with other tools like the Rocket, more software, etc..
 
My question:

Are you primarily a camera house or post house oriented person?

If you are camera oriented I'd say buy the rocket! Make clients pay for storage. If you make most of your money doing color correction or editing then I'd agree with everyone else and say focus on storage first. Which tool will help you make the money to get the other piece of the puzzle faster?
 
As long as your drive is not fragmented even a USB 3.5" drive will do for playback. Add another drive on another bus and that's about as fast as it will get for rendering. As said Red footage tops out at 40-something Megabytes/s so that and some headroom is all you need.

Now for archiving huge amounts of footage or Epic, speed will come in handy for sure.

(Yes, I have a Rocket and render footage all the time)
 
USB is painfully slow, not sure how anyone could recommend it for RED workflow. I guess if that's all you have, then that's it. But IMO, it's worthless for use where time is a factor, offloading, etc.. USB3 is gaining some ground, but frankly I would prefer it just go away. It's still no faster than eSATA overall and has much higher latency. USB2 tops out at about 45MB/s maximum throughput, which is not enough to guaranty real-time playback of some of the higher REDCODE modes we have now, let alone what's coming up with Epic. FW800 is twice as fast and most of us here still feel it's too slow for any performance or time-critical task. We offload 16GB RED CF media via eSATA readers as fast as the cards will physically allow us. USB2 is not up to the task, FW800 typically is a bottleneck for full CF card images. For doing verified copies, every second counts. Even when sitting at my desk on my own time, I don't want to wait a second longer than I have to.

As for transcode write speeds, depends what you are transcoding to. In addition to the read requirements, you need an additional sustained 250MB/s if you want to write 2K DPX in real-time. If you're transcoding to 2K ProRes, 50~75MB/s sustained writes will be sufficient on even the fastest Mac Pro right now because of the inefficiencies of the codec itself and you won't get all the way to real-time performance, even on the latest CPUs, due to these inefficiencies.

But what I'm really saying here is that USB is a bad way to go for storage. If clients are giving me storage, I tell them up front that if they're giving me 800GB of R3Ds to transcode and they hand it to me on an external USB drive, it will take about 5 hours just to read all the data off that drive, so don't expect same-day turn around on the job. Whereas if it's a decent 1TB eSATA drive they give me, I can offload all the data from it in about 90 minutes. And yes, we offload and log all such client jobs into our systems before going to work, having both online masters as well as work files. If the client hands you a 1TB USB2 HDD and you do the transcode straight from that, you're making a serious noob mistake.

There are lots of considerations here. We're not even sure what requirements the OP has or what he would like to do. Different people have different priorities, I know several who don't have an interest in the RED Rocket or faster storage. They don't mind their transcodes taking all night or even a couple days. To each their own....
 
Thanks for the answers guys

I'm basically trying to figure out what are the data outputs of the Red workflow and see if I can get away with it in most cases by owning a good RAID system and bypassing the Rocket...for now.
Or if owning a Rocket would make a fast storage somewhat redundant, since you can transcode your footage super fast to some low data rate codec and then your default storage will be able to handle the problem. But that, I think, would limit my choices only to compressed lossy codecs, right ?

After reading your answers I think my assumption was correct. As i see even if you own a Rocket there s not much you can use it outside a Redcine-x or Scratch for now.
I will probably use CS5 or Avid for cutting and AE beefed up with some plug-ins for color correction. After things get moving i'll probably want to move to more high end software.

So to sum this up. For playing back realtime at full res only the R3D files in a program like Redcine-x,Avid or CS5 i would need about 100MB/s sustained read speeds.
To transcode realtime to DPXs or TIFFs I would need over 200-300MB/s sustained ?

The CS5 and AE Cuda support only works for playback or also for transcoding and effects processing ?

Which RAID systems do you recommend. Can I find some decent internal solutions, like the controller included with the motherboard ? or should I go with separate PCI controllers or external solutions ?

I'm new to the storage stuff if you could recomend a few brands so I can do some research it would be great. A company that I liked is CalDigit ar they at the top of the game or somewhere in between ?

thanks
 
Sorry for the confusion, I certainly wouldn't recommend using USB 2 for storage and workflow. But to say you NEED fast storage in every part of the Red workflow is the other extreme. I regularly play full res debayered footage from a single drive.

What hasn't been mentioned properly I think is that without a Red Rocket full res transcodes will take forever. You can NOT play back full res in RedCine-X without a Rocket and even with a Rocket you can't in CS5. The word about Rocket support in Avid is not out.

So forget about 200-300MB/s for storage to write your DPX sequences, you will not need it without a Rocket unless you want to use it for playback.

You CAN actually grade in real-time with a Rocket, depending on what your definition of grading is. For a one-light it certainly works and you can see the results in real-time.

There's a lot more to it with a Rocket then you read here on the forums though. For instance your computer won't go to sleep anymore, it gets hot and it will make a lot of noise.
 
Mike N said:
So to sum this up. For playing back realtime at full res only the R3D files in a program like Redcine-x,Avid or CS5 i would need about 100MB/s sustained read speeds.
To transcode realtime to DPXs or TIFFs I would need over 200-300MB/s sustained ?
Click to expand...

To playback full res 4k or transcode realtime to DPX you need a Rocket. Period.

Mike N said:
The CS5 and AE Cuda support only works for playback or also for transcoding and effects processing ?
Click to expand...

Adobe is also actively working on Rocket support.

Mike N said:
Which RAID systems do you recommend. Can I find some decent internal solutions, like the controller included with the motherboard ? or should I go with separate PCI controllers or external solutions ?
Click to expand...

I do just fine with Hitachi 2TB drives in software RAID 0. Just 2 drives gets me ~200 MB/sec. Double that if I relocate my boot drive to the second optical bay and stripe 4 of them.
 
I have a rocket and I wouldn't trade it for the world. I would rather raid two internal drives on a Mac Pro than go without the rocket.

Makes life wonderful and removes all barriers.

Do you really need 300MB/s? I don't. As long as I can edit Pro Res at 1080p I'm happy.

Realtime R3Ds is worth it's weight in gold!
 
Mike N said:
The CS5 and AE Cuda support only works for playback or also for transcoding and effects processing ?
Click to expand...

CUDA is accelerating effects only, not transcoding. But the whole Mercury Engine is making better use of many cores with enough RAM and speeds up software as well, but not as much as the GPU effects.
 
Curran Giddens said:
To playback full res 4k or transcode realtime to DPX you need a Rocket. Period.


If only for playback why couldn't i view 4k in realtime with a faster storage since the original R3Ds are about 50-100MB/s as Jeff Kilgroe pointed out in one of his earlier posts.

Is it because besides the fast storage the processing power required is still not available at the moment ?


@jimhare:

Yes I agree with you but what if after the editing is over you would want to relink with the original footage and grade them as DPX or TIFF files ? Rocket would not help you then.

How much would one gain in terms of transcoding speed in an application like REdcine-X with faster storage vs only a Rocket. 30% for the storage only and 80% with Rocket with normal storage. If that's the case I might reconsider my buying order.

Thanks.
Click to expand...
 
You can't play back realtime 4k without a Rocket because it would require a lot of processing power, much more than 8 or 12 cores can give you.

When your hard disks can keep up with reading/writing Red footage then there's nothing to be gained from faster storage. So it all depends on what you have already.
 
You are not going to be happy until you have them both, bottom line.

You're beginning to split hairs here because the answer is you want both and you probably need both. You're not going to be happy with just one or the other, but if push comes to shove, get a good RAID first, then the Rocket card. You are fooling yourself when you think you're not going to need good, fast, voluminous storage capacity. A Rocket card makes transcoding speed ridiculously fast compared to a system without it, a magnitude of 20x faster. But, if you don't have the storage, what good will it do? It's not imperative to be able to playback in 4K on your computer, do you have a 4K monitor? What is your final output going to be? For 98% of folks it's no greater than 1080p at this point in time.

Jim makes a good point about editing in ProRes at 1080p. It's what I do too. But where are all of those files going to reside, even if they're not the original R3D files? And where are you going to have the original files in case you want to go back and render those DPX or TIFFS you mentioned? Rocket card ain't helping that. My earlier point still stands. When the camera first came out, we all had to render without the benefit of the Rocket. Now that it's out, for those of us who have one, it's wonderful and changes the way we work. But there are MANY who still operate without it, just working around in various workflows. It's not pretty, but can be done.
 
OK guys you made your point. RAID now, Rocket as soon as possible.

Thanks.
 
It really all depends on workflow. If you're working with native R3D files and transcoding to 1080p ProRes, get the Rocket first -- a RAID will literally get you nothing except for faster file copies (and only then when copying to/from another RAID -- which you're often not), while the Rocket will get you vastly faster transcoding as well as real-time playback in RedCine-X.

On the other hand, if real-time playback of uncompressed HD and/or DPX is a critical part of your workflow, a RAID is non-negitiable while a Rocket will merely be a convenience (but a major convenience).

Finally, if what you're looking for is fast transcoding to uncompressed HD or DPX, then you really need both the RAID and the Rocket -- there's not really a way around that. Although if you had to choose, Rocket + an internal softare RAID 0 on a couple of drives (which costs, obviously, next to nothing) would give you much faster transcodes than a high-end RAID but no Rocket.
 
An earlier poster really hit one of our assumptions - post vs camera work. I assumed in my earlier post that one is doing a lot more with their setup than just dealing with one lights, transcodes and renders up front.

I do full editing and post on about half the projects I work on. And for that, fast storage is critical, and the rocket is not. As Jeff said earlier, the big question is - how much post are you doing?
 
You must log in or register to reply here.
Back
Top