Welcome to our community

Be a part of something great, join today!

Register Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Komodo REDCODE RAW Explained

One issue that RedUsers will need to look at is sustained read speeds when working with files in the 500-900MB/s range. With the current DSMC2 "speed limit" of 300MB/s you don't need bomber storage and I/O to work with R3Ds. Full res, real time playback/editorial is another issue - but read speed is not a bottleneck unless you are on a single spindle hard disk. For the last few years I've been living on Samsung T3/T5 SSDs that can read/write at +/- 500MB/s for RED projects. Post is typically on RAIDs. There are several options in commodity systems that can handle those data rates - TB2/3, USB3-SS (with USAP), USB3.1, USB-C, eSATA - assuming decent chipsets and enough lanes.

Based on early notes, the new world of DCT Redcode moves the minimum read speed spec up to 800-900MB/s for real time playback. This is beyond the capability of most 4 or 5 drive mini-RAIDs (some can do it) and if you are using a RAID type other than 0, you might actually want an 8 drive RAID to avoid issues. M.2/NVME solutions should work a treat, but the cost per TB is still pretty steep. SSD based RAIDs are another option; cheaper than M.2, but still many times the cost of spinning rust.

In the short term I'll likely move to the T7 Samsungs that R/W around 980MB/s, 2TB is around $400USD currently. For archive there's always LTO and I'll likely use >5TB hard drives at roughly $20/TB for backups as well.

Cheers - #19
 
Many raw neophytes attempting the transition from traditional video cameras, tools, and workflows have a hard time grasping the extreme range of creative imaging possibilities that deep raw format cinema cameras offer compared to their video camera counterparts. The learning curve can be steep.
 
Blair S. Paulsen said:
Based on early notes, the new world of DCT Redcode moves the minimum read speed spec up to 800-900MB/s for real time playback.
Click to expand...

What is this calculation based on? Upscaling towards a future 8K-12K DCT-Red camera?

As far as we know know everything regarding DCT happens within the Komodo and therefore within Cfast 2.0 Spec, which is theoretically 6Gbit/s maxing out at 750MB/s, more practically around 600MB/s and even more realistically calculated somewhat below 500MB/s write speed on the actual card. So what is 800-900MB/s? Just curious.
 
My note was based on recording times to 1TB media per Phil's posts. Even if the chunkiest footy is "only" 600MB/s, it's still more than the T3/T5 SSDs can handle. Under 800MB/s will be OK on most 4/5 drive mini-RAIDS - which would be nice since I have a couple on hand.

IAC, the key point is making sure your setup can read from storage fast enough for real time playback. Moreover, having some headroom above the data rate needed is usually a good idea. YMMV.

Cheers - #19
 
I grudgingly acknowledge that the industry seems to be going in the direction of DCT based schemes for compressed RAW.
To me, Wavelet based compression has a much more organic quality to it's compression artifacts and naturally allowed for
higher compression ratios.

I always considered the ease of playback to be a secondary issue as 1) many workflows still transcode to a more post friendly codecs like ProRes, DNx, DPX and 2)RED has already made a lot of progress getting better playback performance of the existing wavelet based REDCODE.

REDCODE already does what it needs to do as an acquisition (digital negative) format and it does it very well.

KOMODO is small(er), has great power consumption, and is apparently very quiet which are all excellent improvements.
I understand that changing REDCODE to DCT is a trade off to achieve those qualities.

Given these accomplishments, Komodo will be great in many filming scenarios.

The one area that I think the new DCT based storage requirements can be an issue is documentary.
It pains me to think that a 2 hour sit down interview can potentially burn well over 1TB of storage.
And that is just for one camera.

This all compounds itself once storage and archive is in the mix.

It's scenarios like that where I hope RED considers refining the codec and perhaps offering alternative recording media (i.e CFAST adapters for SSD drives) to ease the data footprint.

Otherwise this camera has so many strengths.
As impressive as BMDs 12k Ursa announcement was, I'm more excited with Komodo mainly because I can see where this camera can tangibly enhance, simplify, and modernize things for me on set.

Brian Timmons
BRITIM/MEDIA
 
Remember how this DWT / DCT discussion started?
I guess Jason was a bit ahead of the announcement last June.
This Rufallo moment was so funny though!
yZIezHK.png
 
Last edited:
Brian Timmons said:
I grudgingly acknowledge that the industry seems to be going in the direction of DCT based compression.
Click to expand...

Not really.

The industry has been DCT for a long time.

ProRes is DCT based.

MiniDV is DCT based.

REDCODE in Wavelet form has been the outlier.

JB
 
Back in the good old/bad old days with 16GB CF cards on the RedOne, doc/nature/adventure projects had similar issues. The good news is that like old CF, the CFast2 cards are very small so you can carry a lot of them in minimal space. Renting a pack of them for a shoot via an internet service like LensRentals is cake. I'd wager you'll have way more headaches with batteries, chargers, airlines, etc than schlepping around a few more tiny media cards. Speaking of batteries, Komodo's "power diet" means longer run times and/or the option to use the smaller/lighter Canon BP type batteries.

To me, the power savings are worth the media footprint expansion. YMMV.

Cheers - #19
 
John Brawley said:
Not really.

The industry has been DCT for a long time.

ProRes is DCT based.

MiniDV is DCT based.

REDCODE in Wavelet form has been the outlier.

JB
Click to expand...

This was related toward compressed RAW formats. I adjusted my text to make that read clearer.
Yes very aware that compressed chroma subsampled RGB formats were mostly DCT.
Out of the few compressed RAW video formats around there seemed to be a greater use of wavelets at least early on (R3D, Cineform RAW).
My biggest issues with DCT is generally the higher storage requirements and the greater prevalence of macro blocking. Wavelet compression tends to get higher compression ratios and looks softer when the compression starts to fall apart and to my eyes that looks a bit more organic.

Brian Timmons
BRITIM/MEDIA
 
There are definitely benefits to entropy codecs like RedCode/JP2K. As Brian notes, DCT can create ugly macro block artifacts when pushed too far.

Like most engineering choices, there is going to be compromise. The key trade off in the DWT vs DCT realm is power/heat vs data footprint. The secondary one is storage requirements vs processing/decoding needs. With storage costs continuing to decline and commodity I/O going from USB2 to TB3/USB4 via TypeC I support RED's choice - even while acknowledging the downsides.

Cheers - #19
 
Brian Timmons said:
I grudgingly acknowledge that the industry seems to be going in the direction of DCT based schemes for compressed RAW.
To me, Wavelet based compression has a much more organic quality to it's compression artifacts and naturally allowed for
higher compression ratios.

I always considered the ease of playback to be a secondary issue as 1) many workflows still transcode to a more post friendly codecs like ProRes, DNx, DPX and 2)RED has already made a lot of progress getting better playback performance of the existing wavelet based REDCODE.

REDCODE already does what it needs to do as an acquisition (digital negative) format and it does it very well.

KOMODO is small(er), has great power consumption, and is apparently very quiet which are all excellent improvements.
I understand that changing REDCODE to DCT is a trade off to achieve those qualities.

Given these accomplishments, Komodo will be great in many filming scenarios.

The one area that I think the new DCT based storage requirements can be an issue is documentary.
It pains me to think that a 2 hour sit down interview can potentially burn well over 1TB of storage.
And that is just for one camera.

This all compounds itself once storage and archive is in the mix.

It's scenarios like that where I hope RED considers refining the codec and perhaps offering alternative recording media (i.e CFAST adapters for SSD drives) to ease the data footprint.

Otherwise this camera has so many strengths.
As impressive as BMDs 12k Ursa announcement was, I'm more excited with Komodo mainly because I can see where this camera can tangibly enhance, simplify, and modernize things for me on set.

Brian Timmons
BRITIM/MEDIA
Click to expand...

On the other hand, sitdown interviews is usually done in quite controlled enviorments so shooting raw in such situation is not always needed.
 
Björn Benckert said:
On the other hand, sitdown interviews is usually done in quite controlled enviorments so shooting raw in such situation is not always needed.
Click to expand...

RAW shooting is like a way of life.:laugh:
In seriousness, it's true, technically speaking not everything has to be filmed RAW but shooting RAW is a big preference.

I also imagine dropping the resolution to 5k or 4k could also help control data rates.

Adapting some of the higher capacity desktop drives might help too.
https://cinediskpro.com/cfast-to-ssd

Brian Timmons
BRITIM/MEDIA
 
Brian Timmons said:
RAW shooting is like a way of life.:laugh:
In seriousness, it's true, technically speaking not everything has to be filmed RAW but shooting RAW is a big preference.

I also imagine dropping the resolution to 5k or 4k could also help control data rates.

Adapting some of the higher capacity desktop drives might help too.
https://cinediskpro.com/cfast-to-ssd

Brian Timmons
BRITIM/MEDIA
Click to expand...

I Cant wait for more Cfst- SSD options.
 
Pro Res Raw

Pro Res Raw

And not to get too spicy, but yes, Apple has chosen a DCT scheme for their ProRes RAW
Isn't Pro Res Raw something Atomos came to Apple to get licensed? Like Apple didn't invent Apple Pro Res Raw. It was an abstract out.
 
You must log in or register to reply here.
Back
Top