Redcode compression vs cores

[Dylan Evans]

Hi,

So like many others on here (judging by the front page of this subforum) I am looking at a new build soon.

The big question is the cores vs clock speed dilemma. For encoding the answer seems simple - even well threaded codecs like ProRes and Dnx dont scale well beyond 10-12 or so cores (source)

However for compressed RAW the answer seems a little more complex. As I understand it Red compression is equivalent to your average compression - as such it should scale relatively well across cores when decoding Red footage.

So theoretically if you had a CPU running 12 cores at 4Ghz vs a CPU running 18 cores at 4Ghz the 18 core should show a marked improvement - this however does not seem to be the case in most tests I have seen.

This article from Pudget System in fact shows all the Intel CPU's above the 7900X performing worse than the 7900X itself, can this purely be explained by clock speed?: https://www.pugetsystems.com/labs/a...-Performance-Core-i9-7940X-7960X-7980XE-1034/ (I have queried this in the comments)

Ultimately cost aside - what would you consider the best CPU on the market today (including the new i9's and including potential overclocks) that balances the ability to process all types of professional cinema camera footage?

 

[Jay Smith]

I don't think it necessarily shows that the newer CPUs are worse than the 7900x. Premiere and Red footage in particular are picky about the balance of core count and clock speed. Having 18 cores is great but each core is running at a slower 2.6ghz, so its a bit tough for it to quickly decode what it needs to, despite having more cores. The Turbo Boost won't always be able to help boost the chips speeds to their full ~4.2ghz so the base clock speed is still a factor.

Playback with the 14 core 7940x is really promising, even pushing a full 8k Prores 4444 in real time and 6k red playback with no overclock (strangely only with a Lumetri filter applied, which must push slightly more info to the GPU).

 

[Michael Lindsay]

I'd too would love some hard broad data on this.. we have a dual 10core (20 real cores) z840 that when decoding in REdcine really works all the cores hard...so I feel cores up to 20 are great. I also want to buy/build a new machine..

The machine I want needs to replace a RRX laptop setup I use for creating editorial files BUT also scream doing some resolve conforms..

Basically what is the minimum spec machine that can take a 6k r3d turn it into a DNx editorial file in RQT or faster.. Full debayer?. If I can get a firm answer to this I'd be delighted..

I am going to test some different setups in a few weeks time so if I learn anything pertinent I will report back..

 

[Curtis boggs]

i'm interested in this as well.

Looking to build a new machine

 

[Misha Engel]

Lots of people on this forum use one of the most innovative camera systems available "RED".

What wonders me often is that many are extremely conservative when it comes to post production.

These days many new kids on the block have great innovations but are often ignored for post production in favor of old-school slow post production soft and hardware.


Some examples:

ProRes originates from 2007(that is really old in the IT world) and is made for Apple. Apple made a lot of hurdles for the users not working on Apple computers to use this codec, still lots of people on this forum think ProRes is the only intermediate codec in the world, lots here even use it as the standard delivery codec.
There are alternatives these days, DNxHR, CineForm and new kid on the block, Daniel2 https://www.daniel2.com/.

An other thing is that people using Mac's for years wonder if they can learn to work on windows or Linux. Well, you learned all the different versions of Mac OS.
You learned how to switch from Final Cut Studio to Final Cut Pro and maybe even Adobe software and Davinci Resolve.

Adobe is an other example. Adobe is not the most stable platform but many see it a the defacto standard. It is badly programmed for multi-thread and it is often slow.
There are alternatives that are cheaper and often better support multi-thread and multi GPU's (Davinci Resolve, Vegas,...).

For post production workstations it's often the same story. Which Intel processor do I need for this or that?
There are alternatives these days (AMD), that give you a lot more horse power for the money.

GPU's same story, which NVidia GPU do I need.
There are alternatives (AMD VEGA FE OpenGL 10 bit color for $ 1.000 vs. $ 1.800 Quadro P5000) which are faster and cheaper.
More and more software supports multi-GPU these days, it might be wise to pick a platform that can handle a lot of GPU's and maybe super fast SSD's to edit RAW in real-time(intstead of buying a terabytes of RAM).

Long story short, start (write down) with what you want/need, then pick the soft/hardware. Same as most of you did when you purchased your $ 50.000+ camera.

It also helps a lot when you read contributions from Phil, Jeff, Christoffer, and many others (it helped me a lot).
This might save you a lot of money and frustration.

 

[Dylan Evans]

If I could singlehandedly educate the whole industry on the limits of ProRes and benefits of other intermediates I would, but you know the TV/Film industry... change is difficult. I can advise post and people who employ me but thats about it!

Anyway - I am trying to establish the first part of your suggestion to "start (write down) with what you want/need, then pick the soft/hardware" - I can find no concrete information on how Redcode compression relates to core use and clock speed and where the money should go.

Does unpacking Redcode scale as easily as decompressing a 7zip archive? If so this would be my logic:
I need lots of cores for unpacking redcode
I also need high clock speed for encoding.
So my best choice is likely something like the i9 7980xe, I would overclock the first 10 cores as much as possible for speed in encoding and the extra 8 cores would lend a hand when unpacking Redcode.

The problem is the above is based on an assumption of perfect scalability across processor cores when unpacking redcode (like you would get during 7zip compression/decompression) - if this is not the case then my assumption is incorrect and I am better off getting a 12-14 core processor (AMD or Intel, the choice between them lies elsewhere... cost/compatibility etc)

So my dilemma continues. A white paper on how Redcode interacts with hardware would be a dream .

 

[Misha Engel]

I don't have a white paper, maybe Jeff Kilgroe or Phil Holland.

The sweet spot looks to be the AMD EPYC 7401P (24c/48t $1.100) + 192 (or 256) GB DDR4-2666 MHz ECC ram + 3 GPU (GTX1080ti or VEGA 56/64) + whatever you want in the rest of the PCIe slots.

 

[Michael Lindsay]

So my dilemma continues. A white paper on how Redcode interacts with hardware would be a dream .

+1

but I imagine the issue is that with the redcode SDK in constant dev it would be out of date soon as it was written..

So maybe some soft guidelines based off current dev direction would be the best compromise.

Basically as of today my understanding is NO wavelets are decoded on GPU and with simple transcodes from r3d to MXF/QT wavelets are the hard part.... therefore what we need to know is there a limit to cores and on ballnece how important is clock speed. From my tests 1-2 years ago cores seemed more important than speed BUT recent testing suggested the opposite?

I want to replace a RRX I have with a workstation to create editorial files and would love to nail a spec..