Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

RedCineX not using all cores for rendering...?!

S

sbrookens

Well-known member
Joined
Oct 24, 2007
Messages
55
Reaction score
0
Points
0
Hello All,

Its been awhile since I have had a client ask for prores renders of red files but it came up yesterday. The shoot is pretty extensive and they need the prores files to be ready to post as soon as they finish production on Wednesday night/thursday morning. I told them that I would be able to keep up with rendering the files without a problem- I would just take the HD's back to my office at night and render them out on my 8-core mac. I was planning on using Red Rushes to do this because in the past I could always select the program to use all 8 cores - but in my tests with rushes this time around all the footage turned green. I called Red and they informed me that red rushes doesn't work with the MX sensor!!! They told me to use RedCineX instead. I'm not a fan of RX but they assured me that RX would automatically use all the processors without having to reset anything. So I did what they said - I set everything up to run over night and now I am back in the office in the morning and the files from yesterday are not finished!! To make matters worse I looked at the CPU usage and its running at 40%... Can anyone help me with this? Is there a way for me to set RedCineX to use ALL cores or am I just screwed?

Thanks in advance!!!
 
This is assuming you don't own a Red Rocket:

What debayer setting were your exports? The settings that have done well by me in the past are: ProResHQ, 720/1080p, 1/4 debayer, and make sure to set your frame rate correctly. That usually gets about 80-90% of the cores going. Yes, it's only 1/4 debayer, but it's just an offline and it serves the purpose.
 
For the above I was using RedCineX 262 - ProRes 422 (NOT HQ) 1080p at 23.98 fps at 1/2 debayer (they want to use these as their online files so I upped the debayer slightly). I noticed that in the new RedCineX 348 there is an option to use "RedLine" and have more cores- but when I tried this it took a bin of 7 clips and turned it into 340+ clips in the export window (it just repeated the same clips over and over again). Not to mention the files didn't render correctly - it said one clip was finished in like 2 minutes and the take was over 3 minutes long- there was no way it was processing faster than real-time. So I checked and the file was corrupted. That being said I just stuck with the released RCX instead of the 348 beta version. But still- is there no way to use all the cores in the current released version?
 
If the client wants real-time ProRes 1/2 debayer as their online files, they should be doing one of the following:
1 - renting you a Red Rocket
2 - paying you extra for your over-night rendering time
3 - doing it themselves.
Just be up front with the client about what they want versus what is possible and try to work out a solution. Are you transcoding during the day while you shoot? That might help some.
This is just IMHO. Good luck.
 
A RedRocket seems over-kill in this situation as its only about 2 hours of footage that needs to be rendered from each day of shooting. I feel like the over-night 12 hour period should be more than enough time for an 8-core mac to render out the prores files, assuming it is using all of its cores... If RedRushes worked I could just set it to use all 8 cores, but with the current released version of RCX it doesn't look like that is a possibility.

I am transcoding during the day on my laptop and hopefully tonight I can get RCX 348 working on my 8-core machine.

Aaron - thanks for your replies - are you from West Michigan?
 
From mid-Michigan, but work a lot in west Michigan. Have we met?

I don't think that bumping up to the newest RCX is going to help--(but could be wrong, I'm not an expert on that). It's not going to hurt either. In my experience, transcoding 1/2 debayer without a RR is just too SLOW.

I just got my Rocket, and am all geeked up about it using on my next gig which starts in two weeks. Before, my method was to create 1/4 debayer ProRes, and from there, create DVD dailies. Quite a time-consuming process. I would never finish the transcodes for the day completely--I'd have to come back the next morning and finish them up, hopefully before any new footage came in. When that was done, I'd pack up the daily drive with reports and get it shipped to post ASAP. I'd have the dailies finished by lunch. But with the Rocket, I'll be able to get all downloading and transcoding done and maybe work an extra 15-20 minutes after wrap.

I think in your situation, for real-time 1/2 debayer ProRes that needs to go out daily, the Rocket is not overkill. With Redrushes, were you getting all 8 cores maxed out doing a 1/2 debayer transcode?
 
sounds sweet! good luck with the upcoming feature and enjoy the rocket card! I work in Grand Rapids with Gorilla Pictures. Peter Johnson works with us on projects pretty often and he mentioned you the other day- talking about how you were doing a feature in town. I think he worked with you on Gun or something like that. Either way, good luck and thanks for your insight.

Best!

Scott
 
Clipfinder is fabtastic for these kind of renders.

Just set cores per render and off you go.

Used to get more than realtime offlines with CF before the rocket on an 8x MacPro.
 
clipfinder is a great tool- I use the old beta version all the time and love it for preview / playback on the fly. but I haven't upgraded to the newest version so I don't have the rendering feature. I suppose I should, but I'm just hoping that Red will put something together that works well so I don't have to cough up the $100 for CF
 
Is it processing with 40% on one core and nothing on the others, or is it 40% across all of them? System performance is limited by specific bottlenecks. Open the bottlenecks to solve performance. Here are the general principles, then how it is applicable to you. Consider these while transcoding or during any performance problem:

Processor - 100% on one cpu and 0% on the others means the application is not multithreaded and must be fixed by the programmer, or changing settings to enable the option. 100% across all cpus means the application is bound by CPU and the only way to up this performance bottleneck is to increase CPU speed or offload it (ie moving video encoding or physics calculations to dedicated processors).

Memory - If your memory is full and swap space is being used you can alleviate this by adding more. If the system uses swap space it will kill your performance.

Network - Not applicable here, but in a network one would evaluate bandwidth used versus capacity and take steps to increase throughput and response time.

Disk - In general, most system performance problems are sourced here. Only some OS monitoring utilities will tell you how much throughput is currently being sent through the disks. You should also run a different disk benchmarking utility that will show you the sustained read/writes that the drive can do so you have something to compare with.

Windows Vista and 7 include a resource monitor that shows current disk throughput (accessible from the task manager). On mac there is an activity monitor that should show disk activity (Not the same as disk usage). If that doesn't work you can drop to a command prompt and type 'iostat -K 3'. Press 'ctrl+c' to quit. This command will show you the current throughput of the drives in KB/s.

Basically if the memory and network are not an issue and the CPU is showing multi-threaded activity, but it is low, and the disk throughput is approaching the maximum sustained values, then you know the application is bound by (ie waiting for) the disk and by increasing the speed there you'll allow the system to take advantage of the unused CPU power.

How does this help you?
- Evaluate if Redcinex is actually using multiple threads or only one. If single figure out how to get into multithreaded mode as you are. But whether it is 40% on multiple or 40% on one, there is still another bottleneck so:
- Evaluate free memory while running to see if that is a bottleneck.
- Benchmark your disks.
- Then review their throughput while encoding and see if they are slowing down the processors. If so see about transcoding between separate spindles to increase throughput.

Cory
 
Cory,

Thanks for your post- it was most helpful. I will run the tests you suggest and see where my bottleneck is at. It very well might be the disk throughput- but I will also test to see if RCX is running 40% on one core or on all of them. Thanks again and I will post updates once I run my tests.

Best,

Scott
 
You might also want to retest with the use Redline option in the new 351 build just released. Hopefully it will have sucessfully fixed all the bugs with that feature.
Just make sure you are not using per frame mode otherwise it generates a movie file for ever frame in each clip!
 
RED Rocket, Only way to fly...

RED Rocket, Only way to fly...

Red Rocket, do it you'll never be sorry, lightning fast and the job will be done in Rocket time...A rocket card should be sold with each RED camera. Problem solved....

Or just rent one for the job...once you try it you'll never go back.
 
Last edited:
It's sorta depending on your needs.

For offlines, you really don't need full debayer, and if you don't need full debayer, you can do more than realtime with for example clipfinder.


If you want to make combo offline/online prores files, (and not online from RAW) you want full debayer 4444 prores files, but those will take up a lot of space.

You just have to decide what you need.

But a rocket is real nice.
All I am sying is that for 100$, you can pump out offline files at more than realtime on an octo-mac... -:)
 
You must log in or register to reply here.
Back
Top