Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Mac Pro 8 core 2.93 Transcode times in Compressor/Qmaster

I, too, now have a new Nehalem, and I'm currently compiling an extensive benchmark on transcode times, utilizing:

Red Rushes
FCP L&T
Compressor
Episode Family

Thus far, I have been unable to get Compressor to properly utilize the Quick Cluster I've created (to further utilize the horsepower of the machine). Has anyone been able to remedy the ability for Compressor to utilize a virtual cluster on the machine?

( http://www.kenstone.net/fcp_homepage/compressor_multi_cores_stitzer.html )

Thanks!
 
muffinny said:
I still don't get how AE doesn't have a better system of administrating the cores and use them all for rendering. Maybe I am missing something here. This is my AE setting.
Click to expand...

The Problem with AE is that it is still a 32bit application. I had the same problem as you - AE started to render fast and after a few frames it began to crawl - not utilizing the cores anymore.
I found out when observing the IOs that the threads were taking more and more memory until there was only a few MB left for the OS and soon everything started to crawl.

I have now restricted AE to use 3.9 GB in total. As soon I set it higher (even 4GB) it crawls.
In the multiporcessing tab set minimum to 0,5 GB.

Please try those settings and tell me if it improved render speed for you.
 
Robert Niessner said:
I have now restricted AE to use 3.9 GB in total. As soon I set it higher (even 4GB) it crawls.
In the multiporcessing tab set minimum to 0,5 GB.

Please try those settings and tell me if it improved render speed for you.
Click to expand...

Hi Robert,

I have bought 16 GB RAM and it runs much better. I am able to use multiprocessing, and I can see in monitor activity much better performance of the 16 cores (a lot of blue in the bars... I love the blue color :-). I tend to leave multiprocessing in the middle between Longer Ram and Faster rendering. (2.09 GR per core, Background memory 12,54GB 6 CPU,I leave 1 cpu for other applications with 1,75 Ram left for them) this has worked pretty good for me.

I also made some tests, with putting up all the RAM for Faster Render. What it happens then is that it seems that AE hangs, and I see all the cores working like crazy, but if you leave it for 4 minutes then starts rendering pretty quickly using I think 8 cores or so. So for really longer renders, I think I am going to use this method and then for normal renders just leave in the middle, as in that render mode starts rendering faster.

Also there is a great script that I found and allows you to Render in the background, so you can keep working on different comps or projects while rendering in the background. This is great, because if AE crashes, your render continues, and also as you start rendering few of them, since I use the background render without multiprocessing, then, each render uses one Core, so I have seen that rendering 3 clips at the same time, it doesn't affect the speed on each other, so technically if you need to render few clips and keep working, I think with this method you can get things really speed up.

But I still wish AE had a better and more efficient way of rendering.
 
RED Transcoding Benchmarks

RED Transcoding Benchmarks

I've been doing exhaustive testing on RED Transcode times with the new MacPros, and would like to share my results...in hopes that that you all may have some speed tweaks.

MacPro, 2 x 2.93 Quad Core,6GB RAM
All Media Local
10.5.6, QT 7.6
RED Clip, Build 17
Utilizing H Quality and ProRes HQ encode preset

I ran 11 clips of varying length (from :03 seconds to 5 minutes). I ran each test twice, and alone, i.e. no other apps running, and one only encode at a time.

Below is ONE test which is "par for the course" as far as my testing, and the ratios are below.

:53 second clip:

RedRushes (3.60)---- 1:40
FCP (6.05), Utilizing L&T ---- 3:05
Compressor (3.05) * ----2:59
Episode Pro Desktop (5.1.2)** ---2:53
Episode Engine (Split and Stitch enabled)(5.1) **Locally ---1.41
Episode Engine (5.1) **Locally ---3.11

Ratios in order of speed ***:

RedRushes: 1.8 - 1.9:1
Episode Engine (Split and stitch):1.8-1.9:1
Episode Desktop: 3.2-3.3:1
Compressor: 3.3-3.4:1
FCP L&T: 3.5-3.6:1
Episode Engine: 3.6-3.7:1

*Compressor preset ProRes 422 HQ Progressive. Unable to setup a virtual cluster as Compressor clusters seem to have issues with RED.
**No preset existed for ProRes, created a custom one with Video encoder set to ProRes 422 HQ. No other tweaks made.
*** i.e. 3:1...3 minutes transcode time for 1 minute of red footage.

Notes: All encodes utilized "all 16" processors, and all hovered around 30% usage on all processors, except RedRushes, which was over 45% usually, and Episode Engine with Split and Stitch enabled.

I know I was not utilizing Episode Engine the way it was designed (i.e. multiple machines) but this was a decent baseline.

I do not think RedRushes or Episode use QT as the backend (or do they? I thought REDLINE was the backend for RR, but is RR using QT?).

If anyone has ANY suggestions for improving single machine performance, or needs more detailed specs, please let me know.

~Michael
 
I'm not very familiar with Episode Engine. Is it a cluster renderer like Qmaster or a transcoder like Compressor? In my test I was using compressor 3.0.5 with Qmaster (Compressor Services) set to use 16 instances. The I selected the 4k proxy and used the default Apple Pro Res Preset in Compressor. Only one workstation was utilized in the cluster. The article that your linking to states that when the instances in Qmaster were increased... so was the performance (core saturation). How many instances do you have Qmaster set to?
 
Andrae Palmer said:
I'm not very familiar with Episode Engine. Is it a cluster renderer like Qmaster or a transcoder like Compressor?
Click to expand...

Telestreams Episode Desktop product (formely POPwire, many years ago) is an encoder, like Sorenson Squeeze, however with a much more robust feature set and efficient proc usage.

Engine is an enterprise product that is "similiar" to Qmaster in that it can distribute over multiple machines / network.

I'm benching them all, and working on the QMaster issue with RED (which you seemed to have remedied ;-) )

~Michael
 
Andrae Palmer said:
I'm not very familiar with Episode Engine. Is it a cluster renderer like Qmaster or a transcoder like Compressor? In my test I was using compressor 3.0.5 with Qmaster (Compressor Services) set to use 16 instances. The I selected the 4k proxy and used the default Apple Pro Res Preset in Compressor. Only one workstation was utilized in the cluster. The article that your linking to states that when the instances in Qmaster were increased... so was the performance (core saturation). How many instances do you have Qmaster set to?
Click to expand...

No matter what tweaks I make in QMaster or QAdministrator, whenevr I send a batch to the virtual cluster, the encode fails immediately "Can't transcode to Quicktime, Source has no video or audio"

This is using any quality (_H, _F, etc). Of course, the raw R3d file cannot be read.

Just how in the heck did you get compressor to see the file and what tweaks did you do to your QMaster and Qadministration settings?

EDIT: More testing shows that ful self contained files and QTs exported to Compressor out of FCP can take advantage of a Quick Cluster and hit all the processors no problem. It's dragging the RED generated Proxies into Compressor and using a cluster that seems to yield the problem.

Maybe something got changed in the camera build with the way the QT refs are written, and thus Compressor can handle it better - and you're using new footage to test? I believe the footage I'm using is Build 16 or 17.
 
michael_lucid said:
Just how in the heck did you get compressor to see the file and what tweaks did you do to your QMaster and Qadministration settings?
Click to expand...

I didn't do anything special at all... just dragged and dropped into Compressor. I have RED FINAL CUT STUDIO 2 INSTALLER & RED QUICKTIME CODEC (BETA). This worked with Build 16 & 17 footage. I don't have Build 18 installed.
 
Andrae Palmer said:
I didn't do anything special at all... just dragged and dropped into Compressor. I have RED FINAL CUT STUDIO 2 INSTALLER & RED QUICKTIME CODEC (BETA). This worked with Build 16 & 17 footage. I don't have Build 18 installed.
Click to expand...

This is puzzing! I, too, have all of those software packages installed, everything revved to current (FCP 6.05. 10.5.6, 7.6, etc) and Compressor still fails when I send a RED proxy thru compresssor....and via the power of google (and even searching this forum) it seems to be a common problem.

Do you mind sharing your QMaster and QAdministrator settings?

Thanks Andrae!

~Michael
 
Now getting 13fps playback at 1/2 res high in RedCine with Mac OS X 10.5.7. This is 4fps higher than what I was getting with 10.5.6. With my older 8 core 3GHz Mac Pro & 16GB ram/RAID was getting 16fps at 1/2 res high. Hopefully with further upgrades/updates will be able to get back to that.
 
You must log in or register to reply here.
Back
Top