Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

8Core Nehalem / Red Rushes madness - absolute madness!!! HELP

J

jacknusa

Well-known member
Joined
Sep 4, 2007
Messages
75
Reaction score
0
Points
0
So, after reading the glowing reviews of the 8core 2.26 I went ahead and bought one of those to replace the now long in the tooth 2xdualcore 2.66 Mac Pro that's been our main Red Rushes render machine.

With the double number of cores, and with the faster Nehalem architecture I was expecting a rough 3x (give or take) speed increase running Red Rushes.

Result: the new 8core is almost 3x SLOWER than the old 4core!

Yes I checked, I didn't mix the machines up.

A full debayer/use Clip RSX/scale to 1080 of 100frames of THE EXACT SAME 3K clip takes 1m57s on the old 4core and 5m12s on the new 8core!!!

WHAT's going on here?

So I start trouble shooting:

A) I take the slightly different disc I/O setup between the 2 computers out of the equation by setting the test up so that I'm reading and writing to a single internal drive - NO CHANGE, 1m57s/100frames on the old 4core and 5m12s/100 frames on the new 8core.

B) On the new 8core I try different settings for the number of cores in Red Rushes. Auto, 8, 4, 2 cores = NO CHANGE, still 5m12s/100frames - when I set it to 1 core the time increases to 6m32s/100frames.

C) Thinking there might be something up with Red Rushes I run the test out of Red Alert - NO CHANGE, same results

D) Thinking there might be something wrong with the RSX file (it was generated on the old computer) I throw it out and generate a new one on the 8core - NO CHANGE.

E) Still suspecting the RSX file of something, I set up Red Rushes to ignore the RSX file and just use the camera look - BIG CHANGE!!! The old 4core takes 1m18s/100frames, the new 8core takes 1m12s/100frames - now the new 8core is faster then the old 4core but only by a slim margin, but the drop from 5m12s/100 frames with RSX to 1m12s/100frames without it makes me feel something very whacky is going on...

F) I take the scaling component out of the process, result: new 8core 5m08s/100frames, old 4core 1m57s/100frames - basically everything stays the same...

G) I change the debayer setting from FULL to HALF(standard), result: BIG CHANGE!!! - new 8core 0m7s/100frames, old 4core 0m18s/100frames..., the old 4core runs roughly 6x faster while the new 8core is roughly 40x faster (compared to a full deabyer) - now the new 8core is outperforming the old 4core by roughly 2,5x

Summary of the madness:

3K, full debayer, use clip RSX, scale to 1080:
Old 4core roughly 3x faster than new 8core Nahalem.

3K, full debayer, ignore clip RSX (camera look), scale to 1080:
Old 4core and new 8core is roughly equal in performance.

3K, Half(standard) debayer, use clip RSX, scale to 1080:
New 8core is roughly 2,5x faster than old 4core.

So to sum it up: WHAT's GOING ON

Specs for the two computers:

Old 4core
---------
OSX version: 10.5.5
QT version: 7.5.5
Red Rushes version: 3.60 (1)
RAM: 5GB
Processors: 2x2.66 Ghz Dual core


New 8core
---------
OSX version: 10.5.6
QT version: 7.6
Red Rushes version: 3.60 (1)
RAM: 6GB
Processors: 2x2.26 Ghz Quad core


Can anyone shed some light on this...
 
I can't tell you for sure, but QT 7.6 and OSX 10.5.6 are toxic for a lot of video applications. Since I've stuck on 7.5.5 and 10.5.5 (because of the warnings about the newer releases) I can't say if that's your problem. But it's a place to look. Read the setup guidlines for redrushes and see if they spec QT version or OS version.
 
I have no idea for test A).
For test E) to me is a predictable result since you are using a single drive setup. if you look at test E) you see that reducing the data output performances improve.
to me in that case the HDDs are the bootleneck.

Basically you can take advantage of the performances of your new Mac you need to be able to handle the data coming out from the processor.

If your HDD can receive 50MB/s you are limited to this output.

Many here in the RED Workflow forum buy top of the line computers but use internal drive or small external FW800/400 for read-write.
HDD arrays are one of the key for performance.
 
michele, thank's for the reply but I think you're missing the point. the point being that disk i/o being equal, the 8core nehalem is almost 3 times SLOWER when rendering 3k full debayer WITH a RSX, but 2.5 times FASTER when rendering half res debayer with a RSX - and to add to the mystery, ignoring the RSX makes the two computers perform on par.

shed some light... please?

j.
 
jacknusa said:
michele, thank's for the reply but I think you're missing the point. the point being that disk i/o being equal, the 8core nehalem is almost 3 times SLOWER when rendering 3k full debayer WITH a RSX, but 2.5 times FASTER when rendering half res debayer with a RSX - and to add to the mystery, ignoring the RSX makes the two computers perform on par.

shed some light... please?

j.
Click to expand...

You may have some things like denoise and olpf compensation turned on in the RSX.
 
Its academic to use full debayer - way more information than you need to go to 1080.
 
I would know the size of the file not the MB/s

100MB , 200MB ?

can you tell me also the clip Length?
 
guys the issue is not what settings he should use.

the issue is why the new mac is so slow.

you only have 6gb ram. you may run out of ram - your 8core eats around 2gb per core... maybe thats it?

best,

Sidney
 
Sidney said:
guys the issue is not what settings he should use.

the issue is why the new mac is so slow.

you only have 6gb ram. you may run out of ram - your 8core eats around 2gb per core... maybe thats it?

best,

Sidney
Click to expand...

ok, I'll try some more RAM - how much RAM do you other 8core Nahelem users have installed?
 
joe hedge said:
You don't need full debayer, try the half res (standard) setting

Also this little trick sped things up quite a bit for me...

http://provideocoalition.com/index.php/awilt/story/play_ping_pong_for_faster_file_flipping/
Click to expand...

Thanks for the tip about separate read and write disks, but it's not the issue here, and all our computers are always set up this way - when I first set the new 8core up, it's disk I/O was set up with a single SATA read drive sustaining 100MB/sec over Infiniband and a 4disk Raid5 write volume sustaining well over 200MB/sec over Infiniband (separate bus).

with this set up red, rushes took 3sec per frame for the 3K full debayer to 1080. Disk I/O is NOT THE ISSUE!

The reason I did all further testing reading and writing to a single internal drive was to a) make the I/O setup between new 8core and old 4core identical, and b) I wanted to pull the host card for the infiniband connections just in case the card was causing problems. In both cases the new 8core is 3x SLOWER when processing a full debayer. At over 5minutes/100frames of 3K to 1080 ProRes, trust me, disk I/O is not the issue here!

j.
 
jacknusa said:
Thanks for the tip about separate read and write disks, but it's not the issue here, and all our computers are always set up this way - when I first set the new 8core up, it's disk I/O was set up with a single SATA read drive sustaining 100MB/sec over Infiniband and a 4disk Raid5 write volume sustaining well over 200MB/sec over Infiniband (separate bus).

with this set up red, rushes took 3sec per frame for the 3K full debayer to 1080. Disk I/O is NOT THE ISSUE!

The reason I did all further testing reading and writing to a single internal drive was to a) make the I/O setup between new 8core and old 4core identical, and b) I wanted to pull the host card for the infiniband connections just in case the card was causing problems. In both cases the new 8core is 3x SLOWER when processing a full debayer. At over 5minutes/100frames of 3K to 1080 ProRes, trust me, disk I/O is not the issue here!

j.
Click to expand...

What do you have set for the num clips and cores per clip?
 
Deanan said:
What do you have set for the num clips and cores per clip?
Click to expand...

number of clips = 1 in all cases
cores per clip: see original post - auto, 8, 4, 2 all yield the same slow results, a setting of 1 slows it down another 20%.

J.
 
have you performed an hardware test?

You can do it with the Apple installation disk
maybe you have a bad ram problem!
 
jacknusa said:
number of clips = 1 in all cases
cores per clip: see original post - auto, 8, 4, 2 all yield the same slow results, a setting of 1 slows it down another 20%.

J.
Click to expand...

Running multiple clips in parallel helps maximize throughput. Different options (debayer resolution, denoise, etc) all affect the speeds.
 
You must log in or register to reply here.
Back
Top