Welcome to our community

Be a part of something great, join today!

Register Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

redline to Qmater?

No but you can tell qmaster to use REDline as that's how it really works.
 
Can you provide any details on how to do that Deanan? Some documentation would be very helpful but I'd even settled for a post with some instructions.

Thanks
Dave
 
You could cheat and in RA's Render window select "Qmaster Cmds", a commands file, put a cluster name in the qmaster name field and then click append. That will make a qmaster command you can use to submit to qmaster.
 
Deanan said:
You could cheat and in RA's Render window select "Qmaster Cmds", a commands file, put a cluster name in the qmaster name field and then click append. That will make a qmaster command you can use to submit to qmaster.
Click to expand...
Niiiiiiiiiiiiiiiiiiiiice!!!!!!!!
 
So can this be used to submit a batch from redrushes into qmaster?

And if so, what might that script look like??
 
Anyone had any luck with this. I am looking to add Qmaster and 3 extra nodes to do the rendering. I really want to use Redline rather than Redalert so whats the drill.

I have tried to add Qmaster to my Crimson workflow but it completely ignores the other computers and just distributes it to the cores on the mac I sent it from. Any advice would be appreciated.
 
The script can be saved from Red Rushes, File... Export Script.

Then the script can be opened in several different ways inside Qmaster, as a general render, etc..
 
can anyone put this process in a manual?
 
Kaku Ito said:
can anyone put this process in a manual?
Click to expand...

Managed to get it working with a 12core Qmaster setup today and it has improved my render times significantly. I am working with one of my UNIX people to write a script that will automate the process a little as at the minute I have to use RedRushes to Export a UNIX redline script and then manually input by cut and paste in to Qmaster to render.

Will get back when I have something more concrete to show in the way of scripts.In the mean time this is how I have got it working.

I have 3 Mac Pro server nodes each is a 2 x 2.4Ghz dual core so I have effectively 12 nodes for rendering. You need to install Qmaster.app on each server. You get this on the FCP or Compressor disc.

I then set up 1 server to run as a Manager and Service. Setting it up to run using one instance this then spreads rendering of a clip across all 4 cores evenly. The other 2 I set to run just as a service again running as 1 instance on each. So when you sent out to render more than one clip it will render 3 at a time farming each one out to each server. I was getting around 106 frames at HD per minute so over 3 servers that is 318 frames per minute. So thats around 4.7 x realtime at 25fps.

You can run with a setup of 4 instances on each server that allows you to render out 4 clips per server on a 4 core machine but I found it reduces performance significantly although I render out more clips at once I get less frames overall over the same duration.
 
wow, you can probably sell that.

I had used Qmaster many times for Compressor transcoding for DVD jobs and it was pretty simple, but using conjunction with REDALERT! is something else.

I had to go through transcoding hell this past few days and more to come, so really looking forward to your script.
 
Although my cluster is working with Compressor, the script from REDrush ends up showing "error" in the Compressor. but I'm getting closer.
 
Well my unix guy has come up with the goods so I now have a nice workflow from RedRushes to Qmaster render farm. We have the script running all the time watching a folder that we export the RedRushes script from. It then parses it into another script with Qmaster commands changing all the syntax to work with Qmaster (painfull process). This new script runs automatically and sends a command line script for each clip to Qmaster and that does the rest. Its all automatic apart from the initial redrushes stage.

Make sure you Qmaster is set up properly and that all the servers or location s you store the .R3D and you renders are accessible by the same mounts on all nodes. This is very important. Getting the Qmaster setup to work properly is job in itself but it is worth it.

Put the script on your ingest station/Qmaster manager machine. It can go in any folder but for ease of use we have created a folder in Applications for it called redline.

Here is the script we have called it redline.sh

#!/bin/bash
cd /Applications/redline
timestamp=`date +"%d_%m_%y.%H.%M"`
for file in *.txt
do
# create script from input file
for line in `cat $file| tr -s " " ":"`
do
echo "#"
echo -n "/Applications/Apple\ Qmaster.app/Contents/MacOS/Apple\ Qmaster -clustername pfl_xserve -command \"Shell\" -options "
echo -n "\<command\>"
echo -n $line | tr ':' ' ' | sed -e "s/[^a-zA-Z0-9._]/\\\&/g" | tr -d "\n"
echo "\<\/command\> &"
done > run.sh

# run the script
sh run.sh
mv $file completed/$file.$timestamp
done


Then to run in the background you need to edit crontab to run it at regular intervals.

* * * * * /Applications/redline/redline.sh > /dev/null 2&>1


Hope this helps as it has made my life a lot more easier.
 
Gawd, Simon that's enormous.

Can I introduce this in my magazine column? Please let me know.
Let me know all of the proper credits to be mentioned, too.
 
Thanks for sharing the script. It's an interesting solution. Working fine here.
 
Kaku Ito said:
Gawd, Simon that's enormous.

Can I introduce this in my magazine column? Please let me know.
Let me know all of the proper credits to be mentioned, too.
Click to expand...

Sure you can, Our Programmer was Andy Bridger I just explained to him what I needed to be done and pointed him towards the commands needed and and he didi the rest, I just helped with some research on the correct redline and Qmaster commands to use. Glad it has been on help.
 
Simon, thanks, I'll let you know when I can figure it out to make it work on my side and let you know when it gets covered.

Kaku
 
I'm only getting 1 instance of REDline when exporting a script from REDrushes to Qmaster. I'm on a Mac pro with 8 cores and 8 instances setup. In Qmaster I add a new Shell... then set environment by loading the script file. I then submit it to my cluster. The cluster is connected to 10 rendering nodes. 8 on the Mac Pro and 2 on a G5. Only 1 instance on the Mac Pro are processing the job. Anyone else having a similar problem?

The Mac Pro is the Services and cluster controller. The G5 is services only. The managed check box is checked on both for "Shake and other shell command processing".

This is the script:

REDline -i /Users/Andrae/Desktop/RED\ DEMO\ REEL\ FOOTAGE/A001_06293F.RDM/A001_C001_0629Q9.RDC/A001_C001_0629Q9_001.R3D --outDir /Users/Andrae/Desktop/redline -w 0 -R 1 --useMeta --filter 5 --fit 1 --QTcodec 0 --resizeX 2048 --resizeY 1080 --noClobber
 
You must log in or register to reply here.
Back
Top