Red Render Farm

[Scott Brown]

One of the BIG issues for us with the Red workflow at the moment is the time taken to convert from REDCODE™ RAW (.R3D) files to something workable in FCP, such as Pro Res.

90% of what we plan to shoot with Red will be for HD broadcast output at 1080 25p and as it stands the workflow is simply NOT practical.

This has started me thinking....now we know that the Red team are capable of miracles of biblical proportions, so would it be possible to come up with a "Red Box" that could take our .R3D files and transcode these into our preferred file format and output these in real-time or certainly at a MUCH faster rate than the present workflow allows? For us personally, 1080p output would be fantastic.

I know that Jim and the team have a few BIG surprises up their sleeves for NAB and I do recall hearing about a 4K playback system and I'm wondering if this is something that they have already thought of for this device?

Anyone care to chip in with this? Surely with all the technology we have around today this could be achievable?

Best wishes

Scott

 

[Irri]

Can't RED Alert do command line i.e. render farm?

 

[jtcoleman]

One of the BIG issues for us with the Red workflow at the moment is the time taken to convert from REDCODE™ RAW (.R3D) files to something workable in FCP, such as Pro Res.

90% of what we plan to shoot with Red will be for HD broadcast output at 1080 25p and as it stands the workflow is simply NOT practical.

This has started me thinking....now we know that the Red team are capable of miracles of biblical proportions, so would it be possible to come up with a "Red Box" that could take our .R3D files and transcode these into our preferred file format and output these in real-time or certainly at a MUCH faster rate than the present workflow allows? For us personally, 1080p output would be fantastic.

I know that Jim and the team have a few BIG surprises up their sleeves for NAB and I do recall hearing about a 4K playback system and I'm wondering if this is something that they have already thought of for this device?

Anyone care to chip in with this? Surely with all the technology we have around today this could be achievable?

Best wishes

Scott

I don't have a firm plan on this, yet, but was thinking on contacting some guys that I know who have a CGI render farm out in LA ... they don't support it yet, I don't believe, but I'm sure the nework and pipeline they have built would be easily adaptable... and I think they are pretty reasonable on pricing.

I'll drop them a line and see what they think ... if it looks affordable and workable, I'll post an update here.

JT
Austin, Tx

 

[BASSAM MSSALATIE]

if it looks affordable and workable, I'll post an update here.

JT
Austin, Tx

thanks .so please if you hear anything .:sarcasm:

 

[Jeff Kilgroe]

REDline is available now for OSX and will be for Win/Lin soon.

Most network render node management softwares will allow you to customize for REDline and set up your network render pipeline. Qube! and Alienbrain should be easily adapted to support REDline. I use ButterflyNetRender for LightWave and XSI and it will work with REDline too.. I'm also thinking Spider (from the defunct Station-X) would be easily adapted too. I'll ask the guy who holds the source code if he's interested. That could be a nice free option for a lot of individual RED owners looking to set up a small farm.

 

[BASSAM MSSALATIE]

That could be a nice free option for a lot of individual RED owners looking to set up a small farm.

thanks Jeff hope you could do something about this i am really interrested
having small farm with running cows in it .

 

[Obin Olson]

what is Redline?

 

[Scott Brown]

I'd be interested to hear what Graeme Nattress has to say on this, after all REDCODE™ RAW is his baby (congrats on your new one too Graeme!) and I think he is the obvious person to get on the case with this.

It seems to me that GPU accceleration is now the big thing as opposed to the brute force grunt of masses of CPU's and I can't help thinking that there must be a way of streamlining this - has anyone come up with a definitive table of render times for REDCODE™ RAW with various computer configurations?

The current render times are keeping me awake at night and are making Red a non runner for our broadcast output.

Scott

 

[Michael Morlan]

I believe REDline is the abbreviation for REDAlert! command line tool as recently introduced in this thread:

http://www.reduser.net/forum/showthread.php?t=8570

 

[Sunshine Whitton]

Red

Red

Can't RED Alert do command line i.e. render farm?

REDLine is so cool. I was playing with it last night. Looks like you could set up quick and dirty batch processing of your r3d files. Divide the work among several computers. I wonder how long before you would saturate your shared storage if multiple nodes processed to the same shared storage for simplicity... hmmmm. This get's me thinking... Has any script kitty out there already done this?

 

[Deanan]

here's a script to generate new QT refs in a directory:
-------------------------------------------
#!/bin/tcsh

#usage: makerefs search directory

find $1 -name "*_001.R3D" -print >> /tmp/filelist.$$

foreach i (`cat /tmp/filelist.$$`)
echo Making refs for $i
./REDline -i $i -w2 --useMeta
end
rm /tmp/filelist.$$

 

[Deanan]

and another script to take an FCP EDL and make a copy list (for copying only the shots in an EDL to say a scratch system). It just generates a list of copy commands but could be modified easily to just copy directly.

-------------------------------------
#!/bin/tcsh

#usage: cpfind edl destination_dir commands_file search_directory

find $4 -name "*.RDC" -print >> /tmp/filelist.$$

touch $3
foreach i (`grep "FROM CLIP NAME" $1 | cut -d"-" -f 2 | cut -d"_" -f 2-4 | sort | uniq`)
echo -n Looking for $i
set found=`grep $i /tmp/filelist.$$`
if (`grep -c $i list` != 0) then
echo -n " "
echo Found: $found
echo -n "cp -r " >> $3
echo -n \"$found\" >> $3
echo -n " " >> $3
echo -n \"$2\" >> $3
echo >> $3
else
echo " ...Not Found"
endif
end
rm /tmp/filelist.$$

 

[Deanan]

you get the idea... quite easy to extrapolate that into whatever you need.

 

[Jeff Kilgroe]

thanks Jeff hope you could do something about this i am really interrested
having small farm with running cows in it .

Take a look at Butterfly Net Render from Liquid Dream Solutions. The "Pro" version and support for 5 render nodes is $125, or 15 nodes for $375. It has multiple jobs / batches, priority, etc.. You can access render services via internet / WAN. ...Keep your farm at home base, submit jobs from your notebook on-set, provided you can transfer R3D files over that fast. Hehe.

 

[jtcoleman]

REDline is available now for OSX and will be for Win/Lin soon.

Most network render node management softwares will allow you to customize for REDline and set up your network render pipeline. Qube! and Alienbrain should be easily adapted to support REDline. I use ButterflyNetRender for LightWave and XSI and it will work with REDline too.. I'm also thinking Spider (from the defunct Station-X) would be easily adapted too. I'll ask the guy who holds the source code if he's interested. That could be a nice free option for a lot of individual RED owners looking to set up a small farm.

I havent done a lot of batch work with Alienbrain, but I find the GUI system to be a bit slow / cumbersome, especially with larger directory trees. I know that you can go "under the curtain" and adjust the files in the working directories, but unlike subversion (or similar) I don't know of any way to initiate checkin or checkout proceedures outside of the GUI. (Maybe I'm just not familiar enough with it, though.)

JT
Austin, Texas

 

[BASSAM MSSALATIE]

Take a look at Butterfly Net Render from Liquid Dream Solutions. The "Pro" version and support for 5 render nodes is $125, or 15 nodes for $375. It has multiple jobs / batches, priority, etc.. You can access render services via internet / WAN. ...Keep your farm at home base, submit jobs from your notebook on-set, provided you can transfer R3D files over that fast. Hehe.

thanks Jeff iam amazed about NODES
it is not too much coast .
then where is the coast will gonna be ( on computer units connected to it)?
so let's say i want to build 10 cows farm so all i need id
10 pc's 10 *2k usdollars =20 k
or
10 mac's 10* 4k = 40 k
is it correct ??:sad: :huh:

 

[Sunshine Whitton]

Sweet! Thanks for that! I can definitely use these scripts... I see how you can extrapolate that in order to push cpu intensive redline tasks to other macs say over ssh...

I'll look into the software solution as well. Very cool.

 

[laguun]

thanks Jeff iam amazed about NODES
it is not too much coast .
then where is the coast will gonna be ( on computer units connected to it)?
so let's say i want to build 10 cows farm so all i need id
10 pc's 10 *2k usdollars =20 k
or
10 mac's 10* 4k = 40 k
is it correct ??:sad: :huh:

bassam, macs are pcs.
only important difference is the operating system.

while the macpros are great machines, for a renderfarm they are certainly not a good choice. to large, to many options you don´t need, etc etc.

for a renderfarm you can easily run osx on usual pcs - depending on your local laws. DISCLAIMER: In germany its legal, even when apple does not allow it, as long you don´t patch it. In other nation laws are stricter and you are not allowed, so make sure that your planning is according to local law.

ideal in the backbones would be linux naturally - as it has much lower cost than the os from apple or microsoft.

our configuration in the renderfarms, which we think is the "sweetspot".
quadcore intel 2.4/2.5 cpu: 200-250 euro.
solid actual motherboard with dual gigabit-lan and onbaord graphics: 100 euro
4 GB ram: 80-90 euro
19´ 1/2he case: 50-100 euro
500 watt poer supply: 50-100 euro

so, you look at a total of 500-600 euro per quadcore unit, or, in $€, ~125-150$ per node. excluding OS, which will raise the price 0-50€ per node depending on your choice.

to speed up things a san storage is recommendable, but a dual gigabit attachement is not slowing things down tooo much until ~32 cores.

 

[BASSAM MSSALATIE]

for a renderfarm you can easily run osx on usual pcs - depending on your local laws. DISCLAIMER: In germany its legal, even when apple does not allow it, as long you don´t patch it. In other nation laws are stricter and you are not allowed, so make sure that your planning is according to local law.

.

thans for calculating this for me so
WHAT IS THE RELATION WITH LOCAL LAW HERE ?
is there is something agains law ?

 

[Dj Joofa]

for a renderfarm you can easily run osx on usual pcs - depending on your local laws. DISCLAIMER: In germany its legal, even when apple does not allow it, as long you don´t patch it. In other nation laws are stricter and you are not allowed, so make sure that your planning is according to local law.

OSX runs on Darwin, which is open source and freely available, thanks to Apple. I wonder if somebody has tried putting just Darwin and create render farms.