Shared Rendering Power

[Harry Clark]

Anyone know anything about the Amazon and Google "shared" server farms that you can subscribe to? It seems that smaller editorial facilities without render farms could use a service like this to crunch footage...
Here's a slideshow from a company called RenderRocket:
http://www.slideshare.net/tracylaxdal/renderrocket-and-amazon-web-services/
Don't know what's involved, but seems interesting.
Cheers,
Harry

 

[Chris Kenny]

Google's system is in beta, and only works for hosting web applications.

Amazon's system could theoretically be used for, say, spitting out DPX files from R3D files once there was a Linux version of RedLine. But if you start running some rough numbers, you quickly realize that any Internet connection a small shop could afford would be far too slow to make this practical. Like, several orders of magnitude too slow. And it would cost hundreds of dollars for every hour of footage processed. I get around $400 just in Amazon data transfer fees -- no data storage or processing fees -- to upload R3D and download 4K DPX output.

 

[Dylan Reeve]

There are many rentable supercomputer clusters available on a rental basis. Mostly Linux/Unix based. Once a suitable application exists (REDline and a manager?) there would be reasonable rental options. Best bet would be to courier a drive.

Otherwise, pushing up a R3D file recorded at ~300Mbit/s is going to be pretty impossible in reality.

 

[Adam Glick]

There is currently no working solution for distributing the transcoding process for a single .r3d file.

My understanding of the processes involved with decoding, demosaicing, decompressing, scaling & color correcting the footage would indicate that "distributed rendering" of a single .r3d file cannot be feasibly implemented.

*edit* however, with some scripting and pipelining, one could probably manage multiple rendering jobs across multiple network machines. Each machine would work on transcoding a different .r3d file...

I do think that pre and post processing options for the .r3d file format will continue to evolve in the pursuit of more elegant & efficient workflows, but the work in this area is mostly incremental.

This is precisely why the development of tools such as Scratch Cine & redBOXX are such important milestones.

With these tools, you may never have to transcode again!

Of course, new solutions for faster software transcoding will always be of importance and interest and you can bet there will be advancements in this area as well.

 

[Dylan Reeve]

There is currently no working solution for distributing the transcoding process for a single .r3d file.

My understanding of the processes involved with decoding, demosaicing, decompressing, scaling & color correcting the footage would indicate that "distributed rendering" of a single .r3d file cannot be feasibly implemented.

I thought that too, but REDAlert has the option built into to it to submit a render to Qmaster, but perhaps that's just adding a single file render to the queue to be distributed to the next free node. I don't know because I've only got one Mac so I can't really test it.

It's certainly possible to farm out the processing of multiple files to a cluster and when REDline exists for Linux I will try it out

 

[M Most]

There is currently no working solution for distributing the transcoding process for a single .r3d file.

There has always been an issue using "wrapped" movie files (i.e., r3d, quicktime, avi, etc.) with render farms. That's one reason why visual effects are almost always handled using individual frame files (DPX, Targa, Tiff, etc) rather than movie files. With individual frames, a distributed render manager can send each frame to a different machine, allowing a lot of flexibility and efficiency. This holds for both sources and rendered file formats, although with the latter it's a bigger problem, as you can't write to the same movie file from multiple machines. It's also a bigger problem if you're sourcing from movie file codecs that use interframe compression, such as MPEG or H.264. These files require that multiple frames be read in order to decode only one frame.

 

[Chris Kenny]

It's hard to think of a reason why, in principle, you couldn't simply make the same R3D file available to several render nodes and have them each work on different frame ranges. So, for instance, if you had a 1000 frame R3D file and four nodes, have the first node process frames 1-250, the second 251-501, etc. If you were trying to render out to, say, ProRes, you would, of course, need an extra step at the end to concatenate all the QuickTime files, but QuickTime allows for that sort of thing.

Of course, any project large enough to justify using a render farm almost certainly consists of many R3D files, so the actual utility of implementing something like this is rather questionable.