6K Edit Station

[Marc Thorpe]

Three scenarios. What would be the ideal system for editing 6K footage into a deliverable format for Broadcast / Distribution within the three categories of No expense spared, Costly but not astronomic and Indy Filmmaker options. ANy overview of component parts, software and price expectations would be cool. For this I am assuming that voice and score are provided by the respective facilities / artists and laid to the project by the editor. Looking more for examples of three price range options.

Cheers,
Marc.

 

[Eric Bowen]

I would suggest the following:

Asus X99 Deluxe or WS
Xeon E5 2690V3
64 GB of ram
Samsung 850 Pro SSD OS
Storage - Raid of some type depending on media and workflow
Once of the coming very soon 900 Series Titan cards or 2 if using Davinci
I would look at Adobe first for the software since it's really the only application out besides Davinci that has the performance for these files realtime right now.

 

[Brandon J.F.]

Eric, will a Xeon E5 2690V3 debayer 6k footage in real-time?

 

[Eric Bowen]

When I did the testing here the 2697 V3 handled 6K full res in Adobe without dropping frames with a 60 to 80% load. I suspect the performance is better now since the realtime playback for R3D is threading very well with the updates to CC 2014. I tested with the initial launch of 2014. Should be np getting 1/2 res in Davinci with 2 GPU's especially with the coming 900 series Titans. Maybe even full depending on the caching models Davinci has. RCX Pro should be the same performance as Adobe that I have seen lately. Maybe a little less but should scale over more than 1 video card.

 

[Marc Thorpe]

I would suggest the following:

Asus X99 Deluxe or WS
Xeon E5 2690V3
64 GB of ram
Samsung 850 Pro SSD OS
Storage - Raid of some type depending on media and workflow
Once of the coming very soon 900 Series Titan cards or 2 if using Davinci
I would look at Adobe first for the software since it's really the only application out besides Davinci that has the performance for these files realtime right now.

Interesting stuff. What kind of ballpark on the ca$hflow are we talking here?

 

[Danko Dolch]

Why to use a more expensive and slower clocked Xeon CPU for a single CPU socket system?
Do you need Intel vPro functions or ECC memory?
Would'nt be a Intel Core i7 5960X more appropriate?

For editing a fast 8 core CPU should be sufficient.
But for 4K realtime grading of 6K R3D fotage I suspect a second CPU for wavelet decompression would be needed for stable every day live performance...

Is anybody using a single CPU setup with 6K R3D in Resolve and getting realtime performance?

 

[Jeff Kilgroe]

Depends, sometimes those couple extra cores make a big difference. In the no expense spared category, I would recommend a dual E5-2690V3 20-core system with 128GB or even 256GB RAM and a trio of the new Titan cards that are about to launch.

An overclocked i7 5960X CPU will perform great and offers the best performance to price ratio for many tasks. But like I said above, sometimes those extra cores can play a significant role.

Eric's configuration above with the X99 should probably fall into the costly category, but could arguably be fitted to indie filmmakers as well. Go with the 8-core i7-5960X and it is definitely something most serious indies can justify, that shaves $800 off the price tag of the system and allows for the use of faster memory, even if you don't do any tweaking/ overclocking. But you do lose those two cores...

 

[Danko Dolch]

Ji Jeff,

you're right about the fact that one can't have enought cores ;-)

If it comes to Xeons...


current shopping situation from my default dealer:
(performance rating estimated based on core count & clock speed)

XEONS:

[A] Intel® Xeon® Prozessor E5-2687W V3, CPU
cost: € 2.399,-
cores: 10
base-clock: 3100 MHz
performance: 25% more cores = 25%
availability: unknown

Intel® Xeon® Prozessor E5-2699 V3, CPU
cost: € 4.679,-
cores: 18
base-clock: 2600 MHz
performance: 125% more cores but 25% slower clock = 100%
availability: unknown

[C]
Intel® Xeon® Prozessor E5-2690 V3, CPU
cost: € 2.349,-
cores: 12
base-clock: 2600 MHz
performance: 50% more cores but 25% slower clock = 25%
availability: in stock


Core i7:

[D] Intel® Core™ i7-5960X, CPU
cost: € 1.129,-
cores: 8
base-clock: 3000 MHz
performance: 25% faster clock by overclocking = 25%
availability: in stock


* pricing incl. tax


that means:

1. All interesting Xeon offerings arn't available and Xeon CPU's cost about 200% for the same performance.
2. If you go for lesser cores and higher clock to get good real world performance with all the "not so good multitheraded" apps out there i7 wins.
3. If cost don't count and you luckily find some in stock you can get 2x i7 performance on one CPU socket or 4x i7 performance on dual socket.
But you have to invest 9385,00 EUR instead of 1.129,00 EUR for the CPU's alone ;-)
4. If you want more than 64GB of RAM you have to go the Xeon route.

--> Whenever you can get the job done by a X99/i7 rig do it!
With Xeon's availability of the interesting models is bad an the price is extraordinary.

@ Jeff: "...E5-2690V3 20-core system with 256GB RAM and a trio of the new Titan cards..."
God: Ok - 2 wishes...
Me: God, can I have a infinit powerful machine?
God: Hmm - yes - will send you a X99 E5-2690V3 20-core system...
Me: Can you make it even faster?
God: *** it! Ok - will send you a dual socket E5-2690V3 20-core system... but you'll cost me a fortune!

)

 

[Marc Thorpe]

Just running out to get a Swahili translation phrasebook.... :wink5:

 

[Filip Vandoorne]

if editing of 6K files is the real purpose, i would recommend a offline-online model...
Learn to use the editing workflow of Final Cut Pro X with the proxy files. 6K files would be still be 3K prores files...
plenty of info and very fluent editing. Then do the online directly in Final cut pro X
or sent the project to Davinci Resolve for final grading, consolidating = keep a RAW master , making broadcast files.
If the project is using dual sound, add Sync-n-link X in the mix and learn to use Roles in Final Cut Pro X.
on the hardware side : MacPro 6 - 8 - 12 core with D700, fast thunderbolt raid, LG 34" or the 4K LG 31".
Silent and mobile, nice and big screens.
Or even a top iMac 5K or MacBook Pro 15" combined with a LG can do the job once the -proxy- files are calculated.

 

[Danko Dolch]

If it's only about editing R3D most notebooks can do it easily without proxys... just set debayer to quad res and ready.
(one reason for proxys would be to save some disc space and transcode to DNXHD36 or even 4MBit MP4 so that one can keep the whole project on a external 500GB HDD...)

If you want to view realtime HQ debayered footage, transcode or grade 4K realtime thats another case - then you need as much CPU & GPU power as you can get...

@Marc Thorpe: a Swahili translation phrasebook helps to understand my post? - cool, need one ;-)

 

[Marc Thorpe]

if editing of 6K files is the real purpose, i would recommend a offline-online model...
Learn to use the editing workflow of Final Cut Pro X with the proxy files. 6K files would be still be 3K prores files...
plenty of info and very fluent editing. Then do the online directly in Final cut pro X
or sent the project to Davinci Resolve for final grading, consolidating = keep a RAW master , making broadcast files.
If the project is using dual sound, add Sync-n-link X in the mix and learn to use Roles in Final Cut Pro X.
on the hardware side : MacPro 6 - 8 - 12 core with D700, fast thunderbolt raid, LG 34" or the 4K LG 31".
Silent and mobile, nice and big screens.
Or even a top iMac 5K or MacBook Pro 15" combined with a LG can do the job once the -proxy- files are calculated.

Wahhhh, so editing as I have been for the past 20 odd years by directly ingesting the media, HDV and Canon etc is out of the window? I understand FCPx, it's what I use at the moment for cutting together documentaries I create etc. When you say using proxy files I take it these are mirror image files of lower resolution which are then conformed as an EDL and applied to master files? Head is exploding? I was thinking I could simply do the same as I have been doing and as such wanted to know what kind of beast computer would allow me to do that.

Cheers.
M.

 

[Eric Bowen]

There is no reason to offline edit the 6K media when there are more than viable affordable solutions to edit realtime. The Xeon 2690V3 is a 12 Core Xeon and more than enough to edit the media at 1/2 res or full res depending on the application. The 5960X is cheaper and will handle the playback realtime at full res provided you have it clocked 4GHz plus with the ram at 2666. However the load is 100% at full res preview or atleast it was before the updates. The 14 Core Xeon was at 60% the majority of the time or less. So the 12 Core Xeon should be more than enough for this. The Xeons handle the realtime load so well because they have more cache than the I7 chips besides the Cores. However the trade off is often slower render times and cost. Either way the system cost should be 6K to 9K cost depending on which way you go for CPU and the rest of the config.

 

[Elsie N]

Hope no one minds a quick question that has been on my mind. Is it best to have original footage on a separate fast SSD HD from the one the applications are on, or is the performance faster when working within the same SSD HD? (editing, transcoding etc.)

 

[Eric Bowen]

Yes it is. The SSD drives are so fast latency that you can read media from them and handle the OS read and write requests. The problem is SSD's start to slow down when you constantly write and delete files. They do a bunch of processes to avoid using the permanent erase write cycles that actually use up the charges for the NAND. However those processes are still not the most efficient and the SSD will slow down if it cant keep up with cleaning those areas up over time. The only way to reset the SSD to it's original speed is write 0's or format the drive. That is easy to do on a media drive but not on the OS.

 

[Filip Vandoorne]

the proxy files of FCPX are really easy to make, you just confirm them at import of even afterwards you can ask the transcoding.
you can also edit directly on the RAW files if you want to, but even on a fast system, the response times of the system will NOT be the same as when editing the proxy's. The FCPX proxy files for 6K will still be 3K resolution, bigger than you HD masters, so plenty of info ... nor really low resolution..
When you need the 'full' 6K RAW, you just flip and switch in FCPX and can render out your HD master, or sent it to your colorgrading package - Resolve.

6K 24fps 7:1 compression are still big files to read from a disk or decompress. we are talking about 110MB/s and lots of processing power is needed or a RED rocket.

Did you try the proxy RED workflow in FCPX, it is not so bad as it sounds, and much more fluid in editing than working directly with the RAW's in FCPX or Premiere.
Editing is not starting en stopping a playback, it is jumping back and forth, fast forward, scrubbing and finding the right image and cut. Much easier on a prores file...

Even working on a fast thunderbolt RAID, editing on a proxy is way more productive. Your HDV-Canon files were only 3,6-5MB/s... 6K RAW compressed files are 30 times as big..
a uncompressed 6K RAW bayer file would be 700MB/s and a RGB 6K version is 3times as big after debayering... a RGB 10bit ultraHD uncompressed is around 800MB/s of data that the systeem needs to process...

also if you buy a high-end pc for editing, it will not be easy transportable -perhaps not important.
it also will make a lot of noise and heat, prepare for some long display and keyboard cables. a Macpro2013 is hand luggage and makes less sound than a persons breath...

 

[Eric Bowen]

Actually the single Xeon X99 systems are not loud at all if you use the Noctua heatsinks. The Xeons cpu's dont generate as much heat as the i7 8 core clocked higher. The 900 series video cards are actually very quiet because they generate far less heat and pull less power than the AMD or 700 series Nvidia cards. So noise is not that much of a factor with that config. Also 6K DPX files for example take far less processing than Red 6K files do. In other words the lower the compression level the easier the processing. Remove the debayer element and the processing is far less. So 6K uncompressed files are as easy to process as Pro Res. The difference though is HDD bandwidth requirement. 4K DPX files take on average 1.2GB/s storage bandwidth to playback. 6K are closer to 2GB/s. That either requires a 16 bay SAS mechanical array for 1.2GB/s or a 6 drive SSD raid 5 for 6K DPX files. However you can easily playback R3D files in 6K with a single SSD or several with a SSD raid 0. So the requirements here are not that great when talking about media like this. In all reality Red's 5K and 6K codecs are the heaviest processing load codecs out in the market right now. So if a X99 can play that back realtime flawlessly then you can playback anything else. The latency on scrubbing through the timeline and such has to do with caching models, codec compression/complexity, and processing latency. The difference that DDR4 brings to the table bandwidth wise more than makes up for the difference between 3K and 4K Pro Res versus other codecs. You talking microseconds or milliseconds difference here in how long the change takes to draw out on the screen. Far less that most people even notice if any. I scrubbed through the timeline forwards and backwards with the 5K 96 frame Red media at 60 frame preview and it was as fluid and fast as realtime. Pro res 3K wouldnt be any faster there. Offline is simply about limited equipment now and not the required workflow. If you have the right system offline is pointless and more time sink required to reconform the edit lists.

 

[Marc Wielage]

Three scenarios. What would be the ideal system for editing 6K footage into a deliverable format for Broadcast / Distribution within the three categories of No expense spared...

Ted Schilowitz's "Devil & Demon" company has some pretty screaming-fast computer workstations in the $50K+ price range...

http://www.devilanddemon.com/

On paper, the systems look fairly incredible. In a serious facility (particularly with this kind of money), I'd keep all the machines and servers in an adjacent soundproofed room, and have the editor, colorists, VFX people, and sound people all in quiet rooms without computers. Maybe have a couple of USB3 hubs to pull in files and hand off work copies and stuff like that, but nothing else.

 

[Elsie N]

Yes it is. The SSD drives are so fast latency that you can read media from them and handle the OS read and write requests. The problem is SSD's start to slow down when you constantly write and delete files. They do a bunch of processes to avoid using the permanent erase write cycles that actually use up the charges for the NAND. However those processes are still not the most efficient and the SSD will slow down if it cant keep up with cleaning those areas up over time. The only way to reset the SSD to it's original speed is write 0's or format the drive. That is easy to do on a media drive but not on the OS.

Eric, FANTASTICALLY useful information!

Follow up question... when finished with a project, could one delete the source files from the SSD, make a complete copy to a thumb drive(suggested method?) of the SSD drive, reformat the SSD and then re-install the OS and apps to an up-to-date state?

 

[Eric Bowen]

I have actually re imaged SSD's to see if it cleans up issues along with performance slow down from a image file made of the entire OS drive. It definitely cleans up volume issues and performance to some extent. However I would not do this on a monthly basis. That would be something to do every 6 months to a year. If the amount of media files you write to an SSD requires you to image it on a monthly basis then using the OS drive for media is not a good idea at all. To give you an idea, 3 days of benchmarking with 4K+ R3D and DPX files slowed down even the latest SSD crop that are most efficient to half their rated speed. Granted I exported atleast 500GB to 600GB of data and deleted probably close to as much in that time period but that gives you an idea of the load it takes to slow these drives down and how quickly it occurs.