REDCINE-X workflow solution for low budget indie feature - suggestions please!

[Willem Grobler]

Hi all (please skip to the questions right at the bottom if you don't wanna go through the wall of text! )

Apologies if this has been covered in other threads before - I scrolled through the first few pages of the forum and also performed some searches, but there were no threads that answered and summarized everything I needed to know succinctly... I'll try and order my questions in as logically as possible. If anyone would care to provide answers (if only to select questions), it would be greatly appreciated.



SYSTEM SPECS

REDCINE-X (transcoding) system

Model Name: Mac Pro
Model Identifier: MacPro3,1
Processor Name: Quad-Core Intel Xeon
Processor Speed: 2,8 GHz
Number Of Processors: 2
Total Number Of Cores: 8
L2 Cache (per processor): 12 MB
Memory: 10 GB
Bus Speed: 1,6 GHz
Boot ROM Version: MP31.006C.B05
SMC Version (system): 1.25f4

Final Cut Pro (editing) system

Model Name: MacBook Pro
Model Identifier: MacBookPro6,2
Processor Name: Intel Core i7
Processor Speed: 2,66 GHz
Number Of Processors: 1
Total Number Of Cores: 2
L2 Cache (per core): 256 KB
L3 Cache: 4 MB
Memory: 4 GB
Processor Interconnect Speed: 4.8 GT/s
Boot ROM Version: MBP61.0057.B0C
SMC Version (system): 1.58f16

Both systems are running the latest updated versions of Mac OS X 6.5



WORKFLOW

Ingest R3D media & backup thrice ->

Transcode via REDCINE-X system to Apple ProRes (proxy) (4k to 1080p 1/2 Debayer Premium, Mitchell software scaling) ->

Do rushes/selects and assemblies on FCP system.

We're going for a very mobile option and so the shoot may have to move from our current location to another in the last week of production, so there's a chance the FCP system may take over from the REDCINE-X system for the transcoding tasks in the last week. Questions on that to follow.



STORAGE SOLUTION

Pro AVIO RAID, mirrored RAID set, connected via two eSATA cables (3Tb storage space, for transcoding)
8 x G-Tech 2Tb drives, for 3 x backup of R3D source material and editing Prores proxies on FCP system (still speculating on this we may need more)

Now, the question:



QUESTIONS:

I wanna have the transcode work as efficiently as possible - so I require a good speed/safety ratio. The following questions all relate to this:

1) Will I see a performance boost running REDCINE-X build 348 (64bit) with Mac OS X 6.5 in 64bit mode? How is this different from running build 348 (64bit) 6.5 in 32bit mode? (Have read all kinds of reports on this, but wanna get some more opinions).

2) In which other ways can I tweak my system to optimize transcoding times? Clearing REDCINE-X's cache? Clearing the system's overall cache? Can anyone recommend any apps to assist with this, otherwise advise how to do it in the terminal?

3) In the event that we need to move locations, and use the MacBook Pro to transcode, will it be safe to transcode from the laptop's internal HDD to an external G-Tech 2Tb drive (via FW800)? Or should I daisy chain a Rugged Lacie (we'll get R3D source material from DIT on these drives) to the G-Tech? Which will be fastest/safest?

4) Since we're transcoding to Prores proxies with 1/2 Premium debayer, many shots often look soft or out of focus, while they're actually IN focus when you look at the R3D source material... I tried using Lanczos (sharper) vs. Mitchell (smoother) for my last transcode, but didn't see a marked improvement, and my transcode took just under 4 hours (and resulted in bigger file sizes) while my earlier transcodes (using Mitchell) were just under 3 hours with smaller filesizes... Any suggestions here?

5) Is it faster to (a) transcode from the RAID onto the RAID, (b) from the RAID onto the internal drives (SATA) or (c) from the RAID onto the external G-Tech drive (via eSATA)?

6) Does REDLine actually, REALLY make a difference in terms of speed? Does it hamper reliability?

***

That's about all I can think of right now... Thanks for reading, and again, apologies if a lot of this has been dealt with in the past - I'm very new to this whole process.

With thanks,
Willem

 

[Cory Petkovsek]

5) eSATA has 3 or 6gbit/s peak and FW800 has 768mbit/s theoretical peak. However you won't see those speeds. Instead your maximum will be limited to the throughput of the spindle itself. Your drives are probably 7200rpm and will likely have a sustained read throughput of 40-100Mbyte/s each (100Mbyte/s == 800Mbit/s or a little over theoretical FW800); maybe half to 3/4ths that for write.

High end raid systems (enterprise class) and some workstation class raid systems (ie 3ware) often provide a write performance benefit by allowing the system to write to cache; however this is only when there is a battery backup module connected to the raid controller. Your "Pro" AVIO raid system most likely doesn't have such a thing so your performance will probably be limited by spindle, as noted above.

Therefore, you'll get the best performance reading from one spindle and writing to another. You can run a disk performance utility and see the relative speeds of your drives.

Gtech says their drives are fast (70-115mbyte/s), but the test results on their website are a little questionable. The laptop drive will be nothing special, I guess 60mbyte/sec. The lacies are 25-40mbyte/sec. In any case if your external devices have multiple inputs (fw/esata) test them all on their different ones and see which are fastest for both read and write. Then pick the best two. They will probably be a g-tech and your raid.

Cory

 

[Willem Grobler]

Hey Corey, thanks man.

Wow that's pretty technical - a lot of it went over my head, I won't lie...

For what it's worth I've been doing tests the whole week, and transcoding has taken between 4.5 to 2.75 hours per (the same) 30 min of Red 4k footage.

The 2.75, which is the lowest so far, was when I did everything on the RAID.

Our G-Techs just arrived so once the test that I'm doing now (RAID to internal SATA) is over, I'll do the same test, RAID to G-Tech via eSATA and let it run overnight.

So I guess it's a trial and error thing - just kinda testing and messing about until you find optimal results...

 

[Cory Petkovsek]

Your workflow gives you the most accurate results. However they are long in coming. I suggested downloading a disk benchmarking utility that will run a quick (few minute) test on each drive to test the read (and maybe write speed) of the drives. It should give you a hard number like 40mb/s. Then you'll know which ones are fastest. Pick the two fastest and transcode from one to the other.

Cory

 

[Frank Glencairn]

WORKFLOW

Ingest R3D media & backup thrice ->

Transcode via REDCINE-X system to Apple ProRes (proxy) (4k to 1080p 1/2 Debayer Premium, Mitchell software scaling) ->

Do rushes/selects and assemblies on FCP system.

Bleeding Christ!

Why don´t you install Premiere CS5, throw the R3D files on the timeline and just start editing in realtime?

I never got it why some guys do all that transcoding/proxy Voodoo Shmoodoo to themselves.

Frank

 

[Dustin Cross]

Willem,

Sounds like your off to a good start, but you should really consider hiring a DIT that has experience to take care of this stuff. If Cory's first post went over your head then it sounds like your not a tech guy. Where are you guys shooting? Do you have any budget to hire a DIT? You can get some competent people for a pretty low rate.


Dusty

 

[Willem Grobler]

@ Dustin - thanks man - yeah, been running tests the whole week - we're getting our first shot later this afternoon as a 'test run' before we start shooting in earnest on Monday. We have a DIT, but much like me it's his first time with the Red workflow... We've been putting our heads together and he's way more technically inclined than I am. At this stage we're very much on the same page and I'm confident that the operation will move along smoothly. I also consulted with a post house next door to our offices who have extensive experience with Red and they also approved of our workflow. I'm sure next week we'll run into some teething problems, but that's to be expected. Once we find our feet I think we'll be pretty good to go from there.

We are shooting in Cape Town, South Africa.

@ Fred - haha dude I hear you. But the director will be very much hands-on with the editing process after we've finished production, and as his previous film was edited in FCP, this one will be too. The other thing is that the producer has a VERY tight budget to work with, and quite frankly, the Final Cut Studio 3 upgrade (from 2) just made more financial sense than Creative Studio 5, or even Premiere Pro... I would love to give Premiere Pro a spin though - I've been editing mainly in FCP the last few years, but I cut my teeth with Premiere when I was still in college... And have only been hearing good things about it since CS5.

@ Cory - thanks again man. Didn't get around to getting a app to test with, but will do so later today. Last night I let it transcode and I got the best results yet transcoding from RAID to G-Tech 2Tb via eSATA - 2 hours 20 minutes... So I think that may be our best course of action...

I'll keep you guys posted... Thanks again.

 

[Willem Grobler]

Bleeding Christ!

Why don´t you install Premiere CS5, throw the R3D files on the timeline and just start editing in realtime?

I never got it why some guys do all that transcoding/proxy Voodoo Shmoodoo to themselves.

Frank

BTW, one more thing - how would 4K hold up on our edit suite if we ran Premier Pro? Would you be able to assemble a WHOLE feature film in 4K running Premier Pro without any hiccups?

 

[Gunleik Groven]

Bleeding Christ!

Why don´t you install Premiere CS5, throw the R3D files on the timeline and just start editing in realtime?

I never got it why some guys do all that transcoding/proxy Voodoo Shmoodoo to themselves.

Frank

Many reasons...

Transcoding is the best faulttest. If there's any copy faults, you'll see it in transcoding, in additional to checksum copy, it is a good way of securing the data.

The transcoding is a good way of checking your dataintegrity.

Security/mobility.

Oflline files are small and quick to work with and don't demand much from the computer. If anything happens to the media, nothing bad really happened. You just generate new. No playback issues. The editor don't have to fiddle with meta-controlls. The project itself is highly mobile. No insurance problems...

On shorter projects with short turnaround, I would like to stay raw all the way.

With longform - offline/online hands down.

Conforming back is the push of a button.
Added flexibility, ease of use and good nights for no wories about media-handling:

Totally fantastic.

All in all you'll save space, time, hazzles and be more flexible for long form.

It's hard to say exactly at what length/how many recording days offline/online outperforms RAW workflows, but I wouldn't consider it for anything longer than 5 minutes delivered.... Yet... -

 

[Bing Bailey]

put 4 x 2tb caviar black drives into the mac pro tower and create a raid 0 across them. you'll get over 500MB speed , you should be using the faster machine to transcode and edit. swapping out the laptops or towers drive for an ssd would speed things up too. unfortunately that all means spending another $1200

 

[mansh]

why transcode

why transcode

Bleeding Christ!

Why don´t you install Premiere CS5, throw the R3D files on the timeline and just start editing in realtime?

I never got it why some guys do all that transcoding/proxy Voodoo Shmoodoo to themselves.

Frank

I am wondering same thing, why would anyone want to trascode . i use premiere pro on windows 7 and r3d is the best thing happened to me recently. If anyone knows any other reason one should use transcoding for please let me know

 

[Cory Petkovsek]

BTW, one more thing - how would 4K hold up on our edit suite if we ran Premier Pro? Would you be able to assemble a WHOLE feature film in 4K running Premier Pro without any hiccups?

Ha! Actually asking if one could run premier pro without any hiccups tells me you've never used it. PPro does work. I've used it on certain projects, but it frankly sucks. It requires lots of system resources and becomes very unstable without them. In contrast, I can run vegas with far less resources, far greater response times, an interface I prefer editing with, it renders considerably faster, and it's far cheaper. Both vegas and ppro can use the r3ds without transcoding.

To answer your question, if you have a 64-bit system, fast processor and a LOT of memory, you should be able to edit 4k in ppro. I can comfortably edit 4k in vegas on my 32-bit 3gb core 2 duo laptop, or even my older 32-bit 2gb pentium 4 desktop! My laptop has difficulty editing HD in ppro, let alone 4k! The desktop can't touch HD in ppro. I wouldn't touch PPro for a feature at any resolution on any system however. I wouldn't flinch to do it in vegas or fcp.

No system is perfect though; vegas has it's own problems, but they are workable.

Cory

 

[Cory Petkovsek]

I am wondering same thing, why would anyone want to trascode . i use premiere pro on windows 7 and r3d is the best thing happened to me recently. If anyone knows any other reason one should use transcoding for please let me know

It's kind of one of those things that if you don't know why you'd want it, then it's not applicable to you. Editing raw is fine if you're on a system that supports it. If you review Gunleik's list and nothing comes up as "facepalm; That's how to fix that issue!", then it's not worth the added time to transcode. His list is generated by the pain of losing hours or days of effort that transcoding before hand could have avoided.

Suffering causes us to look for better solutions. You still use PPro because the pain is below your threshold. I've suffered enough and will never use it again if I can avoid it.

Here's one small demonstration of when transcoding might be useful for you. Try editing some H264 HD footage in ppro. Then transcode it to lagarith or prores if on a mac and edit it. You should notice a difference in editing and playback speed.

Cory

 

[Willem Grobler]

Cory, can you recommend any hard drive speed test apps for Mac? Been searching for a few mins, can't find anything reliable...

I've encountered the strangest thing - copying to my RAID via Rugged Lacie is SUPER slow, whilst going from Lacie to the internal drives to my G-Tech external is MUCH faster... Shouldn't it be the other way? Even G-Tech to RAID via eSATA is VEEERY slow...

 

[Gunleik Groven]

Aja's system test is free and works well.

I think Blackmagic may have one, too...

 

[Cory Petkovsek]

Only a few minutes? I don't use mac but my first google search pulled up xbench, disk speed bench, and disk speed as the first three hits which are all free and exactly what you need.

http://www.google.com/search?sourceid=chrome&ie=UTF-8&q=disk+speed+benchmark+mac

Cory

 

[Willem Grobler]

Haha Cory, I don't think I possess the right vocabulary - my search didn't turn up the same results cos I don't think I used the right key words in my search... Thanks for that link... Gunleik's AJA recommendation also helped.

Thanks guys.