Brook's fast CF card reader

[Sanjin Jukic]

It's sort of a DIY thing. The laptop only has one lane of PCI available to it. That means that even if your input [say, from multiple CF readers] is fast, the only output you will have available to you is over FW800. Your overall transfer speed will always be severely limited.

Then you would need Magma enclosure with one or more PCIEXpress slots...

 

[Imran Farouk]

So....going with a magma enclosure with one more PCIExpress slots would expand my one lane PCI into multiple lanes? Or would i still be ham-stringed at one lane? (Sorry about my lack of knowledge, sort of new to this whole DIY thing...)

I figured a system like the pocket rocket though would leave that one lane available to the rest of the system still so using a similar set up it would be able to create more 'pipes' to feed the multiple CF card readers into the MBP...bringing the potential for that 2min transfer time...

 

[Alex Carr]

Port Mulitplier is much cheaper than a magma + Sata Card, But On a laptop you are severly limited anyway. However, even if your target drives are on the portmulitplier it is faster than FW800.

I made a three CF card reader to e-sata using a Port Multiplier. Its still very fast, and checksums in under 2 minutes. Transfers are fast,

My laptop is a backup, to my desktop. If you are needing all this equipment to try to make something work on a Lappie, Believe me Get a Desktop!

 

[Cail Young]

Your overall transfer speed will always be severely limited.

I can get 80MB/s over a single-lane eSATA card (from one port to the other) - mind you this is between two hard drives, but for the work we do, that's enough for me...

As soon as budgets in this country can afford me bringing a Mac Pro on set you better believe I will. However Melbourne isn't LA, and won't be for a long time :/

 

[Brook Willard]

Even if you get a Magma enclosure, your laptop's max speed will still be one lane. It's almost like daisy chaining FW800 drives together. You could plug in 10 if you wanted, but you're still forcing all of that data through one tiny pipe.

A laptop is acceptable on lots of jobs and in some markets... but that will change rapidly. Dealing with ~30-40MB/s of footage from a camera or two is not a big deal. But in 6 months when DITs or Digital Loaders are dealing with cameras that shoot 50MB/s to 100MB/s... it'll be different. Add in color correction to that mix? It's over.

I've seen a lot of people spend thousands of dollars on their laptops to try and make them something they're not. Adding expansion chassis, RAIDs, giant battery systems, fancy laptop stands, giant carry bags, etc. There's a point at which your "laptop" cost $15,000 and weighs 60lbs [plus the cart you had to buy just to hold the thing!].

And no matter how much stuff you add to the laptop, the producer will always see a laptop. Why should they pay more than $50 a day for it? After all, "we had this one kid who did it on the last job for $25 a day, so I'm helping you out."

It's time to start looking to the future. In my opinion... the laptop is dead. Elevate yourselves! Or you'll always be that guy with the laptop who drags and drops files in the Finder.

 

[Evangelos Achillopoulos]

So....going with a magma enclosure with one more PCIExpress slots would expand my one lane PCI into multiple lanes? Or would i still be ham-stringed at one lane?

The PCIe ver 1.0 port at all laptops including the new MBP 17" are 1x lane, that's 2.5Gbits / 8 bits = 312Mbytes/sec minus 20% overhead equals to 250Mbytes/sec absolute maximum throughput...

Magma or anyone else, to make a two slots out of one, it has to use a lane switching mechanism that adds another level of overhead... so the two ports will deliver theoretically 125Mbytes/sec each, that makes available (if I deduct a -25% overhead for the switching/latency mechanism) roughly something like 90Mbytes/sec for each slot as an absolute maximum bandwidth...

Its understood that real life numbers could be lower...

Someone would say a 90Mb/s would be enough for RR to work... but this is not totally correct.

In order to playback a 25fps 1080 signal you have processing time of 1/25th of a second to load the frame, debayer, scale and finally output the frame... The speed of loading a frame is crucial... because if it doesn't load fast enough the GPU would not have the time to process it...

The engineers that designed RedRocket, didn't put an x8 interface just for the fun of it...

With the REDCODE 42 coming in one month time I wouldn't feel very safe with just 90Mb/sec transfer to RR...

I don't want to mention REDCODE 100 for Scarlet or 225 for EPIC that hopefully will come in few months...

If I was you, I would wait for real life benchmark tests from people that will have all the pieces together...

All what I'm writing is just plain engineering facts that we must have in the back of our mind...

 

[Imran Farouk]

Ok, well if laptop is out of the question, how far has anyone travelled with a Mac Pro? Just wondering how that transportation would happen if it was over great distance because like Cail said, its not feasible in some places to have a Mac Pro, just because no one is willing to pay for it. I'm flying solo so really I have my Mac Pro fixed up, if anyone has travelled from one place to the other then maybe it is time to fix up the system with that CF Card Reader, (by travelled I don't mean from your studio to the location in the same city, I mean between cities or countries even)

My only thought it is, Brook how many Mac Pro's do you have on set? Currently mine has a black magic intensity, graphics card, red rocket + SDI out...thats four slots taken up...that would mean the need for two Mac Pros on set if you wanted to colour grade in real time with the help of the Rocket...right?

Apologizes if this is hijacking the thread, just trying to get my head around this, not the most tech savy of people but you have to learn the game at some point right?

 

[Brook Willard]

I have two on set. I've taken them all over the country... they just travel in the camera truck with the rest of the gear. If we're shooting overseas, production generally puts a container on a ship for the camera gear. I haven't had to fly pack in a while... but when I do, I just carry 2 or 3 laptops. A job where everything is flying is generally a job that will accept the limitations of a portable system.

At this point I don't have a Rocket on set... my personal card is in the hands of Gamma & Density so that they can update 3cp to work in realtime with RED footage. Since I'm currently on Southland - a show that I do not do any rendering on - it's not an issue. I'm personally not a fan of the DIT becoming a "mobile post house" and rendering. I've actually never done it... not on a single one of the hundreds of RED jobs I've completed. That said, my equipment package has the power and throughput to handle it... particularly when the Rocket is installed.

When the Rocket SDK is updated to enable non-R3D output over HD-SDI, it'll be a huge update. In other words, when I can put non-RED footage [ProRes, Canon, etc.] out over HD-SDI to a good monitor... it'll replace my Kona or BlackMagic. Until then, it'll be awesome for RED footage only.

On Southland, I use the same tower for data backup as I do color correction. The other tower stands by as a backup. That said, I carry additional backups for just about everything [graphics cards, hard drives, PCI cards, cables, readers, etc]. I've lost a logic board, power supply and monitor in the same day [back on the Southland pilot]. I still made it out at wrap.

Plan for everything!

 

[Dane Brehm]

"I'm personally not a fan of the DIT becoming a "mobile post house" and rendering"

Especially on TV shows when your shooting 400GB+ per day. Don't set yourself up to fail or not make a deadline. You can't possibly compete with a room full of computers. Now on Commercials that's a whole other deal....

In addition to being a Red Tech, I'm also a Phantom Op, S.Two, P2, Weisscam, SI-2K, and Local 600 DIT. My cart has to meet multiple formats, operating systems, and data needs. I use multiple methods to transfer data like SAS, esata, 1gig-e, 10gig-e, fiber, infiniband, and Firewire800 to transfer my Cinema Data. I have to be ready for anything! So should you. This is why we make Operator rate or more to be the CYA for Production.

Organize and Anticipate your needs and the needs of your department.

 

[CJ Roy]

"I'm personally not a fan of the DIT becoming a "mobile post house" and rendering"

Especially on TV shows when your shooting 400GB+ per day. Don't set yourself up to fail or not make a deadline. You can't possibly compete with a room full of computers. Now on Commercials that's a whole other deal....

On virtually every commercial & music video I now work on, I transcode on set. It's certainly a help to create the offline for fast turnarounds. And depending on the DP, I'll do a one light as well. We'll set a look, punch that look into the camera and render it out, tweaking here and there.

An 8-core is definitely a must.

 

[Tony Lorentzen]

Brook - you must be charging a pretty penny to afford such a vast setup?! I'd love to learn more about how you built your business, but I guess that's a trade secret (which I completely understand). I'll just have to build my own slowly... nobody here wants to pay for anything these days. Many will basically just rent more drives and cards and send them directly to the post house facility for backup and further processing if it's a medium to low budget feature.

 

[Jim Perry Jr.]

If you take one H380, one fan cable and 4 CF readers and build a clean
power system for the CF readers, you'll end up with 4 full-bandwidth
SATA II CF readers. If you're offloading to a fast enough RAID
[My primary RAID 6 reads/writes about about 1GB per second],
you'll be looking at 2-3 minutes to offload 4 CF cards simultaneously [via the Finder].

Brook
what are you using as your RAID 6? 1gb per second.....
is this a DIY or off the shelf?

 

[Brent J. Craig]

Many will basically just rent more drives and cards and send them directly to the post house facility for backup and further processing if it's a medium to low budget feature.

Really?!?

I have never heard of anyone doing that, on any size shoot. Drives and computers are so cheap and readily available that even the very smallest shoots at least make backups on set.

 

[Dane Brehm]

I agree that some productions have been stupid paranoid into thinking that if they buy say, 50-16GB cards that you can treat it like film. This in my book defeats the purpose of having a data based system which having multiple clones of the original data.

What Productions forget is that drives can be "re-used". I have a commercial client that has reused the same 3 1TB drives since we purchased them 6 months ago. Thats a 500 dollar cost savings that can go towards say my rate or art direction.

 

[Brook Willard]

I have to be ready for anything! So should you. This is why we make Operator rate or more to be the CYA for Production.

Exactly true. A DIT should be ready for anything and always be up to date and ready on new technology.

DITs should also remember that they are, indeed, Digital Imaging Technicians. Not VTR, "RED Techs", "Data Wranglers", "DAS", "DCE", "DAM", you name it. Try putting any of those [minus VTR] on your timecard and see if a check shows up and your healthcare hours get taken care of. The desire to invent new positions with new names just won't happen at the union level. The positions already exist: Loader and DIT.

Brook - you must be charging a pretty penny to afford such a vast setup?! I'd love to learn more about how you built your business, but I guess that's a trade secret (which I completely understand). I'll just have to build my own slowly... nobody here wants to pay for anything these days. Many will basically just rent more drives and cards and send them directly to the post house facility for backup and further processing if it's a medium to low budget feature.

The truth is that there is no business or trickery to it. To do my job in the way that I need to at the speed that I need to, I need the gear that I own. That gear rents for a certain price. If they can't afford to pay for it, I'll happily recommend other DITs to them. Maintaining a quality level, price and standards in the position is what keeps it professional. There are always concessions and discounts, that's just the way it works... but never big ones.

Just like a RED ONE package goes for a certain price from a particular rental house, my equipment goes for a certain price and I don't come out without my gear. Just like a rental house may give a discount to a particular job, I may give a discount as well. But are they getting my gear for $200 or $300 a day? Nope, sorry. Give them that price for one job as a favor and they'll want it for less next time. There is a standard and people have to stick to it... the same way that the film industry has stuck to it for decades and decades. Those that undercut pricing dramatically to get work are only hurting the industry, the people that work in it and devaluing themselves.

A friend of mine is a gaffer and owns a truck. He gets calls all the time from people asking him to just "help them out" because they have a feature coming up and they'll "pull him on" if he hooks it up. "We only have $1,000 for the week, but we really really need your 18Ks, a generator and the whole truck... plus these subrentals. Can you help?" If it's reasonable? Sure. If it's a grand for the week? He'll move on. Give people the 80% discount and you're the 80% discount guy. Losing a job like that is worth it, because taking a job like that means breaking your own rules.

Now we all have friends, buddy gigs, situations in which we'll help people out. I'm not a hard ass by any means [you guys know me pretty well by now...]. But I'm talking about full-scale jobs in this particular case.

Now not everybody does larger scale jobs. Lots of people don't even want to! That's cool, this advice doesn't really apply there. I guess my thoughts are pretty targeted in this regard.

If people are renting a zillion cards or drives to eliminate the on-set loading... well, there's not much to be done about that. Some people think that's the way to do it and they won't be convinced otherwise. I hope they have an awesome job... but there's no place for me on it and that's fine.

Here's a thought experiment:

Imagine a person who shows up in a room full of studio producers with a laptop and a RED. That person shoots something, pops it on the laptop and shows them. Look! It's this easy and fast! You can finish in 4K on a laptop!

Those producers decide to do their next job on RED and start shopping around. They call me and I give them my rate and equipment rental. Wait a second! At the meeting I was told I could do it on a laptop! What's different?

The real world is not the same as a tech demo. Things need to happen fast, they need to happen perfectly, they need to happen with very specific accuracy [think DI vs. REDCINE] and and they need to have a dozen backup plans. One cable is none and two cables is one. If the footage needs to be backed up and accurately color corrected right now, but there's a location move, you need to be off the generator and get into the stake bed before the 2nd 2nd loses his shit... but three hard drives are failing, this wire just got ripped apart, you're in the middle of a transfer and the monitor needs to be recalibrated... what the hell are you gonna do with that laptop on a popup table?

In the major markets [LA, NY, etc.] on large shows, I desperately hope that no DIT ever does an Epic job with a laptop. Ever.

Brook
what are you using as your RAID 6? 1gb per second.....
is this a DIY or off the shelf?

DIY, but there are lots of options out there.

I agree that some productions have been stupid paranoid into thinking that if they buy say, 50-16GB cards that you can treat it like film. This in my book defeats the purpose of having a data based system which having multiple clones of the original data.

On my next film job I'm going to rent 30 ArriCam LT shoulder mags because I don't want to have any problems. That shit's light-sensitive, you know???

On Southland I have 32 cards. I had 40 on ER. That allows me to keep yesterday's footage untouched on the cards about 6 hours into today. So if there's something I didn't catch but the post house did during the dailies process... the original media is still untouched. I think that's worth it.

 

[Tony Lorentzen]

Really?!?

Really! Sadly! :undecided:

 

[Tony Lorentzen]

Thanks for your thoughtful and very insightful reply, Brook. I completely agree that selling yourself and your gear for less than everyone else is bad for everyone's busines. I have turned down jobs on several occasions because they just didn't want to pay enough - or if theirs and my expectations just didn't match. Unfortunately - the business in my country is so small and well paid high-profile jobs are hard to come by. A typical feature here is produced for around $3-5 million US - low budget features around up to $2 mill, so they will typically say they can't afford a DIT on the shoot unless they can pay him the same as a runner/assistant (which is close to $1000 per week here) or if he can do multiple jobs; DIT, Video-assist, AC and so on. I've done a few jobs of those and they will kill you in no time.

I have people calling me on a weekly basis asking if they can borrow my RED for a 1-2 day shoot for free - or if I'll work a 12-hour-per-day job for $1000/week. It's absolutely ridiculous and it's driving me mad. That's how the business is here at the moment.

 

[Jeff Butcher]

Brook, I really appreciate you taking the time to give us this info. It's important to set the bar early on since this position seems to differ in responsibilities and expectations between different people.

 

[Vico Martin]

Give people the 80% discount and you're the 80% discount guy. Losing a job like that is worth it, because taking a job like that means breaking your own rules.

Amen!!!

 

[Marcus Vasques Osorio]

Give people the 80% discount and you're the 80% discount guy.

amen to that too, even in low budget country like Sweden where the budgets of main features is about that of low independent film in the US this rule applies...

One film this year even employed a junior with no pay to be DIT.... gues how well that went. Guess who is trying to save them afterwards.

One very important point to make is that if you pay less you get what you pay for, over here nearly all DIT create offline dailies, if this is done the wrong way what the production would end up paying in post would easily cover my costs or the costs of a better DIT

Marcus