Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

recommendations to production for storage/shuttle drive

Matthew Love

Matthew Love

Well-known member
Joined
Dec 22, 2010
Messages
292
Reaction score
0
Points
0
Age
42
Location
Los Angeles, CA
hey guys, quick question.

I'm helping a pal plan out a 4 week shoot (20 days total) on 2 epics. Probably shooting 8:1 a majority and 5:1 on some keying shots (far and few).

I told him if he goes with 2 4TB RAIDs that should get him through the first week and they can plan accordingly from there. Money is an issue so I dont think going with large 12TB master and backup systems would work.

I told him that 2 4TB RAIDs (master and backup) would be sufficient for a week to 1.5 weeks possibly 2 weeks and he can use a 1TB drive as the shuttle drive for the Prores422 HQ transcodes.

Just wondering what you guys on here would recommend.

thanks!
 
Well 4TB for a week seems too little to me - even if you have a conservative director and DP, you'll constantly be looking at the remaining free disk space... The last couple movies I DIT-ed on constantly shot anywhere between 800GB and 2TB. Also, you have to consider that the read/write speed will suffer if that RAID fills up. So I would definitely recommend having more than 4TB per backup per week. Can it be done with 4TB? Sure, but the production will somewhat restrict itself. The director deciding whether or not to do another take based on remaining disk space is not a good thing...

If money is a big issue, you can always just get a couple bare drives and a docking station and do multiple backups manually.
 
I would get on newegg and order a bunch of 7200rpm 2-3TB bare drives and a portable drive chassis for each. Something like a rosewill for $50 that has your fastest connection port (usb3, esata, etc). You'll have to install the drives yourself, but that's easy and cheap. Then just make sure you have two copies of everything and plenty of labels. Or the docking station Tobias suggested is good too.

For workflow, experiment with copying on your system. You could start a copy from the media to one drive. Then start another to another drive. On windows I have seen this pull the second copy from the cache so it does not saturate the pipe to the media drive with two streams (ie. the transfer rate of the first copy stays the same). This gets you two copies quickly so you can recycle the card.

On our last feature we were doing around 250GB per day per camera of 5k FF.

Cory
 
I used Glyph Drives with eSATA and I am quite happy with the performance. 4Tb gives you approximately 23hrs at 8:1 which is about 12 hrs/ camera. That should definitely be good for Week 1 including some sequences at 5:1.
 
For the cheapest price ratio: I would buy a 12TB drive to have the whole film in R3D on one disk (easy for the conform process) and the backup on separate 3TB dokable disks.
12TB could hold the hole shooting if you shoot max 500GB/day.

http://www.g-technology.com/products/g-speed-q.cfm

Pat

PS: does anyone have a Redcode compression GB/minutes calculator?
 
Doesn't sound like near enough storage...and you don't want to shoot say over 75% of your hard drive....i try to stick with 70% to 75% max of my HDD space, so a 4TB drive called FULL to me is at 3tb. Otherwise, i've had complications with slow media. And, your backup....but wow, calling it a 4TB per week sounds like very little shooting.
 
I'm likely to be going down the 'copying to dual bare drives via a docking station ' (one master and one backup) route... Of course I'll be using 7200rpm drives, but does anyone have any perspective on the transfer speeds? I'll be connectivng via USB3 and was wondering if the speed of the hard drives themselves will likely bottleneck my transfer rates
 
Sulekh Suman said:
I used Glyph Drives with eSATA and I am quite happy with the performance. 4Tb gives you approximately 23hrs at 8:1 which is about 12 hrs/ camera. That should definitely be good for Week 1 including some sequences at 5:1.
Click to expand...


Er... That's some seriously low output specifications there. We know nothing about their shooting schedule, subject matter, etc.. If I were going into a shoot not knowing how much I might actually shoot each day, I would do 2TB (maybe 3TB) per camera per day. And I would double that up for redundancy.

We're stabbing in the dark here anyway, we know nothing about the production, nothing about other hardware they want to use and connect their storage to, nothing... For productions looking to offload to physical hard drives for storage, I highly recommend using a 2.5" RED Station module (or two) and using 1TB 2.5" 7200rpm HDDs for the storage. If going with 3TB HDDs, the WD Caviar Green are an excellent value and relatively quick for variable spindle, economy drives. If bare drives aren't desirable for shuttling about, then there are lots of options for throwing one of those drives in an enclosure from Rosewill, Bytecc, and many others to give it a host of common interfaces on one unit. Although, it's often cheaper to just buy an external HDD to begin with.

For RAIDs, you're starting to get expensive and that may or may not be what the production needs. You need higher calibre HDDs to provide the right performance and reliability in a RAID setting -- no more Caviar Green or Blue drives, but you really need the Blacks (or Seagate Barracuda or Hitachi UltraStar). Preferably the RAID-specific models, to do it right. I think their primary offload and footage manipulation / transfer station(s) should have a decent RAID to work with. Otherwise the data bottleneck could kill the production if they are shooting any respectable amount of footage, doubly so with the two cameras. Actual storage, shuttle drives, backups, etc.. don't need to be RAID.

Single HDDs are going to bottleneck all transfers to around 65~95 MB/s depending on various factors, but probably an average of about 75~80MB/s. Doesn't matter if USB3 or eSATA are used. Mechanical HDDs only go so fast and one by itself is not "fast".
 
Having just finished three weeks of data wrangling for CampRED, I totally concur with Mr. Kilgroe's numbers.

One thing I noticed, that no one seems to be talking about, are ethernet Gig/E transfer rates.

While I was averaging about 86MB/s copying to single drives via USB 3, I was averaging over 100MB/s when transferring data over the Ethernet port. Simultaneously.

This really came in handy when turning around 250+ Gigs of footage coming coming in at noontime from the morning shoots so that it was ready to edit on the 15 Z-1 workstations after lunch.

I did not have a chance to gauge the throughput of bridged network connections as the Z-1s only have a single ethernet port, and the network switch couldn't handle bridging from the Z800 towers I was using.

I think it would be interesting to see what numbers we can get when binding two or more ethernet adapters, or using a 10Gig/E port to a similarly equipped server or NAS device.
 
thanks guys. Unfortunately this production doesn't really understand the gravity of shooting 5k.. i've gone over it with them quite a bit but I feel like it is what it is.. i just know that it depends on the amount of material being shot and that I just dont know.. I've had days where I shot 250gb on an EPIC and days where I've shot 750-1TB a day..

i guess the first week will tell.
 
Matthew Love said:
thanks guys. Unfortunately this production doesn't really understand the gravity of shooting 5k.. i've gone over it with them quite a bit but I feel like it is what it is.. i just know that it depends on the amount of material being shot and that I just dont know.. I've had days where I shot 250gb on an EPIC and days where I've shot 750-1TB a day..

i guess the first week will tell.
Click to expand...

Then at this point, the best you can do is say "Look, we could be shooting up to 2TB a day. If that happens, you will need more money for hard drives." That way, if it happens, then at least they were warned and you've done all they've allowed you to do.

That being said, it truly does depend on the shooting style. 2 Epics makes me think that the cameras are going to be rolling a lot each. I DIT'd a broadcast shoot that was timelapse-ish and used less than 80GB of Red Raw for the whole commercial. The next day, same director, same camera, same ad agency, different product-line, something like 500GB of raw (4k QuadHD at 120fps alllllll day). With transcodes and a few other odds and ends, it was 1.2TB for day 2, versus 120GB for day 1.
 
You must log in or register to reply here.
Back
Top