Welcome to our community

Be a part of something great, join today!

Register Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

I need a hard drive to work off of!

J

John Pike

Well-known member
Joined
Apr 12, 2021
Messages
53
Reaction score
2
Points
8
I need a good hard drive to work off of with my Red Komodo files. I have a Glyphy 12 TB USB 3 and that clearly doesn't cut it. In the past I've used a bunch of Samsung T5/T7 s but I don't have enough to hold all of my files while I'm editing.

My machine is a 2018 Mac Mini with an eGPU. So thunderbolt 3 helps.

What are suggestions for a good hard drive to edit from?

I'm considering going with a G-Tech Raid Drive https://www.bhphotovideo.com/c/prod...logy_0g10414_g_raid_28tb_2_bay_removable.html

or maybe going to a Qnap NAS- (https://www.bhphotovideo.com/c/product/1365502-REG/qnap_ts_453bt3_8g_us_4_bay_thundebolt_3_nas.html)

To me the advantage of the nas is the ability to easily setup to backup with Backblaze.
 
I use the G Drives with Thunderbolt 3 and they work pretty well for me as long as I have it hooked up over TB3/TB2.

I have a 2013 MAc Pro and go TB3 to TB3 with the Apple adapter and have no problems with media in FCPX. It's not perfect, but I don't really need to make proxies for basic editing and playback in FCPX. I can do 1/2 res in Red Cine X too and playback is smooth.

They seem to be the best bang for the buck I can find
 
Komodo can survive off of a single "good" SATA SSD. However, you'll have more success with these working volumes:

- SATA RAID with SSD or Spinning Disks
- NVME
- NVME RAID

In terms of bandwidth, USB 3.1 Gen2 or Thunderbolt 3/4/USB4 are ideal for external connected media before you get into very expensive setups.

Komodo is aboujt a 300 MB/s data rate, as such you should target that or over provission.

USB 3.1 Gen 2 = 10Gbps, or 1250 MB/s theoretical. In practice typically more like 800-900 MB/s.

Thunderbolt 3/4/USB4 = 40Gbps, or 5000 MB/s theoretical. In practice usually about 2000-3500 MB/s.

I say theoretical because you need to know the top speed of what's in there plus the bandwidth of your system.

i.e. A single SATA spinning disk is usually around 180-240MB/s, but but 2-8+ of them in a RAID 0 configuration through Thunderbolt 3/4/USB4 and you have something worth working off of.

SATA SSDs usually get you around 400-ish MB/s from the get go.


REDCODE RAW is alarmingly flexible in post due to how the SDK taps into debayering the footage. You can work like 1/8th or 1/4 res debayer and be highly functional then switch over the full quality for export. However, as with any of this, the question is how fast do you want or need to go and how many coins are in the piggy bank.

Personally, spinning SATA disk. Aim at TB3/TB4/USB4 4X and up configurations. You can get away with 2 disk RAID for Komodo, gets the job done.

SATA SSD, you can do a lot with a 2 disk setup, but more is more better. I have a 16X SATA SSD RAID in one box as a working volume and it sings.

You got the itch for speed, NVMe is the way to go, but at the moment it's the costliest option. My main mobile workstation (Dell) has 4X 8TB NVMe SSDs in there.


A good analogy is all of these cars drive, but some get you there faster than others.
 
Thanks for that write up. I'm also concerned about redundancy. It seems so tricky as I'll quickly get to 30TB and not have a single drive that I could copy everything to to backup. That's one attraciton I have to say the QNAP NAS since it runs Thunderbolt 3 (kind of, is somehow slightly different and basically using a thunderbolt version of 10gbe, to my limited understanding) and then also could easily backup to Backblaze.

Any thoughts on that?

I'm trying to find a good SATA SSD set up but haven't yet. I'm very tempted by the NVME SSD set up from OWC posted above.

But for simplicity and the need to get through a project over the summer I may just pick up a G-Tech Raid.
 
I build my systems with three factors in mind.

Work Volume = Drives you edit and do things off of.

Export Volume = I never write from where I read.

Backup = should always be separate media, tape, disk, cloud, whatever you choose. Double or Triple Redundancy highly recommended. These are your hero copies of material that goes onto Work Volumes.

On my bigger workstations I have a few other considerations like an Asset Volume that contains a bunch of things like graphics and regularly used bits and bobs.

Everybody has different requirements in terms of how much space and how fast they would like to work. Often tied to project size and duration.

SATA SSD, I use the Icydock internal stuff for one of my workstations. I built an Akitio RAID for one external box too. My general rule of thumb about mid/late 2010s onward has been no more spinning disks internally on any of my systems. But I do use them for external RAIDs and backup. My current shoot we are using Glyph 4TB drives USB-C, 2-4X (rolling about 4-8TB a day w/ redundant copy) depending on schedule and I have an additional backup volume with the entire shoot living with me, that's a spinning disk RAID in this case. Next leg should be between 32-42TB of material. This material goes back to the post house that gets tossed onto their filesystem. Usually shipping drives a couple times a week when possible. Last location, we just held onto the footage until wrap due to post schedule.

Tricky stuff. Every shoot we explore whatever workflow works best. My current one is an international project and this is the way. It's been an ongoing shoot for the better part of 8 years now which "phases" and yearly releases. But I use this workflow for a lot of my travel and regular work too.

Prepping for a 60 day feature right after this. Looking at the data footprint knowing I'll be filming in 8K, which is about 1TB an hour on Monstro. Likely shooting realistically 1-4 hours day.

I've made a handy tool for whipping up data calculations across projects if you are looking for something like that:
http://phfx.com/tools/framesToDataRate/

Asked RED last week for the official Komodo data rates, I'll add them immediately once I have them.
 
Phil Holland said:
I build my systems with three factors in mind.

Work Volume = Drives you edit and do things off of.

Export Volume = I never write from where I read.

Backup = should always be separate media, tape, disk, cloud, whatever you choose. Double or Triple Redundancy highly recommended. These are your hero copies of material that goes onto Work Volumes.
Click to expand...


Hi Phil, thanks again for all of your excellent info. Forgive me for not fully understanding your workflow, but I'm very interested in learning your set up. The Work Volume makes senes to me, but the Export Volume confuses me a bit. You don't write from where you read you say, so would you have copies of all the files you are working with on the Export Volume, and then say reconform to those files and then export? Or did you mean you don't write TO where you read?

I'm just one person working on personal art projects at the moment so I can keep things pretty simple but at the same time my project involves saving lots of footage. I started trying out Final Cut Pro as I enjoy the file management capabilities and simplicity for what I need to do but I'm finding that it's quite limiting in regards to using multiple disks, so I may just go back to Resolve.
 
Matthew P Jones said:
I use the G Drives with Thunderbolt 3 and they work pretty well for me as long as I have it hooked up over TB3/TB2.

I have a 2013 MAc Pro and go TB3 to TB3 with the Apple adapter and have no problems with media in FCPX. It's not perfect, but I don't really need to make proxies for basic editing and playback in FCPX. I can do 1/2 res in Red Cine X too and playback is smooth.

They seem to be the best bang for the buck I can find
Click to expand...

I have this same set up...except am using the G-Shuttles (I have four of them) and the read/write speed is sufficient that I have no problems. https://www.bhphotovideo.com/c/prod...0g10819_1_g_speed_shuttle_xl_thunderbolt.html
 
John Pike said:
I need a good hard drive to work off of with my Red Komodo files. I have a Glyphy 12 TB USB 3 and that clearly doesn't cut it. In the past I've used a bunch of Samsung T5/T7 s but I don't have enough to hold all of my files while I'm editing.
Click to expand...
We use a whole bunch of G-Speed Shuttle RAIDs at the office. We go with spinning drives for big, long-term storage (about 100TB each) and then use a 16TB SSD RAID to store current projects. The regular drives hit 1000MB/s in Resolve with TB2 or TB3, and the SSDs will easily go over 2000MB/s, which is more than fast enough. We media-manage projects so just the necessary material is on the SSD, and it fits with no problem.

For editing, what we recommend to clients is they cut with low-res proxies -- maybe ProRes 422LT -- and then we link back to the original 4K/6K/8K original files for final color and delivery. I don't think it's necessary to cut with camera originals when it's just the decision-making process.
 
I ended up picking up the OWC Thunderbolt 3 NVMW. For now I went with 4 1tb Crucial SSDs. I'm going to have to manage my projects well and develop a good workflow. I'm going to start a disciplined approach to my import by first deleting unwanted clips and then using R3D trim on those I want to keep before placing them on my work disk. I'm mostly shooting basically landscape images right now for a project and rarely need more than 30s to 1 min of each clip so I can save a lot by trimming.

After setting up the drive yesterday it really soars in Resolve, FCP and RED CineX. Got speeds around 2500 read/write in BlackMagic Speed test. Set it up in Raid 0.

I'll likely probably pickup another two or four disk spinning disk raid for more storage as my 12TB USB3 Glyph is basically worthless for anything other than an archive drive, can't really watch footage at all off of it.
 
Marc Wielage said:
For editing, what we recommend to clients is they cut with low-res proxies -- maybe ProRes 422LT -- and then we link back to the original 4K/6K/8K original files for final color and delivery. I don't think it's necessary to cut with camera originals when it's just the decision-making process.
Click to expand...

This is exactly how we work Marc. We shoot 6K/5K/4K but edit with (or give clients) graded 422 Proxies. Final grade relinks to masters easily with EDL.
 
Been working with the OWC ENVOY PRO FX 2TB via TB3 and USB3.
Super fast and solid.
Small as a REDMAG :)
Speeds via TB3 on 2019 Mac Pro are 1700write and 2700read.
I also have the Apple TB 2 converter for anything lower but haven't had a chance to test.
 
Nick Morrison said:
This is exactly how we work Marc. We shoot 6K/5K/4K but edit with (or give clients) graded 422 Proxies. Final grade relinks to masters easily with EDL.
Click to expand...
Yeah, that's a proven process that works well. We generally recommend to client that the temp color offline daiiies copies have "spoiler" vis timecode that identifies the specific takes and project name and date, which will help later on when the final conform is being done:

cJC5qcA.jpg
 
You must log in or register to reply here.
Back
Top