Best Way to Store, Catalog and Find Media

[Will Luker]

We shoot footage for internal use and some B-Roll for our businesses to use. We are shooting footage on the RED and have a small production team. The challenge we're facing is the fact we've never designed the right workflow, and have been working out of a folder structure using Premiere and After Effects.

Challenges

• No consistent metadata tagging
• Can't find assets
• If we want to find footage, we have to ask somebody who knows where it is (and some don't work here anymore)
• Working with 8K RAW RED footage is taxing on the hardware
• No easy way to collaborate and get feedback on shots and cuts
• Moving files and archiving are all manual

Currently

• 2 RED cameras (Weapon & Helium)
• 120 TB of footage
• Lots of duplicates
• Adobe CC
• 45 Drives Turbo 60 XL running FreeNAS with 225 TB usable
• 10 GbE infrastructure from servers to Mac Pros

We've been told we need a MAM system to manage the tagging, ingestion and moving the data. What do you guys suggest? It seems like MAM systems are the "old school" way of dealing with footage and seem out-of-date.

How do the best solve the same issues we're facing? There has to be a better way to manage Storage, Catalogs and Search.

What tools do you use?

 

[Marc Wielage]

There are a variety of problems and solutions here. I can give you some general suggestions:

1) hire a designated hitter who's basically a post supervisor and a workflow person who understands I.T. to the point where they can either set this stuff up optimally or at least hire a consultant who is.

2) write everything down so that if the key people are unavailable or hit by a truck, everybody in the office will understand the correct methodology to label and organize files, and where and how to set metadata. (Having smart assistant editors is critical.)

3) come up with a logical system to organize material, maybe on the basis of clients and project names, and partly on shoot dates.

4) get big mofo drives that can hold everything, and come up with a reliable backup method so that everything is retrievable should loss occur. (Power backups are good, too.)

5) decide what formats to use on the basis of how and what you deliver. If you're not going to deliver 8K (and I suspect you are not), then my advice would be to transcode to 4K twice: once a compressed format with a down-the-middle look for editing, and once as a near-lossless format (like ProRes 444) for color, in RedLogFilm. You could theoretically come up with a benign inbetween format like 6K for just the media-managed takes in post provided the hardware could handle the stress and you had sufficient storage. And you could also do the initial cut in HD just for space reasons.

6) I don't think Premiere and After Effects alone will solve your workflow. You need to also have a room dedicated for the sound mix (my choice would be Pro Tools), and you'd need a room dedicated to conform and color (and I lean towards Resolve).

There are a dozen different ways to go, and a lot depends on your expectations, the amount of space you have, and your available budget. There might even be some projects of yours that are big enough that it makes more sense to hire an outside post house to deal with it, so all you have to do is just all the initial archival and offline editing. Let them do the color and sound mix and final delivery. Time and budgets are big factors here.

 

[Steve Sherrick]

Try NeoFinder. This guy has been around for 20 years improving this product. I would say it's one of the more entry level systems but might be able to get it up and running quickly. Otherwise you are looking at solutions like CatDV with worker nodes and all of that will take time and money (consultation). But also will give you tons of flexibility.

What you don't want is terrabytes of data with no easy way to search it. You will go crazy trying to find things in a crunch.

Ideally you would have everything catalogued and proxied so you could pull clips from your asset management software and begin editing right away if need be.

 

[Eric Santiago]

I don't use an enterprise system so I have RAIDs and JBOD everywhere.

I like using foolcat for the simplicity.

Wish I had a tape system maybe someday.

 

[Will Luker]

The challenge is we have almost 100 TB of unique footage with no proxies or easy way to search. I'm trying to clean the mess, get organized and find methods and tools to prevent this from happening again so we can actually search and leverage the footage

 

[Randall Brown]

Yeah you a need a DAM/MAM. CatDV is a well liked system, and Axle is gaining in popularity. These can scan your media and create low res viewable proxies and then you can begin tagging and building your metadata which is the most important but also time consuming part.


I would say start by sitting down and imagining exactly how everything would ideally work from start to finish at your facility. From project initiation to project archiving. With that blueprint in hand I would talk to an IT centric company that gets video/ production. These aren't new challenges, I just helped an internal content team at a Fortune 50 company solve this for their team. If you need some help. PM and I can recommend some folks, or help direct.

 

[Adrian Weinbrecht]

Try NeoFinder. This guy has been around for 20 years improving this product. I would say it's one of the more entry level systems but might be able to get it up and running quickly. Otherwise you are looking at solutions like CatDV with worker nodes and all of that will take time and money (consultation). But also will give you tons of flexibility.

What you don't want is terrabytes of data with no easy way to search it. You will go crazy trying to find things in a crunch.

Ideally you would have everything catalogued and proxied so you could pull clips from your asset management software and begin editing right away if need be.

We use NeoFinder and love it. We write everything to 2x Sata drives and 2x LTOs we name the pairs the same simply have an A and B version of each.

We then simply drag the whole SATA into NeoFinder and it catalogues everything on it, pointing back to the specific Sata or LTO e.g. Archive040A you can then keyword if desired. Searching is fast and easy, I think we have over 500k clips and Stills.

 

[Will Luker]

We ended up trying out CatDV and are currently running it in our environment for testing. I'm trying to finalize our Metadata fields and determine what (if any) folder structure we should be using. Any thoughts?