Ask Mike Most Anything

[danibam]

LTO backup .... BRU ??

LTO backup .... BRU ??

Recently ran into a problem in a project that backed up all data on LTO-4 tapes using BRU on Mac.

The tapes were sent abroad for restore and it was assumed they would be able to restore them there,
but their LTO drive was attached to a Windows system and apparently there is no BRU for Windows.

I guess my question is if going BRU was a wise decision ... and what is the Industry standard in this
regard. What software should be used for creating LTO back up?

This particular project was backed up to LTO-4 tapes using an LTO-5 drive. LTO-4 tapes were used
because that is what the client abroad could read. When discussing how data was going to be handled
all that was discussed was the type of LTO media to be used ... not the software used to make such
backup. I've now learned that a BRU backup can only be restored using BRU software, which only runs
on Linux or Mac. Is that the case? Any universal LTO format available or widely usedin our industry?

Daniel Perez

 

[Marc Wielage]

I'd also add that if you were shooting a TV series, with a ton of footage being generated, and the mandate was to make the female stars look as good as possible, saving any diffusion for the end of post wouldn't fly if you are going to be judged mainly by how it looks on the HD monitors on set and how it looks in dailies. By the time post is finished and the series goes to air, the DP may be long gone if he didn't make the female leads look good.

Very good point. Though I know a major, major 1-hour show that routinely shoots with only a little bit of diffusion during production, then does a ton of defocus on all the female actresses later on. Really, we mostly do it on the close-ups. (And I can think of one show still on the air where they do about a 7% vertical stretch in order to make the star look thinner. I refer to this as "the 7% solution," when dieting doesn't work.)

I agree that it makes sense to go for less filtration during photography, and then adding more later in post. At least this way, we can reduce it on wide shots or just confine it to skin tones. I've seen some really spectacular filters (like Tiffen Glimmer Glass) that does stuff we can't exactly duplicate in digital, so this makes a good case where an organic approach works better. And the advantage of that is, this is definitely a "what you see is what you get" approach.

How, witin the traditional telecine controls offered in systems like mystica/speedgrade (specifaclly those that do not have curves), is one suposed get fast and steep toe and shoulder compression that allows one to produce a really contrasty picture that keeps as much shadow and highligt details as possible?

I haven't used Speedgrade, but my gut feeling would be to maybe try pulling a highlight key and then exclude those details in order to save them from being compressed. This is one of those things that's easier to demonstrate than to explain.

Otherwise, just tell them to crush the mids as much as possible while trying to retain the whites and blacks. I bet there's a way to get close to this look with traditional controls. But in this specific case, no question, custom curves and/or an adjusted LUT would help make this process faster.

 

[Marc Wielage]

Recently ran into a problem in a project that backed up all data on LTO-4 tapes using BRU on Mac. The tapes were sent abroad for restore and it was assumed they would be able to restore them there, but their LTO drive was attached to a Windows system and apparently there is no BRU for Windows.

They can't just go out and rent a Mac for a couple of days?

I know of cases where LTO's have gone bad in less than 10 years. But it's still the most reliable backup format in the VFX industry (after 35mm negative, that is). I recently had a case where some DTF-2 backups couldn't be used, mainly because of the difficulty in locating working DTF drives. So it's not just the backup format or the operating system -- it's also the choice of drives.

Worst case, you could restore the LTOs to FAT32 hard drives, then send the hard drives to the company. Those will mount on all current versions of Mac OSX, Windows, and Linux, assuming normal DPX or R3D files.

 

[M Most]

Recently ran into a problem in a project that backed up all data on LTO-4 tapes using BRU on Mac.

The tapes were sent abroad for restore and it was assumed they would be able to restore them there,
but their LTO drive was attached to a Windows system and apparently there is no BRU for Windows.

I guess my question is if going BRU was a wise decision ... and what is the Industry standard in this
regard. What software should be used for creating LTO back up?

This particular project was backed up to LTO-4 tapes using an LTO-5 drive. LTO-4 tapes were used
because that is what the client abroad could read. When discussing how data was going to be handled
all that was discussed was the type of LTO media to be used ... not the software used to make such
backup. I've now learned that a BRU backup can only be restored using BRU software, which only runs
on Linux or Mac. Is that the case? Any universal LTO format available or widely usedin our industry?

Unfortunately, there is no universal standard, and this fact is very widely misunderstood. The most common format is "standard" Unix tar format, which has the advantage of being both widely deployed and open, but the disadvantage of not being very flexible. Going forward, LTFS is quickly achieving wider use. It has the distinct advantages of being open and flexible, easily appended, and effectively random access, allowing individual file saving and restoring, and interactivity through standard operating system file tools. My personal belief (and hope) is that LTFS emerges as the consensus standard. BRU is popular in the Mac community for a number of good reasons, but the primary one is that it is tailored to OS X and being a commercial product, tends to simplify the process of backup and restore. But being commercial, it is not open. The BRU people are wisely adopting LTFS as an option, so it remains a good option for OS X users going forward.

 

[M Most]

Otherwise, just tell them to crush the mids as much as possible while trying to retain the whites and blacks. I bet there's a way to get close to this look with traditional controls. But in this specific case, no question, custom curves and/or an adjusted LUT would help make this process faster.

IMHO, crushing the mids is a sure ticket to a lack of detail and definition and a very unnatural image, especially with digitally originated material. You're better off leaving the mids alone and crushing blacks but with proper color balance. This will often yield more "apparent" detail because the lower midrange will be preserved.

Once again, only my opinion.

 

[Marc Wielage]

IMHO, crushing the mids is a sure ticket to a lack of detail and definition and a very unnatural image, especially with digitally originated material. You're better off leaving the mids alone and crushing blacks but with proper color balance. This will often yield more "apparent" detail because the lower midrange will be preserved.

It depends on a lot of factors. I'd have to see the actual material and know what I was dealing with to say for sure. I'm not sure there are any universal truths here, because of all the variables. There's also a difference between crushing the mids somewhat, vs. crushing them to death.

This would definitely be a case where log and linear adjustments might yield different results -- even both used together. But I have zero experience with Speedgrade, and don't know how far you can push it before things get ugly.

 

[Michael Lindsay]

I don't have any real experience with Mistika, so I can't offer any specific suggestions on that. Some programs - Scratch comes to mind - have S-curve controls that offer a continuously variable S-curve, which would to a great degree accomplish what you're looking for.

Yes that would work if Mistica has this control.. Also the other option is a low soft and high soft setting (as davinci has in the soft clip tab). This would allow us to hard push but not clip. Even though not as powerful as a curves this would probably work. We def need compression at both the top and bottom. Hopefully we'll find a way. Powerwindows are more tricky in 3d and less suitable for the the material I have...

thank you for your response

Michael

PS anyone know where I can download the Mistica manaul?? thank you!

 

[Marc Wielage]

Yes that would work if Mistica has this control.. Also the other option is a low soft and high soft setting (as davinci has in the soft clip tab). ...PS anyone know where I can download the Mistica manaul?? thank you!

The manual doesn't appear to be on their website (www.sgomistika.com). You may want to contact the UK head of sales, Geoff Mills, at gmills@sgo.es. I suspect because of Mistika's open architecture, there may be a plug-in that could get the look you want. I also get the impression that because this is such a new product, they probably have the ability to add features relatively quickly. Custom curves is a pretty basic feature that's been around for over a decade (more like 15 years), so I bet they could work this out.

Mistika is a very interesting company, and it'd definitely be one I'd consider if I was starting a big facility from the ground up. (I almost said "if I was starting from scratch," but that would be confusing.)

 

[Evangelos Achillopoulos]

Michael I can provide you with LUT's for Mistika... if you want it...

Also about Film Log and Linear REC709 debate I have made a thread explaining my position:

http://reduser.net/forum/showthread...of-CC-workflow%92s-Film-Log-vs.-Linear-REC709

 

[Michael Lindsay]

Michael I can provide you with LUT's for Mistika... if you want it...

[/url]

Hi Evangelos

Thanks for the offer... We got through the session. I may take you up n your offer next week when I go back in for some tweaks.

kind regards

Michael

 

[Michael Olsen]

Mike,

I've recently been reading some of your musing here and elsewhere and just wanted to ask about how you view the seemingly inevitable approach of compressed and uncompressed RAW footage. It seems that you are a pretty big supporter of the compressed-RAW approach championed by RED, but I wondered if I could get you to expound upon the instances in which you find that compressed RAW or even mathematically lossless compressed RAW is notably inferior to 'True' RAW. It seems that there are quite a lot of people who are saying "Finally! A 'True' RAW camera! So much better than all that compressed-RAW nonsense..."

to which I can only wonder "...is it?"

 

[M Most]

Mike,

It seems that there are quite a lot of people who are saying "Finally! A 'True' RAW camera! So much better than all that compressed-RAW nonsense..."

to which I can only wonder "...is it?"

As far as I can see, no, it isn't.

I really do feel that given the state of the art in modern data compression techniques, the fact that the original images are of a much higher resolution than any digital images we've dealt with previously (at least in terms of motion pictures), and the experience of having seen Red's compressed RAW images for almost 4 years now, that those who are still citing the supposed superiority of uncompressed RAW are just not seeing reality. The only real advantage of uncompressed data is that processing it is sometimes faster, despite the larger size, because no decompression step is required. However, with modern GPUs and other acceleration techniques, that is much less of an issue than it used to be. And while both Arri and
Sony are currently showing only uncompressed RAW solutions, my feeling is that this will change, in Sony's case, perhaps prior to the official product launch in January. Not a prediction, just a belief.

 

[Michael Olsen]

Thanks, Mike. That puts a lid on it for me.

 

[danibam]

Thunderbolt

Thunderbolt

Hi Mike,

Trying to keep up with Thunderbolt, and somehow disappointed as there are no desktop workstation
Thunderbolt solutions yet (unless iMac is considered a desktop workstation) I read somewhere that,
essentially being a PCI-e bus extension, there would be no add-on cards available ever for today's PCI-e
computers. That the nature of the technology required that Thunderbolt controller chips are part of the
motherboard for fast enough access to the bus, meaning no PCI-e Thunderbolt cards would ever exist.
Is that true?

I remember you were one of the first to talk about Thunderbolt in your blog. How do you feel about it now?

What about Windows or Linux based Thunderbolt solutions? I think I remember reading that SONY will be
releasing Thunderbolt solutions (maybe with a different name).

 

[Tom Lowe]

Mike, I am getting ready to cut a 45-min picture, with all Red 3K and 5K material, plus 5.6K RAW timelapse material from Canon 5D2s. Current plan is to build a super-duper dual-Xenon PC with 96GB DDR3 RAM, etc, and cut the picture using Adobe PP and AE.

I am planning to get a large, fast 16TB+ RAID 5 array for the main footage, but my question is, how can I efficiently and affordably back up this 8- to 10TB of actual footage data? I would like to back the main RAID 5 array up every couple of days, or at least once a week, then carry the backup offsite, to protect it against fire, theft, natural disaster, flooding, etc. The LTO storage tapes I am looking at seem kind of small? 1.5TB? What would you suggest for backing up this RAID 5 periodically? Maybe just a cheaper, slower RAID 5 to mirror the main one?

 

[M Most]

I remember you were one of the first to talk about Thunderbolt in your blog. How do you feel about it now?

Pretty much the same. I think it's a very good technology and over time you'll see many products to exploit it. I don't think it obsoletes things like Fibre Channel, and it doesn't really allow for SAN implementations easily. But for what it is - which is essentially a replacement for device interconnects like Firewire, video interfaces such as DVI, and bus expansion implementations - it's got a lot of potential, particularly once it's available on things other than Macs.

What about Windows or Linux based Thunderbolt solutions? I think I remember reading that SONY will be
releasing Thunderbolt solutions (maybe with a different name).

All of those things are on the way.

 

[M Most]

Mike, I am getting ready to cut a 45-min picture, with all Red 3K and 5K material, plus 5.6K RAW timelapse material from Canon 5D2s. Current plan is to build a super-duper dual-Xenon PC with 96GB DDR3 RAM, etc, and cut the picture using Adobe PP and AE.

I am planning to get a large, fast 16TB+ RAID 5 array for the main footage, but my question is, how can I efficiently and affordably back up this 8- to 10TB of actual footage data? I would like to back the main RAID 5 array up every couple of days, or at least once a week, then carry the backup offsite, to protect it against fire, theft, natural disaster, flooding, etc. The LTO storage tapes I am looking at seem kind of small? 1.5TB? What would you suggest for backing up this RAID 5 periodically? Maybe just a cheaper, slower RAID 5 to mirror the main one?

The current generation of LTO is LTO5, which has a capacity limit of 1.5TB. Only on RedUser would someone say that seems "small." :laugh:

Committing to spinning drives as a sole solution - even if it's multiple groups of spinning drives - is, to me, a questionable practice. While the reliability of drives is often higher than it's made out to be, physical drives, by definition, have physical limitations and are not impervious to damage by things like physical jarring and even dust buildup. A second raid setup is good because it allows for fast recovery, but for longer term on the shelf storage, tape is a better technology today. I'm assuming that the backups you're talking about will be living where editorial is happening, or nearby, and thus we're not talking about on-location backup operations. That being the case, in my experience LTO5 is a very reliable technology that offers some insurance against a single technology backup strategy. Your backups can be automated and don't necessarily have to be done all at once or immediately, particularly if you have a second Raid setup to use as a buffer.

 

[stu aull]

Hi Mike - just stumbled upon - and read - this entire thread. And realize I am wa-a-a-a-a-y out of my depth here, coming from a FCP/Beta SP edit/novice color-correction (read: self-taught) background. But now jumping into the deep end with a RED and wonder if you can point me to resources/tutorials/sites that can give me some A-B-Cs on all this REDCODE, LUT, REDLOGFilm,etc. that can give me some fundamentals about color correction/management for an Indy Shooter/editor. That way I can promise not to harass you with nubie Qs and cause a lot of eye-rolling for others reading here...

I sense there is some amazing information being imparted here. I just need a translator to grok it!

Stu Aull
Alaska

 

[M Most]

Hi Mike - just stumbled upon - and read - this entire thread. And realize I am wa-a-a-a-a-y out of my depth here, coming from a FCP/Beta SP edit/novice color-correction (read: self-taught) background. But now jumping into the deep end with a RED and wonder if you can point me to resources/tutorials/sites that can give me some A-B-Cs on all this REDCODE, LUT, REDLOGFilm,etc. that can give me some fundamentals about color correction/management for an Indy Shooter/editor. That way I can promise not to harass you with nubie Qs and cause a lot of eye-rolling for others reading here...

I sense there is some amazing information being imparted here. I just need a translator to grok it!

Normally, I would say to spend some time either in a post facility or with someone with experience and learn directly from them, but since you're in Alaska, it's doubtful you'll find a lot of local resources for that. So I guess the best advice I can give is to look at things like FXPHd, look at some of the videos on the Red website, or possibly come down to L.A. for either a Reducation class or just to spend some time making the rounds and talking to people here who deal with these things regularly. Short of learning from those who already know I don't have a lot of other advice. One of the mistakes I see a lot of people making - and this includes a fair number of people who are taking on DIT roles - is trying to self educate. For many things, that's entirely practical and sensible, especially if the subject is rather cut and dried and has little room for personal interpretation. That is not the case in this industry. There are different approaches to everything - even though there are some basic absolutes - and everyone chooses their approach based on what they're doing and what works best for them. So many of us who have been around for a while have learned through experience and contact with others. That's the best way to learn something that has a lot of sides to it. But basic information, like that available in the classes I mentioned, always helps.

I wish I could give you a better answer. Perhaps others here can.

 

[stu aull]

Thanks Mike - yep, living in The Last Frontier (tm) does have its pluses but this isn't one of them.
I guess rather than trying to take on the Whole World of CC I need to start with the idiosyncrasies of RED to start, tho I realize there is an overlap. I seem to observe there are some RED-specific pipelines/roadmaps that must be followed just to get decent images OUT to the Whole World of CC. Perhaps what I have asked here is for a Race Car Driver to teach me how to Drive, maybe I need the Driving Instructor first, check in with folks like you (or at least hang around the pits) once I can steer/brake/turn....

With Scarlet seemingly about to step out shortly, the number of REDRAW newbies like myself is going to explode - all seeking knowledge in your Black Arts. Hopefully there will be a roadmap to follow. Been shooting for 25 yrs - still a few things to learn (thank goodness).

My thanks

Stu Aull
Alaska