Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

The Art of Compression and Encoding

Steve Sherrick

Steve Sherrick

Well-known member
Joined
Dec 28, 2006
Messages
7,275
Reaction score
54
Points
48
Location
Boston
It's no secret that the web has become a major platform for media distribution at all levels, from sharing home movies on Youtube all the way up the chain to major television shows and feature films. In our little world of media production, one of the things we like to do is share things we've worked on, whether it's a camera test or a finished piece of content. Quite often we struggle to maintain quality, yet keep file sizes manageable. We deal with gamma issues, motion artifacts, banding, etc. I thought maybe we could use this thread to share tips and techniques for getting the most out of our footage when we need to share our video online where it is quite often the wild, wild west. I've reserved some spots here for being able to segment into categories if need be. For Example, so far I have Youtube, Vimeo, Blu Ray, DVD, and Windows Media. As people post tips and techniques, we'll evaluate and then move to these segmented categories so the info is easily accessible when people find the thread.

I know guys at RED have been very interested in this as well with their adventure with Tattoo, so perhaps they will share anything they have learned as well. I'm doing my own barrage of tests right now with MPEGStreamClip and X264 (latest version) to see how various settings affect things. Going to be a long process but I will make this testing available at some point.

So, let's have at it. And please, I do ask that we stay on topic. A lot of this is going to be subjective. So, I would suggest we keep it on point as much as possible and try not to let things get off track. Having links to samples, and providing as much detail as possible on how you arrive at settings will go a long way to keeping this on track.
 
Encoding for the Web
(Tips from Tom Wong)

quicktime h264
whatever your master file is, bring it into premiere cs5 (not straight to media encoder) and export it out of the timeline with whatever quicktime settings. for full res full quality, i never render with max bit depth, but max rendering quality. max bit depth expands the contrast. and for some reason dropping directly into media encoder I've noticed slight differences. but by doing this, i noticed a 0 gamma shift at all, and the colors stay as true as they can getting crammed back into a 8 bit container. from that point on

Vimeo
i just upload that highest quality h264 I made. and let vimeo do the job, it produces great results. Also, utilize the Show 1080P in Vimeo.

youtube
you can upload pro res, even up to 444 directly to it, and let youtube do the rest of the work.

basically my mentality, is the Hollywood or even the RED mentality, always start at the highest quality first. so if vimeo only takes h264, i put in the highest quality possible. the whole "you have to throttle the bandwidth before upload" is bull. you just shove the biggest, lowest compressed file you can that's compatible and let their compression do the rest of the work, any lower and the double compression screws you.


 
BLU-RAY, DVD, RED RAY, etc
(Tips from Tom Wong)
Blu-Ray
Adobe Media Encoder all the way. if the piece is short enough, getting the min to max bit rate from 30-35 gives really good results. i use h264 blu ray no mpeg, as it saves on file size and there's no visual difference.

DVDs
I use Apple Compressor, i actually use the stock "cram" presets(http://compressorpack.com/index.php). it's pre tweaked and cheap, and has given me the best looking DVDs. i don't use Adobe Media Encoder for that because the aspect always comes out weird, i always get black bars on the left and right no matter what.
 
How about we start with a discussion and publicly posted Epic RC Compression from 1:3 up to 1:18 in a test environment. Would love to see where the point of diminishing returns is at acquisition since the cost of SSD are very high and if double the data rate gets nothing visible then it is a waste to record and manage. Lots of anecdotal posts about the subject - have not seen any RC science posted yet. It seems that a conversation about compression should start at the beginning.
 
Fair enough Josh. I wasn't originally thinking about a Redcode discussion, but I'd be willing to do this test if enough people are interested. The key is to capture a highly detailed shot that also has some movement and some high contrast. Trees tend to be a good candidate. Also worth testing is greenscreen. I can devote a section to this. Thoughts?
 
I fully support this thread. I've often wondered how people get such good SD/DVD encodes from HD footage. It's crazy how well that stuff holds up compared to using standard compressor/Adobe Media Encoder presets...

Hell, a better example, how are apple trailers as good as they look when they're a measly 8.5mbit/s or whatever?
 
My secret sauce has been Adobe Media Encoder. It really churns out some amazing stuff when you're trying to encode for the web or DVD - and although I've used it for Blu-Rays before, they've not been in a very stressful environment (i.e. all I had to do was fit a 30 minute short on a Blu-Ray - at that point you can just max everything out.)

What's incredible about AME is that you can go straight from a Premiere project and do encodes from the project and you don't have t lock up Premiere. I also simply export a QT reference from either FCP or Avid and feed it to AME and it churns out great results as well.

YouTube compression is a bit more complex than Vimeo - because YouTube makes a habit of playing 1080p video, while vimeo limits you to 720p unless you're a "pro" subscriber. Most people's computers can't even handle 1080p, though, so it's debatable weather it's even a good idea to go to 1080p. If you're super picky on quality, uploading a 720p master to Vimeo and 1080p master to YouTube will give you less resizing issues.

My solution for encoding gamma issues has been to simply encode on a PC. It staves off Gamma issues and (again) by using AME on a PC, I'm able to fully utilize 64 bit and multiple, hyper-threaded cores for rendering as fast as possible. There was a Cow article recently about this: http://magazine.creativecow.net/article/opening-my-mind-has-opened-doors.

I don't claim to know how YT and Vimeo's encoding works, but I have found that if I have rock-solid h.264 encoding it will translate to no discernible difference when re-encoded into YouTube or Vimeo. They use h.264, so perhaps it's just a pass-though situation. When using Adobe Media Encoder, it's important to use the h.264 setting, and NOT the QuickTime-wrapped h.264 setting. This produces an M4V, but it's still the same thing - readable by the same programs, same encoding, same data - but the AME options are MUCh more robust for the h.264 encoder.
 
My 2 cents: for DVDs on a Mac I find that iDVD works better than Toast. I prefer Vimeo to Youtube, don't know why really. For that I use MpegStreamClip, H264, multiple passes (takes more time but it's wirth it), limiting compression to 2000 Kbps for 24 mins. docs, looks good on small size screen for clients, not good if blown to full screen, plus, you can run it with limited buffering times.
 
It's funny you mention this Steve because I just watched Contagion on our Apple TV in HD and sure enough, banding. Lots of it. Made me feel good knowing I wasn't the only one with the issue. I had a scene in a project a couple years ago that was problematic because it was so clean and loads of banding because of it. So I had to dirty it up to make it go away. A while back I purchased some pre-packaged settings for Compressor that really work well. All of the hunting around for the proper settings was driving me nuts. I'm still not satisfied with the size of some of the files, I suppose it's a trade off. But, it does seem like there are those among us (Michael Cioni perhaps) that have the secret recipe.
 
Great! Look forward to seeing a whole lot of X264 in there. MPEG Stream Clip is too limited however, as are Adobe Media Encoder and Compressor. My favoured GUI for X264 is MeGUI. I have always relied on Debugmode FrameServer, which works on ever major Windows based NLE and MeGUI as the X264 frontend. Or X264VFW to get X264 directly out of the NLE or finishing app (notable exception - Resolve beta, hopefully they will integrate VFW) - that's a very effective solution. My favoured preset in X264 is "Very Slow", with some tweaks based on the source material. Banding is a problem with super compressed material, but here's where MeGUI's AVISynth serving shines through. As part of the encode, you can run a Gradfun2DBmod pass, which works like magic - eliminating all banding. Of course, there's a solution to every problem in AVISynth filters.

X264 is also by far the best option for Bluray, with the --bluray-compat string.

As for DVD, CinemaCraft is the best encoder out there. After that comes HCEnc, which is free. Many frontends utilize it.

PS: MSU has a detailed comparison of various H.264 encoders. (http://compression.ru/video/codec_comparison/h264_2011/). No surprises - X264 is the only option. The gap is even larger as we crank up resolution - upto 4K.
 
Fantastic idea. If I had my EPIC I would definitely help. Both redcode and compression tests are essential parts of the whole red content production. Can't thank you enough, Steve. If other redusers could help on this would be excellent. A maximum is always "less compression the better" , but how much of this is visible for broadcast/web?

May I suggest a test of conversion in redcine x pro of various red code flavours to Prores 4:4:4 and then a quick push on the levels to search for different types of compression artifacts?

Also, different frame rates and compression (this more for the epic) to see the acceptable compression for the faster frame rates?

Once again, thanks to Steve and all those taking the time for this. Essential read, sticky level post!
 
Wish we had a full-time compressionist way in on their method.

I've recently been using Adobe Media Encoder with the mainconcept h.264 codec and have received great results without the gamma issue.
 
Assuming X264 as the encoder, the best compromise between quality and compression is always around CRF 18. For web stuff, even upto CRF 21 is excellent quality and "visually lossless" to majority audience except for tough scenes. CRF targets quality rather than bitrate and is definitely the best method. Based on the presets and settings used, the bitrate/file size will reduce for identical quality, at the expense of longer encode times. At CRF 21 with Very Slow preset, it's no stretch to see excellent 1080P video at as little as 4 Mb/s or 0.5 MB/s. This can be further tweaked with the nature of the footage. E.g. still, long takes require a different approach to aggressively cut, chaotic motion. Or for black and white material, using 4:0:0 effectively cuts your file size by half with zero loss in quality.

I recently edited a short project shot at 60 fps - works absolutely fine with X264 and level 4.2. At level 5 the max frame rate is 72.3 fps at upto 2K. 4K is restricted to 30 fps so far (level 5.1).

As an aside, X264 is extremely flexible and can be used as a codec for almost every scenario. Lossless, 4K, 10-bit, 4:4:4, intraframe, you name it - it can be achieved with X264.
 
Subhadip Sen said:
Assuming X264 as the encoder, the best compromise between quality and compression is always around CRF 18. For web stuff, even upto CRF 21 is excellent quality and "visually lossless" to majority audience except for tough scenes. CRF targets quality rather than bitrate and is definitely the best method. Based on the presets and settings used, the bitrate/file size will reduce for identical quality, at the expense of longer encode times. At CRF 21 with Very Slow preset, it's no stretch to see excellent 1080P video at as little as 2 Mb/s or 0.25 MB/s. This can be further tweaked with the nature of the footage. E.g. still, long takes require a different approach to aggressively cut, chaotic motion. Or for black and white material, using 4:0:0 effectively cuts your file size by half with zero loss in quality.

I recently edited a short project shot at 60 fps - works absolutely fine with X264 and level 4.2. At level 5 the max frame rate is 72.3 fps at upto 2K. 4K is restricted to 30 fps so far (level 5.1).
Click to expand...

Noobie question: is X264 the new H264? Also what is CRF?
 
You must log in or register to reply here.
Back
Top