Apple + RED

[gabrielB]

It's unclear to me a well if RED's patent covers all forms of compressed including mathematically lossless raw. The latter to me falls under a certain kind of gray area and perhaps that is why we've seen companies introduce a flavor of lossless raw capped at 3:1 such as rawlite and the current iteration of prores raw. Perhaps the licensing works in "tiers" and once you get into the lossy raw (3:1 and greater) then you need to show RED the money, which is perhaps something Canon for instance hasn't been willing to do and the reason why we don't have something like 8:1 raw on Canon. In any case this is all speculation.

Now that the petition has been denied they can do business. Apple had to try and it seems like Jarred gets that and hopefully Apple will start writing checks so we can get more compression ratios options with prores raw. And somehow I hope Canon will get more compression ratios options as well. I like their cameras, in certain occasions Canon is what I reach for.


And Phil, I think this is great moment to do something I've always wanted to do. To thank you for your dedicated contribution over the years, your tests, your posts, your passion & obsession at experimenting and learning every aspects of your tools, and your openness and generosity to share with the community. Truly, thank you...

 

[Phil Holland]

Appreciate that Gabriel!

 

[Misha Engel]

It's unclear to me a well if RED's patent covers all forms of compressed including mathematically lossless raw. The latter to me falls under a certain kind of gray area and perhaps that is why we've seen companies introduce a flavor of lossless raw capped at 3:1 such as rawlite and the current iteration of prores raw. Perhaps the licensing works in "tiers" and once you get into the lossy raw (3:1 and greater) then you need to show RED the money, which is perhaps something Canon for instance hasn't been willing to do and the reason why we don't have something like 8:1 raw on Canon. In any case this is all speculation.

Straight from the RED patent: an image processing system configured to compress and store in the memory device the raw image data at a compression ratio of at least six(6) to one(1) and remain substantially visually lossless , and at a rate of at least about 23 frames per second.

Let's say you compress your RAW sensor data 4 times and log it from 16 to 12 bits you get a compression of 4 x 16/12 = 5.33 and hence don't infringe the RED patent(less than 6).
Canon RAW Lite uses a total compression of around 3.5(assuming they use 16 bits sensor data).

 

[Phil Holland]

Read the rest of the patent for sure. As common with much of this discussion to date, one line doesn't tell the full story whatsoever.

 

[Misha Engel]

Read the rest of the patent for sure. As common with much of this discussion to date, one line doesn't tell the full story whatsoever.

With a motion video camera: above 2k horizontal resolution AND atleast 23 frames per second AND raw sensor data AND a compresion ratio of 6:1 or above AND Visually lossless(whatever that means) AND Bayer Pattern, did I forget anything of importance?

 

[Phil Holland]

did I forget anything of importance?

Yes, the entirety of the rest of the document.

 

[Gregory Karydis]

Yes, the entirety of the rest of the document.

Correct.

A few years ago, I've spent a rather long flight going through the patent in order to get better acquainted with REDCode and how to get the most out of my cameras.
Brilliant stuff and as I have a fairly good background in a few of the disciplines involved in putting this together, I trust that the patent will hold fast for years to come!
- side note, I love nerding out and reading this stuff!

Had a quick read through the court ruling. I'll give Apple an A for effort, as they put up a decent argument.
All in all, standard stuff between companies.
Thank God, Jim, Jarred and Graeme for making the patent airtight!

 

[Lee Saxon]

Man Apple's RDF (and/or marketplace influence), though. Thanking them graciously for suing you is a new level.

 

[Misha Engel]

Yes, the entirety of the rest of the document.

Canon got away with it since june, 2017 with Canon RAW Lite with a compression ratio betwen 3:1 and 5:1 and Kinefinity doesn't seem to care, they just skipped the US as a market for their equipment (you can buy them in Canada).
I have a feeling more are going to challenge this patent...

 

[David Rasberry]

Canon got away with it since june, 2017 with Canon RAW Lite with a compression ratio betwen 3:1 and 5:1 and Kinefinity doesn't seem to care, they just skipped the US as a market for their equipment (you can buy them in Canada).
I have a feeling more are going to challenge this patent...

There seems to be at least some IP exchange between Canon and Red, possibly in both directions, Like the electronic RF mount on Komodo. Could be they licensed some of Red's IP for Rawlite.

 

[Dave Del Vecchio]

RED has a number of different patents, but two of the key ones are Patent No. 9,245,314 (https://patents.google.com/patent/US9245314B2/en) which covers a camera recording RAW using lossy compression and Patent No. 9,230,299 (https://patents.google.com/patent/US9230299B2/en) which covers a camera recording lossless compressed RAW using Huffman encoding.

Patents usually include several sections, including Background, Summary of Inventions, Diagrams, and Claims. The most important section of any patent is the Claims section; that is the section that actually describes what is protected by the patent. The other sections (like the background, summary, etc.) are really just to provide context for the Claims, the Claims are what matter in determining what a patent actually covers.

So in the case of the RED patents, the mention of possible compression ratios like 6:1, 12:1, etc. are generally included as examples in the Background or Summary sections. The Claims section does not mention specific compression ratios at all, and as a result, Patent No. 9,245,314 covers any form of lossy compressed RAW recording in-camera, regardless of the compression ratio used. The patent is pretty broad from that standpoint.

The lossless compressed RAW patent (No. 9,230,299) also does not mention compression ratios in the Claims section, but it does mention a specific lossless compression technique (Huffman encoding). So this makes the scope of the lossless compressed RAW patent a little narrower than the lossy compressed RAW patent. Huffman encoding is one of the simpler and more popular lossless compression techniques and is the one used for lossless compression in the original DNG 1.0 specification. But it's certainly not the only viable approach. The original lossless JPEG standard included arithmetic encoding as an alternative to Huffman encoding and JPEG-LS (a later lossless JPEG standard) used the LOCO-I algorithm (based on Golomb coding). JPEG-LS delivers slightly better compression ratios for image data than Huffman encoding, but with faster compression times than arithmetic encoding. PNG images use yet another lossless compression approach, which is based on the Lempel-Ziv algorithm (specifically DEFLATE) which is also used in the compression of ZIP files. Version 1.4 of the DNG specification also added support for ZIP-based compression of RAW files.

In theory, I believe that any of these non-Huffman based lossless compression approaches could be used without infringing on RED's lossless compressed patent No. 9,230,299 (or at least the part of it that deals with compression anyway). Note that many of the image-specific lossless compression techniques were intended more for RGB image data so it's possible they would perform differently on RAW image data. The DEFLATE algorithm used in ZIP was designed as a more general purpose lossless compression technique, but it tends to be a bit less efficient on image data than some of the other approaches.

The downside of any of form of lossless compression, is that it tends to be somewhat limited in terms of its compression ratio. While the actual ratio can vary based on the content being compressed, for most image data, lossless compression tends to be in the range of 1.5:1 to 3:1. Since the compression ratio is not constant, this also means that there isn't really a strong guarantee on the maximum data rate with lossless compression (although it can't exceed the uncompressed rate). In some cases, lossless compression may not reduce the file size at all (as anyone that has put a ZIP file inside of another ZIP file can attest to). With real world image data though it's pretty rare to encounter this kind of uncompressible edge case.

From a video recording perspective using lossless compression means that file sizes may vary somewhat, and that faster guaranteed write speeds are needed on the recording medium to handle this. With lossy compression, it is easier to hit specific compression ratio targets and constrain the data rate into a particular range (so that you can determine how fast the storage media needs to be, and how many minutes of video will fit in a given amount of storage space). But obviously with lossy compression you do run the risk of adding compression artifacts, which is impossible with lossless compression.

I don't think I have come across any definitive technical description of Canon's Cinema RAW Light format, but given that it has compression ratios up to 5:1 it doesn't seem possible that this is a truly lossless format (as that would be significantly better than the best lossless compression techniques currently known for image data). Additionally the data rate for Cinema RAW Light stays constant (1 Gbps) over a wide range of different frame rates (24 to 60 fps) and color depths (10-bit/12-bit) which means that Canon varies the compression ratio depending on what is being recorded (from a low of about 2.4:1 for 24 fps video up to 5:1 for 60 fps footage). Setting specific compression ratio targets like this generally isn't possible with lossless codecs.

 

[Bob Gundu]

I believe Reds patent also states “internal recording” which means other companies have gotten around the patent by using an external recorder.

 

[Misha Engel]

I don't think I have come across any definitive technical description of Canon's Cinema RAW Light format, but given that it has compression ratios up to 5:1 it doesn't seem possible that this is a truly lossless format (as that would be significantly better than the best lossless compression techniques currently known for image data). Additionally the data rate for Cinema RAW Light stays constant (1 Gbps) over a wide range of different frame rates (24 to 60 fps) and color depths (10-bit/12-bit) which means that Canon varies the compression ratio depending on what is being recorded (from a low of about 2.4:1 for 24 fps video up to 5:1 for 60 fps footage). Setting specific compression ratio targets like this generally isn't possible with lossless codecs.

REDCodeRAW, Canon RAW Lite and ProResRAW are not lossless. All commen raw format's (except X-OCN) are logged from a bit depth to a lower bit depth hence not lossless.
Who determines what visually lossless is (some old fart with bad eye sight working at the patent office or a pixel peeper working at a movie studio with all the equimpment in the world to help him/her).

 

[Terry VerHaar]

RED has a number of different patents, but two of the key ones are Patent No. 9,245,314 (https://patents.google.com/patent/US9245314B2/en) which covers a camera recording RAW using lossy compression and Patent No. 9,230,299 (https://patents.google.com/patent/US9230299B2/en) which covers a camera recording lossless compressed RAW using Huffman encoding.

Patents usually include several sections, including Background, Summary of Inventions, Diagrams, and Claims. The most important section of any patent is the Claims section; that is the section that actually describes what is protected by the patent. The other sections (like the background, summary, etc.) are really just to provide context for the Claims, the Claims are what matter in determining what a patent actually covers.

So in the case of the RED patents, the mention of possible compression ratios like 6:1, 12:1, etc. are generally included as examples in the Background or Summary sections. The Claims section does not mention specific compression ratios at all, and as a result, Patent No. 9,245,314 covers any form of lossy compressed RAW recording in-camera, regardless of the compression ratio used. The patent is pretty broad from that standpoint.

The lossless compressed RAW patent (No. 9,230,299) also does not mention compression ratios in the Claims section, but it does mention a specific lossless compression technique (Huffman encoding). So this makes the scope of the lossless compressed RAW patent a little narrower than the lossy compressed RAW patent. Huffman encoding is one of the simpler and more popular lossless compression techniques and is the one used for lossless compression in the original DNG 1.0 specification. But it's certainly not the only viable approach. The original lossless JPEG standard included arithmetic encoding as an alternative to Huffman encoding and JPEG-LS (a later lossless JPEG standard) used the LOCO-I algorithm (based on Golomb coding). JPEG-LS delivers slightly better compression ratios for image data than Huffman encoding, but with faster compression times than arithmetic encoding. PNG images use yet another lossless compression approach, which is based on the Lempel-Ziv algorithm (specifically DEFLATE) which is also used in the compression of ZIP files. Version 1.4 of the DNG specification also added support for ZIP-based compression of RAW files.

In theory, I believe that any of these non-Huffman based lossless compression approaches could be used without infringing on RED's lossless compressed patent No. 9,230,299 (or at least the part of it that deals with compression anyway). Note that many of the image-specific lossless compression techniques were intended more for RGB image data so it's possible they would perform differently on RAW image data. The DEFLATE algorithm used in ZIP was designed as a more general purpose lossless compression technique, but it tends to be a bit less efficient on image data than some of the other approaches.

The downside of any of form of lossless compression, is that it tends to be somewhat limited in terms of its compression ratio. While the actual ratio can vary based on the content being compressed, for most image data, lossless compression tends to be in the range of 1.5:1 to 3:1. Since the compression ratio is not constant, this also means that there isn't really a strong guarantee on the maximum data rate with lossless compression (although it can't exceed the uncompressed rate). In some cases, lossless compression may not reduce the file size at all (as anyone that has put a ZIP file inside of another ZIP file can attest to). With real world image data though it's pretty rare to encounter this kind of uncompressible edge case.

From a video recording perspective using lossless compression means that file sizes may vary somewhat, and that faster guaranteed write speeds are needed on the recording medium to handle this. With lossy compression, it is easier to hit specific compression ratio targets and constrain the data rate into a particular range (so that you can determine how fast the storage media needs to be, and how many minutes of video will fit in a given amount of storage space). But obviously with lossy compression you do run the risk of adding compression artifacts, which is impossible with lossless compression.

I don't think I have come across any definitive technical description of Canon's Cinema RAW Light format, but given that it has compression ratios up to 5:1 it doesn't seem possible that this is a truly lossless format (as that would be significantly better than the best lossless compression techniques currently known for image data). Additionally the data rate for Cinema RAW Light stays constant (1 Gbps) over a wide range of different frame rates (24 to 60 fps) and color depths (10-bit/12-bit) which means that Canon varies the compression ratio depending on what is being recorded (from a low of about 2.4:1 for 24 fps video up to 5:1 for 60 fps footage). Setting specific compression ratio targets like this generally isn't possible with lossless codecs.

Thanks so much for chiming in with the very helpful and informative commentary. It's the first time in this thread that I have a sense someone does actually understand this topic!

 

[David Bourla]

Can't wait for OS X Redcine-X to get Metal acceleration.

 

[Joey Lacroix]

I wonder if this means the Apple accelerate framework will be optimised for R3D playback on all Apple devices?

Would be really great to have better R3D playback on older Mac (thinking trashcan, maybe even last gen 5.1 MacPro with a graphic card that can supports Metal) :smiley:

RED integration with Apple’s METAL framework for realtime R3D playback is coming along well and the work that the two teams are doing together is exceeding expectations. We are very excited for the new Mac Pro and the new XDR pro display and the power they bring to the entire RED workflow.

Not yet

Thumbs up to the RED & Apple team for making R3D workflow on Mac better ! :thumbsup:

 

[Andy Maser]

Right...should we be reading that previous gen Apple computers with sufficient GPU power and Metal support will see R3D performance increases along with the newest MacPro?

 

[Jarred Land]

We have finally completed the METAL GPU accelerated support for the SDK which will be rolling out in the next 24 hours.

3rd parties can start integrating into their apps and FCP should get in the next major release. Getting the Metal SDK done took priority over re-writing REDCINE-X to support metal so somewhat ironically you will likely see all the NLE's support it before REDCINE-X does.

And for the windows folks don't worry you have some CUDA love coming your way over the next 24 hours as well....

 

[Mike P.]

SURPRISE!

What are the minimum GPU requirements for Metal support? (And when is the next FCPX/Resolve dropping?)

 

[Samir Patel]

And for the windows folks don't worry you have some CUDA love coming your way over the next 24 hours as well....

Loveeeee!