Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Intel's new Core-i7, big impacts on performance

D

David Newman

Well-known member
Joined
Dec 29, 2006
Messages
718
Reaction score
0
Points
0
Website
cineform.blogspot.com
Thanks David
 
Wow, even better than I expected. That's the biggest increase I've seen yet. After Effects, Cinema 4D etc all look good (25% or more) but your results are in a different league! Great optimization, man!

Bruce Allen
www.boacinema.com
 
One item I didn't expand on in the blog entry is how effective the new Hyper Threading mode is on these chips. All the test where with HT enabled, to give the best numbers, but with some test the HT offered an amazing 30% gain. So while the memory and processor achitecture has improved greatly, if your application can also benefit from the Hyper Threading, there is up to 30% additional performance bonus. Previously we did a good job supporting the HT on P4s, yet the performance boost max'd out around 15%.
 
Rushes STAMINA!

Great.
 
Hi David,

this is really amazing improvement.

Any idea when the first i7 systems are expected to hit market?

Your chart has an entry for R3D to CF RAW conversion speed - I guess you create an artificial RAW from the RGB out of the SDK, is that right?
That would also explain the comparable slow speed (due to the SDK's deBayer being involved).

If RED is really hesitating about access to their RAW with bypassing the deBayer step they provide, I'd suggest them to force all SDK users to at least include an optional deBayer via REDs SDK as a must, but enabling different deBayer technologies for the RED RAW data stream.

I got the feeling that would be a valid compromise that makes sure that RED can always get the same render quality via their own SDK out of any product, but at the same time enabling high speed editing / grading / compositing using other methods.

What do you think?

Axel
 
The creating of bayer sampled RAW back form 4:4:4 is very fast, but it is the creation of 444 from bayer in SDK's R3D decode that is slow, and this is the function we hope will be controllable through a RAW option in the SDK. We absolutely have no issues offering the user choice between the R3D SDK debayer and third party post debayers -- we do that now and choice is good (and we will continue offering this to the customer when a RAW mode is added.) As we are already providing alternative debayering solutions, all the SDK does by not providing RAW is slow down the usage of R3D files -- by a factor of 3X is my measurements (maybe more now with the Core-i7 as the debayer is not scaling as well as the base wavelet decodes -- getting back onto the thread's subject.) So many votes for RAW mode in the SDK (including an option to completely turning off all color processing.)

Back on subject: The Intel Core-i7 is available now, including the required motherboards. The Core-i7 and new motherboards support triple pumped, three channels of DDR3 all connected directly to the CPU. It is bloody amazing -- sorry I've never been so thrilled by a new CPU before -- never has one bypassed the previous generation by such wide margins (and I thought Core-2 was a pretty awesome step up.) Here are some vendors selling the part I've tested http://www.pricewatch.com/cpu/core_i7_965.htm and some motherboard vendors http://www.pricewatch.com/motherboards/core_i7.htm. So new not all system builders have yet to added Core-i7s to their line ups, nothing from HP yet, here are Dell's Core-i7 gaming systems http://www.dell.com/content/product...sktop-xps-730x?c=us&cs=19&l=en&ref=dthp&s=dhs.

Another example of how fast this desktop core quad part is (i.e. not even for "workstations") is the speed in can encode 4K RAW (if only Red One still had a RAW Port.) Using Dalsa 16-bit 2:1 4K RAW DPX files, encoding to CineForm's highest quality (think REDCODE 65 in RED ONE terms), this stock desktop machine encoded 12-bit 4K at 43 frames per second. So with a little overclocking (which these parts support) or a second CPU, 60fps 4K from a RED ONE would have been possible -- now. Oh well.

Seriously, get Core-i7 Mac or PC, you are going to love it.
 
David Newman said:
As we are already providing alternative debayering solutions, all the SDK does by not providing RAW is slow down the usage of R3D files -- by a factor of 3X is my measurements (maybe more now with the Core-i7 as the debayer is not scaling as well as the base wavelet decodes -- getting back onto the thread's subject.) So many votes for RAW mode in the SDK (including an option to completely turning off all color processing.)
Click to expand...

Are you going back to raw from rgb and why would you do that considering you're just compounding artifacts?
 
Not compounding artifacts, and we have proven this to be very worth while with a 2.5X reduction is data rate over 4:4:4 with no negative impacts on quality and some positive given you now have debayer controls in post. I posted samples a couple months back, this link has some http://www.reduser.net/forum/showpost.php?p=283812&postcount=161. And the argument is off, as the whole point that bought this feature up was that it is all optional, you can use the advantages of bayer sampling or not. Advantages are size, speed and subtly quality, it is hard to pass up if you using a CineForm workflow.
 
David Newman said:
Advantages are size, speed and subtly quality, it is hard to pass up if you using a CineForm workflow.
Click to expand...

Looking at your samples, it looks like it gets softer with rgb to cineform raw and then when you sharpen it, it also amplifies the noise alot more. It doesn't look like an advantage to me.
 
While your interpretation isn't quite correct, you don't have to use it, remember options are good. Plus is it not always obvious what to look for (you really want to know how well you can push them in post.) The two images to compare are:

RED SDK output with OLFP off (OLFP compensation can make nasty ringing in the highlights, see above link.) So this it is best setting. This is also what the CineForm 444 encoded output looks like, completely preserving the debayer calc from the SDK.
R3DDeNoise6-DPX.jpg


Compare to CineForm RAW with Detail 1 (the default of out of four detail settings -- I only show two, you can choose the detail level you need -- it is not a sharpen filter.)
CineFormDetail1.jpg


Now given that they look basically indentical (I sure a double blind test would find most saying they are the same) they can pushed in post indentically (I've tried it) and the second is 2.5 times smaller, faster, and also you to change the demosaic as needed, most find it the easiest choice once you consider workflow advantages (is that what compression is all about, making the workflow easier, otherwise we would be all using 50MB DPX files.) The reason back sampling bayer source from 4:4:4 works, is the function of the demosaic is to guess the missing pixels, it doesn't need to re-guess the one is actually has, so only the original bayer samples are stored in a raw mode transcode (the one that weren't guessed.) The 4:4:4 image has no more real data the bayer original, yet it take more space to store the 4:4:4 than RAW, so the back sample is the best workflow (until we have direct RAW in the SDK.)

Readers, lay the images on top of the other an flip between them and you will see how close they really are.

Now back to the thread topic.
 
My interpretation is not correct? 20 years doing post and image processing and I can spot differences very easily. The CF raw looks softer and no doubt you get some space savings there. But to me that defeats the purpose of RAW when you should be doing RGB.
 
There are four settings, each impact noise levels and sharpness, none of the them are inherently correct, just as RED output is not inherently correct -- this a creative process remember. In your 20 years I'm sure you have seen a few camera settings, and may more in post options that change the way to resolve an image. The back sampling has no less information, I've just shown you a different a demosaic that is under your control. If you see as very slight softening in Detail 1, is not less information, just a the Detail 3 setting is not more information with it enhanced detail, they are just demosaic options (just like Red's own detail and OLPF comp. options), only difference is we aren't baking it in. Detail 1 from CineForm RAW and the output of REDSDK with detail at max. are close match but not a digital match, true. I happen to prefer Detail 1 for my work (I think can push it little more), that's why I show it, some may prefer Detail 2 or Detail 0 (depends on the shot also), it is your choice in post as nothing is lost. If I had Red demosaic math it would be indentical, not necessarily better, just the same as the 4:4:4. So we just gave you 4 more choices, you don't have to use them, but my 20 years in post production engineering tells me choice is good (as in the business for every opinion there is a complete opposite.)
 
Well you were first saying that your way was better and faster and now it's a creative choice? Sometimes it's hard to filter between your cineform hype and what the real intention is.
 
I'm saying all of that, and isn't it all related? Choice helps creativity, speed helps creativity, size helps keep you storage cost down or more online which helps creativity. My recommendation to CineForm users working at 4K, there is no practical advantage to exporting to CineForm 444 over CineForm RAW -- I even show samples to make my case (not hype.) Exports at 2K or 1080p then 4:4:4 is the way to go, but not at 4K. This is just what I recommend, and I do so as you have more flexibility (quality controls in post), higher playback speeds and greatly reduced file sizes without impacting quality. If you are not a CineForm user that doesn't directly pertain to you, unless you want to become one. If you just don't like or understand 444 to RAW sampling, then you should be for the RAW mode in the SDK which will remove the need (which is what Axel was asking about) and increase the transcode speed 3 times.

And speed was the reason for this thread, remember it was about Core-i7 performance, and I used CineForm decodes to showcase that (and why not showcase our technology, is that intention unreasonable?) My posts will be generally CineForm related, but this one has I bit more general cross-over as the new Intel parts will benefit post production in general.

P.S. Intel is not sponsoring this post, although I do thank them for the early access to their hardware.
 
David,

I totally second your comments, especially on the reasons for going back to Bayer from 444 - which is hopefully "obsolete" when a new SDK revision is showing up, but not removed then (I need it for other purposes too).

I am totally after speed. RAW is speed. Not RGB444. In a good RAW workflow, RAW is kep as long as in any way possible. So - if you are not going to add effects, but color grading - the only moment to convert from RAW to RGB is when you render out the final image from grading. I can also imagine that RAW could be a release format at some point.

I still see no easy way to showcase 4K RAW "for the masses", but RAW is already there on certain computers and certain software. Its a huge leap ahead in that context.

The new Intel CPUs seem to bring a very big improvement, the same with GPUs recently. The combination of both makes it thinkable to work full 4K. Question remains what displays we will have at that point.

Going back to RAW from RGB is what I was looking for and I did this artificially on my own, with similar observations like those David explained. If others disagree on the quality, their point, not mine. I know to many people making online masters from FCP RED proxy editing. Who cares about quality in THAT moment? I was looking for this functionality and now I found it, makes me happy :)

And grading in RAW 4K in realtime is good enough for an HD mastering in the end.

The discussion about which workflow option a user is going to use is obsolete, as long as the options are given. Everybody can choose what he/she likes best, what best fits their needs. Thats the good thing when you have choices.

Thanks for you efforts and sharing them, David,

Axel
 
ok...
question about going from Red/SDK rgb...to Cineform Raw through back sampling...or really remosaicing(did I just invent a word?)

as far as I can tell...mathematically in an ideal situation there would be no loss of quality....

correct me if I'm wrong of course...I may be overlooking something

If the bayer pattern is known....and I assume it is....
Then a bayer pixel would be known to be either R,B,or G
If a bayer pixel is known to be G....and the developed image(demosaiced) contains anything other than G....then the resulting R and B values can be discarded....Since this is an RGB Raw sampling to begin with, the other color values should not affect luminance...the way YUV would.

the same would also hold true for the Blue and Red bayer pixels as well.Each known pixel...(known as in there position and color filtering on the bayer sensor) would have the other 2 non native color channels discarded.
Being that seperate channels are recorded for each value this should be simple.

Some caveats....
1.If the bayer pattern is unknown(how many are there really?) then this is assumption
2.If there is some baked in sharpening ect ect that would be anything but the Raw RGB
3.If the origional debayering gave a pixel....green for example....more or less green then was natively recorded due to it's neighbouring pixels via some color weighting algorithm

If these are all not factors, and unless I'm missing something(quite possible) then the mathematics of remosaicing is quite simple, and should be mathematically perfect. At least the remosaic part.

The algorithm that would again demosaic it, of course would be different...and could lead to different results
 
You must log in or register to reply here.
Back
Top