Let's make STAR LENS TESTER to get best lens copy easily (Airy disk-PSF-point source)

[Ilya O.]

Photo and cine lens sample variation is reality. And there's even more copy-to-copy variation if we look at vintage or used / serviced lenses. More so with any zooms.

The second part of the bitter truth is that nearly every real-world lens (especially those after service) is facing centering/tilt/rotational errors to some extent which lead to on-axis coma, astigmatism, spherical aberration...Meaning you have bad lens copy with mediocre picture.

High resolution video sensors of 8K and above do see all these flaws clearly.

How do we cope with that? We try to do the chart/trees-on-the-sky/brick wall testing of the several lens (3 to 5 at best) copies before purchase. Well, guys, that is just not the way we all deserve to have...Besides, this generally helps only to distinguish between the very bad copy and the moderate one but not between the best and the average. And what to do if you want the good copy of costly cine prime or zoom, or some vintage classic — when there's no easy way to plug&test, compare, return and proof the bad sample?

All in all, this approach needs several samples or reference lens to choose the best, but what to do if you do not have one? So, how can we know for sure if the lens is optically OKAY or not by itself, without the reference? I think we need a device to check any lens objectively.

So what I suggest for all us lens passionates is to crowdfund STAR LENS TESTER device, portable and easy to use so we can take it to the shop or rental to choose the best of the pack. It can also check the lens before/after service optical performance.

Star test (aka Airy disk, or point source, or PSF pattern test) is rather simple yet extremely sensitive to many lens aberrations and even glass molding defects. It is well-known and widely used in optical labs and by astronomers but what is strange I find no such device for sale for us! The basic draft of such tester we see like this:

The test uses visible light to go through a tiny pinhole, then into a collimating lens to get rays parallel, then into a prism and to the tested lens forth to the mirror and back to the prism and an eyepiece (that is why AUTOcollimation) — and you see the result: aberration free lens renders point as an nice Airy disk:

This disk is a basic element of good image forming process on our sensors. The better it is the higher MTF would be, the better the overall picture IQ. You can not expect good picture performance out of bad disk, and if you have aberrated disk on axis, imagine it would get much much worse off axis and towards the image edges. So what would we face in real-world lens examples? On-axis coma plus a bit of misalignment is very common and looks like this when software generated:

Here you are the same aberration but as Star tested out of OKS1-100-1 100mm f2 lens:

Star test shows clearly other typical aberrations of a bad copy/wrongly assembled lens:

astigmatism:

spherical aberration:

pinched optics:

and a bunch of all aberrations which are actually the case with real lenses when many of them are mixed together:

Despite the beliefs, the bunch is very common, and here is a Star test of Zeiss Pancolar 50mm f1.8 for you to quest out what aberration types are present:

***
So, do you guys need such a device?
The point is not in aberration guess games, but to choose a lens which renders point as close as possible to Airy disk pattern.

Important note: if you find a nearly ideal lens with this Star Tester, that does NOT mean the lens looses its character and look! It just means that the lens is optically aligned, all elements are centered on one axis and you can expect max IQ, sharpness and resolution for this particular lens design. As for the lens look it'll be all there since star test shows little of vignetting, bokeh, CA, of-axis aberrations etc...

I consider this post as STAGE 1: market demand inquiry. It will last till we see some numerical response.
If OK, I go to STAGE 2: R&D and prototyping.

For the sake of easy analysis I strongly recommend everybody to post using this guide:

NEED? YES or NO

PRICE EXPECTATIONS: just put plain numbers here

COMMENTS: any input is welcome!

If you are not registered on RedUser, just send me email to StarLensTester@gmail.com with StarLensTester in subject and YES/NO in the text body.

 

[Lee Kelly]

Not sure if its my browser but your images did not work?
I will say its a YES from me.

I own a few used lenses and even some brand news ones and I dont know if any of them are any good. I know some are bad but some are bad in a good way. For some applications a shitty lens is great and other applications you want the highest image quality you can get.
I think if we had access to this tool we would be upset with the result and fixate on that 'negative' aspect of a lens that had previously served you very well.
Its pixel peeping all over again.

As for cost, I dont know. I guess if you are a lens technician charging people to fix their lenses then you should have one of these and could expect to pay good money but if youre just a punter looking on ebay for old stills lenses you might be happy to pay $100 for something like this?

 

[Vance Colvig]

Unable to view jpgs

 

[Dan Kanes]

YES.

I'll buy a few.

 

[Ilya O.]

With all the repect, I dare to invoke Jacek, our beloved trickster, to either skyrocket or trample down the thread!

 

[David Kellermann]

Really interested, so YES

 

[Ilya O.]

So guys let me roll out some test shots to show why Airy disc pattern is crucial for finding the best lens. The goal is to show you how good, normal and bad Airy disc lenses perform both on its own and at 6k resolution.

I used good old Sony Alpha NEX-7 cause it has superb sensor with very dense pixel package, the pixel pitch is 3.88µm! Red Helium has 3.65µm, 50Mpix Canon EOS 5DS R has 4.13µm. Pixel pitch tells you the distance from the center of one photosite to the center of the other. The smaller the pixel pitch, the higher resolution we can squeeze out of the camera. NEX-7 is APS-C sensor camera, 23.5 x 15.6mm, image resolution is 6000x4000, sensor resolution is ~128 lp/mm. All shots done at ISO 100 (max DR for this sensor), with Front Curtain Shutter ON (less vibration), 10 seconds delay shot (to avoid after shutter button vibration), Picture Profile is Neutral (0,0,0), Noise Reduction is off.

Well then, lets take the good and the ugly first! Can you guess out of these two sexy Mir-1 f2.8 37mm?:

Do not discriminate Zagorsk one (left)! It can easily be better than KMZ one (right), it just depends...
But not now, and this particular Mir-1 from ZOMZ (Zagorsk optical Plant), shown on the left, made in 1963, represents a bad sample today. Why? Just look at its horrible Airy disc pattern:

Meanwhile, a good guy pictured on the right is Mir-1 #011423, by KMZ, has been adjusted to its max:

Okay, here what these disks or dots mean for the resolution first. Take a glance at the optical bench pattern which measures visual on-axis resolution of a lens by itself, no sensor — as a limiting factor — involved in this bench scheme, right click to open in high res:

Pretty shocking, yeah?
These patterns are made for precise visual/optical resolution measurement. The point here is to find the number of the tiniest component (each consisting of 4 stroked square elements) in which you can distinguish all lines and spaces in all angles. So at this samples, if only viewed at 100% scale, one can say that good Mir-1, on the right, resolves 13th component with minimum contrast, but nevertheless. How to find that 13th component? Can you see 10th component, entitled '10' on the right, then move your sight down, from left to right, unnamed 11th next to the vertical dash, etc till the 13th component). The Ugly can do only 6th component, and that's according to over optimistic attitude Others you can not really count since huge astigmatism lets the Bad resolve 45Deg squares only, and coma smashes the image to the bottom left direction...

So what are the optical on-axis resolution numbers of these lenses at minimum contrast? The 13th component of good disc Mir-1 means — shazam — ~535 line pairs per mm. Bad one gives 6th component, it is ~356 lp/mm, 1.5 times worse than the good one. 535 lp/mm seems to be unbelievable for Mir-1 lens, but it is not There's so called Rayleigh criterion which limits any ideal lens' (with ideal Airy disc, aberration free) resolution. The pretty conservative version of this formula is R=1.22*lambda(=wavelength in nanometers)*F-stop. So according to this theoretical point, for absolute aberration free Mir-1, for 555nm green light (our eye best sensitive to), the resolution limit, or a radius of an Airy disc, would be: R=1.22*555*2.8=1895,88 nanometers, or ~1.9µm (micrometers). This is the limit, 1.9µm radius disc for f2.8 lens. Now we take that 535 lp/mm of good Mir-1, and find out to what Airy disc this corresponds. It's just simple, one should divide 1 by 535, = 0,001869...or 1.9µm. What this means is that this KMZ Mir-1 is close to perfect

I hear you, guys, these numbers here nobody loves, forget them. Ok, now I show you image samples of both Mir-1's. Here's the Ugly, at f2.8, 6k on right click:

Well well well, you might think, — not so bad for a vintage 54 year old photo prime with such an Airy disc pattern-) As for me, I always believed Mir-1 to deliver much more And it IS! Here's the Good, wide open:

Now let me dive more into pixel peeping. Central, focused-at area, left is the Bad, right is the Good, mirrored (horizontal flip) for convenient outwards-the-center comparison:

WOW, open it in new tab, zoom in, and just look at all this little details, bricks, little tiles, windows, trees and now you can see even a Hebrew star in the center ! Mir-1, a good one, indeed delivers a level.

What I here find surprising is that corner/border performance is pretty CLOSE to each other! Of course, the Good still beats the Ugly, but not so drastically as we saw at the image center:

I was expecting the Ugly to go into complete mash...and the Good to be far better... Well then, its the point where we all should start talking about the vintage character and 3d fall-off I guess ah aha ha h

Now lets see more night and day difference at the daylight shot. Mir-1 #632420, the Bad, wide open, focused at the Hebrew star at the center:

Here's the lucky opponent from KMZ, same focus point:

Now the same shot, central region head-to-head:

High contrast of the daylight scene gives the Bad more chances, and you feel it's OK just till the moment you start looking at the right Now you see how much clarity (=high frequency resolution; notice tiny square tiles above the tree line) gives the optically OK lens? That's wide open, infinity test, guys,.. That's how 50+ yrs old grandpa, if in the good mood and being fed and slept well, can one day kick some ass! This Mir-1 of KMZ is vibrantly sharp and seems to be indeed (as its creator Prof. Volosov mentions somewhere) very close to APO performance since zero CA or lateral color at this high contrast white buildings.

...see part 2 a post lower

 

[Mike Finegan]

do you ever relate to lp/mm?

 

[Ilya O.]

OK with Mir-1, a photo lens, then, what about cine lenses? And what's the difference between the good and no-so-good/mediocre Airy disc to resolution/performance influence?

Here it goes I selected out of four 28mm classical LOMO primes, found one which is quite ok (but not the best): OKS7-28-1 28mm f/2.0 #770009 lens, and the other is same 28mm model, but with mediocre Airy disc (but not as bad as that Mir-1 above), #790562:

Before that I put both on the 1600mm collimator optical bench to get Airy disc images and resolution patterns, left is worse, right is near OK:

So what's the optical resolution of close to perfect OKS7-28-1 f/2.0 lens, on axis? I can count with quite an effort 10 lines of 15th element here. I choose 15th, not the 14th because I can clearly (and better than at this picture) see 15th countable with an eye through an eyepice and a microscope lens (that's why visual/optical resolution is not limited by the NEX-7 — or any — sensor). So, 15th means ~802 lp/mm. As for the mediocre 28mm, on the left, it resolves 10th element, ~601 lp/mm, 1.33 times less.

High on-axis visual resolution tells us that the lens can resolve very well in the central spot of an image. So let's see some test shots. Here's OKS 28mm, the good, represented by right patterns above, at f/2.0, 6k, same settings Sony NEX-7 23.5 x 15.6mm sensor size, focused at Hebrew star in the center:

And the left OKS, mediocre Airy disc:

Left image part is for some reason very soft, while the right one is OK, it's not clear for me now why that happens. Maybe some tilted element?
Anyhow, here is head-to-head both OKS 28m comparison, now the left one is flipped to represent better edge:

Here we do not have such a tremendous difference as seen in Mir-1 samples, but again, one can clearly see the better definition of an OKS with OK Airy disc pattern. It is like the mediocre OKS had been shot through the dirty curvy window glass or there were Gaussian blur 1-3 applied in Photoshop

Finally let's check out how disc pattern effects edge performance, left is mediocre, right is OKAY OKS 28mm:

Again, here we notice the difference with the good one being noticeably sharper at the edge.

To sum up, these test shots of Mir-1 and OKS7-28-1, both with bad and good Airy disc pattern, show how disc/elemenary dot/PSF shape directly and unmistakably effects lens performance.

So if only we had a device to see that Airy disc...Please participate, guys!

 

[Tommaso Alvisi]

YES ! ! ! !

I'd buy one for sure!

 

[Jiri Vrozina]

Dan,
Just curious. What is qualification of your chief lens disigner Mr F.Schultz?
Has he designed any other lenses before?

Regards
Jiri

 

[Les Hillis]

Thanks for all the lens information you've been sharing Ilya.

I'd like to see you produce your Star Lens Tester, but I don't actually need one.

I would use one though, just out of curiosity and to learn more about lenses with, if such a thing were affordable and easily obtainable.

I don't think it would be cheap to make such a device though, with all that would have to go into it's manufacture.

I think also something like your Star Lens Tester would be of most value to someone who could use it to help make adjustments to a lens, or use it to make a choice between several samples of the same type of lens. Not sure how many people would be doing that. I think when it comes to assessing lenses, people in general just rely on what their eyes see on the screen and leave it at that, not caring too much why the lens makes the pictures look like it does.

Maybe you could make the device anyway, and use it or sell it to people who can provide a lens testing service for others?

 

[Mark M]

Very interesting.

Beyond several of us Redusers, I'd assume most camera shops who buy/sell used lenses would find this a compelling 'must have' product too.

 

[Les Hillis]

Is there anything else already available that's currently being used by people to check lenses like this?

 

[Dan Kanes]

Dan,
Just curious. What is qualification of your chief lens disigner Mr F.Schultz?
Has he designed any other lenses before?

Regards
Jiri

He's extremely qualified.

This thread is not about us or our company though.

I should add, he's our chief designer. That doesn't mean he's our only designer.

Not sure what this has to do with Ilya's awesome product / project here though.

 

[Jiri Vrozina]

That sounds like Donald Trump answer..��

 

[Dan Kanes]

Come say hello at Cinegear.

We are in booth P112

 

[Ilya O.]

Dan, wish you all the luck and good guys around at Cinegear!

 

[Ilya O.]

...I would use one though, just out of curiosity and to learn more about lenses with, if such a thing were affordable and easily obtainable...
I think also something like your Star Lens Tester would be of most value to someone who could use it to help make adjustments to a lens...Is there anything else already available that's currently being used by people to check lenses like this?

Les, appreciate your reply much. Very reasonable and friendly

Yes, the point is to make the price as low as we can without sacrificing the device main functionality.

Me myself and other lens technicians use this star test to tweak a lens IQ, but nobody I heard of have a device that is portable and light and easy to operate. We just use bulky, not portable and very pricey optical benches with the tons of functionality. Leviathan 1600mm with the 150mm aperture looks like this:

So the goal is to make one-purpose small diagnostic device.

 

[Jacek Zakowicz]

Thanks for all the lens information you've been sharing Ilya.

I'd like to see you produce your Star Lens Tester, but I don't actually need one.

I would use one though, just out of curiosity and to learn more about lenses with, if such a thing were affordable and easily obtainable.

I don't think it would be cheap to make such a device though, with all that would have to go into it's manufacture.

I think also something like your Star Lens Tester would be of most value to someone who could use it to help make adjustments to a lens, or use it to make a choice between several samples of the same type of lens. Not sure how many people would be doing that. I think when it comes to assessing lenses, people in general just rely on what their eyes see on the screen and leave it at that, not caring too much why the lens makes the pictures look like it does.

Maybe you could make the device anyway, and use it or sell it to people who can provide a lens testing service for others?

Les has a very good realistic understanding of the industry. Listen to him. The quote above are my experiences exactly after 4 small portable camera/lens test devices (designs and products) under my belt.....