Breakthrough development in CMOS technology

[Iannis Holwech]

A link to this was posted some time ago, but when I went back to check the responses, to my surprise there was none.

I hope a new thread with a different presentation of this project will make people here more interested.

The MEGAFRAME project! :bigear:

If I understand this right, this breakthrough will have an enormous impact on the development of future CMOS sensors and digital cameras both in light sensitivity and possible framerates.

The highlights;
The first stage of this has been targeted at sensors small enough to fit into a medical microscope, but larger sizes should be no problem now when the basic design has been made.

The first sensor is 32x32pixel that can capture 1024 individual photons at one million frames a second to within 100 picoseconds, when the photon arrived at each detector.

They have gone from 0.35 micrometer CMOS technology size to 0.13 micrometer CMOS technology.
In a pixel that is 50 microns-by-50 microns with a lot of functionality in it, only maybe 1% of the pixel can be used to detect light which has a photosensitive areas that are just 10 microns in size.

Data capture transport of high-speed, highly parallel readout capable of achieving 10Gb/s.

“CMOS, or complementary metal-oxide-semiconductor, is very cheap to produce, once the design has been mastered,”

"In the meantime, Megaframe is always interested in meeting potential research partners, and is particularly keen to meet investors who are interested in developing the 3D camera technology." :thumbup1:

Now, I hope somebody here can put this developement in context comparing it to today’s CMOS technology to better understand the impact on the future of Image Capture.

The relevant Links where I have taken the text from;
The Megaframe project
3D CMOS camera for your mobile?
Video camera that records at the speed of thought
Filming photons, one million times a second

For those that don't want to read all the articles, I have tried to copy/past some of what I thought was the most relevant information for understanding the development of this concept .

The Megaframe project, which developed a CMOS video camera that can capture 1024 individual photons at one million frames a second. It can record, to within 100 picoseconds, when the photon arrived at each detector.
It is insanely fast.

“We started the project with the redesign of a SPAD, at the time available only in 0.35 micrometer CMOS technology, a technology for which we had pioneered SPAD design,” notes Charbon. “We soon realised that if we wanted to fit a stopwatch in each pixel we needed to go further down in feature size. We needed to go to technologies that are much more compact, so we chose a 0.13 micrometer CMOS technology.”

The stopwatches are essential to record when the photon arrives, SPADs are necessary to detect the light, and both need to be present in each pixel. And the consortium wanted to do a 32 by 32 array of pixels, making a total of 1024 individual SPADs and stopwatches!

“We spent a good year and a half optimising the design of SPADs in 0.13 micrometer CMOS technology. So the challenges were mostly to reduce noise, increase sensitivity, and to miniaturise the device.”

Then they had to decide which stopwatch they needed to use, the TDCs or TACs mentioned earlier, which led to one of the most exciting of the project’s many breakthroughs.

Megaframe phase I simply sought to establish that a photon-resolution, high-speed CMOS camera was possible at all. Both the TDCs and the TACs met the design targets, but instead of eliminating one, they discovered that they could potentially use the two in different ways.
The upshot is that their camera can time the arrival of a photon to within 100 picoseconds, and they can do it up to 1 million times a second. “And every microsecond we can count up to 64 photons,” declares Charbon.

Of course, all these electronics in 32 by 32 array produce a staggering amount of data, so the partners needed to develop new and improved ways of moving the data from the pixels into storage.
In the end, they developed a high-speed, highly parallel readout capable of achieving 10Gb/s.

If you have a pixel that is 50 microns-by-50 microns, with a lot of functionality in it, the area that is sensitive to photons can only be a small proportion of this pixel. So you can imagine that only maybe 1% of the pixel can be used to detect light,” Charbon explains.
It was a complex task, but the high-precision lenses work amazingly well to lead the photons onto the photosensitive areas that are just 10 microns in size.

And the main reason Megaframe’s camera could be applied to this particular field is that it is incredibly cheap. “CMOS, or complementary metal-oxide-semiconductor, is very cheap to produce, once the design has been mastered,” explains Edoardo Charbon, coordinator of the Megaframe project.

But by far the most exciting result from Megaframe’s work will be the science that it will enable in dozens of unrelated fields, everything from molecular biology and medicine, to automotive and aeronautic testing and development, entertainment and 3D television, and many others.


It means the technology could be applied in any number of devices and areas where it would serve a useful purpose. The utility of the camera has just begun to be explored.

That is because the camera does three things very well. It can detect very low light levels, ultimately a single photon, it can detect a photon a million times a second and, crucially, it can detect when the photon hits the detector to an incredible degree of accuracy. The Megaframe chip has a time resolution that ranges from 50-200 picoseconds.

The team achieved that by combining SPADs, or Single-Photon Avalanche Diodes, high-precision chronometers and microlenses into each pixel, and then created a 32 by 32 array of these pixels. It was an incredible demonstration of miniaturisation.

This accuracy is an enormous boost to ‘time-of-light’ 3D video. Time of light judges the depth of field, or the distance of a particular object in a scene, by the time it takes light fired at the object to be reflected back. It is incredibly accurate when well done, but it requires enormous precision.

The Megaframe camera has that precision. “If you want to have a depth resolution of a few millimetres, you have to have a time resolution of a few picoseconds. That is another case in which time resolution is very important,” explains Charbon formerly with EPFL and now at TU Delft.

Megaframe has that resolution and Charbon believes that 3D cameras based on SPAD technology could be appearing in mobile phones in the next five years. A similar technology could appear in cars, too, and be used for 3D parking cameras.

Second phase

For now, the team is concentrating on the second phase of the EU-funded project, where they hope to expand their pixel array from a 32 by 32 array to a much larger one. It will dramatically increase the area that can be filmed at any one time.

Charbon would also like to see another phase to cover a series of possible applications and develop prototypes for those applications.

He predicts the project’s work will find its way into real-world applications soon. “The 3D camera could be on your mobile phone in five years, and biomedical applications in 10 years, or perhaps even less,” he emphasises.

In the meantime, Megaframe is always interested in meeting potential research partners, and is particularly keen to meet investors who are interested in developing the 3D camera technology.

Megaframe had a budget of just €2.46 million, with €1.85 million provided by the European Union. Given all the innovation, the vast potential applications and its world leadership in the field, the project has proven to be great value for money.

 

[Tom.Wong]

and i bet someone will still complain about rolling shutter in their microscope! jk, but yeh, possibilities with this when you increase the sensor sized based on this technology is gonna be intense...

 

[Milan Nikolic]

This is very interesting!! But, it terms of camera makers, what you could offer: CMOS image sensor in 24K (for example) for 5.000 frames per second ? Please, make it more clear along those lines.
Thanks for info.

 

[Erik Widding]

In a pixel that is 50 microns-by-50 microns with a lot of functionality in it, only maybe 1% of the pixel can be used to detect light which has a photosensitive areas that are just 10 microns in size.

So basically, it can see a single photon if it happens to fall within the photo sensitive area. Assuming an even distribution of the photons hitting the surface of the sensor, it will miss the other 99...

Alternately, you bascically have the area equivalent of a 5 by 5 micron pixel on a 50 by 50 micron pitch...

I'm not that impressed, yet. Maybe the next generation will prove more interesting.

 

[Iannis Holwech]

So basically, it can see a single photon if it happens to fall within the photo sensitive area. Assuming an even distribution of the photons hitting the surface of the sensor, it will miss the other 99...

Alternately, you bascically have the area equivalent of a 5 by 5 micron pixel on a 50 by 50 micron pitch...

I'm not that impressed, yet. Maybe the next generation will prove more interesting.

Not exactly.
Each pixel has a lens that leads the photons onto the photosensitiv earea.
So the size of the "photon catcher lens" is likely to catch "all" photons hitting the lens.

As described here in an article that got lost when I first posted this and is now added to the OP.

“There were two more very important challenges. One was the optical challenge. If you have a pixel that is 50 microns-by-50 microns, with a lot of functionality in it, the area that is sensitive to photons can only be a small proportion of this pixel. So you can imagine that only maybe 1% of the pixel can be used to detect light,” Charbon explains.

The consortium included a partner specializing in microlenses, which are able to concentrate the light to the sensitive element. It was a complex task, but the high-precision lenses work amazingly well to lead the photons onto the photosensitive areas that are just 10 microns in size.

http://cordis.europa.eu/ictresults/index.cfm?section=news&tpl=article&BrowsingType=Features&ID=90916

The breakthrough as I understands it, is both the small size of the photosites which will make it possible to make very high resolution sensors.

But maybe more important, that being able to record photons that hits the photosites at a very high speed you will be able to makes CMOS sensors that has very much higher light sensitivity than todays sensors.

That is because the camera does three things very well.

It can detect very low light levels, ultimately a single photon,

it can detect a photon a million times a second,

and crucially,
it can detect when the photon hits the detector to an incredible degree of accuracy.

The Megaframe chip has a time resolution that ranges from 50-200 picoseconds.

http://cordis.europa.eu/ictresults/index.cfm?section=news&tpl=article&BrowsingType=Features&ID=90924

 

[Jannard]

The problem with this type of sensor is noise.

Jim