New lens-less camera for Computer Vision

Computer vision requires a lot of computing to process high-resolution images. Therefore, the image resolution of many datasets is very small. Recently, engineers have used a glass, a photoelectric detector, and some software to develop a "fluoroscopy" camera. This camera without a lens can take very low-resolution images, and Abstract The main contour of the object. Therefore, images taken using this camera can greatly reduce the computing power required by computer vision. At the same time, they are also developing technologies that use machine learning algorithms to process more complex images.

Previously, cameras were equipped with their own specialized devices, such as lenses and films, and they had to go to the photo studio. Later, mobile phones, tablets, laptops, and video game consoles all had their own camera functions. Now, the camera seems to become as inconspicuous as the glass one day, and even it no longer needs a lens.

According to new research, photodetectors pressed on the edge of a window can detect reflected light inside the glass, just as light signals pass through the optical cable. Some clever processing of the detected tiny light can make the glass panel a huge camera lens.

Although granular images produced by such cameras (IMAGINE pixel-based, distorted, and low-resolution images) cannot be compared with traditional cameras for the moment. However, for many computer vision tasks, the resolution of a window glass or a car windshield is sufficient to meet the information required by image processing algorithms or neural networks.

This is their effect on the LED array image:

The left column is the original image, the middle column is the input received by the image sensor, and the right column is the computer-reconstructed image.

Rajesh Menon, associate professor of electrical and computer engineering at the University of Utah, said many of the images taken by modern cameras are actually invisible to the naked eye. They can only be seen by camera algorithms or automated vehicle image sensors used to handle security, and the images that we cannot see with the naked eye are becoming more and more.

Therefore, Menon asked, "If the machine has more images and videos than humans can see, why don't we consider re-designing the camera for the machine? This allows us to look at the camera from a non-human perspective .」

In other words, computer vision algorithms do not always require high resolution and high image fidelity as the human eyes do. They can get a lot of information from the "transparent, lens-less camera" of Menon and ganghun Kim, even if the image quality is not high, the cost and occupied area will be greatly reduced. Their technology has been patented and has no requirements for the visual media itself, such as glass, plastic, or organic glass.

They connect a ready-made photoelectric detector (8 Resolution, 640x480 pixels) to the edge of the plexiglass, smooth connection to the edge and are ready to connect to the imaging device. They then placed reflective bands around the rest of the plexiglass. Menon says they can imaging without a reflector, but the reflector increases the signal-to-noise ratio.

For this concept verification, the experiment only needs to maintain a simple field of view. They placed a 32x32 LED light in front of the pane. Then, when 1024 bundles of light are individually illuminated, they observe the signal reaching the photodetector. Therefore, any image from the LED array, at least in the first-level approximation, will be only a linear combination of each separately lit LED lamp signal.

If a machine can see images and videos that humans can see, why don't we consider re-designing a camera for the machine? -- University of Utah, Rajesh Menon

Menon said that in this project, they developed a traditional signal processing algorithm that can reconstruct the image using the signals received by the photodetectors. They call this a "inversion problem" because their algorithms use complex and chaotic signals as input and use the photon detected by the detector to generate possible targets.

「 We are detecting the distribution of "photon" in the space corresponding to a specific target, 」 he said, 「 we like to see a one-to-one graph. This is exactly how the camera works. Here we use a one-to-multiple graph, so we need to solve the inversion problem .」

This is why these glass panel "cameras" perfectly fit with computer vision-related projects. Image quality and information that can be decomposed may be good enough for computer vision, but it cannot (or may never) replace traditional cameras based on lenses and photos.

Menon mentioned that his team is currently developing a machine learning algorithm to learn more complex images, such as handwritten numbers that can be detected and recognized as numerical values. He pointed out that the technology may be first applied in VR or AR glasses. The image generation and Image Display hardware of these glasses are already very heavy, and eye tracking cameras may be cumbersome. Therefore, it would be nice to use this camera-free fluoroscopy camera to track users' eye movements and obtain high-quality information.

It sounds ironic to use a technology with a quality far below the current best level as a breakthrough. But Menon says it may be a big step for us to change our mindset and redesign "high quality enough" technology in AI and image processing systems. Just like the eyes of flies, the most important thing in the AI world is not the quality of individual data, but the proliferation of data sources.

This is also why cameras and glass panels are getting closer and closer in the future, at least for computer vision algorithms.

Thesis: Computational imaging enables a "see-through" lens-less camera

Address: https://www.osapublishing.org/oe/abstract.cfm? Uri = oe-26-18-22826

Abstract: traditional cameras mask scenes that need to be recorded. Here we place an image sensor (without a lens) on the edge of a transparent window and observe the object image through this window. In this process, we can first collect scattered light through an image sensor, and then reconstruct the image by solving the inverse problem of light scattering. Therefore, we can form a simple image with a spatial resolution of 150 line-pairs/mm at a distance of 0.1mm and a focal length of 10mm. We further demonstrate the imaging of two types of objects: LED arrays and traditional LCD screens. Finally, we demonstrated color and video imaging.

Link: https://spectrum.ieee.org/tech-talk/computing/software/a-lensless-camera-built-specially-for-ai-and-computer-vision-programs-sorry-humans

New lens-less camera for Computer Vision