Intelligent programmable meta-imagers: A timely approach to task-specific, noise-adaptive sensing

Sensing systems are becoming prevalent in many areas of our lives, such as in ambient-assisted health care, autonomous vehicles, and touchless human-computer interaction. However, these systems often lack intelligence: they tend to gather all available information, even if it is not relevant. This can lead not only to privacy infringements but also to wasted time, energy, and computational resources during data processing.

To address this problem, researchers from the French CNRS came up with a concept for intelligent electromagnetic sensing, which uses machine-learning techniques to generate learned illumination patterns so as to pre-select relevant details during the measurement process. A programmable metasurface is configured to generate the learned patterns, performing high-accuracy sensing (e.g., posture recognition) with a remarkably reduced number of measurements.

But measurement processes in realistic applications are inevitably subject to a variety of noise. Noise fundamentally accompanies any measurement. The signal-to-noise ratio can be particularly low in indoor environments where the radiated electromagnetic signals must be kept weak.

Therefore, Chenqi Qian and Philipp del Hougne furthered previous research and presented an intelligent programmable computational meta-imager that not only tailors its illumination pattern to a specific information-extraction task like object recognition but also adapts to different types and levels of noise. They published an invited research article on their results in Intelligent Computing on Dec. 2, 2022.

"We hypothesize that the optimal coherent illumination patterns to be used by an intelligent programmable meta-imager to efficiently extract task-specific information from a scene will profoundly depend on the type and level of noise," the researchers said, pointing out that noise may profoundly impact the optimal meta-imager configurations because, besides latency constraints which limits the number of allowed measurements, noise also limits the amount of information that can be extracted from the scene.

"In this paper, we systematically explore how the combination of latency constraints and noise impacts intelligent multi-shot programmable meta-imagers," the researchers said. To evaluate their hypothesis, the researchers considered a prototypical object-recognition problem, for which they proposed a microwave computational programmable meta-imager system. Such systems could be deployed in indoor surveillance, earth observation, etc.

In their considered system, one microwave dynamic metasurface antenna (DMA) radiated a sequence of coherent wavefronts to the scene using a single transmitter, and a second DMA coherently captures the reflected waves using a single detector. A differentiable end-to-end information-flow pipeline was formulated, which comprised the programmable physical measurement process including noise as well as the subsequent digital processing layers.

The essential elements of this pipeline are the same for all wave-based information-extraction problems, including imaging, sensing, localization, and object recognition. "The only significant difference lies in the task-specific cost function that is to be optimized for good performance," they explained.

The same approach that the authors applied to object recognition can hence also be used in parameter-estimation problems such as localization. "This pipeline allows us to jointly inverse-design the programmable physical weights (DMA configurations that determine the coherent scene illuminations) and the trainable digital weights."

It is this joint optimization—task-specific end-to-end joint optimization of the trainable physical parameters and trainable digital parameters—that endows the measurement process with task awareness, such that it could discriminate between task-relevant and task-irrelevant information over the air in the analog domain.

The researchers tested the performance of this programmable meta-imager that generates a sequence of task-specific and noise-specific scene illuminations and found it advantageous over conventional compressed sensing with random configurations when the information that can be extracted from the scene is limited through latency constraints and/or noise. The performance gains for a signal-independent and a signal-dependent additive noise type were both demonstrated. The "macroscopic" features of the learned illumination patterns, namely, their mutual overlaps and intensities, were found to be intuitively understandable despite the "black-box" nature of the approach.

According to the researchers, the transition toward a system that self-adaptively detects the type and level of noise and updates accordingly its utilized sequence of DMA configurations without additional human input is straightforward. "We faithfully expect that our results can be transposed to information-extraction problems based on other wave phenomena (e.g., optics, acoustics, elastics, and quantum mechanics) and/or with other types of in-situ programmable measurement hardware," they concluded.