Intensity control of projectors in parallel: A doorway to an augmented reality future

One challenge to adopting augmented reality (AR) in wider applications is working with dynamic objects, owing to a delay between their movement and the projection of light onto their new position. But, Tokyo Tech scientists may have a workaround. They have developed a method that uses multiple projectors while reducing delay time. Their method could open the door to a future driven by augmented reality (AR), helping us live increasingly technology-centered lives.

Technological advancements continue to redesign the way we interact with digital media, the world around us, and each other. AR, which uses technology to alter the perception of objects in the real world, is unlocking unprecedented landscapes in entertainment, advertising, education, and across many other industries. The use of multiple projectors plays an important role in expanding the usage of AR, alongside a technique called projection mapping. However, an obstacle to the widespread adoption of AR is the application of this method to moving, or "dynamic," targets without the loss of immersion in the AR space.

This technique, known as dynamic projection mapping, relies on a blend of cameras and projectors that visually detect target surfaces and project onto them, respectively. A critical aspect is the need for high speed in information transfer and low "latency," or delay between detection and projection. Any latency leads to a misalignment of the projector, which affects our perception and reduces the effectiveness of the AR space.

Other issues like changes in shadowing and target overlap are solved easily by using multiple projectors. However, the addition of new projectors correspondingly drives up the latency. This is a result of the need to calculate the intensity at every pixel simultaneously for every frame of a moving scene. Simply put, more projectors lead to longer and more complex calculations. The latency is a massive hurdle towards AR taking a true foothold in broader applications across society.

Thankfully, a team of scientists at Tokyo Institute of Technology (Tokyo Tech), led by Associate Professor Yoshihiro Watanabe, might just have the necessary solution. They have developed a novel method to calculate the intensity of each pixel on a target in parallel, reducing the need for a single large optimization calculation. Their method relies on the principle that if pixels are small enough, they can be evaluated independently. While based on an approximation, their results, published in IEEE Transactions on Visualization and Computer Graphics, suggest that they could achieve the same quality of images as conventional, more computationally expensive methods, while drastically increasing the mapping speed and thereby reducing the latency.

"Another advantage of our proposed method is, as there is no longer need for a single global calculation, it allows the use of multiple rendering computers connected through a network, each only controlling a single projector," explains Dr. Watanabe. "Such a network system is easily customizable to incorporate more projectors, without major sacrifices to the latency."

This new method can allow large spaces with many projectors for efficient dynamic projection mapping, taking us a step closer to broader AR applications, as Dr. Watanabe describes: "The presented high-speed multi-projection is expected to be a major part of important base technologies that will advance spatial AR to derive more practical uses in our daily life."