Intuitive machine control using speech recognition

Researchers at the Fraunhofer Institute for Digital Media Technology IDMT in Oldenburg have developed a speech recognition solution for use in industrial manufacturing.

The system works reliably even in noisy environments and can be flexibly adapted to the user's needs. Employees use intuitive voice commands on the factory floor, allowing them to keep both hands free and work much more efficiently. The Hannover Messe Preview on February 15, 2023, will provide a first look at the research project, including a hands-on demonstration.

The practice of using voice-controlled machines in manufacturing was previously considered prone to errors, and as a result, it was hardly ever adopted. Now, Fraunhofer IDMT in Oldenburg has developed a solution that allows machines to be reliably controlled using voice commands.

Even on a noisy factory floor, the speech recognition system functions reliably. Employees speak into a wireless headset or a stationary microphone—and in the future, they could use smart hearable technology, which the institute branch for Hearing, Speech and Audio Technology HSA is also working to develop. Loud ambient noise is almost entirely tuned out using a combination of directional microphones and an effective noise-canceling system.

Less walking—and free hands

Marvin Norda, the project manager for Voice Controlled Production at Fraunhofer IDMT, says, "For the first time, our technology is enabling people to control machines in manufacturing using voice commands in a reliable, intuitive way. For manufacturing companies, this means increased efficiency and lower costs."

The technology also has advantages for employees, as contactless machine operation is safe and hygienic for people. When employees are controlling multiple machines, it reduces how far they need to walk—they can use a mobile device to issue voice commands to the equipment from a distance.

They have both hands free, allowing them to place a workpiece in position in the work area while giving the robot an instruction such as "lower the arm" or "grip the workpiece" at the same time.

Controlling a machine through voice commands is more efficient than using a control panel or touch panel—clicking through nested menu structures is a cumbersome and error-prone process, whereas direct voice commands involve simple instructions.

"The speech recognition technology can handle hundreds of individual commands depending on the application and is not limited to a particular voice. New or modified commands can be quickly added and trained into the system," adds Mr. Norda.

The researchers from Oldenburg can draw on many years of experience and expertise in researching and developing audio technologies and speech recognition software—something that also benefits the more than 50 member companies of AiP (Audiotechnologie für die intelligente Produktion, audio technology for intelligent production), an industrial working group founded by Fraunhofer IDMT in Oldenburg.

"We are developing adaptable system solutions for industry customers. The voice control system can be configured to meet their individual needs and quickly put into operation," says Mr. Norda. The speech recognition software can be integrated in the cloud or the company server, depending on the company's requirements. It is also possible to incorporate a mini PC or even integrate the system into a machine's programmable logic controller (PLC) without any issues.

The solution, which is sponsored by the Ministry of Science and Culture of Lower Saxony and the Volkswagen Foundation, is now ready for the market and is already being tested by some industry customers. Experts from Fraunhofer IDMT will provide a first look into the project to a specialist audience at the Hannover Messe Preview on February 15, 2023. At the Hannover Messe from April 17–21, 2023, visitors will be able to experience the technology and explore its possibilities live at the Fraunhofer booth in Hall 16, booth A12. Demonstrations will take place using a voice-controlled milling machine.