Keeping up with the conversation
Most people find it difficult to concentrate on a specific voice in a busy environment, but for those who are hard of hearing it's especially challenging. Now, however, a new type of hearing aid, developed with the assistance of Fraunhofer researchers, is designed to render speech more intelligible against a background of noise, thereby making it easier to follow a single speaker.
According to the German Association of the Deaf (DSB), some 15 million Germans are hard of hearing. Hearing impaired people can find it difficult to follow a discussion in a loud environment, particularly when the conversation involves several people. Often they have trouble picking out individual voices. This is because they are unable to make use of what is known as the cocktail party effect: the ability to focus on one speaker and filter out any other background noises.
Current hearing aids are unable to establish the special link between the ear and the brain that accounts for the process of selective hearing. "For people with normal hearing, the connection between the ear and the brain is unimpaired," explains Dr. Axel Winneke, research fellow at the Fraunhofer Institute for Digital Media Technology IDMT in Oldenburg. "This is how they know which direction to face whenever they hear a signal. For hearing-aid wearers, this ability is severely restricted. Even high-end devices are unable to pinpoint the source of an acoustic signal of interest. We therefore need to retrieve this information from the brain. Using an electroencephalograph (EEG), we can analyze the brain activity of a hearing impaired person and determine who it is they are listening to." This EEG analysis is conducted by Fraunhofer IDMT-HSA and the University of Oldenburg. Together with partners from industry and research, Winneke and colleagues are developing a system designed to improve the intelligibility of speech in busy environments for the hard of hearing. This technology is based on a combination of EEG, audio-signal processing, and electrostimulation of the auditory cortex. It functions as follows: an EEG measures brain activity in order to determine the direction in which the hearing impaired person is trying to hear—i.e., toward the source of the signal—thereby forming an interface between the brain and a microprocessor. This information is transmitted to the hearing aid, which focuses a directional microphone—known as a beamformer—in the corresponding direction. This beamformer amplifies the specific audio signal that the hearer is trying to focus on, simultaneously filtering out all other sources of noise, including other voices. The third element of the system is a process from the field of neuroscience called transcranial electrostimulation (tES). This employs a very low electrical current to stimulate the auditory cortex on the basis of this voice signal, thereby improving the intelligibility of the person talking. The stimulation process and requisite hardware are being developed by the company neuroConn GmbH in partnership with the University of Oldenburg.
Hearables of the future
Design studies have already been carried out in order to visualize what these new hearables might look like. The concept and the design are intended to emphasize—rather than conceal—the brain's interaction with the device. At the same time, the design seeks to highlight the benefits for the wearer, thereby consciously steering against the still widespread stigma attached to the wearing of a hearing aid. In the future, components developed for the project, including sensor technology, could be incorporated in a wearable headpiece. Another option would be to retrofit existing hearing aids with these new modules and an EEG sensor. "The current prototype is not yet available in the form of a portable hearing aid," Winneke explains. "For that, it still needs to be substantially miniaturized." Early trials with the device, which were conducted with people without a hearing impairment, have shown that the principle works well. Studies with people hard of hearing are now being planned.
Mobile neurotechnologies
A wearable EEG headpiece could also be used for other applications, such as measuring the listening effort required by employees in the workplace. The technology could also be used in a medical context—in neurology, for example, to monitor conditions such as epilepsy. "And using a wearable EEG device would make it possible to observe patients outside of the clinical environment," says Winneke. "In the mEEGaHStim project, we use the technology to measure brain activity in order to control a hearing aid, but it could also be used to analyze brain activity in people with neurological disorders." Winneke belongs to the Mobile Neurotechnologies group at Fraunhofer IDMT. This group is working on EEG-based multisensor platforms for use in concrete scenarios where the analysis of brain activity can provide support—for example, in medical applications or safety-critical work situations.
