Neuroscientists are racing to turn brain waves into speech

Neuroscientists are striving to give a voice to people unable to speak in a fast-advancing quest to harness brainwaves to restore or enhance physical abilities.

Researchers at universities across California and companies, such as New York-based Precision Neuroscience, are among those making headway toward generating naturalistic speech through a combination of brain implants and artificial intelligence.

Investment and attention have long been focused on implants that enable severely disabled people to operate computer keyboards, control robotic arms, or regain some use of their own paralyzed limbs. But some labs are making strides by concentrating on technology that converts thought patterns into speech.

“We are making great progress—and making brain-to-synthetic voice as fluent as chat between two speaking people is a major goal,” said Edward Chang, a neurosurgeon at the University of California, San Francisco. “The AI algorithms we are using are getting faster, and we are learning with every new participant in our studies.”

Chang and colleagues, including from the University of California, Berkeley last month published a paper in Nature Neuroscience detailing their work with a woman with quadriplegia, or paralysis of the limbs and torso, who had not been able to speak for 18 years after suffering a stroke.

She trained a deep-learning neural network by silently attempting to say sentences composed using 1,024 different words. The audio of her voice was created by streaming her neural data to a joint speech synthesis and text-decoding model.

The technique reduced the lag between the patient’s brain signals and the resultant audio from the eight seconds the group had achieved previously to one second. This is much closer to the 100-200 millisecond time gap in normal speech. The system’s median decoding speed was 47.5 words per minute, or about a third the rate of normal conversation.

Many thousands of people a year could benefit from so-called voice prosthesis. Their cognitive functions remain more or less intact, but they have suffered speech loss due to stroke, the neurodegenerative disorder ALS, and other brain conditions. If successful, researchers hope the technique could be extended to help people who have difficulty vocalizing because of conditions such as cerebral palsy or autism.

The potential of voice neuroprosthesis is beginning to trigger interest among businesses. Precision Neuroscience claims to be capturing higher resolution brain signals than academic researchers, since the electrodes of its implants are more densely packed.

The company has worked with 31 patients and plans soon to collect data from more, providing a potential pathway to commercialization.

Precision received regulatory clearance on April 17 to leave its sensors implanted for up to 30 days at a time. That would enable its scientists to train their system with what could within a year be the “largest repository of high resolution neural data that exists on planet Earth,” said chief executive Michael Mager.

The next step would be to “miniaturize the components and put them in hermetically sealed packages that are biocompatible so they can be planted in the body forever,” Mager said.

Elon Musk’s Neuralink, the best-known brain-computer interface (BCI) company, has focused on enabling people with paralysis to control computers rather than giving them a synthetic voice.

An important obstacle to the development of brain-to-voice technology is the time patients take to learn how to use the system.

A key unanswered question is how much the response patterns in the motor cortex—the part of the brain that controls voluntary actions, including speech—vary between people. If they remained very similar, machine-learning models trained on previous individuals could be used for new patients, said Nick Ramsey, a BCI researcher at University Medical Centre Utrecht.