Mind-to-Speech AI Translates Brain Waves Into Speech

Scientists have made a major breakthrough in restoring natural speech for people with paralysis, using AI-powered brain implants to turn brain waves into spoken words in real-time.

Scientists at Radboud University together with UMC Utrecht developed brain implant technology with AI that translates neural signals into spoken words at accuracy rates between 92-100%, as reported by Neuroscience News. The research published this week in Nature Neuroscience seeks to develop communication tools for paralyzed people.

The research used epilepsy patients with short-term brain implants to create associations between neural signals and verbal expressions. The development cuts down on the previous delay that made communication difficult for speech-impaired people.

“Our streaming approach brings the same rapid speech decoding capacity of devices like Alexa and Siri to neuroprostheses,” explained Gopala Anumanchipalli, a co-principal investigator of the study, as reported by New Atlas. “Using a similar type of algorithm, we found that we could decode neural data and, for the first time, enable near-synchronous voice streaming,” he added.

The system works by capturing brain activity through high-density electrodes placed on the brain’s surface. The AI then deciphers these signals, reconstructing words and sentences with remarkable accuracy.

In addition, a text-to-speech model trained on the patient’s pre-injury voice ensures that the generated speech sounds natural, and closely resembles their original voice.

One of the study’s authors, Cheol Jun Cho, explained how the system processes thoughts into speech: “What we’re decoding is after a thought has happened, after we’ve decided what to say, after we’ve decided what words to use and how to move our vocal-tract muscles,” as reported in New Atlas.

The breakthrough significantly improves on past technologies. Additionally this AI-powered system now provides one-second speech initiation which enables uninterrupted fluid conversations. This technology could be life-changing for people with severe paralysis, locked-in syndrome, or conditions such as ALS, by enabling their ability to communicate naturally with others.

However, despite these advancements, the technology still faces challenges. As explained in Neuroscience News, this AI requires extensive training on a person’s neural data, and it may not work effectively for those who lack prior speech recordings.

While the system can decode words, achieving completely natural pacing and expression remains difficult. Additionally, current models struggle to predict full sentences and paragraphs, focusing mainly on individual words.

Additionally, non-invasive versions using EEG helmets have lower accuracy, around 60%, compared to implanted electrodes, as noted in New Scientist.

Moving forward, researchers aim to enhance the system’s speech speed and expressiveness, making conversations feel even more lifelike.