Turning Thoughts Into Words

I recently read an article written by Alan Mozes, a HealthDay Reporter, on the mind-reading AI that turns thoughts into spoken words.  A team of researchers have used artificial intelligence (AI) to turn the brain signals into computer-generated speech.  Five epilepsy patients assisted with the accomplishment of this feat.  They had all been outfitted with various types of brain electrodes as part of their seizure treatment.  Researchers were allowed to conduct very sensitive brain monitoring, called electrocorticography.

The results represent a major leap towards the goal of brain-to-computer communication.  There were prior efforts focused on simple computer models that were able to produce audio that sound similar to the original speech, but not intelligible at all, according to study author Nima Mesgarani.  Mesgarani is an associate professor with Columbia University’s Zuckerman Mind Brain Behavior Institute in New York City.

State-of-the-art AI was used in the new study to reconstruct the sounds from the brain that were more intelligible compared to previous research, which is a huge milestone they weren’t sure could be reached.

Each participant listened to short stories and number lists while brain activity was tracked while being read to by four different speakers.  Brain patterns were recorded while the patients listened, then fed into a computer algorithm without any indication of which pattern matched which number.  The AI program went to work cleaning up the sound.  This technology is used by Amazon Echo and Apple Siri.  They resulted in a final product of robotic-voiced audio tracks of both male and female that articulated each number between Zero and nine.

A group of 11 listeners found the computer-generated sounds as recognizable about 75% of the time.  The algorithm is the first to generate a sound actually intelligible to human listeners.  The ultimate goal is to develop technologies that can decode the internal voice of a patient who is unable to speak.  This also means better brain-computer interfacing that would open up whole new platforms for man-machine communications.

Future tests will focus on more complex words and sentence structure and they hope the system could be part of an implant similar to those worn by some epilepsy patients that translates the wearer’s imagined voice directly into words.

This breakthrough is an important step along the pathway of decoding the brain patterns that underlie thinking.  It provides hope to people who have difficulty communicating thoughts due to injury or disease.  The findings can be found in the journal, Scientific Reports.

Dr Fredda Branyon