Neuro-engineers at New York’s Columbia University say they have created a system that can translate human thoughts into recognizable speech, which would revolutionize not just medicine but communication.
By monitoring brain activity of subjects, the researchers at Columbia’s Mortimer B. Zuckerman Mind Brain Behavior Institute were able to train artificial intelligence to translate thought patterns into intelligible sentences, says a paper published on Tuesday in the journal Scientific Reports. The authors see patients with speech impaired by disease or trauma as first adopters of the incipient technology.
“We've shown that, with the right technology, these people's thoughts could be decoded and understood by any listener,” said Dr. Nima Mesgarani, the paper’s senior author.
After the researchers’ early attempts to translate brain activity into recognizable speech failed, they turned to a computer algorithm that can generate speech, called a vocoder. The algorithm improves the more it is “trained” by recordings of human speech.
Researchers Translate Brain Signals Directly Into Speechhttps://t.co/qEStGOoPOW
This breakthrough, which harnesses the power of speech synthesizers and artificial intelligence, could lead to new ways for computers to communicate directly with the brain. #ai #BCI #speech #science
— Neuroscience News (@NeuroscienceNew) January 29, 2019
“This is the same technology used by Amazon Echo and Apple Siri to give verbal responses to our questions,” said Dr. Mesgarani, who is also a professor at Columbia's Fu Foundation School of Engineering and Applied Science.
The vocoder was trained to interpret brain activity with the help of Dr. Ashesh Dinesh Mehta, a neurosurgeon at Northwell Health Neuroscience Institute in Long Island, and the paper’s co-author.
“Working with Dr. Mehta, we asked epilepsy patients already undergoing brain surgery to listen to sentences spoken by different people, while we measured patterns of brain activity,” said Dr. Mesgarani. “These neural patterns trained the vocoder.”'
Read more: Thought-to-speech: Scientists say their AI can read out what happens in your brain