A noninvasive brain-computer interface capable of converting a person's thoughts into words could one day help people who have lost the ability to speak as a result of injuries like strokes or conditions including ALS.
In a new study, published in Nature Neuroscience today, a model trained on functional magnetic resonance imaging scans of three volunteers was able to predict whole sentences they were hearing with surprising accuracy—just by looking at their brain activity. The findings demonstrate the need for future policies to protect our brain data, the team says.
Speech has been decoded from brain activity before, but the process typically requires highly invasive electrode devices to be embedded within a person's brain. Other noninvasive systems have tended to be restricted to decoding single words or short phrases.
This is the first time whole sentences have been produced from noninvasive brain recordings collected through fMRI, according to the interface's creators, a team of researchers from the University of Texas at Austin. While normal MRI takes pictures of the structure of the brain, functional MRI scans evaluate blood flow in the brain, depicting which parts are activated by certain activities.
[...] Romain Brette, a theoretical neuroscientist at the Vision Institute in Paris who was not involved in the experiment, is not wholly convinced by the technology's efficacy at this stage. "The way the algorithm works is basically that an AI model makes up sentences from vague information about the semantic field of the sentences inferred from the brain scan," he says. "There might be some interesting use cases, like inferring what you have dreamed about, on a general level. But I'm a bit skeptical that we're really approaching thought-reading level."
It may not work so well yet, but the experiment raises ethical issues around the possible future use of brain decoders for surveillance and interrogation. With this in mind, the team set out to test whether you could train and run a decoder without a person's cooperation. They did this by trying to decode perceived speech from each participant using decoder models trained on data from another person. They found that they performed "barely above chance."
This, they say, suggests that a decoder couldn't be applied to someone's brain activity unless that person was willing and had helped train the decoder in the first place.
"We think that mental privacy is really important, and that nobody's brain should be decoded without their cooperation," says Jerry Tang, a PhD student at the university who worked on the project. "We believe it's important to keep researching the privacy implications of brain decoding, and enact policies that protect each person's mental privacy."
Journal Reference:
Tang, J., LeBel, A., Jain, S. et al. Semantic reconstruction of continuous language from non-invasive brain recordings. Nat Neurosci (2023). https://doi.org/10.1038/s41593-023-01304-9
(Score: 3, Interesting) by jb on Thursday May 04, @04:44AM (2 children)
I wonder whether it makes any difference what language the subject is "thinking in" at the time (at least, for those of us who are verbal thinkers)...
If not, then they may be close to implementing the universal translator envisaged in so many sci-fi books & movies.
But I suspect that language would matter, as semantics tend to differ widely between languages.
(Score: 2, Insightful) by Anonymous Coward on Thursday May 04, @06:21AM (1 child)
I'm vocalizing each word I'm typing in my head. They could just be converting the virtual sounds into text.
An advanced version of this could pull audio out of your head so it could tell what song you're thinking of or that you're imagining text read in Arnold Schwarzenegger's voice.
I can think of some ways to get around that.
(Score: 2) by acid andy on Thursday May 04, @02:14PM
Me too but best not to give them ideas.
That won't worry them. It's not like it stopped the roll out of polygraphs or facial recognition cameras, is it?
Master of the science of the art of the science of art.
(Score: 0) by Anonymous Coward on Friday May 05, @07:06PM
Thinking of the cartoon where they invent a machine that allows dogs to talk. Biscuits!