Woman With Paralysis Speaks Again Through AI Avatar Surrogate

Image generated on AI

Ann Johnson spent the last 18 years unable to speak after a cataclysmic stroke had left her paralyzed.

Recently, though, she successfully held a conversation with her husband, telling him, “Anything is possible,” when he asked for her thoughts regarding the Toronto Blue Jays’ chances for the 2023 MLB season.

Anything is possible, indeed. Johnson’s words were conveyed through a brain implant and communicated via an avatar on a computer screen. The revolutionary artificial intelligence setup was developed by researchers at the University of California, San Francisco, and the University of California, Berkeley, who have outlined their remarkable findings in the Nature scientific journal.

An implant was positioned on top of the part of the brain in charge of speech and language, picking up on her brain signals. These were directly translated and expressed by an animated face resembling the woman, who chose the avatar herself. To mimic her voice, the team processed audio from her wedding toast 20 years ago.

It’s the first time speech and facial expressions have been directly synthesized, the New York Times reports.

Instead of bombarding the algorithm with words from a dictionary, the researchers trained it to recognize phonemes, or sound units, that are building blocks for all kinds of words. “‘Hello,’ for example, contains four phonemes: ‘HH,’ ‘AH,’ ‘L’ and ‘OW,’” they explain in a press release.

The technology could translate brain activity into audio waveforms, which became verbalized through an onscreen character that could smile and show emotion. It was a worthy middleman for Johnson, a mother and former teacher and volleyball coach who was often described as an animated person.

The group’s leader, Dr Edward Chang, the chairman of neurological surgery at the University of California, San Francisco, tells the Times that the team just wants to “restore who people are.”

They never intended to add the avatar or audio components, but when the technology superseded their expectations, they decided to just “go for it,” says David Moses, the project manager. The team reached out to facial animation company Speech Graphics, which converted the woman’s muscle movements into moving facial features.

“Do not make me laugh,” Johnson told her husband during their chat, revealing her vivacious side once again.

As impressive as this all is, the technology isn’t perfect. While the average conversation is made up of 160 words per minute, Johnson communicated at a rate of 78. The system also sometimes wrongly decoded her speech, with the talking head proclaiming, “Maybe we that name,” when the speaker wanted to say, “Maybe we lost them.”

For now, the setup can only work when the implant is connected to a computer. The team, however, is working on a wireless version.

Regardless, this a dramatic leap for those who have lost their ability to verbalize their thoughts after a stroke or being struck with neurological conditions like cerebral palsy.

In an email to the Times, Johnson says she now feels “like a whole person again.”

[via New York Times and NBC News, cover image generated on AI]