Working with a team of friends, an IT major with spastic cerebral palsy developed an AI-based app that can help transcribe the voices of people with speech impairments into natural voices.

He is Nguyen Duc Thuan, 20, a student from Bac Ninh Province, northern Vietnam, which is about 30 kilometers from the capital Hanoi.

During his middle school years, Thuan was always ridiculed by his mischievous friends because of his neurological condition.

Sometimes, he was even beaten by some children who called him 'a paralyzed boy.'

"We hope the app we developed can help people with speech disabilities communicate more easily so that their lives have fewer difficulties," Thuan said. 

Accompanying the son

In the memory of Do Thi Hoai, Thuan's mother, her son's childhood meant a series of long days in the hospital.

Because he was born with spastic cerebral palsy, Thuan had to receive so many treatments to be able to go to elementary school. 

Because of his illness, Thuan often stayed at the National Hospital for Acupuncture in Hanoi for nearly 12 months a year.

He only took one week off a month to go home and had to return to the hospital for treatment.

In the early days at school, Thuan had little difficulty in learning only by drawing characters.

However, in the third grade, when he had to take notes more quickly, his characters turned into something like 'noodles' that he eventually could not read by himself. 

He tried to memorize the material and then wrote down as much as he could on later tests. 

"While his friends took about 50 minutes to think and 10 minutes to write down the answer on a math test, my son took 10 minutes to think and 50 minutes to write it down, quite the opposite," Thuan's mother recalled. 

Although his health is not too weak, he cannot sit on a chair in a stable posture and has issues walking more than a few steps. 

In addition, the muscles in his legs and arms become stiff, and his body is so wobbly on his feet that he is in danger of falling at any time.

"What I can do is encourage him to do his best to overcome the obstacles in life and also in his studies," Hoai said.

"I advised him to see the challenges as steps on the way to higher achievements."

The breakthrough came when Thuan was given an old computer by some friends who were co-workers of his father. The young man developed a special passion for programming. 

Hoai, who stood by her son in all circumstances, accompanied him almost every day to Bac Ninh High School for the Gifted in Bac Ninh City, where Thuan could pursue his passion.

The mother and her son patiently traveled from their home in Que Vo District to Bac Ninh City daily, despite the unpredictable weather, especially the heavy rain and biting cold on the worst days.

Their efforts finally paid off. As an 11th-grade student, Thuan won the third prize in the national exam for excellent students in computer science. 

Thanks to this achievement, he was awarded a place in grade 12 at Bac Ninh High School for the Gifted, where he had previously failed the entrance exam. 

'Speaking' on behalf of the disabled

Thuan spends a lot of time reading books on meditation and Buddhism, including many by the late Zen teacher Thich Nhat Hanh. 

His favorite, however, is The Alchemist (Portuguese: O Alquimista), a novel by Brazilian author Paulo Coelho, translated into Vietnamese by Le Chu Cau.

Thuan was given The Alchemist by a teacher, Do Duc Dong, whom he met a few years ago at a summer camp of the Ho Chi Minh City Youth Union in outlying Can Gio District.

Dr. Dong, an IT lecturer at Thuan's school, the University of Engineering and Technology under the Vietnam National University-Hanoi, advised him to try to 'develop' the coding in his brain accurately before presenting it on the computer to reduce the time for wrong calculations, which is harder for him than others because of his condition.

He also encouraged Thuan to join the research team, led by Associate Professor Dr. Le Thanh Ha, to find ways to recognize the voices of people with speech disabilities using AI software and convert them into normal voices.

"Considering Thuan's neurological condition, he has managed to do what is difficult even for an able-bodied person," Dong remarked.

"We can see that he must have remarkable strength and intelligence."

The software developed by Dong and his team can help transform the voices of people with speech problems into more natural ones with more rhythm.

The algorithms can 'self-learn' to effectively transcribe a person's voice based on personal data. So, if the software was used with each individual, it could work more accurately, according to Thuan.

Thuan wants to use the software for deaf-mutes, people with speech disorders, and those who, like him, cannot communicate normally with others. 

His team is still finalizing the software to integrate the app with smart devices such as smartphones and laptops. 

In order to carry out this project, Thuan had to acquire as much knowledge as possible, including foreign languages, AI skills, and advanced coding skills.

Right now, team members are working on a technological solution to regenerate the voices of people with speech impairments, a condition that often occurs in the elderly or disabled.

According to Thuan's research, there is now a lot of foreign software that supports people with speech disabilities, but none supports the Vietnamese language.

"The pronunciation of our language is quite complicated and has different accents," Thuan explained.

"Moreover, there is not enough data to 'train' AI-based software for it, which results in the app unable to distinguish between very similar words.

"Therefore, we are trying to find solutions to the problem."

Like us on Facebook or follow us on Twitter to get the latest news about Vietnam!