ArrowArtboardCreated with Sketch.Title ChevronTitle ChevronIcon FacebookIcon LinkedinIcon Mail ContactPath LayerIcon MailPositive ArrowIcon PrintIcon Twitter
Technology

SoftBank to offer AI-based, real-time sign language translation

Japan mobile carrier aims to make system available to public by 2024

Softbank's sign language app uses AI to convert signs to Japanese text, allowing those with hearing impairments to converse with those who don't know sign language.  (Source photos by Nikkei)

TOKYO -- SoftBank will start providing an AI-based sign language translation service as soon as 2024 using a smartphone app that can turn 5,000 signs into Japanese text in one second or less, Nikkei has learned. 

There are more than 300,000 people with hearing or speech impairments in Japan. SoftBank hopes the sign language translation service can promote smooth communication between hearing people who do not understand sign language and those with hearing impairments.

The system employs artificial intelligence to detect sign language and translate it into Japanese during a video conversation. The AI also adds postpositional particles to make full sentences. When a hearing person speaks, the AI automatically converts the speaker's words to text on screen. The people having the conversation do not have to type, which allows them to enjoy communicating and to see each other's facial expressions.

SoftBank in 2017 began basic research on the system with Tokyo's University of Electro-Communications. It has also collaborated with a Tokyo-based AI startup Abeja, in which Google invests.

Abeja had the AI learn from 50,000 sign language videos, which enabled it to detect the characteristic movements of each signed word. SoftBank aims to develop the service in multiple languages in the future.

Newly developed artificial intelligence from SoftBank can translate sign language into Japanese text in under a second using a smartphone app. (Photo courtesy of Softbank Group)

The Japanese telecom company is offering the service on a trial basis to organizations for disabled people in Tokyo and Fukushima Prefecture since April, and a total of nine municipalities and organizations have started using the service.

SoftBank will begin offering the service to health care facilities and public transportation companies within two years. It will roll out the service to the general public free of charge as early as 2024.

The company is now focused on improving the accuracy of the system. As with spoken language, people who communicate using sign language talk at varying speeds. The position of their hands and arms also varies. The AI can translate signs with more than 90% accuracy, but if the system is insufficiently trained, accuracy can fall to less than 50%. To raise the rate of sign recognition, it is necessary to read the movements of more than 100 people per word.

SoftBank in July began offering an app through which anybody can register samples of sign language. Tokyo-based AI company Preferred Networks provided the technology to automatically generate computer graphics as a sample from sign language videos. By showing the sample, the company is now calling for more people who do not understand sign language to register the samples as volunteers.

Sponsored Content

About Sponsored Content This content was commissioned by Nikkei's Global Business Bureau.

You have {{numberArticlesLeft}} free article{{numberArticlesLeft-plural}} left this monthThis is your last free article this month

Stay ahead with our exclusives on Asia;
the most dynamic market in the world.

Stay ahead with our exclusives on Asia

Get trusted insights from experts within Asia itself.

Get trusted insights from experts
within Asia itself.

Try 1 month for $0.99

You have {{numberArticlesLeft}} free article{{numberArticlesLeft-plural}} left this month

This is your last free article this month

Stay ahead with our exclusives on Asia; the most
dynamic market in the world
.

Get trusted insights from experts
within Asia itself.

Try 3 months for $9

Offer ends October 31st

Your trial period has expired

You need a subscription to...

  • Read all stories with unlimited access
  • Use our mobile and tablet apps
See all offers and subscribe

Your full access to Nikkei Asia has expired

You need a subscription to:

  • Read all stories with unlimited access
  • Use our mobile and tablet apps
See all offers
NAR on print phone, device, and tablet media

Nikkei Asian Review, now known as Nikkei Asia, will be the voice of the Asian Century.

Celebrate our next chapter
Free access for everyone - Sep. 30

Find out more