Avatars help deaf people to understand online text and interact with
29 August 2014
Computer scientists from Saarbrücken, Germany are developing
animated online characters to display text from web pages in sign
language. In the long term, the aim is that deaf people could use
the technique to communicate on online platforms via sign language.
To do this, users would only need readily available devices.
Sign language was first acknowledged as a separate language in
the sixties, which is quite recent. Similar to spoken language, it
evolved from different cultural backgrounds. Every country has its
own sign language with various dialects, which are based on
different rules than the spoken language. For the deaf, sign
language is their native language. Therefore, it is not easy for
them to learn spoken language, which is why they may struggle with
text reading and comprehension even after their graduation.
Although several websites provide video clips in which sign
language interpreters translate the text, much Internet content
remains cryptic for the deaf community. To inform deaf people
quickly in cases where there is no interpreter on hand, researchers
are working on a novel approach to provide content: avatars. These
animated characters could be used in the context of announcements at
train stations, or on websites.
“We have already gained initial experiences with avatars”,
explains Alexis Heloir, who is the leader of the Sign Language
Synthesis and Interaction research group at the Multimodal Computing
and Interaction Cluster of Excellence and also a researcher at the
German Center for Artificial Intelligence. “If we try to animate
them like human beings, deaf people have issues with understanding
The researcher assumes that this is caused by the greater variety
of emotional expressions of humans compared to avatars. To deal with
that problem, Alexis Heloir and Fabrizio Nunnari create avatars that
make more accentuated movements. The researchers are closely
cooperating with Peter Schaar, who is deaf and is a lecturer for
German sign language at the Saarland University Language Center and
the College of Engineering and Commerce in Saarbrücken.
“Our method should be inexpensive and easy to use so that every
member of the deaf community will be able to use it”, says Fabrizio
Nunnari. To capture the motions of deaf people, the scientists make
use of affordable cameras and sensors that are typically used by
teenagers for computer games. A computing method transfers the
movements of the entire body onto the avatar. In the long term, the
researchers want to create a collection of short sign language
sequences that can be used by the deaf to interact on the web.
Computer scientists from Saarbrücken in Germany
have developed an approach where animated online characters (upper
row) display content in sign language.
collaborate with Peter Schaar (lower row) who is deaf and a lecturer
for sign language. Credits: AG Heloir
See the Sign Synthesis project website: