It’s a world first. A sign language spoken language dictionary has just been released. It is the result of research in linguistics and artificial intelligence at UNamur.
In Belgium it is sign language is practiced by four thousand people. until now, no dictionary was dedicated to him, to no singing language. It is finished: UNamur Linguistics and Computer Science researchers just put one online Dictionary of the sign language of French-speaking Belgium – Frenchs. Financially supported by the Baillet Latour Fund, it has been designed for hearing families with a deaf child, signers who need to switch to French and in general for anyone who occasionally needs to bridge the gap between French and sign language, and for the bilingual classes that make up this project have implemented.
A Benedictine task that occupied UNamur’s linguists for almost twenty years.
“Until the birth of this first sign language French dictionary We teachers had to play the role of repertoire for the children. Since sign language cannot be written, we had to make videos that represent the sign of a new word, put it in context and attach these videos via QR code to the sheets distributed in the class. It was a monumental task,” explains Magaly Ghesquière, coordinator of the bilingual French-sign language course and teacher at the Sainte-Marie school community in Namur. There, small groups of deaf children are integrated into general education classes for hearing children. “On the one hand, the dictionary enables, for example, complex expressions in novels written in French to be understood almost immediately. For the latter, by helping them acquire the necessary characters to converse with their deaf friends, he is the key that opens the door to another world.” That of silence.
the Necessary material consists of one computer or tablet or smartphoneequipped with a camera and connected to the internet with good flow. Next you have to join the dictionary website.
Those who want to know what character corresponds to a French word need only introduce it for a video showing a person performing the character to come to life. Examples that allow the user to see the context of the character being used are of great help. “As a matter of fact, In sign language, as in all languages of the world, there is no direct translation from one word to another. The context plays an important role,” explains Laurence Meurant, Director of the French-speaking Belgium Sign Language Laboratory at UNamur.
The dictionary is also functional in the other language direction. Facing the webcam, the user enters a character by positioning his face in the dotted circle and the platform displays its French translation. “For 80% of the inquiries, the correct sign is in the top 5 of artificial intelligence (AI)”, says Jérôme Fink, doctoral student in computer science, who has been the linchpin of this project for five years.
To achieve such a feat, Artificial intelligence researchers were based on a rich corpus of over 36 hours of sign videos, annotated and analyzed individually by the signers. It comprises about 180,000 characters, which corresponds to 4,600 characters repeated multiple times by about a hundred signers. A work of the Benedictines that has occupied the linguists of UNamur for almost twenty years. “A sufficiently large database was essential for the AI to be able to recognize a sign in front of each webcam,” specifies Laurence Meurant.
” To help the AI in this task, we computer-controlled and automatically glued onto each signer a kind of digital skeleton consisting of dots placed on the body elements used in sign language: the signer’s chin and mouth, outlines of his face, Corners of eyes, but also on elbows, hands, fingers,” explains Benoît Frenay, Professor at the Namur Digital Institute. Thanks to this system, the AI knows at every moment the position of each body point involved in the character to be recognized. To hone her skills, she trained and continues to do so with tens of thousands of videos of characters created by signers with special skill and morphology using different colors and lights.
“In order for the AI to identify a character and transmit its French translation, it had to train at least thirty occurrences of each character,” adds Anthony Cleve, co-pilot of this project with Laurence Meurant. In the corpus, this is the case for no fewer than the 750 characters most commonly used by signers. Or 35% of their basic lexicon. In the direction from French to sign language, the task was much easier and about 2,100 characters of nouns and verbs are already accessible.
Over the next few years, this database will grow and the AI will improve. Enough to promote the inclusion of deaf and hard of hearing people in society.