Automatic Recognition Of British Sign Language Using The BSL Corpus.pdf

Although recent advances have been made in sign language recognition, most computer scientists in this area lack in-depth knowledge of sign language and often have no connection with the Deaf community or sign linguists. This has led to technology that doesn't address the challenges, such as sign-language gloves. However, automatic sign language recognition can help speed up the process of annotating and analysing sign language data on video, which is extremely labour-intensive.

The British Sign Language Corpus (BSL Corpus) provides a large dataset that has been precisely and consistently transcribed and translated, which is needed to train computers for automation. The ExTOL project aims to build the world's first British Sign Language to English translation system and the first practically functional machine translation system for any sign language.

To achieve this, the computer must be able to recognise not only the shape, motion, and location of the hands but also nonmanual features, including facial expression, mouth movements, and body posture of the signer. The technology for recognising hand, face, and body positions and movements is improving, which will allow significant progress in speeding up automatic recognition and identification of these elements.

The project has two aims: to create a system that can translate BSL to English and to create automatic annotation tools to be integrated into the software annotation package ELAN. Initial progress on the ExTOL project includes automatic recognition of English mouthings, which is being trained using 600+ hours of audiovisual spoken English from British television and TED videos, and translation of IDGloss to Free Translation.

Baseline results on a subset of the BSL Corpus, which contains 10,000+ sequences and over 5,000 unique tokens, using the state-of-the-art attention based Neural Machine Translation approaches, have been reported. Although free translation cannot be achieved via ID glosses alone, this baseline translation task will help contribute to the overall BSL to English translation process.