AmberScript: live transcription for people with hearing problems

SIDN Fund supports voice recognition engine developer

Person talks with sign language

The Netherlands is home to more than 1.5 million people who are deaf or hard of hearing. It can be difficult for such people to negotiate the digital landscape. But assistance is now at hand from a start-up called AmberScript. The firm was set up to help public bodies fulfil their statutory responsibility to make online material accessible to people with cognitive disabilities. Voice recognition software is used to generate transcripts and subtitles for audio content. AmberScript is now working on an idea to help people with hearing problems, many of whom currently depend on signers or family members to follow discussions and make themselves understood. A lot of things that most of us take for granted – following school and college classes, for example – can present challenges for people who can't hear well. The AmberScript team is therefore developing a Dutch-language speech recognition app that can convert conversation and audio to text in real time. We asked co-founder Peter-Paul de Leeuw about the background to the initiative, collaboration with SIDN Fund and his firm's plans for the future.

Born of frustration

AmberScript is an easy-to-use online tool for converting audio to text. It was born out of personal frustration. "I found it incredibly time-consuming to write up interviews for my qualitative study. I could see that it would be much better to use speech recognition technology," recalls Peter-Paul. Indeed, the technology was already available. However, no one had really integrated it into a working product, certainly not for the Dutch market.

Peter-Paul de Leeuw, co-founder van AmberScript
Peter-Paul de Leeuw, co-founder of AmberScript

So Peter-Paul got together with Thomas Dieste and Timo Behrens, to develop a speech recognition engine. Their tool's very easy to use: you upload an audio file, and it's converted into a text-file transcript that's 80 per cent accurate. The transcript is then linked to the audio for tweaking with the tool's online editor. Anyone who runs into problems can get help from AmberScript’s transcribers to remove any errors from the text.

AmberScript brings down barriers to communication

Over the last two years, 200,000 people have tried the tool, which now supports twenty-nine different languages. And people soon started using AmberScript in innovative ways. "After we'd been going a while, we became aware that students with hearing problems were recording lectures and then using AmberScript to convert their audio files to text. And we thought, hey, there's an idea." People who are deaf or hard of hearing nearly always need sign interpreters to understand audio content and to express themselves. That can be quite an issue, because of the cost and the scarcity of signers. The challenges are particularly great where highly technical or specialised subject matter is involved. We realised that live transcription could provide a solution. AmberScript is therefore developing an app that will let people with hearing loss see real-time on-screen transcripts of messages and conversations – entirely free of charge. By giving the deaf and hard of hearing access to aural information like everyone else, the tool is expected to bring down barriers to communication. A prototype should be ready by March of this year. After that, AmberScript wants to link up with a hardware manufacturer to get the technology installed on devices for people with hearing problems.

SIDN Fund

SIDN's mission is connecting people and organisations for a problem-free, opportunity-rich digital living. SIDN Fund was set up in 2014 to support that mission. The foundation works to build a better internet for everyone by providing grants to projects that help to make the internet stronger, promote user empowerment or utilise the internet in innovative ways. By doing so, it contributes to the prosperity and wellbeing of the nation.

SIDN Fund support AmberScript

SIDN Fund is supporting the team's initiative. "This project is opening the way for the live transcription of conversation, making it much easier for people with hearing impairments to participate. What's more, the app will be free to use, meaning that someone who's deaf or hard of hearing can use it at work, for example. All in all, the project is perfectly aligned with our goal of Tech for Good," explains the Fund's Project Coordinator Marieke van der Kruijs. Peter-Paul adds, "Financial support from SIDN Fund is allowing us to develop a working prototype of the app. But the Fund gives us more than money. We also have access to the Fund's network of people such as privacy experts who can advise us on how to make sure our app is privacy-friendly."

Transcription at the touch of a button

Ultimately, AmberScript hopes to make all audio and video content available in text form at the touch of a button. With that aim in mind, the team is making its technology available via an Application Programming Interface (API) to other software developers looking to incorporate speech transcription capability. For more information about the API and developments involving AmberScript's app for people with hearing disabilities, visit amberscript.nl.