Allison interviews Nicholas Wilkins from Sign-Speak about their service that uses AI to recognize sign language or speech and convert that to an avatar that signs the message, allowing you to share your signed message with others. The service also supports sign-to-text and speech-to-text conversion.
Sign-Speak ensures privacy and safety and is accessible 24 hours a day, 7 days a week, year round. Users can trust the accuracy of Sign-Speak’s models with the flexibility to switch to a human interpreter whenever needed.
The service offers budget-friendly options. Sign-Speak solutions are available at the link below.
Learn more at https://sign-speak.com/
Using a Screen Reader? click here
Transcript of Interview:
Allison: One of the things you learn as you start learning about accessibility is that if you don’t have a disability right now, that you’re supposed to call yourself currently abled. Because if you live long enough, you’re probably going to have something that you’re going to need to have fixed.
My granddaughter is deaf in one ear and has a cochlear implant and she’s got a hearing aid in the other. So I’m really interested in some of the accessible tech for those with hearing impairments. We are in the Sign-Speak booth and I’m here with Nicholas Wilkins to tell us about a really, really cool innovation.
Nicholas Wilkins: Awesome. So we are Sign-Speak. We do sign language recognition and sign language avatars. In particular, any deaf person can sign into our software. It will recognize what they’re signing and voice it out. And then at the same time, the hearing person can respond back. And that’s shown through a lifelike avatar. So if we go over to the monitor. Let me start over the video.
Nicholas Wilkins: So the way that this works is that if someone speaks into the system in a minute, the avatar will sign exactly what was said. And this is actually completely AI-generated. And then over here, as Nico signs back, what will happen is down here, whatever he signed will be shown in text and also voiced out.
Allison: So basically a deaf person and someone without a hearing impairment would be able to have a full-on conversation. One person hearing, one person seeing sign.
Nicholas Wilkins: Indeed. And in addition to that, also provide captions for those who don’t rely on sign. So that way all individuals can have full —
Allison: Say that one more time.
Nicholas Wilkins: So in addition to sign language access, we additionally provide captioning to provide access to those people who might rely on captioning and not on sign language. So for example, late deafened individuals. The goal of this is to allow hearing individuals and deaf or hard-of-hearing individuals to interact. And if at any point the technology is not working, you can also transfer over to a human interpreter. We have human interpreters on call that can be brought into a conversation. So over here, if you see, I’m about to press the button. And then a human interpreter picks up and continues the conversation from there to make sure that accessibility is rendered.
Allison: So back to the captions, I have one question. If you have captions, that would mean that someone who is deafblind could participate in the conversation with a Braille display, no?
Nicholas Wilkins: So we’ve done some initial experiments with deafblind individuals. We’ve noticed that for deafblind individuals, all of — and I mean in general. Each individual is unique in terms of their accessibility preference. So some just want customization to be able to make some text larger. Others want an output to like a Braille display. But yeah. We have made this system screen reader compatible so that it can be used that way.
Allison: These silly people. Everybody seems to be different. I don’t know how that happened, right?
Nicholas Wilkins: Yeah. Yeah. I feel like the world would be easier but a lot less interesting if everyone was the same.
Allison: There you go. Exactly. I think Steve actually had a question.
Steve: Yeah. I had a question about at one point in the display, one of the faces was blurred. What was the blurring for? So that was during the part where it was the human interpreter came in?
Nicholas Wilkins: So right now, because we’re showing this publicly, we decided that we wanted to protect the identity of our interpreters. So we decided to blur that. But because this AI avatar is a completely fake person, doesn’t really have any — it doesn’t — anyone’s face.
Allison: Until they get rights.
Nicholas Wilkins: Yeah. Or somehow they get birthed. But one day maybe.
Allison: So this doesn’t require any kind of hardware. You’re just sitting in front of your Mac laptop here. You have the camera on. You can have this conversation today through the software. So is it a web service?
Nicholas Wilkins: Right. So you’re absolutely right. This requires no external hardware. Just a normal RGB camera that’s on any webcam. Right now we provide it as a software as a service. Both in this package, which is our automated interpreting system, our AIS, which we provide at $200 a month. And then in addition, we also have an API that people can use, which is metered per minute usage.
Allison: So I’m guessing — I’m just spitballing here, but I’m guessing that a human interpreter costs more than $200 a month to always be there for you.
Nicholas Wilkins: Oh, yes, definitely. And that’s not the only issue. Human interpreters are fantastic for a lot of situations, but unfortunately, there’s a huge interpreter shortage. So they’re just not available a lot of the time. Oftentimes you’ll hear about interpreters being requested for appointments and then the interpreter calling in sick and the appointment needing to be moved. Other times interpreters are cost-prohibitive.
Allison: This poor avatar, he’s just stuck in that room all the time. He’s always there for you.
Nicholas Wilkins: Yeah, we just keep on working him. Keep on working him 24/7.
Allison: Okay. If people wanted to learn more about Sign-Speak, where would they go?
Nicholas Wilkins: Yeah, thank you. You can go to sign-speak.com. That’s sign-speak.com.
Allison: Very good. Thank you, Nicholas. This is really neat. I’m very excited about it.