SignScribe
Award-winning bionic robotic hand that translates speech to ASL finger spelling in real time using an open-source ASR kit and rendered GUI through Blender.
SignScribe is a bionic-like robotic hand that translates spoken English into American Sign Language finger spelling in real time. I built it as project leader during the UMD Cyber-Physical Systems Engineering (CPSE) Summer Program.
The hand uses VOSK, an open-source speech recognition toolkit, to convert audio into text in real time. A Python match-case statement parses each letter and dispatches the corresponding servo commands to 7 hand servos, allowing the hand to sign any word in the English language including names and common references. The wrist has 2 degrees of freedom. A Blender-rendered GUI displays the signing hand in 3D alongside a live speech transcript, and a configuration file exposes 7+ user-adjustable settings including sign speed, word and letter pause timing, auto-save transcript, and word censoring. The physical hand sits on a 3D-printed SolidWorks-designed base with a clear acrylic frame and rubber grip pads, with a custom perforated board controlling status LEDs that communicate boot stages.
I was selected to solo present SignScribe at Demo Day to an audience of 200+ people, including Nobel Physics Prize winner Dr. Bill Phillips (MIT alum), Maryland lawmakers, UMD professors, and tech startup founders. SignScribe won the Best Team Award, recognizing market potential, presentation quality, teamwork, and innovation.