AutoChemplete - Making Chemical Structural Formulas Accessible

This research project is published and available in the W4A 2023 proceedings and received the judges’ award for the Accessibility Challenge.The tool has been released as open source and can be downloaded from GitHub.

Project Abstract: Despite their interests, blind and low vision students avoid STEM (science, technology, engineering, and mathematics) subjects. Research attributes this to a lack of accessible material. Annotating STEM content such as chemical structural formulas requires expert domain knowledge, is time consuming, and frustrating. We conduct interviews with blind and low vision chemists and accessibility professionals to derive requirements for tool support. On this basis, we develop AutoChemplete, an interactive labeling tool for chemical structural formulas. It ingests images and uses machine learning to predict the molecule. With a similarity search in the solution space, we enable even novices to simply pick from options. From this we are able to generate accessible representations. We conduct fifteen think-aloud sessions with participants of varying domain expertise and find support of different annotation styles simultaneously. Not only does AutoChemplete strike a balance in skill-support, participants even find it entertaining.