Generative AI and Accessibility

The proliferation of Generative Artificial Intelligence (GenAI) tools has brought a critical shift in how people approach information retrieval and content creation in diverse contexts. Yet, we have limited understanding of how blind and low vision people use and make sense of GenAI tools. In a recent study, we interviewed 19 blind individuals to understand how they incorporate mainstream GenAI tools like ChatGPT, Microsoft Copilot, Google Gemini, Claude, and Be My AI in their everyday practices. Our findings revealed how blind users navigate accessibility issues, inaccuracies, hallucinations, and idiosyncracies associated with GenAI and develop interesting (but often flawed) mental models of how these tools work. We discussed key considerations for rethinking access and information verification in GenAI tools, unpacking erroneous mental models among blind users, and reconciling harms and benefits of GenAI from an accessibility perspective.

In another study, conducted in collaboration with Google researchers, we focused on accessibility of AI-generated images. State-of-the-art image generation models do not output alternative (alt) text with their images, rendering them largely inaccessible to screen reader users. Moreover, we know less about what information about this new visual medium would be most desirable to screen reader users. To address this, we invited AI image creators and SRUs to evaluate alt text prepared from various sources and write their own alt text for AI images. Through a mixed-method analyses, we illustrated creators perspectives and screen reader users’ information needs from alt text of AI images. Finally, we discussed the promises and pitfalls of utilizing text prompts written as input for AI models in alt text generation, and areas where broader digital accessibility guidelines could expand to account for AI images.

Screenshot of ChatGPT 3.5 interface. ChatGPT icon and the user's avatar near the prompt and the response are highlighted with the text 'icons and avatars announced as graphic'. Buttons for copying the response, regenerating another response, and downvoting are highlighted as unlabeled buttons. In the image, the user asks a question about describing a character from a show to someone who is blind. ChatGPT responds with a detailed description of Kim Possible, highlighting her as a high school student and secret agent with red hair, a green crop top, cargo pants, black gloves, and her attributes of intelligence, bravery, and a strong sense of justice.

Collage of five AI generated image, some with AI aberrations. Each has visual watermark that was blurred for anonymity. Image 1: A graphic of a collection of hands making sign language-like gestures against a white background with some hands holding a pink flower with a green stem. Some hands’ shapes are unusual without clearly recognizable fingers or palms, which could be aberrations of the AI model. Some green stems do not have a flower attached. “Model_name” is written near the bottom right corner. Image 2: A collection of overlapping human hands with fingers spread out against a white background. Some hands are strangely fused together; some fingers’ shapes are unusual with blunt tips and non-existent or strange nails as if the fingertips were cut off, which could be aberrations of the AI model. Most hands are viewed from the back side except for a palm view of the hand furthest back. “15” is written near the top right corner and “Model_name” is written near the bottom right corner. Image 3: A cartoon-style image of a birthday party. A gray mouse with pink legs, ears, nose, and tail is smilingly standing in the middle on its hind legs, wearing a green pointy party hat and holding a small blue and yellow object in its hand. A chocolate cake with light yellow frosting and some candles are placed on a blue circular table. The yellow-colored room is decorated with party materials. Hanging on the wall are a few stars and a birthday-style multi-colored banner that says “KIRRY ARIHOA” with some illegible letters, a possible aberration of AI models. Green, red, and pink pennant triangle flag banners are also hanging against a white tapestry with light pink vertical stripes. Pastel pink, blue, yellow and green balloons are floating, and red, yellow, pink and green gift boxes lay on the floor. “Model_name” is written near the bottom right corner. Image 4: Five yellowish green cartoon turtles are driving a yellow car down a road. The background shows New York City’s downtown skyline in shades of blue, and a light blue sky with floating clouds. The car has a black and white checkerboard design in the front where the license plate and logo should have been. “Model_name” is written near the bottom right corner. Image 5: Painting of a metallic robot as seen from behind at an angle. It is standing with a paintbrush in its left hand pointed towards a canvas that shows a painting of an identical robot but with a pointed object projecting from its head. The canvas rests on a wooden easel. Painting plates, a bottle, and other items lay nearby on the ground. The painter robot stands on a dirt trail lined with green trees and grasses that leads to a lake where a waterfall cascades from green mountains, with blurry mountain ranges in the distance under a soft afternoon glow in the sky. “Model_name” is written near the bottom right corner.

Relevant Publications

Rudaiba Adnin and Maitraye Das. 2024. “I look at it as the king of knowledge”: How Blind People Use and Understand Generative AI tools. In Proceedings of the International ACM SIGACCESS Conference on Computers and Accessibility, (ASSETS ’24). [ACM DL Link]

Maitraye Das, Alex Fiannaca, Meredith Ringel Morris, Shaun Kane, and Cynthia Bennett. 2024. From Provenance to Aberrations: Image Creator and Screen Reader User Perspectives on Alt Text for AI-Generated Images. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI ’24). [ACM DL Link][Video]