Alt Text: The Hidden Descriptions that Make Art Accessible

---

According to a recent New York Times article, as of February 2021 only about 40 percent of website home pages contain alt text, a pillar of accessible website design that allows people who use screen readers to access visual imagery. In 2019, only 0.1 percent of over one million tweets did. Given the human labor involved with creating meaningful alt text, that is understandable. But the lack of alt text comes at a cost to many people with low vision, which is why we’re committed to writing hand-crafted descriptions of the images on our site.

---

Screen readers and image descriptions

Many people who are blind or have low vision rely on screen readers to navigate the web. Screen readers are a form of assistive technology that turns written text into audio output that can be listened to rather than seen, much as one listens to an audio book. Screen readers are not able to interpret visual images, however. Instead, they depend on alt text (short for alternative text), a written description of the image which, in the case of a web page, is often embedded in the HTML code and hidden from view.

What this means is that if you’re not using assistive technology, you’re likely to be completely unaware of the presence of alt text. But if you’re using a screen reader and it comes to an image with alt text, it reads the description aloud as it does any other text, thus giving a person with low vision access to the image that they would not otherwise have had. If there’s no alt text, then the screen reader passes over the image as if it were not there.

A.I. versus human-crafted alt text

When we envisioned our new website (which went live in July 2021), having our content meet the most up-to-date accessibility standards was a high priority for us. Because so much of our content is visual (in the form of paintings and drawings), we also knew that the inclusion of alt text would be essential to achieving this goal.

Initially, and naively, we assumed that somehow, magically, the accessible design protocols our website developers were using would automatically generate rich, meaningful alt text worthy of the visual imagery that populates our site. We were therefore surprised when at one of our monthly design meetings, the project manager mentioned that we would have a big job ahead of us, custom crafting alt-text for each of the 600+ images on our site. At the time, the task of having to write hundreds of unique, individually crafted descriptions for all that visual material was rather daunting. Surely, we thought, technology must be at such a place where some kind of smart application could come to our rescue.

What we discovered was that, while some companies (like Facebook and Google) use artificial intelligence (A.I.) to generate alt text, the resulting descriptions are often at best woefully anemic, and at worst completely inaccurate. To accompany the Johannes Vermeer painting below, for instance, Facebook A.I. generates the following description: “May be an illustration of 1 person and indoor.”

This and other startlingly inept alt-text descriptions have inspired us to take the time to carefully observe and write about each of the paintings and drawings on our site with great care (though I should say that as of now, we have only covered a fraction of the images). By way of contrast to the Facebook description above, our handcrafted alt text that accompanies the Vermeer painting is as follows:

“A full-color oil painting of a 17^th-century domestic interior depicting a young woman standing in a room lost in thought. There is an open window to her left, and to her right a table, on which sits a plate and a water pitcher. The room is bathed in diffuse natural light. The woman, who wears a full-length blue dress and white headdress, rests one hand on the open window sash to her left, and one hand on the handle of the water pitcher to her right. The scene presents a poignant moment where time appears to have stopped.”

The challenge of drafting meaningful alt text

On the surface, it seems that alt text ought to be relatively straightforward: write a sentence or two describing an image. But, reminiscent of Blaise Pascal’s famous statement, “I would have written a shorter letter, but I did not have the time,” drafting meaningful alt text requires time, attention, and discernment, which is perhaps one of the reasons A.I. fails to generate meaningful descriptions. Alt text needs to be concise, thus describing an image well involves a process of selection. (One criticism we would have of our Vermeer description above is that it is too long; we are continuously returning to our alt text to revise it, much as one would a poem.) It also needs to provide context and to transmit the image’s emotional valence. In terms of identity and representation, it raises the question of what kind of information should be included (a source of great debate among proponents of alt text).

Our creative director, Brian Schumacher, who is himself a painter, is spearheading this work. Yet far from the onerous chore we feared this would be, it has become a fascinating and transformative journey toward a greater understanding of visual material and how to make art available to people with low vision, a mission we are committed to pursuing as part of our work at the V&AP.