Colors Passing
San Diego International Airport, part of the Figures of Speech exhibition, February 2018 - February 2019
A collaboration with Gabi Schaffzin and Devin Higgins

Colors Passing is an object built for the International Arrivals terminal at the San Diego International airport. It responds poetically to its environment and the passersby. When something or someone approaches it, the lights inside the object reflect the color that it sees, and it produces a unique poem about that color.




Colors Passing translates color into poetry, but only through a translation of qualitative human experience into machine-readable code. It creates its unique poetry by sampling individual phrases or words from a corpus of public domain poetry that were translated into English from other languages. Some of the original languages are from cultures in the places to which airplanes from San Diego fly.

It is thus engaged and implicated in the politics of translation at multiple registers: when it translates the physical world into color data, and in its use of poetry that has been translated from other languages into English.

When a viewer first approaches Colors Passing, a sensor on the device collects color information—from, for example, the shirt of the viewer. This color information is translated into machine-readable language, represented in hexadecimal notation. The values collected are used as a key to compile the unique poetry about the specific color that the device “sees.” This poem is displayed on the screen on the front of Colors Passing.

In order to do what we set out to do — use a corpus of poetry in translation from the countries to which people from SAN might fly in order to produce mashed-up poetry about the color the computer vision system was seeing at any given moment — we needed help. We worked with a digital collections expert at the Michigan State University Libraries, Devin Higgins, to build a corpus of poetry in translation that was originally written in languages spoken in locations to which planes from SAN might fly.  

After we assembled the corpus of texts — mainly books of poetry culled primarily from the academic collections in the Hathi Trust database — we had to go through them quickly and do the best we could to clean them of any improperly recognized artifacts. The texts, which we had as .txt files, had been put through optical character recognition (OCR) after the original books had been scanned. While relatively accurate, OCR is far from perfect. Ligatures, for example, are widely misrecognized as numbers or other letters, and the quality of a book’s printing (as well as its age) has a certain bearing on how well the OCR system can “read” the text on the page.

While we were doing our best to clean up the text files, we needed to create a way to search through the corpus for colors and to categorize those colors, lumping lines about similar colors together. So we established a hierarchical taxonomy of colors (e.g., “azure” is a subset of “blue”) for any color that might be mentioned in the poems. We then wrote a script to find all the mentions of any given color within the corpus of texts, and pulled the line that mentions that color along with the lines before and after it (to identify context). Once we did that, we were able to produce a database of lines of poetry that pertained to general colors and more specific subsets of those colors. It is this database that Colors Passing uses to generate its mashed up poetry about the colors that it is seeing.

But which colors does it actually see? Using camera input in an ultrasonic proximity sensor, the system detects objects that are a few feet away and then grabs the color value of the pixel in the center of its field of “vision.” This hexadecimal value is translated into a color name, which is then used to query the aforementioned database.

Linguistic anthropologists have, for over the past half-century, studied the ways various cultures refer to otherwise similar colors. That is, what American English speakers might consider to be “blue” may be closer to “green” in Japan. Some cultures do not have words for colors that other cultures do. Human language, of course, is a complicated and fluid communication mechanism. Machine language—insofar as it is a production of human programming—can act as a universal equivalent: the hexadecimal value FF8300 (a bright orange) can be read and represented by any computer programmed to do so, which includes nearly every personal computer, tablet, smart phone, and more. Even in a machine-readable context, however, there is a required delineation: each hexadecimal value represents a specific spot on the color spectrum—there is no in-between, for instance, FF8300 and FF8301 in a 24-bit color system. Our eyes, if trained to perceive that in-between, may beg to differ.

Colors Passing uses poetry to illustrate this translation from a common machine-readable value into human language. On the one hand, “sense” is lost when multiple poems are combined into one using natural language processing. On the other, the poetic forms allow us to go beyond common narrative prose. Just as the name or meaning of a color is affected by a viewer’s position relative to that color (i.e., the culture in which they were raised, their own visual abilities, physical location in relation to its source, etc.), the meaning of Colors Passing’s poetry is squarely situated within the viewer’s subjectivity.