Showcase :Metropolitan Museum of Art

Digital Media Lab Expo, December 2014

Date :October - December 2014

Skills :Openframeworks

Live Site :

Paintings Uncovered

Paintings Uncovered is an interactive interface exploring hidden layers underneath paintings. Painters frequently paint over paintings for various reasons — sometimes with a completely different subject. Examining the underlying surfaces through reflectography —an infrared technique to detect layers beneath the top surface of a painting— provides valuable information to art historians about the artwork.

These underlying images reveal fascinating stories about intent and process that are not obviously evident when visitors see the paintings. So I wondered, how can an interactive interface successfully teach people about hidden images in art?

History of the Portrait of Madame X

To explore this question, I chose to work with the Portrait of Madame X by John Singer Sargent. The portrait shows a woman, Virginie Amélie Avegno Gautreau, posing in a black dress, her pale skin a striking contrast against the dark garment and background.

This portrait was the subject of controversy when it was unveiled at the Paris Salon in 1884. In the original painting, the strap on Gautreau’s right shoulder rested seductively off her shoulder -- viewers were shocked, causing a scandal that resulted in the mockery of both Sargent and Gautreau. Critics also ridiculed the color of her skin and pose. Sargent, who had painted Gautreau as she was originally dressed, painted over the strap so that it was securely on her shoulder. In 1916, he sold the painting to the Metropolitan Museum of Art.

This is one of my favorite stories about a painting at the Met. I wanted to experiment with sharing the story in an interactive way, so I created an interface that reacted to infrared light to reveal the original painting that lies underneath.

The interactive installation

With Paintings Uncovered, users point an infrared flashlight at a digital version of the painting. An infrared camera tracks the position of the light to create a circular mask on corresponding areas of the painting that creates the illusion of "burning through" the top layer, revealing the underlying surfaces that Sargent deliberately covered. Moving the flashlight onto certain areas then triggers specific questions about the work, making it primarily an inquiry-based exploration.

I created this capability by using openFrameworks (a C++ toolkit aimed at designers) and the PS3 Eye, a small, portable camera for the PlayStation 3. I modified the PS3 Eye by removing the infrared filter and installing a visible light filter instead (view the Instructable). I then took apart a flashlight I purchased at the dollar store (the cheaper the flashlight, the easier it is to take apart) and switched its LED with an infrared LED.

This idea was inspired by my childhood dream of becoming an archaeologist; by using a flashlight, users can "role-play" as archaeologists or researchers. I wanted to experiment with physical movement rather than creating a touch-based or mouse-based application. The portrait of Madame X is nearly eight-feet tall, and in an ideal scenario, the digital version of the painting would be projected at the same size as the original. By prompting the user to move their arms in order to examine the painting, the interaction emphasizes the portrait's striking size.

Finally, I wanted to engage users with critical thinking rather than giving them the answers. As a docent for The Fralin Museum of Art, I was trained to teach people how to look at and think about art by asking visitors questions instead of immediately jumping into a lecture. I continued that methodology in this project by creating an inquiry-based experience that encourages viewers to examine the painting more closely.


This project was originally created with the intent to enhance tours, although further improvements should include creating an interaction where visitors can find answers without a tour guide. The challenge made me think about how technology could be used to tell the stories behind the artworks in museums and eventually move away from traditional labels and brochures. After all, learning about the history and intent of the artworks truly enhances the museum experience and makes art memorable.