By Bethany Farrell

Sometimes museum visits seem like a twenty-first century gauntlet run—huge crowds, near misses with selfie sticks (which have now been mostly banned by museums), and accidental photo bombing. In an age of exorbitant ticket prices, it is understandable that visitors attempt through photos to absorb and commemorate all that they see in the esteemed halls of the Uffizi, Louvre, and Met. Although a snap-and-run visit seems in opposition to the benefits of looking at art, it has been suggested that taking a selfie with an object, especially when the visitor interacts meaningfully with it (by mimicking the pose or seeing one’s own twin…more on that later), engages the viewer in a form of art appreciation. For scholars, these photo mementos provide a potential new resource for discovering not-well-known or misplaced artworks.

As mentioned in an earlier blog, the artist I study, Bronzino, had a court position which required him and his workshop to copy paintings over and over again. Of varying quality, the copies can number into the twenties and thirties as discovered by scholars in the 1970s and 80s. Since these paintings have significantly less value than their “originals” they are often on the move, passing regularly through the art market. And unlike Leonardo da Vinci, reattributions and discoveries related to Bronzino are not uncommon. The sum of these issues makes digital technologies indispensable to unearthing or relocating paintings.

Since I am looking for secondary versions of paintings, reverse image search engines have been particularly fruitful. I prefer Tineye’s platform (and occasionally Google’s reverse image search) because their metrics compare an image to thousands of databases and websites. Consequently, internet published selfies can answer the key question, “where is it now?”, filling in provenance data gaps. Reverse image search engines provide other helpful services, art historians regularly contend with only having black and white, Xeroxed, or low-quality images. In the case of Tineye, its sorting function allows the use to look for the most changed or largest file, which provide scholars with better images of the art they are researching. For me, this functionality means that Tineye can capture not just the original but its copies due to its ability to capture variations and include them in the results.

A month ago, Google Arts & Culture Project launched a new feature on their app which allows a user to find their historic twin. Just in time for 2018’s #MuseumSelfie day on January 17, it blazed through social media. Users were both flattered and aghast at the comparison that the ai feature made through analyzing their biometrics and matching it to paintings. Google is not the only company interested in this technology. Apple’s newest phones use the face of the owner as an unlocking feature (to varying degrees of success). Facebook has been tagging faces with a similar technology for a long time. The Arts & Culture feature demonstrates the advancing sophistication of biometrics. Seen below, the app was able to match my face’s orientation to a painting with a three-quarter view because the ai technology analyzes the underlying structure of a face.

Google’s Art & Culture Art Selfie Feature (The Author and Her Twin?)

As I mourned my visage’s lack of ego-inflating comparisons, I began to think of the potentials of biometric ai for art historians. Biometrics could find the same sitter or model across multiple artist’s works and mediums. Identifying figures in artworks is fraught territory in art history. For example, the sixteenth century painter Pontormo’s Portrait of a Halberdier has been identified as Duke Cosimo I de’ Medici and Francesco Guardi. In this particular case, the historical evidence and visual cues only slightly lean in favor of a Guardi id. Biometrics could aid in solidifying an identification by comparing the portrait to sculptures and paintings that definitively depict Cosimo.  When I tried the Arts & Culture feature on Pontormo’s painting, it successfully matched it with the Portrait of the Halberdier. In the case of a young portrait of Cosimo by Bronzino’s workshop, it was able to use the underlying structure of his face to match it with a print of the duke when he was at least ten years older.

Varying reasons have kept the field of art history from adopting these computational methods, but what the Arts & Culture face match feature demonstrates is the possibilities in data and computer driven art history. Although it cannot replace traditional methodologies, biometrics and other ai programs (such as the style classification project at the Art and Artificial Intelligence Laboratory) as well as data science could greatly enhance certain art historical inquiries.



Further Reading:

Selfies and the Museum:

Rosenbloom, Stephanie. “The Art of Slowing Down in A Museum.”

Wasmer Andrews, Linda. “Time Spent in an Art Museum Can Be Good Therapy.”


Google Arts & Culture:

Held, Amy. “Google App Goes Viral Making An Art Out of Matching Faces to Paintings.”

Sutton, Benjamin. “Why Google’s Art Selfie App Doesn’t Work in Texas and Illinois.”


Pontormo’s Portrait of a Halberdier:

Cropper, Elizabeth. Pontormo: Portrait of a Halberdier. Getty Publications, 1997.


Computational Art History:

Elgammal, Ahmed, Marian Mazzone, Bingchen Liu, Diana Kim, and Mohamed Elhoseiny. “The Shape of Art History in the Eyes of the Machine.” arXiv preprint arXiv:1801.07729 (2018).

Manovich, Lev. “Data Science and Digital Art History.” International Journal for Digital Art History (2015).