

{"id":10119,"date":"2018-02-26T16:08:00","date_gmt":"2018-02-26T20:08:00","guid":{"rendered":"https:\/\/sites.temple.edu\/tudsc\/?p=10119"},"modified":"2026-10-06T16:14:16","modified_gmt":"2026-10-06T20:14:16","slug":"reverse-image-search-and-biometric-ai-in-art-history-research","status":"publish","type":"post","link":"https:\/\/sites.temple.edu\/tudsc\/2018\/02\/26\/reverse-image-search-and-biometric-ai-in-art-history-research\/","title":{"rendered":"Reverse Image Search and Biometric AI in Art History Research"},"content":{"rendered":"\n<p>Originally posted on February 26, 2018 by Bethany Farrell<\/p>\n\n\n\n<p>Sometimes museum visits seem like a twenty-first century gauntlet run\u2014huge crowds, near misses with selfie sticks (<a href=\"https:\/\/web.archive.org\/web\/20240423020533\/https:\/\/news.artnet.com\/exhibitions\/are-museum-selfies-endangered-museums-ban-selfie-sticks-246203\">which have now been mostly banned by museums<\/a>), and accidental photo bombing. In an age of exorbitant ticket prices, it is understandable that visitors attempt through photos to absorb and commemorate all that they see in the esteemed halls of the Uffizi, Louvre, and Met. Although a snap-and-run visit seems in opposition to the benefits of looking at art, it has been suggested that taking a selfie with an object, especially when the visitor interacts meaningfully with it (by mimicking the pose or seeing one\u2019s own twin\u2026more on that later), engages the viewer in a form of art appreciation. For scholars, these photo mementos provide a potential new resource for discovering not-well-known or misplaced artworks.<\/p>\n\n\n\n<p>As mentioned in an <a href=\"https:\/\/web.archive.org\/web\/20240423020533\/https:\/\/sites.temple.edu\/tudsc\/2017\/11\/28\/provenance-matters-digital-tools-for-object-history-research\/\">earlier blog<\/a>, the artist I study, Bronzino, had a court position which required him and his workshop to copy paintings over and over again. Of varying quality, the copies can number into the twenties and thirties as discovered by scholars in the 1970s and 80s. Since these paintings have significantly less value than their \u201coriginals\u201d they are often on the move, passing regularly through the art market. And unlike Leonardo da Vinci, reattributions and discoveries related to Bronzino are not uncommon. The sum of these issues makes digital technologies indispensable to unearthing or relocating paintings.<\/p>\n\n\n\n<p>Since I am looking for secondary versions of paintings, reverse image search engines have been particularly fruitful. I prefer <a href=\"https:\/\/web.archive.org\/web\/20240423020533\/https:\/\/www.tineye.com\/\">Tineye<\/a>\u2019s platform (and occasionally Google\u2019s reverse image search) because their metrics compare an image to thousands of databases and websites. Consequently, internet published selfies can answer the key question, \u201cwhere is it now?\u201d, filling in provenance data gaps. Reverse image search engines provide other helpful services, art historians regularly contend with only having black and white, Xeroxed, or low-quality images. In the case of Tineye, its sorting function allows the use to look for the most changed or largest file, which provide scholars with better images of the art they are researching. For me, this functionality means that Tineye can capture not just the original but its copies due to its ability to capture variations and include them in the results.<\/p>\n\n\n\n<p>A month ago, Google Arts &amp; Culture Project launched a new feature on their app which allows a user to find their historic twin. Just in time for 2018\u2019s #MuseumSelfie day on January 17, it blazed through social media. Users were both flattered and aghast at the comparison that the ai feature made through analyzing their biometrics and matching it to paintings. Google is not the only company interested in this technology. Apple\u2019s newest phones use the face of the owner as an unlocking feature (to varying degrees of success). Facebook has been tagging faces with a similar technology for a long time. The Arts &amp; Culture feature demonstrates the advancing sophistication of biometrics. Seen below, the app was able to match my face\u2019s orientation to a painting with a three-quarter view because the ai technology analyzes the underlying structure of a face.<\/p>\n\n\n\n<p>(Image no longer available.)<\/p>\n\n\n\n<p>As I mourned my visage\u2019s lack of ego-inflating comparisons, I began to think of the potentials of biometric ai for art historians. Biometrics could find the same sitter or model across multiple artist\u2019s works and mediums. Identifying figures in artworks is fraught territory in art history. For example, the sixteenth century painter Pontormo\u2019s <em>Portrait of a Halberdier<\/em> has been identified as Duke Cosimo I de\u2019 Medici and Francesco Guardi. In this particular case, the historical evidence and visual cues only slightly lean in favor of a Guardi id. Biometrics could aid in solidifying an identification by comparing the portrait to sculptures and paintings that definitively depict Cosimo.&nbsp; When I tried the Arts &amp; Culture feature on Pontormo\u2019s painting, it successfully matched it with the Portrait of the Halberdier. In the case of a young portrait of Cosimo by Bronzino\u2019s workshop, it was able to use the underlying structure of his face to match it with a print of the duke when he was at least ten years older.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img decoding=\"async\" src=\"https:\/\/web.archive.org\/web\/20240423020533im_\/https:\/\/sites.temple.edu\/tudsc\/files\/2018\/02\/IMG_7616-300x300.jpg\" alt=\"\" class=\"wp-image-4720\" style=\"width:300px;height:auto\" \/><\/figure>\n<\/div>\n\n\n<p>Varying reasons have kept the field of art history from adopting these computational methods, but what the Arts &amp; Culture face match feature demonstrates is the possibilities in data and computer driven art history. Although it cannot replace traditional methodologies, biometrics and other ai programs (such as the style classification project at the Art and Artificial Intelligence Laboratory) as well as data science could greatly enhance certain art historical inquiries.<\/p>\n\n\n\n<p>Further Reading:<\/p>\n\n\n\n<p><strong>Selfies and the Museum:<\/strong><\/p>\n\n\n\n<p>Rosenbloom, Stephanie. \u201cThe Art of Slowing Down in A Museum.\u201d<\/p>\n\n\n\n<p><a href=\"https:\/\/www.nytimes.com\/2014\/10\/12\/travel\/the-art-of-slowing-down-in-a-museum.html\">https:\/\/www.nytimes.com\/2014\/10\/12\/travel\/the-art-of-slowing-down-in-a-museum.html<\/a><\/p>\n\n\n\n<p>Wasmer Andrews, Linda. \u201cTime Spent in an Art Museum Can Be Good Therapy.\u201d<\/p>\n\n\n\n<p><a href=\"https:\/\/www.psychologytoday.com\/blog\/minding-the-body\/201703\/time-spent-in-art-museum-can-be-good-therapy\">https:\/\/www.psychologytoday.com\/blog\/minding-the-body\/201703\/time-spent-in-art-museum-can-be-good-therapy<\/a><\/p>\n\n\n\n<p><strong>Google Arts &amp; Culture:<\/strong><\/p>\n\n\n\n<p>Held, Amy. \u201cGoogle App Goes Viral Making An Art Out of Matching Faces to Paintings.\u201d<\/p>\n\n\n\n<p><a href=\"https:\/\/www.npr.org\/sections\/thetwo-way\/2018\/01\/15\/578151195\/google-app-goes-viral-making-an-art-out-of-matching-faces-to-paintings\">https:\/\/www.npr.org\/sections\/thetwo-way\/2018\/01\/15\/578151195\/google-app-goes-viral-making-an-art-out-of-matching-faces-to-paintings<\/a><\/p>\n\n\n\n<p>Sutton, Benjamin. \u201cWhy Google\u2019s Art Selfie App Doesn\u2019t Work in Texas and Illinois.\u201d<\/p>\n\n\n\n<p><a href=\"https:\/\/hyperallergic.com\/421889\/why-googles-art-selfie-app-doesnt-work-in-texas-and-illinois\">https:\/\/hyperallergic.com\/421889\/why-googles-art-selfie-app-doesnt-work-in-texas-and-illinois<\/a><\/p>\n\n\n\n<p><strong>Pontormo\u2019s <em>Portrait of a Halberdier<\/em>:<\/strong><\/p>\n\n\n\n<p>Cropper, Elizabeth.&nbsp;<em>Pontormo: Portrait of a Halberdier<\/em>. Getty Publications, 1997.<\/p>\n\n\n\n<p><strong>Computational Art History:<\/strong><\/p>\n\n\n\n<p>Elgammal, Ahmed, Marian Mazzone, Bingchen Liu, Diana Kim, and Mohamed Elhoseiny. \u201cThe Shape of Art History in the Eyes of the Machine.\u201d&nbsp;<em>arXiv preprint arXiv:1801.07729&nbsp;<\/em>(2018).<\/p>\n\n\n\n<p>Manovich, Lev. \u201cData Science and Digital Art History.\u201d International Journal for Digital Art History (2015).<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Originally posted on February 26, 2018 by Bethany Farrell Sometimes museum visits seem like a twenty-first century gauntlet run\u2014huge crowds, near misses with selfie sticks (which have now been mostly banned by&#8230;<\/p>\n","protected":false},"author":34028,"featured_media":10120,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"footnotes":""},"categories":[2],"tags":[],"class_list":["post-10119","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-grad-students"],"_links":{"self":[{"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/posts\/10119","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/users\/34028"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/comments?post=10119"}],"version-history":[{"count":1,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/posts\/10119\/revisions"}],"predecessor-version":[{"id":10121,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/posts\/10119\/revisions\/10121"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/media\/10120"}],"wp:attachment":[{"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/media?parent=10119"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/categories?post=10119"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/tags?post=10119"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}