How are museums using artificial intelligence, and is AI the future of museums?
March 28 2019
By Lauren Styx
Artificial intelligence, or “AI,” is a powerful tool being used all around us — but it’s not exactly new. Although it was first fully developed in the 1950s, the idea goes back to the 4th century B.C. when Aristotle thought up syllogistic logic, the first formal deductive reasoning system. It’s defined as “the study and design of intelligent agents” where an intelligent agent is a system that perceives its environment and takes actions that maximize its chances of success. Research uses tools and insights from many fields including computer science, psychology, linguistics, probability, and logic..
If you’ve talked to Siri or Alexa, you’ve used AI. If you found a new favorite documentary on Netflix or avoided traffic with Google Maps, you, too, have benefited from AI. The truth is, it’s everywhere and it has many purposes. The technology allows us to get answers more quickly, make less errors and sometimes, funnily enough, better understand our fellow humans. The technology and its uses have been growing, and the museum industry has been taking note.
Artificial intelligence has been a topic of discussion in museums for some time. At MuseumNext 2018 in London, Sara Boutall of big-data analytics service Dexibit said, “AI has been seeping into our lives, often without us actually realizing, on a daily basis. And we use it to accomplish absolutely fundamental tasks”
She’s right. Services we use everyday to stream our favorite music, buy products online, or get a ride across town all use AI to get the job done. But because many of us don’t necessarily understand how AI works, we take it for granted. In her MuseumNext presentation, Boutall uses the analogy of baking a cake to help us better understand artificial intelligence technology. In the past, programming computers meant giving precise instructions to determine its actions in the same way that ingredients and instructions are included in your favorite cake recipe.
“But with AI, rather than telling it what to do, we give it a thousand different ideas around what a delicious cake might look like, and then it starts teaching itself how to bake a cake. And this will run in iterations of thousands and millions … on a daily basis. It’s that type of scale that makes AI so incredibly powerful.”
AI and Museum Visitors
When it comes to museums, AI can be incorporated across the spectrum, from visitor experience to behind the scenes, and the technology can — and has — come in many forms.
In 2016, Paris’ Musee du quai Branly made a home for Berenson, the robotic art critic that quietly wandered the museum’s halls donning a bowler hat, coat, and scarf. Berenson was created by anthropologist Denis Vidal and robotics engineer Philippe Gaussier and used artificial intelligence to record people’s reactions to works of art and, in turn, develop its own taste. The question that presented itself to Berenson’s inventors was, would the robot be able to build aesthetic preferences as it interacted with museum visitors? It did. The robot works like this: Through a camera in its eye, it records the visitors’ reactions, then these recordings are shared with a computer elsewhere in the museum. Green circles represent the positive reactions and red circles, the negative ones. The color of these circles then determines whether Berenson will smile or frown at the artwork on its own.
“In the museum, the learning is controlled first by a set of visitors who are asked to show Berenson one object they like the most in our experimental area but also one object they did not like (or found less interesting),” Vidal said in an interview with Vice’s Creators. “At the end of each day, Berenson has learned 10 to 20 statues, and for each of them, tenths of local views. Each local view is associated thanks to a classical conditioning mechanism to a positive, negative, or neutral value.”
Just as humans evolved, so have museum robots — but at an exponentially faster pace. One example came two years later in 2018, when the world was introduced to Pepper, a humanoid robot developed by the United States’ Smithsonian. Six of these robots occupy three Washington-based museums of the Smithsonian (the National Museum of African Art, the National Museum of African American History and Culture, the Hirshhorn Museum and Sculpture Garden, and the Smithsonian Castle) with the purpose of answering visitors’ questions and telling stories using voice, gestures, and an interactive touch screen. Visitors love to interact with Pepper, and according to the New York Times, the bots will even pose for selfies. The Smithsonian plans to introduce more Peppers to other museum locations in the future.
Thanks to movies and television, many of us pigeonhole artificial intelligence as near-human robots much like Pepper and Berenson. However, visitors can interact with — and benefit from — AI in a myriad of other ways. Websites, chatbots, and analytics tools can all play a role in improving the visitor experience. Some of these tools can even improve the access visitors have to the museum in the first place. For example, by using AI to predict the amount of no-shows who took advance passes to a museum that operates at capacity, that museum can increase actual capacity and release more tickets in advance, eliminating that loss of visitors and bolstering museum visitorship.
Accessibility is a major area that museums are trying to address in new and innovative ways. One example of how AI plays into this is The Museum of Tomorrow in Rio de Janeiro’s IRIS+ chatbot, which was introduced in 2017. The original IRIS came with the museum’s opening as its digital assistant. Through IRIS, each visitor uses a chipped card to personalize their experience throughout the museum within different exhibitions. Now, IRIS+ uses AI to use the data collected from those interactions, converse with visitors, and connect them with social and environmental initiatives that focus on bettering the future.
Museums using AI to Engage
An exciting use for artificial intelligence involves audience engagement and can be achieved both inside and outside a museum’s four walls. In 2016, Tate partnered with Microsoft to issue its IK prize to digital creatives who could use a form of AI to allow the public to explore, investigate, or understand Tate’s collection of British art in new ways. The winner was Recognition, a matching game of artworks and up-to-the-minute photojournalism. The program scanned 30,000 digitized artworks to create the pairs. For example, the program matched a Reuters photo of two women applying makeup with a painting from 1660 with a similar composition. Both images featured two seated women wearing similar colors against red drapery. The best matches were entered into a searchable online gallery accompanied by explanations as to why the program made the match, and a corresponding exhibition allowed visitors to compare the machine’s matches to their own.
A similar use case went viral when Google’s Arts & Culture app launched its portrait lookalike update Art Selfie in the U.S. Users of the app were asked to take a selfie and, with the help of facial recognition, the app could then search thousands of artworks to find the closest matching subject.
Image: Kumail Nanjiani
The feature quickly took off, and users shared their side-by-side comparisons across social media. Because Google Arts & Culture partnered with famous museums from around the world, it brought greater global exposure and might have increased patronage as visitors traveled in search of their high-art doppelgängers.
Behind the Scenes
Flashy examples of AI get attention from the public upon interaction with visitors, but the technology can be even more helpful within museum operations. Websites, chatbots, and analytics tools are just a few of the systems that rely on AI to make decisions and improve museums for both visitors and staff.
Angie Judge, CEO of Dexibit, points to some examples of AI currently used in museums from visitation forecasting to understanding collections by using machine vision to help recognize, classify, or pattern images. Some applications are still in experimental phases, while others, such as visitor forecasting, are up and running in some commercial settings.
“Notably, the world is still in the phase of ‘training the toddler’ when it comes to AI, helping it deal with real life situations as they emerge,” Judge says. “And it is definitely always being used in a hybrid human-machine decision context, where real people are still very much involved in contextualizing AI outputs and ultimately making decisions.”
These applications aren’t just feats of innovation and engineering, they make systems more efficient and can save museums both time and money. Chris Michaels, digital director of the National Gallery in London, has worked on experimental projects on both the visitor and collections side of museums.
“The major applications of AI will come under the hood of museum operations,” Michaels says. “In the way we measure and forecast visitor behaviours, in the way security systems work and the way energy and other resources are managed. (Artificial intelligence) should allow the realisation of cost savings in the management of our buildings. Those are often the biggest single source of operating cost in museums, and efficiencies driven by AI could transform under strain business models.”
One example of AI’s time-saving capabilities is sentiment analysis, which can be used to analyze and interpret visitor comments. Here, natural language technology is used to understand volumes of freeform visitor comments for satisfaction, emotion, and key themes or words. Judge notes sifting through these comments by hand would take much more work and assumption.
“Machine learning models trained on historic data can pick up all sorts of minute details not obvious to the human eye and quickly make granular and accurate predictions that would take months of manual analysis to achieve,” she says. “Having this sort of insight easily available, and democratized for museums of all sizes, and museum professionals of many disciplines, means their decisions are more likely to be insight-informed as opposed to guess work.”
Into the Future
New applications are being thought up every day in hopes of making life both more enjoyable and easy to understand. The aforementioned New York Times article makes note of Elizabeth Merritt, director of the American Alliance of Museums’ Center for the Future of Museums, who points to a possible AI application in which visitors could eventually interact with historical figures at history museums through chatbots that use the figures’ published writings, archives and oral histories. Imagine having a chat with your favorite painter who’s hundreds of years your senior.
The possibilities are seemingly endless for artificial intelligence’s role in museums, but there’s also a need to exercise caution as the technology evolves and museums address issues of privacy, bias and general awareness.
“(Artificial intelligence) is risk and opportunity,” Michaels says. “The critical question about any major technology, and the companies who create them, is how we make them fit to the public purpose of our institutions and how we retain value in the public sphere. We can take great benefit from AI, but in so doing we have to make sure we know the ethical responsibilities we have, the rights of our audiences that must be maintained and how we create socio-economic value for ourselves and our private-sector partners from what we do.”
As AI further enmeshes itself into our daily lives, as well as industry applications that move from experimental stages to visitor- and staff-facing usage, the question remains: Will museums continue to press on into this digital frontier?
“Just as with the age of the internet, then the digital revolution, AI will quickly create a world of the haves and have nots,” Judge says. “And I hope the museum sector will find itself of the right side of that equation.”
Image Credits: Shutterstock
About the author – Lauren Styx
Lauren Styx is a magazine editor and freelance journalist in Chicago. Her storytelling explores health, culture, sustainability, and the ways in which those areas intersect.