Museum Uses Artificial Intelligence to Watch Visitors

It may sound sinister at first, the idea that modern surveillance techniques might be deployed against visitors who turn up at museums. After all, this is just the sort of thing you might expect of a dystopian novel with a Big Brother-like presence. However, the Bologna Museum, in northern Italy, has introduced just these sorts of high-tech measures to show how much attendees appreciate works of art in its collection. An Italian research and development agency, known as ENEA, developed an artificial intelligence system that it claimed would be able to keep track of how visitors look at artworks in a gallery setting. Now their system has been installed at the Instituzione Bologna Museia to assess its effectiveness in the field.

Museum curators would often like visitors to study the art on show in museum galleries at greater length. For example, Slow Art Day, an international project that operates annually asks gallery-goers to spend up to ten minutes looking at a single artwork. However, no one can really say for sure how long the average visitor spends looking at individual examples of art. That looks set to change because the team of researchers in Bologna have now turned the tables somewhat and turned their gaze – albeit an electronic version – onto art lovers themselves.

ENEA, which is a national Italian government agency designed to promote and develop new technologies, installed 14 miniature cameras to watch the comings and goings of gallery visitors at specific pieces of art in the museum. Crucially, artificial intelligence (AI) process the images that each camera captures so that the reactions of people can be studied. This means that things like facial expressions, the posture of people and even how close or far visitors stand to a work of art when viewing it can all be determined electronically. With the system in place monitoring Bologna’s municipal art collection, so it is hoped that the data collected on a centralised server will offer sufficient information to make assessments and broader conclusions about which artworks are most viewed, responded to and why.

ShareArt and Data Processing

ENEA’s project, known as ShareArt, was installed at Bologna’s municipal museum in July. It is currently gathering data from its cameras. Monitoring visitors’ appreciation of the art on show, it is expected that the innovative system will allow curators and others to determine how much attendees appreciate a certain work, for example. The information captured is digitised from the video recordings of facial expressions and then analysed through big data application systems. This is effectively a huge number-crunching process that assigns certain values to the information the cameras pick up. In other words, no humans are involved in assessing the footage – the entire process is automated through big data AI processes.

Some of the tech experts at ENEA, including Riccardo Scipinotti, Stefano Ferriani, Giuseppe Marghella, and Simonetta Pagnutti, have made grand claims about ShareArt. To begin with, they think their system will automatically detect faces looking at any piece of art and be able to distinguish this from those passing by and other face-shaped objects. The relationship between individual works of art and the ShareArt system is achieved by using a camera that is positioned close to the painting or sculpture in question. According to ENEA, ShareArt acquires data based on the behaviours of the people it has identified as observing the art.

Such data includes information on the path that might have been taken to approach a work of art, useful information for any curator who wants to work out the best way to arrange multiple artworks in a gallery for maximised impact. Another metric the system collects is the number of observers who are looking at a work of art at any point in time and overall. Crucially, the time and the distance of each observation is also monitored, allowing curators to better understand how the space in front of each artwork might be better planned. The ENEA team also claim that the gender, age, and social class of an observer can be determined, as well. What’s more, they also think that their system can work out the mood of gallery attendees as they view art based on their facial expressions.

Museum Reaction

“What is the nature of the appreciation of art and what are the variables that might affect it?” asked the Bologna Museum’s president, Roberto Grandi. “These are questions that often heard within the walls of a museum.” The president went on to say that traditional answers do not cut it so technological solutions are the way forward. Grandi reckons that the project will help curators to better comprehend visitors’ behaviours when they are looking at art. “[The system will offer]… a deeper understanding of the dynamics of perception in art appreciation by using AI to process a great deal of data,” he said.

For Grandi, and perhaps many other museum directors around the world, the system could reveal many hitherto misunderstood truths about the way people interact with exhibits within cultural institutions. As the president of the Bologna institution put it, the project is nothing less than a ‘fascinating journey’. “We are pleased to be making it with such an excellent scientific institution as ENEA,” he added.

Other Applications

Interestingly, the ShareArt system also comes with a number of other data acquisition devices that are equipped with a camera. These can be used to collect information and share it with a central storage and processing server for purposes other than curatorial ones. For example, ShareArt can also be used to improve the safety of museums, ENEA claims. Just one suggested technique would be to verify that attendees are using masks correctly and applying social distancing rules.

It has even been considered that the system could be adapted such as that a visual signal could be deployed to remind visitors of their requirement to comply with any such regulations. Is Big Brother watching attendees at the Bologna Museum for benign reasons? For now, that certainly seems to be the case but, as with all technology, new applications for it could be thought of at any time.