How can we put the rich history of cinema literally at the fingertips of visitors?
At the Eye Filmmuseum in Amsterdam, cinematic memory as been captured in all its glory. But with more than 55,000 films stored and managed at the museum across a range of genres and time periods, how should a visitor go about exploring the collection? That is a challenge that the museum looked to address in a unique and innovative way.
By bringing together teams from within the museum and engaging with external partners from Studio Louter and Kiss the Frog, a new immersive installation known as the Film Catcher was launched in December 2022.
The 360-degree projection installation allows visitors to traverse 1,000 of the museum’s film archive from one of nine tablets in the exhibition space. Irene Haan, Manager of Digital Presentations at the museum, says,
“We wanted to present something that made the audience aware that there is a variety in our collection and that they can take an intuitive approach to finding films from our archive.”
The Film Catcher is a unique and experimental project utilising AI technology to create an interactive and visual search method that draws connections and comparisons between films that might otherwise not be found.
As Franka Schaap, Operational Director and Project Manager at Studio Louter, the content design agency responsible for the Film Catcher explains,
“What we’ve tried to show is that a digital audio-visual collection like this can be presented to a broad audience, rather than a very specific research audience.”
Much of the charm of this immersive experience comes from the intuitive and interactive format, which puts visitors in control of what they see and do in the Film Catcher environment. The panoramic space is designed initially to surround visitors with films, providing them with an authentic sense of the scale of the museum’s collection.
Instead of performing a traditional search – via film title, director or release date, for example – a visual search involves applying filters to moving images from a heads-up display. With more than thirty filters to choose from, including colours (e.g. blue or magenta), activities (e.g. dancing or cycling), shapes (e.g. circle) or framing (e.g. close up), the potential search possibilities are huge. From there, a visitor can watch a fragment from a film and, if they choose, watch the complete film.
This playful and intuitive format for exploration turns the simple act of searching into an activity that is fun and accessible.
The role of AI
Artificial Intelligence played a critical role in making the Film Catcher search mechanism a reality. By applying Azure AI Video Indexer – an AI tool from Microsoft – it was possible to make a catalogue of 1,000 films searchable, scene by scene.
The AI technology enabled fragments to be classified by everything from shot framing to rolling credits. As Irene explains, “If we had to tag each clip from a scene across a thousand films manually it would take us years. Only by using AI have we been able to create the Film Catcher within a realistic timeframe.”
Ramon van Bezouw is the software developer at digital design agency Kiss the Frog responsible for much of the technological work on the Film Catcher. He says,
“There are often thought to be some scary aspects of working with AI models. But if we use them as a tool to gather data and responsibly apply that data in innovative and powerful ways, projects like this show that we shouldn’t be frightened of investigating ways to lean on AI. Because it can, in some ways, enhance creativity rather than replace it.”
Irene adds, “The Film Catcher isn’t just about the technology, of course. It’s important that we use the tool alongside human curation to ensure that the experience we are creating is interesting and fit for purpose. It has to be appealing to the audience.”
Testing for success
Early testing of the Film Catcher initially took on an unusual format for the Eye Filmmuseum, thanks the challenges of Covid-19. Online workshops were established to present search options and characteristics that could pique an audience’s curiosity and keep them engaged.
Once the world had opened back up again and the installation was under construction, Irene says that in-person user sessions were fundamental to getting the immersive experience right:
“We didn’t have a complete installation for our testing but with just two projectors and tablets we could give visitors an idea of how they would interact with the Film Catcher. A follow-up interview then helped us to find out what worked well and what could be adjusted.”
Franka says, “One of the most interesting things we learned was how to improve the user experience of swiping from a tablet to a projection on the wall. We had to communicate to people how to interact with the Film Catcher and follow those transitions without providing lots and lots of direction.
“We worked hard to find the balance between how much text, how many illustrations and how much audio worked best to help people navigate through the app. Additionally, we incorporated haptic feedback using a vibration sensor in the tablet to redirect the visitor’s attention from the projection back to the tablet screen.
The end result was a multi-sensory, immersive installation that is much more than a search tool or catalogue listing. It is an activity station and discovery hub that challenges and encourages visitors in new and unexpected ways.
Not only has the Film Catcher received a warm reception from thousands of visitors to the Eye Museum, it has also seen the museum win the 2023 Museums & Heritage Award for Best Use of Digital – International.
In the future, the museum will look to add more films to the Film Catcher to make an increasing portion of the collection accessible through the installation. New filters and search criteria will also be tested and trialled to see how improvements can be made to the user experience.
As Irene says, “AI tools like Azure AI Video Indexer are advancing all the time so we expect that we will be able to enhance the installation over time as identification and tagging advances.”
Franka adds, “What’s great about this project is that we’re making it possible for visual collections to be searched in a visual way.
“This approach can be applied across many other areas in addition to film, of course. The techniques we have used open up the possibility of people exploring and discovering without needing to use the traditional options that need to be typed into a search bar.”
MuseumNext offer online learning for museum professionals striving for engaging, relevant and flexible professional growth content. Learn more about our virtual museum conferences here.