Augmented Reality in full flight: exploring digital storytelling in 3D with Wintor and the National Military Museum

Freek Teunen, CCO of Wintor AR Tours, shares his company’s recent experience of working with the National Military Museum to incorporate augmented reality into its storytelling.

In 2023, applying augmented reality in a museum setting can’t be considered new or unique. But finding ways to integrate AR into exhibitions in meaningful and authentic ways remains a challenge for cultural institutions all over the world.

At the National Military Museum (NMM) in Soesterberg, Netherlands, Freek Teunen and his colleagues from Wintor have been collaborating with the museum’s team to deliver an AR experience that is both engaging and informative.

“I see so much potential in AR and VR technology and yet relatively few organisations are really using it to its full potential. Part of that, of course, is the perception of cost. People are scared to try things because they maybe don’t have much knowledge and they think they have to spend a lot of money on extended reality experiences.

Havng written extensively on the subject of AR/VR, consulted for numerous organisations and delivered workshops on the potential of immersive technologies, Freek is better placed than most to comment on the potential for these emerging technologies. During his first foray into extended reality experiences, Freek applied a VR installation at Holland’s largest theme park, Efteling – providing an accessible virtual solution for disabled visitors for one of the park’s most popular rides.

“What we are trying to do at Wintor is make augmented reality, and particularly augmented reality tours, simple, so that anyone can start to experiment, play and learn to use it in storytelling.”

Freek notes that the methodology for planning out tours and presentations in 3D is quite different from working in two dimensions:

“I often see people initially thinking about creating AR experiences with a 2D mindset – as if they are preparing a Powerpoint. Only through experimentation and experience is it possible to understand how to place things in context in 3D.”

Freek suggests that there has been a definite shift in recent years towards AR and that cultural institutions are now becoming more open to experimentation in AR. He puts this largely down to the fact that AR is gradually being normalised – thanks to applications on social media, in retail and other everyday tech.

“I think the big shift is that augmented reality has become accessible on our mobile devices. Although, funnily enough, people don’t always realise that they are using AR when they are adding filters to their videos on Instagram or TikTok.

“The general familiarity with AR makes it much easier for museums to try more things, because they know that visitors already have the hardware.”

AR as a guiding tool and storytelling device

Asked about the role that he sees AR playing within museums over the coming years, Freek says, “I think there are two main areas: there is an inherent need for museums to guide visitors through exhibitions and collections; and there’s an inherent need to tell stories that bring exhibits to life.”

Freek suggests that finding the right balance of information to share with visitors has always been a challenge for museums. Sometimes the signs in front of an exhibit have too little information to do a piece justice; other times too much information can overload visitors. With AR tools, however, he suggests that visitors are given much greater flexibility:

“Adding a digital layer allows you to develop the storytelling, provide context and increase the information available. Importantly, this can also be done using multimedia. By adding in sound and video as well as text, stories can be told more effectively. Because not everyone enjoys reading lots of text.

“Doing this with fixed hardware would mean bombarding an exhibition with lots of screens and speakers – which is not as cost effective or practical as using AR and mobile devices.”

It is this flexibility and functionality, Freek says, that appealed most to the team at the NMM. Utilising AR tour has helped the museum to “tell the stories they’ve always wanted to tell”.

More specifically, the flexibility of AR tours has enabled the museum to bring a new context to its installations in light of the war in Ukraine. Launched ahead of the one-year anniversary of the war, the military museum was able to explain how specific pieces of equipment were being used at that precise time in a real-world military environment.

“War is constantly evolving and changing, which presents a problem for traditional exhibitions that want to tackle current events. But, through an AR tour, the National Military Museum are able to perform updates and enhancements that can be created in a single afternoon.”

Unpacking the challenges and solutions of AR

One of the fundamental challenges presented by augmented reality technology in museums is the perceived barrier or distance created by interacting with exhibitions through a mobile screen.

“While it’s great to be able to offer a personalised experience through a phone or tablet,” Freek says, “we don’t want it to cause people to become isolated or less engaged with the objects and people around them.

“When working with museums I think it is incredibly important to test and demonstrate tours as they should be experienced. Getting buy-in from a museum takes more than telling them what AR can do or even showing them a promotional video. People need to go through the experience to really appreciate how it works and understand what is the right blend of AR features for them.

“One thing I hear a lot is that museums want to attract a younger audience. But they worry that AR tours will only encourage them to spend more time on their phones – something which young people already dedicate many hours to, of course.

“This is an important conversation to have. In my opinion, we should try to excite the ‘digital first’ generation in a way that gives them the opportunity to do something useful and culturally significant with the devices they are so comfortable with.”

Another challenge that Freek and the team at Wintor are looking to address is helping institutions better understand how to design AR tours – changing their thinking from 2D into the 3D. He says,

“Often people think about an AR visual like a picture that just appears in front of you through the screen. But actually it doesn’t have to be a flat picture. Instead, we suggest blending 2D assets with 3D models to frame images and videos naturally within an experience.

Similarly, one of the most exciting aspects of AR is 3D modelling, which can be used to show objects that can’t or shouldn’t be present in real life. Freek says,

“For example, in our case, we can make ammunition such as missiles or bullets visible through a visitor’s device. Users can also bring things closer that are too far away to see clearly, or make things that are relatively small look bigger. This might be a small feature or instrument in an airplane cockpit.

“I can imagine it would also be cool to have the Mona Lisa enlarged on a wall, without 4 layers of bulletproof glass in front of it.”

One other benefit of AR is that it can help to showcase artefacts and collections that aren’t currently on display at a museum – due to space restrictions or because they are being toured elsewhere in the world. Freek suggests that, although there’s nothing quite like seeing a real museum piece, 3D scanning makes it possible to provide visitors with full-time access to a recreation of a museum’s entire collection – even if the real artefact is in storage:

“In this way, AR can help to remove the limitations of the museum’s walls or exhibition halls.”

One key learning that Freek says he has taken from recent experiences with the National Military Museum is the value of an accompanying marketing plan to ensure that visitors can easily access and utilise the full potential of an AR tour. He says,

“This can begin with a QR code on an admission ticket to make it easy to download the AR tour app. It may also be supporting videos online or on social media to show what the tour can do – giving familiarity before a visit. Or it could be suitable signage at an exhibition to remind people that there is more they could be learning if they download the app.”

Becoming immersed in the future

As Wintor’s work with the National Military Museum and other venues expands, Freek says that they will be looking to dive deeper into the opportunities for creating immersive media experiences. As tools such as 3D modelling become more accessible, this will pave the way for more detailed and widespread immersive elements.

Alongside this, Wintor AR are working with the NMM to create more virtual guides – holographic representations of individuals such as fighter pilots or tank drivers to give a more authentic feel to tours of collections or exhibits.

“After all, the best guide to offer a tour of an aeroplane and explain its capabilities is the pilot, of course. But as they are rarely available in person, we can instead capture their presentations and deliver them as virtual guides.

“We are actually working currently to create a workflow that will see us introduce more virtual guides over time. The team at the National Military Museum have now installed their own green screen (or, in this case, a blue screen to accommodate veterans wearing largely green clothing), which enables them to add new content whenever a veteran or military personnel visit.

“It’s a very exciting time for the museum and for Wintor. And I hope that other museums can learn from our own experiences and see that AR is something that can now be used in a cost effective and valuable way.”

Find out more about how museums are utilising the latest strategies, technologies and tools to enhance the work they are doing for their audiences and communities at May’s Digital Museums Summit.

Learn more about the National Military Museum here.