Film: Going beyond the traditional heritage audio guide with The Lost Palace

Tim Powell, Digital Producer, Historic Royal Palaces

The Lost Palace is a unique immersive heritage experience that brings the hidden history of Westminster in London to life.

Developed by Historic Royal Palaces through an innovative R&D project which combined technical and creative expertise, the experience gives visitors a multi-sensory digital experience without screens.

Developed over two years, Tim Powell shares his lessons from this innovative project, how can an open call lead to this innovative project, how do you create an immersive augmented reality experience without screens and what can we learn from the way in which the public responded to the Lost Palace.

Hello. I’m glad to see you all here, we thought we were running late, but you are here, which is great.   So, I’ll start that.   I’m Tim Powell, from Historic Royal Palaces, here to talk about a project called The Lost Palace. Well, not just talk about the project, talk about what we’ve learnt from that project, and we’ll be playing forward, in terms of the way we create vista experiences using technology.

So, just firstly, the bit I’m obliged to do. Historic Royal Palaces, is an independent charity. We look after six unoccupied royal palaces, we welcome 4.5 million visitors a year. And, we don’t receive any funding from the Government or the Crown, which has, kind of, shaped the way we do things, we’re a financial independent charity. And, we operate under a cause, which is to help everyone explore the stories of how monarchs and people have shaped society, in some of the greatest palaces ever built.

I work within a part of the organisation called, Creative Programming and Interpretation, which is a slightly clunky title. But, essentially, what we do is, the permanent vista offer, temporary exhibitions, and then a new strand of work, which is new commissions of art and performance. We feel that this is returning the palaces back to what they were intended to be. They were places of spectacle, they were places which were full of people, and they, sort of, sit nowadays on a spectrum somewhere between a museum and a theatre, we think.

So, the challenge, for us, is how do we use all of the incredible storytelling tools, that are available to us digitally, without undermining what is unique about the visit to a palace. And, that is, when we ask our visitors, they say, we come here to travel back in time, we come here to escape the real world, and what we don’t want is a load of technology visibly in front of us. So, that’s part of it.

This is the intro, obviously, and we’re going to talk about the Lost Palace Project, the results we achieved, the process we followed, and then the main bit will about the lessons we’ve learnt about creating experiences, and what that means for an organisation.

So, this is, believe it or not, this is the problem, okay. This is Banqueting House, in Whitehall. It’s the best room in London, it’s designed by Indigo Jones, the ceiling is painted by Ruebens. Now, the problem is with this room is that, there’s 1,499 rooms missing, because Banqueting House is the last surviving part of Whitehall Palace. And, Whitehall Palace was, at the time when it burnt to the ground in 1698, the largest palace in Europe, 1,500 rooms, 23 acres. It stretched from what is now Big Ben, to Nelson’s Column, so it’s really the heart of London, the core of London.

And, the history that happened there is absolutely incredible. It’s where Henry VIII met and married Anne Boleyn, it’s where Shakespeare performed several of his pieces, not the new one about Trump though, he didn’t do that there. That didn’t land. And, the key, what we like to do is make links between the things that happened in these palaces, and the way society is shaped today.

For this palace, it’s incredible, because the reason politics is there, the reason 10 Downing Street is there, the reason all of Government is based there, is because this palace was there. Power transferred from royal to, sort of, political. We all, I’m sure, if you’re European here, you’re very much enjoying British politics at the moment, anyway, this is where it’s happening.

So, the challenge was, how do we give experiences of this lost history to our visitors? So, this is just a short video, to show how we did it.

[Video start 00:04:04]

In 1698 Whitehall Palace, once the greatest palace in Europe burn to the ground. The Lost Palace is an experimental resurrection of its spaces and stories.

The Lost Palace

Starting in Banqueting House, a projection mapped model shows how past maps onto present. Then, on a route around modern Whitehall, through the heart of British Government, to the places where history once happened.

Burnt wooden installations, inspired by the architecture that would once have stood on that spot. Technology is hidden in a screen-less wooden device, allows focus on the stories and frees the imagination, allows you to step into historic scenes, or use as a historic surveillance device. Gestures allow participation in historic events, actions directly affect the experience, and even hold the beating heart of a condemned King in your hand.

A modern busker bleeds into the past with a 21st Century version of a 17th Century song.

[Video end 00:06:03]

So, this is the creative team. This is a shameless humble brag photo. How do I show the creative team? Oh, let’s show us winning an award. We’re very pleased to have just won the Museums and Heritage Award for Innovation. But, the brilliant team that made this were … so, it was commissioned by us at Historic Royal Palaces, the interaction designers, [Chunkon Rosia], a theatre company, called Uninvited Guests, with the sound artist Lewis Gibson, and software developers, [Calbium].

So, what we’ve got here is, inside that strange wooden object is a lot of technology. It is a phone hacked to a micro processer, it uses GPS, it uses NFC, it uses all the, kind of, sensors on the phone. But, it’s hidden away, it’s all encased in this wooden box. The other key technology we’re using, is something called Binaural Sound, which is … have people heard of Binaural Sound? It’s, sort of, getting more …

Essentially, Binaural Sound is a way of recording sound, and replicating the way the human ears perceive sound. We know how far away something is and which direction it’s coming from. Now, if you record using this fellow here, in the middle, this is the actual microphone, that dummy head, if you record using that and play it back through headphones, it splits the signal between left and right ear, and it feels that the sound is moving around you. So, if you have an actor whispering in your ear, it really feels like they’re just there.

And, we used that as the, kind of, key way of … Well, there were two types of audio in it, the guide, these are our two guides, Sharon and [Lakisha], they spoke to you from the 21st Century. But, when you went to a place where a particular historic event had happened, it had been reimagined by the theatre company, and performed by actors. But, performed around you, so it was like the historic event was happening around you, rather than in front of you, or to you.

The route was, I mean, you’ll see from the video, but just to be clear that it was outside. We started in the building, and then it was a route around London. And, around the route, they met these architectural installations, and they were inspired by the pieces of architecture that would have stood in that spot. They were made of burnt wood, so the idea is that they were the remains of a fire. So, there were eight of these in total. And, just under the skin of them, they’re lined with NFC tags, so that, when people arrived at them, they could touch devices to them, and they checked in, and then they, kind of, experienced the event that happened through that doorway, through that portal.

A large part of this is really messing with what’s now, and what’s then, what’s in your head, and what’s in the real world. So, we also had a live performer, who played the role of busker. They were singing a contemporary version of a 16th Century song, but you could only hear them through the headphones. So, if you took the headphones off they were miming. So, for all the normal passers-by down the streets, nothing was happening. And, the visitors were just …

So, the key interactions were, to check in. When we could take you to where the history happened, we did, and you checked in. When we couldn’t, you used the device to scan and find it. At times, it became a sword, it becomes a drum, it becomes a cockerel, at some point, you used to have to shake a chicken before you threw it into a cock fight, so we did that. And then, it also tracks motion and gesture, so the scenes that are on the river, as you move, it gives the noise of rowing, and when you stop it stops, it responds to you.

And, just to say, this is my obligatory Brexit slide. This was last summer for us, so this really does happen in a place that is a very current place of protest. Not the easiest place to be running a visitor offer. But, the results, we were just absolutely delighted with. These are the, kind of, general measures of technology, storytelling, practical delivery, and the overall experience.

But, this is, for me, the key ones were the 93% of people said it was unique to other experiences they’d had at visitor attractions, 92% said it brought the history to life for them, and 90% that it made them feel more connected to the past. So, these are those, kind of, emotional responses, that we’re looking for.

And, everyone uses the word, immersive now, it’s amazing, this was a visitor word cloud, you know. Not everyone liked it, this is a true visitor statement, ‘I thought it a lot of dumbed down tosh.’ And, being straightforward with you, we didn’t achieve the level of technical stability that we wanted over the summer. And, also, it’s a very complicated thing to communicate, and we really struggled with the marketing at the start. Because, what is it? You know, it’s not an exhibition, it’s not an audio guide, it’s something new. Word of mouth was actually the most effective way of people finding out about it, rather than the messaging we put out there.

And, next, we’re rerunning it this summer, and we are going to use Binaural sound for … This was, yes, this was a little prototype, we’re not allowed to put these up, unfortunately. This R&D based approach we’re carrying forward. We will carry on working with a lot of these collaborators, and we’re using Binaural sound for future audio guide, so there is a kind of direct legacy.

The process by which we did this, was the key to what it was, really. We started with an open call competition. We said to the creative industries, what can this be, how should we do this? And, we had 3,000 viewings of the brief, we had over 100 organisations through to open days, and 90 full submissions. Which is a lot more than we put time aside to go through, so it was a really difficult, but wonderful, process.

Of those submissions, we made five. We actually built five working prototypes, and during summer 2015 we tested these. We recruited groups, families, and adult groups, to come and test it, all through HRP’s social media, actually. And then, what’s the slide for collaboration, I don’t know, there’s some animals.

And then, the two most popular prototypes, we asked to work together to develop the full visitor offer for summer 2016. And, I mean, it’s probably obvious, but a massive part of this was stakeholder work, because this is the Ministry of Defence, this is 10 Downing Street, this is the Cabinet Office. London is a place of great security nervousness, at the moment, and this is the most security sensitive part of London, so we really had to work with them. And, in fact, a lot of their feedback influenced the designs, not only just the way we ran it.

And then, from there forward, I really felt … I didn’t have a post-it note slide, so I made one last night. It was an iterative design process from there. Every interaction, every part of it, we prototyped, and we tested, there were over 15 rounds of testing. And, the lesson we learned was, you have to kill your babies. We made this interaction we thought was the cleverest thing in the world, where it as about proximity GPS and compass, and you had to find all these things, and we just tested it with people, and they just had ten minutes of chicken noises, it was terrible, so it had to go in the bin.

And, different ways of working. So, you know, this is a theatre company, this is a software development company, and this is a design studio, and then this is the police, and then this is the planning process. So, the real challenge for me, was making all these bits work together, which I did to some degree of success, but that’s the real challenge.

And then, this is just a real personal bugbear of mine, that we’re always told that we need to do things different digitally, and I really think that we should hold fast for our expertise at delighting people. And, just because it’s digital, I don’t think that expertise needs to be put in the bin. My big thing is that, we’re always told about these minimal viable products, well, a minimum viable product, is not a minimum viable visitor experience. A visitor experience, is something that should be way beyond a minimum viable product, so I’d guard against people telling you that.

So, these are our lessons. We really believe that we are in an experience economy, people do not come to us to learn facts, they go to Wikipedia to learn facts. We cannot be guardians of knowledge anymore, so we must be the best people at interpreting and telling the stories of the things, of the places, and the objects, and the stories we hold. And, all of these quotes on these slides, are ones direct from visitors. So, this isn’t just me saying this, this is the actual feedback we received.

So, the invisible interface was a really big part of this. We spend too much of our lives looking at screens, particularly phones, and we found a real benefit from hiding the technology, and changing the interface, changing the way in which people interact with it. And, with this, we were able to make human actions, and interactions between physical objects, the triggers for digital content. And, what we found is, that allows people to focus on the spaces, and on the stories, and it really frees their imagination, actually, and they’re not focused in, they’re focused out into the world.

And, here is just, like I said, we’re continuing working with some of the developers, so this is our next adventure in the invisible interface.

[Video 00:16:27 – 00:17:05]

So, that’s actually looking now, how can you make the actual objects that in your space the triggers for technology. So, there’s no visible, kind of, you know, screens or triggers, or anything, and how can you do that. How can you make touching wood trigger content about how it was used, and what people did there?

The other is, when we first started talking about this project, everyone assumes, oh, what, you’re going to do the thing where you hold the iPad up, or, you’re going to put goggles on people. And, I really didn’t want to do that. I really didn’t want to do that. But, I always argue against the fact that, I do believe we did do virtual reality, I think we created a really rich virtual reality, but it was a non-visual one, it was a virtual reality using sound. And, we did augment reality, but we layered history onto it, not visuals. And, the power of sound, I can under estimate.

But, this happened also. So, if you can see in the background, this is one of our bloody installations, and someone put a Pokémon there. So, yes. And, the aim is, I think, for us, is always to connect people. And, it’s a, kind of, bold statement, but we understand the past best, when we connect to the characters, when we feel empathy, when we can understand their motivations. And, people engage with human stories, but they need a believable human to tell those stories.

And, this is one of my favourite phrases, The Uncanny Valley, has anyone heard of this? So, it’s a term in robotics, and artificial intelligence, where the closer things come towards being human, you know, to lifelike, the more freaked out we are by them, the more unsettling and strange they are. This is film called Polar Express, it’s just horrible, isn’t it, it would have been so much nicer if it was a cartoon, but it was … Yes, anyway.

Give people just enough agency. So, people absolutely want to feel some autonomy in what they’re doing, they want to feel that they’re in control. But, they also want to be looked after, and we need to look after them, especially if we’re taking them around the streets, across busy roads, etc., etc. And, I think the way I always think about this is, a really good director is always telling the audience where to look.

And, if you create a strong enough story, they will want to follow the story, and then that becomes the, kind of, drive through the experience. So, we started thinking maybe a game mechanism was the right way to do this. But, actually, what we settled on, was just quite straightforward theatrical narrative. And, we found that that was enough, the story was enough to drive people through the experience.

And then, the power of sensory technology. And, you know, one of the highlights of the Lost Palace, is this moment where this device, which has been quite playful, it’s been a sword, it’s been a chicken. There’s a lot of chickens, isn’t there. It’s the execution of Charles I, and this device starts physically beating in your hand, and it becomes his heartbeat. And, the heartbeat quickens and quickens, until the point of execution, at which it stops.

And, we’ve not used haptic technology at all, up until that point. And, the effect of combining a sensory, you know, something you feel through your hands, and something you hear through your ears, it had quite a profound effect on people.   And, I think there’s so much to be done with combining different sensory inputs to give emotional effects.

One of my favourite things was, a lady who came back and said, how did you make the smoke smell. So, there’s a scene where you’re listening to the building burning, how did you make the smoke smell?   There wasn’t a smoke smell. She looked at me in horror, and said, we could smell smoke. I said, will you email me that, and that’s the quote from her. So, something happens when you play with senses.

And then, this, okay. So, I was thinking, how can I convince people that the visual is not always the most powerful thing to do? So, what’s something really powerful that happens when you can’t see? Why do you close your eyes when you kiss? Okay. And, I looked it up and, of course, someone has done a scientific study into this, that the visual sense overrides all of the others, it’s a, kind of, animal fight or flight thing, I suppose. And, when we kiss, the tactile stimuli, or whatever, are so much, that we have to shut down sight, so we shut our eyes, that’s the theory.

So, this is … Yes, there’s a different reason why they’ve closed their eyes, incidentally. So, this is a convoluted way of saying, there’s so many senses other than sight, and they have a really profoundly emotional impact on the way people experience stories, and the way they remember those stories.

So, it’s just really, what we did is, we created a series of moments, and we really just thought about how all of the elements combined, to create the emotional experience of that moment. So, how do we use the space? How do we use the story? In what way are we telling the story? How do we ask them to interact and take part in that scene? But, the guiding thing is, how do we want them to feel? Should they feel happy? Should they feel scared? Should they feel sad?

And then, four organisations. Have I got two minutes? Yes, great. It’s really hard to know what to do, as an organisation, there are so many options out there. And, we’re at a stage, really, when it comes to technology, everything is possible. So, how do you decide what to do? Well, the first thing is, never ever to start with the technology. The technology always comes as a solution, as a, kind of, the way to do what you’ve been trying to do. And, if you can do it already, don’t do it digitally. You don’t need to put a, kind of, treasure hunt on an iPad.

But, the question is, I think, what are the things that you’ve always wanted to do, but not been able to? And, the question is, can you do those digitally? So, the challenge, I suppose, to you is, what is the impossibly analogue version of your organisation? And, I think, that’s the digital decision that should be made.

These are great too. We’ve got a physical space, this is our R&D studio, it’s three rooms, and we can just use these to experiment, to play. And, it’s absolutely transformative in the way you work, because you don’t have to commission people, the people that are working are on site, they’re inspired by the actual building that they’re working on. And, if it doesn’t work after two weeks, three weeks, one week, whatever, then you’re not committed, you know, you haven’t commissioned. And, I think it’s a really rewarding way to work with artists and creatives.

And, I have to show this, this is my latest product of the R&D studio, which was a Jacobean drag show in Polari. In the UK, it’s the 50th anniversary of the decriminalisation of homosexuality, so we’ve done a lot of programming. We turned it into a drag club for a week.

So, where do we fit into the process? Did people do this thing at the New Institute yesterday? This is great. You hold a blue screen up, and then you can choose what surface you put on it. So, what do we do? So, we’re bringing artists in, we’re bringing creatives in. What’s our job? Well, I think we’re always the guardian of the visitor experience over the artistic vision, we’re the ones that always have the visitor in mind. And, we are the ones that were focused on the final product, rather than the process. Artists love the process, but the product is always the goal for us.

And that, I think, is it from me. So, here are my, pithy or not, statements to summarise. And, the Lost Palace is running again this summer, so if any of you are in London, it’s on from 11th July to 5th September, and we’d be very, very pleased to see you.

Thank you very much.