Film: Inviting Tech Companies into the Art Gallery

The presentation was made by Ros Lawler, Digital Director, Tate at MuseumNext Melbourne on February 16th 2017.

In it she talks about Tate’s The IK Prize which invites tech companies into the art gallery to come up with an idea that uses digital technology to innovate the way we discover, explore and enjoy British art.

Ros Lawler:  And thank you ever so much to Museum Next for inviting me; I’ve never been to Australia before, and this is such a great event, I’ve met so many interesting people and heard amazing presentations. I’m aware we’re coming up for lunchtime, and you’re probably getting a bit hungry, so I shan’t go on for too long. But I’m going to start with a very brief history lesson, because it kind of sets the context for what we’re doing with the IK prize.

So in the 1890s, Henry Tate was a sugar magnate, and of great wealth and fame throughout the land, and he had the privilege, through his wealth, to go to many private houses and see fantastic art collections, and he found the experience of seeing this art to be really transformational; he learnt more about himself, more about the world … and he thought, well, if the public could see this as well, which they couldn’t at the time, perhaps it could have the same transformational effect on them, and perhaps it could be of a real benefit to society. So in the 1890s, he opened Tate, and put in there a public art collection, with the goal of transforming society.

So today at Tate, that’s our vision and our mission still, our vision is to champion the role of art in society, and we’ve done it by expanding our galleries, so we’ve now got four; this is Tate Britain, Tate Original; we’ve got Britain, Modern, St Ives and Liverpool … but the place where we really have the biggest reach is online, so we have over 15 million people a year come and visit our website, we’ve got various apps and games out there … so what Tate saw was that through the use of digital and technology, we could reach new audiences in an engaging way, and one of our ambitions, really, building on Henry Tate’s vision, is, how do we reach those people who really don’t have much access or experience of art? How are people a bit timid about it, have limited knowledge … so digital and technology is a really key means to us delivering that goal.

So, Tate’s recognised that the space where art and technology meet is important, and that we want to invest in it, but how do we do it? They also recognise it’s really quite hard to do, particularly on top of a day job; [we’re] all stopping servers from falling over, and making sure the wi-fi works. So in 2013, they founded the IK prize … [said], why don’t we have a prize just purely dedicated to tech and innovation, which would be funded, so we’ve got the resources to do it? So a very generous benefactor funded it initially, and he named it after his Auntie Irene, his Auntie Irene [Kripesman], which is why it’s the IK prize, and she was a long-term volunteer at Tate Britain, and she was also fascinated in technology. So, the format for the competition’s quite simple; there’s a very open brief goes out, and we promote it particularly to the creative and technical industries, and it says, give us an elevated pitch for how you would use technology or digital to engage people with art. Hundreds of applications come in, and we have a panel, different every year, but really interesting people, so we’ll have people from the museum, we’ll have artists, we’ll have media journalists, we’ll have technologists, who sift through these vast piles of entrants and select four, and then those four go on to really work up a more detailed idea, and then we pick a winner, and the winner gets £10,000 to keep, and they get £60,000 … it was just recently put up to £90,000, actually … to implement that idea and turn it into reality. There we go. So let’s have a quick [wander] through what we’ve done so far.

So the first winner … did anybody visit After Dark? Does anybody know what I’m talking about? A few of you … good. So, the winning idea at year number 1 was to have robots in the gallery at night; here you can see one, it’s all kind of spooky … and the idea was, I think we had four robots, and anybody from around the world, anywhere, could log on and they could remotely control a robot and drive it round. It was hugely popular. We had in the first week 100,000 people log on to try and do it … this is what they saw, these are the webcams, [obviously] going round in the dark, it’s a bit spooky … but you can see we’ve got John from China there, we’ve got Tom from Hong Kong, we’ve got someone from Clapton in London, someone else from London … so if you didn’t manage to get to log on, you could watch other people going round. So we had, in the duration of this, 500 people from five contents managed to get on and actually do the driving, and then hundreds of thousands of other people watched. So … and the press loved it, they really went to town on it, it was very popular.

And year 2, we had Tate Sensorium. So quite interesting contrasts, because year 1 had been all online, and Tate Sensorium was actually all in gallery. So the sort of thought behind Tate Sensorium is that when you go into a gallery or a museum, it’s all visual stimulation, or primarily visual stimulation, but that’s not how we experience life; we have five senses, and those other senses have a massive impact on how we see things. So the smell of bread, apparently, is more likely to make you buy a house, or you can fall in love with someone because of their scent, and retailers play right music to make you spend more money in their shops. So all these things have a very sophisticated impact on us. So within the gallery, we picked four paintings, and each painting had an experience to go with it, so different audio, different smell … we used haptics for a sense of touch, and one of the paintings was quite [a grimy] … Francis Bacon … had a chocolate to go with it, which some people quite liked it; I thought it tasted like licking an ashtray … I imagine that’s what it tastes like. But all of these really added to your sense of experiencing the artwork. So, this lady here, she’s smelling a block of something which had quite a lemony smell to it, and I wasn’t aware, but actually, your sense of smell affects the way you see colours, so when you smelt this lemony thing and looked at that painting, the blue really leapt out at you. It was quite striking, really incredible.

And here’s a painting called full stop. It’s quite an intense piece … but the lady there has got her hand in a sensor, which is haptic, so it’s giving her a sense on her hand, and she’s got headphones on, so she’s got audio coming on, and this was an incredibly intense experience, I really wish we could do it here, if we could have like a mass sensorium, it would be great. But it felt like that black hole was kind of vibrating and sucking you in; it was really intense. So four people could go [round at] a time, they each had a wristband on which measured their response through their pulse and their sweat, and at the end of the experience, everybody got an individual print-out, which said what their reaction to the art was, and then it said what other artworks they might like to go and visit in the gallery, based on their reaction to that. So you got a kind of tailor-made tool, with a really great take-up, people loved it, and went off to look at the rest of the gallery.

So last year, we took a slightly different tack. Microsoft came on board as our sponsor last year, and we looked at narrowing the brief slightly, so rather than just a general … any technology, we decided to focus on artificial intelligence, because … great with the zeitgeist, but also very important to Microsoft at the moment, and something we’d kind of toyed with playing with. So this is one by an Italian agency, a creative agency … very simply, we had a live newsfeed from Reuters, which brought in up-to-the-minute press photography from around the world, so you could log on online and you could see this happen in real-time, and then the winning agency built an algorithm which took those press photos and ran them against our database [for] 50,000 British artworks.

So there were four things that it looked at within these comparisons: it looked at objects; so cars, trees, buses … [were they in] the images … we used Microsoft’s facial recognition technology, which is incredible; it will identify faces, it will give them an age, a sex and an emotion; we used composition, so light, colour, shapes; and we used context, so the algorithm also read any data that we had about the image, and whatever was written around the news image. So you could watch it running through real-time, online, or you could also come into the gallery, we had an installation where you could see it up in the gallery … you’d kind of see it thinking … and then once an hour, it would select a match of the best thing it had seen in that hour, and publish it onto a gallery. So here’s an example: you’ve got a Syrian refugee woman with a child, fleeing, and then you’ve got some cottage children, by Gainsborough. So you can see what it’s put together here, it’s picked up with the facial recognition, it’s got that face there, similar angle … it’s perhaps picked up something about children, as well, that was within the metadata, and some of the compositional elements are similar.

But what we found was very fascinating with the recognition was how deeply it made people look at these artworks; they didn’t just go, oh yeah, snap … or not snap. People would look really deeply into the artwork. So here you might think, oh, here’s a kind of … two images but have very little in common, but actually, on … thought, the sort of slightly more romantic 18^th century picture of children feels … you know, the details there, those are children who have some hardship in their lives, they’re obviously very poor, there’s no adult there, are they alone, what’s their context? So perhaps brings up a theme of children in difficult situations, perhaps there’s something we should be doing about that.

This shows you if you wanted to click through on some of that detail, so this is what you would see on the website on in the gallery, to see how the algorithm is thinking. So it’s pulled out there, so we’ve got, again, fleeing refugees; there were a lot of images of that last year, obviously brought up some very interesting, quite political matches … but here, so it’s pulled up … it’s picked up the faces, and it’s picked up some of the prints and put those together as a match, and you could dig quite deep into how it brought those together. There were some of the matches … actually, these were some of my favourite … that were just like purely visual. Just really beautiful. Like a bunch of cars and a … [nice] Patrick Heron there.

But some of the most interesting ones, which made people look really deeply, where you think, what … those images have nothing in common with each other, we’ve got some people looking at a stock exchange and a rather forlorn-looking woman. But these were the places where people started to look really deeply. So again, if you go into the data on that, and why it was thinking … so it said, it’s matched some objects … so it’s seen white shirts, a black shirt, it’s seen some colours there … it’s not matched faces, because whilst you can see her face, there’s no faces over here … but it’s gone very big on composition. So [it’s just said], OK, look, we’ve got an arm here, we’ve got this at the angle, we’ve got similar colours in it … so these were the ones that would make people look really deeply into the artworks, and if you were in the gallery space as well, you could make your own matches.

So we had some human input within the gallery space. And this ran for two months, and we had hundreds of thousands of people looking at it, really interesting responses, and I think out of the three, the last year was very interesting, firstly because it worked in gallery and online, so you didn’t have to come in, but if you did, there was something to do there … but also in terms of really engaging people with art, and really making people look … we’re doing some evaluation, but this seemed to take people much deeper into the artwork, and if you’d had no knowledge of art, it would take you quite deeply, and you might learn something about composition. But also, I think, going back to Paula’s presentation, there’s a huge discoverability piece here, because there’s over 50,000 artworks in this database, and even at the end of this two-month project, I was finding art that I’d never seen before.

So those are the three that we’ve had. So, risk, that’s what we’re talking about today, isn’t it? So what are the risk of museums working with technology companies? First of all, a kind of fundamental question … will it work? We have no idea. Will anybody like it? We don’t know. But we’ll do our best to mitigate that. But one of the big challenges, really, is that kind of culture clash. So where museums, galleries, generally plan things two months … two years in advance, technology companies [move in] very quickly, so … and when we open this to any entries, I have to go and say to the director, we might need your gallery in a few months, but we’re not sure what for, and what it’s going to look like … and similarly, saying that to our front-of-house staff can be a challenge. But then there’s also our ability to help support this and deliver it; we don’t want to clamp down the [tech] winner with rules and regulations, what we can and can’t do, but also, there’s challenges to us, we’ve got antiquated tech … are our servers going to fall over, is our API going to work, can we actually help them deliver what we want to deliver? Challenges to our front-of-house teams? Particularly with something like Sensorium, it was a real guided tour, it was a real specialist kind of theatrical production. For our AV guys, really difficult to install, really state-of-the-art technology, with sound, with haptics, to keep all that maintained and running.

So big challenges on the organisation, which is good, because we’ve learnt a lot through that. And also, making sure that our objectives stay aligned throughout this, so letting the tech company have the creativity to deliver, but making sure that it really hits what we need to do about engagement and reach with the audiences. And then, last but not least, [Melissa] touched on reputational risk, so [Warren Buffet] has a nice quote that it takes 20 years to build a reputation, it takes five minutes to ruin it; I don’t want to be that five minutes. And as we were building the recognition, artificial intelligence, so at the same time, I think some of you might remember, Microsoft put an artificial intelligence bot onto Twitter, and within hours it had become foul-mouthed, racist, misogynist, and they had to take it down. But I think that says more about people on Twitter than it does about AI. But there you go.

So whilst that was happening, we did review what interaction we were going to allow people to have with our artificial intelligence, and decided that we would keep it in the gallery, but we thought, we’ll get this up in beta, we’ll have it running in the office for a few weeks and see what it comes up with. And the next two slides I’m going to show you I haven’t shown to anybody outside of Tate before, but you’re all lovely discreet people, so I’m sure that’s fine. So, we left it running, we were watching these matches that it came up with. Some of you may recognise these gentlemen: we have George Osborne, who was our former chancellor of the exchequer; we have Boris, who is surprisingly our foreign minister, and recognition matched it with this, which I’ve censored, because this is a family show … but we have George the Bleep and Gilbert the Bleep. The uncensored version of this is on our website, if you want to find out what those words are. So this was post-Brexit, and a lot of us rejoiced and thought, God, this artificial intelligence is very clever, isn’t it? But actually … and there were numerous ones that came up like this, with Trump and with celebrities, and whilst they were very entertaining, and some of us thought politically quite relevant, we did review this from a risk-factor, and decided, actually, this isn’t our business. We wanted to really keep the focus on art. I mean, this would have driven lots of controversy and lots of traffic, but we really wanted to keep the focus on art, so we did mitigate that risk by moderating it, and taking some of the more choice, slightly risqué, artworks out of it, so we mitigated the risk a bit there.

So what are the benefits? So this is your standard Saturday morning in August in London, it’s chucking it down. But what is not standard is queue of people right round the block at 9.00 o’clock in the morning to try and get into the Sensorium. It was so popular, and we had limited places. Lots of these people had never been to Tate before, some of them had never been to a museum before, and regardless of … it was first come, first served … regardless of whether or not they could get a place on the Sensorium, they still came in and they looked at the art, and they had an experience within the gallery, which is fantastic. So really ticking that box for us for reaching new audiences. You could say the same for the robots, we could say the same for recognition.   Not only reaching new people, but giving them an engaging experience, with a really emotional response; so many people have been in touch with us following these projects. So it really ticked the boxes for us. There’s also been a lot of learning, so as I mentioned, we’ve learnt a lot about how we put on exhibitions, how we deal with audiences, with new experiences, and we’ve learnt a lot online, and we still have our robots in the basement; we’ll bring them out again; we still own this algorithm. So they’ve got some life in them yet, so a great learning experience for us how to adapt … but also, we hope, to give people experiences of a museum that are different, a place that they perhaps thought wasn’t for them, but that they can engage with.

And also, other things we found along the way. So this one didn’t win, but we thought, God, that’s a really brilliant idea, we’re going to do it anyway; so Tate Worlds on Minecraft … how do you reach 15 million kids in one go? So we set up some worlds. This is Carnation Lily, Lily Rose, I don’t know if you’re familiar with the artwork by Sergeant, it’s one of the most famous pieces that we have at Tate Britain … this is re-imagined in Minecraft. So how to reach millions of kids. So for the IK prize at the moment, we’re on a slight hiatus, it is quite demanding to run this year after year; we’re just having a slight pause to really evaluate what’s happened over the past three years, and to think what direction we want to be now. We’re talking to different partners; if you’re interested, come and talk to me later. We’re looking for tech people, we’re looking for sponsors, so really open to ideas and collaborations at the moment.

And that’s me.

Male voice:   Thanks Roz, fantastic. We have plenty of time for questions, and we have a few. Can you [unintelligible 00:16:31] the microphone, please? [There’s just] one here.

Male voice:     Hello, thanks, that was a fascinating presentation. I’ve got a question just about the Google News image matching project, and you talked a little bit how you did a bit of risk mitigation in terms of what was selected from the collection.

Ros Lawler:  Yes.

Male voice:     I’m interested if you could tell us just a little bit more around the other end of risk mitigation, I suppose, in terms of the images that might have come through newsfeeds, and also around any copyright or IP issues you might have had in terms of reproducing those photographers’ works in the gallery.

Ros Lawler:  Yes. So Reuters dealt with the IP, so that basically, they provide a newsfeed anyway, and we [bought them in] their standard terms, so they dealt with all of the image copyright. You’re absolutely right, we had to filter the news images that were coming in, so we didn’t want dead bodies, we didn’t want to identify individuals in difficult situations, that is not what we’re about. So … and anything too traumatic. There was a lot of traumatic stuff anyway, which represented international events, refugees, but we didn’t want to single out individuals, and Microsoft were very keen not to have guns in it, and we kind of went with that, we were happy with that. Yeah.

Male voice:     We have another question down the front.

Female voice:  Hi. I was just wondering … so having been involved in the IK prize a couple of times, I think that the shortlist … you said it’s open-entry, but the shortlist of agencies and groups that worked, that in all three years have all shown a certain … certain characteristics, and certain type, and I thought it might be quite useful for groups, when we talk about collaboration, when we talk about that, to kind of make … to get your views on who are creating these sorts of works.

Ros Lawler:  Yeah, and you’re absolutely right, they have all had similarities. Actually, the majority of entrants have probably come from people with a similar background.   But we are looking … one of the things that we’re looking at the for the future of the IK prize is actually how we can vary that. We’re looking at potentially the kids’ IK prize, which could be really interesting, to work with schools. But, yeah, it does seem to be coming from this kind of a background. One of the things that filtered out perhaps slightly people from different backgrounds who were entering it, so we’d have some great entries from, say, A-Level students, but we’d have to vet them for their ability to actually deliver on their idea, which for a lot of people was a stumbling block, and within the scope of this competition, we didn’t necessarily have the support to give them, which is a shame. So people would have to be able to deliver on quite a lot of what they’d pitched for.

Male voice:     I mean … [so I’d mean] the first two, almost all … I mean, you can check the shortlist, it’s very interesting. You get these very small groups of new, young agencies, who are looking to develop and … I mean, it’s more for everybody looking for partners, and that they would have that combination of cultural background, but with high-tech capability, and I do … I mean, I admire that, it’s a very interesting idea. It changes it slightly when you’re looking for ideas from the rest of the population, rather than from those … that little group of cultural innovators.

Ros Lawler:  Yeah.

Male voice:     And where do you feel that the IK prize is … I mean, when it was founded, what … if you want to go back to the start of your talk, its purpose, who … what’s it for in that regard? Is it …

Ros Lawler:  Well, the purpose was to be … just be very broad, and to see how technology on that very broad level could help people interact with art. Very, simple, very broad level. Yeah. But I do take your point, that there are more interesting things we could do to get more diversity of the people who are inputting ideas on this, and to help them to fulfil that.

Male voice:     We still have plenty of time for questions. No? Be bold. Take risks. We have one over here … we have two. Fantastic, [I don’t know where it is].

Female voice:  Just one question, it’s probably a silly question, and it shows where I come from … how did you stop people driving the robots into the artworks?

Ros Lawler:  Brilliant question. The health and safety on that was quite extreme, so obviously, our curators were worried about the Rodin and some of the Pre-Raphaelites, so we road-tested it, and we had … it had got some sensors on it that wouldn’t let it go too near things. But, yeah, there was a lot of nervousness about that. The health and safety for Tate Sensorium was quite extensive, as well, like, could you put those [bands] on pregnant women, and could they eat the chocolate? You know, what do haptics do to small children? There was all kinds of health and safety around that. So yes, some interesting challenges for the gallery.

Female voice:  Hi, look, I’m just really interested in … you said that the sensory thing gave you a learning of what artworks people liked, depending on their reaction. I think that’s such a great learning for an art gallery, a museum as a whole, as to what works they would like. I mean, what were you gauging it on, was it their … and obviously increased heart-rate, was –

Ros Lawler:  Yeah.

Female voice:  – excitement, but it could also be fear, so I just thought that was quite a fine line.

Ros Lawler:  Yeah, and some of them, they were quite intense experiences. So you got a little graph which kind of showed your panic levels rising at certain points, and some people … we all compared afterwards … some people just flatlined, they had no emotion at all, whereas other people … really all over the place, it was fascinating. So people actually got a lot out of just comparing their reactions to each other, and then it was the curators who’ve kind of gone through and said, work of this intensity, if you’re responding well to this intensity, then you might like this other work which is quite intense, so …

Female voice:  You just mentioned comparing the levels to each other; was there a social media element, or some way that people could engage with one another about these intentions?

Ros Lawler:  People did it on social media anyway. We didn’t need to facilitate that. There was loads of these graphs popping up all over the place. So it kind of happened anyway, yeah. There’s perhaps more we could have done with time and resources actually on that data, to have done an overall round-up of what the responses had been like. I think we’ve still got it somewhere.

Male voice:     In the time the project’s run, have you had something that you would have termed a failure, and what did you learn from that?

Ros Lawler:  I don’t think I would term anything a failure; I mean, they’ve all actually … one of the successes is about how well people have responded to them. With the recognition, we would have liked a media partner. Unfortunately … we had a deal, but unfortunately, because of the state [of the world’s press], that fell through towards the end. Something like the Sensorium’s just slightly frustrating it wasn’t more scalable. But actually, that’s one of the things that was the appeal of it, was people kind of like that bespoke element to it. So I wouldn’t call anything a failure, no. We’ve learnt so much from it, anyway. And that’s one of the kind of nice challenges of having it like this, is a little bit, like Paula was talking about, getting things out the door. When you’ve got that set time of doing a competition; there’s a deadline, it’s got to be there, whether it’s … whether you would like to do more to it or not. There was so much more we could have done with recognition in terms of usability and shareability, but building algorithms is quite hard, and websites to go with it, it was quite an intense time, so …

Male voice:     One or two more? I have a question. So given the success of these projects, is there an appetite at Tate to incorporate these kinds of skill sets, this kind of thinking, this kind of practice into [core] programming?

Ros Lawler:  Yeah … so we do quite a lot of innovation with our core programming anyway. We would very much like to have a lab in house; we just don’t have the funding for it at the moment, but we’d love this to be a kind of permanent work stream, outside of the IK prize, yeah

Male voice:     OK, shall we leave it there?

Ros Lawler:  Thank you.

Male voice:     Thanks, Roz.

[End of recorded material at 00:24:36]