Making Museum Digital Success

 

High-performing digital teams are critical to making organisational change happen. But how do organisations identify how big their digital teams should be, how they are structured, and where they belong in the organisation? How do they define and measure digital success? We surveyed 56 GLAM organisations across the world to find out. We explore how digital is being shaped and is shaping museums and cultural institutions today. We reveal how organisations are reconfiguring their digital teams to define success and drive change. And we suggest next steps in helping organisations to make change happen on the journey to digital maturity.

Speakers

Kati Price
Head of Digital Media
V&A

Dafydd James
Head of Digital
Amgueddfa Cymru — National Museum Wales

Filmed in at the MuseumNext Digital Summit 2019

Kati Price: Hello, and thank you everybody, and thank you, Jim, for inviting us to speak. It’s slightly daunting, it feels like we’re about to start a burlesque show. Daf’s just sorting out his tassels here in the side. Thank you so much. So my name’s Katie Price. I am Head of Digital Media and Publishing at the V&A in London.

Dafydd James: My name is Dafydd James, I’m the Head of Digital at Amgueddfa Cymru, National Museum Wales.

Kati Price: So we’re here to talk to you today about making digital happen. So we’re going to explore a bit of research that we’ve recently done. And to give you a bit of background, Daf and I are both a bit nerdy when it comes to team structures and organigrams. So having poured over a number of them, we thought there’s really something here that’s of interest, well, at least we think so, to the broader sector.

So we started some desk research looking at how people structure digital teams, what good looks like in that sense, but we found there wasn’t really much out there. So, that prompted us to do a survey of nearly 60 organisations. And then we followed those up with some in-depth interviews with some leaders across the globe, exploring what makes for a successful digital team.

We’ve recently done a follow-up survey, which is still open if you’re interested, to really probe a bit deeper into measuring digital success. And we’ve had nearly 80 organisations contribute to that. So thanks to all of you who’ve participated. And that really focuses on the measurement side, which Daf will explore in a bit. So that’s the background.

But what are we talking about today? We’re exploring the idea that to make digital happen, we really have to accept that we, as a sector, have some problems. We have a skills problem. We have a measurement problem. And we have a commitment problem. And I’m going to start with the first one. It’s a skills problem.

We asked our survey, what are the most important skills for a digital team or people? And this is what you told us. And these are in priority order. So the most important skill for a digital team to have is technical leadership. So that’s about translating business and user needs into digital platforms, making sure you’ve got a really robust architecture that can scale up for the future, and that’s sustainable.

And then there’s content management and editorial. Obviously that’s the stuff that sits on top of that technical architecture. Data management and analysis is obviously key in understanding what’s working and what’s not. And then we have social media, a means of getting that content out there, and not locking it up on your own platforms. And then we’ve got product management, which is still relatively recent to our sector, in that we are moving away from thinking about big digital products. And instead we’re thinking about digital projects, and thinking more about digital products, thinking more about the life cycle of the things that we build and who we build them for.

We then asked, what are the digital skills that are most lacking on digital teams? And these are they. Again, a priority list. And you’ll see there’s a few other interesting areas at the bottom there that we consider to be lacking on our digital teams. So digital design being one of them, but also some of the deeper skills involved around e-commerce, IP, et cetera.

But I want to draw your attention to the top there. They look familiar, don’t they? So the things that are most lacking are data management and analysis, web and app development and technical leadership. Which struck us as quite worrying. What’s going on here? Why are technical leadership and data management and analysis right up there in terms of the most important skills, yet on the other hand are most underrepresented on our teams? So we asked ourselves, does it have something to do with pay? And the answer is yes; yes, it probably does.

We then asked very directly, do we pay digital folk enough in the museum sector? And the answer is no. Chances are, if you ask a bunch of people, are you paid enough? They will say no, we are not paid enough. And this survey was no different. But what’s really interesting here is 60% of us think we’re not really paid enough. Only 25% agreed or slightly agreed with the sentiment that they’re remunerated enough for their work. But what’s also interesting here is the geographical breakdown, actually in the UK, we’re a bit more positive than our US counterparts.

But it’s not just a skills thing or just a pay thing. It’s a literacy thing too. And I’m going to explain a bit more about this. Digital skills and literacy are cited as one of the major barriers to digital ambition and growth within the sector. And that’s from a report recently out by Nesta, but it’s also supported by other reports out there. And that was actually confirmed by our survey. So we asked, what is the right location for digital teams? And actually the answer was just put it where it fits best. And what was far more important than the location was actually the digital literacy of the person that you’re reporting into. That person needs to be an ambassador and a champion for digital. And they need to understand what they’re talking about.

So, onto some practical advice. What are we going to do about the skills problem and the related pay and literacy problem? We suggest you think about doing some salary benchmarking amongst similar institutions to yourselves. And really these can help make the case for ongoing investment in these missing skills. We’d also say check out the art museum salary transparency spreadsheet, a very illuminating spreadsheet that reveals what people are paid across our sector. But also think about ways that you can start articulating what the absence of these skills is having on your digital estate and activity. What is that detrimental effect that the lack of skills is having, and really clearly articulate that to the organisation.

Or think about, if not recruiting directly onto your team, think about consultancy and training, but also getting somebody on a retainer. That’s something we’ve recently done at the V&A. We don’t have a digital analyst, we failed to recruit because of a pay issue. And so we’ve actually got an agency on a retainer basis, and though not ideal, that’s providing an excellent support for us.

Also, you can think about how to explore improving digital literacy within your organisation. And when you’re thinking about that, do think about an organisation-wide approach. There’s obviously brilliant initiatives out there, like One By One who are looking at exactly this issue. But also think about mentoring and coaching opportunities outside your institution, but also within it.

We’re talking about the immediate lack of skills, particularly around technical and data management, but also think about the things that you already have in-house that you are doing well. What are you doing to continually invest in those areas like around content development, product management and social media. But also don’t think just about the here and now, think about future skills. Think about emerging areas such as AI and 3D scanning and the skills that those are going to demand on you as a team. And making sure that you’re thinking not just about the immediate short-term, but into the longer term.

I’m now going to hand over to Daf to talk about problem number two.

Dafydd James: Okay. So we’ve got a measurement problem. And-

Kati Price: You don’t have notes.

Dafydd James: I don’t have notes.

Kati Price: Let me switch it into notes mode.

Dafydd James: So be prepared for lots of graphs in this. We did a whole survey on what you’re doing to measure digital success. And I’m going to talk a lot about KPIs. So a KPI is a target that says how well you’re doing, or not. And a more formal explanation there. There are of course, other ways to measure progress. Okay. So we asked, do you measure the impact of your work? And you told us, mainly yes. However, rather worryingly 42% of you don’t measure digital success. So what’s going on here? Why are 42% of us not measuring digital activity?

So, we also asked what makes it difficult to set and measure your KPIs? You told us. So at the top there, lack of time. I mean, if you’re investing a lot in your digital products and your digital activity, it’s probably a good idea to measure how well those things are doing. I think that’s a pretty poor excuse.

We don’t have the right skills. I mean, that’s fair enough. Lack of interest within the organisation or lack of organisational buy-in also rated fairly highly here. As did technical issues. A handful said, I don’t know. And a handful said, it’s all good.

But generally, I mean, all of these things, they basically lead on to a bigger question. I’ll come to that later. So what are you measuring? Okay. You told us, mainly reach, which is fair enough. You’ve got things like reach, which is reach? Oh, it’s vanity metrics-

Kati Price: Overall impressions, etc.

Dafydd James: Overall impressions, things like that. Yeah. So impressions of social media posts and things like that. Then you’ve got engagement and volume. I mean, engagement, you’re thinking through things like comments, a bit of the deeper analytics. Volume, this is scale, this is vanity metrics, this is things like web visits.

Conversion, you’re thinking a bit more about how to get ticket sales and memberships and sale of products in your online shop, things like that. Access, digital collections, things like that. And then reputation, monitoring things like TripAdvisor.

So we also asked, if your organisation was able to measure one thing, which would be most useful? So in the blue, this is what you’re measuring, in the red, that’s what’s important to you. So you see a couple of those areas, that there’s a bit of a gap. So, if you’re looking at reputation, obviously that’s what you’re measuring, but is it that important? Maybe not. Volume. That’s an interesting one. Again, coming back to the vanity metrics, there’s a huge gap between what you’re measuring and what is important.

So the question is, why aren’t we measuring the things we value the most? And as the brilliant Chris Unitt says, “Because it’s a lot easier to measure reach and volume than conversions or reputations.” But why is that? It’s because of the streetlight effect. So a type of observational bias that occurs when people only search for something where it’s easier to look.

So why do you measure digital impact? Let’s dig into this a little bit. I’m not sure if you can quite read what’s going on there, but basically the big ones here are to continuously improve what we do. Brilliant. To understand the impact our work is having. Very important. And to help us prioritise our roadmap or workload, those are kind of peaks there. And those are very valuable points to make.

However, what about this bit? To keep trustees and senior management happy, to keep government and funders happy, to secure more funding. I mean, we gave them an option of checking three, but this on its own, if it was just to keep trustees happy, it would be quite worrying. But there is a point here. There is an element of measuring that we have to do to keep our funders happy. After all, they want to know what we’re doing with their pounds, especially in those organisations that are fully funded and free entry. So it’s worth keeping an eye on that. And also I think there’s a point here about informing our funders. What do we see as valuable in terms of measurements?

Okay. We also asked, what happens if you don’t meet your KPIs? So, maybe lower income, that means that we don’t sell as many tickets. We change our strategy. Maybe we get a little bit less resource. So, trustees don’t put some money into the team or maybe the government doesn’t give you as much, as we touched on earlier.

But this is worrying, little or nothing. So you told us that what happens if you don’t meet your KPIs is basically it doesn’t matter. Should we be more accountable? For those that are measuring KPIs, should we be more accountable? Should there be some consequences if we don’t meet those KPIs? After all, we’re setting ourselves the targets here.

Okay. So what to do on the measurement problem. Work out what to measure first. Focus on impact rather than deliverables. It’s not about creating the thing, it’s understanding how effective the thing is. Engagement is a good start for museums. You’re digging a bit deeper than those vanity metrics. And then conversions, these matter more for the more commercially minded organisations. So it could be performing arts, it could be those that have massive paid for exhibitions, like Katie and the V&A.

Okay. So the other thing to do is to get the team on board. So make sure you’ve got a shared understanding of what success looks like. And then communicate that. Build up data analysis skills. Of course, one of those skills that is lacking within our organisations. Be transparent on how you’re progressing against the success criteria. You need to be transparent. If you’re failing, you need to augment that. But more importantly, you need to understand why and demonstrate what you’re able to do to mitigate that.

Oh, and finally, very important, align your digital objectives around those of the organisation. So as Ros points out quite wisely, “The digital vision should ultimately support the mission for the organisation. A digital strategy should never be separate from the overall strategy, but a means to achieving.”

So I’m going to move on to the third bit, quite swiftly now. We’ve also got a commitment problem, as Tarzan states here. And that’s because, lo and behold, we’re not investing enough in digital. So I need more money immediately. Okay. So we asked, is your digital budget big enough to match your ambitions? No. To reiterate, no. So you told us that basically most of you have an operational budget of less than £50,000. Obviously there are lots of small to medium-sized museums that fed into this survey, but it is interesting to see that. And there’s a small cluster with more than 500K.

Rob quite wisely states, “Breaking budgets out of yearly cycles for things like main platform development seems to be key – seeing digital infrastructure as core capital investment.”

We also asked, do you rely on sponsorship to fund some aspects of your digital offer? You told us that actually more or less occasionally, and yes you do. Which is interesting I think. And I’ll get to that in a bit.

So what do we do about the money problem? Well, obviously we need more money. But apart from that, I think you need to make the case for continuous rather than continual investment. What are the long-term goals and how can you advocate for them?Manage organisational expectation of what is realistic with existing, as well as future, budget. Quantify the return on investment. How do you do that? You measure, obviously. And consider a blended mix of core funding and sponsorship to deliver different types of projects. Because we’re still too distracted by the bling.

And as Oliver points out, “Without adequate revenue to invest in ongoing digital development, attention and effort will constantly be pulled in the direction of highly visible digital initiatives that might be more opportunistic than they are strategic.”

And I’ll pass you back to Kati.

Kati Price: So we’ve stepped through three of the key problem areas for us in our sector. And we do appreciate, you probably are familiar with all of them. But we, as well as offering a few tips on how you might tackle some of those problems, we think there’s a bigger issue here. And that we think that that’s around team structures. And that this might actually partially help us towards tackling some of those problems at a more systemic level.

So we’d like to talk to you a little bit towards the end of this presentation about different team structures and how they can actually help, support or hinder your digital success and impact.

So when we did our first survey, we thought team structures might look a bit like this. So on the left-hand there, you have outsourced. So that’s typically a smaller organisation, sometimes with just one person heading up digital. And they will obviously often rely on a team of freelancers to outsource their digital operation. In the middle is probably a model that you’re very familiar with. This is the top-down kind of centralised approach. And then on the other side, you see a decentralised approach. So this is what we thought teams structures might look like.

But then we digged into the data a little more, and we found out two other forms of digital team structure. They are hub and spoke and holistic. And actually you’ve got a connection between the centralised and the hub and spoke. They’re two different versions of centralised, where one’s very much more top-down where you’ve got a digital leader informing activity at the core. And then you have a hub and spoke model where you have a digital core, but you have other centres of digital excellence that are working in a highly coordinated networked way. And then you’ve got holistic, which is where digital is truly dispersed across the organisation with high degrees of digital literacy.

So our results said that most of us right now are gathering digital resource in one location using a centralised model. And that’s probably the right decision right now. But we also noticed that the structures of digital teams are evolving as they draw in other functions, such as visitor experience, marketing, publishing, for example. And it was this idea of evolution that really got us thinking about whether there is a way that we might be stepping through these different team structures.

And this is when we struck upon this. We realised that actually digital team structures do map onto the idea of digital maturity, i.e. those organisations who’ve truly embraced digital at a deep systemic level, at an audience level, to really revolutionise and transform how they operate.

So just to step through these different stages. On the left-hand side there you have the sceptics. So this is where you see the decentralised model where things are slightly fragmented. There is digital stuff happening, but they’re still quite new to the digital game. Next up you have got the adopters using the centralised model, and that’s where you see a deeper investment in both skills and infrastructure. Hub and spoke is the collaborators, where for digital to succeed, it’s truly genuinely networked across the organisation. So you have everything in place to support that, including governance, training, et cetera, and good solid communication. Differentiators are beginning to use digital to really differentiate in terms of their customer experience. So this is where you are using and leveraging data to transform your offer to customers and to visitors. And we’d say most of us have not reached this point yet. Although in our latest version of our survey, some of you have apparently so, we’re all ears. We’d love to know more about how you’ve achieved this, because it is hard to get to that point.

As two very wise people once said, “Digital teams are part of a shifting culture rather than a permanent structural fixture.”

Dafydd James: Wow, that is so insightful.

Kati Price: It’s deep isn’t it?

Dafydd James: Yeah, deep.

Kati Price: So here are some summary actions that we think you might want to take on board. We’d like you to think about how you might actively structure your teams to push forward towards digital maturity. So if your organisation hasn’t yet centralised its digital provision, that’s something we’d encourage you to think about. Most teams will need to go through that centralised mode rather than skip straight forward to this holistic approach. And if you are currently centralised, think about how you might move towards a more networked hub and spoke model. And if you’re already at that stage, think about what you need to put in place to really achieve that nirvana of holistic approach. That will require you to have those skills, particularly around data and tech leadership to get to that point.

So we hope that we’ve stepped through some of the insights from our survey, some responses, some tips that we’ve got for you. But we also want to talk about some concluding thoughts around how you really can make digital happen. Digital happen for you as an organisation and for your audiences most importantly.

We would encourage you to be brave, be prepared to try new things, seize an opportunity, and to make mistakes. Be prepared to pivot, change, or stop projects that aren’t working. Or restructure roles and teams if they’re not getting results.

Be you. Understand what’s unique about your organisation and how digital can support it. Don’t blindly follow what others are doing if it doesn’t work for you.

Zak concludes with a really positive thought for us all that, “Digital teams are still in their infancy in our sector. We’re building the future digital teams right now, and we should share, share and share some more.”

We’d like to thank you for inviting us here to talk today. We’ve got loads more research we want to do so please get in touch if you’d like to help us in that. Especially if you’ve got money. We do have a survey open at the moment, so we’ll post a link on Twitter about that. But thank you very much.

Dafydd James: Thank you.

Sarah: Thank you so much. Please join me on stage both of you, Dafydd and Kati. Thank you so much for doing your research. And we already have got a lot of questions on Slido. But you are here the entire day, so if your question wasn’t mentioned by me, please go to Katie and Dafydd in the break. So the most popular question is, what does digital literacy mean in practise?

Dafydd James: Wow.

Sarah: Okay. Thank you.

Dafydd James: Okay. So if you want to dig into this, I would point you towards Lauren Vargas, who’s working on the One By One project-

Sarah: There she is, here.

Dafydd James: And we’re doing a lot of work with that project on breaking down the understanding of what digital skills are. And there are different components to that such as digital capability, digital literacy, as well as part of those skills, and digital capability. Yeah.

So yeah, but literacy, the way we were referencing here is around that kind of shared understanding, I guess, and appreciation for what digital is, how it needs to work. But more importantly, especially when advocating for resources that what is required. And what is required in those teams. So that kind of understanding of what needs to be done and how you can help it as well.

Kati Price: I think when we pointed out the skills gaps, what we’re saying is you can be literate and digital without being an expert in tech leadership, app development and data management. So as a leader, you need to understand why those things are important, broadly understand how they work, understand the technical architecture diagram, not necessarily be the one to make it, understand data and insights and why they’re important, and why they are connected to driving impact.

So just being able to understand and read across those different areas from editorial, design, product, to technology. And being fluent in the language that all of those entail.

Sarah:  So in other words, just be very precise and holistic actually also in your approach, like every factor should be noticed.

Kati Price: Yeah.

Sarah: It’s impossible to be an expert in everything in house, AI, 3D, interactivity, et cetera, where to focus your investments, when to hire external expertise in?

Katie Price: It’s a really good question. And please don’t assume that we were saying, you’ve got to have all of those skills.

Sarah: Please tell us how to. Yeah.

Kati Price: We noticed from our survey results that at their core, digital teams are content teams. And we’re here to make brilliant stories about our collections, about things that matter to us as institutions. So that’s us at our core. We need a technology that underpins that, that might not be supported in house, that might be outsourced. But it is incumbent on the people in house to understand how technology works so that they’re commissioning and spec’-ing out the right things and thinking about technology in product terms. So having a fluency, as we were just saying earlier, in how those things work means you’re able to commission them out of house. So we’re not expecting everyone to become AI and 3D scanning experts, but when you are considering those opportunities, you really need to understand enough about them to maybe dismiss them, to maybe say, we don’t need VR right now, we actually just need some really good storytelling on social.

So I would encourage you to think about what you do best, and invest in those areas. And what you do best will need to connect to your overall organisational mission. That helps you decide what to prioritise, what to do, always refer back to that mission.

And we noticed in the first round of our survey, not everybody is, not everybody’s linking digital vision to organisational mission. So, that is the best starting point.

Sarah: Okay. Thank you. Did you find major differences in terms of digital maturity, structure and investments across different countries?

Dafydd James: Yes.

Sarah: Three please. Sorry.

Dafydd James: Oh, wow. Okay. So it’s really hard to kind of break it down because obviously being UK-based, we’ve got a lot of input from UK. And obviously to have a team, you need a certain scale of organisation as well to go through those processes. So yes, yes we did.

Sarah: So, what was the … can you just mention one example?

Kati Price: I can’t think of an example off the top of my head to be honest, but I think we found that most people hadn’t achieved the holistic model yet. So, that was across the board. I think we saw, probably in UK and US, more evolution towards the more mature stages than some of the other geographical territories that we explored. But overall there were broadly similar patterns. But I’d say certainly in the UK and US, we saw greater investment in digital and that obviously helps support more maturity along the way.

But then our most recent results have, as I said, suggested that some of us have actually achieved that holistic model. Which we don’t believe by the way. So we want to dig into that a little bit more.

Sarah: So you’re going to be a surprise visitor and see if they actually are holistic. Very good.

Kati Price: Mystery shopping.

Sarah: Okay. Final question by [inaudible 00:28:43] who also join us here on stage later today. How do you tackle measurements of return on investment across the digital and physical space? How digital engagement can lead to conversion to the museum?

Kati Price: Can you repeat the question? Sorry.

Sarah: How do you tackle measurements of return on investment across the digital and physical space? Maybe just …

Katie Price: I’ll give you an example of what I did to get a major investment in our revamping/transforming our digital estate. And that was to look at the number of people who come to the physical space, who say their starting point was the website. And we get that data through market research. And so we look at the number of people online, and the number of people in the building who say that they came as a result of an interaction through our digital estate. And that turned out to be about 8%. And so we proposed that if we just upped that to 10%, so create a better user experience, create better content to convert more people who are considering a visit to the V&A to actually come.

Just by a 2% increase, that more than paid for our digital investment, because everybody who walks through the door, we have a figure that we know what their value is because they will buy tickets, they’ll buy coffees, they’ll buy merchandise in the shop, for example. And so there’s the average spend and therefore profit per person who comes through the door. And so we just did some funnel mapping like that.

And so there are different techniques. I think ROI is a really difficult territory. You can go down the sort of financial route, but you can also say, we need a brilliant digital presence because it’s part of our reputation. I think we’re probably all familiar with using analogies to do with the physical space. At the V&A we’ve got this courtyard with some grass in it, and every three months it starts looking terrible. You can see all the sort of threadbare bits of earth showing through. And we just accept, as an organisation, we need to re-turf that courtyard every three months. And that’s just a cost, because we need a good looking courtyard. And digital is much the same. So you’ve got choices about whether to go down the ROI route or the reputational route.

Dafydd James: But with the ROI route, it’s important to understand that investment. Digital teams, digital budgets are really small compared to the overall operational budget of the institutions. And this is where you can actually leverage that to make the point for more investment, because you can then clarify the impact that you’re having.

Sarah: Right, personal question, are you already holistic organisations, do you already …

Dafydd James: No.

Kati Price: No.

Dafydd James: But it’s given us a target. I think what’s important, we’re somewhere between centralised and hub and spoke. So it’s starting to develop those areas, especially around content and editorial really, where we’re building our expertise in other teams. So, we’re making the case for it slowly but surely.

Sarah: You’re getting there. All right. Thank you so much, Kati and Dafydd. Let’s hear it for them. Thank you.