(S8E8) Research Impact Heroes: Stories and Learnings from the AESIS Conference

Intro: 00:00:00

Welcome to the Research Culture Uncovered podcast, where in every episode we explore, what is research culture? And what should it be? You'll hear thoughts and opinions from a range of contributors to help YOU change research culture into what you want it to be.

Ged Hall: 00:00:22

Welcome. My name is Ged Hall. And I'm an Academic Development Consultant for Research Impact at the University of Leeds.

All of the episodes I contribute to the Research Culture Uncovered podcast focus on some aspects of research impact, and they are all available via a playlist, which is in the show notes. In this episode, I'm reflecting on my experience of attending the Societal Impact of Scientific Research Conference organised by the Network for Advancing and Evaluating the Societal Impact of Science, or AESIS, at the University College Dublin, or UCD.

Those of you who aren't scientists, please don't think this isn't for me, because the context around impact is really not very dependent on your disciplinary home. And anyway, AESIS also organise a sister conference called Societal Impacts of Social Sciences, Humanities and Arts. Both of them are annual events, with the Science one usually in June, and the more arts focused one in October.

I've been hoping to go to this conference for what seems like ages, this is actually the 12th, but June is usually a really busy month for me with training and development delivery. But this year I actually got my planning head on early enough to make sure I blocked out the dates. Easy when you remember to do it.

one when it was in Cardiff in: 2023

Now working at a Yorkshire University, and hopefully you've heard the jokes about Yorkshire people being very tight with their money, you should have seen how small my room was. Lots of people at the conference looked at the picture of the room with outward empathy and inward joy that they have more generous hotel allowances.

Before I dive into my conference reflections and those of some of the attendees, I wanted to say a little bit about AESIS itself. It is a member organisation and is based in The Hague, in the Netherlands. There is a free membership tier for individuals, as well as a paid tier, which is currently 95 Euros. And an institutional tier, which covers up to 10 people.

The network attracts people from a range of organisations, not just universities from around the world who work in roles, such as evaluators, policymakers in the research, innovation, and higher education spaces, research funders, knowledge mobilization, professionals, and others. I'll put a link to the network's website in the show notes.

For me, working at a UK university, one of the really appealing things about AESIS and its conferences is the much broader focus than just the Research Excellence Framework. Yes, it was mentioned, but because the audience was from South Africa, all across Europe, North America, and Asia. As well as Australia and Aotearoa, New Zealand, there was no way it could turn into a therapy session for those of us who experience the REF's effects.

This meant that I could really focus on the why and how of research impact for three lovely days without thinking about the REF once. well almost. The conference home was the amazing University College Dublin campus in South Dublin, probably the best campus I have ever visited. During the conference, I caught up with Liam Cleere.

Who heads up UCD's Research Analytics and Impact team, who is instrumental in bringing the conference to UCD. Liam also discussed how UCD use an annual impact awards program to stimulate the development of impact case studies. The other driver for engagement with development, development of these case studies was their adaptation into impact statements for funding applications for a range of funders in Ireland.

Hi, so here I am again at the AESIS conference, and I've managed to get Liam Cleere, who's from UCD. Now, Liam, you were really instrumental in bringing AESIS to UCD, so why was it you wanted that, and has the conference kind of stacked up in terms of your expectations?

Liam Cleere: 00:04:52

Yeah, well, first of all, I'd say the conference very much has exceeded my expectations.

And to tell you the truth, I wasn't, this is my first AESIS conference, but I've had many colleagues. who've gone to them and came back with really good reports. And from looking at the agendas and previous ones, I was thinking, yeah, we have to get this one here. We have to bring people here. We have to learn.

So that's, that's really the motivation.

Ged Hall: 00:05:18

And I'm also really interested in, in the Irish context. So without the REF driver, you know, without your, your kind of research assessment driver. Yeah. What is it that you're doing that really helps motivate your academics to engage with impacts here at UCD?

Liam Cleere: 00:05:34

I think it really comes down to our funding agencies.

Virtually all of them require impact in their funding applications. And they also requires to report any impact after the project has completed. So initially that has been the big driver. So our top funder in Ireland is called Science Foundation Ireland, and they've adopted impact and impact assessments quite early on.

But our second highest is the European Union, and they're very much on that train right now. Requiring impact, especially on the more kind of, you know, applied side and societal challenges type of calls.

Ged Hall: 00:06:16

This focus on impact was really refreshing for me because most of the discussion about it in the UK is driven through the Association of Research Managers and Administrators, or ARMA.

act that I released in August: 2023

Now, this point really resonated with me because personally, I don't see my identity as a research manager or administrator. Although Simon Kerridge, who's one of the editors of the Magnum Opus, that is, 'The Emerald Handbook of Research Management and Administration around the World', assures me that I am.

However, I see myself as a research impact professional, maybe I'm just wanting to feel special, with the specific needs that that identity brings with it. The AESIS conference was the closest I've come to satisfying those needs so far. One of my impact heroes, and I have a lot, is Julie Bayley, and she put it brilliantly when I asked for her reflections at the end of day one.

Julie Bayley: 00:07:44

Well, the biggest thing is how international is this? This is ridiculous. Canada, Australia, UK, all of that. All in the room trying to make impact work, and that's glorious, isn't it?

Ged Hall: 00:07:53

It is, yeah, yeah. For us evangelists.

Julie Bayley: 00:07:56

And I get to sit next to the legend that is Lizzie Gadd. So, I'm just delighted with that, and David Phipps is over there, so I'm just going to sit happily.

Ged Hall: 00:08:03

Now because I do have lots of impact heroes, and since attending this conference I've gained a few more, I found it nearly impossible to choose which session to go to in each parallel session. Which is why I love Tamika Heiden's approach to the Research Impact Summit, because you can really consume.

Everything. If you haven't registered for that yet, then you really should. I'll even let you pause this podcast and do it now. The link is in the show notes. So what stood out for me across the three days? Well, first was the opening by Professor Kate Robson Brown, UCD's Vice President for Research, Innovation, and Impact.

The impact nerd in me loved the fact that she has impact in her title. And her description of our research impact is embedded in most of the Irish government strategies as well as in most of the 44 research funders and universities in the Irish HE sector. With nearly all of the 44 funders requiring impact in their applications, I could understand why Liam had mentioned that that was a big driver.

Kate talked about prioritising impact across the well known theory of change, from inputs, to activities, onto outputs and outcomes, and finally impact. Now that wasn't surprising, but Ross Laurence's question was. He asked if progress had been made convincing research funders to support the early phases in developing research, relationships, et cetera, which is at the inputs end and in general, most founders expect you to do that before you get the funding.

So you bring those relationships with you. His question was phrased to suggest that in his context, and he works for one of the Crown Research Institutes, um, in New Zealand, he was hopeful this could start happening soon. So when I caught up with him, I really wanted to find out more.

Ross Laurence: 00:10:04

So I think the degree of optimism is Because when we went through the journey of coming to realise that that was important, part of that journey of our collective Crown Research Institutes, we involved the key funding and policy organisation in New Zealand, that's MBI (Ministry of Business, Innovation and Employment) , in that journey.

Um, and therefore, now when we're actually having the conversations around the major, um, change in the system, it's actually become part of that, more part of that conversation.

Ged Hall: 00:10:35

Brilliant. Thanks Ross. Thanks for getting those insights. Ross also gave a presentation on the changes to the contract that governs the funding his institute receives from the New Zealand government.

He described how they had moved for an outputs driven contract with key performance indicators for those outputs. For instance, the growth in the number of species added to their collections. They do research around the land environment in New Zealand. Or how quickly a request for access to the collection or database was actioned.

So that moved to a principles driven contract was really interesting. There were seven principles and I can't describe all of them because I don't know them in detail. I only have what came from Ross in his presentation, but they included beneficial to New Zealand. And, collaborative and connected. These obviously needed lots of work to unpick them and clearly define them before attention could then turn for how to assess success in those areas.

For the Beneficial to New Zealand principle, there is a basket of measures that include one, Three case studies per annum that demonstrate how the collections and databases enable excellent research and other benefits to New Zealand. Two, tracking and listing the publications, and that's on an annual basis, that include the use of the collections and databases.

And three, usage indicators of the collections and databases that are tracked and reported annually. This change in contract and the KPIs that lie within it has led to a very different relationship with the funder, because what is in the contract gets measured, and what gets measured gets managed.

Unfortunately, I missed Ross's other contribution to the conference, which was part of a session on inclusive impact and ethics with Sarah Ann Buckley from the University of Galway and Mary Jane Bopape from South Africa's National Research Foundation. Lizzie Gadd, from Loughborough University, was also a plenary speaker on day one.

She talked about the Coalition for Advancing Research Assessment, or CoARA. Now I've engaged quite a bit with CoARA, but it was still fascinating to hear about all of its activities, including its cascade funding for three different types of projects. The first being institutional pilot calls, The second being teaming calls, were the description, which explore the adaptability of good practices in new settings.

Finally, institutional change calls, which are there to drive lasting change. I caught up with Lizzie and asked her how it had gone.

Lizzie Gadd: 00:13:19

Well, everyone else used to answer that question rather than me.

Julie Bayley: 00:13:21

It was marvellous.

Lizzie Gadd: 00:13:21

But Julie's nodding, so I'm happy with that. Um, yeah, well, this is my first time at AESIS. Um, and what struck me, I think, is just how advanced and nuanced the impact community are around assessment.

Because you don't see this level of kind of understanding and sensitivity towards challenges faced by assessment and alternative forms of assessment in any other community. So I've just been quite impressed by that. Yeah.

Ged Hall: 00:13:52

Yeah. Yeah, I think, I think, thankfully, we, we are a little bit more aware of that with the, with the challenge to demonstrate impact that we kind of got.

People are involved in all of this and we do have to kind of really think carefully about what are we asking them for, what are we burdening them with and all those things.

Julie Bayley: 00:14:09

I think that impact is one of, is part of the research ecosystem that we have to action daily. Yeah. It's not just principles.

Principles are there. But it's in the action, it's trying to preserve the space for research, and deliver on government agendas, and all of that stuff. So, so many people in the room, you know, so many of us work at that intercept every single day, and it's I think it's borne out, and you've seen it, like, today.

Yeah, for the first time. It's there, that's what we're doing.

Lizzie Gadd: 00:14:34

Yeah, yeah. And you can't, you clearly can't count impact. No. You can't count it, you can't put it on your scales and weigh it. You know, unlike publications and kind of academic impact, we've been counting it for so long. And so I guess this means you've had to be more creative about how it does that.

And so it's just open and inspired and yeah, it's really encouraging.

Ged Hall: 00:14:54

Moving on into the second day, the opening plenary included Professor Ellen Hazelcorn, who is a managing partner at BH Associates, and Emeritus Professor at the Technological University Dublin. Ellen reminded us that we have come a long way with research assessment, that there was still much more to be done.

For instance, although metrics are problematic, she reminded us that peer review is too. This self regulation in other professions is much less tolerated, but academic culture and geopolitical competitiveness have focused too much attention on elite models of knowledge creation and self reflective modes of accountability.

The increased impact emphasis on impact can lead to a threat of trust or politicisation of research, which needs to be countered with more effective and less biased accountability mechanisms. We then moved on to Sean Newell from Elsevier, who described their new tool he hoped could move the conversation about research impact into a more common language.

Ged Hall: 00:16:02

He talked about nine areas for framing those stories about research impact, which included significance and reach, which are no surprise for UK people as the REF uses these, tangibility, evaluating how practical or tangible outputs are, the novelty, of the research and those outputs, the durability of those, um, of those ideas, how collaborative, um, the research has been and the, and the routes to impact have been, how transferable things are between one sector and another, the policy relevance and the level of engagement.

The tool uses lots of existing data from X, X, Elsevier, and other platforms for including ResearchFish and the Faculty Activity Reporting, a US tool, as well as the user, the user of the tool's own data. This is then fed into an AI tool with what Sean described as an unbelievably complex prompt to generate narratives in those nine areas.

The talk definitely generated lots of debate about how responsible this all was. I've put links to posts on X and LinkedIn to ensure I'm not misrepresenting people's concerns about it. However, the tool is still in development and I hope these concerns will be taken on board during the beta program.

Which Elsevier are inviting institutions to join. Personally, I try to avoid saying a blanket no in situations like this, as long as the process for developing the tool is responsible in all of the many ways that it needs to be. And then of course, responsibility comes in again, when people and organisations start using the tool.

In the afternoon, I went to an impact clinic with the illustrious impact doctors, Giovanna Lima from Erasmus University, Esther de Smet from the University of Ghent, and Áine Mhic Thaidhg from the University of Galway. Not only was this loads of fun, but was really efficient and effective. Everyone who came along got a few minutes to write down a challenge using a specific template.

Then they described this challenge to the others at the table and one of the illustrious doctors, who then got a small amount of time, I think how long general practitioners have to diagnose and prescribe, to determine a prescription. Everyone, when they were a patient, scribbled loads of notes about the various prescriptions they received.

I'm really hoping I can get Giovanna, Esther and Áine to repeat this on the podcast in the future. This was a real example of the community coming together to really help each other. Something like the hive culture that Emma talked about in her episode about bees. Now, when it comes to impact, that's really important.

Well, anything really. Because real world leading expertise, I believe, is a collective phenomenon. Tamika Heiden and I discussed this in a post for the London School of Economics Impact Blog, and the link is in the show notes. The hive thing got me thinking about who was missing from the conference. I was mulling this over, and the really obvious answer didn't hit me at all.

See what I mean about expertise being a collective phenomenon? But when I spoke to David Phipps of York University in Toronto, he gave me the answer almost instantaneously. So David, as an, as my impact hero, what's your reflection, and you've been heavily involved today on panels and workshops, etc. What's your reflection been on it?

David Phipps: 00:19:33

Do you know what's interesting? It's the reflection, I've been at many of these AESIS conferences, amazingly talented people. But we miss the other side of the table. We miss the practitioners, not practitioners of impact, but people who are using evidence, people who are policy makers, community organisations, industry.

We don't have that partner side. And I, I really think that, um, the conversation would benefit from having that perspective.

Ged Hall: 00:20:06

That's interesting. You mentioned that actually, just Just brought it up because, uh, you know, Vicky Ward at, uh, St Andrews. Yeah. So she chairs the UK Knowledge Mobilisation Forum. And, uh, and that does get practitioners in the room, practitioners mainly in the, you know, in that knowledge, mobilisation, brokerage role, often charities, quite often health space as well, mainly because of where it grew from.

But yeah, that's, uh, It's fascinating they don't, even there, you don't get the, you know, maybe the patient, but you do hear that voice through the presentation. So, yeah.

David Phipps: 00:20:45

Um, I do cause, uh, Research Impact Canada, one of my organisations runs the Canadian Knowledge Mobilisation Forum. And we do have a great diversity of participants from academia, from nonprofits, from think tanks, not so much industry, um, but, um, certainly funders.

We get a lot of funders there. Um, but not those, end users, not the patient organisations, not the people with lived experience. So I do think that's an opportunity. But the thing is, I get paid for being here, right? They wouldn't get paid. So, so there's got to be, um, some enabling to allow them to participate.

Ged Hall: 00:21:23

Absolutely. Thank you, David.

David Phipps: 00:21:24

You're welcome, Ged.

Ged Hall: 00:21:26

I had three amazing days in Dublin at the AESIS conference and came away with the following thoughts and observations. 1. Impact happens through relationships, we all know that, and those relationships need to be characterised by mutual benefit. The conference and all the attendees really modelled that, but as David pointed out, it'd be great to have some next users to really expand the dialogue.

Differences in context between countries really aren't as big as we sometimes say they are. The challenges were really similar, whoever I talked to, and wherever they were from, we are all pulling in a similar direction and hitting the same bumps in the road.

So Ross, it's a long way from New Zealand to come to, to come here to Dublin.

So what was so important about this conference for you?

Ross Laurence: 00:22:15

I think it's the collegiality and the confidence you get from hearing stories that are the same as your own stories. I think, um, when you are, um, championing something that's not necessarily, uh, supported by people around you, you actually need colleagues.

And I know I'll go back from this conference with a lot more confidence and a lot more energy to push. And, and it's always helpful telling your managers that this other organisation is doing this. Um, why aren't we? That gets their attention.

Ged Hall: 00:22:48

But really the big realisation for me was how far the dialogue around impact has come since I joined it in 2011.

Back then it was probably characterized by the questions, what's that? And how do I do it? And now that's really moved on to how do we do it better, more equitably, and with less negative effects on researchers themselves and on the communities they're working with. Unfortunately, the three days passed really quickly, and I've missed being immersed in the people and the debates, so I really hope I get a chance to do it all again.

Thanks for listening.

Intro: 00:23:26

Thanks for listening to the Research Culture Uncovered podcast. Please subscribe so you never miss out on our brand new episodes. And if you're enjoying the discussions, give us some love by dropping a five star rating and written review as it helps other research culturists find us and please share with a friend and show them how to subscribe.

Thanks for listening, and here's to you and your research culture.