(S6E1) Enhancing Research Culture: How Providing Interview Questions in Advance Levels the Playing Field

Intro: 00:00:00

Welcome to the Research Culture Uncovered podcast, where in every episode we explore what is research culture and what should it be. You'll hear thoughts and opinions from a range of contributors to help you change research culture into what you want it to be.

Emma Spary: 00:00:24

Hi, it's Emma, and for those of you joining me for the first time, I'm the Head of Researcher Development and Culture at the University of Leeds. In this season of our Research Culture Uncovered podcast, I'm going to be introducing some of the initiatives we're trying here at Leeds and also talking to colleagues working across the sector on their ideas and plans. Today, I'm tackling interviews and introducing an approach that we've been using at the University of Leeds to try and improve the experience on both sides of the panel. The pilot work that this podcast is based upon has also been written up as a case study, and I'll make sure I drop a link into the show. Notes so there's no getting around the fact that interviews are stressful. It doesn't matter what side of the panel you're sitting on, our candidates will often spend a lot of time and energy preparing their application and any tasks that they're expected to do as part of the interview process. They also have to deal with the inevitable fallout in dealing with the outcome at the end. We know that candidates often lose out on their ideal job simply because they can't give their best performance on the day, whether that's due to nerves, the environment that they're in, or simply minds going blank when asked a question. Something I'm sure we can all relate to and when we actually think of it. Is an interview the best way of testing someone's suitability for a job? Most candidates will never face a barrage of competency based questions in the role that they're applying for. So why do we insist on using this process to determine if they are suitable for the job? So we asked, is there a better way to conduct interviews? A way that allows candidates to demonstrate their breadth of knowledge and suitability for the role, but that also creates a level playing field between candidates? And we think there is. As part of our work to create a more inclusive, collaborative and enabling research culture, we wanted an interview process that was built on transparency, providing a supportive process that enabled our candidates to shine. Our Dean for Research Culture, Professor Kat Davis, had seen an example of providing interview questions in advance on a Twitter feed from Jennifer de Winter, and wanted to see whether or not this would work in our institution. Could providing interview questions in advance feed into our work to improve research culture? Could we create an environment where candidates have the time to prepare and think of their best answers in advance, so that the questions become a test of a person's ability, rather than how well they can survive a pressurized interview situation. So when we came to a point where we had opportunities available within our teams, we decided to introduce pilots to see if this form of recruitment would work. Now, we did go into the process with some preconceptions. We were hoping that we would see more confident, relaxed and well prepared candidates providing higher quality answers rather than those that they had to think of on the spot. But we also wondered whether or not this would lead to longer, more detailed answers because people had had the time to prepare. Would this then make it harder to keep the interview to time? Would we perhaps lose some of the spontaneity that you get when people think on their feet? And would the interview turn into a performance? With candidates having rehearsed their answers? Would it be obvious that they were perhaps reading from a script, particularly as these interviews were being conducted online? Would this approach actually make it harder for the selection panel to decide? And would we lose insight into an individual's creativity or spontaneity through using this approach? So, without jumping ahead and giving you all the answers, we really did find that these interviews were more relaxed from both sides and that a lot of our preconceptions were unfounded as interviewers. We commented on how there was less emotional labor needed to put the interviewees at ease and to enable those interviewees to give their best performance as a recruiter. Seeing a candidate struggle to answer questions and become increasingly anxious can be really challenging and we often feel the need to jump in and save the individual. We also didn't need to think of ways to steer the question in or to rephrase the questions on the spot to help the interviewees along. Because they'd had them in advance, the responses were more concise, particularly for the competency based questions where we're asking for examples of a skill that they've used. We also found that individuals were able to draw from more than one experience and to bring in a range of experiences in response to the different questions. Often when asked on the spot, they rely on one or two that they've remembered and use this against multiple questions. And as a result, the interview turned into more of a discussion. So having those questions in advance we thought had been a clear advantage. More than one of the candidates remarked that particular questions would have been really difficult to answer on the spot and they might have struggled to think of well-rounded answers. Candidates were also more concise and those lengthy, unstructured, rambling answers that we were worried about didn't materialize and they thought that was due to interview nerves being handled better or because they didn't feel that they had to fill the time. So I've jumped ahead a bit and what I actually need to do is to take you back and explain what we did and it really is quite simple. The interview panel met in advance to decide what questions we were going to ask. And we made sure that these questions were short, clear and most importantly, that we were only asking one question. Quite often in interviews, two questions are buried in one. We then included the interview questions in an email invite, asking them to attend an interview. Now, what we didn't do was provide a detailed explanation of why they were getting these questions. And we know at least one candidate who thought they'd been sent in error so quickly copied them before the email was recalled. So what we've done in subsequent rounds is to make sure that we provide a clear explanation for why people are getting these questions and how we hope that will help their preparation. The other thing that we include are the timings for the interviews, just to let the candidates know how long the interview is going to take and how much of that time would be allowed for their questions and answers. The reason for doing this is to enable candidates to see how much time they have, how many questions there are, and then they can build that into their planning. And that really is all there is to it. The panel meet in advance to decide the questions and then those questions are shared with the candidates, making sure that they have enough time to be able to use them effectively. So what did we actually find as part of this process? Well, I think one of the biggest benefits is the candidate experience. Our candidates said they had time to prepare and to think of their answers in advance. So they really did feel like they could show their best and true potential for the role. For candidates who may be great at their job but perhaps don't perform well in the interview, this approach can help them with their confidence coming into the process. There's also a lot of time associated with preparing for an interview, so providing questions in advance can help candidates to plan their preparation time more effectively. And this can be incredibly beneficial for people who have caring responsibilities or may have limited time in order to be able to prepare for an interview. Now, from an employer's perspective, it also recognizes the effort required for people when they apply for our roles. And any willingness for the organization to streamline the effort for interviewees can only really be regarded as a positive. I've also mentioned creating a level playing field and this is really important. We need to create a level playing field not only between the candidates, but also between the candidates and the interviewers. So between candidates it does help those who are less experienced or less confident to prepare better in advance. And they really do feel like they are able to give their best in the interview situation. But it can also help neurodivergent individuals or interviewees whose first language may not be English. It really does give everybody the opportunity to be at their best now between candidates and interviewers. It provides an equal environment between both sides of the interview panel, shifting that power imbalance and giving the interviewee a more equal footing. This is the complete opposite to how most interviewees experience interviews where they often feel powerless within the process or being at the receiving end of someone else's goodwill. We believe it also leads to better selection outcomes. Having a relaxed, confident candidate gives a better insight into how they operate and whether or not they would be a good fit for the role and that has to be positive. The higher quality answers also highlight the range of experience that they have and means that recruitment panels are often in a better place to assess the suitability of a candidate. Another big positive for us working in research culture is how this process can send a positive signal about the culture that we work in and our university values. It does send a clear signal to the candidate that their experience matters and that the organization cares about them within this process and that we really do want to give them the best chance to succeed. If any other institutions are thinking about using this approach, it can also give you a competitive advantage with you standing out as an employer of choice, an employer that cares. This approach is also important when we consider talent management and succession planning. In larger organizations like higher education institutions. We are encouraging people to change roles, but candidates don't always appreciate the extensive relevant experience that they've got. Providing the interview questions in advance really does allow individuals to think about the experience and how that is transferable to a different role, which is a lot harder to do when we put them on the spot. So I've talked a lot about how it was from a recruiter perspective and that side of the interview panel, but what was it actually like for our candidates? And at this point I really do want to thank all of our candidates who engaged with the process and who were also very generous in their feedback. We followed up with some questions around how it was for them, what they found, the pros and cons and that's what I'm going to share with you now. So the first question we asked was did having the questions in advance help you to prepare for the interview? And the responses we received really did view this positively, but also noted that it was unexpected. Having the questions in advance was seen as being helpful and removed a lot of the stress involved with the interview preparation. It helped them to save time as they were able to prepare effectively and that they were able to drill in and focus on the areas that they knew would come up at interview. It also helped those candidates who know that they tend to give long unstructured answers or who perhaps struggle to listen properly in pressurized situations. So it helped them to feel more confident that they were answering the question being asked, and not one that they thought they had heard or possibly misheard. But as I mentioned before, we hadn't explained that the questions would be shared in advance, so one candidate did wonder whether or not they should have received them and if someone was going to get into trouble for sharing these with the candidates. So one piece of advice to any institutions that are thinking of using this approach is to be upfront about it. Tell them that they are going to get these questions, tell them why they're getting them, and it really isn't a mistake. The next question we asked was whether or not this process had changed the way they approached the interview. And on this question we had mixed responses. So while some said it hadn't changed the way they approached the interview, it had helped them to focus on areas they knew they were going to be asked about, and it helped them to use that information strategically. They knew how many questions they had, how much time they had for that interview, so they were able to work out approximately how much time they could give on each answer, giving them an opportunity to really refine their answers. Another commented that having the questions in advance really does only help you if you've got adequate time to use them. So if the questions are sent quite close to the interview, the benefits aren't necessarily there. But it did help them in terms of removing some of the stress. Another applicant said they'd broken down their interview preparation over several days, tackling it in two questions per day rather than trying to cram it all into one session. And that really had helped them to fit their preparation around other responsibilities that they had. So next was the biggie and whether or not this approach actually helped to make the interview process less stressful for them. All our respondents agreed that the process was less stressful and that's got to be a win. The candidates were able to approach the interview with much more confidence and knowing that they had prepared to the best of their abilities. That also helped them to feel more relaxed when entering the interview and removed the anxiety over what they might get asked. They were more present within the interview because they weren't constantly thinking about whether or not they had already used a certain example or whether it was something they could draw on again and what question was coming next, et cetera. And this process really was highlighted as a positive for those more reflective thinkers or for those people who find it more challenging to recall information on the spot. One candidate explained it really well if somebody asks you a question in a meeting at work and you don't know the answer, you go away. You have time to think about it and then you get back to them and in an interview situation you can't do that. So for candidates this means they're not necessarily able to answer the questions to the best of their ability, even though they could have done it with a bit more time. And finally we asked whether or not the candidates thought this approach influenced their answers. And again, this one was mixed. Some candidates felt their answers were more concise because of the preparation. They'd been able to refine them, worked out how much time to dedicate on each question, whilst others felt their answers were longer. And that might not necessarily have been a good thing. And on reflection they try and focus on some key points rather than trying to cover everything. But one thing they did agree on was that they had an opportunity to demonstrate the breadth of their experience across their answers and that there was less repetition. With this question, candidates did raise one of the negatives that we hadn't perhaps anticipated. Could there be a perception that because the questions had been shared in advance, they had to produce the perfect answer and that they had to come up with new information so that it wasn't a replication of what was on the application form and that they hadn't used before? And could this actually increase the pressure that candidates felt? So after all of that, we asked the interview panel and the candidates to summarize the main pros and cons that they'd identified as part of this process. In the pros column we had that it's less stressful, a more effective use of time, and helps with balancing other responsibilities. There's more ability to highlight experience and relevant examples. That it led to shorter, focused answers, that the process itself was more inclusive and transparent and was better suited to people who need time to consider their answers. And that can also be helpful if you're interviewing multiple candidates in one day. Importantly, one of the ones that we hadn't anticipated or expected was how it helped individuals to cope when they were unsuccessful. That it was a lot easier to accept that they hadn't got the role because they felt they'd been given an opportunity to do their best. On the flip side of that, there were some cons, the temptation to try and memorize your answers, which can then become rehearsed and sound, as though you're reading from a script that the answers can be longer with too. Much detail in an attempt to try and cram everything in that the spontaneity may be lost or the pressure to produce that perfect answer creeps in. And finally, if the ability to think on your feet is a requirement of the job, then this interview approach doesn't test that. And you do need to think about how that might be tested in different ways. So having reflected on this process, would we do things the same way? Well, we are still using the approach at the university of Leeds. But I think it's fair to say we have refined our approach and whilst this approach has been met positively from both candidates and the appointing panel, there are always improvements that we can make going forward. Our biggest recommendation is to make sure that candidates know that they are going to be getting the interview questions in advance and providing that explanation for why, making sure that they're getting them with enough time to be able to use them effectively. So we now make sure that when we send out an invitation to interview, the questions are included, the timings for the interview are included and there is a description of why we're doing this, highlighting the benefits from both the candidate and the recruiter perspective. One of the things we haven't done yet, but I really do think it's worth considering, is referencing this approach in the original job application. This would really ensure that candidates know what to expect, but also send that positive message about our organizational culture. It may even lead to a greater pool of candidates. We also need to acknowledge that even when questions are provided in advance, they are still open to interpretation. Candidates may read questions in quite different ways and this is an important part of the process as it does give the panel an insight into an individual's experience and knowledge. If we consider the increased pressure that a candidate might feel to produce the perfect answer, which perhaps leads them to longer, more detailed wandering into Waffling type answers, then how can we best avoid that? One of the things we can do as a panel is to advise candidates not to try and cover everything in an answer and that we're expecting their answers to be brief and focused. It is really important as a recruiter we don't make judgments on the amount of preparation that somebody makes. The panel must remember that not every candidate is going to have enough time to prepare these outstanding answers and this could be due to factors outside of their control. So we don't advise using the amount of preparation a candidate seems to have done as the only indicator for how engaged they are with the role. And we also do need to consider what to do for roles where a candidate's technical expertise really does need to be evidenced. There is a concern that by providing questions in advance, the candidates might be able to answer the questions through extensive research rather than through their own ability. And again, just noting that if the role requires people to be able to think on their feet and respond in fast situations, then perhaps this approach needs to be supplemented by other interview elements to evaluate the suitability for the role. One of the approaches we used with the competency based questions was to follow up with additional questions to dig deeper if required. And we have now built on this approach and as part of that initial invitation to interview. We do say that one or two additional questions that we aren't providing in advance can be used knowing that candidates can use the information strategically. So the number of questions they're going to have, the time that they know that they have to fit them into, it's really important when you send out those timings to include time for introductions that welcome and leave an opportunity for them to ask questions at the end. So for example, you're sending out information for a 45 minutes interview. Make sure that the candidate knows they don't have 45 minutes for their answers. Oh, and one other tip make sure you tell them that you are going to ask them the questions and that you don't expect them to just give you one long narrative of their answers. And we're really keen to see how this approach might be perceived by the sector. And as we've gone through these pilots, we have encountered some who argue that the higher education is competitive and that having a competitive, stressful interview experience is needed to select those who can thrive in this environment. I'm not sure we agree, but what we do agree on is that the process still needs to be rigorous. We need to ensure that we're getting the best people for the role and putting people into a role they can actually do. We've also heard over time that there are concerns that questions may be shared online or between institutions, with some critics saying comparisons will be made around the validity and rigor of the selection processes. I think what we would say is most of these questions are out there anyway. All we are seeking to do is make the process of the interview itself a more positive and supportive experience. So at this point I thought it'd be really useful to get the insights from our careers with research consultant Ruth Winden. Ruth has been working at the university for quite a few years now, supporting our researchers with their careers. And part of that is the interview process and how to prepare. So Ruth, I'm wondering if you can tell us from your perspective and also the perspective of our candidates how this approach might benefit know.

Ruth Winden: 00:21:28

The number one issue I see, Emma, is that the researchers worry so much about what are they going to ask me and what do they want to hear and they really fret over this. And what I see is that complete nervousness about not knowing what to expect and will I get it right and will I make the right impression. And that takes so much energy away from what we want them to show. I understand the focus and the worry, but at the end we want candidates to show themselves, to show what they can contribute. It's not that we sit there thinking, oh, there's the one answer we want. Or they're worried that they don't know the answer and go blank or all these kind of things. And I think from what I found so enlightening with this process was it really takes that stress away because people can look at these questions in advance and they can really take their time. And so we see the best of them. And my worry was often, oh, will we get regurgitated replies or do they go on the Internet and look for the perfect answer? And I'm so delighted to say no. We can really see the differences in candidates. They all had the same approach, the same time, the same openness. And the other point that I think I wanted to make is, let's not forget universities are global institutions with very diverse members of staff, and many of them are international. I know this. I'm not a native speaker, so when I'm nervous and I go to interview, my English is not at its normal level because I get nervous, it's normal. So for many of our international colleagues, it's a really big barrier to think, oh, I'll know I'll be a little bit stressed and nervous, and I know that my English will disappear right in front of my eyes, and that is additional pressure. And I think that often for colleagues who don't speak another language, that is not necessarily understandable, but it is a real fact. And I thought this move to issuing the questions beforehand makes it really easier for our international colleagues. And fairer, you could say, couldn't you, Emma?

Emma Spary: 00:23:35

Absolutely. I think that's one of the benefits that we've talked about a lot, that leveling the playing field between candidates. So the other thing that I wanted to ask you was how you see this as somebody who's been working with candidates, applying for roles, how you think this plays into their decision making. How about that organizational culture that we're trying to build?

Ruth Winden: 00:24:00

Many of the people who went through the process with us since have said, you know what, I understand I didn't get the position, but I really enjoyed the process and I thought it was a fantastic opportunity to bring myself forward and I can't thank you enough for them. They're really surprised because it is so unusual, isn't it? And so we've been fair and we've given them a great opportunity and we built those networks and sadly, it wasn't for them. But I'm absolutely convinced these people had such a positive experience, they will remember us fondly, and I think they will not hesitate to come forward again should there be another opportunity. So in many ways, you can argue it helps us also not only leave a good impression with our candidates and become an employer of choice, I think it's also a really sound reflection of what's important to us with our research culture. If we go out and want to have a positive, collaborative and equitable research culture, recruitment is a key part of that. We can't have a negative recruitment process and then suddenly switch to being all collaborative and compassionate and supportive. That doesn't work together well, does it, Emma? So for me, it's a great beginning and a beautiful continuation. And also it sends a strong signal to people this is what we mean by creating this positive research culture, and the recruitment process is the beginning of it.

Emma Spary: 00:25:28

Brilliant. Thank you. And finally, for any institutions or any individuals who are thinking of trying this approach, what would you say to them?

Ruth Winden: 00:25:38

I'd say give it a go. I admit I was a little bit skeptical with my recruiter hat on. I thought, will we really be able to differentiate between candidates? And I would say go for it, because I think it makes a really big difference for our candidates. I think you really see the better of them because you see them more relaxed. You see them giving well thought out answers and you can really see what they bring to the role more so than if we put them into a high pressure situation which, at the end of the day, interviews are for most of our people. So I would say give it a go and let us know how it works for you.

Emma Spary: 00:26:15

So, in summary, did the approach work for us? Yes. Did we get everything right? No, we didn't. Would we do things differently? Absolutely. And we already are. Going through this process has highlighted many positives, but it's also raised some of the challenges. With this approach, we continue to reflect and refine on what we're doing here at the University of Leeds and to see whether or not this is an approach we will adopt more widely going forward. But if there is one reason that we continue to use this approach and continue to push for change, it has to be feedback from one of our candidates who said it was the most open, honest and transparent interview they had had in a long time. And isn't that something we should all be striving to achieve?

Intro: 00:27:05

Thanks for listening to the Research Culture Uncovered podcast. Please subscribe so you never miss out on our brand new episodes. And if you're enjoying the discussions, give us some love by dropping a five star rating and written review as it helps other research culturists find us. And please share with a friend and show them how to subscribe. Thanks for listening and here's to you and your research culture.