The EFMD business magazine

The EFMD business magazine

Producing high-quality research has long been an important strategic objective for business schools. Faculty research has typically been evaluated by articles appearing in highly regarded academic journals and by citation counts. Increasingly, however, government funding bodies are seeking more than just appearances in 4* journals, they are looking for meaningful societal impact. To this end researchers, in various countries, must submit case studies to funding bodies outlining concrete examples of impact in addition to the original research articles. In the future, the financial reward for high-impact case studies will significantly supersede the reward for journal prestige and citation counts.

There is little doubt that business schools create impact through their teaching. Many students, ourselves included, have benefitted from the cumulative knowledge base developed and taught in business schools over the past century, but it is not the goal of this short article to cover this ground. Instead, we will be looking at a more contentious element of impact: the impact of business school research and knowledge generation. We all profess to being driven by rigour and relevance in our research. Rigour tends to be defined by appropriate methodology and through recognition by being accepted in a ‘prestige’ journal. Relevance is another matter altogether. Relevance to whom? To gain legitimacy from other business school academics? To the journal editors because it fits in with the themes covered in their particular journal? Or is it relevance and reach to people outside of the narrow confines of the academy?

In this paper, we will explore the debate around relevance generally, and its manifestation as impact more specifically. We will do this by reflecting on some of the articles which have appeared in Global Focus Research Volume One – Perspectives on the Impact, Mission and Purpose of the Business School and will appear in Global Focus Research Volume Two – Business School Research: Excellence, Academic Quality and Positive Impact.

Importantly, we will also look at the various research evaluation approaches taken by those government research funding bodies and agencies that go beyond just using journal ratings or bibliometric data as proxies for impact and relevance. In doing so, we will invariably look at the evolution of the Research Excellence Framework (REF) in the United Kingdom, but also at some of the groundwork carried out in Australia which foreshadowed the REF. We will also look at more recent systems introduced elsewhere.

For better or worse, the systems that evaluate impact have developed specific criteria for evaluation, rather than the vaguer notions of impact, which sometimes circulate in business schools and dare we say occasionally in the pages of Global Focus. Certainly, for business school academics in the UK and those countries where impact is measured, research impact is a very important metric – not just because of ‘doing the right thing and making an impact with research’ – which seems important to us if not to everyone, but because of the extensive amount of research funding related to impact.

Since the 2014 REF in the UK, impact has been defined as the effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia (REF 2021). The assessment of impact has been made through the evaluation of submitted case studies which are then assessed for ‘significance and reach’.

Depending on whose analysis one prefers, the value of a (top) 4* case study continues to increase. In the 2014 REF, a 4* case study was worth about 5 times that of a 4* journal article and thus generated approximately £45,000 per annum for the duration of the period up to the following REF (Reed and Kerridge, 2017). In the 2021 REF, this had increased to a range of 5 to 12 times the value of a journal article (McKay, 2017). Looking forward to REF 2028, a top research impact case study will be worth 10 times the funding of a top 4* journal article for a smaller institution submitting up to 30 researchers to the research assessment. For larger institutions of over 120 researchers submitted, an impact case study can be worth 20 research articles (Kerridge, 2023). Thus, in a larger institution, Kerridge states that a 4* impact case study is worth up to £250,000 per annum for each of the years leading up to the subsequent REF. So, the question arises of how did we (in the UK in any case) get here?


Government spending on higher education research and development in the UK is reported as being approximately £8 billion annually. According to the National Science Foundation in the US, 2021 spend on university research was nearly $90 billion. In Germany, the German Statistics Agency states that universities received €111 billion in 2021. While these figures are clearly measuring investment differently, one thing is certain – a lot of money gets invested in research and development at universities around the world and governments want value from their research investment portfolios.

It would not be an exaggeration to state that the traditional academic research structure based on journal ratings and bibliometrics can frustrate these funding bodies as well as individual academics to no end. The quest for rigour, legitimacy and journal prestige at the expense of a visible impact outside of academia has been increasingly criticised and has gained in prominence from university-originated bodies like the San Francisco Declaration of Research Assessment (DORA) of 2013 and the Leiden Manifesto for Research Metrics of 2015. It is also regularly criticised by government bodies and politically active think-tanks. (Dowling, 2015; O’Grady, 2023; Espinosa 2015).

Ultimately, publicly-funded research funding agencies, whether in the UK, the US, Germany or elsewhere want to be assured that money is being spent in ways that are beneficial to society in the broadest sense. Among the earliest attempts to assess research output was the 1986 Research Selectivity Exercise in the UK which asked university research units to write a short synopsis of their research and put forward their 5 best papers. This was followed by the renamed Research Assessment Exercise (RAE) which expanded the process. The RAE required research units to list significantly more publications which were then assessed by a selected group of academic ‘experts’ in the various fields. Geuna and Martin (2003) criticised the ever-expanding assessment framework and posited that it had cost something in the order of £100M to evaluate 50,000 individual researchers and nearly 200,000 submissions. Reacting to this type of criticism, the UK’s HM Treasury suggested that one could go back to retaining bibliometric measurement indicators. Martin (2011) notes that this also caused an outcry (universities are quite good at being ‘whatever it is, we’re against it’).

As Wróblewska (2021) notes, efforts to measure impact have been progressively observable worldwide since the early 2000s. She points at Claire Donovan’s (2008) article which is self-explanatory, and is entitled The Australian Research Quality Framework: A live experiment in capturing the social, economic, environmental and cultural returns of publicly funded research. Donovan noted that the plans to assess research on the basis of economic, social, environmental and cultural benefits “sent mixed messages and created confusion about what impact is, and how it is best measured, to the extent that this bold live experiment did not come to fruition.”

The UK, never a country to be deterred by evidence that something may not work all that well, invariably decided that it could outdo Australia, with the result being the first effort at adding an assessment of impact not as a live experiment, but as a matter of official policy and funding. The RAE was renamed as the Research Excellence Framework (REF). It has been conducted in 2007, 2014 and 2021. Each iteration has seen some alterations to the criteria used and has intensified the focus on research impact. The 2028 REF, as mentioned earlier, will have the greatest influence yet on the monetary value of the impact case studies.

Over time, the REF has become an unavoidable feature of the UK’s academic life. It has gone on to inspire a number of studies of academic impact including Norway’s Humeval exercise for the humanities (Research Council of Norway 2017) and later assessments of other fields. In Hong Kong, the University Grants Committee and in Poland, the Ministry of Education have also followed suit.

Results from REF 2021

The most recent REF 2021 evaluated 1,878 submissions from 157 higher education institutions across all disciplines. All told, 76,132 academics provided 185,594 research outputs and 6,781 case studies based on research conducted by the various research departments. The number of required case studies was derived from the overall size of a group’s submission. More people and research outputs meant more case studies. 1,120 academics and ‘research users’ were divided into 34 subject-based units of assessment (UoA) to evaluate the submissions. The Business and Management Unit of Assessment was UoA 17.

All case studies from all subject areas were presented in comparable forms through strict templates of how they were to be submitted. Each had to choose the type of impact it was trying to describe, whether the impact was of a cultural, economic, societal, technological, health, legal, political or environmental nature. Each case study judged against criteria of significance and reach and the selection of case studies (as well as of the journal articles and books) were rated as either 4* (world-leading), 3*(internationally excellent), 2* (recognised internationally) or 1* (recognised nationally). According to the rules, 4* articles could appear in relatively unknown journals, while it was possible, if unlikely, that a 1* article may have appeared in a very prestigious journal. Similarly, a 4* case study did not have to be derived from research which appeared in a prestigious journal.

The academic game was clearly to submit as many 4* and 3* submissions as possible as they received the highest funding. Basically, 2* and 1* papers and case studies were considered failures by their submitting universities. Judging articles and case studies not only occupied the assessment panels, but was preceded by extensive internal university assessment of their own submissions. It had the additional benefit for the numerous members of previous assessment panels in creating a lucrative ‘side-hustle’ as external reviewers for the universities. Individual submissions, while having been graded, were not publicly ‘marked’. The assessment only reported back on the proportions of an institution’s submission in the 4/3/2/1* categories.

More positively, all case studies are publicly available on the REF 2014 and REF 2021 websites and are well worth perusing for any academic interested in impact – certainly for those who have similar systems already or in the works and definitely for anyone, crazy as they may be, considering a move to the UK. By cross-referencing the overall proportion of highly-rated case studies with submissions, one can note that 5 UK institutions had 75% or more 4* success in REF 2021. This includes the Universities of Exeter and Manchester who submitted a large number of faculty, as well as Middlesex, Westminster and the School of Oriental and Asian Studies (SOAS) whose submission was either smaller or a smaller proportion of their overall eligible faculty member contingent.

So, what do good case studies look like in the UK context? Here are a few examples which were highly rated in REF 2021. For context, 504 case studies were submitted to the Business and Management UoA 17 category. As mentioned, one cannot be assured that the following are 4* case studies, but they will be 4* or 3* as that is what the following institutions received for all case studies they submitted. None were 2* or 1*.

1. The Value of Marketing

Rajesh Chandy (2021) of London Business School conducted a randomised control study of just under 1000 South African small and medium enterprises. The overall group was divided into three sections. One received several hours of training per week in Finance, another in Marketing and the third, the control group, did not receive any initial training. Six and twelve months after the training, company profitability was compared. The control group had flatlined. The finance group increased sales by 41% and the marketing group by 61%. Additionally, the marketing group hired on average 57% new staff. As a result of this study, the World Bank, which had sponsored the study, directed millions of dollars to similar efforts in Nigeria and in Peru and stated that without the study, they would have thought marketing training was a waste of money for SMEs.

2. Driving an Evidence-based Approach to Diversity and Inclusion, Policy and Practice

Richard Urwin (2021) and his team at the University of Westminster Business School investigated the factors that drove an improvement in diversity and inclusion in member organisations of Chartered Institute of Personnel and Development (CIPD) and the Black Solicitor’s Network (BSN) in the UK. The research, drawing on related studies that Urwin and his team had conducted concluded with two recommendations. First, an unexpected finding that unconscious bias training was a waste of money and did not improve matters at all. This led to a change in the training provided by the UK’s civil service. The second was that businesses should ‘publish their data, as well as their long-term, aspirational diversity targets and report against their progress annually’ as ‘making this information public will motivate organisations to tackle this issue with the determination and sense of urgency it deserves’.

3. Improving the Stability of the UK Financial System Through Improvements of England’s Stress Testing Procedures

At Exeter University’s Business School, Richard Harris (2021) and his colleagues looked at how to improve the stability of the financial system. In response to the 2007-08 financial crisis, the Bank of England (BoE) – the Central Bank and financial regulator in the UK – initiated a programme of annual ‘stress testing’ for the UK’s major banks in 2014, in order to better gauge the vulnerability of the UK banking system to a future crash. The key issue that emerged from the early tests was the need for the BoE to better understand the linkages between financial market volatility, market sentiment and the ‘real’ economy. Professor Harris’ expertise in modelling financial market volatility and its relationship with the wider economy has achieved the following impacts:
·       Improved the formulation and calibration of the annual stress test exercise;
·       Improved macro-economic forecasting for the Monetary Policy Committee; and
·       Influenced broader aspects of BoE financial stability policy-making.
Within this broader policy landscape, improved stress tests enable the BoE to refine its response to market volatility and thereby have played a critical role in safeguarding the stability of the UK financial system and the wider economy.

What Then Makes a Good Impact Case Study?

In addition to the small industry created around external evaluation of a business school’s or university’s submission, is a parallel industry around the mechanics and preparation of case studies. The general advice for the conceptualisation of case studies is eminently sensible and it should occupy all academics. Iglehart’s (2021) ‘Seven Lessons’ below, summarises well:
1. Consider impact from the outset of your research
2. Recognise the difference between pathways to impact and actual impact
3. Identify potential research users early and continuously
4. Develop mutually beneficial long-term relations with research users
5. Plan tangible ways in which research can be actioned
6. Track research impact implementation
7. Gather and record impact evidence continuously

More specifically, the preparation of case studies has also spawned what we all knew would happen: a bunch of journal articles assessing the preparation and presentation of case studies. Reichard et al. (2020) bless us all with ‘Writing impact case studies: a comparative study of high-scoring and low-scoring case studies from REF 2014’ while Wróblewska (op. cit.) ‘draws on linguistic pragmatics and Foucauldian discourse analysis’ to consider research impact evaluation, noting that the creation of the impact case study evaluation process has created an emergent language hitherto unknown, which is true.

All attempts at humour to one side, Reichard et al.’s (2020) study is really interesting. Written by a group of seven communications faculty members, the paper draws on two studies looking at the linguistic differences between high- and low-scoring case studies. What becomes immediately clear in highly-rated case studies are that the significance of impact is described in very concrete terms: a government changed policy due to a study; the study was cited in the House of Commons or Lords; an organisation revised their HR policy due to a study. In terms of reach, examples include: that the US revised their immigration policy because of a study; that a study led to the creation of three test sites across the UK and the EU. Particularly high-scoring terms included cited in; used to; improve the; and resulting in.

Low scoring examples on the other hand were often waffle: stakeholders were involved in the work; the research was disseminated in a report; an event about the research was advertised in the newspaper – exactly the points Iglehart (op. cit.) warned against when differentiating between pathways to impact and actual concrete, clearly described impact.

Extending the viewpoint beyond the UK, one finds systems that differ considerably from the UK system insofar as case studies are conducted, but there is not the same lure of significant cash attached to them. In Australia, there are impact case studies listed in the Data Portal of the Australian Research Council with short synopses of the submitted studies. In Hong Kong, according to their RAE 2020 Guidelines, impact can be understood as ‘the demonstrable contribution, beneficial effects, valuable changes or advantages that research qualitatively brings to the economy, society, culture, public policy or services, health, the environment or quality of life whether locally, regionally or internationally, and that are beyond academia’. The HK University Grants Commission also lists case studies on their website.

Lessons and Reactions to the Two Global Focus Annual Research Volumes

There have been two Global Focus issues devoted to research and impact published as open access volumes. The 2022 volume is entitled Perspectives on the Impact, Mission and Purpose of the Business School. The forthcoming 2023 volume is entitled Business School Research: Excellence, Academic Quality and Positive Impact (Cornuel, Thomas and Wood 2022; 2023). So, in reading through the two volumes, what do we tell ourselves about our positive impact? Well, we certainly claim to create a positive impact consistently. In the 2022 volume, mission statements about the positive role of business schools are impossible to avoid. Some are general statements about ‘improving society’, others are specifically related to the UN’s Sustainable Development Goals (SDGs). Others still are framed in the context of the reflection promoted by the Business School Impact System (BSIS). Where there is fundamental agreement is that the teaching of students and of executive education participants imparts management knowledge to those participating and thus creates value and impact in this manner.

Where we are collectively on wobbly ground concerns the themes raised in this article. Given that we have the dual missions of teaching and conducting research, what is the impact of our research beyond the claims in our mission statements? Kalika and Cornuel (2022) tackle the question in considerable detail based on their experiences with 60 schools which have been thoroughly examined by the EFMD’s Business School Impact System (BSIS). They begin by positing that definitions of research impact are ambiguous. Clarity about impact in academic circles (journal quality, citations, etc.) may be argued over, but the system is long-standing and constitutes the rules of the game. The other component, research impact beyond academia is more ambiguous. The authors state that “measuring the impact of management research is a complex objective for which a universal methodology has not yet been officially developed.” Well, yes and no. There is not a universally accepted methodology for impact assessment, but there are certainly the methodologies used in the UK, Australia, Hong Kong and elsewhere which are ‘the law’ in those countries. Additionally, there is the experience gained from the 60 schools summarised in the BSIS article from Kalika and Cornuel.

About a third of the authors in Volume One are UK-based. They consistently mention the REF exercises, but narrowly focus on the publications side of the exercise, but not the impact component. Surely they would be aware of the ever-increasing amounts of funding tied to the impact component. One may like the system or dislike the system, but it is there, front and centre, and doesn’t seem to be going away anytime soon.

The closest to assessing impact comes from the article by Tsui, Bitner and Netessine (2022) reviewing the progress of the Responsible Research in Business and Management (RRBM) network founded by three outstanding US-based academics. The overview of the 108 award-winning articles and 15 book awards across the categories of management, marketing and operations comes tantalisingly close to honing in on impact, but there is one common problem across many of the articles – they describe the path to impact, but not the actual impact of the article. For example, we read that folks are healthier if they make healthy food choices, but we are not privy to whether or not anyone actually has followed the advice. If so, how many? Among those, what were the benefits? Longer healthy lives? Weight loss? Was there a control group? A positive note comes from an article mentioned in Tsui et al. by Bone, Christensen and Williams (2014) which showed that banks had changed their loans criteria following their study on minority access to loans in the US.

Volume 2, given its specific focus on business school research makes progress in its approach to assessing research impact. There are more mentions of the impact assessment systems in place in the UK, Australia and Hong Kong with references provided to the REF in general, and the database of case studies more specifically. We would all acknowledge what Renate Meyer states (in Haley and Jack, 2023) that research impact is hard to pin down as “it unfolds in a non-linear way” and that allocating causality is a fraught endeavour. Assessment is clearly no easy task, but the attempts to do so are continuously increasing.

An interesting element of Volume 2 are the case studies of the research informed initiatives to influence two specific areas. The first is the work described by Ogbonna (2023) around creating a forum in Wales to counteract systemic racism. The author notes that the framework has been established and they are now at the implementation phase with the Welsh government. Significant progress has been made. Also worth noting are the efforts described by Barin-Cruz et al. (2023) of HEC Montreal describing the SEED project which is aimed at Scaling Entrepreneurship for Economic Development. If one were to classify these, one would probably suggest that they are hybrids. They are research-informed if not totally research-led, are active beyond academia, and are clearly ‘doing the right thing’.


It has not been the goal here to recap the content of both research volumes, but simply to offer some reflections from reading through them as well as reviewing other sources and examples of impactful research. The article has also attempted to describe the nature of the more forensic impact criteria now being applied by funding bodies and government agencies and emphasise the importance of being very clear about the societal and economic value and impact of business school research. Funders have argued that no longer will broad, general descriptions of the pathways to achieve impact be sufficient for research reports submitted to either funding bodies or other stakeholder sponsors. It is now considered imperative that the specific details of implementation success should be provided as well as thorough explanations of how significant policy and other practical solutions emerged from the research findings.

In our business school experience, there have been three clear phases in the evolution and growth of impactful research over the last 15 years. First, the period 2005-2013, around the global financial crisis, in which the impact and practice arose mainly from translating business school ideas, sometimes through customised executive education programmes, in areas such as organisational development, human capital strategy and strategic change. Second, the period from 2013-2019, in which students, faculty and organisations recognised the need for establishing sound principles for responsible management education and research. In this period, strong linkages between business schools and their stakeholders led to jointly funded projects in areas such as ESG and sustainability management, and strong research collaborations and partnerships focusing on ecosystem development and issues of societal impact. The third period from 2020 to the present, following the Covid pandemic and resultant societal disruption, saw the rapid adoption of technology-enabled research tools and digital business applications. This led to even wider tri-sector and interdisciplinary collaboration addressing so-called societal grand challenges involving regional and public policy goals in areas such as inclusive growth, implementing the UN’s SDGs and climate change. Thus, the future research landscape has been framed increasingly in terms of underlying principles of stakeholder value maximisation. This has generated a wide range of initiatives and evidence about the nature of significant research impact including the following:

  • The production and collection, over 15 years, of a considerable number of business school impact studies from EFMD’s Excellence in Practice initiative.
  • Around 60 case studies of business school impact developed over the last 10 years through the EFMD’s BSIS system.
  • The 123 award-winning articles and books from the RRBM community since 2015.
  • The case studies developed in the UK government’s 2021 REF framework in which research case studies had to include an in-depth analysis of how research impact was achieved and implemented.
  • The case studies of practical societal research impact provided in the upcoming second EFMD volume.

Taken in total, what the future research impact landscape clearly requires above all else is specific and clear details about how impact was achieved in each case setting. As researchers we may not like it, we may say that it impedes blue-sky research, we may think of many reasons we are ‘whatever it is, we are against it’ but we need to come to terms with the use of much broader criteria than bibliometric citations if we are to play the research game successfully in the future.

The Impact of Business School Research


Australian Research Council case studies
Barin-Cruz, L., N. Ponce Morales, K. Picone and L. Beaugrand-Champagne (2023) Empowering Vulnerable Populations Through Transformative Approaches and Research, Global Focus Annual Research Volume 2 (Cornuel, E., H. Thomas and M Wood eds) pp.90-95.
Bone, S., G. Christensen and J. Williams. (2014). Rejected, Shackled, and Alone: The Impact of Systemic Restricted Choice on Minority Consumers’ Construction of Self, Journal of Consumer Research 41, pp.451-474. 10.1086/676689.
Chandy, R. (2021) REF case study f44e1a3a-9ea6-4ded-8671-c1314c0c504f?page=1
Cornuel, E., H. Thomas and M. Wood, eds (2022) Global Focus Annual Research Volume 1 -Perspectives on the Impact, Mission and Purpose of the Business School.
Cornuel, E., H. Thomas and M. Wood, eds (2023) Global Focus Annual Research Volume 2 – Business School Research: Excellence, Academic Quality and Positive Impact.
Donovan, C. (2008) The Australian Research Quality Framework: A live experiment in capturing the social, economic, environmental and cultural returns of publicly funded research. New Directions for Evaluation 118 pp.47-60.
Dowling, A. (2015) Dowling Review of Business-University Research Collaborations, London: Royal Academy of Engineering.
Espinosa, J. (2015) Universities wasting public money on ‘pointless’ research, says think tank 23 July 2015 Daily Telegraph.
Geuna, A. and B.R. Martin (2003) University research evaluation and funding: an international comparison. Minerva, 41, pp.277-304.
Haley, U.V.C. and A. Jack (2023) Measuring Societal Impact in Business and Management Studies: From Challenges to Change Sage Business White Paper London: Sage Publications.
Harris, R. et al. (2021) REF case study impact/9fe66d45-d0b5-4d78-8dd9-e16bbec46ab2?page=1
Hong Kong University Grants Committee case studies
Iglehart P. (2022) REF 2021: Seven Lessons learned about impact cases
Kalika, M. and E. Cornuel (2022) The search for meaning: BSIS and its role in promoting business schools’ societal impact Global Focus Annual Research Volume 1 (Cornuel, E., H. Thomas and M. Wood eds) pp.20-24.
Kerridge, S. (2023) REF 2028: Impact becomes more valuable accessed 16 October, 2023
Martin, B.R. (2011) The Research Excellence Framework and the ‘impact agenda’: are we creating a Frankenstein Monster? Research Evaluation 20(3) pp.247-254
McKay, S. (2017) Effects of REF 2021 decisions on impact weighting
Ogbonna, E. (2023) Making Wales an Anti-racist Nation: A ‘Public Value Mission’ in Action, Global Focus Annual Research Volume 2 (Cornuel, E., H. Thomas and M Wood eds) pp.78-83.
O’Grady, C. (2023) ‘Quietly revolutionary’ plan would shake up the way U.K. universities are evaluated 16 Jun 2023 Science.
Reed, M. and S. Kerridge (2017) How much was an impact case study worth in the UK Research Excellence Framework? https://www.fasttrackimpact. com/post/2017/02/01/how-much-was-an-impact-case-study-worth-in-the-uk-research-excellence-framework
REF 2021 (2021)
Reichard, B., M.S. Reed, J. Chubb, G. Hall, L. Jowett, A. Peart & A. Whittle (2020) Writing impact case studies: a comparative study of high-scoring and low-scoring case studies from REF 2014, Palgrave Communications, 6(31) pp.1-17.
Tsui, A.S., M-J. Bitner, S. Netessine (2022) What Topics Should Business Schools Focus on? Global Focus Annual Research Volume 1 (Cornuel, E., H. Thomas and M Wood eds) pp 25-32.
Wróblewska, M.N. (2021) Research impact evaluation and academic discourse. Humanities and Social Sciences Communications 8(58) pp.1-12.

Stay connected
Search Global Focus
Subscribe to the
Global Focus Newsletter