For several centuries, measurement has been at the heart of the scientific approach and in the world of business schools. W. Edwards Deming’s quote ‘You can’t manage what you don’t measure’ has become a mantra. Indeed, measurement plays a key role in many managerial processes – without measurement, no PDCA (plan-do-check-act) process can be implemented. Nevertheless, questioning the choice of what we measure or how we measure it is at the centre of many discussions and debates both in society and within organisations.
‘You can’t manage what you don’t measure’
The most publicised controversy in the last decade is probably around the criticisms of GDP – and especially the measurement of growth or “success” based on GDP. Measuring the totality of economic exchanges is indeed an important factor in understanding the economic dynamics of a country. Nevertheless, it is easy to understand the limitations of a measurement tool when most people do not remember how it is constructed (and therefore what it actually measures). A road accident, a burglary, or a natural disaster often contributes to an increase in GDP, while in contrast, volunteering in projects that contribute to the well-being of a part of the population tends to diminish it (since this volunteer work can potentially replace a commercial exchange).
Measuring requires, of course, an upstream decision about what to measure and why to choose these indicators. Measurement is, therefore, a technical but also a political act that allows us to understand what is important for a group of people at a given moment.
It is very likely that most of the decision-makers who are now committed to reducing CO2 emissions had no idea of the importance of this metric even ten years ago and did not imagine that they would one day measure it.
Since the advent of sustainable development and the deployment of CSR in organisations, the question of what to measure is absolutely key. Less easy and universal than measuring a turnover or a margin, measuring actions in favour of sustainable development (or their impacts) requires agreement on the scope of responsibility of the organisations concerned. A very effective method consists of making an exhaustive list of the actual or potential impacts of an activity on each of the stakeholders of its ecosystem. Once analysed and weighted, organisations can define the indicators that they will implement and monitor.
The specificity of higher education institutions lies in the fact that they have two levels of responsibility. On a day-to-day basis, Business Schools, like any organisations, have an immediate impact on their environment in social, economic, and environmental terms (schools consume resources, produce waste, and are not always exemplary in terms of accessibility…). As providers of education, they also – indeed, probably above all – have an impact on the people they train as well as on the partners they support.
Because they help shape the behaviour and decisions of future managers and business leaders, focusing solely on the carbon footprint and social performance of their campuses is not enough.
It is, therefore, crucial to understand the full scope of their responsibilities and the impact of their activities, both positive (e.g., employment) and negative (e.g., potential discrimination, pollution).
Once the scope of responsibility has been analysed, the institutions will be able to set up a certain number of actions and indicators allowing them to measure and monitor their individual and combined impacts.
For several years, the university community and their stakeholders have been reflecting on ways to measure the integration of sustainability at different levels of their organisations.
Many initiatives have built assessment tools to evaluate the university as a whole (as is the case with accreditation, national labelling, ranking & rating…), while others focused on a specific programme, or even on a specific course. Finally, some will assess what every student should know at the end of his or her programme.
At an institutional level, it is possible to measure elements concerning governance and strategy, but also to implement indicators related to social or environmental policy.
In many countries, associations or networks of higher education institutions have been creating tools to measure at the institutional level for the past fifteen years. While some have unfortunately been abandoned, the American Stars initiative (led by AASHE), the Green Plan in France (led by the CGE and CPU and the supervisory ministries) or the Sustainability Leadership Scorecard in the UK and Ireland are still working well. More recently, some have moved from self-assessment tools to labels & endorsements.
Accreditors, key stakeholders in measuring the performance of business schools, have tried to integrate elements related to social responsibility in their accreditation criteria. As such, since 2013, EQUIS has included not only a dedicated ‘ethics and responsibility’ chapter, but also has a specific paragraph on the topic in each of the other chapters of its evaluation grid.
More recently, dedicated measurement tools have emerged. In 2017, a group of researchers, NGOs, students and representatives of the business world started to think about the creation of a new tool to evaluate the contribution to the construction of a sustainable world. The PIR (Positive Impact Rating) currently measures students’ perception of their school’s governance, culture, programmes, teaching methods, student support, institution as a role model and public engagement. Starting in 2020, the Times Higher Education Impact Rankings attempted to measure universities through the lens of the 17 Sustainable Development Goals (SDGs). The third edition of the rankings was published in April 2021, including assessment of more than 1,200 institutions from 98 countries and regions. This is an interesting sign of the growing interest in these issues within schools and amongst young people. It is interesting to note that this movement is also appearing at the regional level, for example in France, where the first edition of the Echo start was the most successful to date.
Beyond these dedicated rankings, the historical ranking providers are thinking about how to best integrate indicators or new methodologies to make conventional rankings evolve. In particular, the Financial Times has integrated CSR criteria such as inclusion or diversity in 2019.
But if we can be pleased with this recent evolution, there is still a long way to go. We might even deplore the mediocre quality of some initiatives, which are often communication operations that can discredit the approach and make greenwashing quite possible. It is in this context that HESI (the UN Higher Education Sustainability Initiative launched at the UN Rio +20 conference in 2012) has produced a guide for the ranking organisations. The most influential ranking organisations – such as the FT, QS or THE – have committed to integrate these principles and we are looking forward to the next UN high-level political forum in July to review the progress of the work.
As with any change, building, equipping, and empowering a coalition of changemakers is crucial. At universities, beyond the work done with some committed teachers (or on courses dedicated to sustainability), it is now time to support the entire faculty community to integrate the sustainability paradigm in conventional courses. Here again, measurement is essential to establish an action plan. In this regard, SDSN, with the collaboration of GRLI and other partners, have initiated the Self-assessment matrix to embed sustainability in teaching. The matrix has been tested through a survey taken by more than one 160 research professors worldwide.
While numerous tools exist to measure input (courses aligned to sustainability, percentage of students enrolled in these courses, etc.), measuring the output of these actions is an important piece of the assessment puzzle. Sulitest is an organisation that equips educators with tools to map and improve sustainability literacy at the individual level. Their best-known tool, Awareness Test, has been used by almost 250,000 people across all continents, offering an opportunity to learn about sustainable development while better understanding where the current level of awareness is at. Sulitest will soon launch a robust assessment, certifying sustainability knowledge.
Moreover, considering the outcome is equally important. Are your students interested in integrating sustainability into their careers? Do they feel equipped to do so? Who are your alumni working for? What initiatives or companies did you entrepreneurial alumni start? What positive changes are they creating? The long-term impact of a university is carried by its graduates, so considering these “future-focused” questions and back-casting what curricular changes need to happen in today’s strategy is vital.
Whether to define a strategy, to manage, to control, to improve, or to provide evidence to stakeholders, measurement is absolutely essential. Building and choosing social and environmental performance indicators is always an exciting task because it forces us to make choices.
Accreditation and ranking bodies have enormous power, and therefore enormous responsibility, to guide the academic sector.
Many of them have become aware of the need to change their criteria for measuring the performance of higher education institutions. Many initiatives have emerged to either improve current standards or create new ones.
Given the expectations of students, the growing statement of higher education presidents, and the many emerging initiatives, it’s a safe bet that the next few years will be full of lessons in the world of measurement. Let’s hope that the emergence of those assessments will push the entire sector to better prepare future generations to become responsible leaders.
See more articles from Vol.16 Issue 02 – GRLI.
