Is it time for a change in faculty recruitment and promotion practices? Edeltraud Hanappi-Egger describes how, and why, business schools need to take into account multiple aspects of performance.
Masters degree – PhD – postdoc – assistant professor – full professor. That’s what most people expect an academic career to look like – a linear, full-time post in academia from a degree programme to a tenured professorship at a renowned university of business and economics.
A career like this often implies that academics are expected to churn out publication after publication. But when it comes to recruiting researchers for senior faculty positions or to promoting academics, it is not enough to focus mainly on the number of publications in top-tier journals; various other aspects play a crucial role in the performance portfolio of academics.
This is especially true because the demands placed on universities have grown over the last few years, not only with regard to research, teaching and other related demands (now often known as “third mission activities”) but also in connection with universities’ role as employers.
What do we mean by performance?
WU (Vienna University of Economics and Business) is breaking new ground in this area and has been discussing a more comprehensive in-house performance assessment system for full professors. In addition, the diversity of researchers’ personal contexts has been taken into account in various guidelines for evaluating the performance of faculty members.
The current discussion on how to measure multiple aspects of performance at WU is inspired by the approach of measuring achievement or performance relative to opportunity, which is primarily used by universities in Australia and New Zealand.
At the core of this approach lies the concept that the traditional model of a full-time, linear, uninterrupted academic career can no longer be taken as the norm and the main yardstick for measuring performance. WU hopes to initiate a process that allows us to rethink the way performance is assessed in academia, both at WU itself and also at other universities.
The goal is to develop a new, more comprehensive understanding of performance and performance assessment methods in the academic field, focusing on the quality, impact and consistency of researchers’ achievements while also taking into account specific personal contexts. With this new approach, WU aims to contribute to a discussion on performance assessment procedures at universities.
Is it still enough to measure performance based on research output alone?
Which criteria should be applied to judge performance? A look at international rankings and rating systems shows that the performance of researchers is usually assessed based on research output, which, in turn, is mainly gauged quantitatively using criteria such as publication counts and impact factors.
But though this approach is widely regarded as neutral and objective, it has some pitfalls.
In particular, the conventional performance assessment system gives an advantage to those researchers who work in well-established fields with a long tradition of publication and who pursue publication strategies that emphasise quantity.
However, these systems do not give adequate recognition to academic achievements in innovative fields that do not have a long tradition of publication opportunities.
Performance measurements focusing mostly on research output also fail to reward important work in teaching and third mission activities.
If universities place paramount importance on publication output, early-career scholars have to work very hard to meet these specific performance standards. This puts pressure on researchers to follow a strict lifestyle that allows them to dedicate themselves fully to research and to work with a tight focus on goals, independence and undivided attention. As a consequence, a standard career path with uninterrupted, linear full-time employment remains a common requirement for being able to perform adequately.
Since career paths have changed significantly over recent years, there are more and more researchers who for a variety of reasons might complete their PhD at age 35 instead of 25 maybe because they had a job during their studies, started their degree programme late or – for whatever reason – took some time off.
Women, particularly, can find it very hard to meet the demands of academia because their academic portfolios and career paths are often “non-standard”.
For these reasons, it is probably a good idea to assess a researcher’s publication output relative to his or her so-called “academic age” – the time passed since completion of the PhD, regardless of actual age. This means that performance measurements will give a better picture of an individual’s capabilities if their life stories and the opportunities they have had are taken into account.
Multi-dimensional performance assessment in academia
In addition to the biographical contexts of academics, it also seems to be necessary to broaden performance assessment approaches by including more factors than just research output understood in terms of publication count.
uLiKe, WU’s multi-dimensional in-house performance assessment system, is intended to include diverse criteria that go beyond publication output and give more weight to teaching, knowledge transfer activities and/or work in university development.
In concrete terms, this means that performance in the area of research is evaluated based on criteria such as the number of original articles, original contributions to journals and books, or third-party funding that is attracted.
At the same time, however, the new approach also looks at achievements in the field of teaching, factoring-in aspects such as courses taught, thesis supervision, assistance provided to early-stage researchers and student ratings.
Third mission activities are also becoming more and more important for universities. For this reason, additional factors should be considered, including work in university management or decision-making bodies, committee membership, work as a reviewer or assessor, active co-operation with partner universities as well as work for the academic community, for example, functions performed for academic societies or journals (work as an editorial board member or reviewer) or contributing to the organisation of conferences.
Third mission activities relevant to performance assessment also include services to society and knowledge transfer, including offering academic expertise to the media, writing popular science articles and giving presentations at non-academic events.
It is important to note that all professors should be involved in third mission work in one way or another.
All these activities could be assessed relative to individual personal contexts. Relevant factors include caring for children or family members, leaves of absence, part-time employment and gaps in academic career paths due to illness or involvement in civil society initiatives.
In some areas, WU has made previous efforts to acknowledge the importance of a comprehensive range of criteria in assessing employee performance. In 2013, for example, a WU working group developed a definition of the job profile of a full professor that bases performance assessment on an extended range of qualification criteria in the fields of research, teaching and third mission activities. To take into account personal aspects in assessing applicants for a position as a full professor, WU drew up an information document for search committees and guidelines for reviewers in senior faculty recruitment, in accordance with the provisions of WU’s Plan for the Advancement of Women.
In addition, a special assessment sheet is available to search committee members and reviewers to help them understand and apply the position announcement criteria and to improve transparency. A note has also been added to the catalogue of questions for employee performance reviews as a reminder that an employee’s publication output should be assessed in the context of his or her biography, taking into consideration, for example, part time arrangements.
Position announcements for full professorships also contain an explicit note that applicants are expected to have an excellent research output relative to their “academic age”. This means that performance is seen in relation to a person’s individual context.
These new announcements send out important signals to society. They encourage young researchers to apply because they know that their research output will be assessed relative to their academic age and other work and responsibilities. This approach is intended to broaden the palette of criteria applied when reviewing candidates’ qualifications. Of course, all these measures by no means imply that we plan to expect less from our applicants.
Further steps towards equal opportunities
There have many been efforts in academia to make performance assessment more transparent and to eliminate certain discriminatory effects of common performance evaluation methods. At some Austrian universities, special workshops are held to raise awareness of these issues. Other approaches seek to improve transparency and equal opportunities by limiting the number of publications considered, for example by basing the evaluation of candidates on a limited number of each applicant’s best publications.
As a next step, these approaches should be combined and refined into a multi-dimensional system that can then be translated into various guiding principles. The goal is to make further steps towards equal opportunities for researchers with non-standard biographies who show excellent performance in activities that go beyond just research output.
‘How many papers is a baby worth’
It goes without saying that the transition towards a more comprehensive performance assessment framework that includes personal contexts also creates certain challenges.
It requires intensive dialogue and discussion on the selection of criteria and personal factors to be included in performance assessment and how these factors should be weighted. This is by no means a trivial process because it is hard to measure how many papers should be seen as equivalent to child care or caring obligations, for example. (See Klocker N, Drozdzweski D. (2012): Career progress relative to opportunity: How many papers is a baby “worth”? Environment and Planning A 44, p. 1271-1277.)
Another issue is that specific biographical factors can only be taken into consideration if they are disclosed. But how can people be encouraged to share private information for the purpose of performance assessment? Disclosing such information obviously blurs the borders between private and work life, which may be seen as problematic by some and is an issue that needs to be addressed.
It is also essential to discuss how we can make sure that the new approach to more comprehensive performance assessment does not lead to a drop in quality standards and that it is not used to justify sub-par performance.
Indeed, the opposite may even be the case with the addition of new performance assessment criteria making it harder rather than easier for people with non-standard biographies to meet all the different demands. After all, would the requirement to engage in third mission knowledge transfer and supervise theses on top of conventional research work pose an additional burden for researchers with child care obligations?
A stimulus for further development
The question of how a more comprehensive performance assessment system can be implemented and put into practice must undoubtedly be answered by each university individually, and the process requires a great deal of discipline and commitment on the part of the entire faculty.
We believe, however, that initiating this process of change is well worth the effort, as is evidenced by the Diversitas award WU received from the Austrian Federal Ministry of Science, Research and Economy in December 2016.
WU has since been approached by several universities that are interested in the project. Ample positive feedback and strong interest from the media are further signs that it is high time to start a large-scale process of change that involves multiple universities.
Originally published June 2017.
See more articles from Vol.11 Issue 02 – ’17 and the 15th Anniversary Issue.
- Back to campus – vaccinatedor unvaccinated? - January 17, 2022
- Lo que significa la digitalización para las universidades - September 22, 2020
- What digitalisation means for universities - February 10, 2020
