View article

Franzoni and Stephan (2023) recommend a probabilistic ‘subjective expected utility’ technique for addressing challenges of uncertainty in research evaluation. Whilst acknowledging strengths in F&S's analysis, this Response highlights a series of important practical, theoretical and methodological deficiencies. The stakes are raised, in that these are widely shared in a growing body of practice across research policy and beyond. This practice seeks to reduce and aggregate real-world complex, ambiguous, qualitative, multidimensional and contested challenges through ostensibly precise calculation. Taking associated problems in turn, this Response shows how F&S: make scientifically dubious claims; understate the depths of uncertainty; overstate the sufficiency of quantification; neglect foundational limits to calculation; and ignore crucial interpretive dimensions of policy making. Highlighting roles for greater …