Coping with Complexity: the Role of Composite Indicators

Stefano Tarantola and Andrea Saltelli,

European Commission - Joint Research Centre Institute for the Protection and Security of the Citizen Unit of Econometrics and Applied Statistics 21020 Ispra (VA), Italy

Presenting author:
Stefano Tarantola, born in 1967, is a Scientific Officer of the Joint Research Centre of the European Commission with an MSc in engineering and a PhD in Science and Technologies for engineering at the Polytechnic of Milan.
He carries out and coordinates scientific R&D tasks in the field of statistical indicators development and assessment. He conducts statistical work on indicators and composite indicators for the structural indicators initiative and the sustainable development strategy. He has experience in systems analysis, modelling and in methods to perform robustness analysis of decision processes to policy assumptions. He combines sensitivity analysis and participatory methods for the construction of composite indicators and develops methodologies for sensitivity analysis. He is the author of papers in the peer reviewed literature, co-author of three books on sensitivity analysis, and an organizer of summer schools and other events on sensitivity analysis.

Co-author:
Andrea Saltelli, 54, is an applied statistician, with a steady flow of publications in disciplinary journals over the last 30 years, on topics from physical chemistry to environmental sciences and applied statistics (H-factor=18). His main disciplinary focus is on sensitivity analysis of model output, a discipline where statistical tools are used to interpret the output from mathematical or computational models. His second focus is on the use of composite indicators. A third focus is on methodologies for the use of scientific information at the science-policy interface. Presently, he leads the Econometrics and Applied Statistics Unit of the European Commission at the Joint Research Centre in Ispra (Italy). The Unit, with a staff of 47, develops econometric and statistic applications, mostly in support to the Services of the European Commission, the field of lifelong learning, internal market, knowledge economy and others, see www.jrc.cec.eu.int/uasa.

Selected papers:

• Saltelli, A., Andres, T., Campolongo, F., Cariboni J., Gatelli, D., Ratto, M., Saisana, M., Tarantola, S., Sensitivity analysis of scientific models, John Wiley & Sons publishers, to appear in winter 2007.
• Saltelli, A., Composite indicators between analysis and advocacy, 2007, Social Indicators Research, 81, 65-77.
• Saltelli, A., Ratto, M., Tarantola, S., Campolongo, F., Sensitivity analysis practices: Strategies for model-based inference, 2006, Reliability Engineering & System Safety, 91(10-11), 1109-1125.
• Saltelli, A., M. Ratto, S. Tarantola and F. Campolongo, 2005, Sensitivity Analysis for Chemical Models, Chemical Reviews, 105(7) pp 2811 - 2828.
• Saisana, M., Saltelli, A., Tarantola, S., 2005, Uncertainty and Sensitivity analysis techniques as tools for the quality assessment of composite indicators, Journal Royal Statistical Society A, 168 (2), 1-17.
• Saltelli, A. Tarantola, S., Campolongo, F. and Ratto, M., 2004, Sensitivity Analysis in Practice. A Guide to Assessing Scientific Models, John Wiley & Sons publishers.
• Saltelli, A. Tarantola, S., 2002, On the relative importance of input factors in mathematical models: safety assessment for nuclear waste disposal, Journal of American Statistical Association, 97 (459), 702-709.
• Saltelli, A., Tarantola, S. Campolongo, F., 2000, Sensitivity analysis as an ingredient of modelling, Statistical Science, 15(4), 377-395.

Composite indicators are increasingly used to summarize complex phenomena where a plethora of different dimensions have to be considered together. The main virtue of these measures is their usefulness in policy analysis since they can illustrate multifaceted and sometimes elusive issues in wide ranging fields, e.g., environment, economy, society or technological development. Composites often seem easier to interpret than finding a common trend in many separate indicators and have proven useful in benchmarking country performance. However, composite indicators builders have to face a relevant degree of scepticism. They are accused to merge "apples and pears" when deriving a unique ranking from a set of indicators expressed in different measurement units. Furthermore the lack of transparency regarding e.g. the selection of the reference framework, the weighting scheme, the aggregation method, could render the index - and the policy inference based on it - arbitrary and vulnerable to rejection by stakeholders. On the other hand, a sound process of construction cannot remedy to an inadequate framework or to poor quality data. Therefore the construction of a composite indicator requires a balance between different aspects, all equally important in defining the quality and finally the usefulness of the composite. This paper reviews the debate around composite indicators by discussing the main steps to be followed in their construction. We stress the need for stringent standards of quality for these statistical measures, due to their policy relevance and use. This is particularly true in the field of social sciences where composite indicators are relatively new.

Download der Langfassung als PDF-Datei