“[…] it is hard to imagine that the debate on the use of composite indicators will ever be settled […] official statisticians may tend to resent composite indicators, whereby a lot of work in data collection and editing is “wasted” or “hidden” behind a single number of dubious significance. On the other hand, the temptation of stakeholders and practitioners to summarise complex and sometime elusive processes (e.g. sustainability, single market policy, etc.) into a single figure to benchmark country performance for policy consumption seems likewise irresistible.”
Andrea Saltelli, JRC
- 22-26 July, 2013 11th JRC Seminar on Composite Indicators and Rankings
A new global benchmarking study on how talent is grown, attracted and retained in 100+ countries
29 Nov 2013
A new Global Talent Competitiveness Index (GTCI), which the JRC helped refine, was recently launched by the world-class business school INSEAD, together with the Human Capital Leadership Institute of Singapore and Adecco. The GTCI is a global benchmarking study that focuses on how talent is grown, attracted and retained in around 100 countries (including all EU members).
The GTCI is an input-output model of nearly 50 variables, in the sense that it combines an assessment of what countries do to produce and acquire talents (input) and the kind of skills that are available to them (output). The two output pillars of GTCI differentiate between mid-level skills (Labour and Vocational Skills) and high-level skills (Global Knowledge Skills). With its focus on talent, the GTCI does not measure a third type of human capital -unskilled labour- although discussions will sometimes embrace lower-level skills. The four input pillars of the GTCI are based on the Enable-Attract-Grow-Retain framework used by corporations to steer talent management. Multinational corporations frame talent management in these terms, defining talent management as an organisation’s efforts to attract, select, develop and retain key talented employees.
The JRC performed a rigorous audit of the GTCI model (Chapter 5 of the report), including its structure, variables and analytical foundations, using state-of-the-art statistical methodologies developed in-house.
Europe clearly dominates this year’s GTCI rankings, with eight countries in the top ten, and 14 in the top 20. An examination of the scores registered by European countries shows that the Grow pillar (especially education) is a common success factor for those countries: there are eight European countries in the top ten of the Grow pillar and seven European countries in the top ten of the Formal Education sub-pillar. In addition to European countries, one finds at the top of the rankings a significant number of traditional “immigration lands”, i.e., typically large countries such as Australia (15th), Canada (11th), and even the United States (9th). Part of this situation can be explained by the ways in which former colonial empires were built, and then dismantled: the respective legacies left by those evolutions in terms of education, language and even administration have added another layer of history on the provision and distribution of skills worldwide.
A handful of dynamic economies at various levels of income are scoring high on the index. Singapore (2nd in the world) is the perfect example in this case, but the experiences of other small economies such as Estonia (23rd), Montenegro (26th) and Malaysia (37th), show how middle-income countries are striving to mobilise the talent base that their specific economic and geographical situations require.
Updated Regional Competitiveness Index provides guidance to regions
20 Nov 2013
The 2013 edition of the Regional Competitiveness Index (RCI) has been launched by the JRC, in collaboration with the European Commission's Directorate-General for Regional Policy. The RCI was first published in 2010 and shows that competitiveness has a strong regional dimension, which national level indicators cannot capture.
The RCI 2013 shows the strengths and weaknesses of the EU regions. EU countries with a high variation in regional competitiveness should consider to what extent the gaps are harmful for their national competitiveness and if these gaps can be reduced. For example, the difference between the capital region and the second most competitive region in Romania, Slovakia and France is very wide, while regional competitiveness in Germany does not show any big jumps. The RCI can provide a guide to what each region should focus on, taking into account its specific situation and its overall level of development.
A lack of regional spill-overs was noted again in particular around the capitals of some of the EU countries which are lagging behind. The crisis in these countries may have limited the potential growth in regional spill-overs, but in the medium-term such spill-overs should be strengthened since the overall competitiveness of a country depends on the performance of all its regions, and not just its capital region.
The RCI update includes more data and method refinements. Numerous suggestions and remarks have been integrated in the new edition, especially those collected from region representatives, journalists, researchers and Chambers of Commerce.
As the result of a close interaction with the World Economic Forum (WEF) group responsible for the Global Competitiveness Index, a summary of the RCI 2013 outcome has been published in the last annual edition of the Global Competitiveness Report, presented by the WEF in China last September.
World Economic Forum: The Global Competitiveness Report 2013-2014
JRC report: EU Regional Competitiveness Index RCI 2013
JRC audits the first ever Environment and Gender Index
19 Nov 2013
The Environment and Gender Index (EGI) was launched by the International Union for the Conservation of Nature (IUCN) today during the United Nations Framework Convention on Climate Change (UNFCC COP19) in Warsaw. The EGI attempts to monitor progress toward gender equality in the context of global environmental governance and provides information on how 72 nations (including 11 EU members) are translating gender and environment mandates into national policy and planning.
The JRC has worked iteratively with the EGI developing team in order to arrive at an index that meets international quality standards. The JRC simulations and validation tests ensuring the transparency and reliability showed that the EGI summarises different aspects of gender equality within the context of global environmental governance in a more efficient and parsimonious manner than what is possible with a collection of relevant indicators taken separately. The EGI country ranks are fairly robust to methodological assumptions related to the estimation of missing data and to small variations in the weights assigned to the six categories: livelihood, gender rights and participation, governance, gender education and assets, ecosystem and country reported activities. Constructive criticism is still welcome for the EGI, which could be refined within two of its six categories - Ecosystem and Country reported activities. This refinement would be instrumental in bringing the EGI to its full potential.
The EGI has been nominated for the Katerva Awards, which are perceived as the “Nobel Prize” for sustainability initiatives, and aim to identify the most promising innovations on the planet in 10 categories. The nomination is based on the rigorous adjudication process with respect to validation, scalability, policy implications and overall impact of each innovation carried out by over 400 experts in specific subject areas.
Six EU members are in the top ten of the EGI
The three most committed nations to tackling women’s advancement in their environmental efforts are Iceland, the Netherlands and Norway (72 nations included). Six EU members are in the top ten of the EGI: the Netherlands (2), Sweden (4), France (5), Finland (6), Spain (8), and Denmark (9). Five more EU members are well positioned in the EGI occupying positions from the 11th to the 22nd place - Poland (11), Portugal (12), Greece (14), Italy (15) and Romania (22). Remaining EU members were not included due to insufficient data coverage.
Towards a new single innovation indicator
On 24 and 25 October 2013, a new single innovation indicator, which the JRC helped develop, has been presented at this week's European Council Summit. The indicator focuses on innovation output and measures the extent to which ideas from innovative sectors are able to reach the market, providing better jobs and making Europe more competitive.
The JRC investigated the conceptual coherence and the statistical robustness of the indicator by using self-developed state-of-the-art statistical methodologies to perform all the necessary simulations and validation tests aimed at improving the quality of the indicator.
The proposed indicator is based on the following four criteria: technological innovation as measured by patents, employment in knowledge-intensive activities as a percentage of total employment, competitiveness of knowledge-intensive goods and services, and employment in fast-growing firms of innovative sectors.
The indicator was developed at the request of EU leaders to benchmark national innovation policies. By honing in on innovation output, it will complement the indicator on R&D intensity foreseen by the EU's 'Europe 2020 strategy for smart, sustainable and inclusive growth'. It will support policy-makers to take new measures or reinforce existing ones to remove bottlenecks which prevent innovators from translating their ideas into successful products and services.
According to the new indicator, Sweden, Germany, Ireland and Luxembourg are the EU Member States getting the most out of innovation. A comparison with some non-EU countries shows that the EU as a whole does well. Switzerland and Japan have a clear performance lead, but the EU is more or less even with the United States on innovation output.
The new single innovation indicator complements the Commission's Innovation Union Scoreboard (IUS) and Summary Innovation Index (SII) which assess the innovation performances of Member States and the EU more widely, against a broad set of 24 innovation indicators including inputs, throughputs and outputs.
Communication 'Measuring innovation output in Europe: towards a new indicator'
The Innovation Union Scoreboard and Summary Innovation Index
Press release: Commission launches new innovation indicator
Participation to the JRC Round Table on "Scientific support for growth and jobs: cultural and creative industries"
Brussels, October 24, 2013
At the Round Table on 'Scientific support for growth and jobs: cultural and creative industries' organized by JRC in Brussels, on October 24th, the HoU of the Unit of Econometrics and Applied Statistics, Andrea Saltelli, talked about the value of the cultural and creative industries to the European economy. This high level JRC event focused on two key issues: the spill-over effects and the contribution of cultural and creative industries to local and regional development.
JRC method to assess statistical coherence of multi-dimensional indices
11 Oct 2013
Are ratings and rankings non-debatable? Does the weight given to one variable actually reflect its real importance? In a recently published article in the Journal of the Royal Statistical Society A the JRC describes its statistical method to assess the quality of multi-dimensional indices (composite indicators). The analysis shows that for several of these indices equal importance is attributed to the underlying variables, while in many cases, only a few variables drive the results and others are merely cosmetic. The JRC method can help developers revise their composite indicators in order to obtain statistical coherence.
Composite indicators are popular tools in public discourse and for the general public and the media they are perhaps the best known face of statistics when it comes to assessing countries performance on human development, perceived corruption, innovation, competitiveness or other complex phenomena. Multi-dimensional indices combine a set of variables using a mathematical formula, which is often a weighted arithmetic average. The weights are meant to reflect the variables importance in the index. The JRC delved into the potential flaws of rankings and ratings built as weighted arithmetic averages.
The JRC method measures the importance (or main effect) of a given variable within existing composite indicators through a statistical measure known as the Pearson correlation ratio. The article demonstrates to what extent knowledge of the main effects can be used to enhance the statistical coherence of the indices and render them more transparent and defensible. The analysis was applied to six well-known composite indicators, including the United Nations. Human Development Index and two popular league tables of university performance, the Academic Ranking of World Class Universities (ARWU) by the Shanghai University and the Times Higher Education Supplement (TIMES) university ranking.
The JRC methodology has already been applied internationally by the Yale and Columbia University in the Environmental Performance Index 2012 and by the Cornell University, the INSEAD leading business school and the World Intellectual Property Organization in the development of the Global Innovation Index 2013, and has also been used in the development of the new innovation output indicator of the European Commission prepared at the request of the European Council.
To develop its statistical method, the JRC distilled its experience gained over the last 10 years after helping over 60 international organisations to fine-tune their multidimensional measures, such as Transparency International, Harvard, M.I.T., INSEAD, World Intellectual Property Organization, United Nations, World Economic Forum, European Central Bank and others.
Participation to the Meeting of Experts for the European Arts Index
On September 27th, the Unit of Econometrics and Applied Statistics (EAS) of the JRC participated to the Meeting of Experts for the European Arts Index, organized by the European Cultural Foundation and the Boekman Foundation in Amsterdam. The proceedings of the meeting explored the possibilities to develop an Index of European Arts, focusing on conceptual and empirical issues such as framework, methodology, and data requirements. EAS contributed with an outline of the Methodology of Composite Indicators developed at the JRC, and with suggestions for the construction of a composite index for European Arts.
The meeting was attended by 20 experts, including representatives from the European Union and the Council of Europe, Belgium, France, Hungary, the Netherlands, United Kingdom, and the United States.
JRC Composite Indicators.
European Cultural Foundation.
JRC Contribution to EC Communication "Measuring innovation output in Europe: towards a new indicator"
16 September 2013, Brussels
As participant in the Inter-Service Taskforce for the EU 2020 Innovation Indicator, the Unit of Econometrics and Applied Statistics (EAS) of the JRC has contributed to the development of a new innovation indicator designed to measure performance in innovation output. The indicator is presented in the EC Communication on 'Measuring innovation output in Europe: towards a new indicator' (COM (2013) 624 final), issued on September 13, 2013. It will support policy-makers in establishing new or reinforced actions to remove bottlenecks that prevent innovators from translating ideas into products and services that can be successful in the market.
In this process, JRC EAS performed all the necessary analysis and robustness tests required for the development of the innovation indicator. These results were discussed at a workshop with Member States on July 12, and they will be presented by the Commission at the next European Summit on October 24-25.
The new innovation indicator was also presented jointly by DG Research and Innovation and JRC at the Research Working Party Council of the EU in Brussels, on 16 September 2013.
Link to COM (2013) 624 final
Link to presentation
“Statistics and Policy”: JRC Scientific Session at the 59th World Statistics Congress
Hong Kong, 30 August 2013
The European Commission's Joint Research Centre (JRC) represented by Andrea Saltelli of the Econometrics and Applied Statistics Unit (EAS), organized jointly with Eurostat the scientific session “Statistics and Policy” at the 59th World Statistics Congress, held in Hong Kong between 25 and 30 of August. The session, designed as a follow up of the joint JRC - Eurostat Seminar 'Scientific Support to EU Decision Making” held in Ispra during 17-18 September 2012, aimed at identifying strengths and challenges in the development, quality assurance and use of statistical indicators in the public discourse and as tools for policy purposes. Andrea Saltelli illustrated the EAS unit’s work on composite indicators and sensitivity auditing.
The World Statistics Congress is a biennial congress of the renowned International Statistical Institute (ISI). This event provides a platform for the international statistical community to share and present the latest knowledge and innovation in statistics.
Download JRC presentation here
Link to the event and papers: http://www.isi2013.hk/en/scientific_list_All_STS.php
Task Force for the EU 2020 Innovation Indicator
Brussels, 29 July 2013
An inter-service task force chaired by RTD and composed by representatives from DGs ECFIN, ESTAT, ENTR, SG and JRC has been set up upon request of the Cabinet of Commissioner Geoghegan-Quinn to reach consensus on one concrete proposal for an innovation indicator which responds to the European Council request in February 2011 and March 2012.The innovation indicator is foreseen as a priority element in the discussion on innovation at the October European Council.
In June 2013, the task force reached consensus on a simple composite indicator zooming in on output oriented innovation, setting focus on four dimensions: the ability of an economy to transform knowledge into marketable innovations, how a well-educated skills supply feeds into the economic structure, the competitiveness of knowledge-intensive sectors, and jobs in fast-growing firms. It also identified areas for future analyses to bring the indicator to its full potential.
The JRC team has computed and tested in total about twenty-five alternative options for the indicator. These calculations have been very demanding and have required intense back and forth of clarification and re-calculations, all this under tight deadlines.
The Task Force for the EU 2020 Innovation Indicator has completed its mandate with the delivery of a report to the Cabinet of Commissioner Geoghegan-Quinn on July 4th.
Six EU Member States among the world's 10 most innovative nations
Geneva, July 1, 2013
Six European Union Member States are in the top ten list of the 10 most innovative nations (Sweden (2), United Kingdom (3), Netherlands (4), Finland (6), Denmark (9), Ireland (10)), while Switzerland retained its place atop the rankings in the Global Innovation Index (GII) 2013, published by Cornell University, INSEAD, and the World Intellectual Property Organization (WIPO). The index was submitted, for the third consecutive year, to an independent statistical audit by the JRC. The JRC recommendations, which aimed to ensure the overall conceptual and statistical coherence of the GII 2013, were taken into account in the final computation of the rankings.
Launch Conference for the Gender Equality Index
Brussels, 13 June 2013
The Unit of Econometrics and Applied Statistics of IPSC was represented by Dorota Weziak-Bialowolska at the launch event for the Gender Equality Index developed by the European Institute for Gender Equality at the proposal of the European Commission. Held in Brussels, on June 13th, the launch conference included speeches of Herman Van Rompuy (President of the Council of the EU), Kathleen Lynch (Presidency of the Council of the EU, Minister of State for Equality, Ireland), Professor Anne Glover (Chief Scientific Adviser to the President of the European Commission), Mikael Gustafsson (Chair of FEMM Committee, European Parliament), and Algimanta Pabedinskiene (Minister of Social Security and Labour, Lithuania). The JRC was actively involved in the panel session “The Gender Equality Index, pain or gain?,” together with participants from Eurostat and the Institute for Gender Studies of Radbout University.
JRC Speech at the IREG Forum on University Rankings
Warsaw, May 16, 2013
Over 130 experts on university rankings and representatives from the academic community from 32 countries met in Warsaw 16-17 May 2013 at the IREG Forum on University Rankings – Methodologies under scrutiny. The conference was organized by the IREG Observatory on Academic Ranking and Excellence and the Perspektywy Education Foundation in cooperation with the Polish Academy of Sciences. The JRC (M. Saisana) was invited by Jan Sadlak, President of IREG Observatory, to give a presentation on the quality criteria for data aggregation used in academic rankings.
Presentation at the JRC Round Table on the Contribution of Universities to Science and Innovation
Brussels, April 11, 2013
Andrea Saltelli illustrated the G03 unit’s work on sensitivity of university rankings in his presentation “Looking at apples and oranges: Learning from university rankings and composite indicators” at the JRC Round Table “JRC – Universities: How to Increase the Contribution of Universities to Science and Innovation” held in Brussels, on April 11th, 2013.
Seminar on Composite Indicators to the World Economic Forum
Geneva, April 18, 2013
A two days seminar was given by Michaela Saisana and Andrea Saltelli to the World Economic Forum in Geneva, on April 18-19, 2013. Participants included researchers working on four WEF indices: Human Capital Index, Global Competitiveness Index, National Resilience Index, and Global Gender Gap Index. It has also been established that JRC will audit the Human Capital Index in early June and the National Resilience Index at a later stage.
Keynote Speech on Quality Tests for Higher Education Rankings
Valencia, March 07, 2013
A seminar (3h) was given by Michaela Saisana to the International Conference on Higher Education Rankings organized by Universidad Politecnica de Valencia in Spain, on March 07-09, 2013. A JRC collaboration was established with Jan Sadlak, president of the International Observatory on Academic Ranking and Excellence (IREG) to work on the Shanghai university ranking.
JRC discusses the Human Development Index (HDI) at the Second Conference on Measuring Human Progress
New York, March 04, 2013
Michaela Saisana of the Econometrics and Applied Statistics Unit of the European Commission was selected as discussant of the Human Development Index at the Second International Conference on Measuring Human Development (4-5/03/2013, New York) before an audience including Nobel prize winners Joseph Stiglitz and Amartya Sen. JRC was selected for its leadership in the field of composite indicators methodology. A JRC paper on the topic is being published these days on the prestigious Journal of the Royal Statistical Society. The Human Development Report Office organized this meeting to consider how best to improve these indices: how these indices can be made conceptually stronger, more statistically robust and easier to use by policy makers, stakeholders and the media. The feedback received from the JRC and other participant organizations (including OECD, Berkeley University, World Bank, Yale University) will be used in the revision of the HDI 2014.
High-level Roundtable on "Scientific Support to Internal Market", Brussels
Brussels, February 21, 2013
At the request of Commissioner Barnier and with the support from Commissioner Geoghegan-Quinn, the Joint Research Centre has organized a high-level roundtable on "Scientific support to Internal Market" on 21 February 2013 in Brussels. This initiative was structured according to three key areas with the greatest growth potential: a more integrated Single Market for services; the better protection and development of intellectual property; the digital economy. This roundtable among high-level scientists, leading consumer NGOs, business representatives, social partners and policy makers has been an opportunity to reflect on the potential role for science and identify priorities where scientific expertise and research should be targeted to help policy makers to find new ways of growth from the single market.
Ratings and rankings: Voodoo or Science?
According to many – including some of the authors of the Stiglitz report – composite indicators have serious shortcomings. Still these measures are pervasive in the public discourse and represent perhaps the best known face of statistics in the eyes of the general public and media. A new JRC paper to appear in the Journal of the Royal Statistical Society – Series A, delves into the potential fallacies of rankings and ratings built as arithmetic averages. The analysis is applied to six composite indicators, including the Human Development Index and two popular league tables of university performance.