European Union: Measuring Success

In his keynote speech to the National Academy of Sciences on April 28, 2009, President Obama said that “science is more essential to our prosperity, our security, our health, our environment and our quality of life than it has ever been before.” He went on to announce massive increases in public support for a range of agencies and activities and to commit the United States to an annual investment in R&D of more than 3% of gross domestic product (GDP). His initiative has its parallels around the world. For the European Union, the 3% goal was set in the Lisbon agenda of 2000. Finland, Israel, Korea, Japan, and Sweden have already surpassed this target, and for the 30 countries of the Organization for Economic Cooperation and Development (OECD), this indicator has risen from 2.06 to 2.29% of GDP during the past decade. The commitment to science as a driver of economic growth and competitiveness has now become commonplace.

Things were not always so. Throughout human history, wealth was first achieved by the exploitation of natural resources, then by the husbanding and better management of natural resources, followed by conquest and the acquisition of the wealth and natural resources of others, and finally by the application of knowledge to the creation of wealth by increasing what the French call the technicité of society.

Adam Smith’s partitioning of wealth in society into land, labor, and capital was appropriate to its time and place. Two centuries of innovation have shown that knowledge is now the principal source and component of wealth. Unlike other resources, it has the advantage that it is inexhaustible, constantly expanding, and for the most part free.

This new partitioning of wealth began in the 1990s as an attempt to account for the value of a company’s intangible assets. It has since been applied at the level of national economies. The essential novelty of this approach is to distinguish among natural capital (such as oil in the ground), produced capital (such as buildings and railways), and intellectual capital (all the rest). Intellectual capital includes the knowledge and skills of individuals as well as the collective knowledge, competence, experience, and memory contained in our institutions.

A World Bank team calculated these three components of wealth for all the countries of the world. It estimated that high-income OECD countries have per capita wealth of $439,000, whereas the poorest countries have a per capital wealth of $7,216. The first notable feature of the analysis is the scale of the disparity between the rich and poor. The top 10 countries (Japan, the United States, and eight European countries), with a population of 959 million, had an average per capita wealth of $502,000. The 10 poorest countries (nine African countries plus Nepal), with 278 million people, had an average per capita wealth of $3,000.

The second notable feature is the sources of wealth in each case. Among high-income OECD countries, 2% of wealth is natural wealth, 17% is created wealth, and 80% is intellectual capital. Among low-income countries, the natural and created wealth are respectively 29% and 16% of the total, leaving the intellectual capital at 55%. Thus, whereas the wealth ratio of rich to poor in overall wealth is 62:1, for natural capital it is 5:1 and for intellectual capital 89:1. This comparison of the extremes is reflected also in comparisons among developed nations; the wealthier countries have a higher proportion of their total value in their intangible or intellectual capital.

It is clear that increases in wealth, and therefore in social progress and prosperity, now flow mainly from the creation and use of new knowledge. Should wealth creation therefore be the main objective of investment in knowledge creation? By what metrics should the investment and the returns be measured? With what models should the investment be planned?

Is GDP enough?

Shortly after he took office, President Sarkozy of France set up a Commission on the Measurement of Economic Performance and Social Progress. The motivation was the perception that “what we measure affects what we do.” The most widely used measure of the wealth produced by a society in any one year is its GDP. The aim of the commission was to identify the limits of GDP as an indicator of economic performance and social progress, and to look beyond it to alternative ways of setting and measuring the goals of society. The OECD has undertaken a similar project.

The limitations of GDP are clear: It ignores nonmarket outputs, does not attempt to capture preexisting wealth, counts some negatives (such as military expenditures and the destructive extraction of resources) as positive, ignores other negatives (such as crime and environmental deterioration), and takes no account of inequality in society. In addition, it is clear that increasing the average wealth in a society does not translate into progress for people on all fronts. High levels of economic growth have in many cases been accompanied by a decline in trust, in personal and family security, in leisure time, and in happiness or general satisfaction with life. Broad environmental challenges such as climate change must also be balanced against increased average incomes.

Among the many attempts to develop metrics for these additional measures of progress, perhaps the most widely acknowledged is the Human Development Index (HDI), used by the United Nations Development Programme since 1990. It has three components: life expectancy at birth, knowledge and education, and GDP per capita. All three of these indicators are highly correlated across countries, so it is no surprise that the ranking on GDP is not very different from the ranking on HDI.

Other indexes, such as the Gini Coefficient (emphasis on equality), Gross National Happiness (sustainability, cultural values, environment, and governance), and the Happy Planet Index (satisfaction, life-expectancy, and environmental footprint), do not align very well with GDP. For example, the highest-ranking OECD country on the Happy Planet Index is the Netherlands, which is ranked 43rd.

The first great challenge in measuring inputs to and outputs from the knowledge economy is therefore the reality that GDP growth alone will not deliver all the benefits that society desires. However, without the generation of wealth, it becomes more difficult to meet the rising expectations of people. Translating wealth into broad benefits is therefore a second phase of harvesting the fruits of the knowledge economy. How effectively this is done depends on the political, social, and cultural structures of the society, and these in turn could be expected to benefit from rising average incomes. GDP is therefore not the end point of this investment but the enabler of the multiple end points. In this sense, it is a broadly reasonable objective for investment, and particularly public investment, in the knowledge economy.

The second challenge in attempting to quantify benefits is that they vary greatly in their nature, timing, and duration. Investments in health technology, for example, might lift life expectancy or quality of life for the whole population for the indefinite future. Investments at the boundaries of science might not bring direct benefits for a generation and therefore require appropriate discounting. Investments in some technologies, for example in the military field, might produce no benefits at all as they are overtaken by competing technologies. Investment at the national level should therefore be seen in a portfolio context in which probabilities, timing, and the extent of impact are highly variable. Developing models to accommodate such variability is, to say the least, challenging.

Linking investment to returns

In an editorial in Science in 2005, U.S. presidential science advisor John Marburger drew attention to the fact that despite massive investments in science and technology by government and business and the almost universal acceptance that this was worthwhile, metrics and methodologies for evaluating investment and returns were poorly developed. He therefore suggested increased attention to what he called the science of science policy. Such a program has now been established jointly at the U.S. National Science Foundation and Department of Energy.

In the European Union (EU), some progress has been made on this front. Because of its nature as a union of 27 independent countries, most of the investment takes place at the level of the individual states. In contrast to the United States, where 94% of public R&D funds are federal, just 7% of public funds flow through central EU programs. The 27 EU countries therefore constitute a sort of rolling experiment in which useful comparisons can be made across countries and time.

The most widely acknowledged assembly of data in this respect is the European Innovation Scoreboard (EIS). It currently uses 29 indicator statistics, 19 of them tracking various measures of investment and the remaining 10 being measures of output or impact. These are then assembled into a composite index that broadly represents the state of development in science and technology in each country.

The metrics used are grouped according to function. The first group, called “enablers,” consists of five measures of education and four measures of financial and technical support. A group of “firm activities” consists of 11 measures of business expenditure and initiative, together with measures of intellectual property. The “outputs” include three measures of innovation at the level of firms and six measures of economic effects as reflected by employment in and sales from high-tech activities and firms. Making sense of these varied statistics is highly challenging and is still a work in progress.

Despite these qualifications, the EIS does show that countries that score high on the sub-indices of inputs see this reflected in their output measures. On both aggregate scores, the Scandinavian countries, together with Germany and the United Kingdom, occupied the highest places and are classified as the innovation leaders. A further group, including the Netherlands, France, Belgium, Ireland, and Austria, also scores above the average of the EU 27 and is classified as innovation followers. The lowest ranks on the 2008 EIS are held by countries that are still emerging into the modern world from two generations of the communist experiment.

Sufficient data on nine of these indicators are available from 48 countries, and this information has been used to develop the Global Innovation Scoreboard (GIS). In compiling the GIS, activities and outputs at the firm level are given 40% of the weighting, human resources and education 30%, and infrastructures and public investment 30%. Sweden, Switzerland, and Finland still top the list, followed by Israel, Japan, and the United States.

These science and technology indicators are proving to be increasingly useful as countries use them to inform public investment with a view to improving competitiveness in the short term as well as growth and prosperity in the longer term. However, as a model for investment and return, they have two major deficiencies. First, they do not take account of the relationships between all of the indicators. Some may be highly and positively correlated and so be partially redundant. Others may be negatively related (as in competition for limited investment funds), so that tradeoffs are required. Second, they are all simultaneous metrics, snapshots of one year’s statistics. As such, they fail to take account of events that transpire over time. Some investments must be sustained over many years if they are to have their desired effect, the outputs can take many years to emerge, and the expected pace of emergence will vary among outputs.

Challenges of this kind have led to the development of efficient models in some more specific fields. One such field is quantitative genetics, particularly in planning genetic improvement in animal populations. The model that has evolved in this case is called the selection index. It begins with a definition of the desired outputs; for example, increased milk production, higher protein or fat content, better fertility, or disease resistance in dairy cows. For each of these traits, an appropriate economic weight is calculated. If appropriate, discount factors are applied to allow for delayed returns. A selection objective is thus defined (comparable in the wider context to GDP). Next, the inputs requiring investment are identified. These include expenditures on recording the performance of large groups for indicator measurements. Then the relationships between all of these input and output indicators are calculated from historical data. Finally, an index of the inputs is derived which maximizes the gain in the selection objective. Secondary statistics can be calculated that measure the contribution of the inputs, singly or in groups, to the gain achieved and also the contribution of gain in each component of the output to overall gain. These statistics help to refine the investment process.

If GDP growth is accepted as the first objective of public investment in science and technology, the immediate task is to define GDP in terms in which impact can be measured. Of the three standard definitions, the one that serves this purpose best defines GDP as the sum of gross value added (GVA) in different sectors of the economy. Because the objective is GDP per capita, this can be achieved by growth in GVA per capita in individual sectors through productivity gains or by an increase in the numbers employed in the sectors with higher GVA per capita, with a parallel reduction in those employed in the sectors with lower GVA per capita or among those not in employment. This makes it possible to build a quantifiable objective function that captures the expected benefits from investment in science and technology. The remaining task is to develop metrics for the various inputs and to quantify the network of relationships among the inputs and between these and the elements of the objective output function. With appropriate data, this model could help to guide investment in building the knowledge economy.

Public investment in R&D is on the order of 1% of GDP in many advanced countries. In the EU, it is approximately $100 billion per year, of which 87% is in the civilian sector; in the United States it is roughly $150 billion, of which 42% is civilian. These enormous funds are invested to create knowledge, and through it progress and prosperity for future generations. This 1% is our seed corn. Better metrics and models to guide its investment are required.