This article looks at the overall benchmarking of energy efficiency performances and policies. It discusses how country comparisons in terms of effort and achieved results can be carried out and compared1.
Introduction: energy efficiency targets and benchmarking in EU countries
There is a strong commitment in energy efficiency in EU countries. EU policy has 3 main targets to reach by 2020 known as 20*20*20: an aim at a 20% share of renewables, a 20% reduction in GHG emissions and 20% in energy savings. A new directive (equivalent to a law) adopted in October 2012, known as the Energy Efficiency Directive (EED), lists a set of requirements for EU member states to reach this 20% target. It replaces a previous directive, known as the Energy Service Directive (ESD) that was adopted in 2006 and fixed a target of 9% energy savings2 in 2016 for each member state in reference to 2008. For ESD and the new directive EED, the countries have to report to the European Commission every 3 years about their energy savings and the policy measures already implemented or planned in a document called the National Energy Efficiency Plans (NEEAPs)3 . In this context, benchmarking becomes a key priority for a number of reasons:
- To benchmark countries’ performances in terms of energy efficiency achievements and understand which countries perform the best;
- To benchmark policy measures, so as to see which measures are the most effective.
Benchmarking can be done at different levels: at a detailed level, such as an industrial branch (e.g. cement or pulp and paper) or an end-use (e.g. heating), or, at an aggregate level such as a sector (e.g. industry, transport) or at the level of final consumers or of the total consumption. This paper will focus on aggregate benchmarking. Benchmarking can be done in terms of level of performance or rate of progress over a given period: a country may have a good performance at the present time but may not be improving its energy efficiency further whereas another country may have poor performance and improve its energy efficiency rapidly. Roughly speaking, new EU member countries from Central and Eastern Europe fall into that second category (as China at world level).
Benchmarking of energy efficiency trends
The simplest indicator generally used to benchmark energy efficiency trends is the annual variation relating the country’s energy consumption to the GDP. This indicator is generally referred to as the primary energy intensity if it looks at the total consumption of the country (TPES or Total Primary Energy Supply), or final energy intensity if efficiency is considered at the level of final consumers. As energy efficiency policies are mainly targeted at final consumers, the benchmarking will often be done at this level4 . Benchmarking will be done at the level of the primary energy consumption if energy savings are also considered in the energy transformation sector (e.g. power or refining sector). Energy efficiency improvement is then measured by a decrease in the primary or final intensities (less energy used per unit of GDP): a 1% reduction in the energy intensity will be considered as a 1% improvement in energy efficiency. An example of such benchmarking can be found in the energy efficiency indicators of WEC (World Energy Council)5. Fig 1 indicates the average annual variation in the primary energy intensity at world level over the period 1990-2010: the countries registering the largest “energy efficiency gains” are the ones showing the largest reduction, i.e. above 3%/year (case of China or most new EU member countries6).
Fig. 1: Trends in primary energy intensity at world level (1990-2010)
Energy efficiency improvement is not the only driver behind the trends observed. Among the other factors that may explain the decrease in the energy intensity, we can quote changes in the structure of the GDP, in the specialization of industrial activities or in the power mix. For instance, if a country has a strong growth in the production of electronic components, i.e. in an industrial production with low energy intensity, and a low growth in the production of cement, its energy intensity will decrease, all things being equal, regardless of energy efficiency improvement. In other words, this approach only gives a rough proxy of energy efficiency gains. To benchmark the countries in terms of energy efficiency, EU countries are now using more sophisticated indicators, such as the energy efficiency ODEX developed in the framework of the ODYSSEE MURE project. The principle is to select indicators for each end use sector (e.g. households, industry, transport, services) capturing energy efficiency improvement at a detailed level (end-uses, branches, modes of transport) (e.g. toe/m2 for household space heating, kWh/refrigerator) and to set up an index of energy efficiency progress by end-use: for instance if the specific energy consumption for household space heating has decreased by 10% between 2000 and 2010, the index will be equal to 90 for household heating. Then to get the overall energy efficiency index for the sector (i.e. households in the example given), an aggregate index will be calculated by weighting the progress for each end–use by their share in the sector’s consumption7 . The same calculation is done for all sectors and an ODEX is then calculated for all end-use sectors, again by weighting the progress in each sector by their share in the final energy consumption8. The rate of energy efficiency progress for final consumers can then be measured from the annual variation of the ODEX indicator9. As shown in Fig 2, five new EU member countries10 have registered strong energy efficiency improvements since 2000 (calculated between 2 and 3%/year using the ODEX indicator). This good performance is explained in large part by the fact that they start from a situation of low energy performance in buildings and industry; inherited from the time they had very low energy prices. Among western EU countries, with a more similar level of economic and technological development, The Netherlands stand out as the benchmark country in terms of energy efficiency progress since 2000, followed by France and UK.
Fig. 2: Energy efficiency trends in EU countries (2000-2010)
Source: Odyssee
Benchmarking energy efficiency performance levels
Accurate comparisons of energy efficiency performances can only be done at the sectoral or end-use level with technico-economic ratios rather than energy intensities. If one wants to benchmark final consumers as a whole, it is only possible to use energy intensities, i.e. the amount of energy required to generate one unit of GDP. The comparison can however be improved by introducing some adjustments to account for specific national characteristics, in terms of climate, general price level, GDP structure or industrial specialization. Three types of adjustments are quantified in the ODYSSEE database to calculate an “adjusted” final energy intensity that is more relevant for countries’ benchmarking in Europe (Fig 3): • Conversion of the GDP in purchasing power parities to account for differences in the general price level (adjustment important of new member countries where the average price level can be up to three times lower than in western European countries) ; • Adjustment to the same heating requirements, based on the number of heating degree days, to account for climatic differences; • Finally, adjustment to the same economic and industry structures to account for differences in the economic and industrial activities. Germany appears as the country with the best energy efficiency performance in 2010, according to this adjusted final energy intensity, followed by UK and Spain.
Fig. 3: Adjusted final energy intensities: indicator of benchmarking (2010)11
Source: Odyssee
Adjusted final energy intensities compare overall performances in energy productivity and not energy efficiency performances from a technical viewpoint: several additional factors are still embedded in the relative values of these adjusted intensities, such as differences, for instance, in the diffusion of household appliances and cars, in behaviours (e.g. preference for large cars), in the building stock structure (e.g. share of single family houses), in the fuel mix or in the process mix in industry. Are the good results observed in terms of energy efficiency trends for The Netherlands, UK and France, and, in terms of level for Germany and UK, correlated to the energy efficiency policies implemented in these countries or to other factors, not quantified above? Then comes the question: how do we best benchmark these policies?
Benchmarking of energy efficiency policy and measures
To benchmark countries in terms of policies and measures on energy efficiency, we first have to define how to measure the policy intensity and effectiveness in the field of energy efficiency. A first indicator could be the number of policy targets: a country with a wide range of ambitious targets may have a stronger commitment than a country with few or no target at all (Fig.4). Germany appears well as a country with a large number of targets, as well as France. However UK and The Netherlands, who experienced good results, as shown in Fig 2 and 3, do not have many targets. This indicator alone is probably not sufficient to capture the policy intensity.
Fig. 4: Number of policy targets in EU countries by type12
Source: Odyssee
Another indicator of the policy intensity could be the number of energy efficiency measures implemented in the different sectors: adding up the number of measures may not be a relevant enough indicator as some measures may have a stronger impact in terms of energy savings or energy efficiency improvements than others. We may therefore consider the impact of the measures implemented as an indicator, as in the ODYSSEE MURE project. For that purpose, each measure in the MURE policy database13 is classified according to its impact in terms of energy savings, with three levels: strong, medium and low impact14 (Fig. 5).
Fig. 5: Qualification of the impact of measures in the MURE database- case of Germany15
Source: MURE database
A first approach to identify the most active countries and rank them according to the potential impact of their energy efficiency measures is to count the number of high impact measures (Fig. 6). Spain turns out to have the most aggressive policy although this is not reflected so far in the results observed. It is followed by Germany. These two countries belong to the three most efficient countries according to the ranking made above with the adjusted final energy intensity (see Fig. 3). The third one, the UK, only arrives in the 9th position. The Netherlands are only in an intermediate position.
Fig. 6: Number of high impact measures in the MURE database by country
Source: MURE database
A second approach, also used in the analysis of measures in the ODYSSEE MURE project, is to weight the number of measures and calculate an indicator of impact as follows: high impact measures count for 5, medium impact measures for 3 and low impacts for 116 (Fig. 7). France now joins the two other countries that were already identified with high impact measures, Germany and Spain. UK and the Netherlands are still in an intermediate position. The analysis of the two indicators of policy impact shows that there is not always a correlation between a large set of high impact measures and the results observed.
Fig. 7: Indicator of measures impact in EU countries
Source: calculated from MURE database
The benchmarking of energy efficiency depends on the quality of the indicators
Can we conclude from this analysis that countries may register good results although they do not seem to have a comprehensive set of measures? We cannot come to such a decisive conclusion with the indicators used. As indicated above, the indicators used are not perfect, although widely used as they are readily available. First of all, measuring the intensity of a policy from the number of measures, even taking into account their impacts, may hide the fact that a single regulation may have a very strong impact on demand if its requirements are very ambitious, while several regulations, that are badly enforced, or a large number of fiscal or financial incentives, that are not used by consumers, will have a limited impact on demand. In other words, quantity does not always mean quality. Secondly, the assessment of the impact of measures is based on ex ante evaluations or expert judgment and may not be enough harmonized across countries, although some kind of harmonization is done in the MURE database. Although the indicators used for benchmarking the results achieved by the different countries in terms of energy efficiency progress and level are already quite advanced, they do not reflect 100% energy efficiency, as explained above for the adjusted intensity. The ODEX indicator may provide a more accurate vision of the real energy efficiency trends but will still embed the effect of non efficiency related factors that are difficult to correct, such as for instance, the effects linked to the economic crisis in Europe since 2009. The benchmarking of countries’ energy efficiency performances should therefore be considered with care as it depends on the quality of the indicators used in the comparison.
Notes:
1 This article relies on the experience acquired for more than 20 years in a project gathering all EU energy efficiency agencies, known as ODYSSEE MURE and supported by the European programme Energy Intelligence Europe.
2 These savings should be made in domestic transport, buildings and small industries.
3 Reports available by country at: http://ec.europa.eu/energy/efficiency/end-use_en.htm#efficiency.
4 Energy efficiency agencies typically focus their programs on industry, buildings and transport.
5 For more information, see: http://wec-indicators.enerdata.eu/
6 For instance, the Baltic States, Bulgaria, Poland, Romania and Slovakia.
7 If for instance space heating represents 80% of household consumption, the contribution of the 10% progress for heating will be equal to 8% in the sector‘s index.
8 For more information on this calculation see: http://www.odyssee-indicators.org/registred/definition_odex.pdf
9 For instance, at EU level, ODEX was equal to 88 in 2010, this means 12% energy efficiency improvement between 2000 and 2010 or an average rate of energy efficiency improvement of 1.2%/year.
10 Poland, Bulgaria, Latvia, Romania and Slovenia.
11 Final energy adjusted to the EU average GDP and industry structure and climate, with GDP converted in Euros at purchasing power parities.
12 A complete review of policy target in EU countries is available at : http://www.odyssee-indicators.org/publications/PDF/Task-4.1.1-Policy-ta…
13 The MURE database gathering all energy efficiency policy measures in EU countries, Norway and Croatia is accessible at http://www.muredatabase.org/
14
15 Measures addressing space and water heating consumption http://www.odyssee-indicators.org/publications/PDF/Building-policies-br…
16 Such an analysis was in particular done in the following publication (see Figure 2-35) http://www.odyssee-indicators.org/publications/PDF/brochures/buildings…