Improving Prediction of Energy Futures
Most energy-economic models do not provide policymakers with the information they need to make sound decisions.
When federal lawmakers pass–or do not pass–legislation related to the production and use of energy, their actions ripple across society. Their decisions affect not only the mix of fuels, the price of power, and the spread of pollution, but also federal deficits, corporate fortunes, and even national security. Thus, policymakers need to have in hand the best possible projections about the future demand, supply, and cost of various energy options. Unfortunately, a growing disconnect exists between politicians and the economists who develop those projections.
Various government agencies, as well as an array of universities, private consulting firms, and interest groups, have developed energy-economic models, some more sophisticated than others. Yet lawmakers increasingly feel that these models fail to answer, or even properly evaluate, their questions about the most effective means to achieve policy goals. Economists, meanwhile, complain that politicians do not ask clear questions of the models.
Part of the communication conflict results from the different natures of modelers and politicians: Whereas economists seek quantifiable measures and mathematical certainty, lawmakers deal with anecdotes, dueling stakeholders, and the human chaos of politics. But more fundamentally, a new relationship must develop between policymakers and modelers. Lawmakers need economists to help highlight the actions that would best achieve elected officials’ policy goals, such as the reduction of greenhouse gases to certain levels. Rather than offering only unsolicited advice on the benefits or shortcomings of particular policies, modelers need to provide policymakers with observations on the most effective legislative and regulatory steps to obtain policy objectives.
Critical but troubled
The energy-economic models that policymakers use are critical, because government policies clearly have an impact on the energy market. The development of electricity-generating technologies, for instance, will differ if Congress approves the Bush administration’s Clear Skies initiative rather than stricter pollution standards. That debate depends, in part, on the interrelated set of issues associated with energy, pollution, and national security, and those issues share complex interactions that energy-economic models can use to help estimate the future results of various policy options.
From a policymaker’s perspective, however, the current state of energy-economic modeling is disappointing. Lawmakers frequently see dueling forecasts as little more than lobbying tools for interest groups. Countering the environmentalists’ optimistic estimates of energy conservation opportunities, for instance, are downbeat studies promoted by industrialists. Policymakers, moreover, note the inaccuracies of past projections, and they wish economists were more upfront about the limitations of their models, the reality of uncertainties, and the range of possible scenarios. Lawmakers are skeptical of models that assume a static status quo, and they would like better accounting for technological innovations and “externalities,” such as pollution, health care, and reliability.
Despite such shortcomings, energy-economic models remain the logical means by which policymakers can plan and prepare for the future. But they must be used wisely. Just as people adjust plans in their daily lives as conditions change, so we must appreciate that energy-economic models are only current best guesses about the future.
Policymakers need to understand the limitations and biases of models, and modelers need to admit that energy projections have not been particularly accurate. During the 1960s, energy-economic models tended to underestimate future energy growth. Projections made in the 1970s, in contrast, tended to overestimate energy consumption and production. The energy shocks of the 1970s and the resulting reductions of energy consumption in response to higher energy prices slowly forced economists to substantially lower their consumption estimates. Those lowered projections proved to be fairly accurate, and modelers take pride in the fact that a key 10-year forecast made in 1990 was within 1.4 percent of the actual consumption of total energy in 2000.
Yet boastful economists largely ignore the fact that this forecast overestimated electricity and petroleum prices by approximately 25 percent. One would have expected cheaper-than-anticipated energy to cause more consumption. The fact that energy use remained low with relatively low prices suggests, first, that modelers did not account for technological and market changes that kept energy demand in check; and, second, that modelers underestimated the potential within the U.S. market for energy efficiency. Some researchers looking back at these modeling efforts have determined that modelers underestimated the rate of energy-saving technological change and thus assumed that measures to reduce energy use would require significantly higher energy costs. Researchers also have noted that later forecasts of oil and natural gas prices have not correlated with reality.
Revisiting predictions is a humbling and sometimes instructive exercise. Noted futurists can offer insights as well as miscalculations. H. G. Wells, for instance, presciently predicted in 1902 that transportation systems would be based on automobiles and freeways, yet he failed to account for the role of airplanes. Even Amory Lovins, who is given much credit for bucking conventional wisdom in the late 1970s and accurately predicting slow energy growth, was way off the mark when it came to estimating renewable energy’s market penetration.
Assumptions affect outcomes
Quite logically, models using different factors and assumptions will generate different results. From a policymaker’s perspective, those differences can be aggravating. Suppose, for instance, a lawmaker wants to understand the economic impact of imposing a carbon tax that is expected to reduce the output of greenhouse gases by 35 percent. One model suggests that such action would raise the nation’s economic activity by 1.5 percent, whereas another says that the gross domestic product would fall by 3 percent. What is a policymaker to think if models cannot give a clear answer to the question of whether this carbon tax will help or hurt the economy? The declaration that assumptions matter is not a satisfying response to elected officials wanting to make informed policy.
Policymakers must deal with an array of factors, yet most modelers focus on prices, in part because costs have a clear impact on consumer demand, but also because prices are measurable (and modelers, essentially, are measurers). As a result, the modeling community often ignores the numerous nonprice factors such as environmental quality, national security, unexpected outcomes, and “anomalous” behaviors that influence energy consumption and technological diffusion.
In addition, modelers largely avoid externalities such as the medical costs associated with health problems that result from the pollutants emitted by fossil fuel-fired power plants. Although these expenses are more than zero but less than infinite, most modelers, wanting to avoid uncertainties, tend to stick with zero. This approach is both unrealistic and distorting.
Most modelers also assume, perhaps inadvertently, that the status quo will continue. They tend to make projections based on historical averages, but the reality is that conditions and averages change, often as a result of new policies or technological innovation. History does not progress in a linear fashion, yet most models assume linear trend lines. Models tend to be useful if one wants to know about an unchanging future, which rarely occurs.
Modelers, moreover, typically underestimate uncertainties. No doubt predicting future social trends and technological change is difficult, if not impossible. Some futurists foresee a dramatically changing world, with mass customization and teleworking being just two of the trends that may transform markets. At the same time, new inventions, such as low-resistance electricity transmission, could revolutionize the generation, delivery, and use of electric power. Such uncertainties suggest that energy-economic models would be more useful if they outlined a broader range of possible developments.
Discontinuities, or rapid changes, also present enormous challenges to forecasters. It is often assumed, for instance, that any changes in Earth’s climate that result from increases in the concentration of greenhouse gases in the atmosphere will follow a linear progression. But some scientists believe that the climate will “snap”–change dramatically–when greenhouse gases reach a certain concentration. Predicting that point, of course, is impossible, even if its possibility is important to consider.
Predictions even sometimes conflict with common sense. One key model used in the United States, for instance, estimates that renewable energy technologies will not grow rapidly even if the price of renewable energy is zero. It is hard for policymakers to understand how free energy would not be popular in the market.
While economists themselves debate such shortcomings, often in journals unread outside the field, policymakers need to provide the direction and resources needed for modelers to tackle, if not totally resolve, the most serious problems. If lawmakers are to obtain the most accurate guidance on energy and environmental issues, then they must engage and challenge the modelers rather than simply be the passive recipients of advocacy campaigns laced with economic charts and tables.
Demanding improvement
The nation’s most prominent energy-economic forecasting tool is maintained by the Energy Information Administration (EIA) within the Department of Energy (DOE). The department uses this National Energy Modeling System (NEMS) each year to develop the Annual Energy Outlook. About one-10th of the EIA’s annual $82 million budget is devoted to this model and the analyses of deviations between its predictions and reality. Because the EIA tries to be policy-neutral, the agency does not assume a law’s impact until the legislation is implemented, and it subsequently struggles to provide clear estimates of the impact and effectiveness of alternative policies.
Numerous other federal agencies possess their own energy models, but there is little coordination and sometimes even outright disagreements. The EIA, for instance, estimates that the price of electricity from photovoltaic cells, which convert sunlight into electricity, will remain at a high 16 cents per kilowatt-hour, whereas the National Renewable Energy Laboratory, another division of DOE, predicts that solar prices will fall to a competitive 7.2 cents. At the same time, projections from the Environmental Protection Agency about the potential for energy efficiency tend to be far more optimistic than those of the EIA.
For policymakers to rely on a single model, of course, would be like putting all the federal eggs in one basket. Bureaucrats by themselves will oppose any integration, protecting their turf by arguing that their approach is the best. Policymakers, therefore, must demand the coordination of modeling efforts and a detailed analysis of conflicts. The federal government needs an interagency review, one that consistently highlights the assumption differences of various models, identifies their strengths and weaknesses, and identifies gaps in coverage.
Policymakers may need to ask more specific questions if modelers are to assess the potential for policy alternatives to achieve particular goals. At the same time, lawmakers must exert themselves as a key audience for the modelers’ work. To meet their needs, policymakers need to demand that energy-economic modelers provide more realistic ranges, cooperate with diverse specialists, account for externalities and nonprice factors, and consider the effects of technological innovation.
Modelers and lobbyists will undoubtedly always use economic forecasts to bolster their particular policy perspectives. In fact, interest groups devote substantial resources to justifying their positions with models, data, and scientific-appearing analysis. They sometimes finance researchers who share their biases and then widely promote the findings of those researchers. Although most models by government agencies and scientists are advanced without preconceived conclusions, even they are influenced by the modelers’ biases and assumptions.
Policymakers, therefore, need to demand that models be less opaque, that their biases and assumptions be made clear. This is certainly possible. As noted more than a decade ago by two respected policy analysts, M. Granger Morgan and Max Henrion, “There are some models, especially some science and engineering models, that are large or complex because they need to be. But many more are large or complex because their authors gave too little thought to why and how they were being built and how they would be used.” Many economists bury their analytical assumptions and inadvertently suggest that models are magical “black boxes” that foretell the future. Yet energy-economic projections simply reflect the modeler’s assumptions, and they are more valuable to policymakers when those hypotheses are made clear. Such clarity also would enable other modelers to replicate and evaluate the reasonableness of the assumptions.
Modelers are an esoteric fraternity. Debates within the energy-economic community can be active, if not heated. Modelers themselves criticize models, trying to highlight unrealistic assumptions and to scrutinize data sets. Like many technical experts, they suffer from disciplinary myopia, having the typical reluctance to cooperate with colleagues who have different expertise.
Because forecasting is only as good as a model’s assumptions, policymakers would benefit by demanding the involvement of diverse experts, including marketing gurus, environmental economists, and corporate planning specialists. Marketers are particularly important to engage, since they can help policymakers obtain a realistic sense of the potential of new technologies. Modelers tend to assume that technologies will be adopted only when their price becomes attractive. However, marketers (and any parent with teenage children at a shopping mall) understand that purchases are often made because of attributes other than price. In the energy world, an industrialist might buy a combined heat and power system because it would enhance reliability and security, not caring as much about the initial cost. The insights of marketers would enrich energy models, identifying the array of incentives that can advance technologies in the marketplace.
Policymakers also need to demand a clearer understanding of the economy’s uncertainty and flexibility. They must encourage forecasters to highlight the large range among energy and economic variables.
Lawmakers, in short, need to redefine their roles with modelers. The two groups, although working in different worlds with different demands and requirements, need each other. Unfortunately, many modelers ignore policymakers totally, focusing instead on arcane debates within their expert community. For those who do address policy, the typical approach is to use models as a lobbying tool for a particular policy. This view implies that modelers should try to influence the priority decisions made by lawmakers. An alternative approach would be for modelers to be responsive to policymaker requests for insights about which actions would most effectively achieve an identified goal.
It is the job of politicians to set policy goals for the economy and environment. They need help identifying what tools, such as incentives or controls, can best achieve those goals. No doubt some modelers will continue trying to influence the setting of goals, but they need to do a better job of analyzing policy tools and of helping policymakers understand the most effective legislative and regulatory actions.
Richard Munson ([email protected]) is executive director of the Northeast-Midwest Institute in Washington, D.C.