Transforming Energy Innovation

The United States must change the way it produces and uses energy by shifting away from its dependence on imported oil and coal-fired electricity and by increasing the efficiency with which energy is extracted, captured, converted, and used if it is to meet the urgent challenges facing the energy system, of which climate change and energy security are the most pressing. This will require the improvement of current technologies and the development of new transformative ones, particularly if the transition to a new energy system is going to be timely and cost-effective.

The Obama administration and the secretary of energy understand that the status quo will not deliver the results that are needed and have significantly increased funding for energy research, development, demonstration, and deployment and are putting in place new institutional structures. But policymakers must also pay much more attention to improving the management and coordination of the public energy-technology innovation enterprise. For too long, a focus on design and management elements necessary to allow government-funded innovation institutions to work effectively has largely been absent from policy debates—although not from many analysts’ minds and reports.

The nation is, in fact, at a historical point where the energy innovation system is being examined, significantly expanded, and reshaped. This provides not only a rare opportunity, but indeed a responsibility, to improve the efficiency and effectiveness of the system to make sure that this investment yields the maximum payoff. The importance of improving and better aligning the management and structure of existing and new energy innovation institutions to enhance the coordination, integration, and overall performance of the federal energy-technology innovation effort (from basic research to deployment) cannot be overemphasized. The technology-led transformation of the U.S. energy system that the administration is seeking is unlikely to succeed without a transformation of energy innovation institutions and of the way in which policymakers think about their design.

Drawing lessons from successful efforts by some large U.S. private-sector research institutions and by the national laboratories, we highlight five particular elements that we believe are key to effective management of the energy innovation institutions: mission, leadership, culture, structure and management, and funding. All of these elements must be healthy and in balance with one another to ensure the robustness of the innovation ecosystem.

Furthermore, the external political and policy environment within which these institutions are embedded and the evolving nature of innovation also determine their effectiveness. Therefore, managing outside intervention, coordinating with broader policies and regulations, and adapting to the dynamic nature of the innovation should be seen as an integral part of the challenge.

Apparently aware of the inadequacy of the structure and management of the current federal energy-technology innovation effort, the administration has promoted several initiatives such as the Energy Frontier Research Centers to support larger research teams working in coordination and the Energy Innovation Hubs to better integrate the innovation steps from research to commercialization as was done by Bell Labs. It has also supported the creation of ARPA-E to try to replicate the success of the Defense Advanced Projects Research Agency (DARPA). Other experts have proposed additional strategies. In Issues, Ogden, Podesta, and Deutch proposed the creation of an Energy Technology Corporation to select and execute large-scale demonstration projects, and a panel assembled by the Brookings Institution recommended the creation of a national network of a few dozen Energy Discovery–Innovation Institutes to address the failure that “most federal energy research is conducted within ‘siloed’ labs that are too far removed from the marketplace and too focused on their existing portfolios to support ‘transformational’ or ‘use-inspired’ research targeted at new energy technologies and processes.”

The S&T community welcomes decisive government action, but the management and coordination of new and old initiatives and institutions are indeed very difficult and complex tasks and are not receiving enough attention from policymakers. The Energy Innovation Hubs, for example, could—if they adhere to the principles we describe—fill an important gap in the U.S. innovation system by better integrating basic and applied science in a mission-driven approach. The design of ARPA-E has incorporated some of these principles from its inception, particularly the independence of the leadership. Its success will be determined in part by the extent to which the other principles are applied in practice.

The technology innovation process is complex and nonlinear; complex because it involves a range of actors and factors and nonlinear because technology innovation occurs through multiple dynamic feedbacks between the stages of the process. Furthermore, the technology innovation system is made up of many institutions, including universities, large firms, start-ups, the federal government, states, and other international and extranational institutions, and the relationships among them.

The complexity of the innovation process is especially great for energy technologies for a number of reasons:

• There are limited and uncertain market signals for energy research, development, and demonstration (RD&D) and for deployment. The externalities of greenhouse gas emissions and energy security, for instance, are not appropriately represented in the market, and thus there is widespread belief that the RD&D and deployment that take place are not commensurate with the challenges facing the energy sector and the technical opportunities that are available.
• The large scale of many energy technologies such as clean coal processes and nuclear power and the long time frames over which their development takes place further hinder the participation of the private sector in the development of such technologies.
• Energy technologies are very heterogeneous within each stage—RD&D, early deployment, and widespread diffusion—offering different challenges, with early deployment and subsequent expansion issues being particularly important.
• Energy technologies ultimately have to compete in the marketplace with powerful incumbent technologies and integrate into a larger technological system, where network and infrastructure effects may lead to technology “lock-in.”

The public-goods nature of energy technologies thus necessitates a multifarious role for the government: ensuring the availability of future technology options, reducing risk, developing more appropriate market signals, and often even helping create markets. As a result, the federal government is not only a major funder and performer of energy RD&D but also a major player in facilitating the introduction of technologies into the market. Given the particularly large scale and scope of the energy area, the design and management of energy innovation institutions and their interactions with the rest of the energy system are extraordinarily complex.

Working out the specific details for each of the energy innovation activities will require further effort, but the literature on the design and management of innovation institutions and the personal experiences of managers do offer some key interconnected principles for the success of the new, and reshaping of old, public institutions and initiatives for enhancing U.S. energy technology innovation.

Guiding principles

The public-sector effort must be pluralistic, reflecting the range of activities and needs relating to the energy sector. For example, in technology areas where the venture capital community or other private-sector entities are active in pursuing the commercialization of technologies and the barriers to demonstration and deployment are lower, it may make more sense to emphasize the university research/spinoff model. In technology areas where infrastructure and networks are essential, public/private partnerships may play a key role. And in many cases, centralized R&D facilities, modeled on large corporate laboratories that focus on long-term research but are informed by real-world needs, may be the most appropriate.

The concept of “open innovation” illustrates the dynamic nature of the innovation system. Pioneered by Berkeley’s Henry William Chesbrough, this model of innovation assumes that useful knowledge is widely distributed and that “even the most capable R&D organizations must identify, connect to, and leverage external knowledge sources.” Chesbrough and his colleagues argue that ideas that were once generated only in large corporate laboratories now grow in settings ranging from the individual inventor, to start-ups, to academic institutions, to spinoffs, and they claim that at least in some high-tech industries, the open-innovation model has “achieved a certain degree of face validity.”

Although we recognize the heterogeneity of energy systems and the ever-changing nature of the relevant innovation enterprise, on the basis of our personal experience and the history of the corporate and national labs, we maintain that it is still possible to identify five elements that are essential to create and run a successful technology innovation institution. These elements are: having a clearly defined mission; attracting visionary and technically excellent leaders; cultivating an entrepreneurial and competitive culture; setting up a structure and management system that balances independence and accountability; and ensuring stable, predictable funding. We believe that they provide a good framework for restructuring current institutions and designing and evaluating proposals, once a functional gap in the system has been identified.

A clearly defined mission that is informed by, and linked to, a larger systems perspective. As is clear from the examples of some large corporate labs (such as Bell Labs, Xerox PARC, and IBM), mission, leadership, culture, and funding are important to create a productive innovative environment. Several examples of successful large U.S. government–driven efforts, such as the Manhattan Project and the Apollo Project, highlight particularly well the importance of having a clearly defined mission.

A well-defined mission facilitates attracting top employees, enables reaching an institution’s objectives more effectively, and adds significantly to the overall integration and coordination of the energy innovation system. The inspiration provided by an exciting mission has often been the most forceful magnet for attracting the best people and for research at the frontier. And clarity about the mission facilitates the design of an appropriate organizational structure for the institution.

The technological underpinnings of an innovation institution should be adaptive and flexible enough to reflect the needs, information, and context of the times. The telecommunications sector, for instance, provides an excellent example of research efforts that are able to adapt quickly to incorporate new technologies, satisfy new demands, and comply with new regulations.

Finally, the mission of the publicly funded innovation institutions should typically focus on providing a public good and/or tackling a market failure. Whenever possible, the licensing or technology-transfer practices of these entities should aim to create an industry rather than a product.

Leadership that has proven scientific and managerial excellence, has a vision of the role of the institution or enterprise in the overall energy system, and is capable of acting as an integrator of processes. An institution or initiative charged with an innovation mandate must have a visionary leader with excellent managerial as well as scientific/technical credentials. The additional complexities of the energy system require leaders to also have an exceptional understanding of the role of his or her institution in the overall system and an ability to integrate the different activities within and outside the institution.

As with the mission, leadership should not only have a systems perspective but also be adaptive and cognizant of the dynamic nature of markets and the innovation system. Leaders must recognize change and manage and structure their organization accordingly.

A leader with strong people-management skills can create an environment where people know the boundaries but are able to flourish. In other words, people must not feel actively managed; they must have freedom but without losing focus. As we will discuss shortly, leaders can also promote this “insulate but not isolate” management approach through different levels of the organization by fostering the right culture and putting in place the right structure.

Fortunately, President Obama has chosen Department of Energy (DOE) leaders whose experience prepares them for the holistic thinking required to manage energy innovation institutions, and this needs to be propagated throughout the energy innovation institutions.

Entrepreneurial culture that promotes competition but also collaboration and interaction among researchers. Cultivating the right culture is often forgotten in discussions about the management of government enterprises. But one cannot overstate the importance of having a culture of excellence that pervades the entire organization from the top to the bottom. A culture of technical excellence helps drive a virtuous circle in which the best will be attracted to the important research in energy-technology innovation partly because that is what the best are working on.

Researchers and managers should operate in an entrepreneurial environment that encourages personal initiative and creativity and the search for new ways of solving problems. This environment should be characterized by openness and healthy competition and a culture for attacking the most pressing and challenging problems. The principle of insulate but not isolate is also useful in describing the balance to which these entities should aspire. Researchers must have independence to test and experiment, and at the same time be accountable for their efforts. In other words, they must have freedom to explore new paths while keeping in mind and being motivated by the long-term questions.

Exploration of multiple approaches should be welcome and encouraged, especially during the early stages of research. Using several approaches, in particular for high-priority projects, increases the chances of overcoming problems and finding a more optimal solution and may reduce product development times. However, the value of pursuing multiple approaches will be realized only with the right culture of collaboration as well as entrepreneurship.

Management procedures and organizational structures that promote independence, and yet give primacy to performance and accountability. Structurally, it is important that the management do all it can to break down the all-too-common separation between basic and applied research and among disciplines. The greater the integration between those doing basic and applied research, the faster technologies will incorporate relevant new fundamental knowledge about processes and materials. As former IBM executive Lewis Branscomb has discussed, corporate laboratories such as the T.J. Watson Research Laboratories of IBM do not attempt to distinguish between basic and applied research; their work is simply referred to as “research.” Energy innovation institutions must also eliminate this divide.

The independence of directors and managers at different levels of the organization and the creation of a critical mass of researchers for each undertaking will also be essential to make progress at the speed required by the energy-related challenges. Research directors and managers must have a high level of independence to adjust their actions at their discretion by allocating resources and people according to new information as it becomes available, while focusing on the long-term mission of the organization. At the same time, the director of the innovation institution needs to have regular access to the Secretary of Energy for strategic reasons and to ensure the nimbleness of the organization to circumvent bureaucratic barriers.

A critical mass of researchers is essential to combine sufficient expertise and points of view to arrive quickly at the best possible outcomes. Intellectual competition with exploration of multiple pathways is critical for success; thus, the structure and the culture of the organization are tightly interrelated.

Recruitment of the best talent is a key to success in innovation and requires the attention of top management. This sends a signal of the importance of recruiting and allows those with deep technical knowledge and breadth of vision to bring on board the appropriate kind of talent for the long-term direction of the organization.

The management of energy-technology innovation institutions should also expend significant effort in developing a cadre of technically skilled managers who can nurture creative scientists, inventors, and problem-solvers and have the capacity and knowledge to carry out meaningful performance reviews of personnel and programs. This ensures that those entering the institution are acculturated appropriately, that there will be a pipeline of trained managers for the continuity of the culture of excellence, and that researchers and managers will continue to bring value to the organization by avoiding knowledge stagnation.

Stable and predictable funding that allows a thorough and sustained exploration of technical opportunities and system-integration questions. Stable, sufficient, and predictable funding is another important requirement for a successful energy innovation institution. R&D projects tend to need several years (although the specific time scale may vary from area to area) before a decision can be made about whether to move forward. R&D managers need to have certainty that funds will be available to meet their goals and to produce enough information to enable managers or directors to make decisions about the future of each project. Block funding at each appropriate management level is important.

The annual appropriation process, at least in the form it has taken in recent years, is not suitable to support research in energy innovation. The year-to-year percentage change in funding over time for fossil energy, energy efficiency, and renewable energy at the DOE serves to illustrate the point about stability. Figure 1 shows the year-to-year DOE funding variation for six fossil energy and energy-efficiency research areas from 1978 until 2009. The average standard deviation of the variation across these six programs was 27%. This means that on average every year there was a one-in-three chance that these programs would receive a funding change (increase or decrease) greater than 27%. The gas program had the largest standard deviation of the year-to-year funding variation (36%), whereas the vehicle technologies program had the lowest standard deviation (18%). Figure 2 illustrates the large year-to-year variation for five renewable energy areas from 1992 to 2008.

Although some changes in overall funding amounts from year to year are expected, the high volatility of the DOE budget reflects department-wide rapid change in priorities and directions. Any good strategy should be flexible enough to be adjusted and improved with time as more information becomes available, but the changes in the DOE’s funding reflect the difficulty of securing block amounts of funding for specific purposes over time. Funding should have second-level, as opposed to first-level, fluctuation constants, by which we mean that after setting a general direction, which may be revised when new analysis and information are available, funding changes should not disrupt a program’s main operations, although minor adjustments may be allowed. Block funding enables research managers to make rapid adjustments within their programs over a couple of years without having to go back to Congress. As with research performance, decisions about funding should be subject to review. The needs for performance reviews and funding stability are intricately linked, because without one, the other one will not work.

Overall, we think that it is useful to think about the fact that there are certain features of the management of innovation institutions that are important at every scale—from the individual researcher, to a research group, to divisions, to the overall institution. They can be thought of as fractal characteristics that are present at different scales or levels. The most important principles to think about at several levels within an innovation institution are the need to insulate but not isolate, to allow independence but with review, to provide funding at appropriate scales, and to focus on people.

The outside world

In addition to the internal factors essential for an institution to successfully perform or fund energy innovation, there are other factors external to the institutions that policymakers should heed.

The first is the need for independence from outside interference, which in the case of public agencies includes the political process. Bureaucratic interference should be minimized because it often collides with the five elements discussed above. The notion of insulate but not isolate is as true for institutions as it is for research groups and individual researchers.

The coupling of research and funding institutions to the external environment has to be delicate. Although their priorities and programs obviously have to be governed by and respond to societal needs, they should not be shaped by political exigencies or issues that are the “flavor of the month.” Like researchers and R&D managers, the innovation institutions themselves need independence, tempered by accountability.

The second main external factor that innovation institutions should manage is the evolving nature of innovation. As Chesbrough and his colleagues have argued, networks and linkages are becoming increasingly important, because in some sectors knowledge is widely dispersed across a variety of settings from garages to national labs. Institutions must be designed to function within the overall system and to evolve with it. They should incorporate an understanding of the role of the private sector, regulators, and consumers, as well as an openness to a certain degree of coupling with these other actors.

To conclude, the much-needed technology-led transformation of the U.S. energy system is unlikely without a transformation of its public energy innovation institutions. To be effective, the internal design of these institutions has to be given careful thought, with particular attention to the five elements mentioned above to nurture research and couple it to application. But the best-functioning institution can become ineffective in the absence of an appropriate external political and bureaucratic environment and appropriate linkages with other actors. Therefore, just as an energy innovation institution is itself a delicate ecosystem, it should also be seen as part of a larger ecosystem, with the understanding that the effectiveness of the overall innovation system depends on the functioning of the components of the system, the relationships among them, and a connection to the broader environment in which it is embedded.