Cloning DARPA Successfully

Confronted with the growing threat of global climate change, today’s policymakers face the challenge of how to create an energy system that emits less carbon and is more efficient, affordable, and secure. Although technologies exist whose immediate adoption would help reduce carbon emissions in the short to medium term, innovations in new technologies will eventually be necessary to stabilize the climate. A carbon price or carbon portfolio standard will clearly be necessary to ensure market demand for such new technologies; however, it alone will not be sufficient to create the needed market. Further, creating a market for carbon reduction solves only half the problem. New technologies are needed to meet that market demand.

The United States has a long, if mixed, history of government efforts to support technology development. In addition to traditional measures such as facilitating technology investment through tax policy, subsidies, and funding for basic research, the government has experimented with a broad suite of technology policy options. Although departures from the traditional policy repertoire are attacked as misguided efforts by high-level government bureaucrats to choose technology winners, a closer look at history shows that choosing winners is not the only technology policy option. One particularly successful and durable innovation policy alternative, which I call bottom-up governance, has been used for over 50 years by the military in the form of the Defense Advanced Research Projects Agency (DARPA).

Born as ARPA in 1958 as part of the response to the Soviet launching of Sputnik, DARPA’s mission was to prevent future technological surprises. Many blamed the U.S. failure to launch a satellite before the Soviets on the rivalry between the military services, and ARPA was designed to bypass that rivalry. ARPA’s first priority was to oversee space activities until the National Aeronautics and Space Administration (NASA) was established. By 1960, all of ARPA’s space programs had been transferred to NASA and the individual military services. Done with space, ARPA focused its energies on ballistic missile defense, nuclear test detection, propellants, and materials. As observed by historian Alex Roland, it was during this period that ARPA took on the role of nurturing ideas that other segments of the nation would not or could not develop and carrying them to the proof-of-concept stage. Roland notes that it was also in the 1960s that ARPA established its critical organizational infrastructure and management style. Specifically, ARPA decided against doing its own research. Instead, it empowered its program managers—scientists and engineers on loan from academia or industry for three to five years—to fund technology developments within the wider research community. Within this environment, there was little to no hierarchy. To fund a project, program managers needed to convince only two people: their office director and the ARPA director.

Today, DARPA’s success is legendary. Staffed with roughly 100 program managers and an annual budget of about $3 billion, DARPA has been credited with founding everything from the Internet to the personal computer to the laser. In recent years, many federal agencies have begun programs intended to replicate DARPA. In 1998, the intelligence community formed the Advanced Research and Development Agency (ARDA). In 2002, the Department of Homeland Security formed HS-ARPA. In 2006, the intelligence community replaced ARDA with I-ARPA. Then in 2007, Congress directed the Department of Energy (DOE) to create ARPA-E. However, it is not clear how much these DARPA wannabes have incorporated from their avowed model besides its initials. Particularly questionable is whether they have copied the critical factors that are responsible for DARPA’s apparent success.

The program manager as key

Many past government and industry observers have written about the DARPA model and its transferability to other contexts. Some have argued that there is no single DARPA model. Others have focused on DARPA’s organizational culture and structure. Richard Van Atta has identified the following key characteristics of the DARPA model: It is independent from service R&D organizations; it is a lean, agile organization with a risk-taking culture; and it is idea-driven and outcome-oriented. These themes are echoed in DARPA’s self-described 12 organizing elements, along with two additional themes: a focus on hiring an eclectic, world-class technical staff and the importance of DARPA’s role in connecting collaborators. Those who argue against the transferability of DARPA to other contexts have focused on what is unique about DARPA’s role for the military. Berkeley’s David Mowery and others underscore the significance of the military’s long-term needs, both in direction-setting and in ensuring researchers an early niche market for their technologies. But more persistent over the decades than DARPA’s organizational structure or promise of immediate markets are the mechanisms used by DARPA’s program managers for governing technology directions in the United States.

DARPA is often portrayed in the popular press as the military’s radical risk-taking venture capitalist. But this image of hit-and-run investment couldn’t be further from the truth. Rather, the little-studied key to DARPA’s success lies with its program managers. Each program manager, who is temporarily on leave from a permanent position in the academic or industrial research community, is given tremendous autonomy to identify and fund relevant technologies in his or her own field that are relevant to specific military purposes. To carry out their roles, program managers must execute four interrelated tasks: learn about current or forthcoming military challenges; identify emerging technologies that have the potential to address those challenges; grow the community of researchers working on these emerging technologies; and be sure, as this community evolves, to transfer responsibility for the further development and eventual commercialization of these technologies either to the military services or the commercial sector. These four tasks are no small undertaking, but the DARPA program managers don’t go it alone. Instead, they set their directions as much from listening to the voices of active researchers, and nudging them in common directions, as they do from listening to their military customers. The result is a symphony of new research activities that can change the technological direction of the nation.

A host of mechanisms exist by which DARPA program managers and the researchers they fund learn about the challenges facing the military and brainstorm about emerging technologies that may address these challenges. Program managers are in regular contact with senior military officers. In addition, since 1968, DARPA has used the DARPA-Materials Research Council, which was later renamed the DARPA-Defense Science Research Council (DSRC), to bring together top researchers. The DARPA-DSRC assembles 20 to 30 of the country’s leading scientists and engineers, along with 20 or so program managers from the DARPA technical offices, for nearly a month each July. The goal of this meeting is “to apply their combined talents in studying and reviewing future research areas in the defense sciences.” The technical direction of the summer conference is chosen by a steering committee composed of seven representative members of the DSRC, who work with DARPA management to select the relevant technical topics. In addition to the technical meetings, DSRC members engage in activities to better understand military challenges, such as visiting base camps, observing training exercises, and engaging in wargame exercises. After the summer conference, smaller task forces are established on specific topics throughout the year. This conference and subsequent task forces serve many purposes. They act as settings in which lead technical researchers in the nation become aware of military challenges and are able to jointly brainstorm about new technologies for meeting these challenges and thus potential new directions in a field. At the same time, these meetings serve for DARPA program managers as one of many settings in which to hear and identify directions for future funding solicitations.

DARPA program managers do not, however, end their emerging technology identification activities at a series of brainstorming sessions with elite scientists. Program managers continually travel around the country to meet with individual members of the research community and learn about their emerging projects and capabilities. During these activities, they not only identify new research directions but also encourage research in those directions by seed-funding researchers working on common themes. Here, the program managers explicitly do not pick technology winners. Instead, they bet on people. In some cases, they will fund disconnected researchers working on the same technologies. At other times, they fund researchers working on competing technologies aimed at solving the same problem. Once they receive funding from DARPA, researchers are then required to share their work with each other in workshops. These mandatory presentations increase the flow of knowledge among disconnected researchers. As a consequence, these researchers begin to develop common ideas, and new research communities evolve around these directions. In making funding decisions, program managers seldom fund individual technologies. Instead, they envision the suite or pyramid of technologies necessary to meet a particular military goal. In the case of computing, this suite included the materials, processing tools, individual chip designs, software, and system architectures all necessary for significant advance. Finally, Federally Funded Research and Development Laboratories (FFRDCs) are not allowed to apply for DARPA solicitations. Instead, DARPA issues contracts to R&D laboratories such as Lincoln Laboratories to help prepare the solicitations, including mapping long-term technology needs common to the military and commercial sectors.

DARPA as a model for ARPA-E

Although DARPA is a defense-sector institution, there is no reason why its mechanisms for bottom-up technology governance can’t become a model for innovation policy in other, civilian-centered sectors. Take, for example, the energy and automotive industries, both of which need innovation. In an attempt to bolster innovation in energy, Congress approved ARPA-E in August 2007. The new organization was funded in the February 2009 American Recovery and Reinvestment Act, but because those funds are equal to the funds for only a single office at DARPA, ARPA-E so far has only a single program manager and no director. Most important, there are no signs that ARPA-E will necessarily adopt the bottom-up technology identification and community-building processes used by DARPA.

So what would a bottom-up governance approach to the energy and automotive crises look like, if led by an institution like an ARPA-E? First, the director of ARPA-E and the respective office directors would be on short-term rotations from Department of Energy (DOE) government labs, industry, or academia. The offices and their respective directors would be organized around technical areas relevant to the energy problem, and each office would be filled by a suite of program managers who are experts in the respective pieces of that area. Program managers would be leading technical specialists in their fields, recruited to work for three to five years. For an office within ARPA-E engaged on the automotive crisis, these program managers could, for example, be technical leaders from the research labs at General Motors and Ford, as well as professors from the nation’s top mechanical, electrical, and materials engineering departments.

Second, ARPA-E program managers would neither act as venture capitalists nor attempt to pick technology winners. Instead, they would engage in the technology identification and technology community-building activities typical of program managers within DARPA. As done at DARPA, they would need to be careful to fund the full suite of technologies necessary to achieve particular goals. For example, if new power train technologies were being funded, they would need to fund emerging common and competing solutions in materials, fuels, metrology, components, and so on so that all the technology advancements required to revolutionize the power train system would be covered.

Third, ARPA-E should use DARPA-DSRC as a model for an Energy Science Research Council (ESRC), which would bring together 20 to 30 of the country’s leading scientists and engineers and 10 to 20 ARPA-E program managers for several weeks to brainstorm on future research areas in energy sciences. As with the DSRC, the technical direction of the ESRC conference would be chosen by a steering committee composed of several representative members of the council, who would work with ARPA-E management to select the relevant technical topics. In planning the initial conference, it will be particularly critical to bring in the very best technical minds in the country in order to help establish the prestige associated with participating. During the first conference, technical challenges should be identified that require additional, smaller task force meetings. One obvious early task force would focus on personal transportation and the automotive industry. Lessons for ARPA-E’s efforts in the automotive industry could be taken from DARPA efforts in the 1980s to revitalize the semiconductor industry against the perceived threat of Japan. Among other actions, DARPA took the controversial step of channeling the initial matching funds to SEMATECH, the nonprofit industry-founded research consortium in semiconductor manufacturing.

Fourth, ARPA-E should not fund the DOE government labs but instead, as DARPA does with Lincoln Labs, leverage the labs’ expertise to improve solicitations and identify common technology needs across the different energy-related sectors.

The need for a market

Those who question the feasibility of extending the DARPA model beyond military contexts suggest that without the immediate high-paying market for new technologies provided by the military, the DARPA model cannot work. However, during the course of DARPA’s history, the military has not always been the primary market or even the primary beneficiary for DARPA-funded technologies. The computing industry, which benefited enormously from DARPA funding and had the government as a primary market in its early years, offers an example. Although the government was the primary source of demand for the infant computing industry, the balance of demand in the industry has in the recent decades changed dramatically. In 1960, when DARPA began funding computing, 1,790 mainframes were sold, and the majority of computers were owned by the government. In contrast, by the 1990s, innovation in commercial information technology was outstripping advances being developed for military applications, and defense secretary William Perry encouraged the military to figure out how to adapt existing commercial products. Despite the fact that government was not the dominant source of demand during this period, DARPA still managed to provide seed funding for new technologies. The military-related work it funded in computer processing led to the development of microprocessors that were used widely in personal computers.

The energy sector does, however, face a critical challenge in instituting an ARPA-like organization. Knowledge of the crisis is much less centralized. In the military, DARPA program managers talk to officers from the Army, Navy, and Air Force and visit military installations to experience military needs in real time. In the energy crisis, there is no single clear customer. To understand national energy concerns, program managers would need to talk to government and academic experts in energy and the environment, national security, and industrial and economic growth to even begin to identify the nature of the country’s need. For the more specific case of the automotive industry, program managers would need to talk to consumers, auto manufacturers, auto suppliers, and environmental leaders to identify existing and emerging challenges in personal transportation. Although an ESRC and an associated ESRC Automotive Task Force could identify technologies to meet pre-identified national needs, an additional separate Energy Crisis Council (with annual meetings and rotating members) may be necessary to identify and benchmark the evolving nature of the crisis itself. Whereas the ESRC would consist of the nation’s leading technologists, the Energy Crisis Council would need to consist of academic, government, and industry leaders intimately familiar with the nature of the problem itself.

Although I do not believe that the military is required as an early customer for ARPA-like organizations, some market is required for new technologies to be commercialized. The military may provide an early market for some energy technologies, as could other government agencies through the procurement of vehicles, heating and cooling equipment, lighting, and other energy-related products. However, for larger industrial and consumer markets to develop, the government will need to find ways, such as carbon prices or carbon portfolio standards, to incorporate the external costs of carbon emissions into the price of carbon-based fuels. Unless this is done, little demand may exist for new carbon-reducing technologies. It will be important for ARPA-E staff to work with the Energy Crisis Council to stay closely attuned to regulatory trends to ensure that they are responding to national needs for which regulatory action has also created markets. Although it would be politically dangerous for ARPA-E to be involved in regulatory debates, the Energy Crisis Council may be able to act as a go-between to regulators.

Should we be using the DARPA model of bottom-up governance to solve all national challenges requiring technological innovation? Of course not. There are many arms of government currently involved in innovation, each of which serves its own unique role in the overall system. DARPA’s mechanisms for bottom-up governance simply add to the suite of options.

Recently, there has been a renewed call for centralized coordination of the U.S. government’s technology-funding activities. Such centralization is the wrong road. Redundancy within the existing noncentralized technology funding model is a strength, not a weakness. The beauty of the existing system is that if a new alternative power train technology isn’t funded by DARPA, it may be funded by the DOE, the Office of Naval Research (ONR), or any of the other funding institutions. And if ONR funds a technology today, DARPA just may fund it tomorrow, and DOE in the future. This seeming redundancy across funding organizations has always been and will continue to be part of the nation’s strength, as innovation across common technical challenges happens where it should: at the level of the individual researcher.