Making Impact Count

In “What We Talk About When We Talk About Impact” (Issues, Summer 2024), David H. Guston discusses the challenges in defining, measuring, capturing, and demonstrating the impacts associated with research and scholarly activities at institutions of higher education. After highlighting numerous efforts aimed at broadening socially impactful research, he concludes that much more needs to be done to expand the institutions’ reward and incentive systems to encompass these varied forms of impact.

At the US National Science Foundation, we are pleased to contribute to this transformational change through a range of new initiatives, the most significant being the establishment of a new directorate—the agency’s first in more than three decades. The Technology, Innovation and Partnerships (TIP) directorate, which was authorized by Congress in the CHIPS and Science Act of 2022, aims to accelerate technology development and translation, grow a new American innovation base for the mid-twenty-first century, and nurture a workforce of researchers, practitioners, technicians, entrepreneurs, and educators across all fields of science, technology, and engineering. TIP was specifically established to help reestablish the nation’s standing in key technology areas for decades to come.

To achieve this mission, TIP is both enhancing and scaling existing programs—such as the NSF Small Business Innovation Research (SBIR)/Small Business Technology Transfer (STTR), Innovation Corps (I-Corps), and Convergence Accelerator programs—and initiating new activities designed to support capacity-building, use-inspired and translational research, economic growth and job creation, and practical experiences to prepare all Americans for these jobs. For example, the Accelerating Research Translation (ART) program specifically targets building and strengthening the underlying infrastructure for technology transfer at institutions of higher learning, seeking to catalyze a culture devoted to creating and enhancing economic and societal impacts. It requires a diverse set of stakeholders, including senior leadership, technology transfer offices, faculty, industry, nonprofits, and investors, to work together.

The NSF Regional Innovation Engines program similarly encourages cross-sector partnerships to harness a region’s unique strengths and ultimately position it as a national and world leader in one or more key technology areas. And the NSF Experiential Learning in Emerging and Novel Technologies (ExLENT) program invests in regional cohorts of internships and apprenticeships.

These initiatives all require educational institutions and others to go beyond historic measures of impact, notably papers and conference proceedings, and take into account a range of practical quantitative and qualitative data such as invention disclosures, patents, licenses, revenues, start-ups established and acquired, talent trained in degree and certificate programs, and so on.

At a moment of intense global competition, the United States faces consequential decisions that will shape the evolution of its innovation enterprise—the envy of the world. We must continue to lead in curiosity-driven, foundational science, but we must also accelerate use-inspired and translational research. To do this well, we must promote a culture at higher-education institutions and other research organizations that not only acknowledges and rewards historic measures of success, but goes much further in welcoming tangible solutions for pressing real-world challenges in communities across the nation.

Erwin Gianchandani

David H. Guston offers valuable contributions to addressing two essential questions: How should we evaluate the societal impact of research? And how should we build impact into the mission and work of research institutions to maximize their contributions to the public?

These two questions are top of mind for public and philanthropic funders. At The Pew Charitable Trusts, we convene the Transforming Evidence Funders Network (TEFN), a diverse collective dedicated to closing the gap between research and societal outcomes. Like universities, funders are increasingly motivated to demonstrate the public impact of their research investments. Many are gravitating toward approaches that explicitly foster dialogue and partnerships among researchers, community groups, service providers, or policymakers, drawing on evidence that these stronger connections can increase the chances of more expansive and equitable public impact. Some funders invest in engaged research to make creating knowledge more participatory. Others are bridging divides between researchers and decisionmakers through training and policy appointments or resourcing organizations and individuals who work where research institutions, communities, and governments intersect. Each of these investments aims to produce societal benefits while also developing useful knowledge for scholars.

While much progress has been made in understanding and measuring the societal benefits of research, refining and applying these measures requires dedicated and ongoing attention. As Guston notes, the path from ideas to impact, and from impact to outcomes, is complex, involving many actors with competing priorities. Measurement must reflect this complexity, holistically recognizing the many ways that knowledge influences the world. To that end, the William T. Grant Foundation, which is part of the TEFN collective, has proposed a framework for understanding the conditions that lead to the use of research, including infrastructure to sustain cross-organizational relationships, mechanisms for knowledge exchange, and trust. One possible evaluative tool would be to assess the degree to which research fosters these conditions, recognizing that using research in nonacademic settings is essential to achieving public impact.

Beyond measurement, expanded incentives and institutional infrastructure are needed for universities to holistically support public-facing scholarship. As our recent white paper showed, Guston’s university is not alone in thinking deeply about evaluating and rewarding impact. Penn State, for example, recently launched a presidential strategic initiative that includes financial awards for outstanding community-engaged research. Duke revised its tenure standards in 2018 to include a provision on public scholarship, offering guidance for faculty review committees to assess research contributions in the policy or public sphere.

These initiatives illustrate the groundswell of efforts to better recognize an expanded spectrum of research contributions, beyond contributions to academia. To build on this progress, the leaders of Penn State and Duke, along with more than a dozen of their peers, have joined a new effort facilitated by Pew: the Presidents and Chancellors Council on Public Impact Research. This Council will, over the next two years, highlight and strengthen efforts to support university infrastructure for public-facing research and demonstrate the many pathways by which research can shape positive outcomes for people’s lives and communities.

Guston’s article, like our work with funders and university leaders, shows the need for additional innovation and experimentation in encouraging research impact. It is vital that we understand what works, so that efforts at institutional change are grounded in rigorous evaluation. By testing promising approaches, including those that Guston outlines, we can design new paradigms in assessment, incentives, and institutional infrastructure that help our research ecosystem rise to its public purpose and address the most pressing issues of our time.

Angela Bednarek

Benjamin Olneck-Brown

While assessing the societal impact of research is a complex and nuanced endeavor, creating an environment to enable this work is equally challenging. Strengthening the capacity for research translation within institutions of higher education (IHEs) requires a holistic approach that addresses several interrelated components. In particular, IHEs will need to develop researcher capacity, bolster their infrastructure, and foster a vibrant research translation culture.

Researcher capacity. A fundamental step in enhancing research translation is to invest in developing the capacity of researchers to engage in translation activities. Workshops, mentorship opportunities, interdisciplinary training initiatives, and other professional development programs can help foster these essential skills. Decades ago, institutions began implementing programs designed to provide basic teaching instruction to PhD students. Today, academic leaders across disciplines should similarly augment doctoral program experiences to include formal instruction on science communication, policy engagement, or related research translation knowledge and skills. Beyond these general supports, IHEs should cultivate innovation-specific capacity for researchers. Institutions that provide access to strategic resources and tools that facilitate the cocreation of solutions involving researchers, practitioners, and community stakeholders will strengthen researcher capacity for research translation.

Infrastructure support. An effective infrastructure is crucial for enabling and sustaining research translation efforts. IHEs should consider establishing dedicated units that focus specifically on supporting research translation activities. These units will build on and expand traditional research enhancement offerings to include, for example, liaison services with community organizations and tools for disseminating research findings effectively. By providing a robust infrastructure, institutions can streamline the process of translating research into practice and make research more accessible to broader audiences. Building institutional capacity also includes a review and alignment of the operational processes, policies, and incentive structures that facilitate (and at times inhibit) research translation activity. IHE leaders have an obligation to provide clear expectations and to implement practices that support those expectations.

Research translation culture. Establishing a culture that supports research translation is crucial. This involves creating an environment where the institution’s values align with the principles of translation. Notably, the values of diversity, equity, inclusivity, accessibility, and belonging can be tangibly advanced through research translation activities. Open access publication, for example, expands and hastens access to new discoveries. If IHEs value research translation and societal impact, then our systems should both support and recognize this essential work. Celebrating successful partnerships and showcasing impactful outcomes can further motivate researchers to engage in this essential work. Effective leadership is vital for guiding research translation initiatives toward a common strategic direction. IHE leaders should clearly communicate the importance of research translation and the role it plays in the university’s mission. By promoting an environment of transparency and open communication, leaders can ensure that all stakeholders understand the institution’s vision for translation efforts.

The needs are pressing, the time is right, and the rewards are substantial. As institutions of higher learning face growing calls to clearly define their value proposition, strengthening research translation can foster understanding and enhance societal impact.

Diana L. Burley