Science Advocacy and Scientific Due Process

One of the FY 1999 federal appropriations bills included a two-sentence amendment that guaranteed public access under the Freedom of Information Act (FOIA) to “all data produced” by federally supported research scientists. This amendment, introduced by Sen. Richard C. Shelby (R-Ala.), has precipitated a highly charged public debate, with industry stakeholders enthusiastically supporting the amendment as a means of challenging scientific studies that underpin costly regulations, tort suits, and higher health risk estimates, and the scientific community fearing increased paperwork, intellectual property searches by industry and academic competitors, unwarranted access to personal information on research subjects, and even deliberate use of the amendment to harass researchers. It did not help that the Shelby Amendment was designed to apply only to nonprofit universities, teaching hospitals, and research centers, and not to federally funded research undertaken by for-profit companies.

Attempts by the scientific community to convince Congress to repeal the amendment have failed, but pressure did persuade the Office of Management and Budget (OMB) to issue guidelines that reaffirm FOIA’s privacy and trade secret protections and limit the data that must be produced to published or cited research used by the federal government in developing legally binding agency actions. The U.S. Chamber of Commerce intends to bring suit to challenge OMB’s interpretation as much narrower than Congress intended.

The amendment may appear to be an isolated instance of overreaction by a conservative senator and his allies who were angered by a particular case in which business interests were denied access to confidential medical data compiled by a Harvard-based research team whose studies prompted the Environmental Protection Agency (EPA) to tighten costly air pollution standards. But the Shelby Amendment is actually just the latest example of a trend for all sides in environmental, health, and safety disputes to concentrate their advocacy “upstream” on the scientific and technical studies that precede regulations. Such probing should be welcome where it helps ensure a sound scientific basis for public policy. Yet at the same time, more “scientific due process,” or at least the wrong kind of it, could undermine the credibility and independence of U.S. science.

Another example of more scientific due process is evident at the National Research Council (NRC). A few years ago, a court victory by the Animal Legal Defense Fund, which objected to the way in which members were selected for a committee on the proper care of laboratory animals, would have forced the NRC to comply with the Federal Advisory Committee Act (FACA) in all its committee work. The Fund argued that FACA required the NRC to select committee members on the basis of “points of view” instead of scientific acumen alone, to deliberate and draft recommendations in full public view, and to allow federal sponsors to control meetings.

Congress responded by passing legislation that exempted the NRC from these requirements but imposed new “sunlight” requirements, which included requiring the NRC to post prospective committee members’ biographies for public comment before final selection, announce open meetings in advance, post meeting summaries, make available any materials presented to a committee, and disclose names of peer reviewers after the fact. These requirements have caused delays and heightened controversy over some NRC committees, such as those formed to address genetically modified crops and the health risks of nuclear radiation. It is not yet clear that the new measures can actually improve the quality of NRC guidance on science-rich national policy issues, but there is no question that the new processes for vetting committee members and conducting open committee meetings have become a good deal more procedurally burdensome.

Government agencies have been steadily increasing the procedural requirements for health risk assessment, peer review, research design, informed consent by research subjects, and oversight by researchers’ home institutions. Yet these efforts still fall short in the eyes of congressional advocates for “regulatory reform,” who since 1994 have sought legislation mandating procedures that govern numerous aspects of research conducted with funding from any federal agency involved in health, safety, and environmental regulation. Reform advocates feel that federal regulations to reduce risk are needlessly costly, in part because they have not been prioritized to address the worst risks first, and in part because they are based on worst-case assumptions about risk that are applied by default without adequate justification or public ventilation and peer review–that is, without adequate scientific due process.

None of the bills has passed, despite their sponsorship by prominent legislators such as former Senator Robert Dole (R-Kan.). Still, strong support currently exists for a bipartisan Senate bill cosponsored by Senators Fred Thompson (R-Tenn.) and Carl Levin (D-Mich.). Interestingly (at least to lawyers), this bill and most of its predecessors would severely curtail the opportunity for judicial review that ordinarily is available for alleged agency failure to follow the procedures and requirements set out in statutes. Why Congress would want to restrict judicial oversight of scientific due process while encouraging judicial review of other agency regulations remains unclear.

Pressure for change also comes from within the scientific community. Leading scientific journals have expanded their procedures to require fuller disclosure of potential conflicts of interest for authors and peer reviewers. High-stakes discoveries and clashing social ideologies drive this version of scientific due process. For example, the authors of a manuscript on interleukin (a human immune molecule that they had sequenced and cloned) filed a multimillion dollar suit against the peer reviewers selected by the journal Nature, to which the manuscript had been submitted. The authors alleged that the peer reviewers stole ideas from the paper to prepare their own patent application.

Going a step further, the American Journal of Respiratory and Critical Care and the American Journal of Respiratory Cell and Molecular Biology have refused to publish any research funded by the tobacco industry, in despair that they could ever fairly or accurately evaluate such research. Other journals such as the American Journal of Industrial Medicine have sought disclosure of authors’ affiliations, funding sources, and financial interests. The trend may be toward broader disclosure, but it has caused more than one scientist to share the concerns raised by Kenneth Rothman in an article in the Journal of the American Medical Association on the “new McCarthyism in science.”

What next?

As the stakes get higher, primarily because of the rapid creation of new intellectual property through genetic research, we may expect peer review policies to become even stricter and more formal. My view is that it should be possible to develop consensus policies that will be policed by journals and scientists themselves, thus avoiding legislative intervention or cumbersome agency guidelines. But these policies will need to be more explicit, and peer review arrangements may need to be made contractual in nature. Otherwise, federal rules governing peer review could become as detailed as the guidelines of the Securities and Exchange Commission on insider trading in stocks.

In its 1993 Daubert decision, the Supreme Court directed lower courts to be “gatekeepers” for science that parties introduce in federal trials. As explained more fully in Margaret Berger’s article in this issue, judges now conduct a separate “Daubert hearing” on whether a scientific theory or research technique is verifiable, has been peer reviewed and published, may mislead because of errors, and has won acceptance in the scientific community. My point here is that such hearings are a pure application of scientific due process. Recent decisions show that the Court thinks Daubert is a success and intends to broaden judicial supervision of scientific proof.

Daubert aside, some scientists whose work bears on a contested issue have become deeply mired in legal controversies that they did not even know about when they began their research. Parties to legal action go to great lengths today to probe the data, reasoning, and bona fides of independent research that their adversaries cite in support of their point of view. Judicially compelled disclosure of research may serve important public purposes, but it may also be used to chill scientific research, such as that on the health effects of tobacco. Critics fear that the Shelby Amendment will exacerbate this trend.

Judges have long had broad discretion to forbid as well as compel research disclosure. They have blocked disclosure by approving confidential settlements and sealing court records to expedite settlement of lawsuits without a trial. As a result, behind voluntary dispute resolution, which is almost universally praised today, lurk scientific disclosure issues of great consequence. Settling parties may agree not only to keep secret settlement amounts, but also to deny access to studies and data that bear on the injuries allegedly sustained. When this happens, research scientists and health officials are denied access to victims and their medical records. Attorneys for plaintiffs as well as defendants may use confidentiality as a settlement tool.

Science advocacy tactics now may go beyond sealed settlements. After the Exxon Valdez oil spill in 1989, some of the scientists hired by the federal government, Alaska, Exxon, and other parties complained that they were not permitted to publish findings, visit damaged beaches, or consult with scientists hired by opposing parties. Parties have also hired scientists as expert witnesses as a way of blocking their adversaries from gaining access to their expertise and data. (Likewise, attorneys themselves sometimes find that they also are targets, to prevent their being hired by the other side.) Restricting scientific data and curtailing collegial consultation and publication via strategic hiring and case settlement became such an issue to the scientific community that the National Academy of Sciences (NAS) hosted a workshop to discuss ways to avoid such strictures.

Scientists and physicians engaged in biomedical and clinical research are concerned that we are on the threshold of a proliferation of property, privacy, and other personal rights for research subjects and of new procedures and duties for medical institutions. David Korn of the American Association of Medical Colleges put it well when he suggested that we must be careful to select the right “equipoise” between protecting the rights of research subjects and patients and allowing access by researchers to patient data so that biomedical research can advance. My fear is that we appear to be moving into an era of formalistic, even adversarial, medical record keeping. We focus more and more on how injurious to an individual’s privacy interests the medical record may be, and less on how that record may best serve the society as well as the patient. Discussions of the “medical record” by hospital personnel are beginning to take on the characteristics of attorneys’ discussions of the “legal record,” which provides the critical repository of information on which decisions are based in regulatory rulemakings, trials, and appeals. Because decisionmakers cannot “go beyond the record” in arriving at their conclusions, lawyers devote an enormous amount of time sparring over the contents of the record. We are in danger of creating an environment in which the content of the medical record becomes a focus of controversy and thus a magnet for elaborate rulemaking.

In recent years, advocacy tactics and legal maneuverings have not been enough for some interests, who have resorted to direct attack on the integrity of researchers. Many scientists believe that federal and state scientific misconduct filings are on the increase, although a hard empirical look might show otherwise. Well-organized campaigns against certain types of research and the researchers who conduct them do appear to be on the rise, however, involving efforts to harass and intimidate researchers by impugning their integrity or motives, chill new research, increase the costs of research, and deter volunteers for research or complicate informed consent.

I do not want to overstate the newness of these developments in science advocacy and scientific due process. Attacking “junk science” and pleading for “sound science” are established traditions. Science and pseudoscience have long been used in political, religious, and economic disputes. Yet the developments discussed above do, I think, reveal that a wider range of stakeholders and their legal advocates is discovering just how fertile the ground for science advocacy is. Although lawyers and law schools remain infamously nonquantitative and sometimes seem oblivious to the revolutions in biology, medicine, cybernetics, and physics (former law dean Paul Carrington has called law school applicants “refugees from science”), the few lawyers who do learn to apply their skills upstream to emerging scientific research are finding that they can achieve far more than they can downstream in regulatory and judicial proceedings that are too far advanced to allow the science to be revised. The Shelby Amendment, the attempt to apply FACA rules to the NRC, Daubert, sealed records, compelled disclosure, and orchestrated attacks on research all may have a profound effect on the science that is relied on in legislatures, agencies, and the courts.

Defining due process

For society the challenge is to determine how much and what kind of process is due as scientific and technological research becomes more important and more polarized. Despite some dark overtones, directing attention upstream toward the science that supports legislation, regulation, and toxic tort proof makes the greatest sense. Getting the scientific facts straight is the first step to sound policymaking and dispute resolution. The more open and reliable the science, the less able are government regulators, industry, nonprofit advocacy groups, and other stakeholders to obscure the facts and distort the data to serve their interests. Also, society has competing interests that legitimately infringe on researchers’ interests: for example, prompt and effective resolution of disputes, access to the facts that are relevant to dispute resolution, the right to defend one’s interests in court, enforcement of contracts, and confidentiality of business and personal information.

With these competing interests in mind, let us consider how a more balanced approach to just one of the due process issues raised above might be addressed: the Shelby Amendment. Senator Shelby surely cannot be wrong that the public and the democracy it sustains have a legitimate interest in access to scientific research produced with tax dollars. Yet, equally surely, there must be a better way to provide it than the two-sentence amendment that simply requires that “all data produced” be made available under FOIA. Although the OMB guidelines limit the mischief that the amendment could cause, a better approach would be to replace Shelby with a process that goes directly to the heart of the reason why the Shelby Amendment was adopted in the first place.

The Shelby Amendment had its origins in the fact that some federal regulations require enormous investment in compliance. Where those regulations rest directly on scientific findings, supporters believe that the Shelby Amendment will enable them to probe what scientists call the “robustness” of the findings before investing tens or hundreds of millions of dollars in compliance. Correspondingly, other stakeholders may want to reexamine agency findings that undercut the need for regulation.

The controversy that precipitated the Shelby Amendment also stimulated a response that could be a viable alternative to Shelby. When the companies demanding data were in a standoff with the Harvard researchers protecting the privacy of medical records, the Health Effects Institute (HEI) of Cambridge, Massachusetts, stepped forward to undertake an independent reanalysis to see whether the data supported the researchers’ conclusions on morbidity and mortality caused by air pollution.

The EPA’s multibillion-dollar particulates regulation raised significant scientific and policy issues that lay beyond traditional scientific debate. In the view of industry critics, ordinary peer review could not begin to provide the searching validation that they thought was needed. Significant replication issues existed, they maintained, that counterbalanced the researchers’ privacy concerns. Maintaining the confidentiality of medical records, the HEI undertook a complete replication of the study with the original data, allowing no variation in the original study design. HEI then applied an extensive but scientifically justified set of alternative models and techniques to the data to test the sensitivity of the original results to alternative means of analysis. The HEI conducted its analysis independently to ensure credibility, but it asked key stakeholders for early input into the scope of the effort. It also involved the original principal investigators in the reanalysis and gave them the opportunity to append their comments to the HEI reanalysis. HEI has conducted reanalyses in other contexts (such as mercury contamination in the Seychelles Islands and health risk analysis for diesel engine emissions), further validating its approach.

One might ask why Senator Shelby and others did not ask the NRC to form a committee to address the particulate standards controversy. Congress regularly calls on the NRC for broad, state-of-the-science consensus reports when controversies similar to the particulate controversy arise. Indeed, NRC panels have reviewed, and continue to review, air particulate science. In fact, the NRC model works well in most cases where the federal government needs further validation of the science relied on in national policymaking. But if a controversy focuses on the narrow issue of whether a certain data set supports the interpretation that researchers have given, NRC committees, which are composed of prominent scientists who volunteer their time, are unlikely to be available to undertake the arduous task of replicating specific study results.

Without an expansion of the duties of NRC committees, the HEI model for reanalysis of critical scientific studies has much to recommend it as an alternative to the Shelby approach. First, an independent body undertakes the analysis, avoiding the potential for stakeholder bias during interpretation of the results. Second, only controversial research that underpins costly regulations would justify the time and expense involved, thus thwarting those who would exploit the nuisance value of broad FOIA requests for “all data” without regard to need. Third, strict replication of results using the full data set (held confidential where warranted) rests on the very foundation of the scientific method: reproducible results. Fourth, involvement of the original researchers promotes efficiency, facilitates the nonadversarial exchange of views, and permits the original researchers to defend their conclusions in the final report.

The HEI model of independent validation of scientific studies where the stakes are unusually high affords a starting point for a much-needed reconsideration of the Shelby Amendment. The HEI model is sound, notwithstanding that some groups enthusiastically endorse Shelby as written and that the Chamber of Commerce will still seek production of the Harvard data via litigation challenging OMB’s Shelby guidelines.

If all involved agree to a reanalysis, and funding is available, then nothing further is required before work begins. However, if Congress adopted the HEI model in lieu of Shelby, then a trigger mechanism (perhaps a decision by a panel of scientists or a review by the Office of Science and Technology Policy) would be necessary to determine that the findings are important enough to warrant reanalysis. Others have suggested a cost threshold for federal regulations for which data may be sought under Shelby. Presumably, the petitioners who want a reanalysis done would be required to fund it, but federal funding may be required if the petitioners can show the need for a reanalysis but lack funds to have it carried out.

Solutions that balance competing interests and counter science advocacy tactics can be found for other emerging issues in scientific due process. Scientists would rather avoid having government provide these solutions. Governments do not legislate or regulate well where scientific research is concerned, as the current debate over restrictions on stem cell research shows. The HEI model, institutional review boards, peer review, NRC studies, and the work of the National Institutes of Health Recombinant DNA Advisory Committee (following on the worldwide voluntary moratorium on recombinant DNA experiments) are all good examples of flexible, nonintrusive mechanisms through which the scientific community has shown a remarkable ability to self-govern when scientists or the public think oversight is due.

Still, the general demand for scientific due process and the aggressive use of science advocacy tactics may have progressed to the point where scientists themselves need to speak out, before scientific research itself becomes an adversarial proceeding. President Bruce Albert’s address to the NAS annual meeting in May 2000 stressed the need for a more effective voice for the scientific community through “activist academies” worldwide. He had in mind activism to combat direct challenges to science, such as creationism. However, the developments discussed here, including the NRC’s experience with procedural due process and science advocacy, suggest that scientists may need to adopt a more assertive stance to ensure the integrity of scientific research. Scientists are reluctant advocates, but their failure to step forward may mean that the field will be ceded to lawyers, legislators, and judges.

At the risk of being criticized for prescribing the toxin as antidote, it may be time to explore the creation of a nonprofit group to defend the interests of scientific researchers when they are personally attacked or when measures are proposed that will significantly impair the conduct of scientific research. The purpose of such an organization would be to build on existing legal doctrine and use existing legal processes to develop a body of law protecting researchers. Protective statutes or regulations would not be sought until all other avenues were exhausted. Such an organization would function as a “science defense fund” and might undertake direct representation, intervention, friend-of-the-court briefs, and legal commentary, all toward defending scientific research when excessively broad demands for data are made, when attacks are made on the religious or personal views or integrity of individual scientists or institutions, when parties seek to bar public access to data needed for research or public health protection, and when other aggressive stakeholder tactics threaten science. Fair-minded scientists should found the organization and populate its board of directors. By this one step, scientists themselves could go far to ensure that science advocacy and creeping proceduralism do not overwhelm the research enterprise.