Science and engineering provide more than the ideas for future products or the foundation for advances in manufacturing. They also provide the basis for making decisions as a society, as corporations and as individuals. While these decisions certainly affect important national, and even global issues, they also affect elements of our lives as basic as how we live and what we eat. For example, we turn to scientists and engineers for answers to questions such as "To what standards should cities' building codes be written?" Engineers, seismologists, geologists and materials scientists may all need to be consulted. Or, "Is the food on the dinner table safe to eat?" "Is a new drug ready for use by humans?" Epidemiologists, microbiologists and pharmacologists, among many others, must inform us.
Though many of these decisions affect our everyday lives, we tend to consider them only when there is a crisis: when buildings collapse in an earthquake, whenE. coli in hamburgers kills children, when drugs cause dangerous side effects. While every individual must exercise his or her own judgment in making decisions--and be willing to accept responsibility for doing so--we nevertheless must of necessity rely on decisions made by our elected officials, regulators, and the courts for decisions that affect our society. When the decisions to be made involve technical issues, decision-makers must have access to and, to a large extent rely on, the advice and counsel of the scientific and engineering community.
Science can inform issues, but it cannot decide them. For example, scientists have told us that the New Madrid faultline in the Eastern U.S. will give rise, on average, to a magnitude 6.0 or greater earthquake every seventy to ninety years. But they cannot tell us whether states in this region of infrequent earthquakes should employ the same building codes as California does. Similarly, some research indicates that the use of fertilizers may have long-term effects on nearby bodies of water due to runoff. But science cannot tell us how we should balance the interests of the farmers who use the land and the fishers who depend on the water, or the interests of the customers who buy and consume the products of both.
To further complicate matters, in many cases science simply does not have all of the answers. This is likely to be true particularly when the issue involves very complex systems, as is often the case with environmental questions--a forest, lake or other ecosystem cannot be put in a test tube for experimentation. Conclusions drawn by scientists in these instances carry varying degrees of uncertainty, and different scientists may derive very different inferences from the available data.
It is at this point that legal and policy decisions become most difficult. Those on both sides of the issue level charges that the other side is doing "bad" science. Each side produces its own contingent of scientists who in turn put forth conflicting interpretations of the available data, if they even agree on that. Accusations are made that the other side's scientists "have an agenda" or are beholden to a particular stakeholder in the issue.
In fact, disagreements among scientists are nothing new; they are actually an integral part of the scientific process, and the means by which old hypotheses or theories are discarded and new ones accepted. The difference is that these disputes among scientists typically take place in the pages of scientific journals or in the presentation halls at scientific meetings, and not on the floors of Congress, in the Courts, or on the editorial pages of newspapers.
The emergence of environmental threats over the last half century has elevated environmental issues to a position of importance ranking alongside the need to protect our national security, improve peoples' health, and strengthen our economy. Mr. McGroddy acknowledged this in his testimony when he stated, "I know of no serious student of history who would today substantively revise [Vannevar] Bush's rationale or conclusions in any major way, other than perhaps to add a fourth area of impact, the improvement of our management of our environment." Properly managing our natural resources, ensuring clean air and clean water for every citizen, and preserving the planet for future generations are concerns shared by every American. The decisions that must be made in order to tackle these issues, however, are at times highly contentious. It is imperative that we focus scientific resources on questions relating to the environment if we are to make informed future decisions in this arena.
Uncertainty and debate may be implicit in the scientific process, but a lack of a clear scientific consensus on an important policy issue makes matters more difficult for decision-makers. However, there are steps that can be taken to better inform the scientific and technical decisions made by regulators, legislators and the courts.
Science...warns me to be careful how I adopt a view which jumps with my preconceptions, and to require stronger evidence for such a belief than for one to which I was previously hostile. My business is to teach my aspirations to conform themselves to fact, not to try and make facts harmonize with my aspirations.
Thomas Henry Huxley | Because there is no more contentious
technology-based decision than one that is based on incomplete scientific data,
we must commit sufficient resources at the federal and state levels to finding
answers to scientific questions that promise to lie at the heart of future policy
decisions. By committing resources early in the process, we decrease the
likelihood that unsound decisions--decisions that end up costing far more down
the road--are made. Whenever possible, research must precede policy, not the
other way around.
As Dr. Roger McClellan, President and CEO of the Chemical Industry Institute of Toxicology, said in his testimony before the Committee, "Good decisions to protect and promote human health require sound scientific information. The development of sound scientific information requires time and resources to conduct research that is targeted to resolve issues. As simple as these statements are, all too often in the past they have not been heeded. The result has been that many past regulatory decisions have been undergirded by very uncertain science leading to decisions that are highly contentious."47 |
Research on a particular subject should not come to a stop once a policy decision has been made, an issue Dr. McClellan addressed in his testimony. "A mentality develops that we've set the standard, there isn't any need for any [more] research. Research [funding] goes down and then, about two years before the next review of the criteria document, there's a sudden realization [that] we've got to get more science...You have to have the time to create the science that's needed for credible decisions and [supply] the resources," he stated.
Applying forethought to funding decisions regarding research agendas that address areas of regulatory policy will likely come with some controversy, as decisions regarding the allocation of limited resources always are. However, making these difficult decisions before the regulatory process has gained unalterable momentum offers the opportunity to address complex questions, such as environmental issues, in a less highly charged atmosphere than that which exists when implementation of regulations precedes scientific consensus as to the nature--or existence--of a problem. Regardless of a policymaker's or regulator's views on an issue, it should go without saying that each ought to agree that more conclusive evidence on a controversial subject should be sought.
For science to play a meaningful role in legal and policy decisions, the scientists performing the research needed to answer questions posed by policy or law must be seen as honest brokers with the proper expertise to render advice. One simple but important step in facilitating an atmosphere of trust between the scientific and the legal and regulatory communities is for scientists and engineers to engage in open disclosure regarding their professional background, affiliations and their means of support.
Disclosure should not be used as a way to exclude particular scientists simply on the basis of their affiliations, as has happened in past debates. Rather, it should allow for broader participation and shift the focus of the debate to the science itself. In addressing this subject, Dr. McClellan expressed concern that, "we sometimes move and exclude individuals who are employed in the private sector from participation in certain deliberations as panel members because of their employment...We need to go beyond that [and] look at the credentials of the individual...their training, their experience...their publications in the peer reviewed arena, how have they interacted with their fellow scientists."
It is not permitted to the most equitable of men to be a judge in his own cause.
Blaise Pascal | That the scientific opinions these experts offer is seen as sound, credible and objective by those who rely on it depends on far more than the establishment of the scientist's credentials. It depends on the ability of the science itself to stand up to challenges from other experts. In the scientific community, a scientist's work is judged to be sound when it passes judgment upon critical review and testing by other scientists who work in the same field or are otherwise familiar with the subject matter being investigated. |
The first step in this process, peer review to determine whether a scientist's results should be published, imposes a strict standard for initial acceptance by the scientific community. Upon submission of an article describing a new scientific result and any conclusions regarding it, the paper is given to a small group of other scientists who are familiar with the subject matter and have been selected by the journal's editor for anonymous review. Only if the article, its data and conclusions pass muster with this group is the article accepted for publication in any respected (peer-reviewed) scientific journal. Papers that have not been subjected to the peer review process are likely to be viewed with some degree of skepticism by other scientists. Note that peer review applies to more than just publication. Hiring and tenure decisions often rely on peer review, and the grant application process is wholly dependent on it.
Because the peer review process is critical in bringing about acceptance of new scientific results and encouraging discussion among scientists, expanding the peer review process to include the science and science-based decisions made in federal agencies will help improve the credibility of the science conducted or supported by these agencies. Regulations should not be made on the basis of science that does not stand up to the rigors of the peer review process.
All we know is still infinitely less than all that still remains unknown.
William Harvey | Peer review for publication is only the first step in the acceptance of a scientific theory or conclusion. Publication of the new results, and the scientists' conclusions or theories based on those results, constitute the beginning, not the end, of the scientific process and the search for understanding. Publication allows other scientists to compare their own results with those of the published researchers, and to attempt to replicate the results of others--both extremely important steps in validating new discoveries or theories. |
The initial peer review process does not result in a stamp of scientific certainty, as there is often still disagreement over published conclusions or even over the data the conclusions were based on. These disagreements do not necessarily indicate that bad or sloppy science was done. Instead, the scientific debates that these disputes stimulate often lead to further clarification or advances in the field and therefore are an integral part of the process of science. As Dr. Lindley Darden of the University of Maryland wrote in an article entitled The Nature of Scientific Inquiry, "Publishing a plausible hypothesis plays the important role of placing it in the marketplace of scientific ideas...Individual scientists consider [alternative explanations] prior to publishing and choose the one that is best supported by the evidence they have at the time. Publication then allows the wider scientific community to continue the same process."48
As Dr. Darden points out, even the best scientists can be wrong on a particular point. For example, Enrico Fermi, Linus Pauling and Francis Crick--three of the most important scientists of this century (all of whom won Nobel prizes)--have all, at one time or another, published theories that later turned out not to be correct. In doing so, however, these scientists did anything but a disservice to their disciplines. Rather, their erroneous conclusions served to drive the science in those fields to a completely new level as other scientists tested--and subsequently rejected--their theories. Uncertainty is a fundamental aspect of the scientific process, and this is particularly true in rapidly developing fields of study.
Dr. Dennis Barnes, President of the Southeastern Universities Research Association, summed up the importance of the constant evolution of thought in the process of scientific discovery when he paraphrased the Austrian zoologist Konrad Lorenz in his testimony: "It is a good morning exercise for a research scientist to discard a pet hypothesis every day before breakfast. It keeps him young." Dr. Barnes then commented, "[Lorenz] explained perfectly the nature of scientific inquiry: constant examination and re-evaluation, a never-ending process of correcting errors and pushing back the frontiers of knowledge."49
Independent replication of scientific results and the attention scientific debate stimulated by uncertainty brings to a particular issue mean that scientific results do not remain forever in limbo. Eventually, scientists generate enough new data that they are able to shed light on previously uncertain findings. Still, this constant progress and initial uncertainty in the scientific process has repercussions for the policy process, which should not remain static in the face of changing scientific understanding.
...even the best scientists can be wrong on a particular point...Uncertainty is a fundamental aspect of the scientific process, and this is particularly true in rapidly developing fields of study | A particularly ominous threat to scientific freedom that would undermine the entire scientific enterprise are lawsuits brought against researchers and universities claiming damages because a researcher did not pursue a particular line of inquiry or published results that were later found to be in error. Researchers must be free to exercise their scientific judgment about which research paths to pursue without worrying that, if at some later date their hypotheses or conclusions turn out to be incorrect, they may be sued. This would have a chilling effect on scientific research, as Dr. Barnes made clear in his testimony. He said, "I think that there is no doubt about the inhibiting influence [of lawsuits] on the free performance of research...Researchers will be more cautious about making bold hypotheses, universities will need to scrutinize more carefully the research conducted by their faculty, and the cost of defending against such suits, whether or not they are actually pursued, will be another overhead burden on research." |
To resolve effectively the problems regulatory agencies seek to address, regulatory decisions must not only be based upon a sound technical foundation, they must also make sense from a practical standpoint. The importance of risk assessment--the process of identifying and quantifying potential risks and of making decisions about how to deal with these risks through comparing various options and potential outcomes--has too often been overlooked in making policy. We must accept as a society that we cannot reduce every risk in our lives to zero, and should instead determine where to deploy our limited resources to greatest societal effect.
As Dr. John Graham, Founding Director of the Harvard Center for Risk Analysis said in his testimony, "The science of risk analysis can help regulatory organizations make better decisions...the failure to perform sound risk analysis can lead to poor decisions that can harm public health and safety."50
Dr. Graham went on to described an exercise one of his graduate students had undertaken to demonstrate the application of risk assessment to regulatory policy across the U.S. government. "[She] estimated that...she could save 60,000 more lives per year than we're currently saving at no increased cost to either taxpayers or the private sector, simply by reallocation. There are enormous opportunities for reallocation of resources to save more lives." Clearly, ignoring the broader picture in making specific policy decisions is not in the public's overall best interests.
All three branches of our democratic system are faced with decisions that depend on science; the judicial system is no exception. Supreme Court Justice Stephen Breyer recently stated in a speech at the American Association for the Advancement of Science's (AAAS) 1998 meeting that the law "increasingly requires access to sound science...because society is becoming more dependent for its well-being on scientifically complex technology." 51 Indeed, whether it is new advances in forensic technology, such as DNA 'fingerprinting' in criminal proceedings, or questions of cause and effect in civil cases, such as those involving breast implants, science and technology play an important role in the courtroom.
The scientific discourse in a trial is usually highly contentious. As Dr. Mark Frankel, Director of the Scientific Freedom, Responsibility, and Law Program at the American Association for the Advancement of Science, said in his written testimony, "The primary way that we educate judges and juries on complex scientific matters is through the use of expert witnesses, almost always retained by the parties to the litigation, airing their differences in an adversarial setting. Serious reservations have been expressed about this approach, however, some by judges themselves...what often occurs is that experts from both parties are pitted against one another, with lawyers on each side trying to destroy the credibility of the other party's witness. Such tactics are not likely to enlighten either judges or juries about the validity of a scientific methodology or of the conclusions drawn from disparate data." 52
In a landmark 1993 decision that could change this focus from undermining the credibility of the other side's scientific expert to the merits of the science itself, the Supreme Court ruled in a civil case, Daubert v. Merrell Dow Pharmaceuticals, Inc., 53 that "federal judges must act as gatekeepers in order to exclude unreliable evidence from the courtroom" according to Dr. Frankel's testimony.
With the possibility that, in accordance with the Daubert decision, increasing numbers of judges will avail themselves of independent, qualified scientists to assist them in addressing complex scientific and technical questions, the identification of these experts promises to be an increasingly important step in the judicial process. In his testimony, Dr. Frankel described a demonstration project the AAAS hopes to implement that would aid judges in this process: "On receiving a request from the court, the project will seek to clarify the specific technical issue on which the expert is expected to advise and what role the expert will play in the litigation. With the assistance of a Scientific Selection Panel and at the end of a rigorous search, the project will provide the court with a slate of at least three possible experts. In some cases, the court may decide to appoint a panel of experts, in other instances, a single expert will suffice." Dr. Frankel also stated that Justice Breyer had specifically endorsed the AAAS demonstration project.
Decisions about science policy are made in a large number of Congressional committees and subcommittees, which can impede the progress and coordination of important projects. In his testimony, Admiral Watkins gave voice to his frustration in dealing with 9 federal agencies and 47 Congressional committees and subcommittees in his work on oceanographic projects.
Having to answer to so many different committees and agencies is an understandable outgrowth of the extent to which science and engineering touches almost all aspects of our lives, but it clearly makes it much more difficult to effectively manage complex technical programs. While it might at some point in the future make sense to consider lessening the number of committees and agencies with significant influence over large, complex technical programs, at a minimum Congress and the Executive Branch should improve their internal coordination processes to more effectively manage, execute, and integrate oversight over these kinds of programs. While the Office of Management and Budget can fill this role in the Executive Branch, no such mechanism exists in the Congress.