American College of Preventive Medicine
American Fisheries Society
American Institute of Biological Sciences
American Public Health Association
American Society of Agronomy
American Society of Limnology and Oceanography
Association of Teachers of Preventive Medicine
Crop Science Society of America
Ecological Society of America
Estuarine Research Federation
Institute of Food Technologists
Soil Science Society of America
Society for Conservation Biology |
Peer review is an integral component of scientific research and publishing. It
allows the scientific community to maintain quality control of research through
the review of research proposals, journal manuscripts and other reports. Academic
peer review, although far from perfect, is the best tool scientists have to ensure
high standards for their professional work.
This idea has been translated into the policy arena through ‘scientific
peer review’ – the review, by scientific experts, of in-house agency
science or the body of science underlying management decisions. These types
of reviews are critically important tools for policy makers. They allow experts
from both inside and outside the federal government to provide technical advice
and analysis, increasing public confidence in federal science, and ensuring
that the best quality information is used in decision making.
However, it is critical that scientific peer review programs be carefully designed
to maintain objectivity, quality and thoroughness. While scientific peer review
is an important tool for decision makers, a poorly designed process can do more
harm than good. It is for this reason that we endorse the following list of
important considerations for government scientific peer review of agency-produced
science and the body of science underlying management decisions.
- The first priority in choosing reviewers should be to engage the
most competent scientists. Therefore, conflict of interest exclusions
must be carefully designed to balance barring those with a direct conflict
of interest and the reality of a finite pool of suitable reviewers. The key
issue in selecting reviewers is whether they bring the necessary scientific
knowledge and objectivity to reviewing the matter at hand.
Scientific peer review should be insulated from politics as much
as possible. Oversight of scientific peer review should be vested
in scientists and science managers within the agencies. This adds assurance
that the composition of panels is not being unduly influenced by politics
and constitutes a representative subset of the scientists most competent to
review and assess the topic. The agencies must be trusted to perform the task
of constituting and overseeing fair and independent scientific peer review
efforts, without interference from political entities.
- Even the best scientific peer review cannot give policy makers
the ‘right’ answer. Scientific peer review can provide
assurances that rigorous, transparent and respected methods were followed,
that the data were reasonably interpreted, and that the stated conclusions
logically follow from the results. However, often more than one interpretation
of the data set can be made, and there may be no way to determine which interpretation
is ‘best’. Where data are limited or other uncertainties abound,
scientific peer review can point these problems out, but it cannot overcome
them.
- Scientific peer review must maintain programmatic flexibility.
While guidelines can help to ensure that certain standards are met and maintained,
an overly rigid process, particularly for scientific peer review of the body
of science underlying policy decisions, will result in inefficient use of
time and resources. It may be overly prescriptive to stipulate the number
of reviewers, the questions they must answer, or the type of report they must
produce for the broad range of agency scientific work.
- All scientific peer review must be based upon an assumption of
integrity. While commonsense measures can be taken to weed out direct
conflicts of interest, an implementable system can never be fully cleared
of all potential conflicts of interest. Instead, fair reviews are the product
of professional standards of conduct that are a fundamental component of training
in scientific research. Scientific peer review must ultimately rest on the
presumed integrity of the reviewers.
- Efforts to revise the process of peer review should acknowledge
the differences in professional culture that often divide scientists, policy
makers, and the public. The academic model of peer review calls on
reviewers to be as critical as possible. This is done so that authors are
able to make improvements where they can and so that the weaknesses of the
work are understood and acknowledged. Thus, results from scientific peer review
that highlight uncertainties, questions and alternative explanations do not
mean that the science was not well done or that its findings are invalid.
Science is inherently uncertain and there will always be unanswered questions
and areas where more research is needed. However, acknowledging uncertainty
should not be equated with an inability to draw conclusions; managers often
must act without complete certainty. Scientific peer review, properly carried
out by competent peer scientists, can reassure managers, decision makers,
and the public that such difficult decisions are based on research that represents
the current state of our scientific understanding.
Last updated: October 14, 2004
|