Dr. Margo Schwab
Office of Information and Regulatory Affairs
Office of Management and Budget
725 17th Street, NW
New Executive Office Building
Room 10201
Washington, DC 20503

December 15, 2003

Dear Dr. Schwab;

Thank you for the opportunity to submit comments on the Proposed Bulletin under Executive Order 12866 and supplemental information quality guidelines, titled “Peer Review and Information Quality”. The Ecological Society of America (ESA) is the nation’s premier society of professional ecologists with over 8,000 members in academia, private industry and government agencies. Our members have been and will be very active in executing peer review of government studies pertaining to the ecological sciences.

We commend the Office of Management and Budget’s (OMB) efforts to increase scientific accuracy and accountability of federal agency work. Peer review is an important tool of the scientific community and can be invaluable in maintaining high quality methods and results. However, we do have concerns with OMB’s Proposed Guidelines.

First, a stringent peer review, particularly outside peer review, is not appropriate for all scientific information that is relevant to regulatory policies. The proposed guidelines sets a uniform standard for reviewing such documents, when, in fact a more nuanced approach is called for. For example, the Environmental Protection Agency (EPA) currently utilizes a three-tiered policy where information is reviewed by an outside panel, a standing advisory panel, or by letters submitted by outside experts depending on the complexity and importance of the information. OMB’s Proposed Guidelines appear to disallow such a cost and time saving approach, calling for identical procedures in all cases. Also, technical work produced by an agency is often produced according to established procedures or models that have already been thoroughly vetted. Such work does not merit the time and expense of a stringent outside peer and can be reviewed internally. While we certainly support the review of major technical work produced by an agency, we ask OMB to refine the definition of work warranting outside peer review and allow for differing processes relative to the importance and complexity of the work in question.

Second, guidelines pertaining to the selection of peer reviewers raise serious concerns. OMB Proposed Guidelines would exclude a scientist who “(i) has any financial interests in the matter at issue; (ii) has, in recent years, advocated a position on the specific matter at issue; (iii) is currently receiving or seeking substantial funding from the agency through a contract or research grant (either directly or indirectly through another entity, such as a university; or (iv) has conducted multiple peer reviews for the same agency in recent years…”. These guidelines lead us to believe that the most qualified reviewers would be excluded from consideration.

Scientific inquiry is a public good and as such is heavily reliant on federal funding. ESA research has shown that fully 35% of federal funding for ecological research comes from regulatory agencies. Thus, a significant proportion of ecologists receive funding from agencies covered under these Proposed Guidelines. It also stands to reason that scientists with expertise on the subject the agency would like reviewed, would be particularly likely to have received, be receiving or have reason to expect future funding from the agency in question. By excluding these researchers, OMB would exclude the very people most likely to provide a useful review to the agency.

Furthermore, agencies do have firewalls between their regulatory and grant awarding divisions that eliminate any incentive for a scientist to seek to please EPA regulators in order to improve his/her chances of receiving a grant. For example, at EPA, one program, the Experimental Program to Stimulate Competitive Research (EPSCOR), gives grants to proposals selected after an outside peer review process. Thus, while the money comes from the agency, the agency does not choose the awardees. Excluding scientists who receive such funding is a particularly onerous requirement of the Proposed Guidelines.

OMB’s requirement that a reviewer must not have expressed an opinion on the specific matter at issue raises additional concerns. Interpreted narrowly, such a requirement would exclude any scientist who had done work and reached conclusions on the topic in question. It is difficult to imagine a scientist well established and broadly published in a particular field who does not have an opinion on most topics in his/her field. This requirement should be tightened to ensure that such a narrow interpretation is not applied.

The guidelines would allow ‘biased’ reviewers to be appointed should that be necessary to have appropriate expertise on the panel. However, it stipulates that in this event, “another reviewer with a contrary bias is appointed to balance the panel”. It is puzzling to consider whom such an individual would be for many of the ‘biases’ outlined above. If a reviewer were considered biased because she received a grant from the agency, would a reviewer with a ‘contrary bias’ be someone whose grant proposal was rejected? This requirement is nonsensical in light of the extremely broad bias definition in the Proposed Guidelines.

Finally, granting peer review oversight to OMB and the Office of Technology Policy (OSTP) is alarming and would destroy any gains in reducing the appearance of bias from the peer reviewer selection guidelines. As extensions of the Executive Office of the President, OMB and OSTP are fundamentally political entities. By granting oversight of agency peer review to OMB and OSTP, the Proposed Guidelines allow for political interference in what should be purely scientific decisions. It is critical that barriers between federal science and politics remain in place. These guidelines appear to weaken that vital divide.

We hope that you will take these comments into consideration as you finalize these guidelines. Peer review is an important component of federal science and ESA is committed to ensuring that the best possible procedures are in place.

Sincerely,

William H. Schlesinger
President

+++++++++++++++++++++++++++++++++++++++
The Ecological Society of America is the country's primary professional organization of ecologists, representing over 8,000 scientists in the United States and around the world. Since its founding in 1915, ESA has diligently pursued the promotion of the responsible application of ecological principles to the solution of environmental problems through ESA reports, journals, research, and expert testimony to Congress. For more information about the Society and its activities, visit the ESA website at http://www.esa.org.