A Guide to TWS Peer Reviews

Periodically TWS is contracted by outside entities to conduct independent and scientific peer reviews of specific reports, white papers, peer reviewed literature, or policies. These reviews are conducted by TWS’ Government Affairs (GA) Team, facilitated by the GA Associate. Although TWS is the facilitator and contracting entity for these reviews, the final product (which may be independent comments, a summary document, or a single detailed review reflecting a coalescence of all reviewers’ comments) does not reflect the views of the Society, nor should it be viewed as the official position of the Society. In addition, the comments generated by the review teams do not necessarily represent the views or positions of each reviewer’s affiliated organization or employer.

Peer review is an integral component of scientific research and publishing and an important means of assuring sound information. It allows the scientific community to maintain quality control of research through the review of research proposals, journal manuscripts, synthesis or summary reports and other documents. Peer review is one of the best tool scientists have to ensure high standards for their professional work. TWS’ role in the peer review process enables our members, as experts in the field, to have a voice supporting scientific integrity. By proving a mechanism to facilitate peer reviews, TWS is also able to meet objectives identified within the strategic plan - facilitating member involvement to increase TWS’ impact on wildlife and habitat conservation worldwide.

The TWS Peer Review process consists of four main phases:

Contracting and/or bidding,
Reviewer solicitation,
Review facilitation, and
Review production and submission.

Every peer review is different, depending on the items under review and the areas of expertise required. However, in an effort to produce effective, consistent, and independent reviews, standardized templates and processes have been established.

Contracting and/or Bidding

In response to a scope of work, TWS produces a consistent peer review proposal that identifies the following:

The item(s) to be reviewed and the number of individuals required for the review team
Past peer reviews facilitated by TWS staff (a few to serve as examples)
Agencies, organizations, or other groups of individuals who may have a conflict of interest (usually identified by the contracting entity in the scope of work)
Collective areas of expertise (identified by contracting entity in the scope of work and by TWS staff)
Questions or issues for reviewers to consider and respond to (identified by contracting entity in the scope of work and by TWS staff)
Relevant staff expertise qualifying TWS to facilitate the review
Task list and schedule for completion
Deliverables
Budget – broken down by task

In addition to the standard Peer Review proposal, resumes/curriculum vitas for the Executive Director, GA Director, GA Assistant Director, and GA Associate are also included.

Reviewer Solicitation

TWS uses the U.S. National Academy of Sciences Policy on Committee Composition and Balance and Conflicts of Interest to identify potential conflicts and biases on the part of potential reviewers. In addition, the contracting entity may identify individuals, agencies, or organizations that should be excluded from participating on the review panel due to potential or foreseeable conflicts of interest. The solicitation process takes advantage of the Society’s reach into the natural resources community and relies on the knowledge of our membership to identify (or self identify) potential qualified individuals. The process has three parts:

Outreach to Council, Section, Chapter, and Working Group leadership to identify potential individuals with relevant expertise in a given category.
Outreach to identified individuals to gauge interest and ability to participate, and to request pertinent information to evaluate relevant expertise (cover letter identifying relevancy to the peer review, resume and/or curriculum vitae, and other documents as needed).
Evaluation of expertise and selection of the peer review team.

A standard solicitation announcement to recommended individuals identifies:

The entity or individual who recommended them for the review,
The review in question,
The collective areas of expertise,
The time commitment and expectations, and
The honoraria a reviewer will receive upon completion.

A reviewer need not be an expert in all areas identified in the peer review proposal, but collectively all reviewers should satisfy the areas of expertise desired by the contracting entity.

Reviewers and their comments may or may not be anonymous (dependent on the scope of work). However, all reviewers will remain anonymous to outside individuals or entities until the submission of the final product to limit interference.

Review Facilitation

Review facilitation is dependent on the level of staff involvement described in the scope of work. The level of involvement is generally reflective of the type of final product requested by a contracting entity. Currently, TWS peer reviews fall into three distinct product categories:

Category 1: Independent reviews paired with a simple cover letter and high level summary by TWS staff.

Category 1 reviews require the least amount of staff time. Each reviewer receives the necessary materials, conducts their review independently of other reviewers, and submits a final set of comments to TWS staff. Staff format and edit comments for punctuation and grammar and prepare a simple summary to tie the package together. This type of review traditionally does not require coordinating conference calls or in-person meetings.

Category 2: Independent reviews paired with a complete, in-depth summary document by TWS staff.

Category 2 reviews requires very little staff time on the front end, but substantial time on the back end as the staffer is responsible for not only the steps described in Category 1, but also coalescing and summarizing both high level and detailed elements across all independent reviews. Category 2 reviews may require coordinating conference calls with the review team.

Category 3: A single document capturing a single review with input from all reviewers.

Category 3 reviews are the most time intensive for staff and the review team. For these reviews, a “Team Lead” must be identified among the review team. Each member of the review team is expected to review the document(s) in question independently, but the Team Lead is responsible for gathering reviewer comments and creating a single document that integrates all reviews. Due to the complexity of this process, multiple conference calls are required to ensure the diversity of opinions is equally represented. Staff are responsible for coordinating and facilitating conference calls, working with the Team Lead to craft a final product, and then formatting, editing, and shaping the final product to meet TWS Peer Review formatting guidelines for submission to the contracting entity.

Facilitation of a review begins with the distribution of the formal confirmation letter to reviewers. This letter identifies:

Document(s) to be reviewed,
Pertinent background relating to the document(s) and uses,
Alist of questions or issues to be addressed,
Review-specific instructions,
Structure of the final product,
Desired format and structure for comments,
Deadlines and timelines,
Details regarding honoraria, and
Other information as needed.

Interested individuals who were not selected for the review process receive a formal thank you letter declining their assistance.

After the review is completed, TWS staff are responsible for coordinating payments from the contracting entity and distributing honoraria accordingly to appropriate reviewers along with a thank you letter from TWS recognizing their involvement in the peer review process.

Final Product Production and Submission

Depending on the review category, final products vary. Regardless, TWS staff are responsible for the final layout, copy editing, formatting, and submission of all final products. Staff use a standard summary template to ensure some level of consistency across a diversity of products.

Prior to submission, all products are reviewed internally by pertinent members of senior staff and approved by the Executive Director.

Final products are submitted to the contracting entity along with an invoice for services no later than close of business of the contract deadline. Once released to the contracting entity, the product is the property of the contracting entity and can be distributed as such.

Approved by Council: November 2011