[Federal Register Volume 89, Number 165 (Monday, August 26, 2024)]
[Notices]
[Pages 68473-68475]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2024-19041]


=======================================================================
-----------------------------------------------------------------------

NATIONAL SCIENCE FOUNDATION


Request for Information (RFI) on National Science Board-National 
Science Foundation Merit Review Commission Review of NSF's Merit Review 
Policy and Processes

AGENCY: National Science Foundation

ACTION: Request for information.

-----------------------------------------------------------------------

SUMMARY: The National Science Board-National Science Foundation 
Commission on Merit Review (MRX) is issuing this Request for 
Information (RFI) to seek input from interested individuals and parties 
to inform the MRX's review of NSF's Merit Review criteria, policy and 
processes.
    Information on the MRX is available at https://www.nsf.gov/nsb/committees/mrxcmte.jsp.

DATES: Interested individuals and parties are invited to submit 
responses to this Request for Information on or before 5:00 p.m. 
Eastern time on Wednesday, September 18, 2024.

ADDRESSES: 
    Online: Respond to this RFI at the following url: https://nsfevaluation.gov1.qualtrics.com/jfe/form/SV_6xOeZ04jar2xmhU.
    Following this link allows you to access an online form where you 
can provide input on up to six topics described in more detail in the 
Supplementary Information section below. You are encouraged to respond 
to only those that are of interest to you. You may, but are not 
required to, provide input on each topic to submit your response.
    Mail: Attn: Portia Flowers, 2415 Eisenhower Avenue, Alexandria, VA, 
22314, USA.

FOR FURTHER INFORMATION CONTACT: Portia Flowers (703/292-7000, 
[email protected]).

SUPPLEMENTARY INFORMATION: Please refer to definitions provided at the 
end of this notice for terms used in these Information Requests.

Information Requests

    1. MRX is interested in identifying opportunities to improve NSF's 
current Merit Review criteria, policy, and processes. Importantly, this 
includes documenting and understanding any areas of misunderstanding, 
gaps, or lack of clarity regarding (a) the three Merit Review 
Principles which are the foundations of the Merit Review Process, (b) 
the two statutory Merit Review Criteria which are used to evaluate all 
proposals to NSF, and (c) the five Merit Review Elements NSF uses to 
assess each criterion. Are the Principles, Criteria, and Elements 
clear? Could they be improved upon? The MRX welcomes feedback on any or 
all of these, and particularly on the Broader Impacts Criterion. 
Chapter 3 of NSF's Proposal & Award Policies and Procedures Guide 
(PAPPG) defines terms in this Information Request. See https://new.nsf.gov/policies/pappg/24-1/ch-3-proposal-processing-review#a-merit-review-principles-and-criteria-af2.
    Individuals responding to this request are encouraged to indicate 
whether their perspectives are informed by experience(s) preparing and/
or reviewing proposals to NSF.
    2. NSF strives to conduct a fair, competitive, transparent Merit 
Review process for the selection of projects. To accomplish this, NSF 
relies on a process that considers both the technical aspects of a 
proposed project and its potential to contribute more broadly to 
advancing NSF's mission using the statutory Intellectual Merit and 
Broader Impacts Merit Review criteria. MRX invites suggestions on the 
implementation of the Merit Review criteria. We especially invite 
feedback that would (a) clarify how they can be used in preparing and 
reviewing proposals, (b) ensure proposals, reviews, and funding 
decisions demonstrate full consideration of both criteria while 
maintaining openness to the full spectrum of potential activities under 
each, and (c) better recognize and support potentially transformative 
and high-risk/high-reward activities.
    Individuals responding to this request are encouraged to indicate 
whether their perspectives are informed by experience(s) preparing and/
or reviewing proposals to NSF.
    3. MRX is interested in the experiences and perspectives of those 
who have considered submitting and/or submitted proposals in the past. 
We invite you to share your insights and describe any opportunities you 
believe would improve implementation of the Merit Review criteria, 
policy, and processes based on your experience as a proposer or 
investigator. This includes any experiences that may have encouraged or 
dissuaded you from submitting proposals to NSF. We are especially 
interested in learning (a) how NSF guidance (e.g., as provided in the 
NSF PAPPG, program solicitations, or other funding opportunity 
announcements), may have played a part in your decision(s) whether to 
submit proposals, and (b) how NSF might best support investigators 
interested in submitting a proposal to NSF.
    Individuals responding to this request are encouraged to indicate 
whether they

[[Page 68474]]

submitted or decided not to submit a proposal, and whether these 
experiences occurred within the past five years.
    4. MRX is interested in the experiences and perspectives of those 
who have reviewed proposals submitted to NSF. We invite you to share 
your insights and describe any opportunities you believe would improve 
implementation of the Merit Review criteria, policy, and processes 
based on your experience reviewing NSF proposals.
    Individuals responding to this request are encouraged to indicate 
whether they served on a panel and/or as ad hoc reviewers, and whether 
these experiences occurred within the past five years.
    5. MRX is interested in exploring how NSF could better support 
awardees in demonstrating and documenting outcomes of their awards in 
advancing knowledge (Intellectual Merit) and benefiting society and 
contributing to the achievement of specific broader or societal 
outcomes (Broader Impacts). We invite you to share your insights on how 
NSF might better support awardees in demonstrating and documenting 
outcomes of their awards without unnecessarily increasing awardees' 
administrative burden of reporting.
    Individuals responding to this request are encouraged to indicate 
whether their suggestions are based on experiences as investigators, 
users of public outcomes reports, or another perspective.
    6. MRX welcomes any other comments on or suggestions for improving 
NSF's current Merit Review criteria, policy, and processes. It also 
welcomes information about aspects of Merit Review criteria, policy and 
processes that are currently working well.

What will NSF do with this information?

    MRX will use the information submitted in response to this RFI to 
inform its assessment of the efficacy of the current Merit Review 
criteria, policy, and processes, and to draft recommendations regarding 
them. The information provided will be analyzed and considered by MRX. 
Respondents are advised that the government is under no obligation to 
acknowledge receipt of the information or provide feedback to 
respondents with respect to any information submitted. No proprietary, 
classified, confidential, or sensitive information should be included 
in your response submission. The government reserves the right to use 
any non-proprietary technical information in any resultant 
solicitation(s), policies, or procedures. All submitted information may 
be subject to disclosure under the Freedom of Information Act (FOIA) or 
other applicable law.
    This Notice does not invite research proposals nor is it a funding 
opportunity.

Background

    NSB and NSF, with the assistance of expert third parties, have 
periodically re-examined and revised the criteria, policy, and 
processes of Merit Review at NSF. The last time the Board 
systematically examined the Merit Review criteria was in 2010-2011 when 
NSB established a Task Force on Merit Review to examine the 
Intellectual Merit and Broader Impacts merit review criteria and their 
effectiveness in achieving NSF's goals in support of science and 
engineering research and education. At that time, Congress was 
considering, and then passed, the America COMPETES Reauthorization Act 
directing NSF to apply the Broader Impacts criterion to achieve a 
specific array of societal goals and charging NSF to develop policies 
addressing it. The 2011 Task Force report concluded that the Merit 
Review criteria remained appropriate for evaluating NSF proposals; 
however, it provided certain revisions and clarifications.
    Recent events have underscored the importance of demonstrating that 
portfolios of funded projects enable NSF to meet its statutory mission 
``to promote the progress of science; to advance the national health, 
prosperity and welfare; to secure the national defense; and for other 
purposes.'' In 2022, Congress passed the CHIPS and Science Act, which 
directed federal research agencies to regularly assess, and update as 
necessary, policies, and practices to remove or reduce cultural and 
institutional barriers limiting the recruitment, retention, and success 
of groups historically underrepresented in STEM research careers, 
including policies and practices relevant to the unbiased review of 
Federal research applications. Reexamining the Merit Review policy and 
process will help ensure that NSF is best placed to meet the 
requirements set out by Congress.

Definitions for Terms Used in This RFI

Merit Review Policy

Principles

    1. All NSF projects should be of the highest quality and have the 
potential to advance, if not transform, the frontiers of knowledge.
    2. NSF projects, in the aggregate, should contribute more broadly 
to achieving societal goals. These broader impacts may be accomplished 
through the research itself, through activities that are directly 
related to specific research projects, or through activities that are 
supported by, but are complementary to, the project. The project 
activities may be based on previously established and/or innovative 
methods and approaches, but in either case must be well justified.
    3. Meaningful assessment and evaluation of NSF funded projects 
should be based on appropriate metrics, keeping in mind the likely 
correlation between the effect of broader impacts and the resources 
provided to implement projects. If the size of the activity is limited, 
evaluation of that activity in isolation is not likely to be 
meaningful. Thus, assessing the effectiveness of these activities may 
best be done at a higher, more aggregated, level than the individual 
project.

Criteria

    Both criteria are to be given full consideration during the review 
and decision-making processes; each criterion is necessary but neither, 
by itself, is sufficient. Therefore, proposers must fully address both 
criteria.
     Intellectual Merit (IM): the potential for a proposed 
project to advance knowledge.
     Broader Impacts (BI): the potential for a proposed project 
to benefit society and contribute to the achievement of specific, 
desired societal outcomes.

Elements

    1. What is the potential for the proposed activity to:
    a. Advance knowledge and understanding within its own field or 
across different fields (Intellectual Merit); and
    b. Benefit society or advance desired societal outcomes (Broader 
Impacts)?
    2. To what extent do the proposed activities suggest and explore 
creative, original, or potentially transformative concepts?
    3. Is the plan for carrying out the proposed activities well-
reasoned, well-organized, and based on a sound rationale? Does the plan 
incorporate a mechanism to assess success?
    4. How well qualified is the individual, team, or organization to 
conduct the proposed activities?
    5. Are there adequate resources available to the PI (either at the 
home organization or through collaborations) to carry out the proposed 
activities?
    This description of NSF's merit review policy is from NSF's 2024 
Proposal and Award Policies and Procedures Guide (PAPPG), part I,

[[Page 68475]]

chapter 3. See https://new.nsf.gov/policies/pappg/24-1.

Transformative Research

    Transformative research is defined as research driven by ideas that 
have the potential to radically change our understanding of an 
important existing scientific or engineering concept or leading to the 
creation of a new paradigm or field of science or engineering. Such 
research also is characterized by its challenge to current 
understanding or its pathway to new frontiers. See NSB's statement 
Enhancing Support of Transformative Research at NSF: https://www.nsf.gov/pubs/2007/nsb0732/nsb0732.pdf.

Broadening Participation

    ``Broadening participation in STEM'' is the comprehensive phrase 
NSF uses to refer to the Foundation's goal of increasing the 
representation and diversity of individuals, organizations, and 
geographic regions that contribute to STEM education, research, and 
innovation. To broaden participation in STEM, it is necessary to 
address issues of equity, inclusion, and access in STEM education, 
training, and careers. Whereas all NSF funding programs might support 
broadening participation components, some funding programs primarily 
focus on supporting broadening participation research and projects. 
Examples can be found on the NSF Broadening Participation in STEM 
website. See https://new.nsf.gov/funding/initiatives/broadening-participation, and the NSF PAPPG, Introduction https://new.nsf.gov/policies/pappg/24-1.

Interested Individuals and Parties

    The phrase used in this Notice, ``interested individuals and 
parties'', is intended to be interpreted broadly and inclusively by 
potential respondents; we anticipate interested individuals and parties 
include, but are not limited to:
     current, past, and prospective NSF proposers, reviewers, 
and staff
     sponsored research administrators and support 
professionals
     representatives of organizations and communities working 
in or supporting the science and engineering research and education 
enterprise
     members of other communities of practice in the science 
and engineering research and education fields and
     members of the general public expressing an interest in 
these topics.

Ann E. Bushmiller,
Senior Counsel to the National Science Board.
[FR Doc. 2024-19041 Filed 8-23-24; 8:45 am]
BILLING CODE 7555-01-P