Dec 8 -- The Office of Evaluation Sciences; Office of Government-wide Policy (OGP); General Services Administration (GSA), invites public comment by February 6, 2024 regarding proposed new data collection activities conducted for the National Evaluation of the American Rescue Plan (ARP). The objective of this project is to provide a systematic look at the contributions of selected ARP-funded programs toward achieving equitable outcomes to inform program design and delivery across the Federal Government. The project will include in-depth, cross-cutting evaluations and data analysis of selected ARP programs, especially those with shared outcomes, common approaches, or overlapping recipient communities; and targeted, program-specific analyses to fill critical gaps in evidence needs.
The goal of this study is to look systematically across the selected subset of ARP programs, to provide an integrated account of whether, how, and to what extent their implementation served to achieve their intended outcomes, particularly with respect to advancing equity. More specifically, the study aims to learn how lessons from examination of ARP programs and interventions with shared outcomes, common approaches, or overlapping recipient communities may inform equitable program design and delivery across the Federal Government. The study aims to address these overarching evaluation questions:
○ To what extent did ARP investments and policy interventions advance equitable outcomes for those they were designed to serve?
○ What strategies contributed to the successes, and where are different strategies needed?
○ Where multiple ARP programs aim to reach similar outcomes, especially among a shared population:
○ To what extent is there coordination across programs in their administration, customer experience strategies, or performance or outcome measurement practices?
○ To what extent are there collective impacts that could be attributed to more than one program? What kinds of impacts, if any, are observed?
○ What kinds of secondary effects are observed that may not be captured in targeted outcome measures?
The list of 32 programs covered in the May 2022 White House report “Advancing Equity through the American Rescue Plan” provided the scope of programs included in the National Evaluation. A partnership between the Office of Management and Budget Evidence Team and GSA's Office of Evaluation Sciences, this study is also guided by leadership from the White House ARP Implementation Team, who participate on the Steering Committee, as well as a team of agency experts across the Federal Government.
To build evidence in support of the study goals, this project includes a series of up to five in-depth, cross-cutting evaluations of selected ARP programs or recipient communities of multiple ARP program investments with shared outcomes, common approaches, or overlapping recipient groups. These evaluations will be selected based on program, population, place, community, or a combination of these factors. A mixed-methods approach is anticipated in order to ensure that appropriate attention is paid to context and that data collection and analysis methods reflect the complexity of program implementation and address the specific evaluation questions identified through the ongoing planning and consultation process.
The ARP National Evaluation will use a multiple-phased approach for this proposed information collection activity. In Phase 1 (current request) the research team seeks approval to carry out consultations with the relevant state and local agencies, community-based organizations, and program participants, including the formal recruitment process to establish community advisory boards for each of the planned in-depth evaluations.
Under subsequent phases of the request, the project will update the information collection request for the instruments tailored to each in-depth evaluation, to reflect the specific evaluation design, information collection methods and instruments, and associated burden. The proposed information collection activities cover mixed-method approaches to implement primarily outcome and process evaluations. Data collection activities for these studies may include: (1) interviews with program administrators and staff; (2) focus groups, (3) short surveys of program participants and/or eligible non-participants, and (4) data requests.
Respondents: State and local program administrators, program staff, community-based program partners, and individuals who participate or are eligible to participate in the relevant ARP programs.
The multi-year evaluation will be conducted by the American Institutes for Research (AIR), with Abt Associates and Decision Information Resources. OES is supporting the contract in partnership with the Office of Management and Budget’s Evidence Team, the ARP Implementation Team, and other agency partners.
National Evaluation of ARP fact sheet:
https://oes.gsa.gov/assets/files/OES-ARP-National-Evaluation.pdf
OES ARP:
https://oes.gsa.gov/american-rescue-plan/
Draft data collection instruments and technical documentation requested from GSA by AEAStat.
FRN:
https://www.federalregister.gov/d/2023-27007