Federal Research Centers: DHS Actions Could Reduce the Potential for Unnecessary Overlap among Its R&D Projects
Fast Facts
The Department of Homeland Security uses Federally Funded Research and Development Centers to work on projects that DHS or contractors can't do as effectively. Since 2004, DHS has committed over $3 billion to projects at the centers, which are run by non-profit organizations.
DHS reviews proposed research and development projects, including center projects, to check if they overlap—i.e., unnecessarily pursue the same goals. But 5 agencies within DHS could be funding research and development projects that may not be checked for overlap.
We recommended requiring overlap reviews of centers' projects with the 5 agencies, and more.
Highlights
What GAO Found
The Department of Homeland Security (DHS) Science and Technology Directorate (S&T) is responsible for coordinating and overseeing the department's research and development (R&D) activities and, with selected DHS components, funding these activities. To help meet its R&D needs, DHS sponsors Federally Funded Research and Development Centers (FFRDCs), two of which are overseen by S&T's FFRDC Program Management Office (PMO).
S&T has a coordination process that includes steps for reviewing proposed R&D projects with DHS component-funded R&D projects that S&T funds, oversees, or otherwise supports. S&T officials told GAO that they review proposed FFRDC projects for unnecessary overlap as part of this coordination process. However, GAO's review of DHS and S&T policies found that the five DHS components that receive R&D appropriations are not required to share their component-funded R&D activities with S&T. Thus, S&T's overall coordination reviews may not always include these DHS component-funded R&D activities. A leading practice from prior GAO work states that establishing a means to operate across agency boundaries can reduce or better manage program overlap. S&T could reduce the potential for conducting similar R&D work by amending its policies to require that officials review proposed FFRDC projects for unnecessary overlap with DHS component-funded R&D projects.
FFRDC PMO is responsible for assessing the performance of the two FFRDCs it oversees. Federal requirements and DHS guidance require FFRDC PMO officials to assess FFRDC performance each year and more comprehensively every 5 years. FFRDC PMO has developed two tools—a performance framework that identifies 11 performance metrics and a FFRDC user feedback survey—to assess FFRDC performance.
Department of Homeland Security's (DHS) Federally Funded Research and Development Center Performance Assessment Process
GAO found that FFRDC PMO's response rates for FFRDC user feedback surveys ranged in recent years from 100 percent to 43 percent across the two FFRDCs. FFRDC PMO officials said they have not analyzed the extent to which these variations in response rates could have impacted the validity of the overall survey data. Low response rates raise the risk that the survey responses do not represent the views of all FFRDC users. If the views of the users who did not respond to the survey differ from those who did, the survey results could produce a different outcome than what would be found across all users. Given the importance of the surveys in assessing FFRDC performance, analyzing the risk of low response rates could help FFRDC PMO identify whether further steps, such as increasing such rates, are needed to mitigate the risk of those responses not representing all users.
Why GAO Did This Study
DHS uses FFRDCs—not-for-profit organizations—to meet special, long-term R&D needs that its components and other contractors cannot meet as effectively. Since DHS established its first FFRDC in 2004, as statutorily required, the department has obligated over $3 billion through fiscal year 2023 on FFRDC contracts. According to DHS, FFRDCs are to provide independent and objective advice on critical homeland security issues.
GAO was asked to review the oversight of FFRDCs. This report addresses, among other issues, the extent to which (1) S&T has reviewed DHS's proposed FFRDC projects for potential unnecessary overlap with other DHS R&D activities and (2) FFRDC PMO has developed tools to assess FFRDCs' performance and receives, analyzes, and shares key performance information.
GAO reviewed DHS and S&T policies and procedures. GAO also selected a sample of 118 out of 732 FFRDC task orders—orders for services placed against established contracts—over a 9-year period to reflect a range in value and volume. GAO also interviewed officials from S&T, FFRDC PMO, selected DHS components, and the FFRDCs.
Recommendations
GAO is making eight recommendations, including that DHS (1) amend policies to require S&T to review FFRDC projects for potential overlap with DHS R&D activities and (2) ensure FFRDC PMO analyzes the risk of low response rates for FFRDC user surveys. DHS concurred with all eight recommendations and identified planned actions to address them.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Homeland Security | The DHS Under Secretary for Science and Technology should update the list of R&D activities on at least a quarterly basis, as statutorily required, and disseminate the updated list to relevant DHS entities, including DHS components. (Recommendation 1) |
When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
|
| Department of Homeland Security | The DHS Under Secretary for Science and Technology should amend policies and procedures to require that S&T review proposed FFRDC projects for unnecessary overlap with R&D activities funded and developed by DHS components that have their own R&D appropriations. (Recommendation 2) |
When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
|
| Department of Homeland Security | The DHS Under Secretary for Science and Technology should require S&T to document its additional overlap reviews of R&D activities funded and developed by DHS components that have their own R&D appropriations. (Recommendation 3) |
When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
|
| Department of Homeland Security | The DHS Under Secretary for Science and Technology should ensure that FFRDC PMO conducts and documents an analysis of the risk of nonresponse bias based on the response rate of the initial outreach efforts for each fiscal year's set of user feedback surveys. Should the analysis indicate a risk of nonresponse bias which would influence FFRDC PMO's understanding of FFRDC performance, FFRDC PMO should take additional steps to increase their response rate. (Recommendation 4) |
When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
|
| Department of Homeland Security | The DHS Under Secretary for Science and Technology should ensure FFRDC PMO documents, in all Annual Assessments and 5-Year Comprehensive Reviews, the response rate, risk of bias, and steps taken to increase the user feedback survey response rate. (Recommendation 5) |
When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
|
| Department of Homeland Security | The DHS Under Secretary for Science and Technology should amend its policies and procedures to require FFRDC PMO to include a comprehensive analysis of open-ended responses to FFRDC user surveys in the Annual Assessments to gain a more complete understanding of user feedback when assessing FFRDC performance. (Recommendation 6) |
When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
|
| Department of Homeland Security | The DHS Under Secretary for Science and Technology should ensure that FFRDC PMO shares, as required, the results of Annual Assessments with FFRDC leadership. (Recommendation 7) |
When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
|
| Department of Homeland Security | The Under Secretary for Science and Technology should ensure that FFRDC PMO establishes a process to review and analyze a selection of task order results that reflect a range of S&T and DHS component experiences to inform the design and implementation of FFRDC task orders. (Recommendation 8) |
When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
|