Homeland Security: Research & Development Coordination Has Improved, but Additional Actions Needed to Track and Evaluate Projects
Fast Facts
The Department of Homeland Security’s Science and Technology Directorate is required to coordinate research and development (R&D) across the department, which helps to prevent duplicative efforts.
We found that the department's R&D project information is stored in various disparate sources (e.g., reports, data systems, etc.)—which makes it difficult to identify and track. By developing a method to align these sources, the directorate can better compile and analyze R&D project information.
We recommended that it do so.
One R&D project connects border security teams to each other in real-time.
A photo of border security agents using R&D technology.
Highlights
What GAO Found
The Department of Homeland Security (DHS) obligated more than $10 billion for research and development (R&D) from fiscal years 2010 through 2017. Seven DHS components have budget authority to conduct R&D, and the Science and Technology Directorate (S&T) obligated nearly 80 percent of all DHS R&D funds during this time period. These components conduct a wide range of R&D, from cybersecurity to border security projects. S&T generally leads or funds R&D projects by providing technology and knowledge products to support four homeland security mission areas:
- Disaster resilience. Improving community resilience to natural disasters through technology and tools;
- Critical incidents. Improving response technological capabilities;
- Border security. Improving the nation's ability to detect, interdict and prosecute illegal activity across air, land and sea.
- Cybersecurity. Developing technologies and tools to secure systems and critical infrastructures against cyberattacks.
S&T strengthened its R&D coordination efforts across DHS, but some challenges remain. In 2015, DHS established an R&D coordination mechanism, to be led by S&T, and in 2017 issued R&D coordination-related guidance. Specifically, to improve coordination, DHS established an Integrated Product Team (IPT) process to serve as the key R&D coordination mechanism within DHS. All ten DHS components that GAO interviewed stated that the IPT process improved visibility into DHS R&D efforts. However, the component that obligated approximately 17 percent of DHS R&D funds in 2017, or $176 million, did not participate in the IPT process in 2018, as required. Nonparticipation poses a risk to R&D coordination efforts across DHS, including R&D project information not being shared among components. Furthermore, ensuring that all required components participate in the IPT process would help S&T maintain visibility of R&D projects in order to fulfill its statutory role of coordinating R&D, and mitigate the risk of potential duplication of effort.
S&T, in its coordination role for DHS, uses disparate information sources to identify and track R&D project information and faces challenges to track progress and other information for ongoing R&D projects. For example, R&D project information is stored in multiple information sources leading to difficulty in integrating complete R&D project information and resulting in reporting that is not comprehensive. By developing a mechanism to address these challenges, S&T can further improve its efforts to report and analyze R&D project information, and have improved visibility on R&D efforts across DHS. GAO also identified challenges in collecting information related to R&D performance. Among other things, DHS is not well positioned to integrate the results and share lessons learned because limited R&D customer feedback information is collected and analyzed. Of the seven DHS components with R&D budget authority, two reported having formal customer feedback mechanisms. As a result, DHS is unable to more fully understand its customers' perceptions and experience which would allow DHS to better assess the performance of its R&D efforts.
Why GAO Did This Study
Conducting R&D on technologies is vital to enhancing the security of the nation. The Homeland Security Act of 2002, as amended, designates S&T as responsible for coordinating all R&D activities of DHS. Questions have been raised about S&T's ability to demonstrate the impact of its R&D investments. Since DHS began operations in 2003, GAO has made recommendations to help improve DHS's efforts to coordinate and oversee R&D.
GAO was asked to review DHS's R&D efforts. This report examines (1) how much DHS has obligated for R&D and what types of R&D DHS conducts, (2) to what extent S&T coordinates R&D across DHS, and (3) how, if at all, DHS identifies and tracks R&D efforts.
GAO reviewed documentation from DHS related to the conduct, coordination, tracking, and evaluation of R&D projects. GAO interviewed DHS officials with responsibilities related to, among other things, R&D financial reporting, performance evaluation, and the IPT process, including officials from the 10 DHS components that participate in the IPTs. GAO also reviewed DHS R&D budget and obligation data from fiscal years 2010 through 2017.
Recommendations
GAO is making four recommendations, with which DHS concurred, including that DHS: 1) ensure all components participate in the IPT process, 2) develop a mechanism that aligns R&D project tracking sources, and 3) collect feedback from R&D customers.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Office of the Deputy Secretary | The Deputy Secretary of the Department of Homeland Security should ensure that all components adhere to IPT participation requirements, in accordance with the DHS directives. (Recommendation 1) |
We found that DHS established an Integrated Product Team (IPT) process to serve as the key R&D coordination mechanism within DHS. All ten DHS components that GAO interviewed stated that the IPT process improved visibility into DHS R&D efforts. However, the component that obligated approximately 17 percent of DHS R&D funds in 2017, or $176 million, did not participate in the IPT process in 2018, as required. Therefore, we recommended that DHS ensure that all components adhere to IPT participation requirements, in accordance with the DHS directives . DHS issued a management directive in February 2020 that, according to the directive, established the DHS policy, responsibilities, and requirements for coordinating R&D activities conducted or funded by the Department, including any of its components. The Directive further delineates component responsibilities, which includes, among other things, identifying and documenting the relationship between capability gaps and R&D activities in progress, funded, planned, or recently completed in the component mission areas. This clarifying guidance will add assurance that components involved in DHS R&D efforts will participate in the Department's coordination efforts and DHS will be better positioned to address research gaps and avoid uncoordinated projects among the components.
|
| Office of the Deputy Secretary | The Deputy Secretary of the Department of Homeland Security should develop a mechanism that aligns processes and information sources for collecting R&D project data from DHS components to ensure that the information can be collected, integrated and result in a comprehensive accounting of R&D projects DHS-wide. (Recommendation 2) |
We found that S&T's efforts to maintain an inventory of R&D projects across DHS had resulted in reporting that was not comprehensive. Specifically, we identified multiple sources of R&D project information, each posing its own challenges or limitations. For example, S&T officials told us that for at least one R&D data source, the list of projects provided to S&T was not comprehensive. Therefore, we recommended that DHS develop a mechanism that aligns processes and information sources for collecting R&D project data from DHS components that would result in a comprehensive accounting of R&D projects DHS-wide. In response to this recommendation, DHS reported in July 2021 that it had analyzed existing R&D project sources, identified inconsistencies and commonalities among them and, based on that information, developed a mechanism, consisting of a new reporting framework and timeline, to report annually to Congress on the Department's R&D projects, as required by the National Defense Authorization Act of 2017. The new framework specifies the collection, integration, and results of the more streamlined mechanism to develop a single comprehensive source of R&D projects, which will allow DHS to have the information it needs to make informed decisions about R&D investments. This framework is consistent with our recommendation.
|
| Office of the Deputy Secretary | The Deputy Secretary of the Department of Homeland Security should direct Office of the Chief Financial Officer program officials to ensure that S&T take steps to more fully incorporate leading practices, such as those included in DHS's budget preparation guidance, into R&D milestones. (Recommendation 3) |
We found that DHS components have processes in place to collect certain indicators of R&D performance, but these processes have limitations. For example, milestones are often used as the basis of an alternative form of performance goal and milestones can be described in a way that makes it possible to discern if progress is being made toward a goal. In our analysis of 14 milestones for seven Science and Technology Directorate (S&T) high-priority R&D projects identified in fiscal year 2018 DHS budget justification documents, we found that only 3 of the 14 milestones fully adhered to DHS guidance for milestone descriptions. DHS's guidance included leading practices for successful milestones, such as milestones that provide a clear understanding of expected outcomes and measurable results. Therefore, we recommended that S&T take steps to more fully incorporate leading practices, such as those included in DHS's guidance, into R&D milestones. In response to our recommendation, S&T provided key practices for developing milestones to S&T offices and programs to use when developing R&D milestones for fiscal year 2021. According to S&T officials, this guidance is now provided as part of S&T's budget justification development and again when finalizing milestones at the start of fiscal year execution, as well as when S&T program managers seek guidance in updating or developing milestones. For example, during the course of planning efforts in 2020, and when developing fiscal year 2021 milestones for an R&D project related to unmanned aircraft systems, S&T program officials documented their consultation of the key practices within their milestone determination summary. This documentation reflected the key practices suggested in our report. S&T's actions are consistent with our recommendation and will help ensure successful implementation of milestones as indicators of R&D performance.
|
| Office of the Deputy Secretary | The Deputy Secretary of the Department of Homeland Security should develop standard processes and procedures for collecting and analyzing customer feedback, applicable to components conducting R&D, for improving the usefulness of existing customer feedback mechanisms to assess R&D efforts and for implementing such mechanisms where absent. (Recommendation 4) |
We found that DHS was not well positioned to integrate the results and share lessons learned related to R&D performance, because limited R&D customer feedback information was collected and analyzed. Specifically, of the seven DHS components with R&D budget authority, two reported having formal customer feedback mechanisms. As a result, DHS was unable to more fully understand its customers' perceptions and experience which would allow DHS to better assess the performance of its R&D efforts. Therefore, we recommended that DHS should develop standard processes and procedures for collecting and analyzing customer feedback, applicable to components conducting R&D, for improving the usefulness of existing customer feedback mechanisms to assess R&D efforts and for implementing such mechanisms where absent. In response to this recommendation, in July 2021, DHS identified R&D customer feedback models and developed a methodology for collecting and utilizing feedback within existing DHS practices. The questionnaire includes questions such as whether the R&D product is currently in use, what the primary benefits of the R&D product has been, and whether the product included cost savings, among others. Additionally, DHS conducted an exercise to validate the methodology, and subsequently incorporated feedback and comments from the exercise to revise and improve the feedback process, which will allow DHS to more fully understand its customers' perceptions and experience and assess the benefits and performance of its R&D efforts. These actions are consistent with our recommendation.
|