Arizona Border Surveillance Technology: More Information on Plans and Costs Is Needed before Proceeding
Highlights
In recent years, nearly half of all annual apprehensions of illegal aliens along the entire Southwest border with Mexico have occurred along the Arizona border. Keeping illegal flows of people and drugs under control remains a top priority for the Department of Homeland Security's (DHS) U.S. Customs and Border Protection (CBP). In 2005, the Secure Border Initiative Network (SBInet) was conceived as a surveillance technology to create a "virtual fence" along the border. After spending nearly $1 billion, DHS deployed SBInet systems along 53 miles of Arizona's border that represent the highest risk for illegal entry. In January 2011, in response to concerns regarding SBInet's performance, cost, and schedule, DHS cancelled future procurements. CBP developed the Arizona Border Surveillance Technology Plan (Plan) for the remainder of the Arizona border. Funding for this Plan for fiscal year 2012 is $242 million. GAO was requested to assess the extent to which CBP (1) has the information needed to support and implement the Plan and (2) estimated life-cycle costs for future investments in accordance with best practices. GAO analyzed Plan documents and cost estimates, compared those estimates with best practices, and interviewed CBP officials.
CBP does not have the information needed to fully support and implement its Arizona Border Surveillance Technology Plan in accordance with DHS and Office of Management and Budget (OMB) guidance. In developing the Plan, CBP conducted an analysis of alternatives and outreach to potential vendors. However, CBP has not documented the analysis justifying the specific types, quantities, and deployment locations of border surveillance technologies proposed in the Plan. Best practices for developing and managing costs indicate that a business case analysis should be rigorous enough that independent parties can review it and clearly understand why a particular alternative was chosen to support mission requirements. Without documentation of the analysis, there is no way to verify the process CBP followed, identify how the underlying analyses were used, assess the validity of the decisions made, or justify the funding requested for the Plan. CBP officials also have not yet defined the mission benefits expected from implementing the new Plan. GAO has previously reported that a solid business case providing an understanding of the potential return of large investments can be helpful to decision makers for determining whether continued investment is warranted after deployment. Defining the expected benefit could help improve CBP's ability to assess the effectiveness of the Plan as it is implemented. CBP does not intend to assess and address operational issues regarding the effectiveness and suitability of SBInet, steps that could provide CBP with information to help make decisions regarding alternatives for implementing the Plan. OMB guidance suggests that a post-implementation review occur when a system has been in operation for 6 months or immediately following investment termination. Such a review could help CBP make the most effective use of existing SBInet systems that, in connection with the Plan, could build a comprehensive and integrated approach for surveillance technology along the entire Arizona border. CBP's 10-year life-cycle cost estimate for the Plan of $1.5 billion is based on a rough order of magnitude analysis, and agency officials were unable to determine a level of confidence in their estimate as best practices suggest. Specifically, GAO's review of the estimate concluded that the estimate reflected substantial features of best practices, being both comprehensive and accurate, but it did not sufficiently meet other characteristics of a high-quality cost estimate, such as credibility, because it did not identify a level of confidence or quantify the impact of risks. GAO and OMB guidance emphasize that reliable cost estimates are important for program approval and continued receipt of annual funding. In addition, because CBP was unable to determine a level of confidence in its estimate, it will be difficult for CBP to determine what levels of contingency funding may be needed to cover risks associated with implementing new technologies along the remaining Arizona border. Thus, it will be difficult for CBP to provide reasonable assurance that its cost estimate is reliable and that its budget request for fiscal year 2012 and beyond is realistic and sufficient. A robust cost estimate--one that includes a level of confidence and quantifies the impact of risk--would help ensure that CBP's future technology deployments have sufficient funding levels related to the relative risks. GAO recommends that CBP document the analysis justifying the technologies proposed in the Plan, determine its mission benefits, conduct a post-implementation review of SBInet and determine a more robust life-cycle cost estimate for the Plan. DHS concurred with the recommendations.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| United States Customs and Border Protection | To increase the likelihood of successful implementation of the Arizona Border Surveillance Technology Plan and maximize the effectiveness of technology already deployed, the Commissioner of CBP should take the following step in planning the agency's new technology approach: ensure that the underlying analyses of the Plan are documented in accordance with DHS guidance and internal controls standards. |
DHS concurred with the recommendation but, subsequently, CBP officials stated that it is not planning further analyses or additional documentation given that officials consider their analyses to be sufficiently documented in the Arizona Border Surveillance Technology Plan (the Plan). As of March 2014, CBP had awarded 5 out of 7 technology contracts. For the remaining two technologies, CBP plans to award the contracts in July 2014 and in March 2015. As CBP has moved forward in awarding contracts for the Plan's technology programs and does not plan to conduct further analyses, we have closed this recommendation as not implemented.
|
| United States Customs and Border Protection | To increase the likelihood of successful implementation of the Arizona Border Surveillance Technology Plan and maximize the effectiveness of technology already deployed, the Commissioner of CBP should take the following step in planning the agency's new technology approach: determine the mission benefits to be derived from implementation of the plan and develop and apply key attributes for metrics to assess program implementation. |
As GAO recommended in November 2011, CBP has identified improved situational awareness and agent safety, among other things, as the mission benefits derived from the majority of these technologies. The U.S. Border Patrol, within CBP, expanded its 2011 Arizona Border Surveillance Technology Plan to the Southwest Border Technology Plan (Plan) in 2014. In November 2014, CBP identified a set of potential key attributes for performance metrics for all technologies to be deployed under the Plan. In May 2017, Border Patrol officials demonstrated a new system that was intended to allow for more comprehensive analysis of the contributions of surveillance technologies to Border Patrol's mission. As of March 2019, Border Patrol is able to generate a performance report, using data collected from multiple systems, on how surveillance technologies have assisted agents during operations, including Border Patrol apprehensions. Border Patrol officials also described how they use a surveillance capability score to represent the combined contributions of individual technology assets and agents on patrol to conduct surveillance in a given area. The score allows for Border Patrol to consider, among other things, how surveillance technologies may have different impacts in various environments. Border Patrol officials stated that the report and surveillance capability score comprise Border Patrol's metrics for assessing the performance of surveillance technologies. Identifying expected mission benefits and developing performance metrics are important efforts to assist those Border Patrol officials assessing the contributions of border technologies, including when determining future investments in technologies, and GAO is closing this recommendation as implemented.
|
| United States Customs and Border Protection | To increase the likelihood of successful implementation of the Arizona Border Surveillance Technology Plan and maximize the effectiveness of technology already deployed, the Commissioner of CBP should take the following step in planning the agency's new technology approach: conduct a post-implementation review and operational assessment of SBInet, including consideration of the Army Test and Evaluation Command's (ATEC) test results, and assess the costs and benefits of addressing the issues identified to help ensure the security of the 53 miles already covered by SBInet and enhance security on the Arizona border. |
In January 2013, CBP released the results of its Post Implementation Review (PIR) and operational assessment for SBInet Block I. The PIR resulted in recommendations to improve the operational effectiveness of the SBInet systems. Additionally, the PIR included information on CBP's efforts to address to recommendations made by the Army Test and Evaluation Command (ATEC). The PIR noted that CBP had yet to address three of the nine recommendations in the ATEC report. In July 2015, CBP stated that the ATEC findings/observations were reviewed but superseded by the PIR, which was determined to be more current and applicable to the SBInet Block I deployment plan. Accordingly, CBP decided that no action would be taken on the ATEC findings/observations. In addition, regarding CBP's efforts to address the recommendations outlined in the PIR, in July 2015, CBP provided GAO with specific details on the agency's efforts to address the recommendations and actionable items in the PIR. CBP provided documentation of its assessment and review of each recommendation in the PIR as well as the associated cost and benefits of each recommendation. Specifically, in September 2013, USBP and OTIA conducted an evaluation to determine the merits of the six PIR recommendations and 12 actionable items. CBP concurred with four actionable items and non-concurred with eight items. In July 2015, CBP officials told us that no further action will be taken to address the PIR recommendations and provided documentation of its assessment of the costs and benefits of addressing the recommendations in PIR. Based on GAO's review of the documentation provided, CBP's actions meet the intent of this recommendation.
|
| United States Customs and Border Protection | To increase the reliability of CBP's Cost Estimate for the Arizona Border Surveillance Technology Plan, the Commissioner of CBP should update its cost estimate for the Plan using best practices, so that the estimate is comprehensive, accurate, well-documented, and credible. Specifically, the CBP's Office of Technology Innovation and Acquisition (OTIA) program office should (1) fully document data used in the cost model; (2) conduct a sensitivity analysis and risk and uncertainty analysis to determine a level of confidence in the estimate so that contingency funding can be established relative to quantified risk; and (3) independently verify the new life-cycle cost estimate with an independent cost estimate and reconcile any differences. |
CBP has taken significant steps toward updating cost estimates for the Plan using best practices, as GAO first recommended in November 2011. CBP provided cost estimates for the Integrated Fixed Tower and Remote Video Surveillance System programs, the two highest cost programs in the Plan, in February and March 2012, respectively. In November 2012 and March 2014, GAO found that CBP developed these estimates in a way that at least partially met best practices; however, the estimates had not been verified with independent cost estimates, as called for in the 2009 GAO Cost Estimating and Assessment Guide. GAO recommended in its March 2014 report that CBP verify the life-cycle cost estimates for the programs with independent cost estimates and reconcile any differences to better ensure the reliability of the estimates. In May 2016, CBP officials stated that the Department of Homeland Security's Cost Analysis Division (CAD) had started piloting an independent cost estimate for the Remote Video Surveillance System program. CBP selected this program for the pilot because these technologies would be deployed along the border, expanding the program beyond Arizona, and thus would benefit most from having an independent cost estimate. CAD approved its independent cost estimate for the program in November 2016, and according to CBP officials, CBP finalized the life-cycle cost estimate and reconciled it with the independent cost estimate in March 2017. CBP reported that the component acquisition executive approved the reconciliation estimate in September 2017. In November 2017, GAO reviewed the approved life-cycle cost estimate and independent cost estimate and found that they met the intent of GAO's recommendations. That same month CBP officials confirmed that the agency does not plan to conduct an independent cost estimate for the Integrated Fixed Tower program, noting that there are no plans to expand the program and that they believe it would not provide any value as CBP is in the deployment stage of the program. GAO concurs given there are no current plans to expand the program. In November 2017, CBP officials reported that they plan to continue to use best practices to guide their cost estimating methods for future technology investments. CBP's efforts have helped to position the agency to better ensure the reliability of its cost estimates for border security technology investments, and as a result this recommendation is closed as implemented.
|