This is the accessible text file for GAO report number GAO-12-239T entitled 'DHS Research And Development: Science and Technology Directorate's Test and Evaluation and Reorganization Efforts' which was released on November 17, 2011. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Testimony Before the Subcommittee on Cybersecurity, Infrastructure Protection, and Security Technologies, Committee on Homeland Security, House of Representatives: For Release on Delivery: Expected at 10:00 a.m. EST: November 17, 2011: DHS Research And Development: Science and Technology Directorate's Test and Evaluation and Reorganization Efforts: Statement of David C. Maurer, Director: Homeland Security and Justice Issues: GAO-12-239T: Chairman Lungren, Ranking Member Clarke, and Members of the Subcommittee: I am pleased to be here today to discuss our prior work examining the Department of Homeland Security's (DHS) Science and Technology Directorate (S&T) and Research and Development (R&D) efforts. The Homeland Security Act of 2002 created DHS and, within it, established S&T with the responsibility for conducting national research, development, test and evaluation (T&E) of technology and systems for, among other things, detecting, preventing, protecting against, and responding to terrorist attacks.[Footnote 1] Since its creation in 2003, DHS, through both S&T and its components, has spent billions of dollars researching and developing technologies used to support a wide range of missions including securing the border, detecting nuclear devices, and screening airline passengers and baggage for explosives, among others. S&T has a wide-ranging mission, which includes conducting basic and applied research of technologies,[Footnote 2] and overseeing the testing and evaluation of component acquisitions and technologies to ensure that they meet DHS acquisition requirements before implementation in the field.[Footnote 3] In recent years, we have reported that DHS has experienced challenges in managing its multibillion-dollar technology development and acquisition efforts, including implementing technologies that did not meet intended requirements and were not appropriately tested and evaluated. These problems highlight the important role that S&T plays in overseeing DHS testing and evaluation. S&T has reorganized to better achieve its goals and provide better assistance to DHS components in developing technologies. In addition to the challenge of implementing its varied mission, S&T is also managing a decline in available R&D resources. S&T's fiscal year 2011 appropriation decreased 20 percent from fiscal year 2010 and, while its fiscal year 2012 appropriation has not yet been enacted, both the House and Senate marks for the agency are lower than what was appropriated in fiscal year 2011.[Footnote 4] As a result, S&T has had to adjust resources and re-prioritize its efforts. In the past, we have reported on issues related to the transformation and reorganization of R&D efforts in the federal government, particularly related to shifting of priorities and managing a reduction in resources.[Footnote 5] In addition, we identified DHS R&D as an area for potential costs savings in our March 2011 report regarding opportunities to reduce potential duplication in government programs, save tax dollars, and enhance revenue.[Footnote 6] Specifically, we reported that DHS could take further actions to improve its management of R&D and reduce costs by ensuring that testing efforts are completed before making acquisition decisions and cost-benefit analyses are conducted to reduce R&D inefficiencies and costs. My testimony today focuses on the key findings from our prior work related to S&T's test and evaluation efforts, S&T's recent reorganization efforts, and key findings from our past work related to federal R&D. Specifically, this statement will address: * the extent to which S&T oversees T&E of major DHS acquisitions and what challenges, if any, S&T officials report facing in overseeing T&E across DHS; and: * S&T's recent reorganization efforts and how key findings from our prior work on R&D in the federal government can inform how S&T moves forward. This statement is based on reports and testimonies we issued from March 1995 to July 2011 related to DHS's efforts to manage, test, and deploy various technology programs; transformation of federal R&D; and selected updates conducted from July 2011 to the present related to S&T's reorganization efforts.[Footnote 7] For the updates, we reviewed recent S&T testimonies and documentation related to the reorganization as well as information on annual S&T appropriations and budget requests from fiscal years 2009 to 2012. For our past work, we reviewed DHS directives and testing plans, interviewed DHS, Department of Energy, Department of Defense, Environmental Protection Agency, and other agency officials, reviewed documentation from these agencies, visited laboratory facilities, and examined agency databases, among other things. We conducted this work in accordance with generally accepted government auditing standards. More detailed information on the scope and methodology from our previous work can be found within each specific report. S&T Could Take Additional Steps to Ensure that DHS T&E Requirements Are Met; Officials Cited Challenges to Overseeing T&E across DHS: S&T Oversight of DHS Testing and Evaluation: In June 2011, we reported that S&T met some of its oversight requirements for T&E of acquisition programs we reviewed, but additional steps were needed to ensure that all requirements were met. [Footnote 8] Specifically, since DHS issued the T&E directive in May 2009, S&T reviewed or approved T&E documents and plans for programs undergoing testing, and conducted independent assessments for the programs that completed operational testing during this time period. S&T officials told us that they also provided input and reviewed other T&E documentation, such as components' documents describing the programs' performance requirements, as required by the T&E directive. DHS senior level officials considered S&T's T&E assessments and input in deciding whether programs were ready to proceed to the next acquisition phase. However, S&T did not consistently document its review and approval of components' test agents--a government entity or independent contractor carrying out independent operational testing for a major acquisition--or document its review of other component acquisition documents, such as those establishing programs' operational requirements, as required by the T&E directive. For example, 8 of the 11 acquisition programs we reviewed had hired test agents, but documentation of S&T approval of these agents existed for only 3 of these 8 programs. We reported that approving test agents is important to ensure that they are independent of the program and that they meet requirements of the T&E directive. S&T officials agreed that they did not have a mechanism in place requiring a consistent method for documenting their review or approval and the extent to which the review or approval criteria were met. We reported that without mechanisms in place for documenting its review or approval of acquisition documents and T&E requirements, such as approving test agents, it is difficult for DHS or a third party to review and validate S&T's decision-making process and ensure that it is overseeing components' T&E efforts in accordance with acquisition and T&E directives and internal control standards for the federal government. As a result, we recommended that S&T develop a mechanism to document both its approval of operational test agents and component acquisitions documentation to ensure that these meet the requirements of the DHS T&E directive. S&T concurred and reported that the agency has since developed internal procedures to ensure that the approval of test agents and component acquisition documents are documented. Challenges in Coordinating and Overseeing T&E across DHS: We also reported in June 2011 that S&T and DHS component officials stated that they face challenges in overseeing T&E across DHS components which fell into 4 categories: (1) ensuring that a program's operational requirements--the key performance requirements that must be met for a program to achieve its intended goals--can be effectively tested; (2) working with DHS component program staff who have limited T&E expertise and experience; (3) using existing T&E directives and guidance to oversee complex information technology acquisitions; and (4) ensuring that components allow sufficient time for T&E while remaining within program cost and schedule estimates. Both S&T and DHS, more broadly, have begun initiatives to address some of these challenges, such as establishing a T&E council to disseminate best practices to component program managers, and developing specific guidance for testing and evaluating information technology acquisitions. In addition, as part of S&T's recent reorganization, the agency has developed a new division specifically geared toward assisting components in developing requirements that can be tested, among other things. However, since these efforts have only recently been initiated to address these DHS-wide challenges, it is too soon to determine their effectiveness. S&T Recently Reorganized and Our Prior R&D Work Could Inform How S&T Moves Forward: Since 2009, S&T has undertaken a series of efforts related to its organizational structure. S&T underwent a new strategic planning process, developed new strategic goals, and conducted a reorganization intended to better achieve its strategic goals. These efforts were implemented after a 2009 National Academy of Public Administration study found that S&T's organizational structure posed communication challenges across the agency and that the agency lacked a cohesive strategic plan and mechanisms to assess performance in a systematic way, among other things.[Footnote 9] In August 2010, S&T reorganized to align its structure with its top strategic goals, allow for easier interaction among senior leadership, and reduce the number of personnel directly reporting to the Under Secretary of S&T. Additionally, after the Under Secretary was confirmed in November 2009, S&T instituted a new strategic planning process which helped inform the development of new strategic goals. The new strategic goals announced in August 2010 include: * rapidly developing and delivering knowledge, analyses, and innovative solutions that advance the mission of DHS; * leveraging its expertise to assist DHS components' efforts to establish operational requirements, and select and acquire needed technologies; * strengthening the Homeland Security Enterprise and First Responders' capabilities to protect the homeland and respond to disasters; * conducting, catalyzing, and surveying scientific discoveries and inventions relevant to existing and emerging homeland security challenges; and: * fostering a culture of innovation and learning in S&T and across DHS that addresses mission needs with scientific, analytic, and technical rigor. According to S&T, the agency has developed a draft strategic plan that provides its overall approach to meeting these strategic goals, which is currently in the process of being finalized. Moreover, according to testimony by the Undersecretary of S&T in March 2011, to ensure that individual R&D projects are meeting their goals, S&T has committed to an annual review of its portfolio of basic and applied R&D and all proposed "new start" projects. According to S&T, the review process uses metrics determined by S&T, with input from DHS components, that are aligned with DHS priorities. These metrics consider: * the impact on the customer's mission; * the ability to transition these products to the field; * whether the investment positions S&T for the future; * whether the projects are aligned with customer requirements; * whether S&T has the appropriate level of customer interaction; and: * whether S&T is sufficiently innovative in the way it is approaching its challenges. We are currently reviewing DHS and S&T's processes for prioritizing, coordinating, and measuring the results of its R&D efforts for the Senate Committee on Homeland Security and Governmental Affairs and we will report on this issue next year. Our prior work related to R&D at other federal agencies could provide insight for S&T as it moves forward with new structures and processes operating within potential fiscal constraints. During the 1990s, we issued a series of reports on federal efforts to restructure R&D in the wake of changing priorities and efforts to balance the federal budget. More recently, we have issued reports on R&D issues at the Department of Defense (DOD), Department of Energy (DOE), the Environmental Protection Agency (EPA), and DHS. Although the specific recommendations and issues vary from department to department, there are key findings across this body of work that could potentially help inform S&T's efforts to meet DHS's R&D needs, as well as Congressional oversight of these activities. Since our assessment of R&D efforts at DHS is currently under way, we have not determined the extent to which these key findings from our prior work are applicable to DHS's R&D efforts or the extent to which DHS already has similar efforts under way. However, our prior work could provide valuable insights into how DHS could leverage the private sector to help conduct R&D, restructure R&D efforts in response to fiscal constraints, and develop comprehensive strategies to mitigate the risk of duplication and overlap. For example: * We reported on federal agencies that have restructured their research and development efforts in response to fiscal constraints. For example, in January 1998, we reported on efforts by federal agencies, such as DOD, the DOE National Laboratories, and NASA, to streamline their R&D activities and infrastructure. We reported that restructuring research, development, testing and evaluation to meet current and future needs required interagency agreements and cross- agency efforts, in addition to ongoing individual efforts.[Footnote 10] Additionally, we reported on five elements that were useful in the successful restructuring of R&D in corporate and foreign government organizations. For example, we found that successful restructuring of R&D activities included having a core mission that supports overall goals and strategies, clear definitions of those responsible for supporting that mission, and accurate data on total costs of the organization's activities. * In addition, we have reported that comprehensive strategies mitigate risk of duplication and overlap.[Footnote 11] For example, we reported in March 2011 that DOD did not have a comprehensive approach to manage and oversee the breadth of its activities for developing new capabilities in response to urgent warfighter needs, including entities engaged in experimentation and rapid prototyping to accelerate the transition of technologies to the warfighter, and lacked visibility over the full range of its efforts.[Footnote 12] As a result, we recommended that DOD issue guidance that defined roles, responsibilities, and authorities across the department to lead its efforts. DOD agreed with this recommendation. * Within DHS itself, we reported in May 2004 that DHS did not have a strategic plan to guide its R&D efforts. We recommended that DHS complete a strategic R&D plan and ensure that the plan was integrated with homeland security R&D conducted by other federal agencies. [Footnote 13] We also recommended that DHS develop criteria for distributing annual funding and for making long-term investments in laboratory capabilities, as well as develop guidelines that detailed how DOE's laboratories would compete for funding with private sector and academic entities. DHS agreed with our recommendations. While S&T developed a 5-year R&D plan in 2008 to guide its efforts and is currently finalizing a new strategic plan to align its own R&D investments and goals, DHS has not yet completed a strategic plan to align all R&D efforts across the department, as we previously recommended. * Our work on DOE National Laboratories provides additional insights related to oversight of R&D efforts that could be useful for DHS S&T. In 1995, we reported that DOE's national laboratories did not have clearly defined missions focused on accomplishing DOE's changing objectives and national priorities.[Footnote 14] DOE, at that time, managed the national laboratories on a program by program basis which inhibited cooperation across programs and hindered DOE's ability to use the laboratories to meet departmental missions. We recommended, among other things, that DOE develop a strategy that maximized the laboratories' resources. In responding, DOE said that it had undertaken a new strategic planning process which resulted in a strategic plan. Though DOE developed a strategic plan intended to integrate its missions and programs, in 1998 we reported that the laboratories did not function as an integrated national research and development system and recommended that DOE develop a comprehensive strategy to be used to assess success in meeting objectives, monitor progress, and report on that progress.[Footnote 15] DOE acknowledged that it needed to better focus the laboratories' missions and tie them to the annual budget process, but that it would take time to accomplish. * More recently, we reported in June 2009 that DOE could not determine the effectiveness of its laboratories' technology transfer efforts because it has not yet defined its overarching strategic goals for technology transfer and lacks reliable performance data.[Footnote 16] Instead, individual DOE programs such as the National Nuclear Security Administration and DOE's Office of Science articulated their own goals for technology transfer at the national laboratories. We recommended, among other things, that DOE articulate department wide priorities and develop clear goals, objectives, and performance measures. DOE generally agreed with our findings. * Lastly, our work on Environmental Protection Agency (EPA) laboratory facilities also offers insights into the importance of planning and coordination in managing R&D.[Footnote 17] Specifically, we reported in July 2011 that EPA has yet to fully address the findings of numerous past studies that have examined EPA's science activities. These past evaluations noted the need for EPA to improve long-term planning, priority setting, and coordination of laboratory activities, establish leadership for agency wide scientific oversight and decision making, and better manage the laboratories' workforce and infrastructure. We recommended, among other things, that EPA develop a coordinated planning process for its scientific activities and appoint a top-level official with authority over all the laboratories, improve physical and real property planning decisions, and develop a workforce planning process for all laboratories that reflects current and future needs of laboratory facilities. EPA generally agreed with our findings and recommendations. Chairman Lungren, Ranking Member Clarke, and Members of the Subcommittee, this concludes my prepared statement. I would be pleased to respond to any questions that you may have. GAO Contact and Staff Acknowledgments: For questions about this statement, please contact David C. Maurer at (202) 512-9627 or maurerd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Individuals making key contributions to this statement include Chris Currie, Assistant Director; Emily Gunn and Margaret McKenna. Key contributors for the previous work that this testimony is based on are listed within each individual product. [End of section] Related GAO Products: [End of section] Homeland Security: DHS Could Strengthen Acquisitions and Development of New Technologies. [hyperlink, http://www.gao.gov/products/GAO-11-829T]. Washington, D.C.: July 15, 2011. Environmental Protection Agency: To Better Fulfill Its Mission, EPA Needs a More Coordinated Approach to Managing Its Laboratories, [hyperlink, http://www.gao.gov/products/GAO-11-347]. Washington, D.C.: July 25, 2011. DHS Science and Technology: Additional Steps Needed to Ensure Test and Evaluation Requirements Are Met. [hyperlink, http://www.gao.gov/products/GAO-11-596]. Washington, D.C.: June 15, 2011. Opportunities to Reduce Potential Duplication in Government Programs, Save Tax Dollars, and Enhance Revenue, [hyperlink, http://www.gao.gov/products/GAO-11-318SP]. Washington, D.C.: Mar. 2011. Homeland Security: Improvements in Managing Research and Development Could Help Reduce Inefficiencies and Costs, [hyperlink, http://www.gao.gov/products/GAO-11-464T]. Washington D.C.: Mar. 15, 2011. Warfighter Support: DOD's Urgent Needs Processes Need a More Comprehensive Approach and Evaluation for Potential Consolidation, [hyperlink, http://www.gao.gov/products/GAO-11-273]. Washington, D.C.: Mar. 1, 2011. Technology Transfer: Clearer Priorities and Greater Use of Innovative Approaches Could Increase the Effectiveness of Technology Transfer at Department of Energy Laboratories [hyperlink, http://www.gao.gov/products/GAO-09-548]. Washington, D.C.: June 16, 2009. Homeland Security: DHS Needs a Strategy to Use DOE's Laboratories for Research on Nuclear, Biological, and Chemical Detection and Response Technologies. [hyperlink, http://www.gao.gov/products/GAO-04-653]. Washington, D.C.: May 24, 2004. Department of Energy: Uncertain Progress in Implementing National Laboratory Reforms, [hyperlink, http://www.gao.gov/products/GAO/RCED-98-197]. Washington, D.C.: Sept. 10, 1998. Best Practices: Elements Critical to Successfully Reducing Unneeded RDT&E Infrastructure. [hyperlink, http://www.gao.gov/products/GAO/NSIAD/RCED-98-23]. Washington, D.C.: Jan. 8, 1998. Department of Energy: National Laboratories Need Clearer Missions and Better Management, [hyperlink, http://www.gao.gov/products/GAO/RCED-95-10]. Washington, D.C.: Jan. 27, 1995. [End of section] Footnotes: [1] Pub. L. No. 107-296, § 302(5), 116 Stat. 2135, 2163 (2002). [2] According to S&T, basic research includes scientific efforts and experimentation directed toward increasing knowledge and understanding in the fields of physical, engineering, environmental, social and life sciences related to long-term national needs. Applied research includes efforts directed toward solving specific problems with a view toward developing and evaluating the feasibility of proposed solutions. [3] S&T's Test & Evaluation and Standards office is responsible for overseeing key requirements that DHS components are required to follow in DHS's Test and Evaluation directive. [4] The fiscal year 2012 appropriations bill passed by the House of Representatives would appropriate about 42 percent less for S&T than what was appropriated in fiscal year 2011, while the bill passed by the Senate appropriations committee would provide almost 5 percent less. [5] GAO, Homeland Security: DHS Needs a Strategy to Use DOE's Laboratories for Research on Nuclear, Biological, and Chemical Detection and Response Technologies, [hyperlink, http://www.gao.gov/products/GAO-04-653] (Washington, D.C.: May 24, 2004); Department of Energy: Uncertain Progress in Implementing National Laboratory Reforms, [hyperlink, http://www.gao.gov/products/GAO/RCED-98-197] (Washington, D.C.: Sept. 10, 1998); Best Practices: Elements Critical to Successfully Reducing Unneeded RDT&E Infrastructure, [hyperlink, http://www.gao.gov/products/GAO/NSIAD/RCED-98-23] (Washington, D.C.: Jan. 8, 1998). [6] GAO, Opportunities to Reduce Potential Duplication in Government Programs, Save Tax Dollars, and Enhance Revenue, [hyperlink, http://www.gao.gov/products/GAO-11-318SP] (Washington, D.C.: Mar. 2011). See also related GAO products at the end of this statement. [7] See related GAO products list at the end of this statement. [8] GAO, DHS Science and Technology: Additional Steps Needed to Ensure Test and Evaluation Requirements Are Met, [hyperlink, http://www.gao.gov/products/GAO-11-596] (Washington, D.C.: June 15, 2011). [9] National Academy of Public Administration, Department of Homeland Security Science and Technology Directorate: Developing Technology to Protect America (Washington D.C.: June 2009). [10] GAO, Best Practices: Elements Critical to Successfully Reducing Unneeded RDT&E Infrastructure, [hyperlink, http://www.gao.gov/products/GAO/NSIAD/RCED-98-23] (Washington, D.C.: Jan. 8, 1998). [11] [hyperlink, http://www.gao.gov/products/GAO-11-318SP]. [12] GAO, Warfighter Support: DOD's Urgent Needs Processes Need a More Comprehensive Approach and Evaluation for Potential Consolidation, [hyperlink, http://www.gao.gov/products/GAO-11-273] (Washington, D.C.: Mar. 1, 2011). [13] GAO, Homeland Security: DHS Needs a Strategy to Use DOE's Laboratories for Research on Nuclear, Biological, and Chemical Detection and Response Technologies, [hyperlink, http://www.gao.gov/products/GAO-04-653] (Washington, D.C.: May 24, 2004). [14] GAO, Department of Energy: National Laboratories Need Clearer Missions and Better Management, [hyperlink, http://www.gao.gov/products/GAO/RCED-95-10] (Washington, D.C.: Jan. 27, 1995). [15] GAO, Department of Energy: Uncertain Progress in Implementing National Laboratory Reforms, [hyperlink, http://www.gao.gov/products/GAO/RCED-98-197] (Washington, D.C.: Sept. 10, 1998). [16] GAO, Technology Transfer: Clearer Priorities and Greater Use of Innovative Approaches Could Increase the Effectiveness of Technology Transfer at Department of Energy Laboratories, [hyperlink, http://www.gao.gov/products/GAO-09-548] (Washington, D.C.: June 16, 2009). [17] GAO, Environmental Protection Agency: To Better Fulfill Its Mission, EPA Needs a More Coordinated Approach to Managing Its Laboratories, [hyperlink, http://www.gao.gov/products/GAO-11-347] (Washington, D.C.: July 25, 2011). [End of section] GAO’s Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select “E- mail Updates.” Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at www.gao.gov. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125 Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548.