Summary
The Decennial Census is mandated by the U.S. Constitution and provides vital data that are used, among other things, to reapportion and redistrict congressional seats. In March 2008, GAO designated the 2010 Decennial Census a high-risk area, citing a number of long-standing and emerging challenges, including weaknesses in the Census Bureau's (Bureau) management of its information technology (IT) systems and operations. In conducting the 2010 census, the Bureau is relying on both the acquisition of new IT systems and the enhancement of existing systems. Thoroughly testing these systems before their actual use is critical to the success of the census. GAO was asked to determine the status of and plans for testing key decennial systems. To do this, GAO analyzed testing documentation, interviewed Bureau officials and contractors, and compared the Bureau's efforts with recognized best practices.
Although the Bureau has made progress in testing key decennial systems, critical testing activities remain to be performed before systems will be ready to support the 2010 census. Bureau program offices have completed some testing of individual systems, but significant work still remains to be done, and many plans have not yet been developed (see table below). In its testing of system integration, the Bureau has not completed critical activities; it also lacks a master list of interfaces between systems; has not set priorities for the testing of interfaces based on criticality; and has not developed testing plans and schedules. Although the Bureau had originally planned what it refers to as a Dress Rehearsal, starting in 2006, to serve as a comprehensive end-to-end test of key operations and systems, significant problems were identified during testing. As a result, several key operations were removed from the Dress Rehearsal and did not undergo end-to-end testing. The Bureau has neither developed testing plans for these key operations, nor has it determined when such plans will be completed. Weaknesses in the Bureau's testing progress and plans can be attributed, in part, to a lack of sufficient executive-level oversight and guidance. Bureau management does provide oversight of system testing activities, but the oversight activities are not sufficient. For example, Bureau reports do not provide comprehensive status information on progress in testing key systems and interfaces, and assessments of the overall status of testing for key operations are not based on quantitative metrics. Specifically, key operations that do not yet have plans developed are marked as making acceptable progress based solely on management judgment. Further, although the Bureau has issued general testing guidance, it is neither mandatory nor specific enough to ensure consistency in conducting system testing. Without adequate oversight and more comprehensive guidance, the Bureau cannot ensure that it is thoroughly testing its systems and properly prioritizing testing activities before the 2010 Decennial Census, posing the risk that these systems may not perform as planned.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director:
Team:
Phone:
David A. Powner
Government Accountability Office: Information Technology
No phone on record
Recommendations for Executive Action
Recommendation: To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously, for the Headquarters Universe Control & Management (UC&M) and Response Processing System (RPS), finalize requirements for 2010 census operations and complete testing plans and schedules for 2010 operations that trace to baselined system requirements.
Agency Affected: Department of Commerce
Status: Open
Comments: Requirements, test plans, and schedules for the Universe Control & Management (UC&M) & Response Processing System (RPS) have been finalized and completed. Traceablilty will not be complete until March 2010.
Recommendation: To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously, for Master Address File/Topologically Integrated Geographic Encoding and Referencing system (MAF/TIGER), establish the number of products required, define related requirements, and establish a testing plan and schedule for 2010 operations.
Agency Affected: Department of Commerce
Status: Open
Comments: For the Master Address File/Topologically Integrated Geographic Encoding and Referencing system (MAF/TIGER), the number of products required have been established, however test plans will not be baselined until September 2010. As of July 2009, two of nine test plans have been baselined.
Recommendation: To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously, for Field Data Collection Automation (FDCA), establish testing plans for the continuity of operations and map printing systems that trace to baselined system requirements.
Agency Affected: Department of Commerce
Status: Closed - implemented
Comments: The Bureau has established test plans for continuity of operations and map printing systems.
Recommendation: To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously, for Paper-Based Operations (PBO), develop baseline requirements and complete testing plans and schedules for 2010 operations.
Agency Affected: Department of Commerce
Status: Closed - implemented
Comments: The bureau has developed high-level requirements and established schedules for the system. In addition, the Bureau has developed a test plan which documents the testing strategy that will be used to verify and ensure that the system meets the developed requirements.
Recommendation: To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously establish a master list of system interfaces; prioritize the list, based on system criticality and need date; define all interfaces; and develop integration testing plans and schedules for tracking the progress of testing these interfaces.
Agency Affected: Department of Commerce
Status: Closed - implemented
Comments: The Bureau has updated their master list of interfaces to define all interfaces. In addition, the Bureau has developed a process to track the progress of testing all interfaces.
Recommendation: To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously establish a date for completing testing plans for the operations removed from the Dress Rehearsal operations and prioritize testing activities for these operations.
Agency Affected: Department of Commerce
Status: Open
Comments: The Bureau expected to complete testing plans for the 5 operations removed from the Dress Rehearsal by June 15, 2009, but only 4 (non-response follow up, group quarters enumeration, update/leave, and field verification) have been completed.
Recommendation: To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously finalize risk mitigation plans detailing actions to address system problems that are identified during testing.
Agency Affected: Department of Commerce
Status: Closed - implemented
Comments: The Bureau finalized its plan for operational and system failures.
Recommendation: To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously establish specific testing metrics and detailed status reports to monitor testing progress and better determine whether corrective action is needed for all key testing activities.
Agency Affected: Department of Commerce
Status: Closed - implemented
Comments: The bureau has required specific metrics and detailed reports to be presented to management for all operations and systems. In addition, the bureau has developed a testing overview which provides detailed status of testing for all systems.
Recommendation: To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously designate a senior manager with primary responsibility for monitoring testing and overseeing testing across the Bureau.
Agency Affected: Department of Commerce
Status: Closed - implemented
Comments: The Bureau named a Decennial Census testing officer whose primary responsibilities include monitoring testing for Decennial Census activities.
Recommendation: After the 2010 census, the Bureau should establish comprehensive systems and integration testing guidance to guide future testing of systems.
Agency Affected: Department of Commerce: Bureau of the Census
Status: Open
Comments: The Bureau agrees with the recommendation to establish comprehensive systems and integration testing guidance to guide future testing of systems and has referred it to the manager in charge of 2020 planning for testing guidance.
