Information Technology: Census Bureau Testing of 2010 Decennial Systems Can Be Strengthened
GAO-09-262
Published: Mar 05, 2009. Publicly Released: Mar 05, 2009.
Skip to Highlights
Highlights
The Decennial Census is mandated by the U.S. Constitution and provides vital data that are used, among other things, to reapportion and redistrict congressional seats. In March 2008, GAO designated the 2010 Decennial Census a high-risk area, citing a number of long-standing and emerging challenges, including weaknesses in the Census Bureau's (Bureau) management of its information technology (IT) systems and operations. In conducting the 2010 census, the Bureau is relying on both the acquisition of new IT systems and the enhancement of existing systems. Thoroughly testing these systems before their actual use is critical to the success of the census. GAO was asked to determine the status of and plans for testing key decennial systems. To do this, GAO analyzed testing documentation, interviewed Bureau officials and contractors, and compared the Bureau's efforts with recognized best practices.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Commerce | To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously, for the Headquarters Universe Control & Management (UC&M) and Response Processing System (RPS), finalize requirements for 2010 census operations and complete testing plans and schedules for 2010 operations that trace to baselined system requirements. |
The Census Bureau finalized the system requirements for UC&M and RPS and established test plans that trace to the system requirements.
|
| Department of Commerce | To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously, for Master Address File/Topologically Integrated Geographic Encoding and Referencing system (MAF/TIGER), establish the number of products required, define related requirements, and establish a testing plan and schedule for 2010 operations. |
The Census Bureau has determined the number of MAF/TIGER products required for 2010 operations and baselined related test plans and schedules. In addition, the Bureau has conducted Independent Testing and Verification to better ensure that the MAF/TIGER system software meets both its functional and system requirements.
|
| Department of Commerce | To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously, for Field Data Collection Automation (FDCA), establish testing plans for the continuity of operations and map printing systems that trace to baselined system requirements. |
The Bureau has established test plans for continuity of operations and map printing systems.
|
| Department of Commerce | To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously, for Paper-Based Operations (PBO), develop baseline requirements and complete testing plans and schedules for 2010 operations. |
The bureau has developed high-level requirements and established schedules for the system. In addition, the Bureau has developed a test plan which documents the testing strategy that will be used to verify and ensure that the system meets the developed requirements.
|
| Department of Commerce | To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously establish a master list of system interfaces; prioritize the list, based on system criticality and need date; define all interfaces; and develop integration testing plans and schedules for tracking the progress of testing these interfaces. |
The Bureau has updated their master list of interfaces to define all interfaces. In addition, the Bureau has developed a process to track the progress of testing all interfaces.
|
| Department of Commerce | To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously establish a date for completing testing plans for the operations removed from the Dress Rehearsal operations and prioritize testing activities for these operations. |
The Census Bureau finalized test plans and completed testing for most operations removed from the Dress Rehearsal. These operations include Group Quarters Enumeration, Nonresponse Follow-up, Enumeration at Transitory Locations, and Census Coverage Measurement.
|
| Department of Commerce | To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously finalize risk mitigation plans detailing actions to address system problems that are identified during testing. |
The Bureau finalized its plan for operational and system failures.
|
| Department of Commerce | To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously establish specific testing metrics and detailed status reports to monitor testing progress and better determine whether corrective action is needed for all key testing activities. |
The bureau has required specific metrics and detailed reports to be presented to management for all operations and systems. In addition, the bureau has developed a testing overview which provides detailed status of testing for all systems.
|
| Department of Commerce | To ensure that testing activities for key systems for the 2010 census are completed, the Secretary of Commerce should require the Director of the Census Bureau to expeditiously designate a senior manager with primary responsibility for monitoring testing and overseeing testing across the Bureau. |
The Bureau developed a document titled "2020 Census Research and Testing (R&T) Phase Test and Evaluation Management Plan (TEMP) dated May 21, 2013 that provides guidance to those groups that are supporting the testing and evaluation of 2020 systems or have a role in enabling them. The TEMP outlines that 2020 R&T system development projects must have a documented Master Test Plan. The plan requirements include a description of the levels of testing to be planned and executed to verify and validate the solution. The plan must include the conditions for determining when testing or evaluation is successfully completed at all levels of testing. Further, the TEMP outlines requirements for preparing test and evaluation products, such as the Integration Test Plan. The Bureau states that the TEMP will evolve as it moves into the System Development Life Cycle phases of the 2020 lifecycle.
|
| U.S. Census Bureau | After the 2010 census, the Bureau should establish comprehensive systems and integration testing guidance to guide future testing of systems. |
As of September 2013, the Bureau issued a Test Evaluation Management Plan (TEMP) to guide systems testing during the 2020 Research and Test Phase. Census has also defined a test metrics process that will begin in Q1 2014 and iteratively improved via lessons learned. According to the Bureau, an update to the SDLC guidelines, called SCLC 2.0, will be baselined in October 2013 and required for all new IT projects at that point in time. After Enterprise SDLC 2.0 is issued, the Bureau will be able to update its TEMP to follow those testing guidelines, including I & T testing guidelines. The Bureau plans for those updates to be available in March 2014.
|
Full Report
Public Inquiries
Topics
Best practicesCensusData collectionDocumentationElectronic data processingInformation technologyOperational testingProduct evaluationProgram managementSchedule slippagesSoftware verification and validationStatistical dataSystem softwareSystems analysisSystems conversionsSystems evaluationSystems integrationSystems integritySystems testingTestingExecutive agency oversight