This is the accessible text file for GAO report number GAO-11-299
entitled 'Electronic Government: National Archives and Records
Administration's Fiscal Year 2011 Expenditure Plan' which was released
on March 4, 2011.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to Congressional Committees:
March 2011:
Electronic Government:
National Archives and Records Administration's Fiscal Year 2011
Expenditure Plan:
GAO-11-299:
GAO Highlights:
Highlights of GAO-11-299, a report to congressional committees.
Why GAO Did This Study:
Since 2001, the National Archives and Records Administration (NARA)
has been working to develop an Electronic Records Archive (ERA) to
preserve and provide access to massive volumes of all types of
electronic records. NARA originally planned to complete the system in
2012, but has repeatedly revised the program schedule and estimated
cost and is now planning to deploy an ERA system with reduced
functionality by the end of fiscal year 2011. As required by the
Consolidated Appropriations Act, 2010, and the Continuing
Appropriations Act, 2011, NARA submitted an expenditure plan to
Congress to support its request for fiscal year 2011 ERA funding. The
legislation also requires that this plan meet six conditions,
including review by GAO. GAO’s objectives in reviewing the fiscal year
2011 plan were to (1) determine whether the plan satisfies legislative
conditions, (2) determine the extent to which NARA has implemented
prior GAO recommendations, and (3) provide any other observations on
the plan or the ERA acquisition. To do this, GAO reviewed the
expenditure plan and other agency documents and interviewed NARA
officials.
What GAO Found:
NARA’s fiscal year 2011 expenditure plan satisfies four of the six
legislative conditions and partially satisfies two. Specifically, it
partially satisfies the condition that NARA meet requirements for
reviewing the progress of capital investments, such as ERA. While NARA
has held regular meetings with senior-level agency management to
review ERA progress, these groups did not document approval of
important schedule and scope changes, and NARA did not validate the
estimated benefits and costs of deployed ERA capabilities. Further,
NARA partially satisfies the condition that the expenditure plan be
approved by NARA and the Office of Management and Budget (OMB). NARA
approved the expenditure plan in October 2010, but the plan was not
approved by OMB. Without approval from OMB, Congress will have limited
assurance of the plan’s reliability and accuracy.
NARA has fully implemented one of GAO’s four prior recommendations and
partially implemented three. It implemented a recommendation to ensure
that ERA’s requirements are managed using a disciplined process by,
for example, developing a process to keep requirements current. NARA
partially implemented three other recommendations. First, to improve
its executive-level oversight, NARA documented meetings to review ERA
progress, but did not document approval of important changes to a
recent phase, or increment, of the system. Second, NARA added
information in its expenditure plan on ERA cost, schedule, and
performance as recommended, but the plan lacks other key information,
such as the estimated costs of an ongoing increment. Third, NARA
documented a plan to strengthen its processes for measuring program
progress, but continues to have weaknesses in this area, including not
accurately portraying ERA program status.
GAO has three observations on the expenditure plan and ERA acquisition:
* The fiscal year 2011 expenditure plan does not provide a reliable
basis for informed investment decision making. For example, NARA’s
cost estimates do not reliably reflect the work to be completed
because of weaknesses in its supporting methodology, and the plan does
not clearly show what functionality is planned to be delivered in the
final year of development, by when, and at what cost.
* NARA’s expenditure plan does not address how remaining multiyear
funds from fiscal year 2010 will be allocated. Specifically, NARA’s
plans for using the remaining $20.1 million are not discussed in the
plan.
* Although NARA recently updated the ERA requirements, the agency has
not yet determined which of the requirements would be addressed before
the end of development in fiscal year 2011 and has not fully
prioritized the requirements to ensure that critical stakeholder needs
will be met.
Without a reliable expenditure plan and adequate management of the ERA
acquisition, it is unclear whether NARA can make substantial progress
in delivering additional system capabilities by the end of fiscal year
2011 to justify its planned investment.
What GAO Recommends:
Congress should consider limiting funding of further ERA development
until NARA addresses weaknesses in its oversight and management of the
acquisition. GAO is also recommending actions for NARA to take to
address these weaknesses. NARA concurred with GAO’s recommendations.
View [hyperlink, http://www.gao.gov/products/GAO-11-299] or key
components. For more information, contact David A. Powner at (202) 512-
9286 or pownerd@gao.gov.
[End of section]
Contents:
Letter:
Conclusions:
Matter for Congressional Consideration:
Recommendations for Executive Action:
Agency Comments:
Appendix I: Briefing for Staff of Congressional Committees:
Appendix II: Comments from the National Archives and Records
Administration:
Appendix III: GAO Contact and Staff Acknowledgments:
Abbreviations:
EOP: Executive Office of the President:
ERA: Electronic Records Archives:
EVM: earned value management:
NARA: National Archives and Records Administration:
OMB: Office of Management and Budget:
OPA: Online Public Access:
SA-CMM: Software Acquisition-Capability Maturity Model:
SEI: Software Engineering Institute:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
March 4, 2011:
The Honorable Richard J. Durbin:
Chairman:
The Honorable Jerry Moran:
Ranking Member:
Subcommittee on Financial Services and General Government:
Committee on Appropriations:
United States Senate:
The Honorable Jo Ann Emerson:
Chairwoman:
The Honorable José E. Serrano:
Ranking Member:
Subcommittee on Financial Services and General Government:
Committee on Appropriations:
House of Representatives:
Since 2001, the National Archives and Records Administration (NARA)
has been working to develop a modern Electronic Records Archive (ERA).
According to NARA, this major information system is estimated to cost
between $456 and $481 million and is intended to preserve and provide
access to massive volumes of all types of electronic records,
independent of their original hardware or software. NARA plans for the
system to manage the full life-cycle of electronic records, from their
ingestion through preservation and dissemination to customers.
Because of the system's complexity, NARA awarded a contract to
Lockheed Martin to develop ERA in five phases, or increments, the
first of which is referred to as the "ERA base." According to NARA,
the ERA base included initial functionality for transferring federal
electronic records into the system and achieved initial operating
capability in June 2008. The second increment includes the Executive
Office of the President (EOP) system or "ERA EOP," and NARA certified
that it reached initial operating capability in December 2008. NARA
originally planned to complete the development of the remaining
increments and achieve full operating capability in March 2012.
However, in acquiring this system, NARA has repeatedly revised the
program schedule and increased the estimated costs. As a result, in
July 2010, the Office of Management and Budget (OMB) directed NARA to
halt ERA development at the end of fiscal year 2011, 1 year earlier
than originally planned.
As mandated by the Consolidated Appropriations Act,[Footnote 1] NARA
is required to submit an expenditure plan before obligating multiyear
funds for the ERA program. On October 1, 2010, the agency submitted
its fiscal year 2011 expenditure plan to support its request for $85.5
million in ERA funding, which includes $61.8 million in multiyear
funds. In the expenditure plan, NARA also included support for
requests at two alternative funding levels--$72.0 million and $61.4
million--based on congressional direction. Subsequently, on October
19, 2010, NARA submitted a summary of its expenditure plan that
included revised requests at $72 million and $65 million funding
levels. According to NARA, both the expenditure plan and the summary
reflect fiscal year 2011 as the final year of ERA development.
As in the previous year, the plan must satisfy six legislative
conditions, including a review by GAO. Our objectives in reviewing the
plan and the summary were to (1) determine whether NARA's fiscal year
2011 expenditure plan satisfies legislative conditions, (2) determine
the extent to which NARA has implemented prior GAO recommendations,
and (3) provide any other observations about the expenditure plan and
the ERA acquisition.
To assess compliance with legislative conditions, we analyzed the
expenditure plan submitted by NARA in October 2010 and reviewed its
budget submission to OMB, along with other program documentation. To
determine the extent to which NARA had implemented our prior
recommendations, we obtained and reviewed agency documents, which
included briefings to senior management on ERA cost, schedule, and
performance; requirements management documentation; and earned value
management processes. To develop observations on the ERA expenditure
plan and acquisition, we analyzed fiscal year 2010 and 2011 cost,
schedule, and planned system functionality information contained in
the expenditure plans and other ERA briefings. In addition, we
reviewed and analyzed requirements management documentation, including
the revised list of ERA requirements. We also interviewed NARA and
Lockheed Martin officials.
To assess the reliability of the data in the expenditure plan, we
interviewed NARA officials to gain an understanding of the data and
discussed our use of the data in this briefing. We concluded that the
data were sufficiently reliable for our purposes. In addition, we
reviewed NARA budget documents, as well as its consolidated financial
statement results for the fiscal year 2010 Performance and
Accountability Report. We did not, however, assess the reliability of
the information in these documents.
We conducted this performance audit from October 2010 to March 2011 at
NARA's College Park, Maryland, location in accordance with generally
accepted government auditing standards. Those standards require that
we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and
conclusions based on our audit objectives. We believe that the
evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
On December 21, 2010, we transmitted the results of our review to the
staffs of the Senate Subcommittee on Financial Services and General
Government, Committee on Appropriations, and the House Subcommittee on
Financial Services and General Government, Committee on
Appropriations. This report transmits the briefing materials we
provided, the matter we suggest Congress consider, and the
recommendations that we made to the Archivist of the United
States.[Footnote 2] The full briefing materials, including details on
our scope and methodology, are reprinted as appendix I.
In summary, we made the following major points:
* NARA's fiscal year 2010 expenditure plan satisfies four of the six
legislative conditions. Specifically, NARA's plan complies with the
agency's enterprise architecture, conforms to the agency's enterprise
life-cycle methodology, reflects certain system acquisition best
practices, and was reviewed by GAO. The expenditure plan partially
satisfies the other two conditions:
- NARA partially satisfies the condition that it develop capital
planning and investment control review processes designed to help
ensure that projects are being implemented at an acceptable cost and
within expected time frames and that they are contributing to
observable improvements in mission performance. While NARA has
conducted regular meetings with senior-level agency management to
review ERA progress, these groups did not document approval of
important schedule and scope changes to a recent ERA increment.
Further, NARA has not conducted post-implementation reviews of
deployed ERA capabilities, such as the recently deployed congressional
records component, to validate estimated benefits and costs. As a
result, NARA has limited ability to ensure that the system is being
implemented at an acceptable cost and within expected time frames, and
that deployed capabilities are contributing to observable improvements
in mission performance.
- NARA partially satisfies the condition that the expenditure plan be
approved by the agency and OMB. NARA approved the expenditure plan in
October 2010, but the plan was not approved by OMB because, according
to NARA officials, OMB will only review the expenditure plan if there
is an existing appropriation with language requiring its review.
Without approval from OMB, Congress will have limited assurance that
the plan is accurate and reliable.
* NARA has fully implemented one of our prior recommendations and
partially implemented three:
- NARA implemented our recommendation to ensure that ERA's
requirements are managed using a disciplined process by updating the
ERA requirements document and developing a process to ensure that the
ERA requirements are kept current following significant changes to the
program, such as significant modifications to the contract.
- NARA has partially implemented three other recommendations. First,
in response to our recommendation that NARA ensure that its investment
review process has adequate executive-level oversight by maintaining
documentation of the results of reviews, NARA documented meetings that
its senior management attended where the ERA program was discussed,
but the agency did not document approval of important changes to a
recent increment through its review process. Second, in response to
our recommendation that NARA provide additional information in the
fiscal year 2011 expenditure plan on what was spent and delivered for
deployed increments, NARA added information on delivered functionality
and additional capabilities planned for fiscal year 2011. However, the
expenditure plan lacked other important information, such as the
estimated fiscal year 2011 costs and the expected completion date of
an ongoing increment. Third, in response to our recommendation to
strengthen its use of earned value processes, NARA documented an
action plan to improve its earned value processes, but continues to
have weaknesses in this area. Among other things, we found that ERA's
earned value data trends do not accurately portray program status, and
future cost overruns would likely be between $195 and $433 million if
the full ERA system were to be completed as originally designed.
[Footnote 3]
* We made three observations related to the ERA program and fiscal
year 2011 expenditure plan and summary:
- NARA's fiscal year 2011 ERA expenditure plan does not provide a
reliable basis for informed investment decision making. Specifically,
NARA's plan does not clearly show what functions have been delivered
to date or how actual costs compare to planned costs. For example, the
plan does not clearly show the system functions of an ongoing
increment that have been delivered and how the actual costs of this
increment compare to the amount NARA planned to spend, as reported in
the fiscal year 2010 expenditure plan. Further, even though NARA plans
to end development in fiscal year 2011, the fiscal year 2011
expenditure plan does not clearly show what functionality is planned
to be delivered, by when, and at what cost during this period. For
example, the plan does not discuss the delivery date of an ongoing
increment, nor the fiscal year 2011 costs associated with completing
this increment. Lastly, NARA's cost estimates, used as the basis for
determining its fiscal year 2011 funding requests, do not reliably
reflect the work to be completed because of weaknesses in the
supporting methodology. For example, while NARA's estimates were based
on information provided by the contractor, including estimates of the
total lines of software code required and the related costs, these
data could not be traced back to their original sources, and NARA did
not validate the information when preparing its cost estimates.
- NARA's fiscal year 2011 expenditure plan does not address how
remaining multiyear funds from fiscal year 2010 will be allocated.
Specifically, in fiscal year 2010, NARA spent $41.7 million of the
$61.8 million in accumulated multiyear funds that it was provided by
Congress to develop the ERA system, which resulted in a balance of
$20.1 million that is available for obligation in fiscal year 2011.
According to NARA officials, the ERA program plans to use the $20.1
million to fund development efforts needed to make the system
available to all federal agencies, as well as to fund other operations
and maintenance costs. However, NARA's plans for using these funds and
their supporting details are not included in the fiscal year 2011
expenditure plan or the revised summary. Until NARA specifies how it
will use the multiyear funds remaining from fiscal year 2010,
appropriators will lack information important for evaluating NARA's
fiscal year 2011 request.
- NARA has not yet determined which ERA requirements will be
implemented in fiscal year 2011, nor fully prioritized the remaining
requirements. In June 2010, we reported that NARA had developed and
documented a set of 853 high-level business requirements for ERA and
planned to complete about 57 percent of them by the end of fiscal year
2010. Subsequently, in July 2010, NARA updated its requirements
document. According to NARA, the original 853 requirements were
decomposed to a more detailed set of 1,577 requirements, portions of
which were allocated to Increments 1 through 5, deferred to post-
Increment 5 (i.e., post-fiscal year 2011), or removed from the scope
of the ERA program. However, NARA could not determine which of the
remaining requirements would be addressed before the end of fiscal
year 2011 because, according to NARA officials, the requirements were
subject to ongoing negotiations with the development contractor. This
uncertainty is further exacerbated because NARA has not fully
prioritized its requirements. Maintaining a prioritized set of system
requirements can better ensure that the requirements that are most
critical to the customer and other stakeholders are addressed quickly.
Until the ERA program fully prioritizes the ERA requirements to be
completed during fiscal year 2011, it will be unclear what system
development work is planned to be completed and whether it will result
in functionality that is most critical to NARA's customers and other
stakeholders.
Conclusions:
While NARA's fiscal year 2011 expenditure plan meets four of the six
legislative conditions, the lack of critical capital planning and
oversight steps--including documentation demonstrating approval of
significant changes to a recent ERA increment, post-implementation
reviews of deployed capabilities, and OMB's approval of the
expenditure plan--limits NARA's ability to ensure that the system is
being implemented at an acceptable cost and within expected time
frames and contributes to observable improvements in mission
performance. These issues are further exacerbated by the agency's
partial implementation of several open GAO recommendations, such as
those related to improving investment oversight and earned value
processes. With significant weaknesses in many basic oversight and
management processes, as well as continued delays in completing
Increment 3, NARA's ability to make significant development progress
in the remainder of the fiscal year will be challenged.
In addition, without a reliable ERA expenditure plan, NARA has not
provided adequate information to assist congressional oversight and
informed decision making related to the use of appropriated funds.
When these weaknesses are combined with the lack of prioritization of
the remaining requirements under negotiation for fiscal year 2011,
Congress has little assurance that additional funds allocated to ERA
will result in significant benefits to potential users. With OMB's
direction to stop development after 2011, it is unclear whether NARA
will be able to effectively address the full range of weaknesses we
identified and still have adequate time to complete significant
development efforts.
Matter for Congressional Consideration:
The identified deficiencies in NARA's expenditure plan and management
of the ERA acquisition make it unclear whether NARA can make
substantial progress in delivering additional ERA system capabilities
that justify its planned investment by the end of fiscal year 2011. As
such, we suggest that Congress consider employing an accountability
mechanism that limits NARA's ability to use funds appropriated for ERA
development until NARA implements an adequate capital planning and
investment control process, updates its expenditure plan to clearly
describe what system capabilities and benefits are to be delivered in
fiscal year 2011, and establishes an associated set of prioritized
system requirements and adequate earned value reporting.
Recommendations for Executive Action:
We are recommending that the Archivist of the United States
immediately take the following two actions while the current system
development contract is active:
* Report to Congress on the specific outcomes to be achieved with the
balance of any previous multiyear funds in fiscal year 2011.
* Ensure that the ERA requirements planned for fiscal year 2011 are
fully prioritized so that those most critical to NARA's customers and
other stakeholders are addressed.
To ensure that any future efforts are completed within reasonable
funding and time constraints, we are recommending that the Archivist
of the United States take the following four actions:
* Ensure that significant changes to ERA's program's cost, schedule,
and scope are approved through NARA's investment review process.
* Conduct post-implementation reviews of deployed ERA capabilities to
validate estimated benefits and costs.
* Submit ERA expenditure plans to OMB for review and approval prior to
submitting to Congress.
* Update the ERA Requirements Management Plan and related guidance to
mandate requirements prioritization throughout the project's life-
cycle.
Agency Comments:
In written comments on a draft of this report, which are reprinted in
appendix II, the Archivist of the United States concurred with our six
recommendations. Specifically, he stated that NARA has sufficiently
addressed the first two recommendations. He further stated that NARA
would be unable to address the final four recommendations in a near-
term action plan since those were specific to a future ERA development
effort. The Archivist also noted that NARA is developing an addendum
to the fiscal year 2011 expenditure plan to provide updated
information on ERA requirements, costs, and the schedule of software
releases.
We are sending copies of this report to the Archivist of the United
States. The report will also be available at no charge on the GAO Web
site at [hyperlink, http://www.gao.gov].
If you or your staffs have any questions concerning this report,
please contact me at (202) 512-9286 or by e-mail at pownerd@gao.gov.
Contact points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff who
made major contributions to this report are listed in appendix III.
Signed by:
David A. Powner:
Director, Information Technology Management Issues:
[End of section]
Appendix I: Briefing for Staff of Congressional Committees:
Electronic Government: Review of the National Archives and Records
Administration's Fiscal Year 2011 Electronic Records Archive
Expenditure Plan:
Briefing for Staff Members of the Subcommittee on Financial Services
and General Government, Committee on Appropriations, U.S. Senate
and the Subcommittee on Financial Services and General Government,
Committee on Appropriations, House of Representatives:
December 21, 2010:
Contents:
* Introduction;
* Objectives, Scope, and Methodology;
* Results in Brief;
* Background;
* Results;
* Conclusions;
* Matter for Congressional Consideration;
* Recommendations for Executive Action;
* Agency Comments and Our Evaluation.
Introduction:
Since 2001, the National Archives and Records Administration (NARA)
has been working to develop a modern Electronic Records Archive (ERA).
According to NARA, this major information system is estimated to cost
between $456 and $481 million and is intended to preserve and provide
access to massive volumes of all types of electronic records,
independent of their original hardware or software. NARA plans for the
system to manage the full life cycle of electronic records, from their
ingestion through preservation and dissemination to customers. It is
to consist of:
* infrastructure elements, including hardware and operating systems;
* business applications that will support the transfer, preservation,
dissemination, and management of all types of records; and;
* a means for public access via the Internet.
Because of the system's complexity, NARA awarded a contract to
Lockheed Martin to develop ERA in five phases, or increments, the
first of which is referred to as the "ERA base." According to NARA,
the ERA base included initial functionality for transferring federal
electronic records into the system and achieved initial operating
capability in June 2008. The second increment includes the Executive
Office of the President (EOP) system or "ERA EOP," and NARA certified
that it reached initial operating capability in December 2008. NARA
originally planned to complete the development of the remaining
increments and achieve full operating capability in March 2012.
However, in acquiring this system, NARA has repeatedly revised the
program schedule and increased the estimated costs. As a result, in
July 2010, the Office of Management and Budget (OMB) changed the
direction of the program, and NARA is now planning to deploy an ERA
system with reduced functionality by the end of fiscal year 2011.
As mandated by the Consolidated Appropriations Act,[Footnote 4] NARA
is required to submit an expenditure plan before obligating multi-year
funds for the ERA program. The plan must satisfy the following
legislative conditions:
* meet the capital planning and investment control review requirements
established by OMB;
* comply with the agency's enterprise architecture;
* conform to the agency's enterprise life-cycle methodology;
* comply with the acquisition rules, requirements, guidelines, and
system acquisition management practices of the federal government;
* be approved by the agency and OMB; and;
* be reviewed by GAO.
On October 1, 2010, the agency submitted its fiscal year 2011
expenditure plan to the relevant House and Senate appropriations
committees to support its request for $85.5 million in ERA funding. Of
this amount, $61.8 million consists of multi-year funds allocated to
ERA. In the expenditure plan, NARA also included support for requests
at two alternative funding levels—$72.0 million and $61.4 million—
based on congressional direction. Subsequently, on October 19, 2010,
NARA submitted a summary of its expenditure plan to the appropriations
committees that included revised requests at $72 million and $65
million funding levels. According to NARA, both the expenditure plan
and the summary reflect fiscal year 2011 as the final year of ERA
development.
Objectives, Scope, and Methodology:
Our objectives were to:
* determine whether NARA's fiscal year 2011 expenditure plan satisfies
legislative conditions;
* determine the extent to which NARA has implemented prior GAO
recommendations: and;
* provide any other observations about the expenditure plan and the Mk
acquisition.
To assess compliance with legislative conditions, we:
* obtained and reviewed NARA's briefings to senior management and OMB.
minutes of Information Technology Executive Committee meetings,
information related to post-implementation reviews of deployed ERA
functionality, and the fiscal year 2011 exhibit 300 submissions
[Footnote 5] to OMB to determine the extent to which the agency has
complied with OMB's capital planning and investment control
requirements;
* obtained and reviewed data on NARA's enterprise architecture to
determine the status of the agency's enterprise architecture efforts;
* reviewed NARA's ERA system life-cycle processes, which include
processes for managing system investments, configuration management,
and managing risks, and reviewed related agency documentation
describing how these processes were implemented for the ERA project,
such as minutes of oversight boards and risk watchlists to determine
if ERA conforms with NARA's enterprise life-cycle methodology;
* obtained and reviewed NARA documentation on the ERA acquisition,
which included internal assessments of ERA compliance with the
Software Engineering Institute's Capability Maturity Model, to
determine if NARA follows system acquisition best practices;
* obtained and reviewed information related to the agency's and OMB's
approval of the expenditure plan; and;
* reviewed and analyzed the fiscal year 2011 expenditure plan
submitted by NARA on October 1, 2010, and a summary of the expenditure
plan with revised information submitted by NARA on October 19, 2010.
To determine the extent to which NARA has implemented our prior
recommendations, we obtained and reviewed agency documents, which
include briefings to senior management on ERA cost, schedule, and
performance: requirements management documentation; earned value
management processes; and the expenditure plan.
To develop observations on the ERA expenditure plan and acquisition,
we analyzed fiscal year 2010 and 2011 cost, schedule, and planned
system functionality information contained in the expenditure plans
and other ERA briefings. We reviewed and analyzed requirements
management documentation, including the revised list of ERA
requirements. We also interviewed NARA and Lockheed Martin officials.
To assess the reliability of the data in the expenditure plan, we
interviewed NARA officials in order to gain an understanding of the
data and discussed our use of the data in this briefing. We concluded
that the data were sufficiently reliable for our purposes. In
addition, we reviewed NARA budget documents as well as its
consolidated financial statement results for the fiscal year 2010
Performance and Accountability Report, We did not, however, assess the
reliability of the information in these documents.
We conducted this performance audit from October 2010 to December 2010
in accordance with generally accepted government 'auditing standards,
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
Results in Brief:
NARA's fiscal year 2011 expenditure plan satisfies four of the six
legislative conditions contained in the 2010 Consolidated
Appropriations Act. Specifically. NARA's plan complies with the
agency's enterprise architecture. conforms to the agency's enterprise
life-cycle methodology, reflects certain system acquisition best
practices. and was reviewed by GAO. The expenditure plan partially
satisfies the other two conditions.
* NARA partially satisfies the condition that it develop capital
planning and investment control review processes designed to help
ensure that projects are being implemented at an acceptable cost and
within expected time frames. While NARA has conducted regular meetings
with senior-level agency management to review ERA progress, these
groups did not document approval of important schedule and scope
changes to a recent ERA increment, Further, NARA has not conducted
post-implementation reviews of deployed ERA capabilities, such as the
recently deployed congressional records component, to validate
estimated benefits and costs. As a result, NARA has limited ability to
ensure that the system is being implemented at an acceptable cost and
within expected time frames, and that deployed capabilities are
contributing to observable improvements in mission performance.
* NARA partially satisfies the condition that the expenditure plan be
approved by the agency and OMB. NARA approved the expenditure plan in
October 2010, but the plan was not approved by OMB because, according
to NARA officials, OMB will only review the expenditure plan if there
is an existing appropriations bill with language requiring its review,
Without approval from OMB, Congress will have limited assurance that
the plan is accurate and reliable.
NAHA fully implemented one of our prior recommendations and partially
implemented three:
* NARA implemented our recommendation to ensure that ERA's
requirements are managed using a disciplined process by updating the
ERA requirements document and developing a process to ensure that the
ERA requirements are kept current following significant changes to the
program, such as significant modifications to the contract.
* NARA has partially implemented three other recommendations. First,
in response to our recommendation that NARA ensure that its investment
review process has adequate executive-level oversight by maintaining
documentation of the results of reviews, NARA documented meetings that
its senior management attended where the ERA program was discussed,
but the agency did not document approval of important changes to a
recent increment through its review process. Second, in response to
our recommendation that NARA provide additional information in the
fiscal year 2011 expenditure plan on what was spent and delivered for
deployed increments, NARA added information on delivered functionality
and additional capabilities planned for fiscal year 2011, However, our
review of the expenditure plan showed that it lacked other important
information, such as the estimated fiscal year 2011 costs and expected
completion date of an ongoing increment. Third, in response to our
recommendation to strengthen its use of earned value processes, NARA
documented an action plan to improve its earned value processes, but
continues to have weaknesses in this area. Among other things. we
found that ERA's earned value data trends do not accurately portray
program status and future cost overruns would likely be between $195
and $433 million if the full ERA system is completed as originally
designed,
We have three observations related to the ERA program and fiscal year
2011 expenditure plan:
* NARA's fiscal year 2011 ERA expenditure plan does not provide a
reliable basis for informed investment decision-making. Specifically,
we have identified the following weaknesses:
- NARA's plan does not clearly show what functions have been delivered
to date or how actual costs compare to planned costs. For example, the
plan does not clearly show the system functions of an ongoing
increment that have been delivered and how the actual costs of this
increment compare to the amount NARA planned to spend, as reported in
the fiscal year 2010 expenditure plan.
- NARA's plan does not clearly show what functionality is planned to
be delivered in the final year of development, by when, and at what
cost. For example. the plan does not discuss the delivery date of an
ongoing increment, nor the fiscal year 2011 costs associated with
completing this increment.
- NARA's cost estimates. used as the basis for determining its fiscal
year 2011 funding requests, do not reliably reflect the work to be
completed because of weaknesses in the supporting methodology. For
example, while NARA's estimates were based on information provided by
the contractor, including estimates of the total lines of software
code required and the related costs, these data could not be traced
back to their original sources and NARA did not validate the
information when preparing its cost estimates.
NARA officials attributed many of the issues we identified with the
reliability of the expenditure plan, in part, to ongoing program
uncertainty due to negotiations with its development contractor.
Without more specific information on what functionality will be
delivered before the completion of development at the end of fiscal
year 2011 and more reliable cost estimates. Congress will continue to
lack important information for evaluating NARA's funding request and
measuring overall program progress.
* NARA's fiscal year 2011 expenditure plan does not address how
remaining multi-year funds from fiscal year 2010 will be allocated.
Specifically, in fiscal year 2010, NARA spent $41.7 million of the
$61,8 million in multi-year funds that it was provided by Congress to
develop the ERA system, which resulted in a balance of $20.1 million
that is available for obligation in fiscal year 2011. According to
NARA officials, the ERA program plans to use the $20.1 million to fund
development efforts needed to make the system available to all federal
agencies. as well as to fund other operations and maintenance costs.
However, NARA's plans for using these funds and their supporting
details are not included in the fiscal year 2011 expenditure plan or
the revised summary. Until NARA specifies how it will use the multi-
year funds remaining from fiscal year 2010, appropriators will lack
information important for evaluating NARA's fiscal year 2011 request.
* NARA has not yet determined which ERA requirements will be
implemented in fiscal year 2011, nor fully prioritized the remaining
requirements, In June 2010. we reported that NARA had developed and
documented a set of 853 high-level business requirements for ERA and
planned to complete about 57 percent of them by the end of fiscal year
2010. Subsequently, in July 2010, NARA updated its requirements
document, According to NARA, the original 853 requirements were
decomposed to a more detailed set of 1,577 requirements, portions of
which were allocated to Increments I through 5, deferred to post-
Increment 5 (i.e., post-fiscal year 2011), or removed from the scope
of the ERA program. However, NARA could not determine which of the
remaining requirements would be addressed before the end of fiscal
year 2011 because, according to NARA officials, the requirements were
subject to ongoing negotiations with the development contractor. This
uncertainty is further exacerbated because NARA has not fully
prioritized its requirements. Maintaining a prioritized set of system
requirements can better ensure that the requirements that are most
critical to the customer and other stakeholders are addressed quickly.
Until the ERA program fully prioritizes the ERA requirements to be
completed during fiscal year 2011, it will be unclear what system
development work is planned to be completed and whether it will result
in functionality that is most critical to NARA's customers and other
stakeholders.
The identified deficiencies in NARA's expenditure plan and management
of the ERA acquisition make it unclear whether NARA can make
substantial progress in delivering additional ERA system capabilities
by the end of fiscal year 2011 that justify its planned investment. As
such, we suggest that Congress consider employing an accountability
mechanism that limits NARA's ability to use funds appropriated for ERA
development until NARA implements an adequate capital planning and
investment control process. updates its expenditure plan to clearly
describe what system capabilities and benefits are to be delivered in
fiscal year 2011, and establishes an associated set of prioritized
system requirements and adequate earned value reporting, We are also
making specific recommendations to NARA to improve its oversight and
management of the ERA acquisition while the current system development
contract is active and in preparation for any future ERA development
efforts.
We provided a draft of this briefing for review and comment to the
Archivist of the United States. In oral comments, the Archives' GAO
liaison stated that NARA generally agreed with our findings, NARA did
not comment on cur recommendations,
Background:
The ability to find, organize, use, share, appropriately dispose of,
and save records-—the essence of records management-—is vital for the
effective functioning of the federal government. In the wake of the
transition from paper-based to electronic processes, records are
increasingly electronic, and the volumes of electronic records
produced by federal agencies are vast and rapidly growing, providing
challenges to NARA as the nation's record keeper and archivist.
Besides sheer volume, other factors contributing to the challenge of
electronic records include their complexity and their dependence on
software and hardware. Specifically, the computer operating systems
and the hardware and software that are used to create electronic
documents can become obsolete. If they do, they may leave behind
records that cannot be read without the original hardware and
software. Further, the storage media for these records are affected by
both obsolescence and decay. Media may be fragile, have limited shelf
life, and become obsolete in a few years. For example, few computers
today have disk drives that can read information stored on 8-or 5v4-
inch diskettes, even if the diskettes themselves remain readable.
Another challenge is the growth in electronic presidential records.
The Presidential Records Act gives the Archivist of the United States
responsibility for the custody, control, and preservation of
presidential records upon the conclusion of a President's term of
office.[Footnote 6] The act states that the Archivist has an
affirmative duty to make such records available to the public as
rapidly and completely as possible consistent with the provisions of
the act.
In response to these widely recognized challenges, the Archives began
a research and development program to develop a modern archive for
electronic records. In 2001. NARA hired a contractor to develop
policies and plans to guide the overall acquisition of an electronic
records system. In December 2003, the agency released a request for
proposals for the design of ERA. In August 2004. NARA awarded two firm-
fixed-price[Footnote 7] contracts for the design phase, totaling about
$20 million—one to Harris Corporation and the other to Lockheed Martin
Corporation. On September 8, 2005, NARA announced the selection of
Lockheed Martin Corporation to build the ERA system. The total value
of the contract with Lockheed through 2012 is about $317 million,
which includes provisions for award fees based on how well the
contractor meets technical, program management, and cost-control
criteria. As of the end of fiscal year 2010, NARA has paid Lockheed
approximately $199,5 million, including $181.6 million for development
and $17.9 million for operations and maintenance.
The final operational ERA system is to consist of six key functions:
* Ingest enables the transfer of electronic records from federal
agencies.
* Archival Storage enables stored records to be managed in a way that
guarantees their integrity and availability.
* Records Management supports scheduling,[Footnote 8] appraisal,
[Footnote 9] description, and requests to transfer custody of all
types of records, as well as ingesting and managing electronic
records, including the capture of selected records data (such as
origination date, format and disposition).
* Preservation enables secure and reliable storage of files in formats
in which they were received, as well as creating backup copies for off-
site storage.
* Local Services and Control regulates how the ERA components
communicate with each other, manages internal security, and enables
telecommunications and system network management.
* Dissemination enables users to search descriptions and business data
about all types of records, and to search the content of electronic
records and retrieve them.
NARA currently plans to deliver ERA system capabilities in five
separate increments, Each increment involves multiple releases that
are to deliver specific functionality. Below is a summary of the
specific releases delivered or planned for delivery in each increment:
* Increment 1 was deployed in two releases. Release 1 established the
ERA base system—the hardware, software, and communications needed to
deploy the system, Release 2 enabled functional archives with the
ability to preserve electronic data in their original format, enable
disposition agreements and scheduling, and receive unclassified and
sensitive data from four federal agencies; according to NARA
officials, this increment was certified as complete in June 2008.
However, additional enhancements were made to Increment 1. release 2
that were completed in March 2010.
* Increment 2 includes the EOP system, which was designed to handle
records from the Executive Office of the President. This increment was
to include the content searching and management for special access
requests.[Footnote 10] The EOP system was certified for initial
operating capability in December 2008. However, NARA did not finish
ingesting the presidential records it received from the George W. Bush
administration until September 2009, 9 months after initial operating
capability.
* Increment 3 is expected to include the following:
- Storage and access capabilities for electronic records of the
Congress. NARA received authority to operate the first release of
Increment 3—-the congressional records component-—in January 2010.
- Public access to provide the public with tools needed to search and
access electronic records, including descriptions of archival records,
copies of electronic records, and digital surrogates of non-electronic
records. NARA completed a prototype of the Online Public Access (CPA)
interface in April 2010 and plans to release the system for public use
by December 2010.
- Planning for preservation to include development of a preservation
framework prototype. According to NARA, this prototype was completed
in April 2010 and included functionality for file transformations and
capturing preservation metadata, such as the relationship between
different file versions.
- Upgrades to the ERA base system to, among other things, integrate
additional commercial-off-the-shelf products. Development and
contractor testing of these upgrades were originally planned to be
completed in March 2010 but, according to NARA officials, were not
completed until November 2010—a slip of 8 months, These delays were
due, in part, to unanticipated additional development work, as well as
issues identified during testing that required additional work,
According to NARA officials, as of December 2010, these upgrades were
undergoing additional government testing.
NARA planned to complete Increment 3 in June 2010: however, due to the
delays in upgrading the ERA base system, this milestone was postponed.
According to NARA officials, Increment 3 is now expected to be
delivered in January 2011-7 months later than planned.
* Increment 4 was originally planned to build upon the base
architecture delivered as part of Increment 3 and insert newly
available technology, particularly for preservation capabilities.
Specifically, in its fiscal year 2010 expenditure plan, NARA stated
that Increment 4 would provide capabilities for, among other things,
managing access-restricted records, managing Freedom of Information
Ace cases,[Footnote 11] and redacting records and managing redacted
versions of records. However, NARA reduced the time allocated to this
increment from 12 months to 3 months and revised the scope. In
particular, all functionality planned to be implemented during this
increment was deferred to Increment 5 or removed. According to NARA,
the revised scope of this increment included, among other things,
supporting the OPA and preservation framework prototypes, and early
requirements work for Increment 5. This increment was completed in
September 2010.
* Increment 5 is expected to expand on system capabilities implemented
in the prior increments and address functionality deferred from
previous increments, According to NARA, its development priorities for
fiscal year 2011 include modifications to the ERA system needed to
store classified records from the Iraq conflict, enable the faster
release of George W. Bush presidential records when they are legally
open to Freedom of Information Act requests in 2014, and store
electronic records from the 2010 Census. NARA is currently negotiating
the work to be completed during this increment with the development
contractor.
According to NARA, as of October 2010, the cost of the ERA program
through fiscal year 2011 was between $456 million and $481 million,
[Footnote 12] which includes not only the development contract costs,
but also program management, research and development, and program
office support, among other things.
Table 1 shows reported spending from the program's inception to the
end of fiscal year 2010.
Table 1: Summary of ERA Spending from Fiscal Year 2002 through Fiscal
Year 2010:
Project category: Development Contract-—Lockheed Martin;
Spending: $195.5 million.
Project category: System Analysis and Design Contracts-—Lockheed
Martin and Harris Corporation;
Spending: $40.8 million.
Project category: Program Management;
Spending: $47.8 million.
Project category: Program Office Support Team;
Spending: $34.2 million.
Project category: Research and Development;
Spending: $27.2 million.
Project category: Integrated Deployment and Support;
Spending: $16.0 million.
Project category: Independent Verification and Validation;
Spending: $8.8 million.
Project category: Security;
Spending: $0.2 million.
Project category: End of Year Balance;
Spending: $0.4 million.
Project category: Adjustment[A];
Spending: $0.0 million.
Project category: Total;
Spending: $374.9 million[B].
Source: GAO analysis of NARA data.
[A]Recoveries at prior year funds, adjustments to obligations
incurred, obligations against prior years, and carryover funds
expiring at the end of fiscal year 2010.
[B]Total number may not equal the sum of individual items due to
rounding.
[End of table]
In fiscal year 2010, NARA spent about $41.7 million of the $61.8
million in multi-year funds that it was provided by Congress to
develop the ERA system, and plans to use the remaining balance of
$20.1 million in fiscal year 2011.
NARA's estimated ERA obligations for fiscal year 2011 in the
President's budget request, including both single-year and multi-year
funds, are $85.5 million. Table 2 shows how NARA plans to distribute
funds across the ERA program in fiscal year 2011.
Table 2: Summary of NARA% Fiscal Year 2011 Estimated Obligations for
ERA:
Project category: Development Contract;
Description: Activities performed under the EPA system acquisition
contract with Lockheed Martin;
Estimated obligations: $61.8 million.
Project category: Program Management;
Description: Salaries and benefits, supplies, equipment, and
telecommunications;
Estimated obligations: $10.5 million.
Project category: Program Office Support Team;
Description: Labor, contracts, and materials to support ERA program
management;
Estimated obligations: $5.4 million.
Project category: Research and Development;
Description: Research performed with other agencies;
Estimated obligations: $4.5 million.
Project category: Independent Verification and Validation[A];
Description: Verification and validation activities;
Estimated obligations: $1.8 million.
Project category: Integrated Deployment and Support;
Description: Interagency agreements for ERA facilities, including
Allegany Ballistics Lab at Rocket Center, West Virginia;
Estimated obligations: $1.5 million.
Project category: Security;
Description: Security clearances for government personnel;
Estimated obligations: $0.1 million.
Project category: Total;
Estimated obligations: $85.5 million[B].
Source: GAO analysis of NARA's fiscal year 2011 expenditure plan.
[A] NARA contracted with Northrop Grumman to perform independent
verification and validation on policies and plans produced by the ERA
program and contract deliverables produced by Lockheed Martin.
[B] Total includes estimated obligations and does not include the
balance of available multi-year funds from previous years. Total
number may not equal the sum of individual items due to rounding.
[End of table]
NARA's fiscal year 2011 expenditure plan also included estimated ERA
obligations at two additional funding levels—$72,0 million and $61.4
million—based on congressional direction. Table 3 shows how NARA plans
to distribute funds across the ERA program at these funding levels.
Table 3: Summary of NARA's Fiscal Year 2011 Estimated Obligations for
ERA at Alternate Funding Levels:
Project category: Development Contract;
Description: Activities performed under the ERA system acquisition
contract with Lockheed Martin
Estimated obligations at $72.0 million funding level: $56.0 million;
Estimated obligations at $61.4 million funding level: $45.4 million.
Project category: Program Management;
Description: Salaries and benefits, supplies, equipment, and
telecommunications
Estimated obligations at $72.0 million funding level: $9.5 million;
Estimated obligations at $61.4 million funding level: $9.5 million.
Project category: Program Office Support Team;
Description: Labor, contracts, and materials to support EPA program
management
Estimated obligations at $72.0 million funding level: $3.4 million;
Estimated obligations at $61.4 million funding level: $3.4 million.
Project category: Research and Development;
Description: Research performed with other agencies
Estimated obligations at $72.0 million funding level: $1.5 million;
Estimated obligations at $61.4 million funding level: $1.5 million.
Project category: Integrated Deployment and Support;
Description: Interagency agreements for ERA facilities
Estimated obligations at $72.0 million funding level: $1.5 million;
Estimated obligations at $61.4 million funding level: $1.5 million.
Project category: Security;
Description: Security clearances for government personnel
Estimated obligations at $72.0 million funding level: $0.1 million;
Estimated obligations at $61.4 million funding level: $0.1 million.
Project category: Total;
Estimated obligations at $72.0 million funding level: $72.0[A million;
Estimated obligations at $61.4 million funding level: $61.4[A] million.
Source: GAO analysis of NARA's fiscal year 2011 expenditure plan.
Note: For both alternate funding levels, NARA estimated obligations of
$30,000 towards independent Verification and Validation, but
this Is not shown In the table because the amounts are equal to zero
when converted Into millions and rounded to one decimal place.
[A] Totals Include estimated obligations only and do not include the
balance of available multi-year funds from previous years.
[End of table]
Prior GAO Work:
Since 2002, we have issued several reports on ERA and its development.
[Footnote 13] In November 2009, we testified that NARA had completed
two of its five planned increments. but experienced delays and cost
overruns. and several functions planned for the system's initial
release were deferred.[Footnote 14] We further testified that although
NARA initially planned for the system to be capable of ingesting
federal and presidential records in September 2007, the two system
increments to support those records did not achieve initial operating
capability until June 2008 and December 2008. respectively. In
addition, NARA reportedly spent about $80 million on the base
increment, compared to its planned cost of about $60 million.
In our review[Footnote 15] of NARA's fiscal year 2010 expenditure
plan, we made three observations about the expenditure plan and ERA
acquisition:
* The ERA system was experiencing cost increases, and system
development was behind schedule. Since 2007, the ERA estimated life-
cycle cost had increased from about $531 million to $567.4 million, or
about 7 percent. In addition, the planned completion dates for two of
the increments were about 1 year later than milestones established in
program planning documents.
* NARA had not fully detailed the system capabilities to be included
in the final two ERA increments. While the fiscal year 2010
expenditure plan provided a high-level description of these
capabilities, it did not have fully defined plans, including specific
dates for completion and what capabilities were to be delivered.
* NARA had not effectively defined or managed ERA's system
requirements. Although NARA developed a baseline set of system
requirements for ERA, it lacked firm plans to implement about 43
percent of them. In addition. NARA had not updated its requirements
document to reflect changes as the project progressed and did not
conduct system requirements reviews as called for in its requirements
management plan.
Accordingly, we made recommendations to NARA to improve its investment
review process and ensure that ERA's requirements are managed using a
disciplined process.
More recently, we reported[Footnote 16] on weaknesses in NARA's
information security controls. For example, while the agency has
maintained and tracked configuration changes for ERA. it has not
consistently documented the status of those changes. As a result, NARA
faces an increased risk that the integrity of the ERA system could be
compromised. We recommended that NARA take steps to address its
information security-related weaknesses.
In addition, the Senate Committee on Homeland Security and
Governmental Affairs and the House Committee on Oversight and
Government Reform asked us to evaluate whether NARA is adequately
using earned value management (EVM)[Footnote 17] techniques to manage
the ERA acquisition, as well as to evaluate ERA's earned value data to
determine the program's cost and schedule performance. Among other
things, we found that NARA has not yet fully implemented key practices
for establishing a comprehensive EVM system. Further, we found that
ERA's earned value data trends do not accurately portray program
status and that if the full ERA system is completed as originally
designed, ERA's costs could be between $195 to $433 million (34 to 76
percent) higher than NARA's previous estimate of $567 million.
[Footnote 18] We are recommending, among other things. that NARA
establish a comprehensive plan for all remaining work, including new
life-cycle cost and schedule estimates for the program; improve the
accuracy of earned value performance reports; and engage executive
leadership in correcting negative trends. We plan to issue a report on
this work in January 2011.
Citing our concerns related to the cost, schedule, and performance of
the ERA system. OMB recently directed changes to the ERA program.
Specifically. in July 2010, OMB directed NARA to halt all development
activities by the end of fiscal year 2011 and develop an action plan
to address our finding on the lack of defined system functionality for
the final two increments of the ERA program and the need for improved
strategic planning. In response. NARA plans to finish ERA development
in 2011 with reduced functionality, as reflected in its fiscal year
2011 expenditure plan. However. as of December 2010. the revised scope
of the program had not yet been determined because of ongoing
negotiations with the contractor regarding the development activities
to be completed in fiscal year 2011.
Despite changes in program direction, the Archivist noted that the
essential goals of ERA would remain unchanged. He stated that
beginning in fiscal year 2012. ERA would fully support the transferal
of electronic records to an archival repository, as well as access and
preservation of electronic archival records. To do this, the Archivist
stated that the agency would work on those elements determined to be
the highest priorities in fiscal year 2011. According to NARA, these
changes may lead to a second phase of the ERA development in the
future.
Results: Legislative Conditions
Objective 1: NARA's expenditure plan satisfies four of the fiscal year
2011 legislative conditions and partially satisfies two.
Table 4: Fiscal Year 2011 Expenditure Plan Provisions for Satisfying
Legislative Conditions:
Legislative condition: 1. Meets OMB capital planning and investment
control requirements;
Status: Partially satisfied;
Expenditure plan provisions: OMB requires agencies to develop capital
planning and investment control review processes that help ensure that
projects are being implemented at acceptable cost and within
reasonable and expected time frames and that they are contributing to
observable improvements in mission performance. In order to do this,
agencies should establish an oversight entity that periodically
reviews capital assets (e.g., the ERA system) to determine how mission
requirements might have changed whether the asset continues to fulfill
mission requirements and deliver intended benefits to the agency and
customers. Further, agencies should indicate (i.e., document) that the
Investment has been reviewed and approved by the responsible oversight
entity. NARA partially met this condition; it has established groups
to oversee ERA's progress and provided them with regular briefings.
However, NARA did not document approval of important schedule and
scope changes to the ERA program. Specifically, Increment 4 was
originally planned to build upon the base architecture and insert
newly available technology: however. NARA reduced the time allocated
to this increment from 12 months to 3 months and deferred originally
planned functionality to Increment 5 or removed it. Approval of these
changes was not documented through NARA's Investment review process
because, according to NARA officials, the agency Instead relies on
briefings to senior management to proceed with these changes. Further,
NARA has not conducted any post-implementation reviews of deployed ERA
capabilities, including the system's first two increments, as well as
the first release of Increment 3—the congressional records component.
According to NARA officials, these reviews were not conducted because
the agency's IT policy does not require it and the ERA program instead
relies on system acceptance testing to ensure that a system performs
in accordance with stakeholder expectations. Without ensuring that key
schedule and scope changes are approved and post-implementation
reviews of deployed capabilities are conducted, NARA has limited
ability to ensure that the system Is being implemented at acceptable
cost and within expected time frames, and that deployed capabilities
are contributing to observable improvements in mission performance.
Legislative condition: 2. Complies with NARA's enterprise architecture;
Status: Satisfied;
Expenditure plan provisions: OMB requires NARA to include ERA in its
agency-level enterprise architecture, which is updated on a yearly
basis. NARA's most recently approved agencywide enterprise
architecture-—version 5.5 updated in May 2009--includes ERA and
consists of several component architectures,
including business, data, systems, application, operations,
performance, and information technology security architectures. In May
2014, NARA released its annual update to its enterprise architecture,
version 5.6; however, it has not yet been reviewed by OMB. In
addition, OMB requires that any major IT Investment be mapped to and
support the Federal Enterprise Architecture. The business case for the
investment must also demonstrate the relationship between the
investment and the business, performance, data services, application,
and technology layers of the agency's architecture. NARA's business
case for the ERA system certified compliance with these requirements
and was approved by NARA's Acting Chief Information Officer at the
time.
Legislative condition: 3. Conforms with NARA's enterprise life-cycle
methodology;
Status: Satisfied;
Expenditure plan provisions: The ERA project conforms with NARA's life-
cycle methodology. For example, the expenditure plan includes
descriptions of the incremental approach the agency has adopted tor
acquiring ERA and its management of program risks. In particular, the
risk management methodology calls for the agency to identify and
categorize risks, qualify the probabilities and consequences of the
risks, specify a strategy to mitigate each risk, communicate risk
status, and formulate actions needed to mitigate the risk. NARA
manages risks using an agency-level risk review board and a program-
level risk review board. In addition, the ERA program office produces
monthly reports that include top identified risks and specify
associated mitigation strategies. The office also generates reports of
pending or active risks from its risk management database that specify
the probability and consequences of identified risks. Further, risk
status is communicated to senior NARA management on a monthly basis,
to OMB through ad-hoc technical review sessions, and to Congress on a
quarterly basis. The quarterly reports also identify executive actions
needed to mitigate risks.
Legislative condition: 4. Complies with the acquisition rules,
requirements, guidelines, and systems acquisition management practices
of the federal government;
Status: Satisfied[A];
Expenditure plan provisions: NARA satisfied this provision by
implementing key processes that reflect best practices for acquiring
software-intensive systems like ERA. The quality of software is
governed largely by the quality of the processes involved in
developing or acquiring it and maintaining it Carnegie Mellon
University's Software Engineering institute (SEI),[B] recognized for
its expertise in software processes, has developed models and methods
that define and determine organizations' software process maturity.
Specifically, NARA conducted internal assessments in 2002 and 2004
that used SEI's SA-CMM[C] methods to determine the maturity of ERA's
system policies, processes, and practices and implemented a process to
address the assessment's recommendations. In addition, NARA's Chief
Information Officer certified that the ERA program continues to be in
compliance with the Clinger-Cohen Act on November 19, 2009[D]
Legislative condition: 5. Approved by NARA and OMB;
Status: Partially satisfied;
Expenditure plan provisions:
* NARA—October 1, 2010
* OMB--NARA officials stated that the expenditure plan was not
reviewed by OMB prior to submitting it to Congress. Officials stated
that OMB will only review the expenditure plan if there is an existing
appropriations bill with language requiring its review. Without
approval from OMB, Congress will have limited assurance that the plan
is accurate and reliable.
Legislative condition: 6. Reviewed by GAO
Status: Satisfied
Expenditure plan provisions: GAO—-December 21, 2010, briefing to
congressional appropriations committees.
Sources; GAO analysis of NARA data and the Consolidated Appropriations
Act, 2010.
[A] We did not independently examine NARA's compliance with
acquisition rules of the federal government. Instead, we limited our
analysis to the agency's use of the Software Engineering Institute's
Capability Maturity Model and certification from NARA's Chief
Information Officer that the ERA program is in compliance with the
Clinger-Cohen Act.
[B] SEI is a federally funded research and development center operated
by Carnegie Mellon University and sponsored by the Department of
Defense. It provides leadership in software engineering and in the
transition of new software engineering technology into practice.
The Software Acquisition-Capability Maturity Model (SA-CMM) identifies
key process areas that are essential to effectively managing software-
intensive system acquisitions.
[C] Among other things, the Clinger-Cohen Act of 1996 required OMB to
establish processes to analyze, track, and evaluate the risks and
results of major capital investments in IT systems made by executive
agencies. As such. OMB developed policy and issued guidance for the
planning, budgeting, acquisition, and management of federal capital
assets.
[End of table]
Objective 2: NARA has partially implemented most of our open
recommendations:
In July 2009 and June 2010, we made recommendations, to NARA to help
improve the expenditure plan and ERA acquisition. NARA has implemented
one of our recommendations and partially implemented three.
Table 5: Status of NARA's Progress in Implementing Prior GAO
Recommendations:
Prior GAO recommendation: Ensure that ERA's requirements are managed
using a disciplined process that results in requirements that are
traceable throughout the project's life cycle and are kept current
(GAO-10657);
Implementation status: Implemented;
Statue as of November 2010: NARA updated its ERA requirements document
and conducted a gap analysis to identify which requirements had been
satisfied and which remained outstanding. The results of this analysis
were entered into an automated requirements tracking tool to
facilitate traceability analyses. Further, in September 2010, NARA
developed a process to ensure that the ERA requirements document is
kept current Following significant changes to the program, such as
significant modifications to the contract or change requests to the
system.
Prior GAO recommendation: Ensure that NAIRA's Investment review
process has adequate executive-level oversight by maintaining
documentation of the results of reviews, including changes to the
program's cost and schedule baseline and any other corrective actions
taken as a result of changes in ERA cost, schedule, and performance
{GAO-10-657};
Implementation status: Partially implemented;
Statue as of November 2010: NARA documented meetings that NARA senior
management attended where the ERA program was discussed and documented
corrective actions that were proposed in response to Issues related to
the ERA program. In addition, NARA conducted and documented briefings
to senior management on the status of the ERA program's cost, schedule
and performance, and the status of corrective actions. However, NARA
did not document approval of significant changes to Increment 4.
Including reducing the Increment's timeframe from 12 months to 3
months, through Its investment review process.
Prior GAO recommendation: Provide detailed information in future
expenditure plans on what was spent and delivered for deployed
increments of the ERA system and cost and functional delivery plans
for future increments (GAO-09-733);
Implementation status: Partially implemented;
Statue as of November 2010: NARA’s fiscal year 2011 expenditure plan
includes additional information on ERA cost, schedule, and
performance. For example, NARA’s 2011 plan discusses the estimated
amount spent on Increment 3 in fiscal year 2010 through August 2010
($18.8 million) and the delivery dates of the first release—the
congressional records component-—in December 2009, and two prototype
systems, in April 2010. It also discusses plans to achieve a
significant milestone in system testing for the final component of
Increment 3—-upgrades to the ERA base system—in early fiscal year
2011. For Increment 4, the 2011 plan includes the estimated amount
spent in fiscal year 2010 through August 2010 ($16.3 million) and a
description of significant changes to the increment’s schedule and
scope, including the removal of planned functionality. Further, the
2011 plan includes the estimated cost of Increment 5 ($49.8 million)
and planned functionality, including OPA. However, the plan does not
fully discuss other important information. For example, although it
estimates the amount spent on Increments 3 and 4 in fiscal year 2010,
it does not show how these costs compare to the amount NARA planned to
spend on these increments, as reported in the fiscal year 2010 plan.
Further, the plan does not include the estimated delivery date of
Increment 3, nor the estimated fiscal year 2011 costs associated with
completing this increment. Lastly, while the plan describes the
expected functionality and costs planned for Increment 5 during fiscal
year 2011, this information is not reliable
because of, among other things, weaknesses in the cost estimating
methodology (as discussed later in this briefing).
Prior GAO recommendation: Strengthen the earned value process so that
it follows the practices described in GAO’s Cost Estimating Guide and
more reliable cost, schedule, and performance information can be
included in future expenditure plans and monthly reports (GAO-09-733);
Implementation status: Partially implemented;
Statue as of November 2010: In response to our recommendation, NARA
developed but has not fully
implemented an action plan to improve its earned value processes to
follow best practices described in GAO’s Cost Estimating Guide.a
However, NARA continues to have weaknesses in its implementation of
earned value processes and has not yet fully implemented key
practices for establishing a comprehensive earned value management
system, ensuring that the resulting data are reliable, and using earned
value data for decision-making purposes. In addition, ERA’s earned
value data trends do not accurately portray program status, and future
cost overruns would likely be between $195 and $433 million if the full
ERA system is completed as originally designed.
Source: GAO analysis of NARA data.
[A] GAO, GAO Cost Estimating and Assessment Guide: Best Practices for
Developing and Managing Capital Program Costs, GAO-09-3SP (Washington,
D.C.: March 2009).
[End of table]
Results: Observations:
Objective 3: Observations about NARA’s 2011 ERA Expenditure Plan and
Acquisition:
Observation 1: NARA’s ERA fiscal year 2011 expenditure plan does not
provide a reliable basis for informed investment decision-making.
NARA’s expenditure plan should include a sufficient level and scope of
information for Congress to understand what system capabilities and
benefits are to be delivered, by when, and at what costs, and what
progress is being made against the commitments that were made in prior
expenditure plans. However, NARA’s plan does not clearly show what
functions have been delivered to date and at what costs, and what
additional system capabilities are planned for fiscal year 2011,
including reliable costs estimates for these capabilities.
The fiscal year 2011 expenditure plan includes three different
spending scenarios: $85.5 million—based on the President’s budget
request, and two additional figures—-$72 million and $61.4 million—-
based on congressional direction. In addition, as previously
mentioned, NARA also submitted a revised summary of its expenditure
plan that includes two different spending scenarios: (1) $72 million
and (2) $65 million.
The fiscal year 2011 expenditure plan does not clearly show what
functionality has been delivered to date, or how the actual costs of
ongoing and completed increments compare to planned costs, as
documented in the fiscal year 2010 expenditure plan. Specifically, it
is unclear from the expenditure plan exactly what Increment 3
functionality has been delivered to date because the information
provided does not consistently provide adequate detail to understand
the status. For example, the plan provides status information for
system functions included in Increment 3, such as the output of assets
(i.e., electronic records) in various formats; however, it does not
provide other important information, such as the types of file formats
that have been completed to date.
Further, while the fiscal year 2011 expenditure plan reports the
estimated amounts spent on Increments 3 and 4 during fiscal year 2010,
it does not compare these amounts to what was planned to be spent on
these increments, as reported in the fiscal year 2010 plan. For
example, in its fiscal year 2010 expenditure plan, NARA indicated that
it planned to spend approximately $16.6 million on Increment 3
development during fiscal year 2010; however, in its fiscal year 2011
plan, NARA estimated that it had already spent $18.8 million on
Increment 3 development as of August 2010—an increase of $2.2 million,
with one month remaining in the fiscal year. NARA’s fiscal year 2011
expenditure plan did not compare these planned or actual amounts or
include a discussion of why costs were higher than planned.
Even though NARA plans to end development in fiscal year 2011, the
fiscal year 2011 expenditure plan does not clearly show what
functionality is planned to be delivered, by when, and at what cost
during this period. For example, as previously mentioned, NARA
originally planned to complete Increment 3 in June 2010, but
experienced delays in upgrading the ERA base system. According to NARA
officials, this increment is now expected to be delivered by January
2011. However, NARA’s fiscal year 2011 expenditure plan does not discuss
the delivery date of Increment 3, nor the fiscal year 2011 costs
associated with completing this increment.
In addition, while best practices call for the development of
realistic and well-documented cost estimates, NARA’s fiscal year 2011
cost estimates are unreliable and do not provide an accurate basis for
realistic budget formulation. A reliable cost estimate is critical to
the success of any IT program, as it provides the basis for informed
investment decision-making, realistic budget formulation and program
resourcing, meaningful progress measurement, proactive course
correction, and accountability for results. Our research has
identified a number of practices for effective program cost
estimating.[Footnote 19] Among other things, the methodology used to
perform the cost estimate should be clearly documented, including
traceability back to the original data sources, and the estimate
should include clearly documented calculations and results. Further,
in cases where estimates are based on data derived by the subjective
judgment of engineers or other technical experts, this data should be
validated before being used in a cost estimate. Validation involves
cross-checking the results, in addition to analyzing the data and
examining the documentation for the judgment.
While NARA has documented the cost estimates used as the basis for
determining its fiscal year 2011 funding requests, there are
limitations in the supporting methodology, resulting in estimates that
do not reliably reflect the work to be completed. Specifically, NARA
stated that the cost estimates for key components (e.g., ERA base, OPA)
were based on information provided by the contractor, including
estimates of the total lines of software code required and the related
costs. According to NARA officials, the lines of code required for
each task are based on the contractor’s analysis of the complexity,
duration, and interdependencies of the task with other requirements.
However, there are weaknesses in NARA’s methodology. While NARA’s
documentation included the estimated lines of code required and the
related costs (as provided by the contractor) for planned development
tasks, NARA did not provide evidence of the traceability of these
estimates back to their original data sources. For example, NARA’s
cost estimates for providing OPA functionality were based, in part, on
requiring 1,800 lines of code to implement user registration; however,
NARA could not provide documentation to support this estimate and
others. In addition, NARA did not fully validate important data
provided by the contactor, such as the lines of code required for
planned tasks, when preparing its fiscal year 2011 cost estimates.
This is important because our analysis of NARA’s documentation showed
that the estimated cost of each line of code ranged from $287 to
$4,737. While NARA officials stated that the lines of code required
would be reviewed during negotiations with the contractor, this has
not yet been completed. Without validating the contractor’s estimates
by, for example, cross-checking the results against other sources, it
is unclear whether such a wide range of costs for each line of code
are justified based on the complexity of the planned development work.
As a result, NARA’s cost estimates do not provide a reliable basis for
informed investment decision-making and realistic budget formulation.
NARA officials attributed many of the issues that we found with the
reliability of the expenditure plan, in part, to ongoing program
uncertainty due to negotiations with its development contractor. NARA
provided descriptions of tasks for fiscal year 2011 development
activities to Lockheed Martin and, as of December 2010, is working
with the contractor to refine them.
Without more specific information on what functionality will be
delivered before the completion of development at the end of fiscal
year 2011 and more reliable cost estimates associated with those
functions, appropriators will continue to lack important information
for evaluating NARA’s request and have limited ability to measure
overall program progress.
Observation 2: NARA’s fiscal year 2011 expenditure plan does not
address how remaining multi-year funds from fiscal year 2010 will be
allocated.
NARA’s fiscal year 2011 expenditure plan mentions a balance of
remaining multi-year funds, but the plan does not specify how the
agency plans to allocate these funds. Specifically, in fiscal year
2010, NARA spent about $41.7 million of the $61.8 million in multi-
year funds that it was provided by Congress to develop the ERA system,
which resulted in a balance of $20.1 million that is available for
obligation in fiscal year 2011. NARA’s fiscal year 2011 expenditure
plan includes planned ERA system capabilities at three different
funding levels-—$85.5 million, $72 million, and $61.4 million, and its
revised summary of the expenditure plan includes an updated list of
planned of ERA system capabilities and associated costs at two
additional funding levels—-$72 million and $65 million, but none of
these estimates include the planned ERA capabilities to be delivered
with the $20.1 million remaining from fiscal year 2010. According to
NARA officials, NARA will use the $20.1 million to fund development
efforts needed to make the ERA system available to all federal
agencies, as well as to fund other operations and maintenance costs.
However, NARA’s plans for using these funds and their supporting
details are not included in the fiscal year 2011 expenditure plan or
the revised summary provided to Congress. Until NARA specifies how it
will use the multiyear funds remaining from fiscal year 2010,
appropriators will lack information important for evaluating NARA’s
request.
Observation 3: NARA has not yet determined which ERA requirements will
be implemented in fiscal year 2011, nor fully prioritized the
remaining requirements.
System requirements describe the functionality needed to meet user
needs and perform as intended in the operating environment and should
be clearly defined and managed throughout the project in a disciplined
way. Among other things, requirements should be prioritized to guide
the acquirer in determining which requirements to include the project
scope.[Footnote 20] As work products are developed and more is learned
about the system that is being developed, information is occasionally
found that requires a change to the original requirements. Maintaining
a prioritized set of system requirements can better ensure that the
requirements that are most critical to the customer and other
stakeholders are addressed quickly.
NARA recently revised its requirements, and our analysis reveals that
it has made less progress in completing ERA than reported earlier this
year. In June 2010, we reported that NARA had developed and documented
a set of 853 high-level business requirements for ERA and planned to
complete about 57 percent of them by the end of fiscal year 2010.
Subsequently, in July 2010, NARA updated the ERA requirements.
According to NARA, the original 853 requirements were decomposed to a
level such that each requirement could only be allocated to one
increment (e.g., Increment 1) and one component (e.g., ERA base). This
resulted in a revised set of 1,577 requirements, portions of which
were allocated to Increments 1 through 5, deferred to post-Increment
5, or removed from the scope of the ERA program.
Our analysis of the 1,577 revised requirements indicates that only 36
percent of the requirements had been implemented by the end of fiscal
year 2010-—a 21 percent decrease in planned progress compared to
earlier this year, even though NARA spent more time than planned on
system development. Specifically, as of November 2010, we found that
NARA had identified 571 (36 percent) of the revised requirements as
implemented. NARA also indicated that roughly 8 percent of the
remaining requirements would be completed when Increment 3 is delivered,
11 percent of the requirements were removed from the program, and an
additional 18 percent are to be addressed after the completion of the
current development contract at the end of fiscal year 2011 (i.e.,
post-Increment 5). For the remaining 430 requirements (27 percent of
the total) NARA could not determine whether they would be addressed in
fiscal year 2011 because, NARA officials said, they were subject to
ongoing negotiations with the development contractor.
Table 6 shows the status of the ERA requirements.
Table 6: Status of NARA ERA Requirements (as of October 2010):
Requirements category: Increment 1;
Requirements implemented: 319;
Requirements not yet implemented: 0.
Requirements category: Increment 2;
Requirements implemented: 158;
Requirements not yet implemented: 0.
Requirements category: Increment 3;
Requirements implemented: 94;
Requirements not yet implemented: 128.
Requirements category: Increments 4 and 5;
Requirements implemented: 0;
Requirements not yet implemented: 430.
Requirements category: Deferred to post-Increment 5;
Requirements implemented: n/a;
Requirements not yet implemented: 277.
Requirements category: Removed;
Requirements implemented: n/a;
Requirements not yet implemented: 171[A].
Requirements category: Total;
Requirements implemented: 571;
Requirements not yet implemented: 1,006.
Source: GAO analysis of NARA data.
[A] According to NARA, 33 requirements were removed because they were
too high-level to accurately track, while the other 138 were removed
because, among other things, they were no longer valid based on NARA’s
business process or the ERA implementation approach.
[End of table]
The uncertainty about which requirements will be completed in the
coming year is attributable in part to NARA not fully prioritizing its
requirements. As previously mentioned, maintaining a prioritized set
of system requirements can better ensure that the requirements that
are most critical to the customers and other stakeholders are addressed
quickly. To its credit, NARA prioritized its major business functions
planned for fiscal year 2011 and used those prioritized business
functions during negotiations with the development contractor. For
example, NARA has prioritized certain functions associated with its
planned OPA system, such as user registration and the ability to
download certain electronic records. However, NARA has not fully
prioritized the remaining 430 ERA system requirements under
negotiation with the contractor. While NARA has prioritized 242 of the
430 requirements (about 56 percent), it has not prioritized the
remaining 188 requirements (about 44 percent), nor used a full set of
prioritized requirements during negotiations with the contractor
regarding the scope of work to be completed in the final year of
development.
The ERA requirements are not fully prioritized, in part, because the
ERA Requirements Management Plan, dated December 2003, only requires
prioritization of the requirements during the program’s concept
exploration phase. However, significant changes to the ERA program
have occurred since this phase ended in 2004, including the recent
updates to the ERA requirements in July 2010. Although NARA recently
issued a supplemental requirements control process in September 2010
to ensure that the ERA requirements are kept current following changes
to the program (such as significant modifications to the contract),
this process does not include evaluating the prioritization of the
requirements after major changes occur. Until the ERA program fully
prioritizes the ERA requirements to be completed during fiscal year
2011, it will be unclear whether planned system development work will
result in functionality that is most critical to the customer and
other stakeholders.
Conclusions:
NARA’s fiscal year 2011 expenditure plan met four of the six
legislative conditions, but the lack of critical capital planning and
oversight steps, including documentation demonstrating approval of
significant changes to a recent ERA increment, post-implementation
reviews of deployed capabilities, and OMB’s approval of the
expenditure plan, limits NARA’s ability to ensure that the system is
being implemented at an acceptable cost and within expected time
frames and contributes to observable improvements in mission
performance. These issues are further exacerbated by problems with
NARA’s information security controls and the agency’s partial
implementation of several open GAO recommendations, such as those
related to improving investment oversight and earned value processes.
With significant weaknesses in many basic oversight and management
processes, as well as continued delays in completing Increment 3, NARA’
s ability to make significant development progress in the remainder of
the fiscal year will be challenged.
In addition, without a reliable ERA expenditure plan, NARA has not
provided adequate information to assist congressional oversight and
informed decision-making related to the use of appropriated funds.
When combined with the lack of prioritization of the remaining
requirements under negotiation for fiscal year 2011, Congress has
little assurance that additional funds allocated to ERA will result in
significant benefits to federal taxpayers. With OMB's direction to
stop development after 2011, it is unclear whether NARA will be able
to effectively address the full range of weaknesses we identified and
still have adequate time to complete significant development efforts.
Matter for Congressional Consideration:
The identified deficiencies in NARA’s expenditure plan and management
of the ERA acquisition make it unclear whether NARA can make
substantial progress in delivering additional ERA system capabilities
by the end of fiscal year 2011 that justify its planned investment. As
such, we suggest that Congress consider employing an accountability
mechanism that limits NARA’s ability to use funds appropriated for ERA
development until NARA implements an adequate capital planning and
investment control process, updates its expenditure plan to clearly
describe what system capabilities and benefits are to be delivered in
fiscal year 2011, and establishes an associated set of prioritized
system requirements and adequate earned value reporting.
Recommendations for Executive Action:
We are recommending that the Archivist of the United States
immediately take the following two actions while the current system
development contract is active:
* Report to Congress on the specific outcomes to be achieved in fiscal
year 2011 with the balance of any previous multi-year funds.
* Ensure that the ERA requirements planned for fiscal year 2011 are
fully prioritized so that those that are most critical to NARA’s
customers and other stakeholders are addressed.
To ensure that any future efforts are completed within reasonable
funding and time constraints, we are recommending that the Archivist
of the United States take the following four actions:
* Ensure that significant changes to ERA’s program’s cost, schedule,
and scope are approved through NARA’s investment review process.
* Conduct post-implementation reviews of deployed ERA capabilities to
validate estimated benefits and costs.
* Submit ERA expenditure plans to OMB for review and approval prior to
submitting to Congress.
* Update the ERA Requirements Management Plan and related guidance to
require requirements prioritization throughout the project’s life
cycle.
Agency Comments and Our Evaluation:
We provided a draft of this briefing for review and comment to the
Archivist of the United States. In oral comments, the Archives’ GAO
liaison stated that NARA generally agreed with our findings. NARA did
not comment on our recommendations. The liaison also commented that
NARA would like a member of GAO’s staff to be assigned to help the
agency address the ERA program’s weaknesses. Establishing such a
consultative working relationship would create the appearance of a
conflict of interest under GAO's professional standard for independence.
However, we remain committed to working with NARA to discuss or
clarify any issues raised in this product.
[End of briefing slides]
Appendix II: Comments from the National Archives and Records
Administration:
National Archives:
Archivist of the United States:
David S. Ferriero:
15 February 2010:
Government Accountability Office:
Director of Information Technology Management Issues:
Mr. David A. Powner:
441 G Street NW:
Washington DC, 20548:
Dear Mr. Powner:
Thank you for the opportunity to comment on the draft report GAO-11-
299, Electronic Government: National Archives and Records
Administration's Fiscal Year 2011 Expenditure Plan. As with prior ERA
related reports, we find the observations and recommendations in this
report to be helpful as we move forward.
The report includes six recommendations. We concur with all and
believe that we have sufficiently satisfied the first two. We will
provide documentation for these recommendations under separate cover.
The last four recommendations are specific to a future ERA.
Since we do not know when new development efforts may start, nor the
scope or cost of such development, we are unable to address them in a
near term action plan.
Please note that we are also developing an addendum to the FY 2011
Expenditure Plan. The addendum is being prepared pursuant to an
agreement reached at an ERA meeting attended by Senate and House
Appropriations Committee staffs, GAO, OMB, and NARA to provide
updated, clarified information on requirements, costs, and the
schedule of software releases. At GAO's request, NARA has also
included additional information related to security, cost estimating,
and requirements.
If you have any questions regarding this memo or our action plan
process, please contact Mary Drak, NARA's Audit Liaison at 301-837-
1668 or via email at mary.drak@nara.gov.
Signed by:
David S. Ferriero:
Archivist of the United States:
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
David A. Powner (202) 512-9286 or pownerd@gao.gov:
Staff Acknowledgments:
In addition to the individual named above, key contributions to this
report were made by James R. Sweetman, Jr., Assistant Director; Monica
Perez-Nelson; Eric Costello; Lee McCracken; Tarunkant Mithani; Karl
Seifert; Jonathan Ticehurst; and Adam Vodraska.
[End of section]
Footnotes:
[1] Consolidated Appropriations Act, 2010, Pub. L. No. 111-117, div.
C, title V, 123 Stat. 3034, 3193 (Dec. 16, 2009). As of March 1, 2011,
Congress had not yet enacted the Financial Services and General
Government Appropriations Act applicable to NARA for fiscal year 2011.
However, the Continuing Appropriations Act, 2011, as extended to March
4, 2011, appropriates funds under the authority and conditions of
applicable appropriations acts, including the Consolidated
Appropriations Act, 2010, Pub. L. No. 111-242, sections 101(7) and 103
(Sept. 30, 2010); Pub. L. No. 111-290 (Dec. 4, 2010); Pub. L. No. 111-
317 (Dec. 18, 2010); and Pub. L. No. 111-322 (Dec. 22, 2010).
Therefore, the legislative conditions that applied in fiscal year 2010
to NARA's expenditure plan continue to apply at this point in fiscal
year 2011. At the time of our briefing, the Continuing Appropriations
Act had been extended to December 21, 2010, and the legislative
conditions continued to apply at that time.
[2] Subsequent to our briefing, on January 14, 2011, NARA provided GAO
with a revised version of its fiscal year 2011 expenditure plan. We
did not analyze the extent to which NARA's revised plan satisfies
legislative conditions.
[3] For more information related to our evaluation of NARA's use of
earned value processes, see GAO, Electronic Records Archive: National
Archives Needs to Strengthen Its Capacity to Use Earned Value
Techniques to Manage and Oversee Development, [hyperlink,
http://www.gao.gov/products/GAO-11-86] (Washington, D.C.: Jan. 13,
2011). At the time of our briefing, this report had not yet been
issued.
[4] Consolidated Appropriations Act, 2010, Pub. L. No. 111-117, div.
C, title V, 123 Stat, 3034, 3193 (Dec, 16, 2009). As of the data of
this briefing, Congress has not yet enacted the Financial Services and
General Government Appropriations Act applicable to NARA for fiscal
year 2011. However, the Continuing Appropriations Act, 2011, as
extended to Dec. 21, 2010, appropriates funds under the authority and
conditions of applicable appropriations acts, including the
Consolidated Appropriations Act, 2010, Pub, L. No. 111-242, sections
101 (7) & 103 (Sept. 30, 2010); Pub, L, No, 111-290 {Dec. 4, 2010);
Pub, L. No. 111-317 (Dec, 18, 2010). Therefore, the legislative
conditions that applied in fiscal year 2010 to NARA's expenditure plan
continue to apply at this point in Fiscal year 2011.
[5] Agencies develop an exhibit 300, also known as the Capital Asset
Plan and Business Case Summary, to justify each request for a major
information technology investment, OMB sets forth requirements for the
exhibit 300 in Circular A-11, Part 7, Planning, Budgeting,
Acquisition, and Management of Capital Assets.
[6] 44 U.S.C. § 22030)(1).
[7] According to the Federal Acquisition Regulation, a firm-fixed-
price contract provides for a price that is not subject to any
adjustment on the basis of the contractors cost experience in
performing the contract. This type of contract places on the
contractor maximum risk and full responsibility for costs and
resulting profit or loss. 46 C.F.R, § 16,202-1.
[8] A record schedule is a document that describes agency records,
establishes a period for their retention by the agency, and provides
mandatory instructions for what to do with them when they are no
longer needed for current government business.
[9] Records appraisal is the process of determining the value and the
final disposition of records, making them either temporary or
permanent.
[10] These are requests NARA receives from the current and former
administrations, the Congress, and the courts for access to
presidential records, The priorities are determined by NARA's Office
of Presidential Libraries based on experience with the records of
previous administrations.
[11] The Freedom of Information Act establishes that federal agencies
must provide the public with access to government information, thus
enabling them to learn about government operations and decisions, 5
U.S.C. § 552.
[12] In June 2010, we reported that NARA's estimated cost for ERA
through 2012 was $567 million, See GAO, Electronic Records Archive:
Status Update on the National Archives and Records Administration's
Fiscal Year 2011 Expenditure Plan, GAO-10-567 (Washington, D.C.: June
11, 2010. Subsequently, as previously mentioned, OMB directed NARA to
end development in 2011 and, as a result, the current cost estimate is
lower than we previously reported.
[13] GAO, Information Management: Changes in Managing and Preserving
Electronic Records, [hyperlink,
http://www.gao.gov/products/GAO-02-556] (Washington, D.C.: June 17,
2002); Records Management: Planning for the Electronic Records
Archives Has improved, [hyperlink, http://www.gao.gov/products/GAO-04-
927] {Washington, D.C.: Sept. 23, 2004); information Management:
Acquisition of the Electronic Records Archives is Progressing,
[hyperlink, http://www.gao.gov/products/GAO-05-802] (Washington, D.C.;
July 15, 2006); Electronic Records Archives: The National Archives and
Records Administration's Fiscal Year 2006 Expenditure Plan,
[hyperlink, http://www.gao.gov/products/GAO-05-906] (Washington, D.C.:
Aug. 18, 2006); information Management: The National Archives and
Records Administration's Fiscal Year 2007 Expenditure Mari,
[hyperlink, http://www.gao.gov/products/GAO-07-997] (Washington, D.C.:
July 27, 2007); Information Management: Chalienges in Implementing an
Electronic Records Archive, [hyperlink,
http://www.gao.gov/products/GAO-08-738T] (Washington, D.C.: May 14,
2008); Information Management: The National Archives and Records
Administration's Fiscal Year 2008 Expenditure Plan, [hyperlink,
http://www.gao.gov/products/GAO-08-1105] (Washington, D.C.: Sept. 26,
2008); and Electronic Records Archive: The National Archives and
Records Administration's Fiscal Year 2009 Expenditure Plan,
[hyperlink, http://www.gao.gov/products/GAO-09-733] (Washington, D.C.:
July 24, 2009).
[14] GAO, National Archives: Progress and Risks in Implementing its
Electronic Records Archive Initiative, [hyperlink,
http://www.gao.gov/products/GAO-10-222T] (Washington, n.C.: Nov. 6,
2049).
[15] [hyperlink, http://www.gao.gov/products/GAO-10-657].
[16] GAO, Information Security: National Archives and Records
Administration Needs to Implement Key Program Elements and Control's,
[hyperlink, http://www.gao.gov/products/GAO-11-20] (Washington, D,C,:
Oct. 21, 2010).
[17] EVM integrates the investment scope of work with schedule and
cost elements for investment planning and control. The method compares
the value of work accomplished during a given period with that of work
expected in the period. Differences in expectations are measured in
both cost and schedule variances, OM requires agencies to use EVM as
part of their performance-based management system for any investment
under development or with system improvements under way.
[18] Our analysis of NARA's projected cost overruns was based on a
NARA cost estimate that included development costs in 2012, As we
discussed previously, OMB directed NARA to end development in 2011
and, as a result, the current cost estimate is lower than we
previously reported.
[19] [hyperlink, http://www.gao.gov/products/GAO-09-3SP].
[20] Software Engineering Institute, Capability Maturity Model@
Integration for Acquisition, Version 1.2, CMU/SEI-2007-TR-017
(Pittsburgh, Pa.: November 2007).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548: