This is the accessible text file for GAO report number GAO-13-557 
entitled 'DOD Business Systems Modernization: Further Actions Needed 
to Address Challenges and Improve Accountability' which was released 
on May 17, 2013. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov.

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately.

United States Government Accountability Office: 
GAO: 

Report to Congressional Committees: 

May 2013: 

DOD Business Systems Modernization: 

Further Actions Needed to Address Challenges and Improve 
Accountability: 

GAO-13-557: 

GAO Highlights: 

Highlights of GAO-13-557, a report to congressional committees. 

Why GAO Did This Study: 

GAO designated DOD’s multibillion dollar business systems 
modernization program as high risk in 1995, and, since then, has 
provided a series of recommendations aimed at strengthening DOD’s 
institutional approach to modernization and reducing the risk 
associated with key investments. The act requires the department to 
report on actions taken relative to its business systems modernization 
efforts and GAO to assess DOD’s actions to comply with the act. In 
evaluating DOD’s compliance, GAO analyzed, for example, the latest 
version of the business enterprise architecture and enterprise 
transition plan, investment management policies and procedures, and 
certification actions for its business system investments. 

What GAO Found: 

The Department of Defense (DOD) continues efforts to establish a 
business enterprise architecture (a modernization blueprint) and 
transition plan and modernize its business systems and processes, in 
compliance with key provisions of the Ronald W. Reagan National 
Defense Authorization Act for Fiscal Year 2005 and amendments. 
Nonetheless, long-standing challenges remain. The following table 
reflects the status of DOD’s actions to fulfill selected requirements 
of the act. 

Table: 

Act’s requirements: Develop a business enterprise architecture; 
Status of DOD actions: DOD continues to develop content for its 
business enterprise architecture, such as business rules, and is 
proceeding with efforts to extend the architecture to its components. 
However, even though DOD has spent more than 10 years and at least 
$379 million on its business enterprise architecture, its ability to 
use the architecture to guide and constrain investments has been 
limited by, among other things, the lack of a detailed plan.

Act’s requirements: Develop an enterprise transition plan; 
Status of DOD actions: The department’s latest version of its 
transition plan included data on more than 1,200 covered defense 
business systems; however, important content, such as time-phased 
milestones and performance measures, is still needed to address the act’
s requirements.

Act’s requirements: Establish an investment approval and 
accountability structure along with an investment review process; 
Status of DOD actions: DOD has taken steps to establish a portfolio-
based approach to certifying defense business systems, including the 
establishment of a corporate-level board to oversee the approach and 
guidance for selecting, controlling, and evaluating the investment 
portfolio. However, it has yet to fully establish the foundation for 
its new portfolio-level investment management process or the criteria 
and procedures for making portfolio-based investment decisions. 

Act’s requirements: Certify any business system program costing in 
excess of $1 million as compliant with the business enterprise 
architecture and as having undertaken appropriate business process 
reengineering; 
Status of DOD actions: DOD’s portfolio-based investment approach 
included reviewing and certifying more than 1,200 business systems for 
fiscal year 2013, totaling about $6.8 billion in funding. However, 
while DOD continues to perform compliance assertions, it has not 
ensured the accuracy of business enterprise architecture alignment 
through validation of individual investments. Further, appropriate 
business process reengineering assertions were not completed and the 
associated results and outcomes have yet to be reported. 

Source: GAO analysis of DOD data. 

[End of table] 

In addition, the Office of the Deputy Chief Management Officer has yet 
to determine and follow a strategic approach to managing its human 
capital needs, thus limiting its ability to, among other things, 
effectively address the act’s requirements. Collectively, these 
limitations put the billions of dollars spent annually on 
approximately 2,100 business system investments that support DOD 
functions at risk. GAO’s previous recommendations to the department 
have been aimed at accomplishing these and other activities related to 
the business systems modernization. However, to date, the department 
has not implemented 29 of the 63 recommendations that GAO has made in 
these areas. 

According to DOD officials, recent turnover, changes to the act’s 
requirements significantly expanding the number of systems subject to 
certification, and the short time frame for implementing the new 
investment review process contributed to the aforementioned 
weaknesses. Until DOD implements GAO recommendations and addresses the 
weaknesses described in this report, it will be challenged in its 
ability to manage the billions of dollars invested annually in 
modernizing its business system investments. 

What GAO Recommends: 

View [hyperlink, http://www.gao.gov/products/GAO-13-557]. For more 
information, contact Valerie C. Melvin at (202) 512-6304 or 
melvinv@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

Challenges in Addressing Previously Identified Concerns Limit DOD's 
Ability to Manage Its Business Systems Environment: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objective, Scope, and Methodology: 

Appendix II: DOD's End-to-End Business Processes: 

Appendix III: Status of Associated GAO Recommendations Made Since 2009: 

Appendix IV: Comments from the Department of Defense: 

Appendix V: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: DOD Business Systems Modernization Governance Entities' 
Selected Roles, Responsibilities, and Composition: 

Table 2: Summary of Steps to Be Taken in DOD's Investment Management 
Process: 

Table 3: Summary of Criteria and Associated Assessments for Making 
Certification Decisions: 

Table 4: Certification Summary for Fiscal Year 2013: 

Table 5: Organizational Components and Key Responsibilities of the 
Office of the Deputy Chief Management Officer: 

Abbreviations: 

ACART: Architecture Compliance and Requirements Traceability: 

AF-IPPS: Air Force Integrated Personnel and Pay System: 

BPR: business process reengineering: 

BEA: business enterprise architecture: 

DITPR: Defense Information Technology Portfolio Repository: 

DOD: Department of Defense: 

iEHR: Integrated Electronic Health Record: 

IPPS-A: Integrated Personnel and Pay System - Army: 

IPPS-N: Integrated Personnel and Pay System - Navy: 

IRB: investment review board: 

IT: information technology: 

ITIM: Information Technology Investment Management: 

NDAA: National Defense Authorization Act: 

OMB: Office of Management and Budget: 

SNAP-IT: Select and Native Programming Data Input System--Information 
Technology: 

[End of section] 

United States Government Accountability Office: 
GAO: 
441 G St. N.W. 
Washington, DC 20548: 

May 17, 2013: 

Congressional Committees: 

For decades, the Department of Defense (DOD) has been challenged in 
modernizing its business systems. In 1995, we designated the 
department's business systems modernization program as high risk 
because of its vulnerability to fraud, waste, abuse, and 
mismanagement, and because of missed opportunities to achieve greater 
efficiencies; and we continue to designate it as such today.[Footnote 
1] In addition, we have reported[Footnote 2] that significant 
potential exists for identifying and avoiding costs associated with 
duplicative functionality across DOD business system investments, 
which includes approximately 2,100 systems.[Footnote 3] DOD's business 
systems include those for personnel, financial management, healthcare, 
and logistics, and cost the department billions of dollars each year.

Since designating this area as high risk in 1995, we have made a 
series of recommendations aimed at strengthening DOD's institutional 
approach to modernization and reducing the risk associated with key 
investments.[Footnote 4] Further, Congress included provisions in the 
Ronald W. Reagan National Defense Authorization Act for Fiscal Year 
2005 (NDAA)[Footnote 5] as amended, that were consistent with our 
recommendations. Specifically, section 332 of the act, as amended, 
requires the department to, among other things, (1) develop a business 
enterprise architecture (BEA) to cover all defense business systems, 
(2) develop a transition plan for implementing the architecture, (3) 
identify systems information in its annual budget submission, (4) 
establish a systems investment approval and accountability structure 
along with an investment review process, and (5) certify and approve 
any business system program costing in excess of $1 million. The act 
directs the Secretary of Defense to submit an annual report to the 
congressional defense committees on DOD's compliance with certain 
requirements of the NDAA not later than March 15 of each year, through 
2016.

The act directed us to report to these congressional committees--
within 60 days of DOD's report submission--an assessment of the 
department's actions to comply with the requirements of the act. 
Accordingly, our objective was to assess the actions by DOD to comply 
with the act.

To accomplish the objective, we reviewed and analyzed the latest 
version of DOD's BEA, enterprise transition plan, and investment 
management policies and procedures, using our prior reports as a 
baseline.[Footnote 6] To assess the system information in the budget 
submission, we analyzed and compared information contained in the 
department's system that is used to prepare its budget submission with 
information in the department's authoritative business systems 
inventory. To address the investment management provisions of the act, 
we reviewed the department's report to Congress, which was submitted 
on March 20, 2013, and its associated documentation. We also reviewed 
the documentation used to support BEA and business process 
reengineering compliance assertions for fiscal year 2013 
certifications for four case studies--Air Force Integrated Personnel 
and Pay System (AF-IPPS), Integrated Personnel and Pay System-Navy 
(IPPS-N), Integrated Personnel and Pay System-Army (IPPS-A), and the 
Integrated Electronic Health Record system (iEHR)--and compared them 
to DOD's BEA and business process reengineering guidance and our 
business process reengineering assessment guide. In addition, we 
reviewed our guidance and reports on federal agencies' workforce 
planning and human capital management efforts to determine the extent 
to which the Office of the Deputy Chief Management Officer had 
implemented a strategic approach to human capital management. We also 
analyzed documentation relative to corrective actions taken and 
planned on our prior open recommendations.

We conducted this performance audit at DOD and military department 
offices from October 2012 to May 2013, in accordance with generally 
accepted government auditing standards. Those standards require that 
we plan and perform the audit to obtain sufficient, appropriate 
evidence to provide a reasonable basis for our findings and 
conclusions based on our audit objectives. We believe that the 
evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objective. Additional details on our 
objective, scope, and methodology are contained in appendix I.

Background: 

DOD is one of the largest and most complex organizations in the world. 
In support of its military operations, the department performs an 
assortment of interrelated and interdependent business functions, such 
as logistics management, procurement, health care management, and 
financial management. Yet, we have previously reported that the DOD 
systems environment that supports these business functions is overly 
complex and error prone, and is characterized by (1) little 
standardization across the department, (2) multiple systems performing 
the same tasks, (3) the same data stored in multiple systems, and (4) 
the need for data to be entered manually into multiple 
systems.[Footnote 7] For fiscal year 2013, the department requested 
about $6.3 billion for its business system investments and about $16.4 
billion for its IT infrastructure investments.[Footnote 8] According 
to the department's systems inventory,[Footnote 9] its environment is 
composed of approximately 2,100 business systems, including 223 for 
acquisition, 9 for defense security enterprise, 8 for enterprise IT 
infrastructure, 285 for financial management, 675 for human resources 
management, 265 for installations and environment, 653 for logistics 
and materiel readiness, and 10 for security cooperation.

DOD currently bears responsibility, in whole or in part, for 15 of the 
30 programs across the federal government that we have designated as 
high risk.[Footnote 10] Seven of these areas are specific to the 
department,[Footnote 11] and eight other high-risk areas are shared 
with other federal agencies.[Footnote 12] Collectively, these high-
risk areas relate to DOD's major business operations that are 
inextricably linked to the department's ability to perform its overall 
mission and directly affect the readiness and capabilities of U.S. 
military forces and can affect the success of a mission. In 
particular, the department's nonintegrated and duplicative systems 
impair its ability to combat fraud, waste, and abuse.[Footnote 13] As 
such, DOD's business systems modernization is one of the department's 
specific high-risk areas and is essential for addressing many of the 
department's other high-risk areas. For example, modernized business 
systems are integral to the department's efforts to address its 
financial, supply chain, and information security management high-risk 
areas.

Summary of NDAA Requirements: 

Congress included provisions in the NDAA, as amended, that are aimed 
at ensuring DOD's development of a well-defined BEA and associated 
enterprise transition plan, as well as the establishment and 
implementation of effective investment management structures and 
processes.[Footnote 14] The act requires DOD to, among other things,

* develop a BEA that covers all defense business systems,

* develop an enterprise transition plan for implementing the 
architecture,

* identify systems information in DOD's annual budget submissions,

* establish an investment approval and accountability structure along 
with an investment review process, and: 

* not obligate appropriated funds for a defense business system 
program with a total cost of more than $1 million unless the approval 
authority certifies that the business system program meets specified 
conditions.[Footnote 15]

The act also requires that the Secretary of Defense annually submit to 
the congressional defense committees a report on the department's 
compliance with these provisions. DOD submitted its annual report to 
Congress in March 2013, describing steps taken, under way, and planned 
to address the act's requirements.

DOD's Approach to Business Systems Modernization: 

The department's approach to modernizing its business systems 
environment, which is part of DOD's overall effort to transform its 
business operations, includes reengineering the business processes 
supported by its defense business systems, developing and using a BEA 
and associated enterprise transition plan, and improving business 
systems investment management. These efforts are to be guided by DOD's 
Chief Management Officer and Deputy Chief Management Officer. 
Specifically, the Chief Management Officer's responsibilities include 
developing and maintaining a departmentwide strategic plan for 
business reform and establishing performance goals and measures for 
improving and evaluating overall economy, efficiency, and 
effectiveness, and monitoring and measuring the progress of the 
department. The Deputy Chief Management Officer's responsibilities 
include recommending to the Chief Management Officer methodologies and 
measurement criteria to better synchronize, integrate, and coordinate 
the business operations to ensure alignment in support of the 
warfighting mission and developing and maintaining the department's 
enterprise architecture for its business mission area.[Footnote 16]

DOD has assigned roles and responsibilities to various governance 
entities and positions related to business systems modernization. For 
example, the Deputy's Management Action Group is a senior-level forum 
that meets several times a month to discuss departmentwide management 
issues, including business-related topics. This group is to convene as 
the Defense Business Systems Management Committee when it reviews 
defense business system portfolios.[Footnote 17] Table 1 describes 
selected roles and responsibilities and composition of key governance 
entities and positions related to business systems modernization.

Table 1: DOD Business Systems Modernization Governance Entities' 
Selected Roles, Responsibilities, and Composition: 

Entity: Deputy's Management Action Group/Defense Business Systems 
Management Committee; 
Roles and responsibilities: Provide strategic direction and plans for 
the business mission area in coordination with the warfighting and 
enterprise information environment mission areas[A]; Recommend 
policies and procedures required to integrate DOD business 
transformation and attain cross-department, end-to-end 
interoperability of business systems and processes; Review defense 
business system portfolios; Serve as approval authority for business 
system investments greater than $1 million; Establish policies and 
approve the business mission area strategic plan, the enterprise 
transition plan for implementation of business systems modernization, 
and the BEA; 
Composition: Meets at the discretion of the Deputy Secretary of 
Defense and is co-chaired by the Vice Chairman of the Joint Chiefs of 
Staff. Includes senior leadership in the Office of the Secretary of 
Defense, as appropriate, such as the Deputy Chief Management Officer 
and the DOD Chief Information Officer. Also includes the military 
department chief management officers, the heads of select defense 
agencies, and participation by other senior management, including from 
the Joint Chiefs of Staff and the U.S. Transportation Command. 

Entity: Defense Business Council/Investment Review Board (IRB); 
Roles and responsibilities: Oversee DOD's investment management 
process and conduct portfolio analysis in support of the review and 
certification of covered defense business system programs.[B]; 
Review functional strategies developed by the principal staff 
assistants and assess component organizational execution plans; 
Recommend funds certification to the Deputy's Management Action 
Group/Defense Business Systems Management Committee; Prioritize and 
approve changes for inclusion in the BEA; Support the development and 
implementation of the department's end-to-end framework; 
Composition: Chaired by the DOD Deputy Chief Management Officer. 
Includes, for example, the principal assistant secretaries of defense 
(Acquisition, Logistics and Materiel Readiness), Joint Chiefs of 
Staff, DOD Chief Information Officer, Deputy Comptroller 
(Programs/Budgets), Chief Operating Officer, and military departments. 

Entity: Principal Staff Assistants; 
Roles and responsibilities: Senior advisors to the Secretary of 
Defense that assist in policy development, planning, resource 
management, fiscal, and program evaluation responsibilities; Develop 
functional strategies that are to describe business functions, 
outcomes, measures, and standards for their respective business 
areas[C]; 
Composition: Composed of the Under Secretaries of Defense for defined 
functional areas (e.g., Comptroller, Acquisition, Technology, and 
Logistics, Intelligence, Policy, and Personnel and Readiness; 
and the DOD Chief Information Officer). 

Entity: Precertification Authority; 
Roles and responsibilities: Ensure component-level investment review 
processes integrate with the investment management system; Identify 
those component systems that require IRB certification and prepare, 
review, approve, validate, and transfer investment documentation as 
required; Assess and precertify business process reengineering and 
architecture compliance for component systems submitted for 
certification and annual review; 
Composition: Chief Management Officer from the Air Force, Army, Navy, 
the director or the equivalent from the defense agencies, and those 
designated by the DOD Deputy Chief Management Officer as appropriate 
for programs that support the business processes of more than one 
military department or defense agency. 

Entity: Office of the Deputy Chief Management Officer; 
Roles and responsibilities: Provide acquisition oversight and lead IT 
acquisition reform; Conduct and execute investment portfolio reviews; 
Reengineer and apply end-to-end processes to improve business 
operations and support audit readiness; Build and deliver the 
department's BEA, Strategic Management Plan, and enterprise transition 
plan; Conduct organizational assessments and develop guidance to set 
forth the priority performance outcomes for the department; 
Manage the day-to-day operations of the Office of the Deputy Chief 
Management Officer (e.g., human resources, budgeting, IT); 
Composition: Composed of five directorates (Investment and Acquisition 
Management, Enterprise Business Integration, Technology and 
Innovation, Planning and Performance Management, and Operations). 

Source: GAO analysis of DOD information.

[A] DOD has five core business missions: Human Resources Management, 
Weapon Systems Lifecycle Management, Materiel Supply and Service 
Management, Real Property and Installations Lifecycle Management and 
Financial Management.

[B] A covered defense business system is any defense business system 
program that is expected to have a total cost in excess of $1 million 
over the period of the current Future-Years Defense Program, which is 
the department's financial plan over a 6-year period.

[C] DOD has eight functional areas: financial management, acquisition, 
defense security enterprise, logistics and materiel readiness, 
installations and environment, human resources management, security 
cooperation, and enterprise IT infrastructure. 

[End of table] 

Overview of DOD's Tiered Accountability for Business Systems 
Modernization: 

Since 2005, DOD has employed a "tiered accountability" approach to 
business systems modernization. Under this approach, responsibility 
and accountability for business architectures and systems investment 
management are assigned to different levels in the department. For 
example, the Deputy Chief Management Officer is responsible for 
developing the corporate BEA and the associated enterprise transition 
plan. Each component is responsible for defining a component-level 
architecture and transition plan associated with its own tiers of 
responsibility and for doing so in a manner that is aligned with 
(i.e., does not violate) the corporate BEA. Similarly, program 
managers are responsible for developing program-level architectures 
and plans and for ensuring alignment with the architectures and 
transition plans above them. This concept is to allow for autonomy 
while also ensuring linkages and alignment from the program level 
through the component level to the corporate level.

Consistent with the department's tiered accountability approach, the 
NDAA for fiscal year 2008 required the secretaries of the military 
departments to designate the department under secretaries as Chief 
Management Officers with primary responsibility for business 
operations.[Footnote 18] Moreover, the NDAA for fiscal year 2009 
required the military departments to establish business transformation 
offices to assist their chief management officers in the development 
of their respective comprehensive business transformation 
plans[Footnote 19] and to develop well-defined enterprisewide business 
systems architectures and transition plans.

Overview of DOD's BEA: 

DOD's BEA is intended to serve as a blueprint for the department's 
business transformation efforts. In particular, the architecture is to 
guide and constrain implementation of interoperable defense business 
systems by, among other things, documenting the department's business 
functions and activities; the information needed to execute its 
functions and activities; and the business rules, laws, regulations, 
and policies associated with its business functions and activities. 
According to DOD, its architecture is being developed using an 
incremental approach, where each new version of the architecture 
addresses business mission area gaps or weaknesses based on priorities 
identified by the department. The department's BEA focuses on 
documenting information associated with its 15 end-to-end business 
process areas. (See appendix II for a list and description of these 
end-to-end processes.) 

The department considers its current approach to developing the BEA 
both a "top down" and "bottom-up" approach. Specifically, according to 
DOD, the architecture focuses on developing content to support 
investment management and strategic decision making and oversight 
("top down") while also responding to department needs associated with 
supporting system implementation, system integration, and software 
development ("bottom up").

Consistent with DOD's tiered approach to business systems management, 
the department's approach to developing its BEA involves the 
development of a federated enterprise architecture, where member 
architectures (e.g., Air Force, Army, and Navy) conform to an 
overarching corporate or parent architecture and utilize a common 
vocabulary. This approach is to provide governance across all business 
systems, functions, and activities within the department and improve 
visibility across DOD's respective efforts. However, we recently 
reported[Footnote 20] that adopting this approach continued to be a 
challenge for the department. We noted that while DOD was making 
improvements, its corporate architecture (i.e., the BEA) had yet to be 
federated through the development of aligned subordinate architectures 
for each of the military departments. In this regard, we reported that 
the military departments had made little or no progress. Moreover, DOD 
had yet to include common definitions of key terms and concepts to 
help ensure that these architectures will be properly linked and 
aligned.

DOD's Approach to Approving Business System Investments: 

The NDAA, as amended in fiscal year 2012, included significant changes 
to the requirements for investment review and certification of defense 
business systems. Specifically, it required DOD to establish a single 
investment review board chaired by the Deputy Chief Management 
Officer, an investment management process, and significantly expanded 
the scope of systems requiring certification to include any business 
system with a total cost of more than $1 million over the course of 
the department's Future-Years Defense Program, regardless of type of 
funding or whether any development or modernization is planned. 
Previously, the certification requirement only applied to business 
system modernizations costing more than $1 million in modernization 
funds over the life cycle of the modernization. According to DOD 
documentation, the expanded scope resulted in a portfolio review of 
more than 1,200 defense business systems, a nearly six-fold increase 
in the number of systems reviewed previously.

According to DOD, to address the changes, the department implemented 
an integrated business framework to manage its business systems and 
issued new guidance for reviewing and certifying business system 
investments in June 2012. Specifically, the department moved from a 
system-based to a portfolio-based approach for the annual review and 
certification of defense business system investments, where 
investments are grouped into eight functional areas and collectively 
certified as portfolios within each of the functional areas. According 
to DOD, the framework enables the alignment of broad departmental 
strategy and functional business strategy with organizational 
investment decisions and enables the review and discussion of 
investments and outcomes by policy and acquisition decision makers.

The June 2012 guidance outlines sequential process steps and their 
outputs, with each element informing next steps in the process flow. 
Table 2 shows the steps to be taken in DOD's investment management 
process.

Table 2: Summary of Steps to Be Taken in DOD's Investment Management 
Process: 

Phase: Build and review functional strategies; 
Summary: The appropriate business line owners create functional 
strategies that define the business outcomes, priorities, measures, 
and standards for a given functional area within the DOD enterprise. 
After the strategies are reviewed by the department's Investment 
Review Board, the Defense Business Council, they are released to the 
components, becoming the foundation for the remaining phases in the 
process. 

Phase: Build and submit organizational execution plans; 
Summary: Each component assembles its defense business system 
investments into portfolios organized by functional area. Each 
component then documents how their portfolios align with the 
functional strategies in organizational execution plans that consist 
of the following elements: 
* Organizational execution plan brief. The brief demonstrates a 
portfolio's alignment to goals and outcomes in the functional 
strategies, identifies the associated business value of each goal, and 
shows how goals will be achieved. It also includes a road map and 
corresponding time line for achieving the desired end state and budget 
data for each investment; 
* Validated system data. The components are to ensure that the data in 
the organizational execution plans are accurate and complete to 
support certification decisions. As the authoritative sources for DOD 
business IT investments, data to populate the plans is pulled from the 
Defense Information Technology Portfolio Repository (DITPR)[A] and the 
Select and Native Programming Data Input System--Information 
Technology (SNAP-IT)[B]; 
* Portfolio certification request memorandum. A Precertification 
Authority oversees the development of each plan and, through the 
issuance of a portfolio certification request memorandum, approves 
each plan for submission to the Defense Business Council. 

Phase: Analyze portfolios; 
Summary: Completed organizational execution plans are analyzed and 
validated using four criteria: compliance, strategic alignment, total 
cost, and utility. 

Phase: Approve and implement portfolios; 
Summary: The Defense Business Council makes certification 
recommendations to the Chair, who is responsible for approving 
business systems certifications. The Chair documents certification 
decisions in investment decision memorandums that state whether an 
individual organizational execution plan was certified (i.e., funding 
has been approved but may be subject to condition(s) that restrict the 
use of funds, a time line for obligation of funds, or mandatory 
changes to the portfolio of business systems) or not certified (i.e. 
funding is not approved due to misalignment with strategic direction, 
mission needs, or other deficiencies). Expected outcomes of the 
certification process include updates to the department's portfolios, 
Strategic Management Plan, enterprise transition plan, and BEA. 

Source: GAO analysis of DOD data.

[A] Defense Information Technology Portfolio Repository (DITPR) is the 
department's authoritative business systems inventory.

[B] Select and Native Programming Data Input System--Information 
Technology (SNAP-IT) is the department's system used to prepare its 
budget submission. 

[End of table] 

Prior GAO Reviews of DOD's Business Systems Modernization: 

Between 2005 and 2008, we reported that DOD had taken steps to comply 
with key requirements of the NDAA relative to architecture 
development, transition plan development, budgetary disclosure, and 
investment review, and to satisfy relevant systems modernization 
management guidance. However, each report also concluded that much 
remained to be accomplished relative to the act's requirements and 
relevant guidance.[Footnote 21] We made recommendations to address 
each of the areas. DOD largely agreed with our recommendations.

However, in May 2009, we reported that the pace of DOD's efforts in 
defining and implementing key institutional modernization management 
controls had slowed compared with progress made in each of the 
previous 4 years, leaving much to be accomplished to fully implement 
the act's requirements and related guidance.[Footnote 22] In addition, 
between 2009 and 2012, we found progress had been made, but long-
standing challenges we had previously identified remained to be 
addressed.[Footnote 23] For example, the department: 

* Issued BEA versions that continued to address the act's 
requirements, but had yet to federate (i.e., extend) the architecture 
through the development of aligned subordinate architectures for each 
of the military departments.

* Included a range of information for 1,657 business system 
investments in its fiscal year 2013 budget submission; however, it did 
not reflect about 500 business systems, due in part to the lack of a 
reliable, comprehensive inventory of all defense business systems.

* Had begun to implement a business process reengineering review 
process but had not yet measured and reported the results.

* Continued to describe certification actions in its annual report for 
its business system investments as required by the act, but the basis 
for these actions and subsequent approvals was supported with limited 
information, such as architectural compliance assertions that were not 
validated.

* Lacked the full complement of staff it identified as needed to 
perform business systems modernization responsibilities.

In 2012, we concluded that DOD's progress in addressing the act's 
requirements, its vision for a federated architecture, and our related 
recommendations were limited, in part, by continued uncertainty 
surrounding the department's governance mechanisms, such as roles and 
responsibilities of key organizations and senior leadership positions. 
Accordingly, we made recommendations to address the issues identified. 
DOD partially agreed with our recommendations.

In addition, our most recent high-risk report noted that, while DOD's 
capability and performance relative to business systems modernization 
had improved, significant challenges remained.[Footnote 24] For 
example, the department had not fully defined and established a family 
of management controls, such as corporate and component business 
architectures and business system investment management processes. 
These management controls are vital to ensuring that DOD can 
effectively and efficiently manage an undertaking with the size, 
complexity, and significance of its business systems modernization, 
and minimize the associated risks. Furthermore, we also recently 
reported[Footnote 25] that, in order to better identify and address 
potential duplication, DOD needs to develop supporting component 
architectures, align them with its corporate architecture to complete 
the federated BEA, and leverage its federated architecture to avoid 
investments that provide similar, but duplicative, functionality in 
support of common DOD activities.

Challenges in Addressing Previously Identified Concerns Limit DOD's 
Ability to Manage Its Business Systems Environment: 

Over the last year, DOD has taken a number of steps to comply with the 
provisions of the NDAA and to satisfy relevant system modernization 
management guidance. In particular, the department released its 
enterprise transition plan in December 2012, followed by an update to 
its BEA (version 10.0) in February 2013, and its annual report to 
Congress in March 2013 describing important steps that have been taken 
and are planned relative to the act's requirements. However, many of 
the challenges that we have identified in prior years still need to be 
addressed. Specifically,

* Even though DOD has spent more than 10 years and at least $379 
million[Footnote 26] on developing its BEA, the department has yet to 
demonstrate that it is using it to guide and constrain investments as 
well as develop the steps for federating its architecture throughout 
the department.

* The department's latest version of its transition plan lacked 
important content, such as time-phased milestones, performance 
measures, and financial resource needs for all business systems.

* DOD has yet to fully implement its plans and establish a deadline 
for reconciling and validating the completeness and reliability of 
information used to prepare its budget submissions, as previously 
recommended.

* The department's recently established portfolio-based approach for 
reviewing and certifying fiscal year 2013 investments lacks an 
established foundation and criteria and procedures for making 
investment decisions.

* DOD has yet to ensure the accuracy of BEA alignment through 
validation of investments, complete appropriate business process 
reengineering assertions on all investments, and report on the results 
of its efforts.

* The Office of the Deputy Chief Management Officer has yet to follow 
a strategic approach to human capital management and many of the 
positions needed to execute the department's responsibilities remain 
unfilled.

* The department has not implemented 29 of the 63 recommendations that 
we have made since 2005 to assist the department in addressing the 
act's provisions on the architecture, transition plan, budgetary 
disclosure, and investment management.

BEA Content Continues to Be Developed, but Use of the Architecture Has 
Been Limited: 

The act requires DOD to develop a BEA that covers all defense business 
systems and their related functions and activities. Among other 
things, the BEA should include data standards, policies, procedures, 
and performance measures that are to be applied throughout the 
department and define a target defense business systems computing 
environment for the department's major business processes. We have 
also previously reported[Footnote 27] that the BEA is an important 
tool for identifying potential overlap and duplication among DOD 
business systems.

DOD has continued to add needed content to the BEA. Specifically, the 
department's most recent version (10.0), released in February 2013, 
focuses on updating content within core business areas such as 
financial management and human resources management and making 
improvements to the list of laws, regulations, and policies documented 
in the BEA. This version includes information aimed at meeting the 
act's requirements, such as: 

* Data standards intended to help provide for a common business 
language for reporting and sharing financial information among DOD 
business systems. For example, the current version of the BEA includes 
data standards associated with DOD's Standard Line of Accounting, such 
as the Security Cooperation Implementing Agency Code, which the BEA 
defines as a single character code that identifies the military 
department or agency that has negotiated or facilitated a foreign 
military sale on behalf of the U.S. government. Identifying such 
standards is important for improving interoperability among business 
systems and enhancing traceability between budgets and expenditures to 
help achieve audit readiness.

* Business rules to help ensure compliance with the laws, regulations, 
and policies incorporated in the BEA. For example, one business rule 
added to inform the development of human resources systems states that 
the secretary concerned must not pay an inactive National Guard member 
for inactive duty training. Such business rules help to ensure that, 
among other things, the department's business systems consistently 
implement DOD policies and guidance.

* Performance measures that show linkages among DOD activities and 
metrics from the department's Strategic Management Plan. For example, 
the BEA shows that the Manage Civilian Staff Acquisition, Manage Labor 
Relations, and Perform Workforce Analysis business activities are 
linked to the Strategic Management Plan metric associated with 
improving the cycle time for hiring external civilians. This is 
important for meeting the act's requirements associated with 
performance measures and to enable traceability of BEA content to the 
Strategic Management Plan.

The NDAA also requires the department to have a BEA that federates to 
all defense components (e.g., Air Force, Army, and Navy). Toward this 
end, since 2005,[Footnote 28] DOD has initiated a number of efforts to 
federate the BEA, including developing a federated strategy, 
conducting federation pilots, and using semantic web 
technologies[Footnote 29] to provide visibility across its respective 
business architecture efforts. Among other things, DOD's approach to 
federation called for the corporate BEA, each end-to-end business 
process area (e.g., Procure-to-Pay), and each DOD component (e.g., Air 
Force, Army, Navy) to establish a common vocabulary and for the 
programs and initiatives associated with these areas to use this 
vocabulary when developing their respective system and architecture 
products. Yet, we have previously reported that the federation 
strategy lacked adequate details needed to achieve the department's 
goals and accordingly recommended that the department's efforts to 
develop the federated BEA be supported by an architecture management 
plan that incorporated the details needed to execute its Business 
Mission Area Federation strategy.[Footnote 30]

However, according to the Chief Architect, the department's approach 
to federation is currently being reassessed, adding that future 
approaches for architecture federation might not involve the use of 
semantic web technologies. Instead, officials stated that the 
department is working with its Component Collaboration Forum, which 
includes representatives from the military departments and defense 
agencies, to generate recommendations for how to proceed with 
federating the architecture. Officials from the Office of the Deputy 
Chief Management Officer added that the department has experienced 
recent turnover in key positions, which has contributed to the 
challenges in developing a federated approach for its architecture.

Although the department has spent more than 10 years and at least $379 
million on developing its BEA and has been attempting to extend the 
BEA to its components since 2005, it has yet to do so. Further, the 
architecture continues to lack critical content that is needed to 
achieve the department's vision for an architecture that can be used 
to guide, constrain, and enable interoperable business systems.

For example: 

* Business activities have not been defined at a level that allows for 
the identification of potential overlap and duplication. For example, 
while the architecture includes business activities, such as perform 
reporting,[Footnote 31] provide geospatial visualization services, 
[Footnote 32] and manage environmental liability information,[Footnote 
33] it does not decompose these activities to a lower level (e.g., 
describe the activities involved in managing geospatial information 
and services). As a result, programs cannot identify whether they 
address such lower-level activities and, thereby, provide the 
department with information that it could use to identify duplicative 
or overlapping capabilities.

* While the architecture, including the transition plan, includes a 
listing of target business systems, it has yet to identify target 
systems applicable to each operational activity. For example, the 
architecture does not identify any systems that support the perform 
legal investigation operational activity. In addition, while it 
identifies families of systems (i.e., groups of similar systems), the 
architecture has not specified all of the individual systems within 
each family of systems. Specifically, the BEA identifies the Personnel 
and Pay Management family of systems, which is a group of systems 
intended to support many existing personnel and pay processes, but 
information about specific systems within this group of systems (e.g., 
the Army and Air Force's Integrated Personnel and Pay Systems) have 
yet to be defined. Moreover, each of the systems within this family of 
systems is intended to replace multiple existing systems, which are 
not identified in the architecture. As a result, the architecture does 
not include sufficient information to allow the identification of 
systems that might support similar business activities, potential 
duplication and overlap, or inform decision making about reusing or 
integrating existing systems.

* While DOD's Strategic Management Plan for fiscal years 2012 and 2013 
states that DOD was to have completed mapping its Hire-to-Retire and 
Procure-to-Pay business processes by the end of fiscal year 2012, the 
department has not done so. Specifically, while the BEA provides a 
description of these two end-to-end business processes, their related 
business capabilities have yet to be defined. For example, business 
capabilities[Footnote 34] have not yet been defined for the perform 
manpower planning,[Footnote 35] manage recruiting,[Footnote 36] and 
manage labor relations[Footnote 37] business activities, which are 
part of the Hire-to-Retire end-to-end business process. In addition, 
business capabilities have not yet been defined for the create 
purchase requisition,[Footnote 38] award procurement 
instrument,[Footnote 39] and manage disbursements[Footnote 40] 
business activities. Without this information, the department risks 
not being able to optimize business processes and the systems that 
support them to reduce business systems spending.

According to the department's Chief Architect, some of the missing 
detail is not intended to be in the corporate architecture (i.e., 
BEA). Instead, the content should be documented in the component 
architectures that will align with the corporate architecture as part 
of the department's approach to federation. However, given that the 
department is reassessing its approach to federation, it remains 
unclear as to when and how some of this content will be reflected as 
part of its federated vision for an architecture that covers all 
defense business systems and their related functions and activities. 
Until the department does so, it will be limited in its ability to use 
the architecture as a tool for managing its business systems 
investments by, for example, more effectively identifying areas of 
potential duplication and overlap.

Enterprise Transition Plan Content Has Been Improved, but Is Still 
Missing Important Content: 

The act further calls for the development of an enterprise transition 
plan that covers all defense business systems and includes a listing 
of the: 

(a) new systems that are expected to be needed to complete the defense 
BEA, along with each system's: 

* time-phased milestones,

* performance measures,

* financial resource needs, and: 

* risks or challenges to integration into the BEA;

(b) legacy systems that will not be part of the defense BEA, together 
with the schedule for terminating those legacy systems, that provides 
for reducing the use of these systems in phases; and: 

(c) legacy systems that will be a part of the target defense business 
systems computing environment, as well as a strategy for making the 
modifications to those systems that will be needed to ensure that such 
systems comply with the defense BEA, including time-phased milestones, 
performance measures, and financial resource needs.

The department's enterprise transition plan, issued in December 2012, 
includes more than 1,200 covered defense business systems due to 
changes in the NDAA for fiscal year 2012. Along with its related 
documentation, such as the organizational execution plans and the OMB 
exhibit 300s,[Footnote 41] the enterprise transition plan includes 
milestone data, such as termination schedule information on legacy 
systems that will not be part of the defense BEA.

While DOD has begun to improve its enterprise transition plan, 
improvements are needed to fully address the act's requirements.

* The plan includes acquisition program milestone information (e.g., 
initial operational capability) in organizational execution plans and 
the department's functional strategies. However, this documentation 
does not provide milestones for all business systems expected to be 
part of the target architecture,[Footnote 42] nor does it provide the 
capability to easily view milestone information that might be 
contained in other sources, such as the Defense Acquisition Management 
Information Retrieval system.[Footnote 43] For example, the 
organizational execution plans do not identify the complete set of 
individual systems and their associated milestones. In addition, while 
selected exhibit 300s include information about key system events, 
such as system start and end dates, these exhibit 300s only capture 
those business systems that are classified as a major investment and 
do not reflect the complete set of systems that will be part of the 
target environment.

* The department's functional strategies include metrics associated 
with improving each of the functional areas (e.g., financial 
management), but do not include performance measures for each target 
business system. According to DOD officials, milestones will be 
addressed in future versions of the plan through the development of a 
systems evolution description, known as an SV-8.[Footnote 44] 
Officials from the Office of the Deputy Chief Management Officer also 
stated that they plan to include acquisition program measures and 
other related measures in future versions of the plan.

* The plan includes information about fiscal year 2013 funding 
approved under the department's business system investment review 
process for each business system expected to be part of the target 
business architecture. However, the plan does not include information 
about financial resource needs for each business system. DOD officials 
stated that future versions of the plan are to include financial 
information (i.e., financial resource needs) obtained from querying 
data currently located in the department's data repositories.

* The plan and related documentation do not discuss each system's 
risks or challenges to integration into the BEA. Instead, DOD 
officials referred to the constraints and enablers documented in each 
organizational execution plan. For example, the Department of the 
Navy's financial management organizational execution plan cited 
"reaching enterprise-wide agreements on the reduction of multiple and 
often competing business processes that are counterproductive and 
ineffective" and the Defense Logistics Agency's organizational 
execution plan cited "synchronization with other DOD efforts" as a 
challenge. In addition, the Army's acquisition organizational 
execution plan cited "end-to-end business process development" as an 
enabler and the Air Force's acquisition organizational execution plan 
cited "use of the Defense Information Systems Agency's modern 
capabilities and enterprise services" as an enabler.[Footnote 45] 
However, these high-level constraints and enablers do not address 
risks or challenges to integration associated with each system. DOD 
officials stated that future versions of the plan will include 
information about integration risks and challenges to integration from 
investment decision memoranda and future organizational execution plans.

* The plan includes a listing of legacy systems that will not be part 
of the defense BEA along with the associated termination dates. For 
example, the plan includes termination dates for 110 legacy systems. 
However, the plan does not discuss how these systems will be 
terminated in phases. Instead, it only includes system end dates, 
which do not reflect a phased approach to termination. DOD officials 
stated that future versions of the plan are expected to include this 
information using department and program-level data to identify 
schedules and dependencies. In addition, termination dates described 
in the plan are not always accurate. Specifically, 10 legacy systems 
have termination dates that occur prior to the beginning of fiscal 
year 2014. DOD officials acknowledged that not all dates are accurate 
and attributed the inaccuracies to the time required to improve the 
data associated with the large number of new systems subject to the 
act's requirements. They stated that they are actively working to 
improve the data and noted that the plan includes notes about data 
quality, including notes indicating that some life-cycle end dates 
occur in the past.

According to officials in the Office of the Deputy Chief Management 
Officer, an internal assessment was conducted in late 2012 to identify 
gaps in the enterprise transition plan, and efforts are ongoing to 
develop a plan for what is required to address these gaps and the 
act's requirements.

In addition to these gaps, the department has also yet to address the 
act's requirement for including a listing of the legacy systems that 
will be a part of the target defense business systems computing 
environment as well as selected information about those systems. More 
specifically, although DOD's approach to addressing this requirement 
involves identifying all of the systems considered to be target 
business systems, the Office of the Deputy Chief Management Officer 
has only recently (April 2013) defined more detailed expectations for 
the target defense systems computing environment.[Footnote 46] As 
such, it remains to be seen how this definition will be implemented in 
a future transition plan. In addition, while DOD does not plan to make 
modifications to legacy systems that will not be part of the target 
BEA, the existing enterprise transition plan documentation does not 
identify a strategy for making the modifications to legacy systems 
that will be part of the target BEA to ensure that they comply with 
the defense BEA, including time-phased milestones, performance 
measures, and financial resource needs. Moreover, regardless of how 
DOD defines a legacy system, it is important for the department to 
include information regarding phased terminations of existing systems 
to help improve planning for the department's future business systems 
environment.

Beyond this, we have previously reported[Footnote 47] that the plan 
did not include information needed to understand the sequencing of 
systems. To its credit, the department has begun to use a top-down 
approach for sequencing investments (e.g., using functional strategies 
to identify strategic initiatives and propose systems transition). 
However, the planned investments have not been sequenced based on a 
range of important factors cited in federal guidance, such as 
governmentwide and agency-specific priorities (e.g., open and 
transparent government), dependencies among investments, expectations 
about investment costs and benefits, and emerging and available 
technological opportunities (e.g., cloud computing).

Officials from the Office of the Deputy Chief Management Officer 
attributed the gaps in enterprise transition plan content to the 
significant increase in the number of systems that the act requires to 
be included in the plan. Specifically, according to the DOD officials, 
the number of systems subject to the act's requirements increased from 
269 systems in the fiscal year 2012 transition plan to more than 1,200 
in the fiscal year 2013 plan. They stated that gaps in the plan's 
existing content will be improved in future versions.

Until the department defines by when and how the enterprise transition 
plan will include the complete set of required information on the full 
inventory of investments across the department (and does so in a 
manner that reflects consideration of the range of variables 
associated with a well-defined transition plan, such as dependencies 
among investments), it will not be able to provide the department with 
a sufficient basis for sequencing business capabilities and system 
functions to be subsumed by a modern system.

Plans for Improving the Reliability of Business System Investment 
Information Are Not Fully Implemented: 

The NDAA requires that DOD's annual IT budget submission include key 
information on each business system for which funding is being 
requested, such as the system's precertification authority and 
designated senior official, the appropriation type and amount of funds 
associated with modernization and current services (i.e., operation 
and maintenance), and the associated Defense Business Systems 
Management Committee approval decisions.[Footnote 48] The budget 
submission relies on business system investment information (e.g., 
funds requested, mission area, and system description) that the 
components entered into the department's system used to prepare its 
budget submission (SNAP-IT). In accordance with DOD guidance and 
according to officials in the Office of the DOD Chief Information 
Officer, the business systems listed in SNAP-IT should match the 
systems listed in the department's authoritative business systems 
inventory--Defense Information Technology Portfolio Repository 
(DITPR). We have previously recommended that the department develop 
and implement a plan and establish a deadline for reconciling and 
validating the completeness and reliability of information in its two 
repositories, and include information on the status of these efforts 
in the department's annual report in response to the act.[Footnote 49]

DOD Chief Information Officer officials have reiterated the 
department's commitment to integrating the two repositories and 
described steps taken and future steps toward achieving this end. For 
example, the officials stated that a planned investment portal that is 
intended to present a unified interface to users for both IT budget 
and system information is in development. Officials also stated that 
this new portal is expected to reach a limited operational capability 
in May 2013 and when planned capabilities are implemented, issues 
associated with previously reported differences between the number of 
business systems reflected in the two repositories would have been 
addressed. In addition, to prepare for the planned integration, DOD 
officials have described steps it has begun to take to improve the 
quality of the data in the systems. For example, DOD's March 20 annual 
report pursuant to the NDAA stated that the Defense Business Council 
directed precertification authorities to update and validate the 
authoritative data sources, and specific feedback on data errors was 
provided to the precertification authorities. In addition, the report 
stated that a community of interest work group was updating the data 
collection repositories and removing conflicting data sources. 
Moreover, DOD has developed detailed plans for the integrated portal 
that includes, among other things, roles and responsibilities, 
requirements, a data dictionary, business rules, and implementation 
time frames for an incremental release, such as achieving full 
operational capability for its final increment by March 2015. 
Nevertheless, the officials stated that efforts to implement these 
planned capabilities for the integrated portal have been impacted by 
changes in the management structure for DITPR and by requirements to 
rehost the repository due to a base realignment and closure. As such, 
it remains unclear as to when DOD will be able to fully implement its 
plans for having an integrated portal.

Until DOD fully implements its plans and establishes a deadline for 
completing the repository integration and validating its completeness 
and reliability, as we have previously recommended, it will not be 
able to ensure that annual budget submissions and oversight decisions 
relying on these repositories will be based on complete and accurate 
information.

Steps to Establish a Portfolio-Based Investment Management Approach 
Have Begun, but Much Work Remains: 

The act, as amended, requires DOD to establish an investment approval 
and accountability structure along with an investment review process. 
In addressing the act's provisions, DOD has taken steps to establish a 
portfolio-based approach to certifying defense business systems, 
including the establishment of a corporate-level Investment Review 
Board (IRB) to oversee the approach and guidance for selecting, 
controlling, and evaluating the investment portfolio. However, it has 
yet to fully establish the foundation for its new portfolio-level 
investment management process or the criteria and procedures for 
making portfolio-based investment decisions.

Investment Review Board to Oversee Portfolio-Based Investment 
Management Process Has Been Established: 

Section 901 of the NDAA for fiscal year 2012 requires the 
establishment of a single IRB to oversee the department's investment 
review process and conduct periodic reviews of all covered defense 
business systems programs. The act also calls for the IRB to be 
established by the DOD Deputy Chief Management Officer, with 
representation from appropriate officials, including the DOD Chief 
Information Officer and appropriate senior officials for the functions 
and activities supported by the defense business systems under review. 
Consistent with the act, our IT Investment Management framework 
[Footnote 50] also calls for the establishment of an enterprisewide 
investment board that is composed of senior executives from IT and 
business units.

To comply with the new requirements of the act, the department changed 
its investment management governance structure by establishing an 
enterprisewide IRB and eliminating the multiple, functionally oriented 
boards used previously. Specifically, in October 2012, DOD established 
the Defense Business Council as the department's enterprisewide, 
single IRB and assigned the council overall responsibility for 
governing the optimization of DOD's enterprisewide business 
environment and the improvement of the department's business 
activities and management structures. In accordance with the NDAA and 
our investment management framework, the council is chaired by the DOD 
Deputy Chief Management Officer and includes members such as the 
Deputy DOD Chief Information Officer, the Assistant Secretary of 
Defense (Acquisition), the Deputy Under Secretary of Defense 
(Installations and Environment), the Office of Joint Chiefs of Staff, 
and representatives from the Air Force, Army, and the Department of 
the Navy. By establishing an IRB composed of appropriate senior 
executives, the likelihood of making decisions that reflect the needs 
of the enterprise is increased.

Foundation for Portfolio-Level Investment Management Has Yet to Be 
Fully Established: 

According to DOD's guidance, the foundation of the new investment 
management process is the set of six functional strategies that 
prioritize and identify the enterprise's immediate needs while 
providing direction for each business area. The guidance requires that 
the functional strategies demonstrate alignment of goals with 
enterprisewide business goals identified in DOD's Strategic Management 
Plan. Other critical elements that must be included are the vision for 
each business area, the goals that will lead to accomplishment of the 
vision, and the expected outcomes for each goal. To track progress 
toward the accomplishment of stated goals and outcomes, the guidance 
calls for performance measures that include baseline and target 
measures, and provide a rationale for the identified targets. The 
completed functional strategies are to inform the components' 
technology portfolios, provide the context for portfolio-based 
investment decisions, become the basis for the department's enterprise 
transition plan, help to expand and refine the department's Strategic 
Management Plan, and become part of the BEA.

For the fiscal year 2013 certification of defense business systems, 
six functional strategies that covered seven of DOD's eight functional 
areas were developed.[Footnote 51] In accordance with the investment 
management guidance, each of the six included many of the critical 
elements called for in the guidance. Specifically,

* Five of the six functional strategies demonstrated linkages to 
business goals in DOD's Strategic Management Plan, but one did not. As 
a result, it lacked the direction needed to guide efforts for 
developing a portfolio that adheres to the strategy's tenets while 
ensuring the successful execution of the department's mission.

* Each of the six included the vision and goals of the corresponding 
functional area, but one functional strategy did not include expected 
outcomes for all functional area goals. Without fully documented 
outcomes, the department cannot adequately assess progress against 
goals, take corrective action when needed, and demonstrate tangible 
improvements.

* Four of the six did not have all performance measures in place for 
assessing progress toward achieving stated goals. In addition, none of 
the six included performances measures that reflected all of the key 
attributes identified in DOD's guidance. Specifically, of the 72 
performance measures documented in the functional strategies, none 
included all of the following: a baseline measure, a target against 
the baseline, and a rationale for the target. Without these key 
attributes, the performance measures cannot function as a standard for 
comparison and provide a clear indication of DOD's progress towards 
its goals.

According to DOD officials, the functional strategies were not 
expected to be fully mature for the fiscal year 2013 certification 
review cycle; instead, initial versions were to highlight areas in 
which additional work was needed and updates would reflect lessons 
learned as well as departmental priorities, goals, and objectives. For 
the fiscal year 2014 certification cycle, the six functional 
strategies were revised. To the department's credit, this resulted in 
the following improvements: 

* All of the six demonstrated linkages to the business goals in DOD's 
Strategic Management Plan.

* Five of the six had fully documented performance measures and all 
six identified one or more expected outcome for each functional area 
goal.

Nevertheless, DOD has not established performance measures in which 
all key attributes identified in DOD guidance are present and, thus, 
the department may lack a clear indication of progress toward 
achieving shared goals. Without fully developed functional strategies 
that include all of the critical elements identified in DOD's 
guidance, the fiscal year 2013 certification reviews lacked the 
foundation necessary for ensuring that sound investment decisions were 
aligned with enterprisewide goals and priorities. Further, the 
department's ability to present a complete enterprise business 
strategy that can be used to update the strategic management and 
enterprise transition plans as well as the business enterprise 
architecture may be limited.

Criteria and Procedures for Making Portfolio-Based Investment 
Decisions Have Not Been Fully Defined: 

Investment portfolios are integrated, enterprisewide collections of 
investments that are evaluated and managed collectively based on 
common criteria. Taking a portfolio perspective enables an 
organization to consider its investments in a more comprehensive and 
integrated manner so that, collectively, the investments optimally 
address the organization's mission, strategic goals, and objectives. 
Managing IT investments as portfolios also allows an organization to 
determine its priorities and make decisions about which projects to 
begin funding and continue to fund based on analyses of the relative 
organizational value and risks of all projects, including projects 
that are proposed, under development, and in operation.

Our investment management framework requires that an organization have 
documented policies and procedures for reviewing, evaluating, and 
improving portfolio performance, including the use of a predetermined 
performance threshold for analyzing actual versus expected performance 
(e.g., more than XX percent over expected cost)--a major factor in 
defining remedial actions for underperforming investments. Our 
framework also highlights the importance of examining costs incurred; 
benefits attained; current schedule; accuracy of project reporting; 
and risks that have been mitigated, eliminated, or accepted to date as 
part of assessing portfolio performance. Altogether, these assessments 
ensure that the overall portfolio provides the maximum benefits at a 
desired cost and at an acceptable level of risk.

DOD's investment management guidance identifies four criteria and 
specifies the associated assessments that are to be conducted when 
reviewing and evaluating components' organizational execution plans in 
order to make a portfolio-based investment decision. Table 3 provides 
a summary of the criteria and associated assessments that DOD uses to 
make a certification decision.

Table 3: Summary of Criteria and Associated Assessments for Making 
Certification Decisions: 

Criteria: Compliance; 
Assessment: The Precertification Authority's memorandum stating that 
each applicable defense business system is compliant with the relevant 
requirements of the NDAA for fiscal year 2012. 
Criteria: Strategic alignment; 
Assessment: The degree of alignment to strategic goals and missions 
documented in the department's Strategic Management Plan and the 
functional strategies; The degree to which investments are being 
managed in accordance with capital planning and investment control 
practices[A] and DOD's Better Buying Power guidance[B]. 

Criteria: Utility; 
Assessment: The portfolio's ability to deliver required capabilities 
for a given function, to demonstrate interoperability with other 
systems, and to show scalability to support additional users or new 
features in the future. 

Criteria: Total cost; 
Assessment: Cost in the context of the defense business system's life-
cycle phase and the estimated return on investment. 

Source: GAO analysis of DOD data.

[A] Capital planning and investment control is an IT portfolio-driven 
management process for ongoing identification, selection, control, and 
evaluation of investments. The process attempts to link budget 
activities and agency strategic priorities with achieving specific IT 
program modernization outcomes.

[B] DOD's Better Buying Power guidance, issued in September 2010, is 
intended to improve affordability and cost control, competition, and 
management of services acquisitions, among other things. Among the 
specific actions called for are increasing competition in services 
contracting and reducing the use of high-risk contract vehicles. 

[End of table] 

However, the guidance does not specify a process for conducting an 
assessment or call for the use of actual versus expected performance 
data and predetermined thresholds. These thresholds, according to our 
investment management framework, should be established to evaluate 
portfolio performance. Without a defined approach that incorporates 
these practices, the department is limited in its ability to compare 
investments to one another within and across the functional areas to 
help ensure that its collections of investments optimally address the 
organization's mission and goals.

In addition, the guidance does not call for assessments to be 
conducted in four key areas identified in our framework: benefits 
attained; current schedule; accuracy of project reporting; and risks 
that have been mitigated, eliminated, or accepted to date. The 
framework stresses that it is important to examine all of these 
aspects since, among other things, some investments may exceed 
performance expectations (e.g., delivering at lower cost, in less 
time, and with better benefits than expected). In these cases, the 
board may wish to accelerate an investment's funding or schedule, 
reallocate resources within the overall portfolio, or make some other 
type of adjustment.

Further, the guidance does not call for the organizational execution 
plans to include critical information for conducting assessments 
associated with the strategic alignment, utility, and total cost 
criteria. More specifically, it does not require the plans to include 
information on alignment with the capital planning and investment 
control practices and Better Buying Power guidance; interoperability 
among systems and system scalability to support additional users or 
new features in the future; and cost in relationship to return on 
investment. Without this critical information, the department cannot 
ensure that certification decisions are being made in accordance with 
established criteria.

DOD officials stated that, although assessment decisions may appear 
subjective, the reviews were based on analytics focused on the four 
criteria and the individual expertise of the business leaders. 
However, according to our framework, board members should apply their 
judgment and knowledge to the evaluation of investment data using 
documented procedures and established criteria.

To its credit, the department issued revised guidance in April 2013 
for the fiscal year 2014 certification of defense business systems 
that is intended to mature the investment management process and 
reflect lessons learned from the fiscal year 2013 certification 
process. While it is too soon to evaluate the department's updated 
approach to business system investment management, we plan to evaluate 
DOD's progress in defining and implementing its updated investment 
review processes in our fiscal year 2014 report on defense business 
systems modernization. As DOD continues to mature its investment 
management process, it is important that criteria and procedures for 
portfolio-level management are fully established and that all 
information needed to conduct portfolio evaluations is provided in key 
documents to ensure that its investments are selected and controlled 
in a manner that best supports its mission needs.

Investment Portfolios Certified, but Compliance Assertions Must be 
Implemented More Effectively: 

* The act also requires that, prior to the obligation of funds, 
investments involving business programs with a total cost in excess of 
$1 million over the period of the Future-Years Defense Program, 
whether appropriated or non-appropriated, be certified to meet 
specific conditions defined in the act, such as demonstrating 
compliance with DOD's BEA and whether appropriate business process 
reengineering efforts have been undertaken.

DOD's March 2013 report describes the department's continuing efforts 
to improve the efficiency and effectiveness of its business operations 
through the use of its Integrated Business Framework, actions it will 
take moving forward related to the evolution of its BEA, and the 
results of its actions to certify supporting IT investments. 
Specifically, according to the report, the department received about 
1,245 certification requests totaling more than $7 billion in eight 
functional areas from 21 DOD components.[Footnote 52] Of the 1,245 
requests within investment portfolios, the Defense Business Council 
approved 1,231 requests totaling about $6.8 billion (see table 4).

Table 4: Certification Summary for Fiscal Year 2013: 

Functional portfolio: Acquisition; 
Count of certification requests approved: 98; 
Sum of certification requests approved: $256,704,000.

Functional portfolio: Defense Security Enterprise; 
Count of certification requests approved: 8; 
Sum of certification requests approved: $44,868,000.

Functional portfolio: Enterprise IT Infrastructure; 
Count of certification requests approved: 5; 
Sum of certification requests approved: $41,231,000.

Functional portfolio: Financial Management; 
Count of certification requests approved: 200; 
Sum of certification requests approved: $785,179,000.

Functional portfolio: Human Resources Management; 
Count of certification requests approved: 435; 
Sum of certification requests approved: $2,919,274,000.

Functional portfolio: Installations and Environment; 
Count of certification requests approved: 133; 
Sum of certification requests approved: $288,561,000.

Functional portfolio: Logistics and Materiel Readiness; 
Count of certification requests approved: 348; 
Sum of certification requests approved: $2,436,253,000.

Functional portfolio: Security Cooperation; 
Count of certification requests approved: 4; 
Sum of certification requests approved: $1,564,000.

Functional portfolio: Total; 
Count of certification requests approved: 1,231; 
Sum of certification requests approved: $6,773,634,000.

Source: DOD's 2013 Congressional Report on Defense Business Operations. 

[End of table] 

The report notes that the investment review process resulted in the 
identification of 99 systems that will be terminated within the next 
36 months and are not expected to receive budget funding beyond the 
next 3 years. In addition, investments within portfolios were 
identified that required further refinement or elimination, which 
resulted in the denial of more than $300 million in funding to those 
investments. According to the annual report supplement, three 
investment requests were withheld until a later time; seven were not 
certified; and one was certified but for a smaller amount than was 
originally requested. The report also includes the status of 
certifications, such as "certified with tasking action item" or 
"conditionally certified," for each business system investment. 
However, while the report documents the number of certifications, it 
does not include the relevant information about known weaknesses 
related to the investment review process and fiscal year 2013 
certifications, such as the lack of BEA and business process 
reengineering-related compliance and the extent to which additional 
actions were needed to ensure compliance.

Further, although the report states that the Defense Business Council 
took actions to significantly improve both the visibility and accuracy 
of its authoritative data, we were unable to verify the number of 
certifications and the funding amount requested using authoritative 
data sources, such as component organizational execution plans 
provided by DOD. For example, while the department reported 1,245 
certification requests totaling about $7.17 billion, we were only able 
to identify 1,223 requests totaling about $7.41 billion (a net 
difference of 22 certification requests and about $240 million in 
requested funds). DOD officials noted that certain data fields in the 
original data sources were incomplete, incorrect, or duplicative when 
the certification request was submitted. These officials added that 
the difference occurred because of in-process changes or updates to 
the original requests. They also stated that extensive manual cleanup 
efforts have been ongoing since the beginning of the investment review 
process. Although these data cleanup efforts are ongoing, the 
officials expected future portfolio reviews to have improved data.

While data cleanup efforts have been significant in the first year of 
this new investment review process and are an important part of 
building the department's baseline portfolios of investments, without 
providing the full and accurate range of information the department 
faces increased risk that it will not be able to provide effective 
oversight over its business systems investments. Further, by not 
reporting the overall weaknesses associated with the certification 
activities, the department does not provide the full range of 
information that is needed to permit meaningful and informed 
congressional oversight of the department's business systems 
modernization efforts.

BEA Compliance Asserted for Most of the Systems, but Assertions 
Continue to Lack Adequate Validation: 

Under the act, funds may not be obligated for covered business systems 
unless the certification requirements with regard to BEA compliance 
are met. For its fiscal year 2013 certification review, investments 
were to follow BEA compliance guidance issued in March 2011. This 
guidance describes an incremental approach to BEA compliance 
requirements, including key elements needed for asserting architecture 
compliance,[Footnote 53] such as specific artifacts and definitions 
for compliance or planned compliance.[Footnote 54] Further, we have 
previously recommended that DOD, among other things, explicitly assign 
responsibility for validating program BEA compliance assessments. 
[Footnote 55]

DOD has certified most, but not all, investments on the basis of 
architecture compliance. According to DOD, precertification 
authorities asserted BEA compliance in their respective organizational 
execution plans for about 89 percent of systems submitted as part of 
the portfolio certification process. For example, for the four 
investments that we reviewed, [Footnote 56] the military components 
and the Tricare Management Activity assessed BEA compliance for each 
investment in accordance with DOD guidance. In addition, the 
appropriate precertification authorities asserted (via their 
respective organizational execution plans) whether each system was 
compliant with DOD's BEA. For example, the precertification 
authority's request in the Army's organizational execution plan 
asserted BEA compliance for the Integrated Personnel and Pay System-
Army (IPPS-A). As supporting documentation for this assertion, the 
program provided a compliance dashboard to document BEA alignment, 
consistent with DOD guidance, and architecture artifacts demonstrating 
alignment to specific architectural content, such as operational 
activities described in the BEA.

However, the remaining 11 percent of systems were identified as not 
being in compliance with the architecture. DOD officials stated that 
no system was denied certification due to the lack of BEA compliance; 
instead, the Deputy Chief Management Officer included an action item 
in all investment decision memoranda tasking components to document 
compliance or a plan to achieve BEA compliance for any business 
systems identified by the precertification authority as not being in 
compliance. However, while the department has asserted compliance for 
many business systems, it has yet to validate these assertions, as we 
have previously recommended.

Office of the Deputy Chief Management Officer officials said that the 
large increase in the number of systems to be reviewed made it 
challenging to complete a review for all the systems. In addition, the 
short time frame for implementing the new investment review process 
resulted in the need to include the action items. They also stated 
that they are tracking responses to these actions for each of the 
portfolios and expect the assertions for fiscal year 2014 reviews to 
be more complete than the fiscal year 2013 reviews.

With regard to validation, DOD officials stated that some information 
associated with the compliance process had been validated, such as 
information associated with complying with DOD's Standard Financial 
Information Structure.[Footnote 57] However, the DOD Inspector General 
recently reported that an assessment of an enterprise resource 
planning system determined, among other things, that the system 
inaccurately asserted compliance based on a future capability rather 
than on the system's actual capability at the time of the assessment. 
[Footnote 58] It further stated that a more robust certification 
process was needed instead of having the Office of the Deputy Chief 
Management Officer rely on a self-certification process that did not 
include validation.

Officials from the Office of the Deputy Chief Management Officer have 
stated their intent to perform BEA compliance validation for selected 
defense business systems as part of the investment review process. 
However, they have not yet executed these activities and were not able 
to provide milestones for when this would be accomplished. Among other 
things, BEA compliance is important for helping to ensure that DOD 
programs have been optimized to support operations. As we and others 
have reported, without proper oversight and validation of compliance 
assertions and plans to be compliant, DOD programs are at increased 
risk of being defined and implemented in a way that does not 
sufficiently ensure interoperability and avoid duplication and 
overlap, which are both goals of the BEA and related investment 
management approach.

Appropriate Business Process Reengineering Assertions Were Not 
Completed for the Majority of Investments for Fiscal Year 2013 
Certifications: 

The act requires that precertification authorities determine that 
appropriate business process reengineering efforts be undertaken for 
the investment prior to certification approval for each covered 
business system. To address this requirement, DOD issued guidance in 
April 2011 to implement its approach for asserting business process 
reengineering compliance, which was intended to ensure that the 
underlying business process supported by the business system 
modernization was as streamlined and efficient as practicable. This 
guidance required submission of a form that integrates questions in 
areas related to, for example, business process definition and target 
process improvements. We have also issued business process 
reengineering guidance that recognizes, among other things, the 
importance of developing a clear problem statement and business case 
for reengineering and analyzing the current and target environments. 
[Footnote 59]

The four investments we reviewed generally followed the department's 
business process reengineering guidance for asserting compliance. 
Specifically, all four programs we reviewed--IPPS-A, IPPS-N, AF-IPPS, 
and iEHR--submitted business process reengineering forms, consistent 
with DOD guidance, and were asserted by their respective 
precertification authorities as being in compliance. These programs, 
all of which are large and complex investments, also had 
documentation, which included, to varying degrees, a general 
description of problems or issues to be solved, process maps that 
graphically documented the target business processes, and process maps 
that graphically documented the current business processes that were 
reengineered.

However, limited documentation of root cause analyses was provided for 
the four investments. For example, iEHR documented root cause analyses 
for some, but not for all five capabilities that were the focus of its 
reengineering efforts. It also did not document the likelihood of each 
probable cause. As we have previously reported, [Footnote 60] there is 
a long history of challenges in trying to achieve electronic health 
records sharing between DOD and the Department of Veterans Affairs, 
and this analysis would be a key document for supporting the business 
process reengineering assertion. Without a root cause analysis to 
trace symptoms back to the underlying factors, an agency may determine 
inappropriate corrective actions, including IT solutions, to address 
the true problems of business process inefficiencies.

While precertification authorities asserted business process 
reengineering compliance for IPPS-A, IPPS-N, AF-IPPS, and iEHR, 
according to DOD, precertification authorities asserted that 
appropriate business process reengineering had been undertaken on only 
about 41 percent of the approximately 1,200 systems for the fiscal 
year 2013 certification reviews. Department officials confirmed that 
no system was denied certification due to lack of business process 
reengineering assertion even though, according to the act, compliance 
is a requirement for obligation of funds. As with the BEA compliance, 
DOD included an action item in nearly all investment decision 
memoranda tasking components to document a plan to achieve business 
process reengineering compliance for any business systems identified 
by the precertification authority as not being in compliance. 
According to department officials, one of the reasons for a number of 
systems that were not asserted for appropriate business process 
reengineering was confusion about the extent to which business process 
reengineering was required for systems in sustainment.[Footnote 61] 
Also similar to the BEA compliance, officials from the Office of the 
Deputy Chief Management Officer stated that the increase in the number 
of certification requests made it challenging to complete a review for 
all the systems. In addition, they added that the short time frame for 
implementing the new investment review process resulted in the need to 
include the action items. They also stated that they are tracking 
responses to these actions for each of the portfolios and expect the 
assertions for fiscal year 2014 reviews to be more complete than the 
fiscal year 2013 reviews.

The department has also begun to validate business process 
reengineering assertions. Specifically, DOD updated its BPR guidance 
in late September 2012 to include actions to be taken to review 
supporting documentation on selected investments. Beginning in late 
November 2012, the Office of the Deputy Chief Management Officer 
selected 35 defense business systems from across the department to 
inspect and validate business process reengineering assertions. 
According to officials, validation efforts have been labor intensive 
and will inform future business process reengineering assertions. 
However, results are not expected to be reported to the Defense 
Business Council in full until June 2013 and it is unclear what 
actions will be taken as a result. DOD's September 2012 business 
process reengineering guidance notes that the Defense Business Council 
actions may include levying conditions on defense business systems or 
decertification of 2013 funds until the business system becomes 
compliant, but no actions had been taken as of yet.

DOD's efforts to track action items and validate precertification 
authorities' business process reengineering assertions would enable 
the department to make business system investment decisions based on 
information that is accurate and reliable. However, the department 
needs to ensure that the outcomes of its efforts inform the business 
process reengineering to be performed on all investments for the 
fiscal year 2014 reviews. Further, it should ensure that appropriate 
business process reengineering assertions have been completed on all 
the business systems submitted for the fiscal year 2014 certification 
reviews prior to the certification of funds. Until it does, the 
department risks not being able to determine whether business 
processes are streamlined and efficient as practicable and not 
complying with the statutory restriction on obligation of funds.

Business Process Reengineering Efforts Need to Be Directly Aligned 
with Business Outcomes and Reported: 

DOD guidance states that business process reengineering should consist 
of, among other things, potential outcomes related to: 

* eliminating or reducing unique interfaces to the greatest extent 
possible and designing necessary information exchanges logically and 
efficiently and: 

* identifying appropriate outcome-based business performance measures 
that are consistent and linked to intended benefits of the investment.

Among other things, our guidance[Footnote 62] also emphasizes the 
importance of having meaningful performance measures to assess whether 
business process reengineering activities actually achieve the 
intended results. For example, business process reengineering results 
should be focused on providing better support to an organization's 
mission, reducing costs, and improving service. We have also 
previously recommended[Footnote 63] that DOD include in its annual 
report to Congress the results of the department's business process 
reengineering efforts, such as the department's determination of the 
number of systems that have undergone material process changes and the 
number of interfaces eliminated as part of these efforts (i.e., by 
program, by name).

For the four investments we reviewed, DOD reported that each one 
intends to reduce or eliminate interfaces. However, while two of the 
investments--AF-IPPS and IPPS-A--provided specifics about the number 
of interfaces needed or the number of interfaces to be reduced, none 
provided detailed assessments of existing interfaces for reuse 
opportunities and documentation related to the complexity of 
interfaces. For example, iEHR stated that it had not yet performed an 
analysis on the elimination of interfaces for any of the business 
capabilities identified in its supporting documentation. In the 
absence of such information, the department lacks the detailed 
justification needed to help it determine whether the investment is 
reducing the cost of fielding business capability.

Moreover, the business process reengineering documentation concerning 
performance measures varied across the four investments. For example, 
IPPS-N provided an extensive list of metrics associated with its 
reengineering processes and included baseline performance levels and 
targeted outcomes, such as specific decreased costs and/or decreased 
time. AF-IPPS included a number of metrics, some of which included 
baseline performance levels and targeted outcomes. On the other hand, 
while IPPS-A only specified targeted outcomes related to its 
architecture products, it did not include specific baseline 
performance levels of the current process and targeted outcomes of the 
reengineered business process. Finally, iEHR included 16 out of 19 
performance metrics with quantitative outcomes for the five 
capabilities' target processes; however, performance baseline values 
were not provided for any of the current processes. Establishing a 
performance baseline from which to measure improvements and defining 
indicators related to quality, cost, and time is key to knowing if the 
reengineered process has produced the desired effect.

Further, while DOD included five examples of business process 
improvements in its most recent annual report, it did not indicate how 
these examples were tied to its business process reengineering efforts 
either for a specific business process or across the larger end-to-end 
business processes. DOD officials noted that they intend to track this 
information better in the future, but did not describe how and where 
this information would be reported. Until the department develops and 
reports on specific performance measures, as we have previously 
recommended, the department and its stakeholders will not know the 
extent to which business process reengineering is effectively 
streamlining and improving its business processes needed to transform 
the department as intended.

Office of the Deputy Chief Management Officer Needs a Strategic 
Approach to Managing its Human Capital Needs: 

The success of any program depends on effectively leveraging people, 
processes, and tools to achieve defined outcomes and results. To 
effectively leverage people, they must be treated as strategic assets. 
As we have previously reported,[Footnote 64] a strategic approach to 
human capital management enables an organization to be aware of and 
prepared for its current and future human capital needs, such as 
workforce size, knowledge, skills, and training. To be effective, our 
research shows that such a strategic approach includes using data-
driven, fact-based methods to (1) assess the knowledge and skills 
needed to execute a program; (2) inventory existing staff knowledge 
and skills; (3) forecast the knowledge and skills needed over time; 
(4) analyze the gaps in capabilities between the existing staff and 
future workforce needs, including consideration of evolving program 
and succession needs caused by turnover and retirement; and (5) 
formulate strategies for filling expected gaps.

The Office of the Deputy Chief Management Officer has yet to determine 
and follow such a strategic approach to managing its human capital 
needs. In particular, in addressing its staffing needs, the office has 
not used a documented, fact-based, data-driven methodology to assess 
needs and existing capabilities, nor has it performed a gap analysis 
of the number of staff required and the specific skills and abilities 
needed to effectively achieve its mission of leading and enabling end-
to-end integration and improvement of business operations in support 
of national security.

In 2012 we reported[Footnote 65] that the office had defined an 
organizational structure and identified the staff resources it would 
need to fulfill responsibilities associated with business systems 
modernization, including leading efforts to develop the BEA and 
enterprise transition plan. This structure had become effective in 
October 2011 when the Business Transformation Agency was formally 
eliminated. However, it had not filled many of the positions needed to 
execute these responsibilities. In particular, as of April 2012, it 
had filled only 82 of its planned 139 positions, with 57 positions (41 
percent) remaining unfilled. Officials from the Office of the Deputy 
Chief Management Officer attributed the unfilled positions to, among 
other things, challenges associated with the length of time between 
when DOD announced that the Business Transformation Agency, which had 
previously addressed many of the Office of the Deputy Chief Management 
Officer's current functions, would be disestablished (August 2010) and 
when the agency was formally disestablished (October 2011). For 
example, some staff chose to seek employment elsewhere due to 
uncertainties associated with the transition. In response, we 
recommended that DOD include an update on the Office of the Deputy 
Chief Management Officer's progress toward filling staff positions and 
the impact of any unfilled positions on the ability of the office to 
conduct its work in its annual report to Congress on its business 
systems modernization.

Since April 2012, DOD has eliminated 42 full-time equivalent 
positions, leaving 97 planned positions in the Office of the Deputy 
Chief Management Officer. According to the Assistant Deputy Chief 
Management Officer, 40 of the 42 positions were eliminated as part of 
the Deputy Secretary of Defense's efforts to find efficiencies within 
the department. As of April 2013, the office reported having filled 
one more than the 82 positions it had filled in April 2012, still 
leaving 14 positions unfilled. According to officials from the Office 
of the Deputy Chief Management Officer, the department has imposed a 
90 percent hiring limit due to sequestration. As a result, these 
officials stated that the office is not able to fill 10 of the planned 
positions.

In March 2013, DOD officials reported changes in the responsibilities 
of the Deputy Chief Management Officer-related functions. These 
changes included the realignment of responsibilities among Deputy 
Chief Management Officer directorates (e.g., moving the responsibility 
for conducting business process reengineering assessments from its 
Investment and Acquisition Management Directorate to its Enterprise 
Business Integration Directorate), the elimination of several 
responsibilities (e.g., the office is no longer executing enterprise 
risk assessments for Milestone Decision Authorities), and the addition 
of other responsibilities (e.g., the Technology and Innovation 
Directorate is now responsible for developing and designing business 
engineering methodologies to advance Business Mission Area 
objectives). While the office has subsumed many of the 
responsibilities that were once managed by DOD's Business 
Transformation Agency, the Office of the Deputy Chief Management 
Officer has fewer staff and contractor resources. For example, 
according to DOD budget documentation, the Business Transformation 
Agency had more than 200 civilian staff and more than 600 contract 
staff during fiscal year 2010. However, as of April 2013, the Office 
of the Deputy Chief Management Officer consisted of five directorates 
and was staffed with 83 government employees and 150 contractor staff. 
Table 5 shows the organizational components of the Office of the 
Deputy Chief Management Officer and key responsibilities.

Table 5: Organizational Components and Key Responsibilities of the 
Office of the Deputy Chief Management Officer: 

Organizational component: Investment and Acquisition Management 
Directorate; 
Key responsibilities: Report to Congress on progress and improvements 
made in the DOD Business Mission Area; Develop the enterprise 
transition plan; Provide acquisition oversight and lead IT acquisition 
reform; Conduct and execute investment portfolio reviews. 

Organizational component: Enterprise Business Integration Directorate; 
Key responsibilities: Reengineer and apply end-to-end processes to 
improve business operations and support audit readiness; Manage and 
oversee the appropriate end-to-end governance model(s) and forum(s); 
Identify and analyze requirements for BEA development. 

Organizational component: Technology and Innovation Directorate; 
Key responsibilities: Build and deliver the BEA; Develop and promote 
enterprise standards for business architecture; Manage and execute BEA 
statutory requirements. 

Organizational component: Planning and Performance Management 
Directorate; 
Key responsibilities: Develop the Strategic Management Plan; Improve 
performance management infrastructure and leverage continuous process 
improvement methodologies; Conduct organizational assessments and 
develop organizational guidance to set forth the priority performance 
outcomes for the department. 

Organizational component: Operations Directorate; 
Key responsibilities: Manage the day-to-day operations of the Office 
of the Deputy Chief Management Officer (e.g., human resources, 
budgeting, IT). 

Source: GAO analysis based on DOD documentation. 

[End of table] 

The Office of the Deputy Chief Management Officer has conducted 
analyses of the functions that the office is to perform, but has not 
conducted human capital analyses. More specifically, to its credit, 
the Office of the Deputy Chief Management Officer identified the 
functions that each of the office's directorates was to perform as the 
office was being established. The office also recently revisited these 
functions. However, the Assistant Deputy Chief Management Officer 
stated that the office has not conducted human capital analyses and 
that no plans exist to analyze and address skill gaps. Instead, the 
Assistant Deputy Chief Management Officer stated that the office 
focuses on addressing its priorities by working with its existing 
staff and leveraging staff from entities and individuals across the 
department who participate in related entities (e.g., the Defense 
Business Council) and efforts (e.g., end-to-end business process 
activities).

Officials from the Office of the Deputy Chief Management Officer 
stated that the current level of staffing and capabilities is 
sufficient to effectively achieve its mission and related functions. 
Nevertheless, the department has had a number of challenges in 
addressing its mission of achieving its vision relative to developing 
a federated BEA, an enterprise transition plan, an investment 
management process, and a strategic plan, as described in this report 
and our recent report on business transformation efforts.[Footnote 66] 
For example, we found that while DOD had made some improvements to its 
Strategic Management Plan, further refining the plan to include key 
information such as performance measures that fully reflect core 
activities needed to assess progress could help DOD better prioritize 
and target its reform efforts to address the underlying causes of its 
systemic business challenges and to achieve results. Unless the Office 
of the Deputy Chief Management Officer adopts a more strategic and 
proactive approach to managing its human capital, it will continue to 
be challenged in its ability to help ensure that the required 
capabilities and mission value for approximately 1,200 business system 
investments--totaling about $6.8 billion in approved funding for 
fiscal year 2013--will be delivered in a timely and cost-effective 
manner.

Key Recommendations Have Not Yet Been Implemented: 

Between 2005 and 2008, we made 48 recommendations to assist DOD in 
developing a well-defined and useful BEA and business system 
investment management approach and in gaining better insight into and 
control over its ongoing business system investments, which are 
consistent with the NDAA requirements. Of these, we reported that DOD 
had implemented 31, but 17 recommendations were not implemented. For 
example, the department had not: 

* Issued an architecture management plan that addresses, among other 
things, how business architecture federation will be governed, how the 
alignment of component business architectures with incremental 
versions of the BEA will be achieved, and what milestones will be used 
to measure progress and results.

* Directed the responsible authorities in the department to use the 
program-specific data in the compliance assessment tool for 
identifying and analyzing potential overlap and duplication, and thus 
opportunities for reuse and consolidation among programs and provide 
programs access rights to use this functionality.

* Explicitly assigned responsibility for validating program BEA 
compliance assertions to military departments and defense agencies or 
issued guidance that describes the nature, scope, and methodology for 
doing so.

Since 2009, we have made 15 additional recommendations to assist DOD 
in its business systems modernization efforts. (See app. III for 
details on the status of these 15 recommendations.) As of April 2013, 
the department had fully implemented 3 recommendations, with 12 
remaining to be addressed. Specifically, the department has fully 
implemented our recommendations[Footnote 67] to: 

* include provisions in its guidance requiring certification 
submissions to include any of our open recommendations for program 
weaknesses as well as the status of actions to address these 
recommendations;

* disestablish the Business Transformation Agency and complete the 
transfer of its various responsibilities to other DOD entities; and: 

* ensure more accurate categorizations of its IT investments.

In addition, the department has begun to take steps to address some of 
the remaining 12 recommendations, including developing new policies 
and guidance and establishing new governance entities, but additional 
actions are needed before the remaining recommendations are fully 
addressed. For example, in response to our recommendation[Footnote 68] 
that the department establish a policy that clarifies the roles, 
responsibilities, and relationships among the various entities 
associated with the development of a federated BEA, the department 
stated that it has a number of policy and guidance documents currently 
under development to help clarify the roles and responsibilities 
associated with development of the BEA and enterprise architecture in 
general across DOD. For example, DOD has established a forum for 
discussing the development of the federated BEA and is currently 
working to establish a BEA Configuration Control Board charter, which 
is intended to document the roles and responsibilities for the review 
and approval of proposed BEA content. However, DOD has yet to issue 
these policy and guidance documents.

Officials from the Office of the Deputy Chief Management Officer cited 
various reasons for not fully implementing our open recommendations, 
including changes to the act's requirements that significantly 
expanded the number of systems subject to certification and approved 
as compliant with the BEA and having completed sufficient business 
process reengineering and prioritizing changes to balance competing 
priorities. It is important that the department move swiftly in doing 
so because these recommendations are aimed at strengthening 
architecture (and transition planning) management activities and 
controlling ongoing and planned business system investments. Until it 
does, the likelihood of sustained incremental improvement to its 
modernization management controls will be diminished and the means of 
holding the department accountable for such improvement will be missing.

Conclusions: 

Over the last year, the department has continued to take a number of 
steps to comply with the provisions of the act, including significant 
changes to the requirements for investment review and certification of 
defense business systems. The department has begun to improve the 
content of its BEA and enterprise transition plan, but additional 
content is still needed to fully address the act's requirements and to 
make the BEA and plan more useful for modernizing the defense business 
systems enterprise. In addition, while DOD has begun to improve the 
information it uses to help manage its portfolio of business system 
investments, it has yet to fully implement its plans for validating 
the completeness and reliability of the information. DOD has also 
taken steps, in accordance with the act, to establish a portfolio-
based approach to reviewing and certifying its defense business 
systems. Nevertheless, the lack of an established foundation and 
criteria and procedures for making portfolio-based investment 
decisions will continue to inhibit the department's ability to make 
sound, portfolio-based investment decisions that align with mission 
priorities. Further, DOD has yet to ensure the accuracy of BEA 
alignment through validation of individual investments. Moreover, 
while the department has begun taking steps to reengineer its business 
systems and processes, the lack of appropriate business process 
reengineering assertions and the reporting of associated results and 
outcomes puts the department at risk of not being able to determine 
whether business processes are streamlined and efficient as 
practicable. Further, it puts the department at risk of noncompliance 
with the statutory restriction on obligation of funds. Finally, the 
efforts of the Office of the Deputy Chief Management Officer have been 
impacted by the lack of a strategic approach to managing its human 
capital needs. To date, DOD has not implemented 29 of the 63 
recommendations in our reports and the modernization program remains 
on our High Risk List.

To successfully accomplish its modernization program, it is important 
for DOD to complete all intended steps for implementation of the 
program: improve its federated BEA, enterprise transition plan, and 
its investment management approach; validate the completeness and 
reliability of its portfolio of business system investments; implement 
and use the BEA and business process reengineering compliance 
assessments more effectively; and address its human capital needs. 
Until the department takes such needed steps, it is unlikely that its 
approximately 1,200 business system investments--totaling about $6.8 
billion in approved funding for fiscal year 2013--will be managed in 
an effective manner that maximizes mission performance while 
minimizing or eliminating system overlap and duplication.

Recommendations for Executive Action: 

We are recommending that the Secretary of Defense direct the Deputy 
Chief Management Officer to take the following six actions to 
effectively implement key components of DOD's business systems 
modernization program: 

* Define by when and how the department plans to develop an 
architecture that would extend to all defense components and include, 
among other things, (a) information about the specific business 
systems that support BEA business activities and related system 
functions; (b) business capabilities for the Hire-to-Retire and 
Procure-to-Pay business processes; and (c) sufficient information 
about business activities to allow for more effective identification 
of potential overlap and duplication.

* Define by when and how the enterprise transition plan will include, 
among other things, (a) milestones, performance measures, and funding 
plans for all business systems expected to be part of the target 
architecture and each system's risks or challenges to integration; (b) 
time-phased end dates associated with terminating legacy systems in 
phases; (c) a listing of all other defense business systems (including 
systems that are considered to be core systems) that will be a part of 
the target defense business systems computing environment and a 
strategy for making modifications to those systems that will be needed 
to ensure that they comply with the defense BEA, including time-phased 
milestones, performance measures, and financial resource needs; and 
(d) information about how systems are to be sequenced according to, 
among other things, dependencies among investments.

* Ensure that the functional strategies include all of the critical 
elements identified in DOD investment management guidance, including 
performance measures to determine progress toward achieving the goals 
that incorporate all of the attributes called for in the department's 
guidance.

* Select and control its mix of investments in a manner that best 
supports mission needs by (a) documenting a process for evaluating 
portfolio performance that includes the use of actual versus expected 
performance data and predetermined thresholds; (b) ensuring that 
portfolio assessments are conducted in key areas identified in our IT 
investment management framework: benefits attained; current schedule; 
accuracy of project reporting; and risks that have been mitigated, 
eliminated, or accepted to date; and (c) ensuring that the documents 
provided to the Defense Business Council as part of the investment 
management process include critical information for conducting all 
assessments.

* Implement and use the BEA and business process reengineering 
compliance assessments more effectively to support organizational 
transformation efforts by (a) disclosing relevant information about 
known weaknesses, such as BEA and business process reengineering 
compliance weaknesses for systems that were not certified or certified 
with qualifications in annual reports to Congress; (b) establishing 
milestones by which selected validations of BEA compliance assertions 
are to be completed; and (c) ensuring that appropriate business 
process reengineering assertions have been completed on all 
investments submitted for the fiscal year 2014 certification reviews 
prior to the certification of funds.

* Develop a skills inventory, needs assessment, gap analysis, and plan 
to address identified gaps as part of a strategic approach to human 
capital planning for the Office of the Deputy Chief Management Officer.

In addition, we are recommending that the Secretary of Defense direct 
the appropriate authority to ensure that complete documentation, such 
as root cause analyses, assessments of existing interfaces for reuse 
opportunities, and performance metrics related to the reengineering 
efforts, is provided as part of the fiscal year 2014 certification and 
approval process for the IPPS-A, IPPS-N, AF-IPPS, and iEHR investments.

We further recommend that the Secretary direct the appropriate 
authority to determine whether funds were properly obligated under 10 
U.S.C. § 2222(a)-(b) for systems for which appropriate business 
process reengineering assertions were not completed.

Agency Comments and Our Evaluation: 

In written comments on a draft of this report, signed by the Deputy 
Chief Management Officer, DOD concurred with two of our eight 
recommendations, partially concurred with three recommendations, and 
did not concur with three recommendations. The department stated that 
it disagreed with many of the assertions put forward by us and 
believed that there was a lack of balance expressed in the report. It 
also stated that the report demonstrated a lack of understanding of 
the department's new investment management process and the significant 
improvements it has produced, such as revamping the business mission 
area governance and establishing an Integrated Business Framework in 
response to the NDAA for Fiscal Year 2012. The department added that 
these changes have resulted in significant progress rather than the 
slow improvements portrayed in the report. Further, the department 
stated that while its business systems modernization efforts will 
continue to evolve, it is important for the department and Congress 
not to lose sight of the bigger picture of what they endeavor to 
achieve by focusing only on the set of challenges cited in our report. 
It remarked that DOD's business mission area is a complex, 
interconnected environment that is unprecedented in size. Finally, it 
stated that there is significant focus in the report on the 
department's failure to implement recommendations that are almost a 
decade old and that have been overtaken by more recent events. DOD's 
comments are reprinted in their entirety in appendix IV.

In contrast to DOD's stated position, we believe the report provides a 
balanced view of DOD's efforts and reflects an appropriate level of 
understanding of the department's new, but still evolving, investment 
management process. For example, in our report, we recognized that DOD 
had continued to add needed content, such as data standards, business 
rules, and performance measures to its architecture, as called for by 
the act. However, we also noted that while DOD had initiated a number 
of efforts to federate its BEA to the components, it remained unclear 
as to when and how it would develop an architecture that covers all 
defense business systems and their related functions and activities. 
As another example, we reported that to its credit, the department had 
made a number of improvements to its functional strategies for the 
fiscal year 2014 certification cycle, but also noted that it had yet 
to establish performance measures in which all key attributes 
identified in DOD guidance are present in the strategies.

Our report also appropriately focused on the scope of work called for 
by the act's reporting requirements and recognized that while DOD has 
initiated numerous management activities, such as the Integrated 
Business Framework, aimed at modernizing its business systems 
environment, it has also been limited in its ability to demonstrate 
results. We also recognize that DOD's business mission area is a 
complex, interconnected environment that is unprecedented in size. In 
this regard, our study focused on the management controls, as defined 
in the NDAA and our prior recommendations, which are vital to ensuring 
that the department can effectively and efficiently manage an 
undertaking with the size, complexity, and significance of its 
business systems modernization and minimize the associated risks. 
Further, our report appropriately focused on the same challenges that 
DOD has been facing in modernizing its business systems and that 
prompted the recommendations in the first place, regardless of their 
age. For example, since 2001, we have provided a series of 
recommendations relative to developing and using a business enterprise 
architecture and establishing effective investment management controls 
to guide and constrain DOD's multibillion-dollar business systems and 
services. In addition, since 2002, Congress has included provisions 
consistent with GAO's recommendations in National Defense 
Authorization Acts, which DOD has yet to fully address. Our report 
also appropriately addressed both the department's progress (to 
include the implementation of 34 out of 63 recommendations), as well 
as the long-standing challenges that we had previously identified and 
that remained to be addressed during our study. DOD has not provided 
evidence to demonstrate progress in addressing a number of these 
challenges and our related prior recommendations and, as a result, the 
modernization program remains on our High Risk List. Further, while we 
support any departmental efforts, whether completed or ongoing, that 
would address the challenges cited in our report, we note that DOD did 
not disagree with the facts as presented in the report. Therefore, we 
stand by our findings and recommendations.

DOD's specific comments on each recommendation, along with our 
responses to its comments, follow.

* The department partially concurred with our first recommendation to 
define by when and how it plans to develop an architecture that would 
extend to all defense components and include missing content. In this 
regard, it agreed that plans for continuing to mature the federated 
BEA should be completed. It also noted that the department had already 
taken the necessary initial steps to extend the BEA to all defense 
components that are institutionalized via the Integrated Business 
Framework, which was implemented during calendar year 2012. In 
particular, the department stated that its Integrated Business 
Framework links the development of functional business strategies, the 
identification of associated BEA changes, and the submission of 
organizational execution plans from the components. It added that the 
information contained in the component-developed organizational 
execution plans constitutes key elements of the components' portions 
of the federated BEA.

We recognize that the functional strategies and organizational 
execution plans include information that might be incorporated into 
the BEA, such as information about enterprise standards and 
performance measures. We also acknowledged, in the report, the steps 
that DOD is taking to mature its BEA content, including establishing a 
forum for generating recommendations regarding how to proceed with the 
architecture. Nevertheless, while the department stated that it has 
taken initial steps to federate the BEA to all defense components, it 
has yet to articulate, as recommended, by when and how it will fully 
achieve its federated BEA. Thus, while the department has various 
efforts in place that may contribute content to its federated business 
architecture, as described in the report, these efforts have yet to 
result in a federated business architecture that extends to all 
defense components, and the architecture continues to lack critical 
content needed to achieve the department's vision for an architecture 
that can be used to guide, constrain, and enable interoperable 
business systems. Thus, we continue to stress full implementation of 
this recommendation.

* DOD partially concurred with our second recommendation to define by 
when and how the enterprise transition plan will include, among other 
things, milestones, performance measures, and funding plans for all 
business systems expected to be part of the target architecture and 
each system's risks or challenges to integration, as identified in our 
report. In its comments, DOD referred to specific actions and 
processes that it states meet the act's requirements. For example, DOD 
stated that milestones, performance measures, and funding plans for 
all business systems expected to be part of the target architecture 
and each system's risks or challenges to integration are explained in 
presentations provided to the Defense Business Council. It also stated 
that the listing of systems that will be part of the target defense 
business systems computing environment and strategy for making 
modifications to systems that will be needed to ensure that they 
comply with the defense BEA, including time-phased milestones, 
performance measures, and financial resource needs, is being met by 
the precertification process and through the defense acquisition 
milestones and performance measures. Further, it noted that 
information about how systems are to be sequenced according to, among 
other things, dependencies among investments is captured in 
presentations and supporting materials provided to the Defense 
Business Council.

However, while noting these actions, the Office of the Deputy Chief 
Management Officer did not provide evidence that such information is 
presented during the precertification process or to the Defense 
Business Council for every defense business system. With respect to 
the Defense Acquisition System, while the acquisition process might 
support and be informed by the development of an enterprise transition 
plan, the existence of the acquisition process does not invalidate the 
need for the plan to include the information called for by the act. 
Regardless of whether such information is presented to the Defense 
Business Council or addressed through other DOD processes, such as the 
department's acquisition process, the act still calls for the 
information to be included in the transition plan. Doing so will help 
ensure informed oversight of DOD business systems investments and help 
provide the department with a sufficient basis for sequencing business 
capabilities and system functions to be subsumed by a modern system.

DOD also referred to actions the department has taken to improve its 
approach to managing business systems investments, including 
establishing priorities in its Strategic Management Plan and 
developing its functional strategies and organizational execution 
plans, which are discussed in our report. In addition, the department 
referred to its acquisition and budgeting processes, which it says 
provides DOD with greater visibility into the business enterprise than 
ever before. While we recognize that these actions and processes are 
important parts of the department's efforts to manage its business 
systems investments, our recommendation focuses on the need for DOD to 
describe by when and how gaps that we identified in the report would 
be included in its enterprise transition plan. Doing so would promote 
visibility across the existing DOD processes and provide a basis for 
coordinating modernization activities and the concurrent development 
of business systems in a manner that limits unnecessary duplication 
and increases the likelihood that systems will be interoperable. 
Consistent with the department's data-driven approach to managing its 
business systems, the approach described by officials from the Office 
of the Deputy Chief Management Officer to addressing its transition 
plan gaps is to leverage information that may already exist across the 
department. Accordingly, if existing processes already collect data 
that are required to be included in the plan, then the department 
should leverage and incorporate these data in the plan. Such an 
approach would provide DOD with an improved basis for sequencing 
business capabilities and system functions to be subsumed by modern 
systems while limiting unnecessary duplication.

Further, with regard to including time-phased end dates associated 
with terminating legacy systems in phases, DOD stated that this 
requirement is being met by the establishment of a category of 
"legacy" investments that are to be retired within the next 36 months 
along with an explanation of what, if any, systems will continue to 
support the business need. The department also stated that all other 
defense business systems are considered core systems and, therefore, 
must demonstrate compliance with the BEA while ensuring appropriate 
business process reengineering. Accordingly, the department stated 
that it does not require that its legacy systems demonstrate 
compliance with the BEA. While we agree that it may not be an 
efficient use of resources to modify legacy systems that will soon be 
retired, the act and our recommendation focuses on the importance for 
all the other defense business systems (including core systems since 
they will be part of the target environment) to be included in the 
enterprise transition plan and taking a phased approach to eliminating 
systems. As such, information about how systems will be retired in 
phases and associated milestones remains an important part of the 
transition plan. Thus, we stand by our recommendation, but have added 
additional clarification to the recommendation and to our discussion 
of this matter in the body of the report to reflect DOD's approach for 
addressing legacy systems.

* The department concurred with our third recommendation to ensure 
that the functional strategies include all of the critical elements 
identified in DOD investment management guidance.

* The department did not concur with our fourth recommendation to 
select and control its mix of investments in a manner that best 
supports mission needs by (a) documenting a process for evaluating 
portfolio performance that includes the use of actual versus expected 
performance data and predetermined thresholds; (b) ensuring that 
portfolio assessments are conducted in key areas identified in our IT 
investment management framework: benefits attained; current schedule; 
accuracy of project reporting; and risks that have been mitigated, 
eliminated, or accepted to date; and (c) ensuring that the documents 
provided to the Defense Business Council as part of the investment 
management process include critical information for conducting all 
assessments. The department noted that our recommendation did not 
recognize the significant improvements the department made in 2012 to 
fundamentally change how it reviews its defense business systems.

With regard to the first part of the recommendation about process 
documentation, DOD stated that its investment review guidance defines 
the review criteria used to evaluate and certify business systems via 
organizational execution plans and was updated based on lessons 
learned in 2012. It also stated that the department intends to 
withhold certification for business system investments that are not 
aligned to the functional strategies, unless a business system is 
required for critical operational support, and that its IRB does not 
intend to review acquisition issues associated with individual 
business systems since that function is performed by the Defense 
Acquisition System. Our report acknowledged that DOD's 2012 investment 
management guidance defines review criteria that the Defense Business 
Council is to use for evaluating and certifying portfolios of business 
systems rather than individual systems and credits the department for 
revising its guidance for the 2014 certification of defense business 
systems. However, the department has not specified a process for using 
the criteria to make portfolio-based certification decisions that 
incorporate the use of actual versus expected performance data and 
predetermined thresholds, which, according to our IT investment 
management framework, is a major factor in evaluating portfolio 
performance and defining remedial actions for underperforming 
investments.

With regard to the second part of our recommendation covering 
portfolio assessments, DOD stated that key components of GAO's IT 
investment management framework are being leveraged to evaluate the 
investment portfolios presented to the Defense Business Council and 
noted that the portfolio-based review process is intended to 
complement rather than duplicate DOD's Defense Acquisition System. 
According to the department, the Defense Acquisition System is the 
core process used by the department to manage defense acquisitions and 
it already contains significant elements of GAO's framework. We agree 
and acknowledged in our report that components of our framework are 
being leveraged in evaluating investment portfolios. However, the 
intent of our recommendation is not to duplicate already existing 
processes, but rather, to ensure that assessments in four key areas 
identified in our IT investment management framework (benefits 
attained, current schedule, accuracy of project reporting, and risks 
that have been mitigated, eliminated, or accepted to date) are 
incorporated and considered when making annual certification 
decisions. As discussed in our report, some investments may exceed 
performance expectations (e.g., delivering at lower cost, in less 
time, and with better benefits than expected) and in these cases, the 
board may wish to accelerate an investment's funding or schedule, 
reallocate resources within the overall portfolio, or make some other 
type of adjustment.

With regard to the third part of our recommendation on documentation 
for conducting assessments, the department stated that the information 
derived for strategic alignment, cost, utility, and compliance is 
already given to the Defense Business Council and incorporates 
objective information derived from authoritative documents and data 
sources. It also noted that the documentation is supported by input 
from the functional area owners and subject matter experts. However, 
as discussed in our report, the department's investment management 
guidance does not call for the organizational execution plans, which 
are the key documents used by the Defense Business Council to make 
certification decisions, to include critical information for 
conducting assessments associated with strategic alignment, utility, 
and total cost criteria.

Notwithstanding the actions DOD has taken in these areas, we continue 
to believe our recommendation, in its entirety, is warranted.

* With regard to our fifth recommendation, the department partially 
concurred. Specifically, DOD agreed with the first two parts of our 
recommendation that the BEA and business process reengineering 
compliance assessments be used to support organizational 
transformation efforts by disclosing relevant information about known 
weaknesses and identifying milestones for completing selected 
validations. However, it did not agree with the third part of the 
recommendation on ensuring that appropriate business process 
reengineering assertions have been completed for all investments 
submitted for the fiscal year 2014 certification reviews prior to the 
certification of funds. The department noted that requiring this 
assertion for systems solely in sustainment or scheduled for 
retirement would divert scarce resources to tasks that would provide 
no benefit. Further, it stated that its latest Business Process 
Reengineering Assessment Guidance of September 28, 2012, provided DOD 
with a business process reengineering standard, a plan for validating 
a sample of defense business programs' compliance assertions, and a 
requirement for precertification authorities to develop a Plan of 
Action for all defense programs that did not assert compliance in 
fiscal year 2013 to become compliant, which was also documented in 
related investment decision memoranda.

We have recognized all of these steps cited by the department in our 
report. Nonetheless, we stand by our recommendation for completing 
appropriate business process reengineering assertions for all 
investments, which is a requirement of the NDAA and in accordance with 
DOD guidance. Specifically, DOD's Business Process Reengineering 
Assessment Guidance describes how the standard DOD has developed 
should be applied to varying degrees depending on the life-cycle stage 
of each defense business system, providing a reduced burden for 
systems in sustainment and eliminating the requirement for those 
systems designated as legacy, which for fiscal year 2013 included 
about 99 systems. We acknowledge DOD's approach as described in its 
guidance, but maintain that a precertification authority would still 
need to make the determination that appropriate business process 
reengineering efforts had been undertaken. The precertification 
authority would also be able to assert and validate compliance for a 
system in sustainment if the system has followed DOD guidance and for 
a legacy system scheduled to be retired within 36 months because no 
reengineering was required. However, for the fiscal year 2013 reviews, 
the use of the action items requiring precertification authorities to 
develop an action plan for all defense business programs, which did 
not assert business process reengineering compliance, resulted in more 
than half of the covered business systems being asserted as non-
compliant at the time they were certified. We also acknowledge that 
the department is working to remedy this situation for fiscal year 
2014, but our recommendation has been made to ensure that asserting 
compliance, as required by the act, is a departmentwide priority. 
Thus, we continue to believe our recommendation is warranted.

* The department did not concur with our sixth recommendation to 
develop a skills inventory, needs assessment, gap analysis, and plan 
to address identified gaps as part of a strategic approach to human 
capital planning. In particular, DOD stated that we did not recognize 
that the Office of the Deputy Chief Management Officer is a relatively 
small component of the Office of the Secretary of Defense and that we 
had directed the office to undertake a level of activity normally 
designed for an entire agency. While we recognize that the office is a 
relatively small component of the overall department, as we reported, 
the success of any program depends on effectively leveraging people to 
achieve defined outcomes and results. Further, without knowing if it 
has the right number of staff required and the specific skills and 
abilities needed, the office will be challenged in its ability to 
effectively achieve its mission of leading and enabling end-to-end 
integration and improvement of business operations for approximately 
1,200 business system investments--totaling about $6.8 billion in 
approved funding for fiscal year 2013.

DOD also stated that the report misunderstands statements made by 
officials within the office and draws conclusions about the impact of 
staffing levels that are not supported by the report's findings. More 
specifically, the department stated that the office thoroughly 
considered the skills of its employees, sought to optimally align them 
organizationally, and has used hiring actions to bring additional 
skill sets onboard that filled identified gaps. While our report 
acknowledged the efforts the office has taken to conduct analyses of 
the functions that it is to perform, we also note that the office did 
not provide evidence that it has used a documented, fact-based, data-
driven, methodology to assess human capital needs and existing 
capabilities, or performed a gap analysis of the number of staff 
required and the specific skills and abilities needed to effectively 
achieve its mission.

The department further noted that the report attributes consequences 
that are beyond what is supported by the report's findings and that 
weaknesses we identified in the department's federated architecture, 
investment management data, and transition plan content are 
departmental weaknesses and are not simply due to the staffing levels 
of the Office of the Deputy Chief Management Officer. While we 
recognize DOD's "tiered accountability" approach to business systems 
modernization, where responsibility and accountability for business 
architectures and systems investment management are assigned to 
different levels in the department, we do not agree with the 
department's assertions. Our report does not link the weaknesses to 
the staffing levels but rather it states that the efforts of the 
Office of the Deputy Chief Management Officer have been impacted by 
the lack of a strategic approach to managing its human capital needs. 
Given the level of oversight entrusted to the office relative to 
business systems modernization, it is important that it address the 
weaknesses identified in our report. Thus, it is imperative for the 
office to help ensure that its workforce is appropriately sized and 
has the right mix of knowledge, skills, and training. Adopting a more 
strategic and proactive approach to managing its human capital will be 
a positive step forward in helping to ensure that such weaknesses will 
be addressed in a timely and cost-effective manner. As such, we stand 
by this recommendation.

* The department concurred with our seventh recommendation to ensure 
that complete documentation, such as root cause analyses, assessments 
of existing interfaces for reuse opportunities, and performance 
metrics related to the reengineering efforts, are provided as part of 
the fiscal year 2014 certification and approval process for the IPPS-
A, IPPS-N, AF-IPPS, and iEHR investments. It stated that its Business 
Process Reengineering Assessment Guidance of September 28, 2012, 
addressed our concerns in its standard questions. We support the 
department's efforts to address our recommendation and reiterate the 
importance of ensuring that complete documentation, in support of the 
standard questions, is provided as part of the certification and 
approval process.

* The department did not concur with our eighth recommendation to 
determine whether funds were properly obligated under 10 U.S.C. 
§2222(a)-(b) for systems for which appropriate business process 
reengineering assertions were not completed. DOD stated that it 
believes that it has already met the conditions of the act through 
reviews by the precertification authorities and the Defense Business 
Council. The department noted, as an example, that it considers any 
existing defense business systems with a retirement date within 36 
months of certification to be legacy systems, so development and 
modernization funds for these legacy systems have been removed from 
the budget. It further noted that the requirement to retire these 
systems is included in all relevant documentation generated; is 
approved by the Defense Business Council and the Defense Business 
Systems Management Committee; and therefore has been reviewed for 
compliance. DOD stated that its approach to legacy systems scheduled 
for retirement conserves resources of the department. The department 
also stated that all other defense business systems (including 
existing systems that will be modernized) are considered core systems 
and, therefore, must demonstrate compliance to the BEA while ensuring 
appropriate business process reengineering. Further, the department 
stated that its revised Business Process Reengineering Assessment 
Guidance memorandum of September 28, 2012, specifically addressed the 
requirement for the precertification authority to develop a Plan of 
Action for all defense business programs that did not assert business 
process reengineering compliance in fiscal year 2013 to become 
compliant; this requirement was also documented in the investment 
decision memorandum for each organizational execution plan as a 
condition to the fiscal year 2013 certification of funds.

Consistent with a determination that appropriate business process 
reengineering efforts have been undertaken, we acknowledge that a 
legacy system scheduled to be retired within 36 months would be 
asserted in compliance because no reengineering was required. However, 
for all other defense business systems that are not legacy systems 
scheduled for retirement, we do not agree that the department has 
already met the conditions of the act through reviews by the 
precertification authorities and the Defense Business Council. We 
maintain that, consistent with the act, a precertification authority 
would still need to make the determination that appropriate business 
process reengineering efforts have been undertaken as a condition for 
obligating funds for covered defense business system programs. We 
acknowledge that DOD has some discretion in deciding what constitutes 
the undertaking of appropriate business process reengineering efforts. 
Further, the act provides alternative bases for certification prior to 
the obligation of funds that DOD could consider in some situations. 
However, the majority of systems did not assert business process 
reengineering compliance at the time they were certified and DOD's use 
of a Plan of Action in lieu of determining that appropriate business 
process reengineering efforts have been undertaken does not directly 
address our recommendation to determine whether funds were properly 
obligated under the act. Indeed, DOD's description of its actions 
actually highlights the apparent absence of compliance with the 
condition on the certification of funds for the majority of the 
systems. In this regard, the subsequent higher-level review and 
certification of these systems does not rectify the absence of the 
necessary determinations by the precertification authorities, as 
required by the act. Accordingly, we reassert our recommendation.

We are sending copies of this report to the appropriate congressional 
committees; the Director, Office of Management and Budget; the 
Secretary of Defense; and other interested parties. This report also 
is available at no charge on the GAO website at [hyperlink, 
http://www.gao.gov].

If you or your staff members have any questions on matters discussed 
in this report, please contact me at (202) 512-6304 or 
melvinv@gao.gov. Contact points for our Offices of Congressional 
Relations and Public Affairs may be found on the last page of this 
report. GAO staff who made major contributions to this report are 
listed in appendix V. 

Signed by: 

Valerie C. Melvin: 
Director: 
Information Management and Technology Resources Issues: 

List of Committees: 

The Honorable Carl Levin: 
Chairman: 
The Honorable James M. Inhofe: 
Ranking Member: 
Committee on Armed Services: 
United States Senate: 

The Honorable Dick Durbin: 
Chairman: 
The Honorable Thad Cochran: 
Ranking Member: 
Subcommittee on Defense: 
Committee on Appropriations: 
United States Senate: 

The Honorable Howard P. McKeon: 
Chairman: 
The Honorable Adam Smith: 
Ranking Member: 
Committee on Armed Services: 
House of Representatives: 

The Honorable C.W. Bill Young: 
Chairman: 
The Honorable Pete Visclosky: 
Ranking Member: 
Subcommittee on Defense: 
Committee on Appropriations: 
House of Representatives: 

[End of section] 

Appendix I: Objective, Scope, and Methodology: 

Our objective was to assess the actions by the Department of Defense 
to comply with section 332 of the National Defense Authorization Act 
for Fiscal Year 2005 and amendments.[Footnote 69] These include (1) 
developing a business enterprise architecture (BEA) to cover all 
defense business systems, (2) developing a transition plan for 
implementing the architecture, (3) identifying systems information in 
its annual budget submission, (4) establishing a system investment 
approval and accountability structure along with an investment review 
process, and (5) certifying and approving any business system program 
costing in excess of $1 million. In addition, we reviewed the 
department's efforts to develop a strategic approach to human capital 
management and address our prior open recommendations.

To address the provision associated with developing a business 
architecture, we analyzed version 10.0 of the BEA, which was released 
in February 2013 relative to the act's architectural requirements, 
DOD's September 2011 Strategic Management Plan, and recommendations 
that our previous annual reports in response to the act identified as 
not being fully implemented. Specifically, we interviewed officials 
from the Office of the Deputy Chief Management Officer and reviewed 
written responses and related documentation on steps completed, under 
way, or planned to address the act's requirements and previously 
reported weaknesses. We then reviewed architectural artifacts in BEA 
10.0 to validate the responses and identify any discrepancies. In 
addition, we reviewed documentation and interviewed officials from the 
Office of the Deputy Chief Management Officer and the military 
departments about efforts to establish a federated business mission 
area enterprise architecture.

To address the provision associated with developing a transition plan, 
we analyzed DOD's enterprise transition plan, released in December 
2012, relative to the act's transition plan requirements and related 
findings documented in previous GAO reports. Specifically, we 
interviewed officials from the Office of the Deputy Chief Management 
Officer and reviewed written responses and related documentation on 
steps completed, under way, or planned to address the act's 
requirements and previously reported weaknesses. We then reviewed 
enterprise transition plan documentation (e.g., organizational 
execution plans and Office of Management and Budget (OMB exhibit 
300s)) to validate the responses and identify any discrepancies.

To assess the system information in DOD's fiscal year 2013 information 
technology budget submission, we analyzed and compared information 
contained in the department's system that is used to prepare its 
budget submission--Select and Native Programming Data Input System--
Information Technology (SNAP-IT) with information in the department's 
authoritative business systems inventory--Defense Information 
Technology Portfolio Repository (DITPR). We were unable to determine 
whether DOD's fiscal year 2014 information technology budget 
submission was prepared in accordance with the criteria set forth in 
the act because the budget submission was not released in time for us 
to review it for this report. We also interviewed officials from the 
Office of the Deputy Chief Management Officer and office of DOD's 
Chief Information Officer to discuss the accuracy and 
comprehensiveness of information contained in the DITPR system, the 
discrepancies in the information contained in the DITPR and SNAP-IT 
systems, and efforts under way or planned to address these 
discrepancies and our prior recommendations.

To address the act's provision for establishing an investment approval 
and accountability review structure along with an investment review 
process, we reviewed and analyzed the department's current policies, 
procedures, and guidance relative to the act's requirements and 
criteria documented in our IT Investment Management framework. 
[Footnote 70] Specifically, to determine whether a single Investment 
Review Board, composed of appropriate senior executives, had been 
established, we reviewed the October 2012 Defense Business Council 
charter. To determine whether the foundation for DOD's new investment 
management process had been fully established, we reviewed and 
analyzed the six functional strategies prepared for the fiscal year 
2013 certification cycle on the extent to which they included the 
critical elements called for in the department's June 2012 investment 
management guidance. We also reviewed and analyzed the investment 
management guidance to determine whether criteria and procedures had 
been fully defined for making portfolio-based investment decisions and 
the extent to which they aligned to our IT investment management 
framework. In addition, we reviewed and analyzed the April 2013 
revised investment management guidance and updated versions of the six 
functional strategies prepared for the fiscal year 2014 certification 
cycle to determine whether improvements had been made. Finally, we 
interviewed cognizant officials in the Office of the Deputy Chief 
Management Officer to gain a better understanding of the process and 
its outputs and planned improvements for fiscal year 2014 defense 
business system certifications.

To determine whether the department was certifying and approving 
business system programs costing in excess of $1 million in accordance 
with the act's provisions, we reviewed and analyzed DOD's fiscal year 
2013 report to Congress, all organizational execution plans and their 
respective precertification request memoranda, and associated 
portfolio attachments submitted as part of the fiscal year 2013 
investment review process. This also included reviewing and analyzing 
investment decision memoranda which documented the decisions made by 
the Defense Business Council. We also compared our results with the 
information reported in DOD's annual report to Congress to identify 
any discrepancies and discussed these with cognizant officials from 
the Office of the Deputy Chief Management Officer. Due to DOD's manual 
data cleanup efforts during the fiscal year 2013 investment review 
process, we were unable to verify the exact number and value of 
certification requests.

As part of our evaluation under this requirement, we also sought to 
identify the process associated with asserting BEA and business 
process reengineering compliance for covered defense business systems 
and the documentation used to support those assertions. To accomplish 
this, we reviewed DOD guidance and related documentation that applied 
to the fiscal year 2013 investment portfolio reviews and relevant 
updates to that guidance issued during our audit. We then applied the 
guidance to four investments that we selected--Air Force Integrated 
Personnel and Pay System, Integrated Personnel and Pay System-Navy, 
Integrated Personnel and Pay System-Army, and the Integrated 
Electronic Health Record System--based on the following criteria: the 
investment was (1) one of the top 25 largest dollar business system 
programs identified using the 2013 IT systems budget submission, (2) 
not the recent subject of DOD Inspector General review that included 
review of business process reengineering activities, and (3) an 
example from one of the military departments or defense agency or 
activity. In reviewing these investments, we analyzed and compared BEA 
and business process reengineering supporting documentation that was 
submitted as part of compliance assertions for fiscal year 2013 
request for certification of funds, to requirements in DOD's 2011 BEA 
compliance guidance,[Footnote 71] DOD's business process reengineering 
assessment guidance,[Footnote 72] and our Business Process 
Reengineering Assessment Guide.[Footnote 73] The supporting 
documentation, among other things, documented the current and target 
processes and described business performance metrics. We also 
interviewed officials from each system's component--the Departments of 
Air Force, Army, Navy, and the Tricare Management Activity--and 
discussed the program-specific documentation submissions with program 
officials. In addition, we interviewed officials from the Office of 
the Deputy Chief Management Officer about steps completed, under way, 
or planned relative to BEA and business process reengineering 
compliance and validation activities. Further, we interviewed DOD 
officials about efforts to understand, track, and report outcomes 
related to business process reengineering efforts. We also reviewed 
DOD's annual report to Congress to determine if business process 
reengineering outcomes had been reported.

To determine the extent to which the Office of the Deputy Chief 
Management Officer had implemented a strategic approach to human 
capital management, we reviewed our own guidance and reports on 
federal agencies' workforce planning and human capital management 
efforts.[Footnote 74] We then met with officials from the Office of 
the Deputy Chief Management Officer, including the Assistant Deputy 
Chief Management Officer to discuss the office's human capital 
planning efforts and analyzed related documentation provided by the 
office on steps completed, under way, or planned to address strategic 
human capital management activities discussed in our guidance.

To determine the extent to which DOD had implemented our prior 
recommendations, we identified 48 recommendations related to DOD's 
business systems modernization efforts that we made in our reports 
from 2005 to 2008 and summarized the implementation status of each. 
For those recommendations that had not been implemented after more 
than 4 years since we issued the associated report, we considered them 
closed and not implemented. We then focused on the 15 recommendations 
related to DOD's business systems modernization efforts that we made 
in our reports from 2009 to 2013. In reviewing these 15 
recommendations, we obtained and analyzed documentation relative to 
corrective actions taken and planned. Documentation that we reviewed 
included DOD's 2012 vision for the BEA,[Footnote 75] the most recent 
version of DOD's BEA compliance guidance, and DOD's 2012 and 2013 
investment management process guidance. Further, we reviewed 
documentation associated with changes to the Office of the Deputy 
Chief Management Officer's staffing levels, as well as the 
department's enterprise transition plans for fiscal years 2011, 2012, 
and 2013, and its fiscal year 2013 report to Congress.

We conducted this performance audit at DOD and military department 
offices from October 2012 to May 2013, in accordance with generally 
accepted government auditing standards. Those standards require that 
we plan and perform the audit to obtain sufficient, appropriate 
evidence to provide a reasonable basis for our findings and 
conclusions based on our audit objectives. We believe that the 
evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objective. 

[End of section] 

Appendix II: DOD's End-to-End Business Processes: 

The department's BEA focuses on documenting information associated 
with its 15 end-to-end business process areas. In particular, the 
department's Strategic Management Plan has identified the Hire-to-
Retire and Procure-to-Pay business process areas as its priorities. 
According to the department, the process of documenting the needed 
architecture information also includes working to refine and 
streamline each of the associated end-to-end business processes.

Business process: Acquire-to-Retire; 
Description: Obtain, manage, and dispose of accountable and reportable 
property (capitalized and noncapitalized assets) through their entire 
life cycle. 

Business process: Budget-to-Report; 
Description: Plan, formulate, create, execute against, and report on 
the budget and business activities of the entity. 

Business process: Concept-to-Product; 
Description: Identify product needs, and plan and execute all 
necessary activities to bring a product from initial concept to full 
production. 

Business process: Cost Management; 
Description: Identify, collect, measure, accumulate, analyze, 
interpret, and communicate cost information to accomplish the many 
objectives associated with control, decision making, planning, and 
reporting. 

Business process: Deployment-to-Redeployment/Retrograde; 
Description: Plan, notify, deploy, sustain, recall, and reset tactical 
units to and from theaters of engagement. 

Business process: Environmental Liabilities; 
Description: Identify environmental cleanup, closure, or disposal 
issues that represent an environmental liability of the department, to 
develop cost estimates and expenditures related to the actions 
required to eliminate an identified environmental liability, and to 
report appropriate financial information about the environmental 
liability. 

Business process: Hire-to-Retire; 
Description: Plan for, hire, classify, develop, assign, track, account 
for, compensate, retain, and separate the persons needed to accomplish 
aspects of the DOD mission. 

Business process: Market-to-Prospect; 
Description: Establish marketing plans, identify target markets, plan 
and define marketing campaigns, execute marketing campaigns, and 
measure and evaluate the performance of marketing campaigns. 

Business process: Order-to-Cash; 
Description: Accept and process customer orders for services and/or 
inventory held for sale. 

Business process: Plan-to-Stock; 
Description: Plan, procure, produce, inventory, and stock materials 
used both in operations and maintenance as well as for sale. 

Business process: Procure-to-Pay; 
Description: Obtain goods and services. 

Business process: Proposal-to-Reward; 
Description: Life cycle of the grant process from the grantor 
perspective. It includes all the business functions necessary to plan, 
solicit, review, award, perform, monitor, and close out a grant. 

Business process: Prospect-to-Order; 
Description: Generate and sustain sales by pursuing qualified leads, 
employing effective sales techniques, efficient order processing, 
maintaining customer relationships and providing support functions to 
include service, personnel, and financial impacts. 

Business process: Service Request-to-Resolution; 
Description: Perform maintenance on materiel/assets requiring repair 
or complete rebuild of parts, assemblies, subassemblies, and end-
items, including the manufacture of parts, modifications, testing, and 
reclamation as required. It also includes the process whereby 
buildings and other fixed facilities are maintained and renovated 
during their life cycle. 

Business process: Service-to-Satisfaction; 
Description: Determine service requirements, secure funding, contract 
with outside vendor, establish service, and measure customer 
satisfaction. 

Source: GAO based on DOD documentation. 

[End of table] 

[End of section] 

Appendix III: Status of Associated GAO Recommendations Made Since 2009: 

GAO report information and recommendation: 

GAO-09-586: DOD Business Systems Modernization: Recent Slowdown in 
Institutionalizing Key Management Controls Needs to be Addressed, May 
18, 2009: 

GAO report information and recommendation: 1. To ensure that DOD 
continues to implement the full range of institutional management 
controls needed to address its business systems modernization high-
risk area, the Secretary of Defense should direct the Deputy Secretary 
of Defense, as chair of the Defense Business Systems Management 
Committee and as DOD's Chief Management Officer, to resolve the issues 
surrounding the roles, responsibilities, authorities, and 
relationships of the Deputy Chief Management Officer and the military 
department Chief Management Officers relative to the business 
enterprise architecture (BEA) and enterprise transition plan 
federation and business system investment management; 
Implemented: In Process; 
GAO assessment: The department stated that it has a number of policy 
and guidance documents currently under development to help clarify the 
roles and responsibilities associated with development of the BEA and 
investment management. For example, the department's most recent 
investment management guidance specifies roles and responsibilities 
associated with managing portfolios of business systems. In addition, 
DOD has established a forum for discussing the development of the 
federated BEA and is currently working to establish a BEA 
Configuration Control Board charter, which is intended to document the 
roles and responsibilities for the review and approval of proposed BEA 
content. However, the department has yet to clarify in policy or 
guidance the roles, responsibilities, authorities, and relationships 
between the Deputy Chief Management Officer and military department 
Chief Management Officers relative to the BEA and enterprise 
transition plan federation. 

GAO report information and recommendation: 2. To ensure that business 
system investment reviews and related certification and approval 
decisions, as well as annual budget submissions, are based on complete 
and accurate information, the Secretary of Defense should direct the 
appropriate DOD organizations to develop and implement plans for 
reconciling and validating the completeness and reliability of 
information in its Defense Information Technology Portfolio Repository 
(DITPR) and Select and Native Programming Data Input System--
Information Technology system data repositories, and to include 
information on the status of these efforts in the department's fiscal 
year 2010 report in response to the act; 
Implemented: In Process; 
GAO assessment: According to DOD, the department is fully committed to 
integrating the data in the system data repositories. DOD Chief 
Information Officer officials stated that a new portal that will 
provide an integrated view of information from its two portals will 
reach a limited operational capability in May 2013. However, DOD has 
yet to fully implement its plans and to complete the integration of 
the repositories and validate the completeness and reliability of the 
information. 

GAO-10-663: Business Systems Modernization: Scope and Content of DOD's 
Congressional Report and Executive Oversight of Investments Need to 
Improve, May 24, 2010: 

GAO report information and recommendation: 1. To ensure that 
Investment Review Board (IRB) certification actions are better 
informed and justified, the Secretary should direct the Deputy 
Secretary to ensure that DOD guidance be revised to include provisions 
that require IRB certification submissions disclose program weaknesses 
raised by GAO and the status of actions to address our recommendations 
to correct the weaknesses; 
Implemented: Yes; 
GAO assessment: The enterprise transition plans for fiscal year 2011 
and 2012 and system dashboards reflected related GAO reports. In 
addition, the department's fiscal year 2014 investment management 
guidance issued by the Office of the Deputy Chief Management Officer 
now requires the precertification authorities to include any open GAO 
recommendations for program weaknesses as well as a status update on 
addressing GAO recommendations as part of the certification requests. 

GAO report information and recommendation: 2. To facilitate 
congressional oversight and promote departmental accountability, the 
Secretary of Defense should direct the Deputy Secretary of Defense, as 
the chair of the Defense Business Systems Management Committee, to 
ensure that the scope and content of future DOD annual reports to 
Congress on compliance with section 332 of the National Defense 
Authorization Act (NDAA) be expanded to include (1) cost, capability, 
and benefit performance measures for each business system 
modernization investment and actual performance against these measures; 
and (2) all certification actions, as defined in DOD guidance, which 
were taken in the previous year by the department on its business 
system modernization investments; 
Implemented: In Process; 
GAO assessment: While the department's fiscal year 2013 report to 
Congress included a list of certification actions, it did not include 
cost, capability, and benefit performance measures for each business 
system investment and actual performance against these measures. 
According to officials from the Office of the Deputy Chief Management 
Officer, the department elected not to seek measures data for fiscal 
year 2012 given the significant changes as a result of the NDAA, but 
it plans to provide measurement data for the fiscal year 2014 cycle. 

GAO-11-684: Department of Defense: Further Actions Needed to 
Institutionalize Key Business System Modernization Management 
Controls, June 29, 2011: 

GAO report information and recommendation: 1. To address the 
uncertainty and pending decisions surrounding the roles and 
responsibilities of key organizations, the Secretary of Defense should 
expeditiously complete the implementation of the announced transfer of 
functions of the Business Transformation Agency and the Office of the 
Assistant Secretary of Defense for Networks and Information 
Integration/Department of Defense Chief Information Officer and 
provide specificity as to when and where these functions will be 
transferred; 
Implemented: Yes; 
GAO assessment: In October 2011, the department formally 
disestablished the Business Transformation Agency, completing the 
transfer of its various responsibilities to other DOD entities, 
including the Office of the Deputy Chief Management Officer. In 
addition, in January 2012, DOD announced the disestablishment of the 
Assistant Secretary of Defense for Networks and Information 
Integration and the transfer of its various responsibilities to other 
DOD entities, including the DOD Chief Information Officer and Under 
Secretary of Defense for Acquisition, Technology, and Logistics. 

GAO-11-902: Organizational Transformation: Military Departments Can 
Improve Their Enterprise Architecture Programs, Sep 26, 2011a: 

GAO report information and recommendation: 1. To ensure that the 
military departments establish commitments to fully develop and 
effectively manage their enterprise architectures, the Secretaries of 
the Air Force, Army, and Navy each should expeditiously provide to the 
congressional defense committees a plan that identifies milestones for 
their respective department's full satisfaction of all of our 
Enterprise Architecture Management Maturity Framework elements. In the 
event that a military department does not intend to fully satisfy all 
elements of our framework, the plan should include a rationale for why 
the department deems any such element(s) to be not applicable (ARMY); 
Implemented: In Process; 
GAO assessment: While Department of the Army officials stated that the 
department plans to address all of the framework elements, the 
department has yet to develop a plan for doing so. 

GAO report information and recommendation: 2. To ensure that the 
military departments establish commitments to fully develop and 
effectively manage their enterprise architectures, the Secretaries of 
the Air Force, Army, and Navy each should expeditiously provide to the 
congressional defense committees a plan that identifies milestones for 
their respective department's full satisfaction of all of our 
Enterprise Architecture Management Maturity Framework elements. In the 
event that a military department does not intend to fully satisfy all 
elements of our framework, the plan should include a rationale for why 
the department deems any such element(s) to be not applicable. (NAVY); 
Implemented: In Process; 
GAO assessment: The Department of the Navy provided a memorandum 
describing its proposed approach for addressing the elements of the 
framework that it considers applicable, but this memorandum does not 
describe the department's milestones for addressing the elements and 
the department did not provide evidence to support that the memorandum 
has been delivered to the congressional defense committees. 

GAO report information and recommendation: 3. To ensure that the 
military departments establish commitments to fully develop and 
effectively manage their enterprise architectures, the Secretaries of 
the Air Force, Army, and Navy each should expeditiously provide to the 
congressional defense committees a plan that identifies milestones for 
their respective department's full satisfaction of all of our 
Enterprise Architecture Management Maturity Framework elements. In the 
event that a military department does not intend to fully satisfy all 
elements of our framework, the plan should include a rationale for why 
the department deems any such element(s) to be not applicable. (AIR 
FORCE); 
Implemented: In Process; 
GAO assessment: DOD officials reported that the Air Force Enterprise 
Architecture Management Plan, scheduled for issuance in July 2013, 
will identify milestones for satisfying a number of the elements 
described in our Enterprise Architecture Management Maturity Framework. 

GAO-12-241: Information Technology: Departments of Defense and Energy 
Need to Address Potentially Duplicative Investments, Feb 17, 2012: 

GAO report information and recommendation: 1. To better ensure 
agencies avoid investing in duplicative investments, the Secretary of 
Defense should direct the Chief Information Officer to correct the 
miscategorizations for the DOD investments we identified and ensure 
that investments are correctly categorized in agency submissions; 
Implemented: Yes; 
GAO assessment: DOD has taken steps to help ensure accurate 
categorizations of its IT investments. Specifically, it has corrected 
miscategorized investments and included the corrections in its budget 
submissions. 

GAO report information and recommendation: 2. To better ensure 
agencies avoid investing in duplicative investments, the Secretary of 
Defense should direct the Chief Information Officer to utilize 
existing transparency mechanisms, such as the IT Dashboard, to report 
on the results of the department's efforts to identify and eliminate, 
where appropriate, each potentially duplicative investment we have 
identified, as well as any other duplicative investments; 
Implemented: In Process; 
GAO assessment: According to DOD officials, in October 2012, DOD held 
an IT portfoliostat review in which it evaluated its IT investments 
for fiscal year 2013. In addition, as discussed in this report, it 
also began to implement a new IRB process for its business system 
investments, which is to, among other things, reduce duplication and 
improve business effectiveness. However, DOD has not yet reported on 
whether its efforts have resulted in the identification and 
elimination of duplicative investments. 

GAO-12-685: DOD Business Systems Modernization: Governance Mechanisms 
for Implementing Management Controls Need to Be Improved, Jun 1, 2012: 

GAO report information and recommendation: 1. To ensure that DOD 
continues to implement the full range of institutional management 
controls needed to address its business systems modernization high-
risk area, the Secretary of Defense should ensure that the Deputy 
Secretary of Defense, as the department's Chief Management Officer, 
establish a policy that clarifies the roles, responsibilities, and 
relationships among the Chief Management Officer, Deputy Chief 
Management Officer, DOD, and military department chief information 
officers, principal staff assistants, military department chief 
management officers, and the heads of the military departments and 
defense agencies, associated with the development of a federated BEA. 
Among other things, the policy should address the development and 
implementation of an overarching taxonomy and associated ontologies to 
help ensure that each of the respective portions of the architecture 
will be properly linked and aligned. In addition, the policy should 
address alignment and coordination of business process areas, military 
department and defense agency activities associated with developing 
and implementing each of the various components of the BEA, and 
relationships among these entities; 
Implemented: In Process; 
GAO assessment: The department stated that it has a number of policy 
and guidance documents currently under development to help clarify the 
roles and responsibilities associated with development of the BEA and 
enterprise architecture in general across DOD. For example, DOD has 
established a forum for discussing the development of the federated 
BEA and is currently working to establish a BEA Configuration Control 
Board charter, which is intended to document the roles and 
responsibilities for the review and approval of proposed BEA content. 
However, DOD has yet to issue these policy and guidance documents. 

GAO report information and recommendation: 2. To ensure that annual 
budget submissions are based on complete and accurate information, the 
Secretary of Defense should direct the appropriate DOD organizations 
to establish a deadline by which it intends to complete the 
integration of the repositories and validate the completeness and 
reliability of information; 
Implemented: In Process; 
GAO assessment: According to DOD, the department is fully committed to 
integrating the data in the system data repositories. DOD Chief 
Information Officer officials stated that a new portal that will 
provide an integrated view of information from its two portals will 
reach a limited operational capability in May 2013. However, DOD has 
yet to establish a deadline by which it intends to complete the 
integration of the repositories and validate the completeness and 
reliability of information. 

GAO report information and recommendation: 3. To facilitate 
congressional oversight and promote departmental accountability, the 
Secretary of Defense should ensure that the Deputy Secretary of 
Defense, as the department's Chief Management Officer, direct the 
Deputy Chief Management Officer to include in DOD's annual report to 
Congress on compliance with 10 U.S.C. § 2222, the results of the 
department's business process reengineering efforts. Among other 
things, the results should include the department's determination of 
the number of systems that have undergone material process changes, 
the number of interfaces eliminated as part of these efforts (i.e., by 
program, by name), and the status of its end-to-end business process 
reengineering efforts; 
Implemented: In Process; 
GAO assessment: The Office of the Deputy Chief Management Officer 
stated that measures such as the number of systems that have undergone 
material process changes, the number of interfaces eliminated, and the 
status of end-to-end business process reengineering efforts were not 
included in its annual report to Congress because the focus for this 
year was on establishing baseline portfolios. The department took 
steps to update its business process reengineering guidance in 
September 2012 and has begun to validate business process 
reengineering assertions on selected investments. However, the 
department has yet to identify and report the results of the 
department's business process reengineering efforts. 

GAO report information and recommendation: 4. To facilitate 
congressional oversight and promote departmental accountability, the 
Secretary of Defense should ensure that the Deputy Secretary of 
Defense, as the department's Chief Management Officer, direct the 
Deputy Chief Management Officer to include in DOD's annual report to 
Congress on compliance with 10 U.S.C. § 2222, an update on the Office 
of the Deputy Chief Management Officer's progress toward filling staff 
positions and the impact of any unfilled positions on the ability of 
the office to conduct its work; 
Implemented: In Process; 
GAO assessment: The Office of the Deputy Chief Management Officer 
provided an update to GAO on the numbers of positions filled and open. 
However, status on staffing and the impact of unfilled positions on 
the ability of the office to conduct its work have not yet been 
included in the annual report to Congress. 

GAO-12-791: Organizational Transformation: Enterprise Architecture 
Value Needs to Be Measured and Reported, Sep 26, 2012: 

GAO report information and recommendation: 1. To enhance federal 
agencies' ability to realize enterprise architecture benefits, the 
Secretary of Defense should periodically measure and report enterprise 
architecture outcomes and benefits to top agency officials (i.e., 
executives with authority to commit resources or make changes to the 
program) and to the Office of Management and Budget (OMB); 
Implemented: In Process; 
GAO assessment: According to DOD, the Defense Business Council is 
currently chartering the BEA Configuration Control Board and will 
require all BEA changes to be directly traceable to a measurable 
business outcome. However, the department has yet to measure and 
report enterprise architecture outcomes and benefits consistent with 
our recommendation or report the results of outcome measurements to 
OMB. In addition, the department did not demonstrate that its military 
departments or its corporate enterprise architecture efforts are 
periodically measuring and reporting related outcomes and benefits to 
top agency officials and to OMB. OMB has requested agencies to report 
enterprise architecture outcomes in reports that are due by May 15, 
2013. 

Source: GAO analysis of DOD and OMB information.

[A] GAO-11-902 focused on the military departments' corporate 
architectures. 

[End of table] 

[End of section] 

Appendix IV: Comments from the Department of Defense: 

Deputy Chief Management Officer: 
9010 Defense Pentagon: 
Washington, DC 20301-9010: 

May 10, 2013: 

Ms. Valerie Melvin: 
Director, Information Management and Technology Resources Issues: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Ms. Melvin, 

This is the Department of Defense (DoD) response to the Government 
Accountability Office (GAO) draft report GAO-13-557, "DOD Business 
Systems Modernization: Further Actions Needed to Address Challenges 
and Improve Accountability," dated May 1, 2013, (GAO Code 310993). 
While there are a few recommendations with which the Department 
concurs, on the whole we disagree with many of the assertions put 
forward by GAO and the lack of balance expressed in the draft report. 

Every day the Department successfully operates one of the most complex 
and diverse business enterprises in the world, and DoD leaders 
routinely take deliberate actions to improve how the Department 
conducts its business. The Department has implemented a business
management model that improves services to the warfighter by aligning 
the Department's business strategies and desired outcomes with 
information technology investments organized as portfolios. The draft 
report demonstrates a lack of understanding of this new investment
management process and the significant improvements it has produced. 
During Fiscal Year 2012, we took full advantage of the mandate in 
title 10 United States Code section 2222 to revamp business mission 
area governance and establish an Integrated Business Framework. DoD 
business leaders now use this framework to vet business strategy, link 
portfolio objectives to enterprise vision, mission, goals and 
objectives, and ensure business process reengineering and business 
enterprise architecture activities are aligned to end-to-end processes 
to make informed business system investment and certification 
decisions. Rather than the slow improvement portrayed in the draft 
report, these changes have resulted in significant progress. 

While our business systems modernization efforts will continue to 
evolve, it is important for the Department and Congress not to lose 
sight of the bigger picture of what we endeavor to achieve by focusing 
only on the set of challenges cited by GAO in the draft report. DoD's 
Business Mission Area is a complex, interconnected environment that is 
unprecedented in size and critical to the Department's overall 
national security mission. Rather than attempt to institute a one-size-
fits-all approach across every aspect of the DoD business enterprise, 
the Department will continue to prioritize actions and activities that 
produce the best return on investment. Implementing GAO's 
recommendations to institute a 100-percent compliance verification 
regime without regard to the lifecycle phase of a program would divert 
scarce resources for tasks that provide no benefit, for example, 
reengineering a system targeted to be retired. To separately track 
milestone and performance data for investments as part of the
investment review process, as GAO also suggests, would be duplicative 
of the Department's robust, existing controls that are part of our 
Defense Acquisition System. 

Finally, there is significant focus in the report on the Department's 
failure to implement recommendations that are almost a decade old and 
that have been overtaken by more recent events. We have made progress 
over the years in implementing those recommendations with which we 
concurred; however, the Department's business modernization efforts 
have evolved significantly since 2005. In March, the Department issued 
revised guidance for FY2014 funds certification that incorporates 
lessons learned during our first year using the new process. Going 
forward, we will continue to mature and refine our efforts to 
rationalize the Department's business system investments to eliminate 
unnecessary redundant systems, as well as identifying both long range 
improvement opportunities and near-term steps to advance current 
business practices. 

The Department appreciates the opportunity to respond to your draft 
report. We look forward to your continued cooperation and dialog 
toward our common goal of improving business systems modernization 
throughout the Department of Defense. Should you have any questions, 
please contact Mr. Thomas Cowley, 703-692-8170, thomas.cowley@osd.mil. 

Sincerely, 

Signed by: 

Elizabeth A. McGrath: 

Enclosures: As stated: 

GAO Draft Report Dated April 30, 2013: 
GAO-13-557 (GAO Code 310993): 

"Dod Business Systems Modernization: Further Actions Needed To Address 
Challenges And Improve Accountability" 

Department Of Defense Comments To The GAO Recommendations: 

Recommendation 1: The Government Accountability Office (GAO) 
recommends that the Secretary of Defense direct the Deputy Chief 
Management Officer (DCMO) to define by when and how the department 
plans to develop an architecture that would extend to all defense 
components and include, among other things, (a) information about the 
specific business systems that support Business Enterprise 
Architecture (BEA) business activities and related system functions; 
(b) business capabilities for the Hire-to-Retire and Procure-to-Pay 
business processes; (c) sufficient information about business 
activities to allow for more effective identification of potential 
overlap and duplication. 

DoD Response: Partially concur. The Department of Defense (DoD) agrees 
that plans for continuing to mature the Federated BEA should be 
completed. It should be noted that the Department has already taken 
the necessary initial steps to extend the BEA to all Defense 
Components. These steps have been institutionalized via the Integrated 
Business Framework which was implemented during calendar year 2012. 
The framework links together the development of functional business 
strategies, the identification of associated BEA changes, and the 
submission of Organizational Execution Plans (OEPs) from the DoD 
Components. Information contained in the Component-developed OEPs 
constitutes key elements of the Component's portion of the Federated 
BEA. In particular, the OEPs and associated authoritative data sources 
provide the relationship and linkage between each Component's 
portfolio of proposed investments and key elements of the corporate 
portion of the Federated BEA. 

Recommendation 2: The GAO recommends that the Secretary of Defense 
direct the Deputy Chief Management Officer to define by when and how 
the enterprise transition plan will include, among other things, (a) 
milestones, performance measures, and funding plans for all business 
systems expected to be part of the target architecture and each 
system's risks or challenges to integration; (b) time-phased end dates 
associated with terminating legacy systems in phases; (c) a listing of 
legacy systems that will be a part of the target defense business 
systems computing environment and a strategy for making modifications 
to legacy systems that will be needed to ensure that they comply with 
the defense BEA, including time-phased milestones, performance 
measures, and financial resource needs and (d) information about how 
systems are to be sequenced according to, among other things, 
dependencies among investments. 

DoD Response: Partially concur. The Department is meeting the 
requirements of section 2222 as follows: (a) milestones, performance 
measures, and funding plans for all business systems expected to be 
part of the target architecture and each system's risks or challenges 
to integration. This information is explained in the presentations 
that are provided to the Defense Business Council (DBC); (b) time-
phased end dates associated with terminating legacy systems in phases. 
This requirement is being met by the establishment of a category of 
"legacy" systems that are to sunset within the next 36 months along 
with an explanation of what if any systems will continue the business 
need; (c) a listing of systems that will be a part of the target 
defense business systems computing environment and a strategy for 
making modifications to systems that will be needed to ensure that 
they comply with the Defense BEA, including time-phased milestones, 
performance measures, and financial resource needs. This requirement 
is being met by the precertification process and through the Defense 
acquisition system milestones, performance measures, etc.; d) 
Information about how systems are to be sequenced according to, among 
other things, dependencies among investments. This information is 
captured in presentations and supporting materials provided to the DBC. 

The Department is using core attributes of the GAO Information 
Technology Investment Management (ITIM) framework and the Capital 
Planning and Investment Control (CPIC) process to manage DoD's budget 
and acquisition of defense business systems. DoD has developed a 
complete investment portfolio, established selection criteria, and 
developed an Integrated Business Framework that continues to evolve 
for selecting, controlling and evaluating investments. The Department 
built its investment foundation by establishing organizational 
priorities through the Strategic Management Plan and aligning 
Functional Strategies that are then executed by DoD Components through 
their OEPs. DoD is focusing on delivering efficient, effective, and 
agile business operations that support and enable the warfighter; 
achieving specific program business outcomes; using criteria related 
to return on investment and quantitative and qualitative criteria for 
comparing and prioritizing alternative defense business systems 
programs; realizing enterprise architecture-driven performance 
improvements and outcomes (e.g., improving mission performance; saving 
money and avoiding costs; enhancing the quality of DoD investment 
portfolios; improving the quality, availability, and sharing of data 
and information; and increasing the transparency of DoD operations) to 
complement existing and enduring budget and acquisition processes 
within DoD. 

The Defense Acquisition System is the core process used by the 
Department to manage defense acquisitions to include the procurement 
of defense business systems. Likewise, the Department's Planning, 
Programming, Budgeting and Execution (PPBE) process is the principal 
process for managing the DoD budget. Taken together, the Department 
has a much greater visibility into the business enterprise than ever 
before. The Department is taking a data-driven approach to manage 
defense business systems as portfolios of investments within the 
Integrated Business Framework. It is our goal to leverage and 
aggregate data from authoritative data sources and tools used by the 
PPBE, acquisition and funds certification processes to track and 
manage the overall performance of portfolios.

The Department considers any Defense Business System (DBS) with a 
sunset date within 36 months of certification as legacy and 
development and modernization funds for these legacy systems have been 
removed from the budget. All other defense business systems are 
considered core and therefore must demonstrate compliance to the BEA 
while ensuring appropriate Business Process Re-Engineering (BPR). 
Modernization to individual systems will be tracked and monitored by 
the Defense Acquisition System; however, the requirement driving the 
modernization will be validated by the Defense Business Council (DBC) 
in accordance with Chairman of the Joint Chiefs of Staff Instruction 
3170.01H, "Joint Capabilities Integration and Development System," and 
Directive-Type Memorandum (DTM) 11-009, "Acquisition Policy for 
Defense Business Systems," to ensure strategic alignment with the 
Strategic Management Plan, Functional Strategies and the BEA. Further, 
the DBC will continue to track the performance of business system 
portfolios as OEPs are presented to the DBC for certification. 

Recommendation 3: The GAO recommends that the Secretary of Defense 
direct the Deputy Chief Management Officer to ensure that the 
functional strategies include all of the critical elements identified 
in DOD investment management guidance, including performance measures 
to determine progress toward achieving the goals that incorporate all 
of the attributes called for in the department's guidance.

DoD Response: Concur. 

Recommendation 4: The GAO recommends that the Secretary of Defense 
direct the Deputy Chief Management Officer to select and control its 
mix of investments in a manner that best supports mission needs by (a) 
documenting a process for evaluating portfolio performance that 
includes the use of actual versus expected performance data and 
predetermined thresholds; (b) ensuring that portfolio assessments are 
conducted in key areas identified in our IT investment management 
framework: benefits attained; current schedule; accuracy of project 
reporting; and risks that have been mitigated, eliminated, or accepted 
to date; and (c) ensuring that the documents provided to the Defense 
Business Council as part of the investment management process include 
critical information for conducting all assessments. 

DoD Response: Non-concur. This recommendation does not recognize the 
significant improvements the Department made in 2012 to fundamentally 
change how it reviews its defense business systems. The FY2012 NDAA 
substantially changed the scope and governance approach for reviewing 
and certifying defense business systems by creating a single 
Investment Review Board (IRB) to review all business systems planning 
to spend more than $1M over the period of the Future Years Defense 
Program (FYDP) regardless of the funding source. In response to the 
NDAA, the Department created the Integrated Business Framework, a 
paradigm shift that aligns business strategies to business systems IT 
spending vice simply looking at IT costs.

Regarding the specific recommendations: 

(a) Process Documentation: The Department created initial 
documentation aligned to the Integrated Business Framework in June 
2012. Section 3.3 of the Defense Business System Investment Review 
Guidance defines the review criteria used by the DBC to evaluate and 
certify business systems via OEPs for certification. Based on lessons 
learned in 2012, we then updated guidance to further strengthen and 
improve the process. As noted in the draft report, the Functional 
Strategies that were updated in 2013 include improved business goals 
and supporting measures that will be used to evaluate business system 
certification requests when they are presented to the DBC later this 
year. It is our intent, unless a business system is required for 
critical operational support, to withhold certification for business 
system investments that are not aligned to the Functional Strategies. 
Likewise, it is not the intent of the DBC (which serves as the 
Department's IRB) to review acquisition issues associated with 
individual business systems, as that function is performed by the 
Defense Acquisition System. 

(b) Portfolio assessments in accordance with GAO IT investment 
management framework: The Department is leveraging key components of 
the GAO ITIM Framework to evaluate portfolios of investments presented 
to the DBC. However the IRB process will complement the Department's 
acquisition system rather than duplicate it. The Defense Acquisition 
System is the core process used by the Department to manage defense 
acquisitions, to include the procurement of defense business systems, 
and it already contains significant elements of the GAO ITIM framework. 

(c) DBC documentation for conducting assessments: This aspect of the 
recommendation does not take into account that the information derived 
for strategic alignment, cost, utility and compliance is already given 
to the DBC and incorporates objective information derived from 
authoritative documents and data sources. Additionally, the 
documentation is supported by input from the Functional Area Owners 
and Subject Matter Experts. 

Recommendation 5: The GAO recommends that the Secretary of Defense 
direct the Deputy Chief Management Officer to implement and use the 
BEA and business process reengineering compliance assessments more 
effectively to support organizational transformation efforts by (a) 
disclosing relevant information about known weaknesses, such as BEA 
and business process reengineering compliance weaknesses for systems 
that were not certified or certified with qualifications in annual 
reports to Congress; (b) establishing milestones by which selected 
validations of BEA compliance assertions are to be completed; and (c) 
ensuring that appropriate business process reengineering assertions 
have been completed on all investments submitted for the fiscal year 
2014 certification reviews prior to the certification of funds.

DoD Response: Partially concur. DoD agrees that BEA and BPR compliance 
assessments be used to support organizational transformation efforts 
by disclosing relevant information about known weaknesses and that 
milestones should be identified for completing selected validations. 
However, DoD does not agree with the GAO recommendation to ensure that 
appropriate business process reengineering assertions be completed on 
all investments submitted for the fiscal year 2014 certification 
reviews prior to the certification of funds. Requiring BPR for systems 
scheduled for retirement or for systems solely in sustainment would 
divert scarce resources for tasks that would provide no benefit. 
Therefore our transition plan for those systems is to sunset the 
program within 36 months (legacy) and, for systems solely in 
sustainment, determine that it is not appropriate to conduct further 
BPR. The revised BPR Assessment Guidance and memorandum of September 
28, 2012 documents the DCMO BPR Assessment plan which includes 
providing the Department a BPR Standard, outlining a plan of action 
for executing a validation of a sampling of DBS programs in which the 
Pre-Certification Authority (PCA) asserted BPR compliance on their 
FY2013 Organizational Execution Plan (OEP). The revised BPR Assessment 
Guidance memorandum also required the PCA to develop a BPR Plan of 
Action for all DBS programs which did not assert BPR compliant in 
FY2013 to become BPR compliant; this requirement was also documented 
in the Investment Decision Memorandum (IDM) for each OEP as a 
condition to the FY2013 certification of funds. The results of the 
Office of the DCMO BPR Assessment validations and the PCA response to 
the IDM/BPR Compliance Plan of Action will be reported to the DBC this 
summer. 

The DoD is an operational organization with emerging and unforeseen 
operational needs, therefore the DBC must retain and use, when 
appropriate, the conditional certification authority as it relates to 
the BPR compliance requirement. This authority has been used by the 
DoD IRBs since the inception of the certification review process in 
2005 and allows the DBC to levy a condition on the DBS program to 
develop a BPR Plan of Action by a specified date. It is the 
expectation of the DBC that all BPR compliance assertions be true and 
verifiable; having the ability to condition the DBS program with a BPR 
Plan of Action supports the DBC's ability to go back and potentially 
assess the BPR effort based on their BPR Plan of Action. 

Recommendation 6: The GAO recommends that the Secretary of Defense 
direct the Deputy Chief Management Officer to develop a skills 
inventory, needs assessment, gap analysis, and plan to address 
identified gaps as part of a strategic approach to human capital 
planning. 

DoD Response: Non-concur. The Office of the DCMO is a relatively small 
component of the Office of the Secretary of Defense. The proposed 
direction could not be accomplished given current resources (both 
personnel and funding) and is a level of activity normally designed 
for an entire agency. The draft report also draws conclusions from 
statements by Office of the DCMO officials as to the impact of the 
Office of the DCMO's staffing levels that are not supported by the 
findings. The Office of the DCMO is constantly working to improve its 
human capital assets in ways that are appropriate to an office with 
less than 100 government employees. As part of the functional work 
assessment and reorganization GAO referenced in the report, the Office 
of the DCMO thoroughly considered the skills of its employees, sought 
to optimally align them organizationally, and has used hiring actions 
to bring additional skill sets onboard that filled identified gaps. 
The report attributes consequences of the Office of the DCMO's 
perceived human capital weaknesses far beyond what is supported by the 
GAO's findings. While the Office of the DCMO sets policy, performs 
oversight activities, and develops important products, it relies on 
content, input, and implementation from throughout the DoD Components 
to move forward. Weaknesses identified by GAO in the Department's 
federated architecture, investment management data, and transition 
plan content are Departmental weaknesses and are not simply due to the 
staffing levels of the Office of the DCMO. 

Recommendation 7: The GAO recommends that the Secretary of Defense 
direct the appropriate authority to ensure that complete 
documentation, such as root cause analyses, assessments of existing 
interfaces for reuse opportunities, and performance metrics related to 
the reengineering efforts is provided as part of the fiscal year 2014 
certification and approval process for the IPPS-A, IPPS-N, AF-IPPS, 
and iEHR investments. 

DoD Response: Concur. The revised BPR Assessment Guidance of September 
28, 2012 specifically addressed root cause analyses in question 5, 
assessment of interfaces in question 10, and performance measures in 
questions 4 and 11 with the expectation that objective evidence is 
available that supports the responses to questions 4, 5, 10, and 11. 
It is the expectation of the DBC that all PCA BPR compliance 
assertions are made using the BPR Assessment Form with objective 
evidence which should yield a more accurate evaluation and provide a 
more tangible outcome for all DBS programs. 

Recommendation 8: The GAO recommends that the Secretary of Defense 
direct the appropriate authority to determine whether funds were 
properly obligated under 10 U.S.C. §2222(a)-(b) for systems for which 
appropriate business process reengineering assertions were not 
completed. 

DoD Response: Non-concur. The Department believes that it has already 
met the conditions of 10 U.S.C. §2222 through reviews by the Pre-
Certification authorities and the DBC. As an example, the Department 
considers any Defense Business System (DBS) with a sunset date within 
36 months of certification as legacy so development and modernization 
funds for these legacy systems have been removed from the budget. The 
transition plan for these legacy systems is to sunset the system as 
opposed to bringing the system into compliance with the BEA. This 
action is in the best interest of the department and conserves 
resources. The requirement to sunset the DBS is included in all 
relevant documentation generated, is approved by the DBC and the DBSMC 
and therefore has been reviewed for compliance. 

All other defense business systems are considered core and therefore 
must demonstrate compliance or a transition plan for compliance to the 
BEA while ensuring appropriate Business Process Re-Engineering (BPR). 
Modernization to individual systems will be tracked and monitored by 
the Defense Acquisition System; however, the requirement driving the 
modernization will be validated by the Defense Business Council (DBC) 
in accordance with Chairman of the Joint Chiefs of Staff Instruction 
3170.01H, "Joint Capabilities Integration and Development System," and 
Directive-Type Memorandum (DTM) 11-009, "Acquisition Policy for 
Defense Business Systems," to ensure strategic alignment with the 
Strategic Management Plan, Functional Strategies and the BEA. Further, 
the DBC will continue to track the performance of business system 
portfolios as OEPs are presented to the DBC for certification. 

Further, to strengthen BPR assertions, the Department issued the 
revised BPR Assessment Guidance memorandum of September 28, 2012, 
which specifically addressed the requirement for the PCA to develop a 
BPR Plan of Action for all DBS programs that did not assert BPR 
compliance in FY2013 to become BPR compliant; this requirement was 
also documented in the IDM for each OEP as a condition to the FY2013 
certification of funds. The results of the Office of the DCMO BPR 
Assessment validations and the PCA response to the IDM/BPR compliance 
Plan of Action will be reported to the DBC this summer. It is the 
expectation of the DBC that a larger number of DBS programs will 
assert BPR compliance in FY2014 (prior to the fund certification 
requests, in either an OEP or out of cycle). As noted, the DBC does 
not require BPR for systems scheduled for retirement. For systems 
solely in sustainment, a determination is made that further BPR is not 
required as it would divert scarce resources for tasks that would 
provide no benefit.

[End of section] 

Appendix V: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Valerie C. Melvin, (202) 512-6304 or melvinv@gao.gov: 

Staff Acknowledgments: 

In addition to the contact named above, individuals making 
contributions to this report include Neelaxi Lakhmani (Assistant 
Director), Debra Conner, Nancy Glover, Michael Holland, Anh Le, 
Jennifer Stavros-Turner, and Donald Sebers.

[End of section] 

Footnotes: 

[1] GAO, High-Risk Series: An Update, [hyperlink, 
http://www.gao.gov/products/GAO-13-283] (Washington, D.C.: February 
2013).

[2] GAO, Opportunities to Reduce Potential Duplication in Government 
Programs, Save Tax Dollars, and Enhance Revenue, [hyperlink, 
http://www.gao.gov/products/GAO-11-318SP] (Washington, D.C.: Mar. 1, 
2011); Follow-up on 2011 Report: Status of Actions Taken to Reduce 
Duplication, Overlap, and Fragmentation, Save Tax Dollars, and Enhance 
Revenue, [hyperlink, http://www.gao.gov/products/GAO-12-453SP] 
(Washington, D.C.: Feb. 28, 2012); and [hyperlink, 
http://www.gao.gov/products/GAO-13-283].

[3] Defense Information Technology Portfolio Repository dated March 
22, 2013.

[4] GAO, DOD Business Systems Modernization: Long-standing Weaknesses 
in Enterprise Architecture Development Need to Be Addressed, 
[hyperlink, http://www.gao.gov/products/GAO-05-702] (Washington, D.C.: 
July 22, 2005); DOD Business Systems Modernization: Billions Being 
Invested without Adequate Oversight, [hyperlink, 
http://www.gao.gov/products/GAO-05-381] (Washington, D.C.: Apr. 29, 
2005); DOD Business Systems Modernization: Limited Progress in 
Development of Business Enterprise Architecture and Oversight of 
Information Technology Investments, [hyperlink, 
http://www.gao.gov/products/GAO-04-731R] (Washington, D.C.: May 17, 
2004); DOD Business Systems Modernization: Important Progress Made to 
Develop Business Enterprise Architecture, but Much Work Remains, 
[hyperlink, http://www.gao.gov/products/GAO-03-1018] (Washington, 
D.C.: Sept. 19, 2003); Business Systems Modernization: Summary of 
GAO's Assessment of the Department of Defense's Initial Business 
Enterprise Architecture, [hyperlink, 
http://www.gao.gov/products/GAO-03-877R] (Washington, D.C.: July 7, 
2003); Information Technology: Observations on Department of Defense's 
Draft Enterprise Architecture, [hyperlink, 
http://www.gao.gov/products/GAO-03-571R] (Washington, D.C.: Mar. 28, 
2003); DOD Business Systems Modernization: Improvements to Enterprise 
Architecture Development and Implementation Efforts Needed, 
[hyperlink, http://www.gao.gov/products/GAO-03-458] (Washington, D.C.: 
Feb. 28, 2003); and Information Technology: Architecture Needed to 
Guide Modernization of DOD's Financial Operations, [hyperlink, 
http://www.gao.gov/products/GAO-01-525] (Washington, D.C.: May 17, 
2001).

[5] Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-1856 (2004), as 
amended by the NDAA for Fiscal Year 2013, Pub. L. No. 112-239, 126 
Stat. 1632 (2012), the NDAA for Fiscal Year 2012, Pub. L. No. 112-81, 
125 Stat. 1298 (2011), and the NDAA for Fiscal Year 2010, Pub. L. No. 
111-84, 123 Stat. 2190 (2009) (codified in part at 10 U.S.C. § 2222. 
Hereafter, we refer to the provisions of 10 U.S.C. § 2222, including 
its amendments, as the NDAA or 'the act.').

[6] GAO, DOD Business Systems Modernization: Governance Mechanisms for 
Implementing Management Controls Need to be Implemented, [hyperlink, 
http://www.gao.gov/products/GAO-12-685] (Washington, D.C.: June 1, 
2012); GAO, Department of Defense: Further Actions Needed to 
Institutionalize Key Business System Modernization Management 
Controls, [hyperlink, http://www.gao.gov/products/GAO-11-684] 
(Washington, D.C.: June 29, 2011); Business Systems Modernization: 
Scope and Content of DOD's Congressional Report and Executive 
Oversight of Investments Need to Improve, [hyperlink, 
http://www.gao.gov/products/GAO-10-663] (Washington, D.C.: May 24, 
2010); DOD Business Systems Modernization: Recent Slowdown in 
Institutionalizing Key Management Controls Needs to Be Addressed, 
[hyperlink, http://www.gao.gov/products/GAO-09-586] (Washington, D.C.: 
May 18, 2009); DOD Business Systems Modernization: Military 
Departments Need to Strengthen Management of Enterprise Architecture 
Programs, [hyperlink, http://www.gao.gov/products/GAO-08-519] 
(Washington, D.C.: May 12, 2008); Business Systems Modernization: 
Department of the Navy Needs to Establish Management Structure and 
Fully Define Policies and Procedures for Institutionally Managing 
Investments, [hyperlink, http://www.gao.gov/products/GAO-08-53] 
(Washington, D.C.: Oct. 31, 2007); and Business Systems Modernization: 
Air Force Needs to Fully Define Policies and Procedures for 
Institutionally Managing Investments, [hyperlink, 
http://www.gao.gov/products/GAO-08-52] (Washington, D.C.: Oct. 31, 
2007).

[7] GAO, DOD Financial Management: Implementation Weaknesses in Army 
and Air Force Business Systems Could Jeopardize DOD's Auditability 
Goals, [hyperlink, http://www.gao.gov/products/GAO-12-134] 
(Washington, D.C.: Feb. 28, 2012).

[8] The DOD IT infrastructure segments account for all business and 
mission infrastructure. Data from the department's fiscal year 2013 
budget request is presented here because DOD did not issue its 
detailed fiscal year 2014 budget request for IT systems in time to be 
included in this report. 

[9] Defense Information Technology Portfolio Repository dated March 
22, 2013.

[10] [hyperlink, http://www.gao.gov/products/GAO-13-283]. 

[11] These seven high-risk areas include DOD's overall approach to 
business transformation, business systems modernization, contract 
management, financial management, supply chain management, support 
infrastructure management, and weapon systems acquisition. 

[12] The eight governmentwide high-risk areas include better managing 
climate change risks, disability programs, ensuring the effective 
protection of technologies critical to U.S. national security 
interests, information systems and critical infrastructure, 
information sharing for homeland security, human capital, mitigating 
gaps in weather satellite data, and real property. 

[13] GAO, DOD Business Systems Modernization: Planned Investment in 
Navy Program to Create Cashless Shipboard Environment Needs to Be 
Justified and Better Managed, [hyperlink, 
http://www.gao.gov/products/GAO-08-922] (Washington, D.C.: Sept. 8, 
2008); DOD Travel Cards: Control Weaknesses Resulted in Millions of 
Dollars of Improper Payments, [hyperlink, 
http://www.gao.gov/products/GAO-04-576] (Washington, D.C.: June 9, 
2004); Military Pay: Army National Guard Personnel Mobilized to Active 
Duty Experienced Significant Pay Problems, [hyperlink, 
http://www.gao.gov/products/GAO-04-89] (Washington, D.C.: Nov. 13, 
2003); and Defense Inventory: Opportunities Exist to Improve Spare 
Parts Support Aboard Deployed Navy Ships, [hyperlink, 
http://www.gao.gov/products/GAO-03-887] (Washington, D.C.: Aug. 29, 
2003). 

[14] 10 U.S.C. § 2222.

[15] The act (10 U.S.C. § 2222(a)) requires the appropriate 
precertification authority to determine that a defense business system 
modernization program (1) (a) is in compliance with the enterprise 
architecture and (b) has undertaken appropriate business process 
reengineering efforts; (2) is necessary to achieve a critical national 
security capability or address a critical requirement in an area such 
as safety or security; or (3) is necessary to prevent a significant 
adverse effect on a project that is needed to achieve an essential 
capability, taking into consideration the alternative solutions for 
preventing such an adverse effect. The NDAA for fiscal year 2012 
requires that the certification and approval requirements apply to all 
business systems programs that are expected to cost more than $1 
million over the period of the current Future-Years Defense Program, 
which is the department's financial plan over a 6-year period. 
Previously, the certification requirement only applied to business 
system modernizations with a total cost in excess of $1 million. 

[16] According to DOD officials, the business mission area is 
responsible for ensuring that capabilities, resources, and materiel 
are reliably delivered to the warfighter. Specifically, the business 
mission area addresses functional areas such as real property and 
human resources management. 

[17] The Defense Business Systems Management Committee was established 
under 10 U.S.C. §186, which requires the department to set up a 
committee to review and approve major updates of the defense BEA and 
to ensure that the obligation of funds for defense systems 
modernization is consistent with the criteria set out in 10 U.S.C. § 
2222.

[18] Pub. L. No. 110-181, § 904(b), 122 Stat. 3, 274 (Jan. 28, 2008).

[19] Pub. L. No. 110-417, § 908, 122 Stat. 4356, 4569 (Oct. 14, 2008).

[20] [hyperlink, http://www.gao.gov/products/GAO-13-283].

[21] GAO, DOD Business Systems Modernization: Progress in Establishing 
Corporate Management Controls Needs to Be Replicated Within Military 
Departments, [hyperlink, http://www.gao.gov/products/GAO-08-705] 
(Washington, D.C.: May 15, 2008); DOD Business Systems Modernization: 
Progress Continues to Be Made in Establishing Corporate Management 
Controls, but Further Steps Are Needed, [hyperlink, 
http://www.gao.gov/products/GAO-07-733] (Washington, D.C.: May 14, 
2007); Business Systems Modernization: DOD Continues to Improve 
Institutional Approach, but Further Steps Needed, [hyperlink, 
http://www.gao.gov/products/GAO-06-658] (Washington, D.C.: May 15, 
2006); and DOD Business Systems Modernization: Important Progress Made 
in Establishing Foundational Architecture Products and Investment 
Management Practices, but Much Work Remains, [hyperlink, 
http://www.gao.gov/products/GAO-06-219] (Washington, D.C.: Nov. 23, 
2005).

[22] [hyperlink, http://www.gao.gov/products/GAO-09-586].

[23] [hyperlink, http://www.gao.gov/products/GAO-11-684], [hyperlink, 
http://www.gao.gov/products/GAO-10-663], and [hyperlink, 
http://www.gao.gov/products/GAO-09-586].

[24] [hyperlink, http://www.gao.gov/products/GAO-13-283]. 

[25] http://www.gao.gov/duplication/action_tracker/1712]. 

[26] This figure only reflects data provided by DOD and reported in 
[hyperlink, http://www.gao.gov/products/GAO-05-702] ($318 million) 
along with a recent update on costs incurred by the Office of the 
Deputy Chief Management Officer in developing the BEA between fiscal 
years 2010 and 2013 ($61 million).The period of time between our June 
2005 report and the beginning of fiscal year 2010 is not accounted for 
because those costs were incurred by the now-disestablished Business 
Transformation Agency.

[27] [hyperlink, http://www.gao.gov/products/GAO-11-318SP] and 
[hyperlink, http://www.gao.gov/products/GAO-12-453SP].

[28] [hyperlink, http://www.gao.gov/products/GAO-06-219].

[29] Semantic web technologies include the use of nonproprietary, open 
standards and protocols to develop DOD architectures to allow users 
to, among other things, locate and analyze needed architecture 
information across the department. 

[30] [hyperlink, http://www.gao.gov/products/GAO-07-451].

[31] The perform reporting business activity includes receiving 
financial and management reporting requirements, preparing the 
information product, and distributing the finished product to the 
requestor. 

[32] The provide geospatial visualization activity manages geospatial 
IT visualization applications and services for DOD business missions 
and creates geospatial information products. 

[33] The manage environment liability information business activity 
identifies and values environmental liabilities, and prepares 
environmental liability information for reporting purposes. 

[34] Among other things, DOD describes capabilities as information 
that describes the ability to achieve a desired effect under specific 
standards and conditions through a combination of means and ways to 
perform a set of tasks. For example, the account for personnel 
business capability is associated with, among other things, accounting 
for time, absence, and labor, and performing leave and absence 
administration.

[35] The perform manpower planning business subprocess is associated 
with projecting manpower requirements, identifying the mission list, 
and developing policy and procedure guidance, to support preparation 
of the DOD budget, and includes both budgetary and executionary 
requirements. 

[36] The manage recruiting business subprocess is associated with 
managing the recruitment process for applicants who apply to the Armed 
Forces. 

[37] The manage labor relations subprocess is associated with managing 
the relationship between the agency and its unions and bargaining 
units. 

[38] The create purchase requisition business subprocess relates to 
the initiation and management of requests for the purchase of goods 
and/or services. 

[39] The award procurement instrument business subprocess results from 
the execution of an approved acquisition/sourcing plan and results in 
the execution of contractual documentation and the legal obligation of 
funds. 

[40] The manage disbursements business subprocess supports all 
activities necessary to execute the payment process for transactions 
that have been authorized for payment.

[41] Each year, agencies submit the exhibit 300 to OMB to justify each 
request for a major IT investment.

[42] DOD considers these systems to be "core" systems. Officials from 
DOD components designated every defense business system as either a 
core or legacy system, according to the department's business system 
investment management guidance. This guidance defines covered legacy 
systems as those that are to be terminated within 36 months of the 
date of their certification approval and all other systems are 
considered core systems. 

[43] The Defense Acquisition Management Information Retrieval system 
is a DOD initiative that provides enterprise visibility to Acquisition 
program information.

[44] According to DOD's architecture framework, an SV-8 describes the 
life cycle of systems and how they change over time. In addition, an 
SV-8 can be combined with other architecture information to show how 
an organization's capabilities change over time. 

[45] The organizational execution plan refers to, among other things, 
leveraging common services for identity, security, communication, and 
workflow. 

[46] The April 2013 guidance defines the target business systems 
computing environment as the "to be" environment consisting of (1) the 
core covered defense business system programs and related resources 
which the DOD will use to conduct its major business processes and (2) 
the supporting enterprise IT infrastructure and related resources, 
such as networks, communications, enterprise shared services, 
enterprise information assurance, in the enterprise information 
environment and other mission areas.

[47] [hyperlink, http://www.gao.gov/products/GAO-09-586].

[48] Due to ongoing budget negotiations, DOD did not prepare its 
detailed fiscal year 2014 budget request in time for inclusion in this 
report.

[49] [hyperlink, http://www.gao.gov/products/GAO-09-586] and 
[hyperlink, http://www.gao.gov/products/GAO-12-685].

[50] GAO's IT Investment Management framework provides a method for 
evaluating and assessing how well an agency is selecting and managing 
its IT resources. The framework, which describes five progressive 
stages of maturity that an agency can achieve in its investment 
management capabilities, was developed on the basis of our research 
into the IT investment management practices of leading private-and 
public-sector organizations. See GAO, Information Technology 
Investment Management: A Framework for Assessing and Improving Process 
Maturity, [hyperlink, http://www.gao.gov/products/GAO-04-394G] 
(Washington, D.C.: March 2004). 

[51] According to DOD officials, a functional strategy was not created 
for the department's security cooperation functional area because it 
does not currently have any defense business systems. Additionally, 
they determined that it would be more useful to create a single 
combined strategy for the acquisition functional area and logistics 
and materiel readiness. 

[52] According to the March Congressional Report, certification 
results were reported as of December 3, 2012.

[53] For fiscal year 2013 certification, investments could assert 
compliance to BEA 8.0 or 9.0.

[54] Generally, if a defense business system has not yet deployed any 
capability into an operational environment, an investment is asserted 
as being "planned compliant." 

[55] GAO, DOD Business Systems Modernization: Key Navy Programs' 
Compliance with DOD's Federated Business Enterprise Architecture Needs 
to Be Adequately Demonstrated, [hyperlink, 
http://www.gao.gov/products/GAO-08-972] (Washington, D.C.: Aug 7, 2008).

[56] See app. I for details on our methodology related to four case 
studies, which looked at documentation used to support BEA and 
business process reengineering compliance assertions for fiscal year 
2013 certifications related to Air Force-Integrated Personnel and Pay 
System (AF-IPPS), Integrated Personnel and Pay System-Navy (IPPS-N), 
Integrated Personnel and Pay System-Army (IPPS-A), and the Integrated 
Electronic Health Record system (iEHR).

[57] The Standard Financial Information Structure is intended to 
provide a standard financial management data structure and uniformity 
throughout DOD in reporting on the results of operations.

[58] DOD Inspector General, Enterprise Business System Was Not 
Configured to Implement the U.S. Government Standard General Ledger at 
the Transaction Level, DODIG 2013-057 (Alexandria, VA: March 20, 2013).

[59] GAO, Business Process Reengineering Assessment Guide (Version 3), 
[hyperlink, http://www.gao.gov/products/GAO/AIMD-10.1.15] (Washington, 
D.C.: May 1997).

[60] GAO, Electronic Health Records: Long History of Management 
Challenges Raises Concerns about VA's and DOD's New Approach to 
Sharing Health Information, [hyperlink, 
http://www.gao.gov/products/GAO-13-413T] (Washington, D.C.: Feb. 27, 
2013).

[61] Systems in sustainment are operational, not currently 
modernizing, and are not slated for retirement.

[62] [hyperlink, http://www.gao.gov/products/GAO/AIMD-10.1.15].

[63] [hyperlink, http://www.gao.gov/products/GAO-12-685].

[64] GAO, National Archives and Records Administration: Oversight and 
Management Improvements Initiated, but More Action Needed, [hyperlink, 
http://www.gao.gov/products/GAO-11-15 (Washington, D.C.: Oct. 5, 2010).

[65] [hyperlink, http://www.gao.gov/products/GAO-12-685]. 

[66] GAO, Defense Business Transformation: Improvements Made but 
Additional Steps Needed to Strengthen Strategic Planning and Assess 
Progress, [hyperlink, http://www.gao.gov/products/GAO-13-267] 
(Washington, D.C.: Feb. 12, 2013).

[67] [hyperlink, http://www.gao.gov/products/GAO-10-663], [hyperlink, 
http://www.gao.gov/products/GAO-11-684], and [hyperlink, 
http://www.gao.gov/products/GAO-12-241].

[68] [hyperlink, http://www.gao.gov/products/GAO-12-685].

[69] Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-1856 (2004), as 
amended by the NDAA for Fiscal Year 2013, Pub. L. No. 112-239, 126 
Stat. 1632 (2012), the NDAA for Fiscal Year 2012, Pub. L. No. 112-81, 
125 Stat. 1298 (2011), and the NDAA for Fiscal Year 2010, Pub. L. No. 
111-84, 123 Stat. 2190 (2009) (codified in part at 10 U.S.C. § 2222.)

[70] GAO's ITIM framework provides a method for evaluating and 
assessing how well an agency is selecting and managing its IT 
resources. The framework, which describes five progressive stages of 
maturity that an agency can achieve in its investment management 
capabilities, was developed on the basis of our research into the IT 
investment management practices of leading private-and public-sector 
organizations. See GAO, Information Technology Investment Management: 
A Framework for Assessing and Improving Process Maturity, [hyperlink, 
http://www.gao.gov/products/GAO-04-394G] (Washington, D.C.: March 
2004). 

[71] Department of Defense, DOD Business Enterprise Architecture: 
Compliance Guidance for BEA 8.0 (Mar. 11, 2011).

[72] Department of Defense, Guidance for the Implementation of Section 
1072 - Business Process Reengineering (Apr. 30, 2011); and Business 
Process Reengineering Assessment Guidance (Sept. 28. 2012).

[73] GAO, Business Process Reengineering Assessment Guide (Version 3), 
[hyperlink, http://www.gao.gov/products/GAO/AIMD-10.1.15] (Washington, 
D.C.: May 1997).

[74] GAO, National Archives and Records Administration: Oversight and 
Management Improvements Initiated, but More Action Needed, [hyperlink, 
http://www.gao.gov/products/GAO-11-15] (Washington, D.C.: Oct 5, 
2010); Information Technology: FBI Has Largely Staffed Key 
Modernization Program, but Strategic Approach to Managing Program's 
Human Capital Is Needed, [hyperlink, 
http://www.gao.gov/products/GAO-07-19] (Washington, D.C.: Oct. 16, 
2006); and A Model of Strategic Human Capital Management, [hyperlink, 
http://www.gao.gov/products/GAO-02-373SP] (Washington, D.C.: Mar. 15, 
2002).

[75] Department of Defense, Same Mission, New Vision. The Future of 
the DOD Business Enterprise Architecture (March 2012).

[End of section] 

GAO’s Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO’s commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
“E-mail Updates.” 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO’s 
website, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov]. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Katherine Siggerud, Managing Director, siggerudk@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, DC 20548. 

Public Affairs: 
Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, DC 20548. 

[End of document]