This is the accessible text file for GAO report number GAO-13-455 
entitled 'Information Technology: HUD Needs to Improve Key Project 
Management Practices for Its Modernization Efforts' which was released 
on June 12, 2013. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to Congressional Committees: 

June 2013: 

Information Technology: 

HUD Needs to Improve Key Project Management Practices for Its 
Modernization Efforts: 

GAO-13-455: 

GAO Highlights: 

Highlights of GAO-13-455, a report to congressional committees. 

Why GAO Did This Study: 

HUD relies extensively on IT to carry out its mission of strengthening 
communities and ensuring affordable housing and has reported that 
efforts are under way to modernize its aging, duplicative, and poorly 
integrated systems. Committee report language mandated GAO to evaluate 
the implementation of project management practices for HUD’s IT 
modernization efforts. The objective was to identify the extent to 
which the department implemented key project management practices for 
the FHA Transformation and NGMS modernization efforts. GAO assessed 
project management artifacts for 9 FHA Transformation and 5 NGMS 
projects in the areas of project planning (charters, work breakdown 
structures, and project management plans), requirements management 
(requirements management plans and traceability matrixes), and 
acquisition planning (acquisition strategies) against best practices. 
GAO also interviewed officials. 

What GAO Found: 

The Department of Housing and Urban Development (HUD) has taken 
initial steps toward applying key project management practices in the 
areas of project planning, requirements management, and acquisition 
planning for its Federal Housing Administration Transformation (FHA 
Transformation) Initiative to address performance gaps in housing 
insurance programs and its Next Generation Management System (NGMS) to 
improve management of its affordable housing programs. However, HUD 
has not yet fully implemented any of these practices in executing and 
managing the information technology (IT) projects associated with 
these efforts. Specifically, while the department had developed 
project management artifacts such as charters and requirements 
management plans, none of these documents included all of the key 
details that could facilitate effective management of its projects 
such as full descriptions of the work necessary to complete the 
projects, cost and schedule baselines, or prioritized requirements, 
among other things. Department officials attributed these deficiencies 
to a lack of project management expertise. The table below summarizes 
GAO’s assessment of key practices for these modernization efforts. 

Table: GAO Assessment of Key Project Management Practices, as of April 
2013: 

Artifact: Project charter       
Description: Formally authorizes a project and identifies high-level 
information; 
FHA Transformation: Artifact developed and contained essential 
information; 
NGMS: Artifact developed and contained essential information. 

Artifact: Work breakdown structure; 
Description: Defines the necessary work and provides a basis for cost 
and schedule estimates; 
FHA Transformation: Artifact not yet developed; 
NGMS: Artifact developed and contained essential information. 

Artifact: Project management plan; 
Description: Primary source for how to execute project objectives and 
measure progress; 
FHA Transformation: Artifact developed and contained essential 
information; 
NGMS: Artifact developed and contained essential information. 

Artifact: Requirements management plan; 
Description: Outlines processes and methods for developing and 
managing requirements; 
FHA Transformation: Artifact developed and contained essential 
information; 
NGMS: Artifact developed and contained essential information. 

Artifact: Requirements traceability matrix; 
Description: Provides the ability to follow a requirement from origin 
to implementation; 
FHA Transformation: Artifact developed and contained essential 
information; 
NGMS: Artifact developed and contained essential information. 

Artifact: Acquisition strategy; 
Description: Describes how the project will manage contracts in 
coordination with other processes; 
FHA Transformation: Artifact developed and contained essential 
information; 
NGMS: Artifact developed and contained essential information. 

Source: GAO analysis of HUD data. 

[End of table] 

Because HUD has not taken these foundational steps to fully define its 
modernization efforts, the department is not well positioned to 
successfully manage or execute the associated projects. These 
incomplete documents limit the department’s ability to fully 
understand the work to be completed or accurately report project 
progress. A major reason for these information deficiencies is HUD’s 
inadequate development and use of its project management framework, 
which did not ensure the quality or completeness of artifacts 
developed. Specifically, the framework did not always include 
essential guidance and, in other cases, the projects did not always 
implement the guidance provided. Further, the governance structure did 
not consistently operate as intended to provide adequate oversight to 
ensure compliance with key project management practices. As a result, 
the department increases the risk of continuing to inadequately apply 
project management practices and may not be positioned to effectively 
manage or report progress of its key modernization efforts. Fully 
implementing effective project management practices is critical for 
the success of these two modernization efforts and others under way or 
planned. 

What GAO Recommends: 

GAO recommends that HUD establish a plan of action to fully implement 
best practices, provide needed project management expertise, and 
improve the development and use of its project management framework 
and governance structure. In written comments, HUD concurred with the 
recommendations to improve its framework and governance, but did not 
concur with the entirety of the recommendation to develop a plan of 
action, and contended that the need for project management expertise 
did not follow from the premises established in the draft report. GAO 
maintains that these actions are necessary as discussed in this report. 

View [hyperlink, http://www.gao.gov/products/GAO-13-455]. For more 
information, contact Valerie C. Melvin at (202) 512-6304 or 
melvinv@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

HUD Has Begun Applying Key Practices for Its Modernization Efforts but 
Has Been Hindered by Inadequacies in Its Project Management Framework 
and Governance: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objective, Scope, and Methodology: 

Appendix II: Summary of HUD's Transformation Initiative Information 
Technology Modernization Efforts: 

Appendix III: Detailed Assessment of FHA Transformation and NGMS 
Project Management Documentation: 

Appendix IV: Comments from the Department of Housing and Urban 
Development: 

Appendix V: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: Summary of FHA Transformation Projects Included in Our Study: 

Table 2: Summary of Next Generation Management System Projects 
Included in Our Study: 

Table 3: Summary of Key Project Management Practices for the 14 FHA 
Transformation and Next Generation Management System Projects, as of 
April 2013: 

Table 4: HUD's Seven Transformation Initiative IT Modernization 
Efforts: 

Table 5: Assessment of FHA Transformation and NGMS Project Charters: 

Table 6: Assessment of FHA Transformation and NGMS Work Breakdown 
Structures: 

Table 7: Assessment of FHA Transformation and NGMS Project Management 
Plans: 

Table 8: Assessment of FHA Transformation and NGMS Requirements 
Management Plans: 

Table 9: Assessment of FHA Transformation and NGMS Requirements 
Traceability Matrixes: 

Table 10: Assessment of FHA Transformation and NGMS Acquisition 
Strategies: 

Figures: 

Figure 1: Simplified HUD Organizational Structure Including FHA 
Transformation and NGMS Modernization Efforts: 

Figure 2: HUD's Governance Structure for IT Investments: 

Abbreviations: 

CMMI: Capability Maturity Model Integration: 

FHA: Federal Housing Administration: 

HUD: Department of Housing and Urban Development: 

IT: information technology: 

LEAP: Lender Electronic Assessment Portal: 

MFH: multifamily housing: 

NGMS: Next Generation Management System: 

OCIO: Office of the Chief Information Officer: 

OCPO: Office of the Chief Procurement Officer: 

OMB: Office of Management and Budget: 

PMI: Project Management Institute: 

PMBOK: Project Management Body of Knowledge: 

PPM: Project Planning and Management: 

SEI: Software Engineering Institute: 

[End of section] 

GAO:
United States Government Accountability Office: 
441 G St. N.W. 
Washington, DC 20548: 

June 12, 2013: 

The Honorable Patty Murray: 
Chairman: 
The Honorable Susan M. Collins: 
Ranking Member: 
Subcommittee on Transportation, Housing and Urban Development, and 
Related Agencies: 
Committee on Appropriations: 
United States Senate: 

The Honorable Tom Latham: 
Chairman: 
The Honorable Ed Pastor: 
Ranking Member: 
Subcommittee on Transportation, Housing and Urban Development, and 
Related Agencies: 
Committee on Appropriations: 
House of Representatives: 

The Department of Housing and Urban Development (HUD) relies 
extensively on information technology (IT) to carry out its mission of 
creating strong, sustainable, inclusive communities and ensuring 
affordable housing. According to the department, in fiscal year 2012, 
IT systems supported HUD programs valued at more than $1 trillion. 
[Footnote 1] However, the department has reported that its systems are 
overlapping, duplicative, not integrated, necessitate manual 
workloads, and employ antiquated, costly-to-maintain technologies. 
Further, as we and others have reported, the department has long 
experienced shortcomings in its IT management capability.[Footnote 2] 
For example, a 2010 assessment[Footnote 3] of the department's IT 
environment concluded that HUD lacked sufficient capability to execute 
basic management functions, including project management,[Footnote 4] 
which inhibited effective IT operations and the successful delivery of 
IT solutions. 

For fiscal years 2010 and 2011, HUD invested in developing and 
modernizing its systems through funding provided by the department's 
Transformation Initiative.[Footnote 5] The department has been 
pursuing seven IT modernization efforts under this initiative, the two 
largest of which are the Federal Housing Administration Transformation 
(FHA Transformation) Initiative to address performance gaps in housing 
insurance programs and the Next Generation Management System (NGMS) to 
improve management of the department's affordable housing programs. To 
provide oversight and inform decision making, Congress established 
limitations on funding for the Transformation Initiative IT 
modernization efforts and required the department to submit 
expenditure plans that satisfied statutory conditions before fully 
obligating the available funds.[Footnote 6] We were required to review 
the department's expenditure plans and, accordingly, have reported on 
the results of our assessments.[Footnote 7] Concurrent with the 
modernization funding, the department also began implementing reforms 
to improve its IT management controls and guide its investments, 
including the establishment of a framework for planning and managing 
such projects. 

Given the important role that HUD's modernized systems are to have in 
supporting its mission, the Senate Report accompanying the 
department's fiscal year 2012 appropriation, as approved by the 
Conference report, mandated us to evaluate the implementation of 
project management practices for its IT modernization efforts. 
[Footnote 8] In this regard, our specific objective was to identify 
the extent to which the department has implemented key project 
management practices for the FHA Transformation and NGMS modernization 
efforts. 

To address this objective, we examined all of the 14 projects for FHA 
Transformation and NGMS that had been identified in the department's 
fiscal year 2011 expenditure plan. We assessed these projects against 
key IT management practices[Footnote 9] in the areas of (1) project 
planning, (2) requirements management, and (3) acquisition planning. 
[Footnote 10] As part of our analysis, we reviewed HUD project 
management documentation for the 14 FHA Transformation and NGMS 
projects, including charters, work breakdown structures (which define 
the necessary work and provide a basis for cost and schedule 
estimates), project management plans, requirements management plans 
and traceability matrixes (which provide the ability to follow a 
requirement from origin to implementation), and acquisition 
strategies. We assessed whether this documentation was consistent with 
project management practices identified by the Project Management 
Institute (PMI) and the Software Engineering Institute (SEI), and best 
practices we previously identified for cost estimating and project 
schedules.[Footnote 11] We also attended the department's project 
management meetings to observe how the identified practices were being 
applied. In addition, we interviewed responsible HUD officials, 
including FHA Transformation and NGMS management officials, 
procurement officials, and responsible IT officials. Appendix I 
provides additional details regarding our objective, scope, and 
methodology. 

We conducted this performance audit from June 2012 to June 2013 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objective. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objective. 

Background: 

To accomplish its mission, HUD administers community and housing 
programs that benefit millions of households each year. Among other 
things, the department provides affordable rental housing 
opportunities and helps homeless families and chronically homeless 
individuals and veterans. The department also administers mortgage 
insurance programs for single-family housing, multifamily housing, and 
health care. 

HUD relies on five main organizational components to carry out its 
mission. Of these, two components have lead responsibility for 
improving access to housing and are the business owners for related IT 
modernization efforts:[Footnote 12] 

* Housing/Federal Housing Administration (FHA): Programs within this 
office are responsible for contributing to building healthy 
communities, maintaining and expanding housing opportunities, and 
stabilizing credit markets in times of economic disruption. This 
office also regulates certain aspects of the housing industry. For 
example, the department currently reports that it provides insurance 
on loans made by its approved lenders for 4.8 million single-family 
mortgages and 13,000 multifamily projects, including manufactured 
homes and hospitals. The FHA Transformation modernization effort is 
managed within this office. 

* Public and Indian Housing: Programs within this office are 
responsible for creating opportunities for residents' self-sufficiency 
and economic independence. Toward this end, this office currently 
oversees a housing choice voucher program to subsidize housing for 
approximately 2.1 million low-income, elderly, and disabled families; 
a public housing program that subsidizes about 1.3 million housing 
units for vulnerable low-income families; and block grants and 
guarantee programs for Native American groups. The NGMS modernization 
effort is managed within this office. 

In addition, to support these organizational components, the 
department relies on various administrative offices to provide 
guidance and tools. These include the department's Office of the Chief 
Information Officer (OCIO) and the Office of the Chief Procurement 
Officer (OCPO). Through coordination with the organizational 
components, OCIO manages IT resources and provides support for the 
department's infrastructure, security, and ongoing projects. This 
office also provides project management guidance and technical 
expertise to modernization efforts.[Footnote 13] For its part, OCPO is 
responsible for obtaining contracted goods and services required by 
the department to meet its strategic objectives. This office is 
involved with initiating acquisition actions upon request by the 
organizational components. 

Further, HUD's Deputy Secretary is responsible for managing the 
department's daily operations, annual operating budget, and 
approximately 8,900 employees. As part of this role, the Deputy 
Secretary conducts biweekly meetings with stakeholders to discuss the 
Secretary's priorities. During these meetings, the scope, milestones, 
risks, and status of action items related to priority issues are 
discussed. The FHA Transformation and NGMS modernization efforts are 
designated as priority and each has its own biweekly meeting. 

A simplified view of the department's housing organization structure 
and the offices responsible for FHA Transformation and NGMS is 
provided in figure 1. 

Figure 1: Simplified HUD Organizational Structure Including FHA 
Transformation and NGMS Modernization Efforts: 

[Refer to PDF for image: Organizational Structure] 

Top level: 
Secretary. 

Second level: 
Deputy Secretary; 
Chief Operating Officer: 
* Chief Procurement Officer; 
* Chief Information Officer. 

Third level, reporting to Deputy Secretary: 
* Housing/Federal Housing Administration: 
- FHA Transformation Initiative; 
* Public and Indian Housing; 
- Next Generation Management System. 

Source: GAO analysis of HUD data. 

[End of figure] 

Overview of HUD's IT Environment: 

According to the fiscal year 2014 President's Budget request for HUD, 
$285.1 million is expected to be spent on IT investments. HUD's IT 
environment consists of multiple systems that, among other things, are 
intended to help the department coordinate with lending institutions 
to insure mortgages, collect and manage state and local housing data, 
process applications for community development, and issue vouchers 
that provide access to subsidized housing. In particular, the 
department's housing programs rely on systems for processing and 
managing these business operations. For example, systems within the 
Office of Housing are expected to process mortgage insurance 
applications, bill and collect premiums, pay claims, manage 
receivables and other assets, track delinquencies and defaults, and 
support staff in providing counseling to first-time home buyers and 
existing homeowners. Additionally, Public and Indian Housing programs 
that use systems are intended to process vouchers for different rental 
assistance programs, as well as to support the processing of 
applications for, and the management of, more than 50 grant programs 
administered by the department. 

However, HUD's current IT environment has not effectively supported 
its business operations because its systems are overlapping and 
duplicative, not integrated, necessitate manual workloads, and employ 
antiquated technologies that are costly to maintain. For example, the 
department reported from 2008 to 2012 that its IT environment 
consisted of:[Footnote 14] 

* Over 200 information systems, many of which perform the same 
function and, thus, are overlapping and duplicative. Specifically, 
different systems perform the same task to separately support grants 
management, loan processing, and subsidies management. 

* Stovepiped, nonintegrated systems that result in identical data 
existing in multiple systems. For example, two organizational 
components store about 80 percent of similar data in separate 
databases that provide information on rental assistance participants. 

* Manual processing for business functions due to a lack of systems to 
support these processes. For example, specific NGMS projects are 
intended to replace existing ad hoc analyses performed in spreadsheets 
and databases with systems that automate and standardize those 
functions. 

* Antiquated technology (15-to 30-years old) and complex systems that 
are costly to maintain. For example, the department relies on 
different programming languages and operating systems, which requires 
specialized skills to operate and maintain. 

Additionally, HUD engaged contractors to conduct an assessment of the 
department's environment. This assessment, issued in January 2011, 
concluded that unclear reporting relationships hindered the 
enforcement of IT policies; contractor performance information was not 
used to inform management decisions; technical standards were lacking 
or not enforced; and data management practices did not support 
business needs.[Footnote 15] 

HUD's Transformation Initiative IT Modernization Efforts: 

Through the Transformation Initiative's IT component, HUD has begun 
addressing challenges to its environment and modernizing its systems. 
In this regard, the department initiated seven IT modernization 
efforts, of which FHA Transformation and NGMS are the two largest. 
[Footnote 16] For fiscal years 2010 and 2011, the department reported 
that the Transformation Initiative funding made available for FHA 
Transformation and NGMS was $58.5 and $41.1 million, respectively. 
[Footnote 17] (See later discussion in this report regarding costs 
associated with the 14 projects in our study.) 

FHA Transformation was initiated to improve the department's 
management of insurance programs through the development and 
implementation of a modern financial services IT environment[Footnote 
18] that is expected to improve loan endorsement processes, collateral 
risk capabilities, and fraud prevention. In August 2009, HUD published 
the FHA Office of Housing Information Technology Strategy and 
Improvement Plan,[Footnote 19] which identified and prioritized 25 IT 
areas with performance gaps for its single-family housing, multifamily 
housing development and rental assistance, health care facilities 
programs, and enterprise applications. In May 2010, FHA Transformation 
began planning and executing modernization efforts aimed at addressing 
the gaps identified in the plan. Specifically, the modernization 
initiative is intended to implement technology within the following 
four functional areas aimed at addressing changes in FHA's business 
model, operating environment, and components of the loan life cycle: 

* Infrastructure and legacy migration: Provide a scalable 
infrastructure to support rules engines, analytics, and reporting 
systems, as well as a mechanism for transferring legacy applications 
to the new platform. Specifically, the Federal Financial Services 
Platform project is intended to provide hardware and standard software 
to support case management and migration of legacy applications (e.g., 
the Computerized Homes Underwriting Reporting System) for all lines of 
business. 

* Borrower/collateral risk management and fraud monitoring: Provide 
tools for analyzing, monitoring, and managing emerging issues and 
trends in the housing market, including borrower and collateral risk, 
appraisals, and fraud, as it relates to the FHA portfolio. For 
example, the Legacy Application Transformation project is expected to 
implement a software service tool that aggregates data to identify 
emerging issues and trends in borrower risk and fraud by analyzing the 
accuracy and validity of verified assets, income, and employment on 
individual loans. Other projects within this functional area include 
business process reengineering and a pilot designed to automate and 
streamline the multifamily housing underwriting process. Using the new 
infrastructure, an automated underwriting tool is expected to be 
deployed to expand the capabilities for processing loan applications 
for insurance programs and replace current systems (e.g., the 
Development Application Processing System). 

* Counterparty management: [Footnote 20] Provide applications for 
improved performance and compliance of lenders and appraisers through 
more proactively identifying risk trends and improving loan file 
review techniques. Specific projects include the Lender Electronic 
Assessment Portal, a web-based automated delivery of electronic 
applications and storage of lender application data that assists with 
reviewing new lender applications and requests for annual 
recertification to participate in FHA programs. Future plans call for 
replacing seven legacy applications. 

* Portfolio analysis: Provide tools intended to augment risk 
monitoring and management; enhance predictive analytics; provide 
timely and flexible reporting; and deliver more accurate, detailed 
information to decision makers. For example, the Portfolio Risk 
Reporting & Analytics project is intended to provide a web-based 
software service tool for modeling FHA program risks. While initial 
use of the software is to include receiving hard-copy reports from the 
third-party vendor, HUD also expects to deploy the tool within the 
department's infrastructure in order for FHA employees to have access 
to reports and the analytics dashboard data electronically. 

Overall leadership for FHA is provided by the Assistant Secretary for 
Housing/Federal Housing Commissioner, who chairs the modernization 
effort steering committee; the General Deputy Assistant Secretary for 
Housing; and the Director for the Office of Program Systems 
Management,[Footnote 21] who is the executive sponsor. The 
modernization effort also has a project management office that is 
responsible for executing and managing the associated projects. As of 
April 2013, FHA Transformation consisted of 10 projects, 9 of which 
were included in our study.[Footnote 22] Table 1 summarizes the 9 FHA 
Transformation projects that we assessed as part of our study. 

Table 1: Summary of FHA Transformation Projects Included in Our Study: 

Project name: Capital Needs Assessment; Purpose: Develop a standard 
software template tool to be made available to all federal agencies 
administering housing programs to support capital needs assessments. 

Project name: Federal Financial Services Platform; Purpose: Install and 
configure hardware and software for a new financial services IT 
environment. 

Project name: Healthcare Automated Lender Application Pilot[A]; 

Purpose: Provide a modernized business process and systems capability 
to support the Office of Healthcare Programs' loan origination and 
production processing for mortgage insurance for residential care 
facilities. 

Project name: Legacy Application Transformation[B]; Purpose: Implement 
analytical tools to support the analysis of aggregate data to identify 
emerging issues and trends in borrower/collateral risk and fraud. 

Project name: Lender Electronic Assessment Portal (LEAP) Automation of 
Lender Approval Workflow; 
Purpose: Develop tools that are intended to monitor lender performance 
and policy compliance, including identification of key performance 
characteristics, risk drivers, and comparative performance across 
lenders. 

Project name: LEAP Institution Manager[C]; Purpose: Automate the manual 
process of reviewing and approving lender documentation for the annual 
recertification of lenders participating in FHA's lender programs. 

Project name: Multifamily Housing (MFH) Development & Underwriting 
Business Process Reengineering/Automated Underwriting Solution; 

Purpose: Automate the multifamily housing underwriting business 
process to streamline operations and increase effectiveness, 
efficiency, and risk management capabilities. 

Project name: MFH Physical Inspections Alignment; Purpose: Enhance 
HUD's Real Estate Assessment Center software systems to eliminate 
duplicative inspections performed by state housing agencies. 

Project name: Portfolio Risk Reporting & Analytics; Purpose: Implement 
a web-based tool that provides reporting and analytics capabilities 
that allow FHA users to model risks associated with its portfolio of 
mortgages. 

Source: GAO analysis of HUD data. 

[A] FHA Transformation officials reported on February 21, 2013, that, 
as a result of reorganization, the pilot was expected to become part 
of the MFH Development & Underwriting Business Process Reengineering/ 
Automated Underwriting Solution project. 

[B] This project was formerly named Emerging Trends Analysis. 

[C] This project was formerly named LEAP Recertification. 

[End of table] 

The NGMS modernization effort is intended to provide an integrated 
system with a seamless view of financial and program data currently 
warehoused in disparate data sources and a new set of monitoring, 
oversight, and software tools directed at ensuring that funds are used 
to assist affordable housing participants and reduce improper payment 
errors.[Footnote 23] In November 2011, the department used contractors 
to develop four prototype software tools aimed at demonstrating 
anticipated NGMS functionality for voucher programs.[Footnote 24] 
However, in July 2012, the department determined that the prototypes 
that had been developed would not address its business needs. As a 
result, the department initiated planning efforts to restructure the 
modernization effort and expand the scope to include all Public and 
Indian Housing lines of business. 

HUD has reported that the aim of the restructured effort is to enhance 
the department's affordable housing program, improve end-user 
satisfaction, streamline complex business processes, and integrate 
disparate IT systems into a common, modernized platform.[Footnote 25] 
The department intends for NGMS to support efforts to improve HUD's 
financial accountability by more accurately quantifying budgetary data 
resources, measuring program effectiveness, and justifying the 
agency's budget formulations and requests. NGMS is expected to help 
department personnel reduce improper payments by identifying anomalies 
in operating costs, reserves, and subsidy payments. Once implemented, 
NGMS is intended to provide staff with a new set of monitoring, 
oversight, and analysis tools to ensure that allocated federal funds 
are used efficiently to assist affordable housing participants. The 
department is taking an incremental approach to developing NGMS and 
expects to deliver initial functionality by August 2013. NGMS system 
and software development projects are designed to support four 
functional areas: 

* Financial management: Provide automated processes for budget 
forecasting and formulation and cash management based on real-time 
data that are expected to allow the department to anticipate cash flow 
needs through precise scenarios and disburse funds on the basis of 
project and tenant records, eliminating reconciliations. For example, 
the Budget Forecasting and Formulation project is intended to develop 
a solution that will include forecasting functionality, data 
aggregation, and analytics to support the budget development process 
for Public and Indian Housing programs such as vouchers, 
administrative fees, family self-sufficiency, mainstream vouchers, and 
housing assistance programs. In addition, this functional area is 
expected to migrate data from HUD's Central Accounting and Program 
system and utilize information gathered from Public Housing 
Authorities regarding subsidized housing programs through an interface 
with the department's New Core system. 

* HUD operations: Provide a single point of access to data and 
information to improve efficiency and reduce administrative burden 
through a New Data Collection system that is expected to replace 
legacy systems (e.g., Public and Indian Housing Information Center) 
and provide new functionality for subsidized housing programs, 
geospatial data on physical housing, real-time occupancy information, 
and energy conservation measures for properties. In the interim, the 
Portfolio and Risk Management Tool project is expected to provide 
aggregated data about Public Housing Authorities through a standard 
business intelligence solution and is expected to expand its use to 
partner operations in the future. 

* Partner operations: Expand the department's operations system to 
provide a web-based single point of access for gathering consistent 
and accurate information from families and landlords to be used in the 
operation of public housing and voucher programs administered by 
Public Housing Authorities. 

* Business support: Provide expanded access and use of NGMS IT 
solutions; grant HUD and program participants better access to 
information and technical assistance through a central point of access 
with live help and self-paced guides; and develop the necessary 
infrastructure and processes to enable timely and accurate answers to 
end users' inquiries. 

Overall leadership for NGMS is provided by the General Deputy 
Assistant Secretary for Public and Indian Housing, who is the 
executive sponsor and chair of the modernization effort steering 
committee. The modernization effort also has a project management 
office that is responsible for executing and managing the associated 
projects. As of April 2013, NGMS consisted of six projects, of which 
five were included in our study.[Footnote 26] Table 2 summarizes the 
NGMS projects that we assessed. 

Table 2: Summary of Next Generation Management System Projects 
Included in Our Study: 

Project name: Affordable Housing Data Architecture; 
Purpose: Create a data model for affordable housing programs and serve 
as the architectural foundation for other modernization efforts. 

Project name: Budget Forecasting and Formulation; 
Purpose: Define and develop a tenant-based rental assistance solution 
to automate the end-to-end process of developing precise, targeted 
budget forecasting scenarios based on timely, accurate, and granular 
data. 

Project name: Cash Management; 
Purpose: Provide the ability to disburse funds based on grantee needs 
supported by near real-time data and continuous budget reconciliations. 

Project name: Integrated Budget Forecasting Model; 
Purpose: Define and develop a solution to improve budget forecasting 
and program funds management for project-based rental assistance 
programs and create reliable mechanisms to support the end-to-end 
financial management life cycle from initial budget formulation 
requests to financial closure and reconciliation. 

Project name: Portfolio and Risk Management Tool; 
Purpose: Develop the capability to analyze portfolio performance and 
risk through a single point of access. 

Source: GAO analysis of HUD data. 

[End of table] 

Key Practices for Modernization Project Management: 

Effective use of project planning and management practices is 
essential for the success of modernization efforts such as those being 
undertaken by HUD. Our prior reviews of federal agencies have shown 
that, when effectively implemented, these practices can significantly 
increase the likelihood of delivering promised capabilities on time 
and within budget.[Footnote 27] Moreover, project management maturity 
is dependent on an agency's standardization and institutionalization 
of such practices. PMI reported in its March 2013 annual survey of 
project management professionals that high-performing organizations 
are almost three times more likely than low-performing organizations 
to use standardized practices throughout the organization, and 
generate better project outcomes.[Footnote 28] 

To guide the application of best practices, we and others, including 
PMI and SEI at Carnegie Mellon University, have issued reports and 
frameworks for effective project management.[Footnote 29] These 
reports and frameworks emphasize practices that include the 
development of essential documentation needed for the execution and 
management of projects in the areas of project planning (charters, 
work breakdown structures, and project management plans), requirements 
management (requirements management plans and traceability matrixes), 
and acquisition planning (acquisition strategies). 

* Project planning: This practice helps establish project objectives 
and outline the course of action required to attain those objectives. 
It also provides a means to track, review, and report progress and 
performance of the project by defining project activities and 
developing cost and schedule estimates, among other things. Project 
planning involves, for example, creating a charter to authorize 
project work, developing a work breakdown structure, and establishing 
project management plans that provide processes for measuring progress. 

* Requirements management: Having a documented strategy for developing 
and managing requirements can help ensure that the final product will 
function as intended. Effective management of requirements involves 
assigning responsibility for them, tracking them, and controlling 
changes to them over the course of the project. It also ensures that 
each requirement traces back to the business need and forward to its 
design and testing. Requirements management practices call for the use 
of requirements management plans to provide a mechanism for 
documenting the process for managing requirements and associated 
traceability matrixes, which are intended to facilitate efforts to 
link requirements to identified business needs to help ensure that 
they will be satisfied by the end product. 

* Acquisition planning: Effective IT project management also involves 
creating strategies to serve as the road map for acquisition planning. 
Such road maps are used for early planning of procurements and are 
developed by a project manager. Among other things, acquisition 
strategies should address plans for how projects will manage risks, 
deliverables, and reporting on contractor performance.[Footnote 30] 

In addition to calling for agencies to apply best practices, federal 
guidance, along with our framework for managing IT investments and our 
prior reviews of federal investments,[Footnote 31] outlines the 
importance of having reviews conducted by management at various points 
throughout a project's life cycle. Such reviews are critical to 
helping ensure that cost, schedule, and performance goals for a 
project are satisfied, and they can provide early detection of risks 
and problems that could impede progress toward those goals. Further, 
management reviews can help ensure that appropriate quality standards 
are achieved and provide input for areas that need improvement. 

HUD's IT Project Management Framework and Governance Structure: 

In order to better manage its modernization efforts, during 2011 HUD 
established new policies and procedures for executing and governing IT 
investments.[Footnote 32] Specifically, in April 2011, the department 
developed a Project Planning and Management (PPM) framework[Footnote 
33] to provide guidance for managing a project's life cycle in 
accordance with best practices. Using the framework, projects--such as 
those related to FHA Transformation and NGMS modernization efforts--
are expected to proceed through life-cycle phases that require 
specific documents to demonstrate project activities and outcomes. The 
framework provides guidance through sample templates, with associated 
instructions and checklists that projects can use in developing their 
documentation. The framework also calls for management reviews that 
are intended to help ensure projects are aligned with the department's 
architecture and technical standards, and that they have developed 
required information before committing resources to the next life-
cycle phase. For example, at the initiation of a project, among other 
things, a charter and schedule are expected to be developed and 
approved by a review committee. In addition, during a project's 
definition phase critical documents, such as a project management 
plan, requirements management plan, requirements traceability matrix, 
and an acquisition strategy, are also expected to be developed and 
approved by a review committee. 

In July 2011, HUD also established a governance policy that set forth 
processes, standards, roles, and responsibilities to facilitate 
decision making around investments, stakeholder relationships, project 
life-cycle management, and other important IT operational areas. 
[Footnote 34] In particular, the policy established an IT governance 
structure consisting of the Executive Investment Board, Customer Care 
Committee, Investment Review Subcommittee, and Technical Review 
Subcommittee. Figure 2 provides a simplified depiction of this 
governance structure. 

Figure 2: HUD's Governance Structure for IT Investments: 

[Refer to PDF for image: Governance Structure] 

Top level: 
Executive Investment Board. 

Second level, reporting to Executive Investment Board: 
Customer Care Committee. 

Third level, reporting to Customer Care Committee: 
Investment Review Subcommittee; 
Technical Review Subcommittee. 

Source: GAO analysis of HUD data. 

[End of figure] 

According to HUD's Policy for Information Technology Governance 
handbook, these governance bodies have the following composition and 
responsibilities. 

* Executive Investment Board: Comprised of senior leaders, including 
the HUD Secretary, Deputy Secretary, and Chief Information Officer, 
with responsibilities for providing strategic direction, managing the 
IT investment portfolio, and overseeing and approving projects that 
cost more than $5 million. 

* Customer Care Committee: Comprised of executives, including the 
Chief Information Officer, the Chief Procurement Officer, and deputy 
assistant secretaries, who manage IT investments and perform project 
management oversight by reviewing and submitting recommendations to 
the Executive Investment Board, and coordinating with the 
subcommittees responsible for approving projects that cost between 
$500,000 and $5 million. 

* Investment Review Subcommittee: Comprised of business area 
personnel, including representatives designated by deputy assistant 
secretaries, who focus on investment management oversight with respect 
to business cases and budget information for the Office of Management 
and Budget (OMB). 

* Technical Review Subcommittee: Comprised of personnel from within 
OCIO, including the Chief Technology Officer, the Chief Architect, and 
the Chief Information Security Officer, with a focus on ensuring that 
the technical architecture is aligned with the department's strategic 
goals and monitoring IT projects through conducting control gate 
reviews that assess whether all necessary documentation has been 
produced. The subcommittee is also responsible for approving projects 
that cost less than $500,000. 

HUD Has Begun Applying Key Practices for Its Modernization Efforts but 
Has Been Hindered by Inadequacies in Its Project Management Framework 
and Governance: 

For its FHA Transformation and NGMS modernization efforts, HUD has 
taken initial steps toward applying key project management practices 
in the areas of project planning, requirements management, and 
acquisition planning. However, the department has not yet fully 
implemented any of these practices in managing the 14 projects in our 
review. In large part, these deficiencies can be attributed to 
inadequate development and use of the department's project management 
framework and governance structure. Without fully implementing these 
practices and effectively developing and using its framework and 
governance structure, HUD risks investing its resources on projects 
that may not meet critical mission needs. 

According to the Project Management Institute and the Software 
Engineering Institute, disciplined project management practices call 
for the development of project details such as objectives, scope of 
work, schedules, costs, and requirements against which projects can be 
managed and executed.[Footnote 35] This step can be facilitated by 
developing project artifacts that include, among other things, 
charters to authorize projects and assign responsibility for their 
execution, work breakdown structures to define the work that needs to 
be done to accomplish project objectives, project management plans to 
define how projects are to be executed and controlled, and 
requirements management plans to document the processes and methods to 
be used for developing and managing project requirements. Further, 
developing requirements traceability matrixes that provide linkages 
between business objectives and detailed system requirements, and 
establishing strategies to ensure adequate acquisition planning are 
practices that contribute to effective project management. Our prior 
reviews of federal agencies have shown that applying these practices 
can significantly increase the likelihood of delivering promised 
capabilities on time and within budget.[Footnote 36] 

HUD had taken initial steps in applying key project management 
practices by developing artifacts, to varying degrees, for the 9 FHA 
Transformation and 5 NGMS modernization efforts in our review. 
Nevertheless, the department lacked information needed for managing 
and executing the projects because the documentation developed did not 
contain a number of essential details that best practices stress as 
being critical to effectively defining a project and measuring its 
success. In this regard, none of the documentation included all of the 
critical information that could facilitate effective project 
management, such as full descriptions of the work necessary to 
complete the projects, cost and schedule baselines, or prioritized 
requirements. 

Table 3 summarizes the key project management practices in the areas 
of project planning (charters, work breakdown structures, and project 
management plans), requirements management (requirements management 
plans and traceability matrixes), and acquisition planning 
(acquisition strategies) for the 14 projects that we assessed. In 
addition, appendix III provides our more detailed assessment against 
best practices. 

Table 3: Summary of Key Project Management Practices for the 14 FHA 
Transformation and Next Generation Management System Projects, as of 
April 2013: 

FHA Transformation: 

Project: Capital Needs Assessment; 
Charter: Artifact developed but lacked essential information; 
Work breakdown structure: Artifact not yet developed; 
Project management plan: Artifact not yet developed; 
Requirements management plan: Artifact not yet developed; 
Requirements traceability matrix: Artifact not yet developed; 
Acquisition strategy: Artifact developed but lacked essential 
information. 

Project: Federal Financial Services Platform; 
Charter: Artifact developed but lacked essential information; 
Work breakdown structure: Artifact not yet developed; 
Project management plan: Artifact developed but lacked essential 
information; 
Requirements management plan: Artifact developed but lacked essential 
information; 
Requirements traceability matrix: Artifact developed but lacked 
essential information; 
Acquisition strategy: Artifact developed but lacked essential 
information. 

Project: Healthcare Automated Leader Application Pilot; 
Charter: Artifact developed but lacked essential information; 
Work breakdown structure: Artifact not yet developed; 
Project management plan: Artifact not yet developed; 
Requirements management plan: Artifact developed but lacked essential 
information; 
Requirements traceability matrix: Artifact not yet developed; 
Acquisition strategy: Artifact developed but lacked essential 
information. 

Project: Legacy Application Transformation; 
Charter: Artifact developed but lacked essential information; 
Work breakdown structure: Artifact not yet developed; 
Project management plan: Artifact developed but lacked essential 
information; 
Requirements management plan: Artifact developed but lacked essential 
information; 
Requirements traceability matrix: Artifact developed but lacked 
essential information; 
Acquisition strategy: Artifact developed but lacked essential 
information. 

Project: Leader Electronic Assessment Portal (LEAP) Automation of 
Lender Approval Workflow; 
Charter: Artifact developed but lacked essential information; 
Work breakdown structure: Artifact not yet developed; 
Project management plan: Artifact developed but lacked essential 
information; 
Requirements management plan: Artifact developed but lacked essential 
information;
Requirements traceability matrix: Artifact developed but lacked 
essential information; 
Acquisition strategy: Artifact developed but lacked essential 
information. 

Project: LEAP Institution Manager; 
Charter: Artifact developed but lacked essential information; 
Work breakdown structure: Artifact not yet developed; 
Project management plan: Artifact developed but lacked essential 
information; 
Requirements management plan: Artifact developed but lacked essential 
information; 
Requirements traceability matrix: Artifact developed but lacked 
essential information; 
Acquisition strategy: Artifact developed but lacked essential 
information. 

Project: Multifamily Housing (MFH) Development & Underwriting Business 
Process Reengineering/Automated Underwriting Solution; 
Charter: Artifact developed but lacked essential information; 
Work breakdown structure: Artifact not yet developed; 
Project management plan: Artifact developed but lacked essential 
information; 
Requirements management plan: Artifact developed but lacked essential 
information; 
Requirements traceability matrix: Artifact developed but lacked 
essential information; 
Acquisition strategy: Artifact developed but lacked essential 
information. 

Project: MFH Physical Inspections Alignment; 
Charter: Artifact developed but lacked essential information; 
Work breakdown structure: Artifact not yet developed; 
Project management plan: Artifact not yet developed;
Requirements management plan: Artifact not yet developed; 
Requirements traceability matrix: Artifact not yet developed; 
Acquisition strategy: Artifact developed but lacked essential 
information. 

Project: Portfolio Risk Reporting & Analytics; 
Charter: Artifact developed but lacked essential information; 
Work breakdown structure: Artifact not yet developed; 
Project management plan: Artifact developed but lacked essential 
information; 
Requirements management plan: Artifact developed but lacked essential 
information; 
Requirements traceability matrix: Artifact developed but lacked 
essential information; 
Acquisition strategy: Artifact developed but lacked essential 
information. 

Next Generation Management System: 

Project: Affordable Housing Data Architecture; 
Charter: Artifact not yet developed;
Work breakdown structure: Artifact not yet developed;
Project management plan: Artifact developed but lacked essential 
information; 
Requirements management plan: Artifact developed but lacked essential 
information; 
Requirements traceability matrix: Artifact not yet developed; 
Acquisition strategy: Artifact developed but lacked essential 
information. 

Project: Budget Forecasting and Formulation; 
Charter: Artifact developed but lacked essential information; 
Work breakdown structure: Artifact developed but lacked essential 
information; 
Project management plan: Artifact developed but lacked essential 
information; 
Requirements management plan: Artifact developed but lacked essential 
information; 
Requirements traceability matrix: Artifact developed but lacked 
essential information; 
Acquisition strategy: Artifact developed but lacked essential 
information. 

Project: Cash Management; 
Charter: Artifact not yet developed; 
Work breakdown structure: Artifact not yet developed; 
Project management plan: Artifact developed but lacked essential 
information; 
Requirements management plan: Artifact developed but lacked essential 
information; 
Requirements traceability matrix: Artifact not yet developed; 
Acquisition strategy: Artifact developed but lacked essential 
information. 

Project: Integrated Budget Forecasting Model; 
Charter: Artifact developed but lacked essential information; 
Work breakdown structure: Artifact not yet developed; 
Project management plan: Artifact developed but lacked essential 
information; 
Requirements management plan: Artifact developed but lacked essential 
information; 
Requirements traceability matrix: Artifact developed but lacked 
essential information; 
Acquisition strategy: Artifact developed but lacked essential 
information. 

Project: Portfolio and Risk Management Tool; 
Charter: Artifact developed but lacked essential information; 
Work breakdown structure: Artifact not yet developed; 
Project management plan: Artifact developed but lacked essential 
information; 
Requirements management plan: Artifact developed but lacked essential 
information; 
Requirements traceability matrix: Artifact not yet developed; 
Acquisition strategy: Artifact developed but lacked essential 
information. 

Source: GAO analysis of HUD data. 

[End of table] 

Most of the Projects Had Developed Charters, but They Lacked Clear 
Accountability: 

A project charter formally authorizes a project and identifies high-
level information that constitutes and assigns responsibility for 
project success. According to project management practices, to be 
effective, a charter should include, among other things, a project's 
purpose or justification; high-level information on such factors as 
requirements and risks, measurable objectives, and related success 
criteria; a summary schedule and budget; project approval requirements 
(e.g., information on what factors will define project success and who 
will be responsible for final sign off at the completion of the 
project); and names, responsibilities, and authority levels of 
assigned leadership such as the project manager and sponsor.[Footnote 
37] 

Of the 14 projects in our review, all 9 FHA Transformation and 3 of 
the NGMS projects had developed charters that included most of the 
relevant high-level information. For example, all of the charters 
included such information as the project purpose, description, and 
high-level risks and requirements, as well as the names of the 
assigned project managers. Regarding measurable objectives and success 
criteria, 10 of the projects included objectives, while 6 had related 
success criteria. Lastly, 10 project charters included a summary 
schedule, and 1 included a summary budget. 

While most of the charters contained high-level information, other 
essential details were not included such as the authority levels of 
project leaders and the requirements for approving the completion of 
the projects. Specifically, while each of the charters generally 
referenced HUD's PPM framework and the associated governance 
committees, the charters did not explicitly state what results would 
constitute project success (e.g., a specified number of project 
objectives met) or what individuals or entities would be responsible 
for final sign-off at the completion of the project. For example, FHA 
Transformation's LEAP Institution Manager project charter included the 
project's purpose, high-level risks and requirements, and measurable 
objectives and related criteria. Specifically, the charter noted that, 
by the end of fiscal year 2014, the project would result in the 
retirement of four systems, eliminating the associated costs for 
operations and maintenance. The charter also incorporated a summary 
schedule and the names of its project manager and sponsor. However, 
while the charter included a total budget figure, it did not include 
details regarding the breakdown of the budget provided, the 
responsibilities and levels of authority given to the manager and 
sponsor identified, and the requirements for approving the completion 
of the projects. 

Additionally, the NGMS Integrated Budget Forecasting Model project 
charter provided the project's purpose, high-level risks and 
requirements, the names of the project sponsors and managers, a 
summary schedule, and measurable objectives with related success 
criteria. In particular, regarding the measurable objectives, the 
charter stated that the project would reduce the average time to 
respond to ad hoc requests for budgetary reports and data from 3 days 
to 1 day. However, while the charter referenced the OMB Capital Asset 
Plan and Business Case Summary (Exhibit 300) for a list of associated 
costs, it did not include a summary of the project's expected budget. 
This charter also did not include the responsibilities and the 
authority levels of the sponsors and managers or the project 
completion requirements. 

FHA Transformation and NGMS officials acknowledged the absence of 
these details in the charters and attributed the deficiencies to the 
general immaturity of the department's project management practices. 
Regarding the remaining two NGMS projects for which charters had not 
yet been developed, project officials stated in April 2013, that one 
project was in the process of developing a charter, while relevant 
information for the other project was expected to be incorporated into 
the Budget Forecasting and Formulation charter. 

In the absence of charters that reflect all of the essential elements, 
HUD lacks clear definitions of what will constitute success for its 
modernization projects and has less ability to hold the responsible 
officials accountable for this success. Moreover, the lack of 
important details that a charter is intended to provide at the initial 
authorization of a project makes it more difficult to undertake other 
project planning activities, such as developing work breakdown 
structures, project management plans, and requirements. 

Work Breakdown Structures Had Not Been Fully Developed: 

A work breakdown structure is the cornerstone of every project because 
it defines in detail the work necessary to accomplish a project's 
objectives and provides a basic framework for a variety of related 
activities like estimating costs, developing schedules,[Footnote 38] 
identifying resources, and determining where risks may occur.[Footnote 
39] According to best practices, this artifact should be a deliverable-
oriented[Footnote 40] hierarchical decomposition of the work to be 
executed by the project team to accomplish the project's objectives. 
Moreover, best practices state that it should represent the entire 
scope of the project and product work, including project management, 
and it should be standardized to enable an organization to collect and 
share data among projects. In addition, it should be accompanied by a 
dictionary that describes in brief narrative form what work is to be 
performed in each of the various work breakdown structure elements. 
[Footnote 41] 

Of the 9 FHA Transformation and 5 NGMS projects, none had developed 
complete work breakdown structures and associated dictionaries; only 
one NGMS project--Budget Forecasting and Formulation--had a draft work 
breakdown structure and associated dictionary. However, while this 
draft work breakdown structure included details regarding the first 
increment of the project, neither it nor the associated dictionary 
provided details for any of the future planned increments. Thus, it 
did not reflect the entire scope of the project and lacked 
descriptions of the work that would be performed following the first 
increment, which is expected to deploy initial functionality by late 
summer of 2013. Further, rather than being organized by deliverables--
that is, unique and verifiable products, results, or capabilities--the 
draft was organized by life-cycle phases such as definition and 
design. As a result, it did not allow for progress to be measured by 
deliverable, which would enable more precise identification and 
effective mitigation of the root causes for any cost or schedule 
overruns. Moreover, developing a deliverable-oriented work breakdown 
structure would show how deliverables relate to one another as well as 
to the overall end product. 

According to NGMS officials, plans are under way to fully develop work 
breakdown structures that represent the first and second increments of 
all projects in late spring 2013. Notwithstanding these plans, as of 
April 2013, a specific time frame for developing the work breakdown 
structures and associated dictionaries for the third increment of NGMS 
projects had not yet been determined. 

Further, regarding the 9 FHA Transformation projects, in April 2013 
officials stated that a work breakdown structure and dictionary to 
represent the entire modernization effort is being developed. However, 
the department was not able to provide a specific date for when this 
documentation would be completed. 

NGMS and FHA Transformation officials stated that work breakdown 
structures were not initially developed for the projects because the 
PPM framework did not require the completion of this artifact. The 
officials added that, in addition to HUD project management practices 
lacking maturity, their staff had not yet developed the expertise 
required to create this artifact. 

Until FHA Transformation and NGMS develop deliverable-oriented work 
breakdown structures and associated dictionaries for all of their 
projects, these efforts will lack critical information for 
understanding the detailed work that needs to be performed to 
accomplish project objectives. Further, by not defining the work to be 
performed, HUD cannot provide reasonable assurance that cost and 
schedule estimates will capture all the relevant information needed 
for the management of these efforts. 

Project Management Plans Included Important Details but Lacked 
Baselines and Management Processes: 

According to project management practices, a project management plan 
is the primary source that defines how a project is to be executed and 
controlled.[Footnote 42] Best practices emphasize the importance of 
having such plans in place to, among other things, establish a 
complete description that ties together all project activities and 
evolves over time to continuously reflect the current status and 
desired end point of the project.[Footnote 43] Moreover, these 
practices state that to be effective, a project management plan should 
identify life-cycle processes to be applied, outline plans for project 
tailoring (i.e., determining what processes and documentation would be 
necessary to accomplish project objectives), provide communication 
techniques to be used, and list management reviews. Further, building 
on the initial summary schedule and budget in the charter, the project 
management plan should include baseline cost and schedule estimates 
developed during planning activities. Moreover, this baseline 
information should be updated as needed and periodically compared with 
actual performance data in order to track and report progress. 
Finally, the plan should include, or make reference to, subsidiary 
management plans that describe how subordinate activities are to be 
carried out for the project.[Footnote 44] 

Six FHA Transformation and 5 NGMS[Footnote 45] projects had drafted or 
completed project management plans that outlined life-cycle processes, 
identified communication techniques and management reviews, and 
incorporated certain subsidiary management plans. For example, the 
plan for FHA Transformation's Legacy Application Transformation 
project included an approach for tailoring the life-cycle processes to 
be used; contained a communication table with details about what 
techniques would be used; and described different types of management 
reviews, including an official team review and a structured 
walkthrough. Similarly, the plan for NGMS's Integrated Budget 
Forecasting Model project indicated that the project was following 
HUD's PPM framework, which includes tailoring life-cycle processes; 
contained communication techniques; and identified different types of 
management reviews, including audit reviews and post-project reviews. 

However, the plans provided for the 11 projects lacked other essential 
information. Specifically, they did not clearly identify cost and 
schedule baselines or consistently incorporate subsidiary plans. For 
example, the plan for FHA Transformation's MFH Development & 
Underwriting Business Process Reengineering/Automated Underwriting 
Solution project listed milestones, such as implementing a solution by 
July 2013, and referenced a total cost of ownership artifact that 
indicated a cost of $1.9 million. Yet, the plan did not indicate if 
these were considered to be the project's schedule and cost baselines 
against which progress would be measured. The impact of the lack of 
clear baselines was evidenced by inconsistencies between the project 
management plan and other project documentation. For example, weekly 
status reports indicated that the solution would be implemented by 
November 2013, rather than by the July 2013 date identified in the 
project management plan. This lack of clarity regarding the project's 
cost and schedule baseline makes it difficult to accurately measure 
and report progress against commitments made to deliver functionality. 

In a similar example, the project management plan for NGMS's 
Integrated Budget Forecasting Model provided project milestones and 
identified cost estimates by life-cycle phase, but it did not specify 
if these figures represented cost or schedule baselines developed as 
part of planning activities. Additionally, the plan that reflected the 
other 4 NGMS projects in our study included subsidiary plans for 
requirements, scope, schedule, cost, quality, human resources, and 
risk management, but it did not incorporate necessary details in the 
acquisition strategy[Footnote 46] and lacked one for process 
improvement. 

According to FHA Transformation and NGMS officials, the project 
management plans did not include cost and schedule baselines, in part, 
because the baseline information had been included in the updates 
provided to OMB. However, including a project's cost and schedule 
baseline in a project management plan is important because the plan 
serves as a primary source of information used to execute and manage 
the project. In addition, such baseline information provides managers 
and sponsors the foundational basis for measuring project progress. 
Relying on information reported to an external entity such as OMB 
rather than on cost and schedule baselines to manage projects may not 
allow the project manager to have accurate real-time information 
available when responding to stakeholder interests regarding the 
status of project progress. Regarding the incorporation of subsidiary 
plans, the officials stated that these plans were not required by the 
PPM framework to complete control gate reviews and, as a result, were 
not fully addressed in all of the project plans. Further, for the 
remaining 3 FHA projects that had not yet developed project management 
plans, FHA Transformation officials said the projects were still 
completing initial planning activities. In accordance with the PPM 
framework, the projects would be expected to develop plans when those 
activities are completed. 

Until FHA Transformation and NGMS have comprehensive project 
management plans that reflect cost and schedule baselines and fully 
incorporate subsidiary plans for process improvement and acquisition 
management, these modernization efforts will continue to lack a 
foundational tool needed for successfully managing their projects and 
for providing stakeholders with insight into the status of the 
projects. 

Requirements Management Plans Lacked Details for Prioritizing and 
Measuring Requirements: 

According to project management practices, effective planning of 
requirements includes documenting the processes and methods to be used 
for developing and managing requirements from initial identification 
through implementation. Such practices state that requirements 
management plans should incorporate the approach for how requirements 
development activities (e.g., collecting requirements) will be 
conducted and how changes will be managed; identify methods for 
prioritizing requirements; and specify the metrics to be used to 
measure products against identified requirements, among other things. 
[Footnote 47] As we previously reported, effective planning for 
requirements development and management activities can reduce the risk 
of cost overruns, schedule delays, and limitations in system 
functionality.[Footnote 48] 

Seven of the FHA Transformation and all 5 of the NGMS projects in our 
review had developed requirements management plans that documented how 
requirements development activities would be conducted, including 
managing changes.[Footnote 49] For example, the FHA Transformation's 
Healthcare Automated Lender Application Pilot project plan outlined 
processes for how changes to requirements would be, among other 
things, evaluated upon submission and analyzed for determining their 
impact on original requirements in order for decisions to be made 
regarding proposed changes. In addition, the NGMS Integrated Budget 
Forecasting Model project plan indicated that requirements would be 
gathered through interviews between the contractor and stakeholders, 
and provided detailed policies and procedures for developing and 
maintaining requirements. However, only 1 of the 12 projects--LEAP 
Automation of Lender Approval Workflow[Footnote 50]--identified 
methods for prioritizing requirements, and none of the projects 
established metrics for determining the extent to which the products 
developed addressed requirements. The over 2,400 functional 
requirements identified for the NGMS Budget Forecasting and 
Formulation project illustrates the significance of this point as the 
lack of prioritization could heighten the difficultly developers may 
face in determining which among the many requirements to focus on 
first. 

FHA Transformation and NGMS officials stated that they had followed 
the PPM framework template to develop the requirements management 
plans for their projects. However, they added that the framework did 
not call for prioritization methods to be identified in the 
requirements management plan, and the department's governance 
committee responsible for project oversight did not provide feedback 
to indicate that the plans needed to include this information. As of 
late April 2013, according to FHA Transformation officials, the 
remaining 2 projects had not yet developed requirements management 
plans because the projects were still in initial planning. 

Without establishing methods for prioritizing requirements, the 
department will lack vital information needed to allocate resources in 
a manner which ensures that higher-priority requirements are addressed 
before lower-priority ones. In addition, until metrics for determining 
how products address requirements are established, the department 
lacks the ability to ensure that products will address business needs. 
As a result of these missing details, HUD increases the risk that 
implemented solutions may not effectively support the department's 
mission. 

Requirements Matrixes Provided Limited Traceability: 

According to best practices, the development of a requirements 
traceability matrix is intended to link business needs outlined in 
high-level requirements to more detailed requirements. Traceability 
refers to the ability to follow a requirement from origin to 
implementation and is critical to understanding the interconnections 
and dependencies among the individual requirements and the impact when 
a requirement is changed. Requirement matrixes provide tracing to, 
among other things, business needs and the criteria used to evaluate 
and accept requirements. Further, the use of attributes (e.g., a 
unique identifier, priority level, status, and completion date) in the 
matrix helps define the requirement to facilitate traceability. 
[Footnote 51] As we have reported, establishing and maintaining 
traceability is important for understanding the relationships among 
requirements--from the point at which business requirements are 
initially established through the execution of test cases to validate 
the resulting product.[Footnote 52] 

Six FHA Transformation projects and 2 NGMS projects had developed 
requirements traceability matrixes to track their requirements. 
However, the eight matrixes that had been developed varied in the 
extent to which project requirements linked detailed functional 
requirements backwards to high-level business needs and forward to 
implementation. Further, attributes intended to allow the original 
business needs to be traced to detailed functional requirements were 
incomplete or missing. For example, the FHA Transformation Portfolio 
Risk Reporting & Analytics project matrix supported traceability of 
requirements back to higher-level business goals and also provided 
specific attributes such as unique identifiers. However, the matrix 
did not link the high-level requirements outlined in the project's 
requirements definition documentation to more detailed requirements or 
trace to documentation that described criteria to be used for 
evaluation and acceptance of requirements. 

In addition, the NGMS Integrated Budget Forecasting Model matrix 
included requirements that were traced from high-level to more 
detailed requirements and recorded specific attributes such as a 
unique identifier and the current status. However, the matrix did not 
provide traceability to criteria for evaluating and accepting 
requirements or consistently record accurate information regarding the 
current status of the requirements. For instance, the matrix included 
two identical requirements, but each stated requirement had a 
different disposition: the information for one requirement indicated 
that it had been "completed" while information for the other 
requirement included a notation of "discontinued," but without any 
associated dates to clarify which disposition accurately described 
whether the requirement had been implemented. Further, the NGMS Budget 
Forecasting and Formulation project developed a matrix that used a 
unique identifier that allowed traceability from 15 high-level 
requirements to more detailed functional requirements. In addition, 
these requirements and the traceability matrix were approved by the 
appropriate stakeholders. However, the matrix did not document several 
other attributes, including status, or provide traceability to 
criteria for the evaluation and acceptance of these requirements. In 
particular, the matrix did not establish priorities for requirements 
to aid in ensuring that those of highest priority are addressed first. 

According to FHA Transformation and NGMS officials, the PPM framework 
guidance was used in creating the matrixes and, in many cases, the 
projects relied on contractors to complete the artifacts. In addition, 
according to these officials, project resources were focused on 
providing the documentation required by the framework and associated 
governance committee. As a result, information that was not explicitly 
identified as being required in an artifact, such as matrixes that 
demonstrate traceability, was not developed. With regard to the 
remaining 3 FHA Transformation and 3 NGMS projects, department 
officials said the projects were still completing initial planning 
activities and had not reached a point where requirements had been 
defined to populate a matrix. 

The incomplete state of the requirements traceability matrixes makes 
it unclear what mission needs have been addressed by project 
functional requirements and are planned to be implemented in a 
solution. Without fully traceable requirements for each project, the 
FHA Transformation and NGMS modernization efforts are limited in their 
ability to know whether necessary requirements are being implemented 
or if those being implemented support defined business needs. 

Strategies Lacked Details for Guiding Project Acquisitions: 

Best practices also state that effective IT project management 
involves creating a strategy for acquisition planning.[Footnote 53] 
The strategy should be based on the needs of each individual project 
and can be formal or informal, and highly detailed or broadly framed. 
The strategy should also be incorporated as a subsidiary component of 
the project management plan. An acquisition strategy serves as the 
road map for effective acquisition planning and should document the 
types of contracts to be used, address contract risks, determine dates 
for deliverables, and coordinate contracts with other processes, such 
as scheduling and performance reporting. Additionally, the strategy 
should reflect early identification of metrics to be used in managing 
and evaluating contractors to help ensure that business needs are 
addressed through contracted support.[Footnote 54] 

FHA Transformation and NGMS each developed one acquisition strategy 
that was intended to represent all the projects being undertaken by 
their respective modernization efforts. In addition, while the 
acquisition strategy for NGMS was intended to represent all the 
projects, one project--Integrated Budget Forecasting Model--also 
developed its own individual strategy. These three strategies 
identified the types of contracts (e.g., time and materials, firm-
fixed price, or interagency agreement) that were planned to be awarded 
for their associated projects. For instance, the FHA Transformation 
strategy stated that indefinite-delivery, indefinite-quantity 
contracts would be awarded and associated task orders would be firm-
fixed price, time and materials, or labor hour. The NGMS strategy 
stated that it would utilize existing interagency agreements and work 
with small disadvantaged businesses[Footnote 55] for its contract 
needs. Further, the NGMS Integrated Budget Forecasting Model project's 
separate acquisition strategy identified the type of contract to be 
used (i.e., blanket purchase agreement with firm-fixed price task 
orders), addressed contract risks (e.g., the unavailability of server 
space), and determined dates for deliverables (e.g., create and update 
detailed functional requirements between January 20 and February 10, 
2011). 

Nonetheless, the three strategies did not fully document details 
needed for effective acquisition planning, such as information on how 
risks would be addressed, determining dates for deliverables, 
coordinating with other processes, and identifying metrics needed for 
evaluating contractors. For example, the FHA Transformation 
acquisition strategy identified dates for projects, but did not state 
how contract dates would be coordinated with schedule processes. 
Moreover, both the NGMS strategy and the individual strategy for the 
Integrated Budget Forecasting Model project did not state how other 
project processes, such as requirements development, would be 
coordinated with acquisitions or identify metrics for assessing 
contractors' performance.[Footnote 56] 

FHA Transformation and NGMS officials stated that the strategies 
developed were based on the PPM framework template and that the 
strategies had been approved by the Technical Review Subcommittee, 
which did not identify the deficiencies. Further, while a strategy 
should guide acquisition planning, OCIO officials said the requirement 
in the PPM framework did not call for developing strategies prior to 
awarding contracts. 

Without strategies that guide planning activities in order to ensure 
that acquisitions are managed in accordance with other processes and 
provide performance metrics, the department increases the risk that 
acquisitions associated with its modernization efforts will not be 
effectively managed and that acquired services or products will not 
meet its needs. 

Inadequate Development and Use of HUD's Project Management Framework 
and Governance Contributed to FHA Transformation and NGMS Weaknesses: 

As previously discussed, HUD's project management framework and 
associated governance structure was established to provide policies 
and procedures for managing the department's IT investments. 
Specifically, the framework provides instructions, templates, and 
checklists intended to help ensure important details are incorporated 
for use during the execution and management of project activities. The 
department's governance structure is responsible for ensuring that all 
necessary documentation is produced for all IT projects through 
control gate reviews conducted by the Technical Review Subcommittee. 

Officials responsible for the 9 FHA Transformation and 5 NGMS projects 
in our review stated that they relied on the department's PPM 
framework to implement project management practices and the artifacts 
discussed in this report. However, guidance discussed in the framework 
did not always include essential information called for by best 
practices. For example, the guidance for developing requirements 
management plans did not specifically direct the projects to identify 
methods for prioritizing requirements. In addition, the projects did 
not develop strategies early enough to guide acquisitions because the 
framework did not call for the strategy to be developed until after 
projects completed initial planning activities. 

In other cases, where guidance was provided, FHA Transformation and 
NGMS projects did not always follow the guidance provided or 
adequately implement the tools provided by the framework in developing 
the documentation we examined. This was particularly evident in the 
development of work breakdown structures. For example, the PPM 
guidance included specific details regarding the importance of 
developing work breakdown structures as the basis for defining project 
work and establishing reliable cost and schedule baselines. However, 
as noted earlier, only 1 of the 14 projects in our study had drafted a 
work breakdown structure. Further, the project management plan 
template and guidance call for incorporation of cost and schedule 
baselines and approaches for how those will be managed for any given 
project. However, the project management plans we examined did not 
clearly incorporate such baselines or how they would be managed. 

Compounding the issue of inadequate development and use of the 
framework was the lack of evidence that the department's governance 
bodies had provided adequate oversight to ensure compliance with 
project management practices. In particular, the department's 
Technical Review Subcommittee did not express concerns regarding the 
alignment of FHA Transformation or NGMS documentation with the 
framework and, when issues were raised, the subcommittee had 
nonetheless allowed the projects to proceed. In doing so, the projects 
were able to move to the next control gate review without critical 
information--a practice which could result in projects proceeding for 
months without correcting flaws or inadequacies in information that 
was vital to effective project management. Specifically, in examining 
documentation for control gate reviews, we found that the Technical 
Review Subcommittee did not consistently operate as intended or use 
the guidance provided in the department's framework. While the 
department's framework outlines processes for conducting control gate 
reviews of projects and provides templates to be used, the reviews 
were conducted without using the framework guidance. For example, the 
control gate review procedures state that documentation should be 
assessed based on (1) its accuracy in capturing necessary information 
for the project's development, (2) its completeness with a level of 
detail sufficient to provide correct and relevant information, and (3) 
the adequacy of information in the artifact to make it actionable and 
informative. The framework also provided a decision document intended 
to capture any issues or concerns identified by the subcommittee. 

However, it was not evident that any of the control gate reviews 
conducted from 2011 through 2012 had assessed the documentation 
against the outlined criteria or that the decision document was used. 
For example, during this time, none of the control gate review 
documents provided for FHA Transformation and NGMS included an 
assessment of the documentation against the criteria in the control 
gate review procedures, and meeting minutes or e-mails were used to 
record high-level issues or concerns identified instead of the more 
detailed information called for in the decision document. According to 
responsible OCIO officials, the subcommittee did not assess compliance 
with the framework, but was focused on reviewing the technical aspects 
of IT projects. The officials also noted that the subcommittee did not 
have the staff needed to fully implement the control gate review 
guidance included in the framework, but that it did look to see if the 
identified artifacts were developed for each project. Further, these 
officials stated that it was the responsibility of the project 
managers and their teams to address issues identified before the next 
control gate review, but that the subcommittee did not enforce any 
specific deadlines. Based on our assessment of the control gate review 
documentation, as well as interviews with OCIO and modernization 
effort officials, it was not clear that the subcommittee consistently 
considered its role to include a full assessment of the artifacts for 
compliance with the framework outlined in control gate review guidance 
and templates. 

According to OCIO officials, the initial implementation of the 
framework focused on attempting to get projects to understand basic 
project management, and as a result, the department limited the focus 
of the first version of the framework. In April 2013, the department 
reported that it was working on a revised version of the framework 
that would be released in September 2013. However, the preliminary 
information provided regarding the revisions planned for the framework 
did not incorporate information to address all the deficiencies 
identified by project officials or highlighted in this report. For 
example, draft documents regarding the planned revisions did not 
explicitly state whether work breakdown structures and associate 
dictionaries would be required documentation to serve as the basis of 
cost and schedule baselines. Further, the preliminary information did 
not specify if methods for prioritizing requirements are to be 
incorporated into the requirements guidance. Additionally, as of 
February 2013, the department had assigned new leadership for managing 
the control gate reviews. According to this official, the control 
gates are expected to be revised to ensure that artifacts are 
evaluated and that the subcommittee takes a more active role in 
assessing the application of project management practices. However, 
the department did not state if it would clarify the Technical Review 
Subcommittee's role or associated guidance outlined in the PPM 
framework or identify time lines for implementing the anticipated 
changes. 

Until HUD has a PPM framework for managing its projects that 
incorporates the above mentioned details, including clarifying the 
role of the Technical Review Subcommittee, and is appropriately used 
in managing its modernization efforts, the department increases the 
risk of continuing to inadequately apply project management practices 
and will not be positioned to effectively manage or report progress of 
its modernization efforts. 

Conclusions: 

HUD has taken steps toward applying best practices by establishing a 
framework for standardizing project management, and to varying 
degrees, the FHA Transformation and NGMS modernization efforts have 
developed basic documentation in the areas of project planning, 
requirements management, and acquisition planning. Notwithstanding 
these initial actions, the limited extent to which its modernization 
efforts implemented key practices in these areas puts its projects at 
an increased risk of failure. Specifically, the absence of complete 
information in foundational documentation intended to guide these 
efforts--such as project charters that define project success, 
deliverable-oriented work breakdown structures that detail the work 
needed to be accomplished, project management plans that include cost 
and schedule baselines, requirements management plans that provide 
methods for prioritizing requirements, traceable requirements to 
desired capabilities, and sound acquisition strategies that guide 
planning activities--means that HUD has not taken the steps to fully 
define its modernization efforts in terms of what they will 
accomplish, what steps are necessary to complete them, what they will 
cost, when they will be completed, what specific functionality is 
needed to meet their goals, and how contractors will be held 
accountable for performance. This indicates that, despite the steps 
that have been taken, the maturity of HUD's project management 
practices does not sufficiently position the department to 
successfully carry out these efforts. 

Contributing to these deficiencies is that the department has not 
developed and used its project management framework in a manner that 
ensured the quality or completeness of project management 
documentation. Additionally, the lack of adequate oversight from the 
Technical Review Subcommittee resulted in projects not fully 
understanding how to develop complete artifacts. Until it addresses 
these weaknesses in applying project management practices, HUD may 
continue to invest resources in modernization projects that will not 
satisfy business needs and support its mission. Moreover, fully 
implementing effective project management practices is critical not 
only for the success of these modernization efforts, but also for that 
of the other five IT Transformation Initiatives or any other projects 
under way or undertaken in the future. 

Recommendations for Executive Action: 

To ensure that HUD effectively and efficiently manages its 
modernization efforts aimed at improving its IT environment to support 
mission needs, we recommend that the Secretary of Housing and Urban 
Development direct the Deputy Secretary to establish a plan of action 
that identifies specific time frames for correcting the deficiencies 
highlighted in this report for both its ongoing projects, as 
applicable, and its planned projects, to include: 

* developing charters that define what constitutes project success and 
establish accountability, 

* finalizing deliverable-oriented work breakdown structures and 
associated dictionaries that define the detailed work needed to 
accomplish project objectives, 

* completing comprehensive project management plans that reflect cost 
and schedule baselines and fully incorporate subsidiary management 
plans, 

* establishing requirements management plans that include 
prioritization methods to be applied and metrics for determining how 
products address requirements, 

* completing matrixes to include requirements traceability from 
mission needs through implementation, and: 

* establishing strategies to guide how acquisitions are managed in 
accordance with other processes and that performance metrics are 
established. 

Further, to improve development and use of the department's project 
management framework, we recommend that the Secretary direct: 

* the FHA Transformation and NGMS steering committees to ensure that 
project management expertise needed to apply the guidance outlined in 
the framework is provided to execute and manage their respective 
projects; 

* the Chief Information Officer to ensure that revisions to the 
framework incorporate specific information to address the areas of 
deficiency in project planning, requirements management, and 
acquisition planning identified in this report; and: 

* the Customer Care Committee to review the role and responsibilities 
of the Technical Review Subcommittee and ensure that the department's 
governance structure operates as intended and adequately oversees the 
management of all of its modernization efforts. 

Agency Comments and Our Evaluation: 

We provided a draft of this report to HUD for review. In response, HUD 
provided a letter, signed by the Acting Chief Information Officer, 
which included a chart containing the department's written comments on 
the draft report. In the chart, the department outlined its views 
related to our four recommendations, and provided other comments and 
technical corrections on information in specific sections of the draft 
report, including the background and appendix I, our discussion of the 
findings on the development and use of HUD's project management 
framework, and the report title page. The department's comments are 
reprinted in their entirety in appendix IV. 

In commenting on our recommendations, the department discussed actions 
it was taking on various aspects of the first recommendation, but did 
not state whether or not it concurred with the entirety of the 
recommendation; stated that our conclusion leading to the second 
recommendation did not follow from the premises established in the 
draft report; and concurred with our third and fourth recommendations. 
Summaries of HUD's comments for each recommendation, along with our 
responses, follow. 

* With regard to the first recommendation--which called for the Deputy 
Secretary to establish a plan of action that identifies specific time 
frames for correcting the deficiencies highlighted in this report for 
both of its ongoing projects, as applicable, and its planned projects--
the department noted activities that FHA Transformation expects to 
undertake in addressing the deficiencies for the six specific items 
listed as part of this recommendation. In this regard, the department 
stated that FHA Transformation acknowledged the need to update project 
charters and project management plans, develop deliverable-oriented 
work breakdown structures, examine and correct the requirements 
management plans and traceability matrixes, and work with support 
offices to ensure acquisition planning occurs at the earliest possible 
opportunity in the project's life cycle. The department added that FHA 
Transformation had recognized the need to update its charters and 
project management plans well ahead of our draft report. Nonetheless, 
updated artifacts for FHA Transformation were not provided during our 
review. Moreover, the department did not address whether or how it 
intends to address deficiencies for its ongoing or planned projects, 
including those associated with the NGMS modernization effort. 
Accordingly, we maintain that it is important for HUD to establish a 
plan of action that identifies specific time frames for addressing the 
deficiencies in its IT projects. As acknowledged in the department's 
comments, efforts to improve these project management practices could 
be applied to the other five IT Transformation Initiatives or any 
other projects under way or undertaken in the future. 

* For our second recommendation, which called for the FHA 
Transformation and NGMS steering committees to ensure that project 
management expertise needed to apply the guidance outlined in the 
framework is provided to execute and manage their respective projects, 
the department contended that our conclusion leading to this 
recommendation did not follow from the premises established in the 
report. The department stated that it has ample talent and that 
providing additional talent would likely yield similar results 
regarding its deficiencies until the underlying steps are taken to 
apply effective project management practices. 

We agree that applying effective project management practices is 
important; however, in our view, it is essential for the FHA 
Transformation and NGMS steering committees to ensure that their 
respective modernization efforts have the expertise needed to do so, 
as it pertains to the development of tools such as work breakdown 
structures and requirements traceability matrixes. During our study, 
department officials stated on multiple occasions, that certain 
artifacts and practices were not implemented because staff lacked 
expertise in these areas. For example, both FHA Transformation and NGMS 
officials stated that their staff had not developed the expertise 
required to create work breakdown structures. Similarly, these 
officials stated that projects had relied on contractors to complete 
requirements traceability matrixes. Additionally, as we noted, the 
officials acknowledged that a lack of project management maturity was 
the cause of many of the deficiencies identified. Moreover, in its 
comments on this report, the department stated that staff training 
for the transition to applying the framework was limited. Thus, for 
these reasons, we believe our recommendation is valid and should be 
implemented. 

* The department concurred with our third recommendation that the 
Chief Information Officer ensure that revisions to the framework 
incorporate specific information to address the areas of deficiency in 
project planning, requirements management, and acquisition planning 
identified in this report. 

* In commenting on the fourth recommendation, the department concurred 
with the need for the Customer Care Committee to review the role and 
responsibilities of the Technical Review Subcommittee and ensure that 
the department's governance structure operates as intended and 
adequately oversees the management of its modernization efforts. 

In other comments, the department stated that the discussion of the 
department's project management framework did not recognize the 
difficulty of implementing this framework over the past 2 years. It 
stressed that tremendous effort had been made by the FHA 
Transformation and NGMS modernization efforts toward applying the 
framework while continuing to make progress on their related projects. 
It also stated that time is needed to fully incorporate the framework 
throughout the department on projects other than these modernization 
efforts. 

* Acknowledging that the department has continued to take actions to 
improve its environment, the focus of our work for this report was on 
the implementation of project management practices for FHA 
Transformation and NGMS, specifically. As such, we did not assess the 
difficulties associated with improving the department's overall 
capacity to manage its IT projects. We do agree that there are 
difficulties associated with applying project management practices 
while concurrently undertaking multiple modernization efforts and have 
previously reported on the progress HUD has made in addressing its 
limited capacity to manage and modernize its IT environment.[Footnote 
57] 

* Regarding the title page, HUD commented that modernization efforts 
historically account for a relatively small percentage of IT projects 
at the department, and that a more comprehensive perspective that 
accounts for all IT investments should be considered in the title of 
our report. Our objective for this report was specifically to identify 
the extent to which key project management practices were implemented 
for the FHA Transformation and NGMS modernization efforts. As such, 
this report did not evaluate all of the department's IT investments. 
However, in this report, we do acknowledge the value of HUD applying 
these practices to all of its IT projects and moreover, we plan to 
undertake future work to evaluate the department's 
institutionalization of its IT governance that we anticipate will be 
more comprehensive in assessing the department's management of IT 
investments. 

* Lastly, the department stated that the report should contain 
historical information illustrating the distribution of modernization 
funding in contrast to funding available for the operation and 
maintenance of IT. Toward this end, we assessed all relevant data that 
the department provided to us regarding its IT funding against the 
data that it reported to OMB. However, we found these data to lack 
consistency and concluded they were not sufficiently reliable for 
inclusion in our report. 

With respect to HUD's technical corrections on the draft report, we 
have incorporated revisions, as appropriate. Specifically, in the 
background section and appendix I, we included a footnote to clarify 
that the Office of Program Systems Management is within the Office of 
Housing/FHA Office of Multifamily Housing Programs. We also updated 
the report section that discussed the development and use of HUD's 
project management framework by removing the specific reference to the 
Deputy Chief Information Officer for IT Operations. In this same 
section, the department stated that OCIO did not concur with 
statements attributed to officials from the Technical Review 
Subcommittee. We modified the statements and the attribution in that 
section to represent more specifically what the officials stated. In 
doing so, we also further clarified the activities conducted by 
members of the Technical Review Subcommittee and comments provided by 
officials from the two modernization efforts. 

We are sending copies of this report to interested congressional 
committees. We are also sending copies to the Secretary of the 
Department of Housing and Urban Development and the Director of the 
Office of Management and Budget. Copies of this report will also be 
available at no charge on the GAO website at [hyperlink, 
http://www.gao.gov]. 

If you or your staff have any questions about this report, please 
contact me at (202) 512-6304 or melvinv@gao.gov. Contact points for 
our Offices of Congressional Relations and Public Affairs may be found 
on the last page of this report. GAO staff who made significant 
contributions to this report are acknowledged in appendix IV. 

Signed by: 

Valerie C. Melvin: 
Director, Information Management and Technology Resources Issues: 

[End of section] 

Appendix I: Objective, Scope, and Methodology: 

Our objective was to identify the extent to which the Department of 
Housing and Urban Development (HUD) has implemented key project 
management practices for the Federal Housing Administration 
Transformation Initiative (FHA Transformation) and the Next Generation 
Management System (NGMS) modernization efforts. To address this 
objective, we examined all 14 projects for FHA Transformation and NGMS 
that had been identified in the department's fiscal year 2011 
expenditure plan.[Footnote 58] This included 9 FHA Transformation and 
5 NGMS projects, which are identified in tables 1 and 2 of this report. 

Because HUD recently began implementing project management practices 
for its information technology (IT) modernization projects, we 
reviewed the implementation of practices during the initial phases of 
the projects' life cycles; these practices establish the foundational 
plans and processes for managing projects throughout their life 
cycles. Specifically, we reviewed project planning and management 
practices essential for the success of modernization efforts in three 
areas: project planning, requirements management, and acquisition 
planning. We identified best practices in these areas included in the 
Project Management Institute's (PMI) A Guide to the Project Management 
Body of Knowledge (PMBOK® Guide), the Software Engineering Institute's 
(SEI) Capability Maturity Model® Integration for Development (CMMI-
DEV) and for Acquisition (CMMI-ACQ), and GAO's Cost Estimating and 
Assessment Guide (Cost Guide).[Footnote 59] 

For the 14 projects in our study, we assessed the three project 
management areas by reviewing six relevant documents to determine 
whether they contained essential information called for by best 
practices. Our assessment evaluated to what extent these documents (1) 
were developed and contained essential information, (2) were developed 
but lacked essential information, or (3) had not yet been developed. 
Specifically: 

* To assess project planning activities, we determined whether 
projects had developed project charters, work breakdown structures, 
and project management plans, and when they had, we compared the 
contents of these documents with project management practices in order 
to determine the extent to which critical elements were incorporated 
or executed on the projects. Specifically, we assessed whether project 
charters addressed important elements such as the project purpose or 
justification, the project manager's responsibility and authority 
level, and the name and responsibility of the project sponsor. We 
assessed whether the work breakdown structures were deliverable-
oriented hierarchical decompositions of the work to be executed and 
had associated dictionaries. Finally, we assessed whether project 
management plans addressed important elements such as the project life 
cycle, results of project tailoring, cost and schedule baselines, and 
subsidiary management plans. 

* To assess requirements management, we determined whether projects 
had developed requirements management plans and requirements 
traceability matrixes, and when they had, we compared the contents of 
these documents with best practices in order to determine the extent 
to which each program was applying specific elements. Specifically, we 
assessed whether requirements management plans addressed important 
elements such as configuration management activities, methods used to 
prioritize requirements, metrics, and a traceability structure. In 
addition, we assessed whether requirements identified in matrixes 
were, among other things, traceable to business needs, opportunities, 
goals, and objectives and whether the matrixes included essential 
information such as requirements change requests and status. 

* To assess acquisition planning, we determined whether the 
modernization initiatives had developed acquisition strategies and, 
when they had, compared the contents of these documents with key 
practices to determine the actions HUD is taking to ensure that the 
acquisitions for FHA Transformation and NGMS are planned in accordance 
with best practices and guidance. Specifically, we assessed whether 
acquisition strategies addressed important elements such as the 
established dates for the contract deliverables, and procurement 
metrics. 

We interviewed relevant HUD officials and staff in the FHA 
Transformation and NGMS project offices, including the General Deputy 
Assistant Secretary for Public and Indian Housing, the Director for 
the Office of Program Systems and Management,[Footnote 60] the Deputy 
Director of FHA Transformation, and the NGMS Program Manager. In 
addition, we interviewed officials from the department's Chief 
Procurement Office, including the Deputy Chief Procurement Officer, 
and the Office of the Chief Information Officer, including the Acting 
Deputy Chief Information Officer for Business and IT Modernization, to 
obtain information on how these offices support the work of the two 
modernization efforts. Further, we attended and observed project 
status meetings, and related review sessions conducted by senior 
leadership, including HUD's Deputy Secretary. 

We determined that information provided by the department, such as 
work breakdown structures and requirements traceability matrixes, was 
sufficiently reliable for the purposes of our review. To arrive at 
this assessment, we conducted reliability testing by comparing 
information with statements from relevant department officials to 
identify discrepancies. However, we did not test the quality of 
certain information, such as cost and schedule data provided by the 
program offices. 

We conducted this performance audit from June 2012 to June 2013, in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objective. We believe that 
the evidence obtained provided a reasonable basis for findings and 
conclusions based on our audit objective. 

[End of section] 

Appendix II: Summary of HUD's Transformation Initiative Information 
Technology Modernization Efforts: 

Table 4: HUD's Seven Transformation Initiative IT Modernization 
Efforts: 

Name: Federal Housing Administration (FHA) Transformation; 
Description: Develop and implement a modern financial services 
information technology environment to better manage and mitigate risk 
across FHA's insurance programs for single-family housing, multifamily 
housing, and the insured healthcare portfolio. 

Name: Homeless Emergency Assistance and Rapid Transition to Housing 
and eGrants Management Implementation; 
Description: Reengineer processes and implement an automated system 
for managing grants that will reduce application time, eliminate 
manual paper processes, and increase the transparency of grant 
management information. 

Name: HUD Agency-Wide Place-Based Performance Management System; 
Description: Provide business intelligence and geospatial tools for 
documenting and assessing progress toward achieving strategic goals 
that will enhance transparency, reduce workload, increase employee 
productivity, and improve data quality. 

Name: Human Resources End-to-End; 
Description: Integrate human resources systems and tools to allow for 
automated recruitment and hiring documentation, reduction of manual 
data entry, and accelerated candidate decision making. 

Name: HUD Integrated Acquisition Management System; 
Description: Integrate an acquisition management system that is 
compliant with federal regulations to reduce inefficiencies, time, and 
duplication in the procurement process across office locations to 
expedite services rendered to the public. 

Name: New Core Financial System; 
Description: Modernize and replace financial management systems 
through an outsourced shared services provider. 

Name: Next Generation Management System; 
Description: Reengineer management processes to establish a technical 
infrastructure that will integrate disparate systems and provide 
consistent information in order to support rental housing assistance 
services. 

Source: GAO analysis of HUD data. 

[End of table] 

[End of section] 

Appendix III: Detailed Assessment of FHA Transformation and NGMS 
Project Management Documentation: 

Our assessment of FHA Transformation and NGMS implementation of key 
project management documentation[Footnote 61] in the areas of project 
planning (charters, work breakdown structures, and project management 
plans), requirements management (requirements management plans and 
traceability matrixes) and acquisition planning (acquisition 
strategies) is described below. 

Project Charters: 

Best practices recognize the development of a project charter as an 
integral step in project planning for establishing and maintaining 
project teams. A charter formally authorizes a project and identifies 
high-level information that constitutes and assigns responsibility for 
project success. This is a critical artifact for creating project 
management plans, documenting business needs, and outlining the result 
a project is intended to achieve. Specifically, to be effective, 
project management practices state that a charter should include: 

* the project's purpose or justification; 

* high-level requirements; 

* high-level project description; 

* high-level risks; 

* measurable objectives and related success criteria; 

* summary milestone schedule; 

* summary budget; 

* project approval requirements (e.g., what results in project success 
and who is responsible for final sign off); 

* assigned project manager, responsibility and authority level; and; 

* the name and authority of the sponsor or other person(s) authorizing 
the project charter.[Footnote 62] 

Table 5: Assessment of FHA Transformation and NGMS Project Charters: 

FHA Transformation: 
Name: Capital Needs Assessment; 
Assessment: Developed, but lacks essential information; 
Explanation: Details regarding the project's purpose, high-level 
requirements, high-level description, high-level risks, measurable 
project objectives, a summary schedule, a summary budget, and the 
names of the project sponsors and project managers were included in 
the charter. For example, the ability for any agency using the Capital 
Needs Assessment tool to exercise its own discretion to waive or 
modify the tool requirements for very small properties was included as 
a high-level requirement. However, while the charter included 
measurable project objectives, it did not identify related success 
criteria or include information regarding project approval 
requirements or the roles and responsibilities of the project sponsor 
and manager. For instance, while the charter listed the name and role 
of the two sponsors and two project managers, it did not include the 
responsibilities for these individuals or their levels of authority 
relative to the project. 

Name: Federal Financial Services Platform; Lender Electronic 
Assessment Portal (LEAP) Automation of Lender Approval Workflow; 
Assessment: Developed, but lacks essential information; 
Explanation: The charter included a description of the project's 
purpose, high-level requirements, a high-level project description, 
high-level risks, measurable objectives and related success criteria, 
and the name of the assigned project manager. For example, the charter 
stated that one high-level requirement would be to provide loan 
origination and endorsement. However, the charter did not include a 
summary schedule, a summary budget, project approval requirements, 
assigned project manager responsibilities and authority level, and the 
name and authority level of the project sponsor. For example, the 
charter referenced cost data provided to the Office of Management and 
Budget (OMB) for the overall FHA Transformation Initiative, but did 
not include budget information for this project, such as the breakdown 
of costs for hardware, software, and installation. According to 
project officials, this charter was also considered the charter for 
the LEAP Automation of Lender Approval Workflow project. 

Name: Healthcare Automated Lender Application Pilot; 
Assessment: Developed, but lacks essential information; 
Explanation: The charter listed the project's purpose, high-level 
requirements, a high-level project description, high-level risks, 
measurable project objectives, a summary schedule, and the names of 
the project sponsor and project managers. However, while the charter 
identified measurable project objectives, it did not include related 
success criteria. In addition, the charter included a total budget 
figure, but it did not provide details regarding the breakdown of 
those funds. For example, the charter stated that the total budget 
estimated was $500,000, but this number was not accompanied by a 
breakdown of how those funds would be applied toward the pilot. The 
charter also did not identify project approval requirements or the 
authority of the project sponsor or project manager. 

Name: Legacy Application Transformation; 
Assessment: Developed, but lacks essential information; 
Explanation: The charter contained high-level information regarding 
the project's purpose, requirements, description, and risks, a summary 
schedule, and the names of the project sponsor and the project 
managers. For example, a delay in the procurement of tools and 
services was listed as a high risk that might affect the project 
schedule. However, the charter did not include measurable objectives 
and related success criteria. For example, the charter stated that an 
objective was to implement and deploy a risk and fraud tool that meets 
FHA requirements and provides initial reporting and analysis 
capabilities, but did not include metrics or details for determining 
what specific FHA requirements would be satisfied. While the charter 
also included a total budget figure, it did not provide details 
regarding the breakdown of those funds. Further, the charter did not 
incorporate project approval requirements, the detailed 
responsibilities and authority level of the project managers, or the 
authority level of the project sponsor. 

Name: LEAP Institution Manager; 
Assessment: Developed, but lacks essential information; 
Explanation: The charter included the project's purpose, high-level 
requirements, high-level description, high-level risks, and measurable 
objectives and related criteria. For example, the charter noted that 
by the end of fiscal year 2014 the project would result in the 
retirement of four systems, eliminating the associated cost for 
operations and maintenance. The charter also incorporated a summary 
schedule, and the names of the project sponsor and project manager. 
However, while the charter included a total budget figure, it did not 
include details regarding the breakdown of the total budget provided, 
project approval requirements, detailed responsibilities of the 
project manager, or the authority levels of the project sponsor and 
project manager. 

Name: Multifamily Housing (MFH) Development & Underwriting Business 
Process Reengineering/Automated Underwriting Solution; 
Assessment: Developed, but lacks essential information; 
Explanation: The project's purpose, high-level requirements, 
description, high-level risks, measurable project objectives and 
related success criteria, a summary schedule and its project manager 
and sponsor names, responsibilities, and authority levels were 
included in the charter. For example, the charter stated that the 
technical solution would be implemented by July 31, 2013, and included 
various interim dates. However, while the charter also referenced 
several other documents for the costs associated with the project, our 
review of these documents identified two estimated costs--$6 million 
and $13 million--making the actual budget for the project unclear. The 
charter also did not incorporate project approval requirements. 

Name: MFH Physical Inspections Alignment; 
Assessment: Developed, but lacks essential information; 
Explanation: The charter incorporated details regarding the project's 
purpose, high-level requirements, description, risks, schedule, and 
the names of the project sponsor and project managers. For example, it 
stated that the project would begin March 1, 2012, with final system 
implementation estimated to be 8 months after contract award. However, 
while the charter identified measurable project objectives, it did not 
include related success criteria. In addition, the charter stated that 
the cost estimate was $1.8 million, but it did not provide a summary 
budget detailing the way these funds would be used or describe project 
approval requirements, detailed responsibilities of the project 
managers, or the authority levels of the project managers and the 
project sponsor. 

Name: Portfolio Risk Reporting & Analytics; 
Assessment: Developed, but lacks essential information; 
Explanation: The project's purpose, high-level requirements, 
description, high-level risks, objectives, summary schedule, and the 
names of the project sponsor and the project managers were included in 
the charter. For example, the solution's ability to be run on-demand 
with data refreshed monthly was listed as a high-level requirement. 
However, although the charter included various objectives such as 
providing structured reports that would allow FHA leadership to make 
proactive risk- and policy-based decisions, it did not provide related 
success criteria. For instance, the charter did not state what quality 
level these reports would need to have in order for them to be deemed 
successful in providing essential information for decision making. 
Similarly, the charter included a total budget figure, but it did not 
provide details regarding the breakdown of those funds. In addition, 
the charter did not include project approval requirements or detailed 
responsibilities of the project managers or the authority levels of 
the project sponsor or project managers. 

NGMS: 

Name: Affordable Housing Data Architecture; 
Assessment: Not yet developed; 
Explanation: As of April 2013, this project had not yet completed a 
charter. According to NGMS officials, a charter for this project is 
expected to be developed in late spring. 

Name: Budget Forecasting and Formulation; 
Assessment: Developed, but lacks essential information; 
Explanation: Details regarding the project's purpose, high-level 
requirements, description, high-level risks, measurable objectives 
with related success criteria, schedule, and the names of project 
sponsors and managers were included in the charter. For example, the 
schedule that was included in the charter stated that system 
deployment would be in September 2013, with interim milestones such as 
designing the system in June 2013. However, the charter did not 
include a summary budget, project approval requirements, the 
responsibilities of the project managers, or the authority levels of 
the sponsors and managers. For example, the charter indicated that its 
budget estimate was not yet available. 

Name: Cash Management; 
Assessment: Not yet developed; 
Explanation: As of April 2013, this project had not yet completed a 
charter. According to NGMS officials, a charter for this project is 
expected to be developed in late spring. 

Name: Integrated Budget Forecasting Model; 
Assessment: Developed, but lacks essential information; 
Explanation: The charter provided the project's purpose, high-level 
requirements, high-level description, high-level risks, measurable 
objectives with related success criteria, a summary schedule, and the 
names of the project sponsors and managers. For example, the charter 
stated that the project would reduce the average time to respond to ad 
hoc requests for budgetary reports and data from 3 days to 1 day. 
However, the charter did not include a summary budget, project 
approval requirements, the responsibilities of the managers, or the 
authority levels of the sponsors and managers. For example, it 
referenced OMB's Capital Asset Plan and Business Case Summary (Exhibit 
300) for a list of associated costs, but did not include a summary of 
the project's expected budget. 

Name: Portfolio and Risk Management Tool; 
Assessment: Developed, but lacks essential information; 
Explanation: The project's charter provided the purpose, high-level 
requirements, description, high-level risks, a summary schedule, as 
well as the names of the project sponsor and managers. For example, 
the charter indicated that the purpose of the project would be to 
provide a single access point where users can access, analyze, and 
interpret all major program performance indicators for all Public and 
Indian Housing programs. However, the charter did not include 
measurable objectives and related success criteria, and while a 
summary budget was provided, it did not include costs beyond 
requirements development and contractor development. Further, the 
charter did not discuss project approval requirements, the 
responsibilities of the managers, or the authority levels of the 
sponsor and managers. 

Source: GAO analysis of HUD data. 

[End of table] 

Work Breakdown Structures: 

According to best practices, a work breakdown structure is the 
cornerstone of every project because it defines in detail the work 
necessary to accomplish a project's objectives and provides a basic 
framework for a variety of related activities like estimating costs, 
developing schedules,[Footnote 63] identifying resources, and 
determining where risks may occur. In the context of the work 
breakdown structure, work refers to work products or deliverables that 
are the result of effort and not to the effort itself. Creating a work 
breakdown structure involves subdividing (or decomposing) project 
deliverables and work into smaller, more manageable components (called 
work packages) that can be scheduled, cost estimated, and managed. 
[Footnote 64] According to best practices, the work breakdown 
structure is a deliverable-oriented[Footnote 65] hierarchical 
decomposition of the work to be executed by the project team to 
accomplish the project's objectives and create the required 
deliverables. Further, these practices state that a work breakdown 
structure should also represent the entire scope of the project and 
product work, including project management, and it should be 
standardized to enable an organization to collect and share data among 
projects. In addition, it should be accompanied by a dictionary of the 
various work breakdown structure elements that describes in brief 
narrative form what work is to be performed in each element.[Footnote 
66] 

Table 6: Assessment of FHA Transformation and NGMS Work Breakdown 
Structures: 

FHA Transformation: 

Name: Capital Needs Assessment; Federal Financial Services Platform; 
Healthcare Automated Lender Application Pilot; Legacy Application 
Transformation; LEAP Automation of Lender Approval Workflow; LEAP 
Institution Manager; MFH Development & Underwriting Business Process 
Reengineering/Automated Underwriting Solution; MFH Physical 
Inspections Alignment; Portfolio Risk Reporting & Analytics
Assessment: Not yet developed; 
Explanation: As of April 2013, a work breakdown structure and 
associated dictionary had not been developed for any of the FHA 
Transformation projects. According to FHA Transformation officials, 
plans are under way to create a work breakdown structure for the 
entire modernization effort; however, a specific date for doing so had 
not yet been determined. 

NGMS: 

Name: Affordable Housing Data Architecture; 
Assessment: Not yet developed; 
Explanation: As of April 2013, a work breakdown structure and 
associated dictionary for this project had not yet been developed. 
According to NGMS officials, plans are under way to fully develop a 
work breakdown structure and dictionary that represent the first and 
second increments of this project in late spring. 

Name: Cash Management; Portfolio and Risk Management Tool; 
Assessment: Not yet developed; 
Explanation: In late April 2013, NGMS provided draft schedules as 
evidence of work breakdown structures for these two projects. However, 
neither schedule satisfied critical elements of a work breakdown 
structure. Specifically, they were not deliverable-oriented or fully 
standardized, and did not represent the entire scope of the projects. 
In addition, no associated dictionaries were provided. Accordingly, we 
did not consider the information provided to be equivalent to 
developing work breakdown structures for these projects. 

Name: Budget Forecasting and Formulation; 
Assessment: Developed, but lacks essential information; 
Explanation: In February 2013, NGMS produced a draft work breakdown 
structure and associated dictionary for this project that contained 
details regarding the first increment, but did not provide details for 
the second increment of the project. In addition, rather than being 
organized by deliverable--that is, unique and verifiable products, 
results, or capabilities--it was organized by life-cycle phase 
processes (e.g., requirements development), which is inconsistent with 
best practices. Additionally, the associated dictionary was specific 
to the first increment of the project, and it did not include the 
other increments. 

Name: Integrated Budget Forecasting Model; 
Assessment: Not yet developed; 
Explanation: As of April 2013, a work breakdown structure and 
associated dictionary had not been developed for this project. 
According to NGMS officials, development of a work breakdown structure 
for future functionality of this project would be considered; 
however, plans for doing so had not yet been determined. 

Source: GAO analysis of HUD data. 

[End of table] 

Project Management Plans: 

As we have previously reported, agencies need to develop comprehensive 
project management plans, and best practices emphasize the importance 
of having a plan in place that, among other things, establishes a 
complete description that ties together all activities and evolves 
over time to continuously reflect the current status and desired end 
point of the project.[Footnote 67] According to project management 
practices, a project management plan is the primary source that 
defines, among other things, how the project is to be executed and 
controlled. Project management plans can be either summary level or 
detailed and can be composed of one or more subsidiary plans to 
address elements of project management. To be effective, best 
practices state that a project management plan integrates cost and 
schedule baselines from planning activities, and this baseline 
information should be updated as needed and periodically compared with 
actual performance data in order to track and report progress. While 
the content of a project management plan will vary depending upon the 
type and complexity of a project, it is developed through a series of 
integrated processes and is progressively elaborated by updates during 
the execution and management of a project. Such plans include: 

* identification of the projected life cycle and processes to be 
applied to each phase; 

* results of project tailoring; 

* how the team will execute the work to accomplish project objectives; 

* project cost and schedule baselines; 

* how the team will maintain the integrity of performance measurement 
baselines; 

* the needs and techniques for communicating among stakeholders; 

* key management reviews; 

* a change management plan that documents how changes will be 
monitored and controlled; 

* a configuration management plan to define those items that are 
configurable, those items that require formal change control, and the 
process for controlling change to such items; and: 

* subsidiary management plans (scope, requirements, schedule, cost, 
quality, process improvement, human resources, communication, risk and 
procurement).[Footnote 68] 

Table 7: Assessment of FHA Transformation and NGMS Project Management 
Plans: 

FHA Transformation: 

Name: Capital Needs Assessment; 
Assessment: Not yet developed; 
Explanation: As of April 2013, a project management plan had not yet 
been developed. According to FHA Transformation officials, this 
project is still completing initial planning activities; 
thus, a date for completing its plan had not yet been determined. 

Name: Federal Financial Services Platform; LEAP Automation of Lender 
Approval Workflow; 
Assessment: Developed, but lacks essential information; 
Explanation: The project management plan for these two projects 
identified the life-cycle processes through a project tailoring 
agreement, communication techniques, and management reviews, and 
included a configuration management plan. For example, the plan 
included a matrix that identified several different methods of 
communication, including weekly meetings, project dashboards, and 
monthly status reports. However, the plan did not describe how the 
work would be executed, cost or schedule baselines, or the maintenance 
of performance measurement baselines. Further, the plan only partially 
discussed change management and did not incorporate all subsidiary 
management plans. For example, the plan described approaches for 
subsidiary activities such as requirements, quality, staffing, 
communication, and risk, but it only partially described procurement 
management and did not discuss how scope, cost, schedule, or process 
improvement would be managed. 

Name: Healthcare Automated Lender Application Pilot; 
Assessment: Not yet developed; 
Explanation: As of April 2013, a project management plan had not yet 
been developed for the project. According to FHA officials, this 
project is being considered for incorporation with the MFH Development 
& Underwriting Business Process Reengineering/Automated Underwriting 
Solution project, and as a result, would be included in that project 
management plan. However, a date for doing so had not yet been 
determined. 

Name: Legacy Application Transformation; 
Assessment: Developed, but lacks essential information; 
Explanation: Details regarding project life cycle, tailoring, 
communication techniques, management reviews, and configuration 
management were included in the project management plan. For example, 
the plan indicated that the project was following HUD's PPM framework, 
which requires project teams to tailor life-cycle processes in order 
to appropriately address project needs. However, the plan did not 
discuss how work will be executed or project baselines, and although 
its requirements management plan included change management with 
respect to requirements, the plan did not discuss change management 
for the entire project. In addition, it did not fully incorporate all 
subsidiary management plans, including schedule, cost, human resource, 
process improvement, procurement, or scope management plans. 

Name: LEAP Institution Manager; 
Assessment: Developed, but lacks essential information; 
Explanation: The life-cycle processes identified through a project 
tailoring agreement, communication techniques, and management reviews 
were included in the project management plan. For example, the plan 
included descriptions of four different types of reviews, including a 
self-review, an informal review, a formal review, and a structured 
walkthrough. However, the project management plan did not describe how 
work would be executed, integrate cost or schedule baselines, or 
describe how baselines would be maintained. For example, the plan 
listed milestones and referenced a total cost of ownership artifact, 
but it did not clearly state whether this information constituted the 
project's schedule and cost baselines. In addition, the plan also 
discussed a generic method for measuring costs throughout the project 
life cycle, but it did not discuss the maintenance of the cost or 
schedule baselines. While the project developed a change management 
plan, it was not referenced in the project management plan. Further, 
the project management plan did not provide a detailed description of 
configuration management or fully incorporate all subsidiary 
management plans, including scope, schedule, cost, process 
improvement, and procurement management plans. 

Name: MFH Development & Underwriting Business Process Reengineering/ 
Automated Underwriting Solution; 
Assessment: Developed, but lacks essential information; 
Explanation: Descriptions of the life-cycle processes identified 
through a project tailoring agreement, communication techniques, 
management reviews, and configuration management were included in the 
project management plan. For example, the plan identified different 
types of communication techniques, including status meetings, 
requirements gathering sessions, design validation, and kick-off 
meetings at the start of each phase of the project life cycle. 
However, the plan did not discuss the execution of work, cost or 
schedule baselines, or the maintenance of baselines, or provide a full 
description of change management. For example, the plan listed 
milestones and referenced a total cost of ownership artifact, but it 
did not clearly state whether this information constituted the 
project's schedule and cost baselines, nor did it discuss the 
maintenance of the cost or schedule baselines. In addition, the plan 
did not fully incorporate all subsidiary management plans, including 
scope, schedule, process improvement, procurement, human resource, and 
cost management plans. 

Name: MFH Physical Inspections Alignment; 
Assessment: Not yet developed; 
Explanation: As of April 2013, a project management plan had not yet 
been developed. According to FHA Transformation officials, this 
project is still completing initial planning activities; 
however, a date for completing its plan had not yet been determined. 

Name: Portfolio Risk Reporting & Analytics; 
Assessment: Developed, but lacks essential information; 
Explanation: The project management plan described the life-cycle 
processes identified through a project tailoring agreement, 
communication techniques, management reviews, and configuration 
management. For example, the plan stated that reviews provide a method 
of verifying and evaluating completeness, consistency, conformity, 
objectives, and clarity of work in progress. However, the plan did not 
describe how work would be executed, integrate cost or schedule 
baseline, or discuss how the baselines would be maintained. While the 
project's requirements management plan included change management, the 
project management plan did not cover changes for areas other than 
requirements. In addition, the plan did not fully incorporate all 
subsidiary management plans, including scope, schedule, process 
improvement, procurement, cost, or human resource management plans. 

NGMS: 

Name: Affordable Housing Data Architecture; Budget Forecasting and 
Formulation; Cash Management; 
Assessment: Developed, but lacks essential information; 
Explanation: The NGMS project management plan was developed and 
reflected the processes for these projects. The plan identified 
projected life-cycle phases, communication techniques, management 
reviews, the maintenance of schedule and cost baselines, change 
management, and configuration management, and it included subsidiary 
plans for requirements, scope, schedule, cost, quality, human 
resources, and risk management. For example, the plan stated that a 
schedule baseline approved by the executive project sponsor would be 
reviewed on a regular basis. However, the plan did not discuss the 
execution of work, cost and schedule baselines or the management of 
process improvement, and it did not fully incorporate a subsidiary 
management plan for procurement management. Additionally, while the 
plan noted that project tailoring would be done, the results of the 
tailoring had only been completed for the Budget Forecasting and 
Formulation project. As of April 2013, officials stated that plans 
were under way to complete the tailoring for the remaining projects--
Affordable Housing Data Architecture, Cash Management, and Portfolio 
and Risk Management Tool. 

Name: Integrated Budget Forecasting Model; 
Assessment: Developed, but lacks essential information; 
Explanation: The project management plan described the life-cycle 
processes identified through a project tailoring agreement, management 
reviews and communication techniques, the methodology for change 
requests, and configuration management. For example, the plan 
described post-project and audit reviews. However, the plan did not 
describe the execution of work, identify cost or schedule baselines, 
discuss the maintenance of baselines, and include subsidiary plans for 
scope, schedule, or process improvement, or fully incorporate plans 
for procurement management. While it identified cost estimates by life-
cycle phase, it did not specifically state if the cost and schedule 
figures were project baselines or describe how baselines would be 
maintained. For example, the plan identified milestones for each phase 
of the project life cycle, but it did not specifically identify or 
refer to a schedule baseline. 

Name: Portfolio and Risk Management Tool; 
Assessment: Developed, but lacks essential information; 
Explanation: In addition to the explanation above for the Affordable 
Housing Data Architecture project, this project also developed a plan 
that provided an overview of the project's development approach, 
challenges, benefits, and risks. However, similar to the NGMS project 
management plan, this specific plan did not discuss the maintenance of 
performance measurement baselines, cost and schedule baselines, or the 
management of process improvement. 

Source: GAO analysis of HUD data. 

[End of table] 

Requirements Management Plans: 

According to project management practices, effective planning of 
requirements includes documenting the processes and methods to be used 
for effectively developing and managing requirements from initial 
identification through implementation. A project's success is directly 
influenced by the care taken in capturing and managing requirements. 
Other essential planning activities such as developing a work 
breakdown structure or estimating a project's cost and schedule are 
built upon requirements developed. Best practices state that in 
establishing requirements, project teams should plan requirements 
collection activities such as conducting interviews, focus groups, 
facilitated workshops, or other techniques, including surveys and 
prototypes. Depending on the type of project, the approach for 
managing requirements can vary, but requirements management plans 
document the approach for how requirements development activities will 
be conducted. In particular, a plan includes: 

* how requirements activities (e.g., collecting requirements) will be 
planned, tracked, and reported; 

* configuration management activities such as how changes will be 
initiated, analyzed, and managed; 

* requirements prioritization methods; 

* product metrics that will be used and the rationale for using them; 
and: 

* a traceability structure outlining attributes[Footnote 69] for a 
traceability matrix and identifying what other project documents 
requirements will be traced to.[Footnote 70] 

Table 8: Assessment of FHA Transformation and NGMS Requirements 
Management Plans: 

FHA Transformation: 

Name: Capital Needs Assessment; 
Assessment: Not yet developed; 
Explanation: As of April 2013, a requirements management plan had not 
yet been developed. According to FHA Transformation officials, this 
project is still completing initial planning activities; 
thus, a date for completing its plan had not yet been determined. 

Name: Federal Financial Services Platform; LEAP Automation of Lender 
Approval Workflow; 
Assessment: Developed, but lacks essential information; 
Explanation: The requirements management plan incorporated for these 
projects information about how changes to requirements would be 
managed and provided high-level plans for requirements development. 
For example, the plan explained that a process to address requirements 
change requests had been established and outlined the details needed 
to analyze the impact of the change. However, the plan did not detail 
how requirement activities would be planned, tracked, or reported; 
explain how the team would prioritize them; identify metrics for 
determining whether each requirement has been satisfied by the final 
product; or establish a traceability structure describing which 
requirements attributes would be captured on the matrix. For example, 
the plan included a section on requirements traceability; however, 
that section discusses the need for traceability without providing any 
details about the processes to be applied to the project. In addition, 
while the plan did not explain how requirements would be prioritized, 
the LEAP Automation of Lender Approval Workflow project identified 
methods in its other requirements documentation. 

Name: Healthcare Automated Lender Application Pilot; 
Assessment: Developed, but lacks essential information; 
Explanation: Details about how changes to established requirements 
would be configured and managed were incorporated into the 
requirements management plan. For example, it outlined processes for 
how changes to requirements would be, among other things, evaluated 
upon submission and analyzed for determining their impact on original 
requirements to support decisions about whether to accept or reject 
the proposed change. However, the plan did not outline processes for 
planning, tracking, or reporting requirements activities or include 
details regarding the methods that would be used to prioritize 
requirements or metrics for determining whether each requirement had 
been satisfied by the final product. For example, the plan did not 
outline criteria for establishing metrics or list any specific metrics 
for assessing the implementation of stakeholder requirements. Further, 
the plan did not identify which requirements attributes, such as 
current status or date completed, will be captured on the traceability 
matrix. 

Name: Legacy Application Transformation; 
Assessment: Developed, but lacks essential information; 
Explanation: The plan identified how requirements changes will be 
handled by the team and outlined high-level activities for gathering 
requirements, but it did not provide details regarding how the project 
would plan, track, or report on requirements activities. In addition, 
the plan did not incorporate plans for the methods for prioritizing 
requirements or metrics for measuring the extent to which each 
requirement had been satisfied by the final product. For example, the 
plan did not identify how project requirements would be categorized to 
ensure that required functionality was focused on first and that "nice 
to have" functionality would be added later, if resources were 
available. Further, the plan did not include a description of what 
requirements attributes would be used as the basis of a traceability 
structure for associated matrixes. 

Name: LEAP Institution Manager; 
Assessment: Developed, but lacks essential information; 
Explanation: Processes for configuring and managing changes to 
established requirements were addressed in the requirements management 
plan. Specifically, the plan outlined processes and described forms 
for requesting changes to requirements, provided categories of changes 
requested, as well as methods for tracking the specific status of 
changes from the time they are requested ("open") until they are 
implemented ("closed"). However, it did not provide specificity about 
other requirement management activities and did not establish methods 
for prioritizing requirements or identify metrics that the project 
will use to determine whether the final product addresses established 
requirements. For example, the plan did not identify processes for 
categorizing project requirements to ensure that required 
functionality would be focused on first and "nice to have" 
functionality would be added later, if resources were available. 
Further, a traceability structure describing which requirements 
attributes would be captured on the traceability matrix had not been 
established. 

Name: MFH Development & Underwriting Business Process Reengineering/ 
Automated Underwriting Solution; 
Assessment: Developed, but lacks essential information; 
Explanation: The requirements management plan established processes 
for configuring and managing changes to requirements; however, it did 
not fully incorporate other requirements management techniques 
outlined by best practices. Specifically, the plan incorporated high-
level plans for requirements development activities, but did not 
provide sufficient detail about how requirements would be planned, 
tracked, or reported. In addition, it did not outline processes and 
methods for prioritizing requirements or metrics for determining 
whether each requirement had been satisfied by the final product. For 
example, the plan briefly stated that the team would use software 
tools to capture and prioritize both functional and nonfunctional 
requirements, but did not identify prioritization methods. Further, 
the plan did not establish a traceability structure outlining specific 
requirements attributes to be captured on the traceability matrix. 

Name: MFH Physical Inspections Alignment; 
Assessment: Not yet developed; 
Explanation: As of April 2013, a requirements management plan had not 
yet been developed. According to FHA Transformation officials, this 
project is still completing initial planning activities; 
thus, a date for completing its plan had not yet been determined. 

Name: Portfolio Risk Reporting & Analytics; 
Assessment: Developed, but lacks essential information; 
Explanation: The requirements management plan addressed how 
requirements changes were to be managed. In addition, the plan 
addressed high-level processes for developing requirements but did not 
describe the processes in detail. For example, the plan stated that 
interviews with subject matter experts and business users for each 
functional capability would be completed and stakeholder commitment 
for resulting requirements would be obtained, but it did not identify 
stakeholders or detail the processes for identifying stakeholders to 
participate in requirements development activities. Additionally, the 
plan did not incorporate details about prioritizing requirements or 
measuring the extent to which requirements were addressed. For 
example, the plan briefly stated that the project would use software 
tools to capture and prioritize functional and nonfunctional 
requirements but did not establish methods for prioritizing these 
requirements that would help teams to address those that were most 
important first. Further, the plan did not establish a traceability 
structure that described which requirements attributes would be 
captured on the traceability matrix. 

NGMS: 

Name: Affordable Housing Data Architecture; Budget Forecasting and 
Formulation; Cash Management; Portfolio and Risk Management Tool; 
Assessment: Developed, but lacks essential information; 
Explanation: The NGMS requirements management plan for these projects 
identified how changes to requirements will be managed and addressed, 
at a high level, how requirements activities would be handled. For 
example, the plan described the intent to maintain and update a change 
management log and to collaborate with HUD stakeholders to reach a 
decision regarding changes to requirements. However, the plan did not 
provide details on how requirement development activities would be 
conducted. For example, while the plan states that the project team 
will meet with stakeholders to discuss, among other things, the 
requirements baseline; it did not identify stakeholders or detail the 
processes for identifying stakeholders to participate in requirements 
development activities. Moreover, the plan did not address how the 
team would prioritize requirements or metrics for determining whether 
each requirement had been satisfied by the final product. Further, the 
plan did not establish a traceability structure that described which 
requirements attributes would be captured on the traceability matrix. 

Name: Integrated Budget Forecasting Model; 
Assessment: Developed, but lacks essential information; 
Explanation: Plans for this project outlined high-level processes for 
gathering requirements from stakeholders through interviews between 
the contractor and stakeholders and also detailed policies and 
procedures for the team and stakeholders to use in developing and 
maintaining requirements. However, the plan did not identify specific 
stakeholders that would be involved in developing requirements or the 
processes for determining participants. In addition, the plans did not 
address how the team would prioritize requirements or metrics for 
determining whether each requirement had been satisfied by the final 
product. Lastly, a traceability structure to describe which 
requirements attributes would be captured on the matrix was not 
included. 

Source: GAO analysis of HUD data. 

[End of table] 

Requirements Traceability Matrixes: 

Project management practices state that requirements traceability 
matrixes are designed to support backward traceability by linking each 
requirement to the broader business objective it supports and forward 
traceability by linking these requirements to more detailed functional 
requirements. Traceability refers to the ability to follow a 
requirement from origin to implementation and is critical to 
understanding the interconnections and dependencies among the 
individual requirements and the impact when a requirement is changed. 
This bidirectional traceability can help management determine whether 
the project addresses all requirements and that those requirements and 
the related deliverables are traceable back to valid business needs. 
According to best practices, requirement matrixes provide tracing to: 

* business needs, opportunities, goals and objectives; 

* high-level requirements to more detailed requirements; 

* criteria used for evaluation and acceptance of the requirements; 

* a set of approved requirements; and: 

* status of requirement changes and requests. 

Further, matrixes should have specific attributes recorded for each 
requirement. Attributes associated with each requirement--such as a 
unique identifier, textual description, priority, version, current 
status, and date completed--should be recorded.[Footnote 71] 

Table 9: Assessment of FHA Transformation and NGMS Requirements 
Traceability Matrixes: 

FHA Transformation: 

Name: Capital Needs Assessment; 
Assessment: Not yet developed; 
Explanation: As of April 2013, a requirements traceability matrix had 
not yet been developed. According to FHA Transformation officials, 
this project is still completing initial planning activities; 
thus, a date for developing its requirements had not yet been 
determined. 

Name: Federal Financial Services Platform; LEAP Automation of Lender 
Approval Workflow; 
Assessment: Developed, but lacks essential information; 
Explanation: While these two projects shared a matrix that 
incorporated two high-level requirements, the matrix did not support 
traceability from business goals and objectives to more detailed 
requirements. The matrix also did not provide traceability to criteria 
to be used for evaluation and acceptance of the requirements, approval 
of the requirements, or the status of requirement changes and 
requests. Further, the matrix did not delineate how requirements were 
prioritized (since the two identified were listed as high priority) or 
provide other attributes such as requirement status or completion 
dates. 

Name: Healthcare Automated Lender Application Pilot; 
Assessment: Not yet developed; 
Explanation: As of April 2013, a requirements traceability matrix had 
not yet been developed. According to FHA officials, this project is 
being incorporated into the MFH Development & Underwriting Business 
Process Reengineering/Automated Underwriting Solution project, and as 
a result, requirements developed would be included with that project. 
Thus, a date for doing so had not yet been determined. 

Name: Legacy Application Transformation; 
Assessment: Developed, but lacks essential information; 
Explanation: This project's matrix enabled traceability from business 
objectives to high-level requirements and then to more detailed 
functional requirements. It also provided requirements attributes, 
including unique identifiers and textual descriptions for 
requirements. For example, one requirement had the description that 
stated it should have the ability to monitor performance for assessed 
properties. However, the matrix did not include other attributes such 
as ones to denote the priority for implementing established 
requirements or report substantive information about the status of 
requirements because all were described as "in progress". 

Name: LEAP Institution Manager; 
Assessment: Developed, but lacks essential information; 
Explanation: Although the requirements listed in the matrix were 
traceable to high-level requirements using a unique identifier and 
listed the requirements as either "in progress" or "complete," the 
matrix did not identify the date of completion for the requirements. 
In addition, the matrix did not allow clear traceability back to the 
business goals and needs for the system or link to criteria used for 
the evaluation and acceptance of requirements, or allow for tracking 
change requests associated with established requirements. Further, the 
matrix did not trace to documentation that indicated that the 
requirements in the matrix were approved by stakeholders. 

Name: MFH Development & Underwriting Business Process Reengineering/ 
Automated Underwriting Solution; 
Assessment: Developed, but lacks essential information; 
Explanation: The matrix for this project was traceable to a set of 
approved requirements and captured most attributes including a unique 
identifier, textual descriptions, and the priority and status of 
requirements. For example, the requirement described as "ability to 
accept lender application data electronically via secure Internet 
access" had a unique identifier. However, the project matrix did not 
clearly provide traceability from business goals and objectives to 
high-level requirements, and on to more detailed functional 
requirements. It also lacked traceability to criteria for evaluating 
and accepting requirements and for tracking changes to established 
requirements as well as an attribute for completion dates for the 
requirements listed. 

Name: MFH Physical Inspections Alignment; 
Assessment: Not yet developed; 
Explanation: As of April 2013, a requirements traceability matrix had 
not yet been developed. According to FHA Transformation officials, 
this project is still completing initial planning activities; 
thus, a date for developing its requirements had not yet been 
determined. 

Name: Portfolio Risk Reporting & Analytics; 
Assessment: Developed, but lacks essential information; 
Explanation: The matrix supported traceability of requirements back to 
higher-level business goals and also provided specific attributes such 
as unique identifiers and brief descriptions for requirements. On the 
other hand, it did not link high-level requirements outlined in the 
project's requirements definition documentation to more detailed 
requirements. In addition, it did not provide traceability to criteria 
to be used for evaluation and acceptance of the requirements, approval 
of the requirements, or the status of requirement changes and 
requests. Further, the matrix listed all requirements as medium 
priority (thus providing no prioritization) and did not provide 
completion dates for the requirements documented. 

NGMS: 

Name: Affordable Housing Data Architecture; 
Assessment: Not yet developed; 
Explanation: According to NGMS documentation, this project will begin 
defining its requirements in June 2013. 

Name: Budget Forecasting and Formulation; 
Assessment: Developed, but lacks essential information; 
Explanation: The matrix links requirements to high-level business 
goals and provides traceability from the 15 established high-level 
business requirements to more detailed functional requirements. The 
matrix also captures attributes such as unique identifiers and textual 
descriptions for each requirement. However, the matrix does not allow 
traceability to criteria to be used for evaluation and acceptance of 
the requirements, approval of the requirements, or the status of 
requirement changes and requests. The matrix also lacked other 
attributes such as the status and priority of requirements. 

Name: Cash Management; 
Assessment: Not yet developed; 
Explanation: According to NGMS documentation, this project will finish 
defining its requirements in June 2013. 

Name: Integrated Budget Forecasting Model; 
Assessment: Developed, but lacks essential information; 
Explanation: The matrix provided traceability from project goals and 
high-level business needs to more detailed requirements. It also 
recorded specific attributes for requirements such as a unique 
identifier and a textual description. For example, a requirement that 
had a unique identifier also listed the description as "the model must 
be capable of delivering alerts and report content via email to 
designated internal HUD personnel." However, the matrix did not 
provide traceability to documentation that indicated that requirements 
had been approved or information about changes to the established 
requirements. Further, the matrix did not incorporate other attributes 
for requirements such as the priority of requirements or completion 
dates. Regarding the attribute for the status of requirements, the 
matrix contained conflicting information. Specifically, it included 
two identical requirements, but provided different dispositions for 
each; one indicated it has been "completed" and the other had a 
notation of "discontinued" without any associated dates to clarify 
which disposition was accurate. 

Name: Portfolio and Risk Management Tool; 
Assessment: Not yet developed; 
Explanation: According to NGMS documentation, this project finalized 
its requirements in October 2012; however, as of April 2013, a 
requirements traceability matrix had not yet been developed. 

Source: GAO analysis of HUD data. 

[End of table] 

Acquisition Strategies: 

According to best practices, effective IT project management also 
involves early planning for and management of acquisitions.[Footnote 
72] The planning process begins with the identification of project 
needs which can best be, or must be, met by acquiring products, 
services, or results outside of the organization. During planning, 
coordination of the acquisition with other project management 
activities, such as budgeting, scheduling, resource estimating, risk 
identification, and requirements definition, should be discussed and 
documented. Most organizations have documented policies and procedures 
specifically defining mandatory acquisition activities for obtaining 
contracted goods or services. The acquisition planning process should 
result in a plan or strategy that describes how management decisions 
will be applied for a particular project. Such strategies serve as the 
road map for effectively planning and managing acquisitions from 
initiation through contract closure.[Footnote 73] In particular, 
project management practices indicate that acquisition strategies 
should provide guidance for: 

* defining the types of contracts to be used; 

* addressing risk management issues; 

* coordinating procurement with other project aspects, such as 
scheduling and performance reporting; 

* setting the scheduled dates in contracts for determining dates for 
deliverables and coordinating with other project management processes; 
and: 

* establishing procurement metrics to be used in managing and 
evaluating contractors.[Footnote 74] 

Table 10: Assessment of FHA Transformation and NGMS Acquisition 
Strategies: 

FHA Transformation: 

Name: Capital Needs Assessment; Federal Financial Services Platform; 
Healthcare Automated Lender Application Pilot; Legacy Application 
Transformation; LEAP Automation of Lender Approval Workflow; LEAP 
Institution Manager; MFH Development & Underwriting Business Process 
Reengineering/Automated Underwriting Solution; MFH Physical 
Inspections Alignment; Portfolio Risk Reporting & Analytics
Assessment: Developed, but lacks essential information; 
Explanation: The FHA Transformation acquisition strategy developed was 
intended to include all 9 projects in our study. This strategy 
identified the types of contracts to be used, stating that indefinite-
delivery, indefinite-quantity contracts would be awarded and 
associated task orders would be time and materials, firm-fixed price, 
or labor hour. However, it did not incorporate how risk management 
issues would be handled or describe how the procurement would be 
coordinated with other processes. In addition, although the strategy 
outlined dates for contract deliverables, it did not discuss how these 
dates would be incorporated into schedule development or additional 
control processes. Further, the strategy lacked metrics that would be 
used to manage the contract and evaluate contractors. 

NGMS: 

Name: Affordable Housing Data Architecture; Budget Forecasting and 
Formulation; Cash Management; Portfolio and Risk Management Tool; 
Assessment: Developed, but lacks essential information; 
Explanation: The NGMS acquisition strategy for these projects 
identified the types of contracts to be used, including utilizing 
existing interagency agreements and working with small, disadvantaged 
businesses. However, the plan did not incorporate how risk management 
issues would be handled or describe how the procurement would be 
coordinated with other processes. In addition, it did not discuss 
dates for contract deliverables or how these dates would be 
incorporated into schedule development or additional control 
processes. Lastly, the project lacked metrics for managing the 
contract and evaluating contractors. 

Name: Integrated Budget Forecasting Model; 
Assessment: Developed, but lacks essential information; 
Explanation: The strategy for this project indicated it would use 
blanket purchase agreements with firm-fixed-price task orders as the 
type of contracts. In addition, the strategy identified contract risk 
issues, such as the unavailability of server space and established 
dates for contract deliverables, and indicated how the deliverable 
dates were going to be coordinated with schedule development. However, 
this strategy did not identify how other project management processes, 
such as scheduling and performance management, would be coordinated 
with the procurement or outline procurement metrics intended to 
support management of contracts and evaluation of contractors. 

Source: GAO analysis of HUD data. 

[End of table] 

[End of section] 

Appendix IV: Comments from the Department of Housing and Urban 
Development: 

U.S. Department of Housing and Urban Development: 
Chief Information Officer: 
Washington, DC 20410-3000: 

June 4, 2013: 

Ms. Valerie C. Melvin: 
Director, Information Management and Technology Resources Issues: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Ms. Melvin: 

Thank you for the opportunity to comment on the Government 
Accountability Office (GAO) draft report entitled, Information 
Technology: HUD Needs to Improve Key Project Management Practices for 
Its Modernization Efforts (GAO-13-455). 

The Department of Housing and Urban Development (HUD) reviewed the 
draft report. HUD's comments on the draft report are listed in the 
enclosed chart. 

If you have any questions, please contact Joyce M. Little, Director, 
Office of Investment Strategies Policy and Management, at 
(Joyce.M.Little@hud.gov), or 202-402-7404. 

Sincerely, 

Signed by: 

Barbara A. Elliott: 
Acting Chief Information Officer: 

Enclosure: 

HUD Comments on GAO-13-455 Draft Report: 

Page: 13; 
Section: Background and Appendix I; 
GAO Comment/Issue: Director of the Office of Program Systems Management 
HUD Comment/Issue: Corrected Language: Director, Office of Program 
Systems Management, Office of Multifamily Housing Programs. Should 
also be corrected on page 44. 

Page: 19; 
Section: HUD's IT Project Management Framework and Governance Structure 
GAO Comment/Issue: ... during 2011 HUD established new policies and 
procedures for executing and governing IT investments; 
HUD Comment/Issue: The GAO draft report is factually correct, based on 
the requirement that federal agencies adhere to practices when 
developing IT systems. GAO, within the report, recognizes and notes 
that HUD recently implemented this standard in 2011. However, at the 
time of implementation of the standard, TI/IT funding had already been 
allocated and projects were moving forward. The result is that both 
the FHA Transformation and NGMS projects are in varying stages of 
maturity of adhering to the standard. Consequently, the report lacked 
recognition of the difficulty related to the transition to the PPM 
framework, with limited training that accompanied the transition, and 
the tremendous effort that has been accomplished by these projects to 
move toward that standard while continuing to make progress on TI/IT 
projects. HUD would like to see acknowledgment of these challenges 
addressed in the report and a recognition that it will take time for 
the full complement of project management to be incorporated into the 
overall framework for the Department, not just FHA Transformation and 
NGMS. 

Page: 38; 
Section: Inadequate Development and Use of HUD's Project Management 
Framework; 
GAO Comment/Issue: According to responsible OCIO officials, including 
the Deputy Chief Information Officer for IT Operations, the 
subcommittee did not have the staff needed to fully implement the 
guidance; 
HUD Comment/Issue: Identification of a specific person/title within 
the OCIO or other Program Office within the Department is inconsistent 
with the rest of the draft report content. Consequently, OCIO requests 
that this information (including the Deputy Chief Information Officer 
for IT Operations) be removed. 

Page: 38; 
Section: Inadequate Development and Use of HUD's Project Management 
Framework; 
GAO Comment/Issue: Further, according to these officials, it was not 
clear that the subcommittee's role was to fully assess the artifacts 
and, in their opinion, this function was the responsibility of project 
managers and their teams; 
HUD Comment/Issue: OCIO does not concur with this statement. The OCIO 
officials interviewed on the TRC did discuss the process for fully 
assessing the artifacts and for providing all comments/concerns from 
the assessment to the project manager and integrated project team 
members to address accordingly. The TRC expected/stated that any 
issued identified would be addressed before the projects' next control 
gate. 

Page: 40; 
Section: Recommendations; 
GAO Comment/Issue: ...developing charters that define what constitutes 
project success and establish accountability; 
HUD Comment/Issue: FHA recognized the need to update project charters 
for all active projects well ahead of the release of this draft report 
and the work is in progress. We also acknowledge the likelihood we 
will need to reconcile the updated charters once Project Planning and 
Management (PPM) version 2.0 is released. It is important to note that 
FHA and NGMS submitted project charters in accordance with PPM version 
1.0 templates and while the current template provisions for 
accountability there are no such provisions or requirements to 
characterize what 
constitutes project success or alternatively a project kill-point. 

Page: 40; 
Section: Recommendations; 
GAO Comment/Issue: ...finalizing deliverable-oriented work breakdown 
structures and associated dictionaries that define the detailed work 
needed to accomplish project objectives; 
HUD Comment/Issue: FHA is constructively examining this best practice 
and where applicable (and practical) will develop deliverable oriented 
work breakdown structures and associated dictionaries for projects in 
the execution phase of the PPM life cycle. FHA's goal, consistent with 
the release of PPM 2.0, is to achieve full integration of this best 
practice as new projects (modernization and otherwise) matriculate 
into the initiation and planning project management phases. The 
explicit need for WBS and WBS dictionaries was likely obfuscated 
because it was not a required deliverable to the Technical Review 
Committee or to transition from one control gate to the next. It is 
important to note that project schedules were developed to support each 
project and many of the elements found in these schedules would 
translate directly vis-a-vis a WBS. Additionally FHA is working with 
OCIO to identify and modernize enterprise rated tools to support 
project planning. The desired tool would have an integrated and fully 
functional WBS modeler and WBS dictionary builder. It is irresponsible 
to waste time engaged in manual production of these artifacts. 

Page: 40; 
Section: Recommendations; 
GAO Comment/Issue: ...completing comprehensive project management 
plans that reflect cost and schedule baselines and fully incorporate 
subsidiary management plans; 
HUD Comment/Issue: FHA recognized the need to update all project 
management plans for all active projects well ahead of the release of 
this draft report and the work is in progress. This effort is 
associated with project charter updates and a critical review of all 
artifacts either submitted or in the pipeline is underway. We also 
acknowledge the likelihood we will need to reconcile the updated 
charters once Project Planning and Management (PPM) version 2.0 is 
released. We plan to work more closely with support offices (e.g. 
OCIO, OCFO, and OCFO) to ensure PPM version 2.0 all project management 
templates meet the burden sufficiency (i.e. cost, acquisition, etc.) 
needed to enable effective project management support. Here again it 
is important to note that FHA submitted project management plans in 
accordance with PPM version 1.0 templates. The project management and 
associated subsidiary management plans (e.g. risk, quality assurance, 
etc.) were submitted as part of the definition phase. The 
documentation burden was very comprehensive and deemed sufficient for 
applicable projects to transition from definition to design phase. 

Page: 40; 
Section: Recommendations; 
GAO Comment/Issue: ...establishing requirements management plans that 
include prioritization methods to be applied and metrics for 
determining how products address requirements; 
HUD Comment/Issue: FHA will constructively examine the current and 
proposed PPM (Le. version 1.0 and 2.0) templates that support 
requirements management planning to determine sufficiency of 
prioritization methods and metrics. We will work with OCIO to correct 
any disconnects in the templates and deficiencies in terms of 
application as we go forward. In the broader scheme of things the 
organization needs to consider a more refined enterprise capital 
investment decision model. Such an instrument would provide a mix-
methods (Le. quantitative and qualitative factors, weighted criteria, 
etc.) approach to project selection linked directly back to mission 
requirements and provide the means to: 1) defend the departments IT 
capital investments, 2) align investments with strategic imperatives 
and funding forecast, 3) optimize the impact of scarce IT investment 
dollars across the department, 4) stratify project focus, and 5) 
articulate the actual requirements for capital investment funding in a 
valid and reliable manner. 

Page: 40; 
Section: Recommendations; 
GAO Comment/Issue: ...completing matrixes to include requirements 
traceability from mission needs through implementation; 
HUD Comment/Issue: FHA concurs that well defined business requirements 
in-turn technical specifications supported by traceability matrixes 
are essential to ensuring the validity between what the authorized 
requested and authorized supplier supplied. HUD will constructively 
examine the requirement traceability artifact(s) and work aggressively 
to correct for "in-flight" projects and reconcile with PPM 2.0 for 
planned projects at the appropriate time. 

Page: 40; 
Section: Recommendations; 
GAO Comment/Issue: ...establishing strategies to guide how acquisition 
are managed in accordance with other processes and that performance 
metrics are established; 
HUD Comment/Issue: FHA concurs that integrated acquisition and budget 
planning should occur at the earliest possible opportunity in the 
project life cycle. We look forward to collaboratively working with 
the support (e.g. OCIO, OCFO, and OCPO) offices to ensure such a 
framework is implemented or developed and to ensure PPM version 2.0 
reflects integrated acquisition and budget templates that meet (or 
exceed) the burden of sufficiency (i.e. valid and reliable strategy, 
performance metrics, etc.) needed to support life acquisition of 
projects. 

Page: 40; 
Section: Recommendations; 
GAO Comment/Issue: ...the Chief Information Officer to ensure that 
revisions to the framework incorporate specific information to address 
the areas of deficiency in project planning, requirements management, 
and acquisition planning identified in this report; 
HUD Comment/Issue: The Chief Information Officer concurs with the 
recommendation. The OCIO will continue to work collaboratively with 
the support offices (e.g. OCFO and OCPO), to equip FHA and NGMS with 
tools to meet the standards set by the PPM version 2.0. 

Page: 40; 
Section: Recommendations; 
GAO Comment/Issue: ...the FHA Transformation and NGMS steering 
committee to ensure project management expertise needed to apply the 
guidance outlined in the framework is provided to execute and manage 
their respective projects; 
HUD Comment/Issue: HUD contends this inference or conclusion does not 
follow from the premises established earlier in this report. The 
provision of additional talent would result in a "marginal 
benefit"...at best. HUD in general and FHA and NGMS specifically have 
ample talent. The chemistry, integration and organizational alignment 
of that talent could be the subject of rigorous debate. The 
application of additional talent to "apply the guidance outlined in 
the framework" would likely yield similar results until underlying and 
foundational determinants of effective project management are 
definitively addressed. The salient point is we must adjust our focus 
to one that is preemptively focused and recognizes the inherent risk 
and challenges associated with any project up front. 

Page: 40; 
Section: Recommendations; 
GAO Comment/Issue: ...the Customer Care Committee to review the role 
and responsibilities of Technical Review Subcommittee and ensure that 
the department's governance structure operates as intended and 
adequately oversees the management of its modernization efforts; 
HUD Comment/Issue: HUD concurs with the recommendation. The OCIO will 
take action as appropriate to ensure the Customer Care Committee will 
be more engaged in the governance of the Technical Review Subcommittee 
going forward. 

Page: NA; 
Section: Title; 
GAO Comment/Issue: None; 
HUD Comment/Issue: Title Page: Title language should be more 
comprehensive and inclusive of the IT project management discipline at 
HUD. While FHA understands the scope (and perhaps spirit and intent) 
of this engagement, to delimit the need just to modernization efforts 
is counterproductive (even misleading) when modernization historically 
accounts for a relatively small percentage of IT projects at HUD. A 
recommended title is HUD Needs to Improve Key Project Management 
Practices for Its 
Portfolio of IT Investments. 

Page: NA; 
Section: None; 
GAO Comment/Issue: None; 
HUD Comment/Issue: Figure 2: Distribution of HUD's IT Budget for 
Fiscal Years 2005 to 2012 (reference: 310990 HUD IT Project Management 
Fact Sheet Part 1 of 2) should have been retained as part of the final 
draft report and ultimately the final report. Perhaps an even more 
longitudinal chart should be presented to dexterously characterize the 
episodic (Le. irregular) nature of modernization versus one that is 
stable thus more predictable. The salient point is even in years when 
funding is deemed sufficient it won't compensate for the likely 
correlation between episodic modernization investments and challenges 
experienced with effective project management of those investments. 

[End of section] 

Appendix V: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Valerie C. Melvin, (202) 512-6304 or melvinv@gao.gov: 

Staff Acknowledgments: 

In addition to the contact above, Teresa M. Neven (Assistant 
Director), Kami J. Corbett, Amanda C. Gill, Lee A. McCracken, John 
Ockay, and Shannin G. O'Neill made significant contributions to this 
report. 

[End of section] 

Footnotes: 

[1] HUD, Fiscal Year 2012 Budget Justifications for Estimates: Working 
Capital Fund (WCF) (Washington, D.C.: Feb. 14, 2011). 

[2] GAO, Information Technology: HUD Needs to Strengthen Its Capacity 
to Manage and Modernize Its Environment, [hyperlink, 
http://www.gao.gov/products/GAO-09-675] (Washington, D.C.: July 31, 
2009); HUD Management: HUD's High-Risk Program Areas and Management 
Challenges, [hyperlink, http://www.gao.gov/products/GAO-02-869T] 
(Washington, D.C.: July 24, 2002); HUD Management: Progress Made on 
Management Reforms, but Challenges Remain, [hyperlink, 
http://www.gao.gov/products/GAO-02-45] (Washington, D.C.: Oct. 31, 
2001); Single-Family Housing: Current Information Systems Do Not Fully 
Support the Business Processes at HUD's Homeownership Centers, 
[hyperlink, http://www.gao.gov/products/GAO-02-44] (Washington, D.C.: 
Oct. 24, 2001); HUD Information Systems: Immature Software Acquisition 
Capability Increase Project Risks, [hyperlink, 
http://www.gao.gov/products/GAO-01-962] (Washington, D.C.: Sept. 14, 
2001); and HUD, HUD Transformation Initiatives IT Expenditure Plan 
(Washington, D.C.: Feb. 2011), and U.S. Department of Housing and 
Urban Development: IT Current State Assessment (January 2011). 

[3] HUD, U.S. Department of Housing and Urban Development: IT Current 
State Assessment (January 2011). 

[4] Project management is the application of knowledge, skills, tools, 
and techniques to project activities to meet project requirements. A 
project is a temporary endeavor undertaken to create a unique product, 
service, or result. 

[5] The Transformation Initiative represents a strategy for reforming 
the way the department does business and comprises four components: 
(1) research, evaluation, and program metrics; (2) program 
demonstrations; (3) technical assistance and capacity building; and 
(4) information technology. Funding for this initiative comes from the 
department's authority to set aside up to 1 percent from specified 
accounts. 

[6] The Consolidated Appropriations Act, 2010, Pub. L. No. 111-117 
(Dec. 16, 2009), and Department of Defense and Full-Year Continuing 
Appropriations Act, 2011, Pub. L. No. 112-10, § 2259, 125 Stat. 38, 
197-98 (Apr. 15, 2011) required HUD to submit expenditure plans that 
satisfied sets of statutory conditions before portions of the 
Transformation Initiative funds were made available for IT 
modernization. Until the plans were submitted, HUD IT modernization 
expenditures were limited to 25 percent of the $122.5 million for 
fiscal year 2010 and 35 percent of the $114.1 million for fiscal year 
2011. Additionally, Transformation Initiative appropriations are made 
available for obligation by the department through fiscal years 2012 
and 2013, respectively. Among other statutory conditions, the plans 
were to identify functional and performance capabilities to be 
delivered, expected mission benefits, estimated life-cycle costs, and 
planned key milestones. 

[7] GAO, Information Technology: HUD's Fiscal Year 2011 Expenditure 
Plan Satisfies Statutory Conditions, [hyperlink, 
http://www.gao.gov/products/GAO-12-654] (Washington, D.C.: May 24, 
2012); Information Technology: HUD's Expenditure Plan Satisfies 
Statutory Conditions, and Implementation of Management Controls Is 
Under Way, [hyperlink, http://www.gao.gov/products/GAO-11-762] 
(Washington, D.C.: Sept. 7, 2011); and Information Technology: HUD 
Needs to Better Define Commitments and Disclose Risks for 
Modernization Projects in Future Expenditure Plans, [hyperlink, 
http://www.gao.gov/products/GAO-11-72] (Washington, D.C.: Nov. 23, 
2010). 

[8] This mandate is contained in the Senate Appropriations Committee 
report, S. Rep. No. 112-83, at 141-42 (2011), as approved by the 
conference committee in the Explanatory Statement, H.R. Conf. Rep. No. 
112-284, at 286 (2011), accompanying the Consolidated and Further 
Continuing Appropriations Act, 2012, Pub. L. No. 112-55, 125 Stat. 
552, 691-92 (Nov. 18, 2011). 

[9] Project Management Institute: A Guide to the Project Management 
Body of Knowledge (PMBOK® Guide), Fourth Edition, Project Management 
Institute, Inc. (PMI), 2008. Copyright and all rights reserved. 
Material from this publication has been reproduced with the permission 
of PMI; Software Engineering Institute/Carnegie Mellon, Capability 
Maturity ModeI® Integration (CMMI®) for Development, Version 1.3, 
CMU/SEI-2010-TR-033 (Hanscomb AFB, Massachusetts: November 2010) and 
CMMI® for Acquisition, Version 1.3, CMU/SEI-2010-TR-032 (Pittsburgh, 
Pa.: November 2010); GAO, Best Practices for Project Schedules--
Exposure Draft, [hyperlink, http://www.gao.gov/products/GAO-12-120G] 
(Washington, D.C.: May 30, 2012) and Cost Estimating and Assessment 
Guide: Best Practices for Developing and Managing Capital Program 
Costs, [hyperlink, http://www.gao.gov/products/GAO-09-3SP] (Washington 
D.C.: Mar. 2, 2009). 

[10] These three areas were based on HUD's recent implementation of 
project management practices and the important role planning 
activities have on executing and managing IT projects. 

[11] SEI, CMMI® for Acquisition and CMMI® for Development; PMI, PMBOK® 
Guide (see footnote 9); [hyperlink, 
http://www.gao.gov/products/GAO-12-120G]; and [hyperlink, 
http://www.gao.gov/products/GAO-09-3SP]. 

[12] The other three components are Fair Housing and Equal 
Opportunity, Community Planning and Development, and Government 
National Mortgage Association (also known as Ginnie Mae). 

[13] In 2009 HUD developed a plan to reorganize its OCIO to better 
align with departmental needs. In January 2013 the department received 
final approval and is expecting to implement the new structure by 
September 2013. 

[14] HUD, Fiscal Year 2013 Target Enterprise Architecture Version 7.0 
(November 2012); FY2012 Information Technology Strategic Portfolio 
Review, Version 1.1 (Washington, D.C.: May 2012); U.S. Department of 
Housing and Urban Development: IT Current State Assessment (January 
2011); HUD Transformation Initiatives IT Expenditure Plans, submitted 
April 2010, September 2010, February 2011, and December 2011; and 
Strategic Portfolio Review FY 2009 (Washington, D.C.: June 2008). 

[15] HUD, U.S. Department of Housing and Urban Development: IT Current 
State Assessment (January 2011). 

[16] See app. II for descriptions of HUD's seven IT modernization 
efforts. 

[17] HUD, HUD Transformation Initiatives Information Technology Fiscal 
Year 2011 Expenditure Plan (Washington, D.C.: December 2011). 

[18] This environment is expected to provide case management for the 
life cycle of a loan and capture data from the loan origination and 
underwriting processes. 

[19] HUD, FHA Office of Housing Information Technology Strategy and 
Improvement Plan, (Aug. 13, 2009). 

[20] Counterparties are the persons or institutions engaging in a 
transaction. HUD uses the term counterparty to refer to lenders and 
other participants in its programs. 

[21] The Office of Program Systems Management is responsible for 
developing and enhancing the automated systems that support 
multifamily programs and for coordinating funding for multifamily 
systems contracts. This office is within the Office of Housing/FHA 
Office of Multifamily Housing Programs, which is responsible for the 
overall management, development, direction, and administration of the 
department's multifamily housing programs. 

[22] FHA Transformation's tenth project--Loan Review--had not been 
identified as an active project at the start of our review, nor was it 
identified in the fiscal year 2011 expenditure plan. Thus, this 
project was not assessed in our study. 

[23] HUD has been working toward reengineering business models and 
processes for its affordable housing assistance programs since 2004. 
Specifically, the department established initial requirements and a 
proposal for reengineering business models and processes across HUD's 
rental housing assistance programs; however, the modernization effort 
was deferred due to competing priorities. After a feasibility study 
completed in November 2009, department officials decided to define 
requirements before developing a system. A business process 
reengineering effort completed in December 2011 identified four 
functional areas to be further explored through prototypes. 

[24] The prototypes included the Financial Management, Portfolio 
Management, and Dashboard Releases and Online Recertification. 

[25] HUD, Exhibit 300 Budget Year 2014: Next Generation Management 
System (Washington, D.C.: April 29, 2013). Exhibit 300s are business 
cases submitted to the Office of Management and Budget as part of the 
budget process each fiscal year. 

[26] When the department determined that the developed NGMS prototypes 
would not address business needs, we adjusted our study to include 
Affordable Housing Data Architecture and Portfolio and Risk Management 
Tool, which were two additional projects that replaced the prototype 
efforts. Another project for NGMS--HUDCAPS Migration--had not been 
identified as an active project at the start of our review, nor was it 
identified in the department's fiscal year 2011 expenditure plan. 
Thus, this project was not assessed in our study. 

[27] GAO, Information Technology: FDA Needs to Fully Implement Key 
Management Practices to Lessen Modernization Risks, [hyperlink, 
http://www.gao.gov/products/GAO-12-346] (Washington, D.C.: Mar. 15, 
2012); USDA Systems Modernization: Management and Oversight 
Improvements Are Needed, [hyperlink, 
http://www.gao.gov/products/GAO-11-586] (Washington, D.C.: July 20, 
2011); and Office of Personnel Management: Retirement Modernization 
Planning and Management Shortcomings Need to Be Addressed, [hyperlink, 
http://www.gao.gov/products/GAO-09-529 (Washington, D.C.: Apr. 21, 
2009). 

[28] PMI, Pulse of the Profession™: The High Cost of Low Performance 
(Newtown Square, Pa.: March 2013). 

[29] SEI, CMMI® for Development; CMMI® for Acquisition; PMI, PMBOK® 
Guide (see footnote 9); [hyperlink, 
http://www.gao.gov/products/GAO-12-120G]; [hyperlink, 
http://www.gao.gov/products/GAO-09-3SP]; and [hyperlink, 
http://www.gao.gov/products/GAO-04-394G]. 

[30] For the purposes of this report, we use the term "acquisition 
planning" to refer to activities for planning contracts, including 
practices described in the PMBOK® Guide as procurement management (see 
footnote 9). 

[31] OMB, Memorandum for Chief Information Officers: Information 
Technology Investment Baseline Management Policy, M-10-27 (Washington, 
D.C.: June 28, 2010); GAO, Information Technology: OMB Has Made 
Improvements to Its Dashboard, but Further Work Is Needed by Agencies 
and OMB to Ensure Data Accuracy, [hyperlink, 
http://www.gao.gov/products/GAO-11-262] (Washington, D.C.: Mar. 15, 
2011); Information Technology: OMB and Agencies Need to Improve 
Planning, Management, and Oversight of Projects Totaling Billions of 
Dollars, [hyperlink, http://www.gao.gov/products/GAO-08-1051T] 
(Washington, D.C.: July 31, 2008); and [hyperlink, 
http://www.gao.gov/products/GAO-04-394G]. 

[32] HUD's website provides an overview of the framework and links to 
documents used to manage IT projects. See [hyperlink, 
http://portal.hud.gov/hudportal/HUD/program_offices/cio/ppm], accessed 
May 7, 2013. These policies include the following: HUD, Policy for 
Information Technology Capital Management, Handbook 3420.1, Version 
1.1 (Washington, D.C.: July 14, 2011); Policy for Information 
Technology Governance, Handbook 3415.1, Version 1.0. (Washington, 
D.C.: July 14, 2011); Policy for Information Technology Strategic 
Planning, Handbook 3425.1, Version 1.1 (Washington, D.C.: July 14, 
2011); and Policy for Information Technology Management, Handbook 
3400.1, Version 1.0 (Washington, D.C.: Apr. 22, 2011). 

[33] According to HUD's fiscal year 2011 expenditure plan, this 
framework was developed using the PMI PMBOK® Guide's structure and 
processes. The PMBOK® Guide standards include processes, tools, and 
techniques that may increase the likelihood of managing a project 
toward a successful outcome (see footnote 9). 

[34] HUD, HUD Policy for Information Technology Governance, Handbook 
3415.1, Version 1.0 (Washington, D.C.: July 14, 2011). 

[35] SEI, CMMI® for Acquisition and CMMI® for Development; and PMI, 
PMBOK® Guide (see footnote 9). 

[36] GAO, [hyperlink, http://www.gao.gov/products/GAO-11-586]; 
Information Technology: Opportunities Exist to Improve Management of 
DOD's Electronic Health Record Initiative, [hyperlink, 
http://www.gao.gov/products/GAO-11-50] (Washington, D.C.: Oct. 6, 
2010); [hyperlink, http://www.gao.gov/products/GAO-09-529]; Business 
Systems Modernization: IRS Needs to Complete Recent Efforts to Develop 
Policies and Procedures to Guide Requirements Development and 
Management, [hyperlink, http://www.gao.gov/products/GAO-06-310] 
(Washington, D.C.: Mar. 20, 2006); and [hyperlink, 
http://www.gao.gov/products/GAO-04-394G]. 

[37] SEI, CMMI® for Acquisition and PMI, PMBOK® Guide (see footnote 9). 

[38] A work breakdown structure should also be used as the outline of 
the integrated master schedule, which includes all activities 
necessary to complete a program. 

[39] [hyperlink, http://www.gao.gov/products/GAO-09-3SP]. 

[40] A deliverable is any unique and verifiable product, result, or 
capability to perform a service that must be produced to complete a 
process, phase, or project. Deliverables are produced as outputs from 
processes performed to accomplish the project work planned and 
scheduled in the project management plan. Deliverable-or product-
oriented means that a work breakdown structure's content should be 
focused on the deliverables or products and not on the processes. 

[41] PMI, PMBOK® Guide (see footnote 9) and [hyperlink, 
http://www.gao.gov/products/GAO-09-3SP]. 

[42] SEI, CMMI® for Acquisition and PMI, PMBOK® Guide (see footnote 9). 

[43] [hyperlink, http://www.gao.gov/products/GAO-11-50]; [hyperlink, 
http://www.gao.gov/products/GAO-09-529]; and [hyperlink, 
http://www.gao.gov/products/GAO-04-394G]. 

[44] SEI, CMMI® for Acquisition and PMI, PMBOK® Guide (see footnote 
9). According to these best practices, baselines are approved plans 
for a project that are compared to actual performance to determine if 
performance is within acceptable variance thresholds. Specific 
baselines include cost and schedule baselines. Subsidiary management 
plans include, but are not limited to, scope, requirements, schedule, 
cost, quality, process improvement, human resource, communications, 
risk, and acquisition management plans. 

[45] NGMS developed one project management plan that reflected four of 
the five projects in our study. This plan included details for the 
Affordable Housing Data Architecture, Budget Forecasting and 
Formulation, Cash Management, and Portfolio and Risk Management Tool 
projects. The remaining project--Integrated Budget Forecasting Model--
developed its own plan. 

[46] For an explanation regarding what details were missing from the 
acquisition strategy, see the discussion for this project in the app. 
III section on acquisition strategies. 

[47] SEI, CMMI® for Acquisition and CMMI® for Development; and PMI, 
PMBOK® Guide (see footnote 9). 

[48] [hyperlink, http://www.gao.gov/products/GAO-06-310]. 

[49] Two of these nine projects--Federal Financial Services Platform 
and LEAP Automation of Lender Approval Workflow--provided the same 
requirements management plan. 

[50] While this project identified prioritization methods and applied 
them to its requirements, this prioritization was not reflected in the 
requirements management plan. 

[51] SEI, CMMI® for Acquisition and CMMI® for Development and PMI, 
PMBOK® Guide (see footnote 9). 

[52] [hyperlink, http://www.gao.gov/products/GAO-11-586] and 
[hyperlink, http://www.gao.gov/products/GAO-06-310]. 

[53] For the purposes of this report, we use the term "acquisition 
planning" to refer to pre-solicitation activities including practices 
described in the PMBOK Guide® as procurement management (see footnote 
9). See the discussion of acquisition strategies included in app. III 
for additional details. 

[54] PMI, PMBOK® Guide (see footnote 9). 

[55] The 8(a) Business Development Program is managed by the Small 
Business Administration and was established to help small 
disadvantaged businesses compete in the market place. 

[56] Although FHA Transformation and NGMS documents call for regular 
reporting from contractors, the acquisition strategies did not 
identify metrics for assessing the contractors' performance. 

[57] [hyperlink, http://www.gao.gov/products/GAO-12-580T]; [hyperlink, 
http://www.gao.gov/products/GAO-11-762]; [hyperlink, 
http://www.gao.gov/products/GAO-11-72]; and [hyperlink, 
http://www.gao.gov/products/GAO-09-675]. 

[58] Since the start of our review, FHA Transformation and NGMS 
identified two additional projects--Loan Review and HUDCAPS Migration-
-that were not in the fiscal year 2011 expenditure plan. Thus, these 
projects were not assessed in our study. In addition, when the 
department determined that the developed NGMS prototypes would not 
address business needs, we adjusted our study to include Affordable 
Housing Data Architecture and Portfolio and Risk Management Tool, 
which were two additional projects that replaced the prototype efforts. 

[59] These best practices are identified in the Software Engineering 
Institute/Carnegie Mellon, CMMI for Development, Version 1.3, CMU/SEI-
2010-TR-033 (Hanscomb AFB, Mass.: November 2010) and CMMI® for 
Acquisition, Version 1.3, CMU/SEI-2010-TR-032 (Pittsburgh, Pa.: 
November 2010); Project Management Institute: A Guide to the Project 
Management Body of Knowledge (PMBOK® Guide), Fourth Edition, Project 
Management Institute, Inc. (PMI), 2008, copyright and all rights 
reserved (material from this publication has been reproduced with the 
permission of PMI); and previously issued GAO reports and guidance, 
including GAO, Best Practices for Project Schedules--Exposure Draft, 
[hyperlink, http://www.gao.gov/products/GAO-12-120G] (Washington, 
D.C.: May 30, 2012); and Cost Estimating and Assessment Guide: Best 
Practices for Developing and Managing Capital Program Costs, 
[hyperlink, http://www.gao.gov/products/GAO-09-3SP] (Washington D.C.: 
Mar. 2, 2009). 

[60] The Office of Program Systems Management is responsible for 
developing and enhancing the automated systems that support 
multifamily programs and for coordinating funding for multifamily 
systems contracts. This office is within the Office of Housing/FHA 
Office of Multifamily Housing Programs, which is responsible for the 
overall management, development, direction, and administration of the 
department's multifamily housing programs. 

[61] SEI, CMMI® for Development; CMMI® for Acquisition; PMI, PMBOK® 
Guide (see footnote 2); [hyperlink, 
http://www.gao.gov/products/GAO-12-120G]; and [hyperlink, 
http://www.gao.gov/products/GAO-09-3SP]. 

[62] SEI, CMMI® for Acquisition and PMI, PMBOK® Guide (see footnote 2). 

[63] A work breakdown structure should also be used as the outline of 
the integrated master schedule, which includes all activities 
necessary to complete a project. 

[64] PMI's Practice Standard for Work Breakdown Structures provides 
guidance and example work breakdown structure templates that can be 
tailored to specific projects. In addition, [hyperlink, 
http://www.gao.gov/products/GAO-09-3SP] also provides a checklist of 
best practices for developing a work breakdown structure and includes 
case studies of agencies GAO has reviewed. 

[65] A deliverable is any unique and verifiable product, result, or 
capability to perform a service that must be produced to complete a 
process, phase, or project. Deliverables are produced as outputs from 
processes performed to accomplish the project work planned and 
scheduled in the project management plan. Deliverable-or product-
oriented means that a work breakdown structure's content should be 
focused on the deliverables or products and not on the processes. 

[66] [hyperlink, http://www.gao.gov/products/GAO-09-3SP] and PMI, 
PMBOK® Guide (see footnote 2). 

[67] GAO, Information Technology: Opportunities Exist to Improve 
Management of DOD's Electronic Health Record Initiative, [hyperlink, 
http://www.gao.gov/products/GAO-11-50] (Washington, D.C.: Oct. 6, 
2010); Office of Personnel Management: Retirement Modernization 
Planning and Management Shortcomings Need to Be Addressed, [hyperlink, 
http://www.gao.gov/products/GAO-09-529] (Washington, D.C.: Apr. 21, 
2009); and Information Technology Investment Management: A Framework 
for Assessing and Improving Process Maturity, [hyperlink, 
http://www.gao.gov/products/GAO-04-394G] (Washington, D.C.: Mar. 1, 
2004). 

[68] PMI, PMBOK® Guide (see footnote 2). 

[69] Attributes help define key information about the requirement. 
Typical attributes used in a traceability matrix may include a unique 
identifier, a textual description of the requirement, priority, 
current status (such as active, canceled, deferred, added, or 
approved), and date completed. Additional attributes to ensure that 
the requirement has met stakeholders' satisfaction may include 
stability, complexity, and acceptance criteria. 

[70] SEI, CMMI® for Acquisition and CMMI® for Development; PMI, PMBOK® 
Guide (see footnote 2); and [hyperlink, 
http://www.gao.gov/products/GAO-06-310]. 

[71] SEI, CMMI® for Acquisition and CMMI® for Development and PMI, 
PMBOK® Guide (see footnote 2). 

[72] For the purposes of this report, we use the term "acquisition 
planning" to refer to procurement activities and contract management, 
including practices described in the PMBOK® Guide as procurement 
management (see footnote 2). Procurement management plans called for 
by the PMBOK® Guide are best practices that are consistent with 
federal acquisition regulations (e.g., the Federal Acquisition 
Regulation, known as the "FAR") for developing acquisition strategies, 
particularly for major systems. See FAR, 48 C.F.R. §34.002. 
Departments and agencies must follow federal acquisition regulations 
as well as additional department-specific regulations, policies, and 
procedures. 

[73] These plans or strategies can be formal or informal, highly 
detailed or broadly framed and are based on the needs of each project. 
They are to be incorporated as a subsidiary component of the project 
management plan. 

[74] PMI, PMBOK® Guide (see footnote 2). 

[End of section] 

GAO’s Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO’s commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
“E-mail Updates.” 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO’s 
website, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov]. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Katherine Siggerud, Managing Director, siggerudk@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, DC 20548. 

Public Affairs: 
Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, DC 20548. 

[End of document]