This is the accessible text file for GAO report number GAO-04-394G 
entitled 'Information Technology Investment Management: A Framework for 
Assessing and Improving Process Maturity' which was released on March 
01, 2004.

This text file was formatted by the U.S. General Accounting Office 
(GAO) to be accessible to users with visual impairments, as part of a 
longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov.

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately.

Executive Guide: 

March 2004 Version 1.1: 

INFORMATION TECHNOLOGY INVESTMENT MANAGEMENT: 

A Framework for Assessing and Improving Process Maturity: 

GAO-04-394G: 

GAO Highlights: 

Highlights of GAO-04-394G, an executive guide.

Why GAO Did This Study: 

In 2000, GAO published an exposure draft of Information Technology 
Investment Management: A Framework for Assessing and Improving Process 
Maturity (ITIM). Built around the select/control/evaluate approach 
described in the Clinger-Cohen Act of 1996—which establishes statutory 
requirements for IT management—the framework provides a method for 
evaluating and assessing how well an agency is selecting and managing 
its IT resources. The exposure draft reflected current accepted or 
best practices in IT investment management, as well as the reported 
experience of federal agencies and other organizations in creating 
their own investment management processes. This new version updates 
the exposure draft to take into account comments that GAO has 
received; GAO’s experiences in evaluating several agencies’ 
implementations of investment management processes and the lessons 
learned by these agencies; and the importance of enterprise 
architecture (EA) as a critical frame of reference in making IT 
investment decisions. Using the framework to analyze an agency’s IT 
investment management processes provides: (1) a rigorous, standardized 
tool for internal and external evaluations of these processes; (2) a 
consistent and understandable mechanism for reporting the results of 
assessments; and (3) a road map that agencies can follow in improving 
their processes.

What GAO Found: 

The ITIM framework is a maturity model composed of five progressive 
stages of maturity that an agency can achieve in its IT investment 
management capabilities. These maturity stages are cumulative; that 
is, in order to attain a higher stage of maturity, the agency must 
have institutionalized all of the requirements for that stage in 
addition to those for all of the lower stages. The framework can be 
used both to assess the maturity of an agency’s investment management 
processes and as a tool for organizational improvement. For each 
maturity stage, the ITIM describes a set of critical processes that 
must be in place for the agency to achieve that stage. The figure 
below shows the five stages and lists the critical processes for each 
stage.

At the Stage 1 level of maturity, an agency is selecting investments 
in an unstructured, ad hoc manner. Project outcomes are unpredictable 
and successes are not repeatable; the agency is creating awareness of 
the investment process. Stage 2 critical processes lay the foundation 
for sound IT investment processes by helping the agency to attain 
successful, predictable, and repeatable investment control processes 
at the project level. Stage 3 represents a major step forward in 
maturity, in which the agency moves from project-centric processes to 
a portfolio approach, evaluating potential investments by how well 
they support the agency’s missions, strategies, and goals. At Stage 4, 
an agency uses evaluation techniques to improve its IT investment 
processes and its investment portfolio. It is able to plan and 
implement the “de-selection” of obsolete, high-risk, or low-value IT 
investments. The most advanced organizations, operating at Stage 5 
maturity, benchmark their IT investment processes relative to other 
“best-in-class” organizations and look for breakthrough information 
technologies that will enable them to change and improve their 
business performance.

www.gao.gov/cgi-bin/getrpt?GAO-04-394G.

To view the full product, click on the link above. For more 
information, contact David Powner, 202-512-4299, pownerd@gao.gov, or 
Lester Diamond, 202-512-7957, diamondl@gao.gov.

[End of section]

Contents: 

Preface: 

Section 1: Introduction: 

Changes from the Exposure Draft: 

Investment Management Overview: 

Section 2: Overview of ITIM: 

The Stages of Maturity: 

Progressing through the Stages of Maturity: 

Section 3: Components of ITIM: 

ITIM Hierarchy: 

Section 4: Uses of ITIM: 

Principles Guiding the Use and Interpretation of the Framework: 

Tool for Organizational Improvement: 

Tool for Assessing the Maturity of an Organization: 

Limitations and Boundaries: 

Section 5: Critical Processes for the ITIM Stages: 

Stage 1: Creating Investment Awareness: 

Stage 2: Building the Investment Foundation: 

Stage 3: Developing a Complete Investment Portfolio: 

Stage 4: Improving the Investment Process: 

Stage 5: Leveraging Information Technology for Strategic Outcomes: 

Appendixes: 

Appendix I: Glossary: 

Appendix II: Conducting an ITIM Assessment: 

Using ITIM to Assess IT Investment Decision-Making Processes: 

Summary of ITIM Assessment Process: 

Appendix III: Acknowledgments: 

Figures: 

Figure 1: Fundamental Phases of the IT Investment Approach: 

Figure 2: The Five Stages of Maturity Within ITIM: 

Figure 3: Critical Maturation Steps Required to Move to the Next 
Stage: 

Figure 4: The Components of an ITIM Critical Process: 

Figure 5: The ITIM Stages of Maturity with Critical Processes: 

Figure 6: The ITIM Stages of Maturity with No Stage 1 Critical 
Processes: 

Figure 7: The ITIM Stages of Maturity with Stage 2 Critical Processes: 

Figure 8: Instituting the Investment Board: 

Figure 9: Meeting Business Needs: 

Figure 10: Selecting an Investment: 

Figure 11: Providing Investment Oversight: 

Figure 12: Capturing Investment Information: 

Figure 13: The ITIM Stages of Maturity with Stage 3 Critical 
Processes: 

Figure 14: Defining the Portfolio Criteria: 

Figure 15: Creating the Portfolio: 

Figure 16: Evaluating the Portfolio: 

Figure 17: Conducting Postimplementation Reviews: 

Figure 18: The ITIM Stages of Maturity With Stage 4 Critical 
Processes: 

Figure 19: Improving the Portfolio's Performance: 

Figure 20: Managing the Succession of Information Systems: 

Figure 21: The ITIM Stages of Maturity with Stage 5 Critical 
Processes: 

Figure 22: Optimizing the Investment Process: 

Figure 23: Using IT to Drive Strategic Business Change: 

Figure 24: Phases in an ITIM Assessment: 

Preface: 

Investments in information technology (IT) can enrich people's lives 
and improve organizational performance. For example, during the last 
decade the Internet has matured from being a means for academics and 
scientists to communicate with each other to a national resource where 
citizens can interact with their government in many ways, for example, 
by receiving services, supplying and obtaining information, asking 
questions, and providing comments on proposed rules. Although they have 
the potential to improve lives and organizations, IT projects can also 
become risky, costly, unproductive mistakes. As we have described in 
numerous reports and testimonies, federal IT projects too frequently 
incur cost overruns and schedule slippages while contributing little to 
mission-related outcomes.

The Paperwork Reduction Act (PRA)[Footnote 1] requires federal agencies 
to be accountable for their IT investments and responsible for 
maximizing the value and managing the risks of their major information 
systems initiatives. The Clinger-Cohen Act of 1996[Footnote 2] 
establishes a more definitive framework for implementing the PRA's 
requirements for IT investment management. It requires federal agencies 
to focus more on the results they have achieved through IT investments, 
while concurrently improving their IT acquisition processes. The 
Clinger-Cohen Act[Footnote 3] also introduces more rigor and structure 
into how agencies are to select and manage IT projects. Among other 
things, it lays out specific aspects of the process that agency heads 
are to implement in order to maximize the value of the agency's IT 
investments and assess, manage, and evaluate the risks of its IT 
acquisitions.[Footnote 4] The E-Government Act of 2002[Footnote 5] 
provides additional guidance on IT management practices across federal 
agencies.

Through our research into IT management best practices and our 
evaluation of agency IT management performance, we have identified a 
set of essential and complementary management disciplines. These 
include: 

* investment management,

* strategic planning,

* software/system development and acquisition management,

* IT services acquisition management,

* human capital management,

* information security management, and: 

* enterprise architecture management.

Using the results of this research and evaluation, we have developed 
various management frameworks and guides. In 1997 we developed 
guidance,[Footnote 6] based primarily on the Clinger-Cohen Act, that 
provides a method for evaluating and assessing how well a federal 
agency is selecting and managing its IT resources. This guidance also 
identifies specific areas where improvements can be made. The 
Information Technology Investment Management (ITIM) Framework enhances 
this guidance by identifying critical processes for successful 
investment and organizing these processes into a framework of 
increasingly mature stages.

Maturity models have been proven to be a highly effective evaluative 
technique for the Software Engineering Institute, which is well 
regarded for its collection of Capability Maturity Models SM (e.g., 
Capability Maturity Model for Software).[Footnote 7],[Footnote 8] Other 
researchers have proposed similar approaches based on maturity 
models.[Footnote 9]

The maturity framework approach generally: 

* offers a comprehensive model for assessing process capability within 
an organization;

* can be applied to multiple types of disciplines, such as IT asset 
acquisition, human capital, and systems engineering; and: 

* can serve as a valuable tool for organizations to use to improve 
their technical development and management processes.

The initial ITIM exposure draft that we issued in May 2000[Footnote 10] 
reflected both a maturation of thinking in the area of IT investment 
management and input we had received from organizations and federal 
agencies based on their experiences in creating their own investment 
mechanisms and processes. This updated version has been modified based 
on comments we received on the initial exposure draft and on our 
experiences in evaluating and learning from agencies that are 
implementing investment management processes. Moreover, this version of 
the ITIM is consistent with and supports other maturity frameworks, 
including GAO's Enterprise Architecture Management Maturity Framework 
(EAMMF).[Footnote 11] Among other things, this version of the ITIM 
addresses the importance of an enterprise architecture (EA) as a 
critical frame of reference for organizations when they are making IT 
investment decisions.

The ITIM can be used to analyze an organization's investment management 
processes and to determine its level of maturity. Since its release in 
exposure draft in May, 2000, the ITIM has been GAO's primary tool for 
evaluating investment management capabilities. In addition, a number of 
agencies have used the framework as they worked to improve their 
investment processes.

If you have any questions about the Information Technology Investment 
Management Framework or the IT investment management approach, please 
contact me at (202) 512-4299 or [Hyperlink, pownerd@gao.gov]; or 
Lester Diamond, Assistant Director at (202) 512-7957 or 
[Hyperlink, diamondl@gao.gov]. Other key contributors to this report 
were Joanne Fiorino, Sabine R. Paul, Tomas Ramirez, Thomas Wright, and 
Neil Doherty.

Signed by: 

David A. Powner: 
Director, Information Technology Management Issues: 

[End of section]

Section 1: Introduction: 

The Information Technology Investment Management Framework identifies-
-and organizes into a framework of increasingly mature stages--thirteen 
processes that are critical for successful investment. The original 
exposure draft of ITIM expanded the widely accepted federal management 
framework for IT investment decision making that was embodied in OMB 
and GAO guidance[Footnote 12] and shifted the content from a guidance-
based focus to an activity-and maturity-based focus. Such a maturity 
framework can be used either to analyze an organization's investment 
management process or to determine the maturity of its investment 
process. The framework provides three key capabilities that are of use 
to many federal agencies: (1) a rigorous, standardized tool for 
internal and external evaluations of an agency's IT investment 
management process; (2) a consistent and comprehensible mechanism for 
reporting the results of these assessments to agency executives, the 
Congress, and other interested parties; and (3) a road map that 
agencies can use for improving their investment management processes. 
It should be noted, however, that an organization's achievement of more 
mature investment management stages depends on its instituting other 
good management practices and attributes, such as strategic planning, 
project management, enterprise architecture (EA) management, human 
capital management, and software and system acquisition management.

In May 2000 we released an exposure draft of the ITIM framework for 
trial and comment. Since that time, the framework has been used by a 
number of federal agencies in developing and enhancing their investment 
management strategies. In addition, we have used it to evaluate 
several agencies.[Footnote 13] This release includes lessons learned 
from our use of the framework in these evaluations and from lessons 
conveyed to us by users of the framework at a number of agencies. In 
order to validate the appropriateness of our changes and to gain the 
advantage of their experience, we provided this release for review to 
several outside experts who are familiar with the ITIM exposure draft 
and with investment management in a broad array of organizations, both 
public and private.

This version also includes a much fuller description of the 
relationship between ITIM and EA. Based on our experience, employing 
ITIM and EA in concert can greatly increase the chances that an 
organization's operational and IT environments will be pursued in a way 
that optimizes mission performance. The EA provides a clear and 
comprehensive picture of the structure of an entity, whether it is an 
organization or a functional or mission area. It defines an 
organization's operations in logical (i.e., information flows) as well 
as technical terms (i.e., hardware and software). The EA also describes 
these perspectives both for the organization's current or "as-is" 
environment and for its target or "to-be" environment as well as for a 
transition or sequencing plan for moving from the "as-is" to the "to-
be" environment.

Changes from the Exposure Draft: 

Stage 2 has been the primary beneficiary of the lessons learned from 
the use of the framework, because most agencies that we have evaluated 
are still operating at Stage 2. In Stage 2 we have tried to clarify 
aspects of critical processes that previously have led to diverse 
interpretations. In addition, we have moved what was previously the 
critical process for Authority Alignment of IT Investment Boards from 
Stage 3 in the exposure draft into Stage 2 in this release; it is now 
part of the critical process for Instituting the Investment Board. 
Through our work, we have found that instituting multiple boards was 
not unusual for organizations working in Stage 2 and that these boards 
occasionally were not well aligned.

Stage 3 has been enhanced to better explain the organization and use of 
portfolio management for investments. In this area we gained knowledge 
from the experiences of others, both directly from individuals using IT 
portfolio management in agencies as well as from literature that has 
been released during the last few years. In addition, we moved the 
critical process for Postimplementation Review and Feedback from Stage 
4 in the exposure draft to Stage 3 in this release. We did this so we 
could ensure that organizations that have completed Stage 3 are meeting 
the requirement for having selection, control, and evaluation processes 
in place, as required by the Clinger-Cohen Act.

Stages 4 and 5 have been modified only to reflect new names for 
critical processes and to relocate to Stage 3 the critical process for 
Postimplementation Review and Feedback. We have not gained substantial 
new experience in these stages, because few organizations are operating 
at these levels of maturity. We anticipate modifying these stages in 
the future, when we have learned more from organizations' experiences.

Investment Management Overview: 

A central tenet of the federal approach to IT investment management has 
been the select/control/evaluate model. This model was initially 
identified in our Strategic Information Management (SIM) Executive 
Guide,[Footnote 14] expanded in the Office of Management and Budget's 
IT investment guidance,[Footnote 15] and then refined in our subsequent 
guidance.[Footnote 16] It provides a systematic method for agencies to 
minimize risks while maximizing the returns of investments. Figure 1 
illustrates the central components of this model.

Figure 1: Fundamental Phases of the IT Investment Approach: 

[See PDF for image]

[End of figure]

During the select phase the organization (1) identifies and analyzes 
each project's risks and returns before committing significant funds to 
any project and (2) selects those IT projects that will best support 
its mission needs. This process should be repeated each time funds are 
allocated to projects, reselecting even ongoing investments as 
described below.

During the control phase the organization ensures that, as projects 
develop and investment expenditures continue, the project continues to 
meet mission needs at the expected levels of cost and risk. If the 
project is not meeting expectations or if problems have arisen, steps 
are quickly taken to address the deficiencies. If mission needs have 
changed, the organization is able to adjust its objectives for the 
project and appropriately modify expected project outcomes.

During the evaluate phase, actual versus expected results are compared 
after a project has been fully implemented. This is done to (1) assess 
the project's impact on mission performance, (2) identify any changes 
or modifications to the project that may be needed, and (3) revise the 
investment management process based on lessons learned.

The investment process does not end with the evaluation phase. A 
project can be active concurrently in more than one phase of the 
select/control/evaluate model. After a project has been designated for 
initial funding in the select phase, it becomes the subject of 
evaluation throughout the control phase for the purposes of 
reselection. Reselection is an ongoing process that continues for as 
long as a project is receiving funding. If a project is not meeting the 
goals and objectives that were originally established when it was 
selected, or if the goals have been modified to reflect changes in 
mission objectives--and corrective actions are not succeeding--a 
decision must be made on whether to continue to fund the project. 
Ultimately, "deselection" can be one of the most difficult steps to 
implement, but it is necessary if funds can be better utilized 
elsewhere. Once projects are operating and being maintained, they 
remain under constant review for reselection.

[End of section]

Section 2: Overview of ITIM: 

The Stages of Maturity: 

ITIM is comprised of five stages of maturity. Each stage builds upon 
the lower stages and enhances the organization's ability to manage its 
IT investments. Figure 2 shows the five ITIM stages and gives a brief 
description of each stage.

Figure 2: The Five Stages of Maturity Within ITIM: 

[See PDF for image]

[End of figure]

Stage 1: Creating Investment Awareness: 

Stage 1 is characterized by ad hoc, unstructured, and unpredictable 
investment processes. For example, in a Stage 1 organization, there is 
generally little relationship between the success or failure of one 
project and the success or failure of another project. If an IT project 
succeeds and is seen as a good investment, it is largely due to 
exceptional actions on the part of the project team, and thus its 
success might be difficult to repeat. Investment processes that are 
important for success may be known, but only to isolated teams; this 
process knowledge is not widely shared or institutionalized.

Most organizations with Stage 1 maturity have some type of project 
selection process in place as part of their annual budgeting activity. 
However, the selection process is frequently rudimentary, poorly 
documented, and inconsistently applied.

The unstructured and unpredictable investment processes that 
characterize a Stage 1 organization also mean that even if it 
recognizes that a given project is in trouble, it may not have adequate 
processes to consistently address and resolve the project's problems. 
Additionally, a focus on project results in terms of business benefits 
is often missing in these organizations.

Stage 2: Building the Investment Foundation: 

One focus of Stage 2 maturity is to establish basic selection 
capabilities. Basic selection capabilities are driven by the 
development of project selection criteria, including benefit and risk 
criteria, and an awareness of organizational priorities when 
identifying projects for funding. No longer are projects being funded 
solely on an ad hoc basis. The basic selection processes established in 
Stage 2 lay the foundation for more mature selection capabilities in 
Stage 3. Therefore, the organization also focuses on defining and 
developing its IT investment board(s), identifying the business needs 
or opportunities to be addressed by each IT project, and using this 
knowledge in the selection of new IT proposals.

An organization working to complete Stage 2 should be starting to 
develop an ITIM decision-making process that utilizes its EA--to the 
extent that an EA exists. An organization's "as-is" architecture may 
provide some of the basic information that is needed by decision 
makers, such as what systems currently exist and what potential 
functional overlap may occur with a new investment. In addition, an 
organization's EA tool may serve as a repository for investment 
information, although this may require modifying the manner in which 
the tool is currently being used. Criteria for selecting new and 
ongoing investments should be established, and the requirement to 
comply with the target EA may serve as an important guide in investment 
decisions. In addition, to gain further confidence that each investment 
is providing specific value to the organization, an organization's 
policies and procedures should provide for identifying the business 
needs and the associated users of each IT project.

An equally important focus is to attain repeatable, successful IT 
investment control techniques at the project level. For an organization 
to develop a sound IT investment process, it must first be able to 
control its investments so that they finish predictably within 
established schedule and budget ranges. In addition, it must be able to 
identify potential exposures to risk and put in place strategies to 
mitigate that risk. In the absence of predictable, repeatable, and 
reliable investment control processes, selected investments will be 
subject to a higher risk of failure despite rigorous analysis of the 
estimates used to justify them. Further, the absence of repeatable 
control processes will result in ineffective evaluation processes and 
contradictory efforts at process improvement.

To ultimately succeed, most IT investments require a relentless focus 
on interim results and successful risk management strategies, among 
other things. Taking this into account, an organization can begin by 
(1) focusing on gaining control of its existing collection of projects 
and (2) following a disciplined process for improving project outcomes 
over time by regularly tracking and overseeing each project's cost and 
schedule milestones and by monitoring expected benefits and risks. 
Supporting these activities requires collecting investment information 
to ensure that the organization knows fundamental facts about its IT 
assets, such as their location, cost, and ownership.

Stage 3: Developing a Complete Investment Portfolio: 

Stage 3 critical processes depend specifically on the successful 
implementation of Stage 2 critical processes. In order to operate 
successfully at Stage 3, the organization must have in place the 
structure and repeatability of the project-centric management processes 
described above. In addition, the project-specific performance data 
being used for oversight and reselection in Stage 2 are crucial for the 
successful management of the investment portfolio. The critical focus 
for Stage 3 maturation is to establish a consistent, well-defined 
perspective on the IT investment portfolio and to maintain mature, 
integrated selection (and reselection), control, and evaluation 
processes. These processes will be evaluated during postimplementation 
reviews (PIR). Once IT projects have been selected and are meeting 
their scheduled performance expectations--as outlined in Stage 2--the 
organization needs to develop an IT investment portfolio using an 
investment process that is consistent with its EA and employs sound 
selection criteria.

The development and use of portfolio selection criteria enable the 
organization to expand its focus from being primarily project-oriented 
to including the broader portfolio perspective. The portfolio 
perspective drives the organization to focus on the benefits gained 
from the synergies to be found among the investments in the entire 
collection, rather than just from the sum of the individual 
investments. Instead of focusing exclusively on the balance between the 
costs and benefits of individual investments, in Stage 3 decision 
makers also must consider the interaction among investments and the 
contribution to organizational mission goals and strategies that could 
be made by alternative portfolio selections. The development of the 
portfolio selection criteria communicates organizational priorities to 
the IT project management community and ensures that each investment 
submitted for funding supports the organization's mission, strategies, 
and goals, as well as project-specific outcomes. The critical process 
for Creating the Portfolio describes how the organization should use 
the portfolio selection criteria to develop an IT investment portfolio. 
Individual investments are reviewed and evaluated following their 
implementation in order to compare actual results with performance 
expectations.

An organization's policies and procedures should provide for specifying 
the relationship between its architecture and its investment decision-
making authority. The links between the EA and the investment portfolio 
should be explicitly defined. In addition, when operating at this 
stage, organizations should be working to align their EA with their IT 
portfolio selection criteria.

Stage 4: Improving the Investment Process: 

An organization at Stage 4 maturity is focused on using evaluation 
techniques to improve its IT investment processes and portfolio(s) 
while maintaining mature control and selection processes. At this 
stage, the organization should also regularly analyze its investment 
portfolio(s) to ensure that its investments continue to be aligned with 
the most current version of its architecture, since small changes in 
either an investment itself or in the EA may have occurred over time 
without being recognized in periodic selection/reselection decisions. 
As described in Stage 3, postimplementation reviews typically identify 
lessons learned from an investment and determine whether the benefits 
anticipated in the business case for the investment have been achieved. 
Analyzing a number of PIRs serves as a basis for creating 
recommendations for changing and improving IT investment processes.

Portfolio categories are used to organize the lessons learned and the 
recommendations gleaned both from PIRs conducted during Stage 3 and 
from other sources of process or investment information. The 
information within these categories is then used to fine-tune the 
investment processes and the portfolios. Additionally, at Stage 4 
maturity the organization has the capacity to conduct IT succession 
activities and thus can plan and implement the "deselection" of 
obsolete, high-risk, or low-value IT investments.

Stage 5: Leveraging Information Technology for Strategic Outcomes: 

Once an organization has mastered the selection, control, and 
evaluation processes, it seeks to shape its strategic outcomes by 
(1) using its EA as a critical frame of reference to ensure alignment 
with the target architecture, (2) learning from other organizations, 
(3) continuously improving the manner in which it uses IT to support 
and improve its business outcomes, and (4) focusing on flexibility and 
becoming a more agile organization that relies on its architecture for 
its vision of the future and the ITIM as a critical means for 
implementing it. Thus, an organization with Stage 5 maturity benchmarks 
its IT investment processes relative to other "best-in-class" 
organizations and conducts proactive monitoring for breakthrough 
information technologies that will allow it to significantly change and 
improve its business performance.

Progressing through the Stages of Maturity: 

Within ITIM, lower maturity stages provide the foundation for higher 
maturity stages. Thus, an organization increases its IT investment 
maturity and management capability as it progresses through the ITIM 
maturity stages. The following section describes the critical 
maturation steps that occur as an organization moves from one stage to 
the next (see fig. 3).

Figure 3: Critical Maturation Steps Required to Move to the Next Stage: 

[See PDF for image]

[End of figure]

Moving from Stage 1 to Stage 2: 

Investment control processes are the essential proficiencies that an 
organization establishes as it moves from ITIM Stage 1 to Stage 2. As 
investment control processes become better established,

* one or more investment board(s) is created to oversee and select IT 
projects;

* investment information such as costs, benefits, schedule, risk 
assessments, performance metrics, and system functionality is collected 
to support executive decision making;

* the organization gains a better perspective on the IT projects in 
which it is investing;

* communicating the status of ongoing projects improves 
organizationwide system acquisition, development, and management 
practices;

* the organization creates and maintains better project-level cost 
information; and: 

* key customers (or end users) and business needs for each IT project 
are identified, and the users are engaged in this process.

Critical to maturing project-level IT investment control processes is 
the ability to recognize the need for and to take swift corrective 
action when a project is having trouble meeting its schedule 
expectations and cost estimates. As it moves through Stage 2, an 
organization develops robust methods to collect data from the project-
level management processes and aggregate it appropriately to provide 
executive management with the information it needs to execute its 
oversight responsibilities. As the organization matures, it also learns 
from past decisions and better manages the causal factors that created 
past problems, thus improving the performance results of ongoing 
projects.

Beyond investment control processes, the organization also begins to 
implement basic selection processes. The core business needs for each 
IT project are identified and the basic portfolio development processes 
are used to select new IT proposals.

Moving from Stage 2 to Stage 3: 

Creation of a mature IT process for selecting investments is the major 
accomplishment that an organization demonstrates as it moves from Stage 
2 to Stage 3 maturity. In addition, well-developed investment control 
processes lead to greater certainty about future IT investment outcomes 
and greater confidence that IT investments, when they are selected, 
will achieve their expected cost, schedule, and performance goals, as 
well as their expected functionality. Thus, once the investment control 
processes have been established, an organization can build on these 
fundamental investment processes to create mature portfolio selection 
processes. Mature selection processes include: 

* the creation and maintenance of portfolio selection criteria,

* the analysis associated with examining the merits of each IT 
investment in the context of the portfolio,

* the use of an EA to help align IT investments with strategic 
objectives, and: 

* the grouping of similar investments together and the development of 
the portfolio.

Beyond the creation of a mature selection process, the organization now 
refines the elements of benefit and risk management in its investment 
control process, because it has installed the supporting tools for 
doing so as its selection process matures. Individual investments are 
reviewed and evaluated following their implementation and are judged 
based on how well they meet their performance expectations.

Moving from Stage 3 to Stage 4: 

As an organization reaches Stage 4 maturity, it has created mature IT 
investment evaluation processes and established a complete IT 
investment management process. In this stable environment, the 
organization can take the lessons it has learned from evaluating its 
investment processes (i.e., based on postimplementation reviews in 
Stage 3) and change these processes with predictably beneficial 
results. By doing so, it also creates the environment and the 
mechanisms for continuous improvement in Stage 5. In addition to 
improving its investment processes, an organization operating in Stage 
4 can manage resource succession--that is, "de-selecting" current IT 
investments--by migrating to successor investments or retiring obsolete 
and low-performing ones and by making these decisions in the context of 
the portfolio created in Stage 3 and a well understood EA sequencing 
plan and "to-be" architecture. Together, the portfolio, sequencing 
plan, and "to-be" architecture provide a full picture of the current 
state of an organization's investments, its vision of the future, and 
its plan for getting there. In this context, the obsolescence of 
systems can be anticipated, and the declining benefits of specific 
systems can be viewed in the light of alternative investments.

Moving from Stage 4 to Stage 5: 

An organization that is moving from Stage 4 to Stage 5 has mature 
selection, control, and evaluation processes in place. It now seeks 
ways to (1) institutionalize the continuous improvement of these 
processes and (2) improve its strategic business outcomes. It 
accomplishes these goals by examining and learning from other 
organizations by means of benchmarking. Benchmarking is used because 
there may be external organizations with specific processes that are 
more innovative or more efficient than its own processes. Beyond 
benchmarking, the organization leverages IT to significantly change and 
improve its business performance and outcomes.

[End of section]

Section 3: Components of ITIM: 

ITIM Hierarchy: 

Like other maturity models, ITIM is subdivided into a hierarchy. Each 
maturity stage consists of critical processes that are composed of a 
number of key practices. These hierarchical components are described 
below.

Maturity Stages: 

Each of the four maturity stages beyond Stage 1 is a plateau of well-
defined critical processes. The five maturity stages represent the 
steps toward achieving a mature, comprehensive IT investment management 
process.

Critical Processes: 

With the exception of Stage 1, each maturity stage is composed of 
multiple critical processes, such as the processes used to create an IT 
investment portfolio. Each critical process contains a set of key 
practices that, when fulfilled, implement the critical process needed 
to attain a given maturity stage.

Key Practices: 

The key practices are the tasks that must be performed by an 
organization in order to implement and institutionalize a critical 
process effectively. Key practices fall into three categories: 
organizational commitments, prerequisites, and activities. An 
explanation and a description of the relationship among these different 
types of key practices is shown in figure 4. In Section 5, each key 
practice is listed, followed by commentary and additional information 
that may assist an organization in understanding or interpreting how it 
could be implemented.

Figure 4: The Components of an ITIM Critical Process: 

[See PDF for image]

[End of figure]

[End of section]

Section 4: Uses of ITIM: 

ITIM identifies critical IT investment processes, establishes the 
presence or absence of these critical processes in an organization, 
assesses an organization's IT investment management capability and 
maturity, and offers recommendations for improvement. Used in this way, 
ITIM can be a valuable tool that (1) supports organizational self-
assessment and improvement and (2) provides a standard against which an 
evaluation of an organization can be conducted.

Principles Guiding the Use and Interpretation of the Framework: 

Regardless of the specific reason for using ITIM, the following 
principles[Footnote 17] should guide each interpretation and use of 
this framework.

* The ITIM is a generic framework intended for broad use. The way in 
which an organization implements the framework will vary, depending on 
its needs for improving its investment processes and its managerial and 
professional judgment.

* The ITIM is a road map for improvement and describes the 
characteristics of an IT investment management process that one would 
expect to see at each maturity stage. The maturity stages prescribe the 
order in which to improve the processes, but not how an organization is 
to improve its processes.

* The ITIM may not exhaustively describe the necessary conditions for 
successful investment management in all organizations. Other components 
of the investment management process may exist and could be considered 
for addition to this framework as greater context sensitivity develops 
to the issues surrounding the process of IT investment management.

* Each ITIM critical process will generally go through a step-by-step 
evolution--consisting of introduction, adoption, development, and 
finally full implementation--within an organization as that 
organization changes over time, modifies necessary functions and 
operations, and reaches a particular maturity stage. ITIM does not 
address all factors that can affect investment success. For example, 
organizational processes and other factors--such as strategic planning, 
availability of funding, risk assessments, and specific technology 
implementations--can strongly influence an organization's investment 
success.

* There is no one right way to implement the ITIM, because the 
framework describes the characteristics of mature and successful IT 
investment management processes, not specific implementation 
techniques. Because of this, the framework is technology independent. 
For example, no specific tools, methods, or technologies are mandated 
by its use. Appropriate tools, methods, and technologies should be made 
available to support the processes that an organization develops within 
ITIM.

Tool for Organizational Improvement: 

ITIM offers organizations a road map for improving their IT investment 
management processes in a systematic and organized manner. These 
process improvements are intended to: 

* improve the likelihood that investments will be completed on time, 
within budget, and with the expected functionality,

* promote better understanding and management of related risks,

* ensure that investments are selected based on their merits by a well-
informed decision-making body,

* implement ideas and innovations to improve process management, and: 

* increase the business value and mission performance of investments.

ITIM can be implemented as a tool for organizational improvement in a 
variety of ways. For example, an organization can create a separate 
improvement program, employ external assistance and support, or use the 
framework as a managerial support tool. Regardless of the 
implementation technique, the following important factors should be 
considered when using ITIM as an organizational improvement tool: 

* Many organizations will have a variety of selection, control, and 
evaluation processes in place. ITIM can help these organizations 
understand the relationships among these processes and determine the 
key opportunities for immediate improvements.

* The framework uses a structured approach that identifies the key 
practices for creating and maintaining successful investment management 
processes. However, it describes what to do, not how to do it. Thus, 
specific implementation methods can and will vary by organization, 
based on specific attributes of the organization, such as size, 
complexity, and culture.

* The developmental nature of a maturity model means that process 
maturation is cumulative. Lower-stage processes provide the foundation 
for upper-stage processes. As additional critical processes are 
introduced into the organization and implemented, the organization 
attains greater process capabilities and maturity. As the organization 
incorporates additional processes at each successive stage of maturity, 
it must maintain the lower-stage critical processes that it has 
previously implemented.

* The framework depends on good project management to form the 
foundation of good performance measurement and the project-level 
control processes that underlie mature investment control processes.

* Where one exists, the use of an EA is a critical frame of reference 
for making investment decisions, and only investments that move the 
organization toward its target architecture--as defined by its 
sequencing plan--should be approved unless a waiver is provided and/or 
a decision is made to modify the EA.

* Critical processes initially may be implemented and practiced within 
individual bureaus or divisions before they are implemented and are 
mature across the organization.

* Business process improvement initiatives are usually not themselves 
considered to be IT investments; they are considered to be parallel 
efforts that may or may not be linked to investments. Thus, ITIM 
assessments do not evaluate individual initiatives. However, if such 
initiatives include IT investments, then the investments should be 
subject to the organization's investment management process.

* Change management should be a cornerstone of process improvement, 
because culture affects the nature of investment decisions. Investment 
decisions are about change, and change affects an organization's 
culture. For example, a decision can be creative or cautious, strategic 
or tactical. Culture emanates from the values of the organization.

Tool for Assessing the Maturity of an Organization: 

Just as ITIM can be used as a tool for organizational improvement, it 
can also be used as a standard against which to judge the maturity of 
an organization's IT investment management process. For example, ITIM 
can be used to support assessments to help ensure compliance with 
industry standards or acceptable practices, independent reviews of 
organizational maturity by oversight bodies, or other external IT 
process reviews. Regardless of the specific use, however, the following 
important factors should be considered when using ITIM as an 
organizational assessment tool: 

* An assessment using the framework can be conducted for an entire 
organization (e.g., an executive branch department) or for one of its 
lower-level divisions (e.g., a branch, bureau, or agency). However, the 
unit or scope of analysis (e.g., branch, bureau, agency, or department) 
must be defined before an ITIM assessment is conducted. Additionally, 
the assessed maturity stage for a lower-level division is not 
necessarily indicative of the maturity stage of a higher-level division 
or of the organization as a whole.

* The use and interpretation of ITIM by organizations may vary with 
their size, culture, and organizational structure--as well as other 
factors. The overriding objective of the framework is to enable senior 
managers to systemically maximize the benefits of IT investments 
through the use of a structured investment process. In achieving this 
objective, different organizations may choose different specific 
implementations of the ITIM, which may be influenced by the factors 
mentioned above. For example, although ITIM addresses the 
organizational need to align and coordinate multiple investment boards, 
an organization with only one IT investment board would not need to 
perform the key practices associated with board alignment. Also, small 
organizations--or those with highly centralized IT management--may not 
require as extensive written guidance as large organizations, because 
their investment management processes are executed by a small, cohesive 
cadre of managers. Ultimately, each organization must use its best 
judgment in determining how to implement ITIM within its own context.

* An organization may be concurrently implementing key practices that 
are associated with several maturity stages. In fact, key practices 
associated with higher stage critical processes are frequently 
initiated while the organization as a whole is at a lower stage of 
maturity. However, organizational maturity is determined by assessing 
at what maturity stage the organization implements all of the key 
practices for all of the critical processes associated with a given 
stage of maturity--in addition to all of those associated with lower 
maturity stages. For example, performing key practices in only some of 
the Stage 3's critical processes does not mean that the organization 
has attained Stage 3 maturity.

Limitations and Boundaries: 

The purpose of ITIM is to describe and improve an organization's IT 
investment management processes so that the strategic plans and 
decisions that it makes can and will be supported by highly effective 
investments. However, like other assessment tools, the framework has 
its limitations and boundaries. For example, while strategic planning 
and executive decision making can greatly influence an organization's 
performance, the framework does not evaluate these. If IT plans and 
business plans are linked, there is a high likelihood that investment 
decisions will be closely aligned with the business.

Similarly, performance measures that are created and used to guide the 
organization and its activities are an integral part of controlling the 
expenditures on an investment and can be viewed as maturing in parallel 
with the IT investment management processes. However, this guide does 
not describe in detail[Footnote 18] the development or implementation 
of these measures.

In addition, the framework does not address IT acquisition (e.g., which 
type of contract to use or how best to conduct price negotiations, 
etc.) as a separate investment management step. While they are 
important, the primary purpose of acquisition-related activities is to 
support the execution of the investment decisions that are made by the 
IT investment board(s)[Footnote 19] Thus, one would expect that the 
acquisition aspects of project development would be embedded in the 
project proposal and analysis steps within the framework. 
Alternatively, the acquisition strategy might be part of the project's 
risk assessment (i.e., the risks of pursuing various acquisition 
alternatives).

Finally, organizations selecting ITIM as an assessment tool should: 

* become proficient with the related GAO and OMB guidance on IT 
investment.[Footnote 20] This is particularly important for those 
seeking to apply ITIM in the federal government. Understanding this 
guidance provides greater insight into the developmental history, key 
issues, and critical success factors associated with the IT investment 
approach.

* become familiar with generally accepted capital decision-making 
approaches and associated analytical tools;

* become familiar with the concepts associated with EA management;

* receive training to become familiar with the basic concepts behind 
maturity models; and: 

* have experience using standardized assessment tools to assess 
organizations.

For further guidance on how to conduct an ITIM evaluation, refer to 
appendix II of this document.

[End of section]

Section 5: Critical Processes for the ITIM Stages: 

Figure 5: The ITIM Stages of Maturity with Critical Processes: 

[See PDF for image]

[End of figure]

The following subsections describe each maturity stage in greater 
detail. The first subsection describes only the attributes of Stage 1 
because no critical processes are associated with this stage. Each 
subsequent subsection describes one of the stages. In each subsection, 
the stage is briefly introduced and its associated critical processes 
are identified, along with a list of applicable criteria. For each 
critical process, a brief introduction and purpose is presented, along 
with a map showing the associated key practices (organizational 
commitments, prerequisites, and activities) that make up the critical 
process and a discussion and interpretation of the key practice. For 
easy reference, each page heading in section 5 indicates which stage 
and critical process are being discussed on that page.

Stage 1: Creating Investment Awareness: 

Figure 6: The ITIM Stages of Maturity with No Stage 1 Critical 
Processes: 

[See PDF for image]

[End of figure]

The following section provides a description of the conditions and 
characteristics associated with an organization operating at ITIM Stage 
1. Within ITIM, Stage 1 is different from the other maturity stages 
because: 

* there are no critical processes associated with Stage 1; and: 

* it is typified by the absence of an organized, executable, and 
consistently applied IT investment management process.

The following description of an ITIM Stage 1 organization is not 
intended to be comprehensive; rather, it provides an overview of the 
general conditions and problems that typically confront a Stage 1 
organization.

Generally, an ITIM Stage 1 organization has ad hoc or undisciplined IT 
investment management processes. This often contributes to escalating 
project costs, unmitigated risks, frequent slippages in project 
schedules, and low-value mission or business benefits. Furthermore, 
while the organization may have "pockets of excellence" in IT 
investment management, the variability in these processes across the 
organization may lead to inconsistency in IT project outcomes.

Select Process: 

The Stage 1 organization's focus is more often on a project's funding 
requirements and lower level organizational requirements rather than on 
(1) its value toward achieving the organization's mission goals, 
(2) its technical and economic risks, (3) its performance problems, or 
(4) cost and schedule overruns. IT is treated as an expense item in 
most organizations' budgets, and it may be intertwined with other 
administrative and management support funding needs. Also, multiyear IT 
projects that are "in the budget pipeline" are reviewed each year 
largely in terms of marginal increases or decreases to the previous 
year's funding base, regardless of cost, schedule, and performance 
results to date.

In short, while some IT projects within a Stage 1 organization may be 
funded because they link to a defined business or mission purpose, many 
projects are funded despite the absence of critical information that 
demonstrates expected and achieved improvements in program, business, 
or mission performance.

Control Process: 

Stage 1 organizations typically have unstructured, ill-timed, and 
inconsistent IT investment management controls. Senior executives and 
line managers may rarely review IT projects' performance data, and thus 
the organization lacks an early warning method for quickly detecting 
and rectifying major problems. Instead, project crises are handled as 
they arise, focusing only on quick fixes rather than considering 
possible systemic causes of the problems. As a result, the success of 
individual projects is unpredictable and may often be the result of 
extraordinary efforts by individuals or the project team.

Additionally, a Stage 1 organization rarely would have an up-to-date 
and complete collection of investment information. For example, 
although it might have an IT hardware (equipment) inventory, it might 
lack a comprehensive list of systems, software applications and tools, 
and licensing agreements. Without a complete inventory of IT 
information, an organization cannot develop an adequate investment 
control process.

Evaluate Process: 

Finally, a Stage 1 organization rarely, if ever, (1) evaluates IT 
investment outcomes or (2) identifies lessons learned from its 
projects. If such evaluations are conducted, they often are triggered 
only in response to outside pressures (e.g., an audit or a budget 
oversight review), and they tend to be poorly staffed and conducted 
without a formal process that delineates method, scope, and 
responsibilities.

Stage 2: Building the Investment Foundation: 

Figure 7: The ITIM Stages of Maturity with Stage 2 Critical Processes: 

[See PDF for image]

[End of figure]

Stage 2 builds the foundation for current and future IT investment 
success by establishing basic IT selection and control processes. This 
stage is defined by five critical processes. Each critical process is 
described below, followed by a set of "Criteria," and a listing of 
documents that establish criteria supporting the use of the critical 
process in ITIM.

* Instituting the Investment Board is the process for creating and 
defining the membership, guiding policies, operations, roles, 
responsibilities, and authorities for one or more IT investment boards 
within the organization.

Criteria: Assessing Risks and Returns: A Guide for Evaluating Federal 
Agencies' IT Investment Decision-making (hereafter referred to as IT 
Assessment Guide) (AIMD-10.1.13), 32, (CCA, OMB M-97-0(2)); Executive 
Guide: Improving Mission Performance Through Strategic Information 
Management and Technology (hereafter referred to as SIM Executive 
Guide) (AIMD-94-115), Practices 2, 10; Evaluating Information 
Technology Investments, version 1.0 (hereafter referred to as OMB IT 
Investment Guide), Office of Management and Budget, 3; Capital 
Programming Guide, version 1.0, Office of Management and Budget, ii.

* Meeting Business Needs is the process for developing a business case 
that identifies the key executive sponsor and business customers (or 
end users) and the business needs that the IT project will support.

Criteria: IT Assessment Guide (AIMD-10.1.13), 15, 16, 17; SIM Executive 
Guide (AIMD-94-115), Practices 4, 9; OMB M-97-16.

* Selecting an Investment introduces a defined process that an 
organization can use to select new IT project proposals and reselect 
ongoing projects.

Criteria: IT Assessment Guide (AIMD-10.1.13), 23-25, (CCA, PRA, EO 
13011, OMB A-11, OMB A-130, OMB A-109, OMB A-94, OMB M-97-0(2)): 

* Providing Investment Oversight is a pivotal process whereby the 
organization monitors projects against cost and schedule expectations 
as well as anticipated benefits and risk exposure.

Criteria: IT Assessment Guide (AIMD-10.1.13), 52, (CCA, PRA, FASA, EO 
13011, OMB A-11, Part 3); OMB IT Investment Guide, 10.

* Capturing Investment Information is the process by which specific 
details about a particular investment are captured and maintained to 
provide asset-tracking data to executive decision makers.

Criteria: IT Assessment Guide (AIMD-10.1.13), 8, 19; PRA; E.O. 13103; 
Capital Programming Guide, ii.

Instituting the Investment Board: 

The IT investment board is a key component in the investment management 
process. This critical process defines the membership, guiding 
policies, operations, roles, responsibilities, and authorities for each 
designated board and, if appropriate, each board's support staff. This 
definition provides the basis for each board's investment selection, 
control, and evaluation activities. The organization may choose to make 
this board the same board that provides executive guidance and support 
for the EA. This overlap of responsibilities may enhance the ability of 
the board to ensure that investment decisions are consistent with the 
architecture and that it reflects the needs of the organization.

Depending on its size, structure, and culture, an organization may have 
more than one IT investment board. This critical process is based on 
the assumption that, for managerial reasons, the key practices in this 
critical process will be implemented consistently across each of these 
boards and that the organization will tailor each board's operations as 
part of this implementation.

Figure 8: Instituting the Investment Board: 

[See PDF for image]

[End of figure]

Purpose: 

To define and establish an appropriate IT investment management 
structure and the processes for selecting, controlling, and evaluating 
IT investments.

Organizational Commitments: 

Commitment 1: An enterprisewide IT investment board composed of senior 
executives from IT and business units is responsible for defining and 
implementing the organization's IT investment governance process.

The enterprisewide investment board is created to (1) define the 
investment board's structure and accompanying processes and 
(2) implement the processes as they are defined. This board is 
comprised of senior executives, including the organization's head or a 
designee,[Footnote 21] the Chief Information Officer (CIO) or other 
senior executive representing the CIO's interests, and heads of 
business units and supporting units such as financial management. When 
the CIO is represented on the board by another senior executive, this 
executive must have knowledge of the CIO's management responsibilities 
and be able to fully represent the technical criteria that are being 
applied in the investment decision process. In cases where lower-level 
investment boards, comprised of individuals from across the 
organization, are chartered to carry out the responsibilities of the 
enterprisewide IT investment board within their own business units, the 
enterprisewide IT investment board still must maintain ultimate 
responsibility for the lower-level boards' activities. These 
subordinate boards should have the same broad representation as the 
enterprisewide board, though at the subordinate unit's level.

The enterprisewide IT investment board is responsible not only for 
major systems that affect multiple departments and users. These 
enterprisewide investments should be elevated to the enterprisewide IT 
investment board to ensure buy-in from senior executives and users 
representing various departments. The enterprisewide IT investment 
board should be actively involved in all IT investments and proposals 
that are high cost or high risk or have significant scope and 
duration.

Commitment 2: The organization has a documented IT investment process 
directing each investment board's operations.

The organization uses the available IT investment process 
guidance[Footnote 22] and defines the unique manner in which the 
guidance will be implemented. The guidance should lay out the roles of 
key boards, working groups, and individuals involved in the 
organization's IT investment processes, and it should explain the 
procedures for assigning responsibility for decision making for a given 
investment or proposal. The guidance should specify that individual 
business or operational units retain decision-making authority for 
unit-specific IT decisions while still following enterprisewide 
standards and procedures, and it should outline the significant events 
and decision points within the processes; identify external and 
environmental factors that will influence the processes (i.e., legal 
constraints, the behavior of key suppliers or customers, or industry 
norms); and specify the manner in which IT investment-related processes 
will be coordinated with other organizational plans, processes, and 
documents--including, at a minimum, the strategic plan, budget, and EA.

In IT organizations that have multiple IT investment boards, the 
enterprisewide investment process guide should document the policies 
and procedures that define each IT investment board's span of authority 
and describe how investment board activities are to be coordinated.

Prerequisites: 

Prerequisite 1: Adequate resources, including people, funding, and 
tools, are provided for supporting the operations of each IT investment 
board.

Executive management is typically responsible for creating the 
investment board(s), defining their scope and resources, and specifying 
their membership. Establishing an investment management working group 
can benefit both the IT investment boards and IT project managers by 
coordinating requests for information and verifying and providing 
responses.

Prerequisite 2: The board members understand the organization's IT 
investment management policies and procedures and the tools and 
techniques used in the board's decision-making process.

Members of the investment board should have an understanding of the 
board's policies and procedures and the experience and skills to carry 
them out. Thus, the organization should consider introducing investment 
concepts to board members with little or no investment decision-making 
experience or relevant education in this area. Orientation sessions 
might be provided to board members in areas such as economic evaluation 
techniques, capital budgeting methods, performance measurement 
strategies, and risk management approaches. In addition, board members 
should be made aware of the specific processes for which they are 
responsible.

Knowledge building and/or orientation sessions might include: 

* briefings specifically designed for new board members,

* educational forums,

* formal seminars, and: 

* executive training programs offering in-depth courses.

Prerequisite 3: Each board's span of authority and responsibility is 
defined to minimize overlaps or gaps among the boards.

When multiple boards execute the organization's IT investment 
governance process, criteria aligning these boards must be defined such 
that there are no overlaps or gaps in the boards' authorities and 
responsibilities. These criteria can be based on cost, benefit, 
schedule, and risk thresholds, the number of users affected, the 
function of the business unit (e.g., CIO, human resources, or program 
office), the life cycle phase of an IT investment (e.g., proof of 
concept, full scale development, or operations and maintenance), or 
other comparable and useful measures. An example would be to manage 
investments with less than a $100,000 life cycle cost at the lowest 
departmental level, but to have investments with more than $100 million 
in life cycle costs managed by the enterprisewide investment board.

Activities: 

Activity 1: The enterprisewide investment board has oversight 
responsibilities for the development and maintenance of the 
organization's documented IT investment process.

As the board responsible for defining and implementing the 
organization's IT investment management process, the enterprisewide IT 
investment board should also have responsibility for developing the 
organization-specific IT investment guide. The board's work processes 
and decision-making processes (i.e., schedules, agendas, authorities, 
decision-making rules, etc.) are described and documented in the 
guidance. In addition, after the guide has been developed, the 
enterprisewide investment board must actively maintain it, making sure 
that it always reflects the board's current structure and the processes 
that are being used to manage the selection, control, and evaluation of 
the organization's IT investments.

Activity 2: Each investment board operates in accordance with its 
assigned authority and responsibility.

For the whole IT investment management process to function smoothly and 
effectively, each investment board must operate within its assigned 
authority and responsibility, so that investments are properly aligned 
with the organization's objectives and are reviewed by the appropriate 
board.

Activity 3: The organization has established management controls for 
ensuring that investment boards' decisions are carried out.

Establishing management controls helps to ensure that management will 
carry out the decisions made by the IT investment board. Without these 
controls in place, decisions made by the investment board might not be 
implemented because of conflicting priorities. To ensure adherence to 
management controls, the structure of the relationship between upper 
management and the investment board must be documented and agreed to by 
both parties,. The investment board must have the confidence of upper 
management when selecting new proposals and ongoing projects for 
funding.

Meeting Business Needs: 

IT projects and systems should be tightly aligned with the business 
needs of the organization, providing support for highly visible core 
business processes. These strategically aligned IT projects and systems 
provide the highest value and most obvious investment benefits to an 
organization and are hallmarks of successful return on investment.

To achieve such a robust level of support, the organization must 
continually identify the business necessity for its IT projects and 
systems. Periodic identification of the business needs ensures that the 
correct and appropriate IT projects and systems are funded and that 
they directly support the organization's strategic plan. The frequency 
of this business verification may range from every quarter for an R&D 
project to every 3 years for systems in operations and maintenance; the 
appropriate interval depends upon the pace of functional changes in the 
system and the evolution of users' needs. Identifying business needs 
ensures that IT projects and systems will maintain an alignment with 
the organization's strategic plans and its business goals and 
objectives.

To the extent that the organization has planning documents--such as a 
strategic plan or a target enterprise architecture--these documents 
should be used as a source of agreed upon business needs. In addition, 
other business needs may surface through the investment process itself. 
In all cases, these business needs should be aligned with specific 
strategic objectives of the organization.

The essence of identifying business needs is for the business case for 
every IT project and system to be periodically reviewed and verified 
with respect to the business need(s) it is supporting. If an IT project 
or system is out of alignment with its strategic plan, then the IT 
investment needs to be resynchronized with the strategic plan or the 
overall strategic plan needs to be changed. Based upon the business 
case review, the most promising IT projects and systems are identified 
for continued investment. The investment board addresses whether 
business and user needs continue to be met in a cost-effective and 
risk-insured manner.

This critical process establishes a mechanism for verifying the 
business case (such as business requirements and rules, congressional 
mandate, and the organizational users) that drives continued support 
for each IT system. Ensuring that an essential link exists between the 
organization's business objectives and its IT strategy and that a 
defined partnership exists between the sponsoring unit and the IT 
solution providers strengthens and institutionalizes the 
organization's investment management process.

Figure 9: Meeting Business Needs: 

[See PDF for image]

[End of figure]

Purpose: 

To ensure that IT projects and systems support the organization's 
business needs and meet users' needs.

Organizational Commitments: 

Commitment 1: The organization has documented policies and procedures 
for identifying IT projects or systems that support the organization's 
ongoing and future business needs.

The organization has policies and procedures that outline a systematic 
process for identifying, classifying, and organizing its business needs 
and the IT projects used to support these needs. In many cases, this 
can be covered in the internal guidance that is used for documenting 
business cases for IT investments. These policies and procedures 
typically specify that: 

* a systematic process for identifying, classifying, and organizing 
business needs is linked to the business planning process,

* business needs or opportunities should be stated in functional terms 
or in terms of desired business improvement and not in product-or 
technology-specific terms,

* each IT project or system fits within the organization's EA and 
established security standards: 

* IT projects or resources that do not support an identified business 
need (and the associated customers or end users) are further examined 
for possible termination,

* there is a procedure by which similar needs or opportunities within 
different operating units are reconciled, and: 

* meeting business needs occurs regularly as part of the strategic 
planning cycle.

Prerequisites: 

Prerequisite 1: The organization has a documented business mission with 
stated goals and objectives.

The business mission, containing the stated goals and objectives is 
typically identified in: 

* strategic management or business plans (e.g., agency strategic plans 
prepared for GPRA),

* business process architecture documents,

* process improvement initiatives, or: 

* performance measurement plans.

Defining these goals and objectives, however, is largely outside the 
scope of ITIM. (See also Section 4: Limitations and Boundaries of 
ITIM.): 

Prerequisite 2: Adequate resources, including people, funding, and 
tools, are provided for ensuring that IT projects and systems support 
the organization's business needs and meet users' needs.

These resources typically involve: 

* funding for these activities;

* managerial attention to this process;

* an executive sponsor for the project;

* staff support for carrying out these activities; and: 

* supporting methods, analytical tools, and processes.

Activities: 

Activity 1: The organization defines and documents business needs for 
both proposed and ongoing IT projects and systems.

Each IT project is directly or indirectly linked to at least one of the 
organization's business needs or mission goals; a direct link is of 
greater value than an indirect link. This link can be established in a 
variety of ways. For example, an organization can: 

* identify a project's business purpose as part of the project's 
initiation activities,

* define an executive sponsor for each project, or: 

* obtain validation from external groups supporting the business value 
of the project.

The business needs for each IT project will generally be documented in 
the business case for the project.

Activity 2: The organization identifies specific users and other 
beneficiaries of IT projects and systems.

Each major IT project or system will have end users or customers who 
will benefit from the system. A given project or system may address the 
needs of multiple sets of end users or customer groups. The primary end 
users or customers will be formally identified by the organization.

Identifying the end users early in the process assists the IT staff 
developing the IT project or system in focusing on the specific, well-
defined goals of delivering value to end users. So that they may 
accomplish their particular work, end users depend directly on the IT 
staff to deliver a project's capability and to provide a system's 
functionality.

Activity 3: Users participate in project management throughout an IT 
project's or system's life cycle.

End user involvement will vary during the different stages of a 
project's system life cycle. During the project's conception, end users 
should be heavily involved in developing the business case and in 
defining how the system will help to meet business needs or 
opportunities. They will be heavily involved again during user 
acceptance testing. During other phases of development, they will play 
a more limited role.

In the final phases of the system's life cycle, especially during the 
operational phase of the system, end users should play a major role in 
helping to identify and document any benefits that are realized from 
the system's implementation. Users should also participate in the 
operational analysis of the system. The analysis should involve 
collecting information about the system's performance and comparing it 
with the initial performance baseline.

Activity 4: The investment board periodically evaluates the alignment 
of its IT projects and systems with the organization's strategic goals 
and objectives and takes corrective actions when misalignment occurs.

This activity permits the investment board to assess a project's or 
system's outcomes and its value in comparison to predefined 
expectations, in preparation for determining whether or not and how 
well the IT project or system is meeting the organization's 
expectations. After deployment, a system's success is measured by its 
ability to continually meet a business or user need.

The length of the period for collecting IT system data prior to review 
and analysis varies from one organization to another. An organization 
could, for example, annually review one-third or one-half of its 
operational IT systems. Another organization could decide to review all 
operational IT systems every 3 years. The essential point is that 
operational IT systems are investments that need to be reviewed on a 
regular basis to ensure that they are still providing value to the 
organization in a cost-effective and risk-insured manner.

Using historical data, system expectations, and other factors as 
criteria, the investment board evaluates every IT system to determine 
its value to the organization. The review cycle should reflect the risk 
and volatility of the project or system being evaluated. Periodic 
evaluation of each IT project or system permits the investment board to 
determine the ongoing value that each investment is providing to the 
organization and its end users. These periodic evaluations are critical 
to determining whether or not to continue to fund an IT system.

When an investment is found to be out of alignment with the 
organization's strategic goals and objectives, immediate action must be 
taken at the project level, with oversight provided by the investment 
board, to realign the project or system. But even a successful system 
will eventually begin to provide diminishing returns as it becomes more 
expensive to maintain. In addition, changing business requirements also 
can make a system obsolete.

Selecting an Investment: 

The purpose of this critical process is (1) to predefine a method for 
selecting new IT proposals and (2) use this method to select new 
proposals. Within ITIM, "new" proposals include both (1) previously 
submitted IT proposals that were not originally selected for funding 
and (2) IT proposals that have never been submitted.

Defining and implementing a selection process is a basic step toward 
implementing the mature IT critical processes for proposal and project 
selection in Stage 3. The key activities implemented within this 
critical process include (1) concurrent review of IT proposals by the 
organization's executives, (2) the use of predefined selection criteria 
to analyze the proposals, and (3) decision making by executives to fund 
some proposals and not others. The EA, where it exists, should be 
reflected in the selection criteria. Investments may come up outside of 
the EA, in which case their value must be considered under the same 
criteria as all other investments. Investments that are not consistent 
with the current EA should either be assimilated into the EA or be 
provided a waiver.

Reselection of ongoing projects is a very important part of this 
critical process. If a project is not meeting the goals and objectives 
that were established in the original selection, the investment board 
must make a decision on whether to continue to fund it.

Figure 10: Selecting an Investment: 

[See PDF for image]

[End of figure]

Purpose: 

To ensure that a well-defined and disciplined process is used to select 
new IT proposals and reselect ongoing investments.

Organizational Commitments: 

Commitment 1: The organization has documented policies and procedures 
for selecting new IT proposals.

The organization has policies and procedures that outline a structured 
method for identifying, evaluating, prioritizing, and selecting its new 
IT proposals.

Using a structured method to select new IT projects accomplishes 
several objectives. First, a structured method provides the 
organization's investment board, business units, and IT developers 
(whether they are internal IT staff or contractors) with a common 
understanding of the process and the cost, benefit, schedule, and risk 
criteria that will be used to select IT projects. Second, whether a 
business unit identifies a business need and develops an IT proposal 
itself or the organization's IT group develops the proposals, 
organizational roles and responsibilities will be defined for each 
participating unit involved in the project selection process. Lastly, 
the data required for decision making and the decision-making 
procedures should be predefined.

A documented selection process can help to ensure consistency when an 
organization is considering multiple investments for funding. 
Transparency in the process can help to create an environment that is 
objective, fair, and rational. Thus, potential investments will be 
judged solely on the merits of their contributions to the strategic 
goals of the organization without undue influence from outside the 
process.

Commitment 2: The organization has documented policies and procedures 
for reselecting ongoing IT investments.

The organization has policies and procedures that outline a structured 
method for identifying, evaluating, prioritizing, and reselecting 
ongoing projects.

A policy-driven, structured method for reselecting ongoing projects for 
further funding can also accomplish several objectives. A structured 
method provides the organization's investment board with a common 
understanding of how ongoing projects will be reselected for continued 
funding. Each ongoing project should be judged based on its success in 
meeting the investment outcomes that were stated in the policies and 
procedures for reselection. The information needed for decisions on 
project reselection should be predefined.

A documented reselection process ensures consistency when an 
organization is considering multiple investments for additional 
funding. Again, transparency in the process will create an environment 
that is objective, fair, and rational. Thus, ongoing investments will 
be judged solely on the merits of their current contributions to the 
strategic goals of the organization without undue influence from 
outside the process.

Commitment 3: The organization has policies and procedures for 
integrating funding with the process of selecting an investment.

The process of selecting investments is not feasible unless the 
policies and procedures for selection and reselection take into account 
how much funding is available for IT investments. No decision to fund a 
project can be considered valid without considering what funds are 
available. It is therefore vitally important to include procedures for 
project funding in the documented policies and procedures for selecting 
investments.

Prerequisites: 

Prerequisite 1: Adequate resources, including people, funding, and 
tools, are provided for identifying and selecting IT projects and 
systems.

These resources typically involve: 

* managerial time and attention to the process, including project 
sponsorship;

* staff support, including, at a minimum, a designated official to 
manage the process; and: 

* supporting tools, methods, and equipment for organizing and analyzing 
the proposals.

Prerequisite 2: Criteria for analyzing, prioritizing, and selecting new 
IT investment opportunities have been established.

The organization has created a process for comparing projects within 
the portfolio of IT investments. Any decision-support process should be 
based on predetermined criteria. In order to maintain consistency, the 
criteria should include quantitative or qualitative measures for 
comparing projects. Projects are compared with one another based on 
criteria such as investment size, project longevity, technical 
difficulty, project risk, business impact, customer needs, cost-benefit 
analysis, organizational impact, and expected improvement. The results 
of such a comparison will help the investment board analyze the 
potential risk and return of investing in a particular project and 
prioritize the portfolio of projects using a scoring mechanism that 
considers strengths and weaknesses. After a careful analysis of the 
various projects vying for funding, senior executives should be able to 
prioritize the list of IT investment proposals based on supporting 
documentation.

Prerequisite 3: Criteria for analyzing, prioritizing, and reselecting 
IT investment opportunities have been established.

The organization has created a process for analyzing and prioritizing 
ongoing projects within its IT investment process. Any decision-support 
process for analyzing ongoing operations and maintenance projects 
should be based on predetermined criteria. There should be consistent 
quantitative or qualitative measures for analyzing projects for 
reselection or, if necessary, termination. If corrective actions cannot 
be implemented to maintain the desired investment outcome, the project 
should be identified, based on developed criteria, for termination. The 
results of such an analysis will help the investment board determine 
the potential risk and return of continuing to fund an ongoing project 
and to prioritize the projects based on decision criteria. After a 
careful analysis of the various ongoing projects competing for 
continued funding, senior executives should be able to prioritize the 
list of existing IT investments for reselection based on supporting 
documentation.

Prerequisite 4: A mechanism exists to ensure that the criteria continue 
to reflect organizational objectives.

The organization has created a process for ensuring that the criteria 
change as organizational objectives change. During project selection, 
decision makers use various criteria to help them assess a system's 
projected outcomes, resource allocations, and benefits and costs. 
Because criteria are usually presented in a hierarchical structure, 
decision makers are able to apply judgments based on the criteria/
objectives deemed important to achieving specific goals. As 
organizational goals and objectives change--and the criteria for 
selecting projects changes with them--decision makers need to have 
management structures and tools in place to help them reassess their 
decision criteria and the effects of those criteria on decisions, 
results, and outcomes.

Activities: 

Activity 1: The organization uses its defined selection process, 
including predefined selection criteria, to select new IT investments.

The organization uses a structured process for submitting IT proposals 
that require funding or organizational support. This activity typically 
occurs within the context of the organization's cyclical budgeting 
process. A designated official manages the data submission and 
screening activities that are associated with the process.

Activity 2: The organization uses the defined selection process, 
including predefined selection criteria, to reselect ongoing IT 
investments.

The part of the process during which organizations tend to need the 
most help is in determining which projects to reselect and which to 
terminate. Competing priorities and differing objectives make it 
extremely difficult for IT decision makers to determine where to 
allocate their scarce IT funds. Faced with a changing laundry list of 
important and potential IT projects that exceeds budget parameters, 
managers need a predefined selection process that will help them choose 
among new and ongoing projects. To help ensure the selection and 
continuation of the most promising projects, ongoing projects should be 
reviewed continually along with new projects and go/no-go decisions 
should be made using predefined selection criteria.

Activity 3: Executives' funding decisions are aligned with selection 
decisions.

The organization's executives have discretion in making the final 
funding decisions on IT proposals. However, their decisions should be 
based upon the analysis that has taken place in the previous 
activities. Additionally, as part of the decision-making process, there 
should be evidence that some proposals are judged less meritorious than 
others and thus do not get funded.

Providing Investment Oversight: 

The purpose of this critical process is to ensure that the organization 
provides effective oversight for its IT projects throughout all phases 
of their life cycles. While the board should not micromanage each 
project in order to provide effective control, it should maintain 
adequate oversight and observe each project's performance and progress 
toward predefined cost and schedule expectations as well as each 
project's anticipated benefits and risk exposure. The board should 
expect that each project development team will be responsible for 
meeting project milestones within the expected cost parameters that 
have been established by the project's business case and cost/benefit 
analysis. The board should also employ early warning systems that 
enable it to take corrective actions at the first sign of cost, 
schedule, and performance slippages.

The investment board has ultimate responsibility for the activities 
within this critical process. However, in larger organizations, the 
board may authorize designated subgroups to carry out some of these 
activities. The investment board must ensure that projects maintain 
alignment with the EA, where one exists.

Figure 11: Providing Investment Oversight: 

[See PDF for image]

[End of figure]

Purpose: 

To review the progress of IT projects and systems, using predefined 
criteria and checkpoints, in meeting cost, schedule, risk, and benefit 
expectations and to take corrective action when these expectations are 
not being met.

Organizational Commitments: 

Commitment 1: The organization has documented policies and procedures 
for management oversight of IT projects and systems.

These policies and procedures typically specify: 

* each investment board's responsibilities when providing investment 
oversight within its domain,

* the procedural rules for the investment board's operation and for 
decision making during project oversight,

* the threshold criteria that the investment board(s) uses when 
analyzing project performance as part of its oversight function 
(threshold is typically based on cost or schedule measures--for 
example, currently more than 10 percent over expected cost--and will be 
a major factor in determining whether to take remedial actions),

* that corrective actions are required when the project deviates or 
varies significantly from the project management plan,

* that changes to the project's commitments to meet cost, schedule, 
performance, or other expectations be made with the involvement of 
affected groups, including: 

* enterprise architecture,

* system engineering,

* software engineering (including all subgroups, such as software 
design),

* hardware engineering,

* project planning and estimating,

* information assurance,

* project stakeholders and champions,

* business units, and: 

* customers and end users.

* that each investment board oversee all changes to new and existing 
project commitments that it has made to individuals and groups external 
to the organization,

* the procedures for escalating/elevating unresolved and/or significant 
issues,

* the conditions under which a project would be terminated and the 
funds redirected to other "successful" projects.

Prerequisites: 

Prerequisite 1: Adequate resources, including people, funding, and 
tools, are provided for IT project oversight.

The organization performs an assessment of the resources needed to 
oversee its IT projects and systems. These resources should include: 

* managers and staff who are assigned specific responsibilities for 
monitoring IT projects and systems,

* tools to support board(s)' oversight operations, which may include 
project summary reports on various metrics and decision support 
applications.

Prerequisite 2: IT projects and systems, including those in steady 
state (operations and maintenance),[Footnote 23] maintain approved 
project management plans that include expected cost and schedule 
milestones and measurable benefit and risk expectations.

Each IT project management team creates and maintains a project 
management plan[Footnote 24] for the project or system for which it is 
responsible. This plan documents a variety of project decisions, 
assumptions, and expectations, including project performance.[Footnote 
25] These expectations could include a cost and schedule baseline 
control system, such as the earned value management system, milestone-
based accomplishment expectations, or other such control systems as are 
commensurate with the project's size, importance, cost, and 
risk.[Footnote 26]

Each project that is in its operations and maintenance (O&M) phase 
should have its own distinct project management plan, one that is 
different from plans for new investments. This requirement is due in 
large part to the differences in how each investment is managed. O&M 
projects typically do not have milestones, and their cost structure is 
more predictable.

Activities: 

Activity 1: Data on actual performance (including cost, schedule, 
benefit, and risk performance) are provided to the appropriate IT 
investment board.

For an organization to establish control of projects in Stage 2, it is 
essential that all performance data including cost, schedule, benefits, 
risks, and system functionality (both expected and actual) for each IT 
project are collected and distributed to the appropriate IT investment 
boards. In addition, to monitor the long-term value of a project or 
system, the organization needs to collect and distribute this 
information to the appropriate IT investment board during agreed-upon 
stages of the project's life cycle.

These performance data may be collected by the board itself or 
collected and distributed in some other manner (e.g., through a 
centralized third party). These data will be key to assisting each IT 
board in its decision making.

IT projects in development, by definition, provide little current 
benefit, but they may provide benefits to the organization upon 
completion. The potential benefits of an IT project are enumerated in 
the project's business case; they are used to conduct an [expected] 
benefit/cost analysis and to persuade executives to select the project 
as a good investment. These potential benefits will be realized after 
implementation is complete. Measuring the actual benefit of a project 
while it is in development is a challenge. One way to measure the 
benefit of development work is to approximate it. Measuring a project's 
actual cost and schedule progression (i.e., evaluating earned value, 
which is a measure of the amount of preplanned work that is actually 
performed in relation to the funds expended) renders an approximate 
value of the project to the organization.

Activity 2: Using verified data, each investment board regularly 
reviews the performance of IT projects and systems against stated 
expectations.

The board typically oversees the project's performance by conducting 
reviews at predetermined checkpoints and/or major milestones, in order 
to interpret the data on project cost and schedule with respect to 
historic project data and expectations.

Project oversight: 

* is conducted at least at the major life cycle milestones for each 
project;

* is managed to limit changes in scope, such as increasing 
functionality requirements (scope creep);

* differs in its degree of depth depending on the size, cost, and 
importance of the project;

* must compare estimated schedule time frames to actual schedules, 
including schedule slippages and/or compressions;

* must compare estimated costs with funds spent or obligated to date, 
any changes in funding, and the impact of these changes; and: 

* ensures that project information and data are valid and that 
corrective actions are verified by qualified and independent audit 
teams, quality assurance groups, or internal verification and 
validation (IV&V) contractors.

Project oversight should also address each of the following project 
management issues: 

* Development/Acquisition. Problems (e.g. contractor management) 
stemming from the selection of a specific project development and 
implementation approach.

* Technical. Technical issues or problems concerning such components as 
hardware, software, or telecommunications.

* Benefits. Evaluation of benefits delivered to date and the 
relationship of the project to specific business objectives.

* Risks. Assessment of the risks encountered to date and how expected 
risks are to be managed.

Activity 3: For each underperforming IT project or system, appropriate 
actions are taken to correct or terminate the project or system in 
accordance with defined criteria and the documented policies and 
procedures for management oversight.

Using estimated and actual cost and schedule data, the organization 
should identify projects that are not meeting their cost and/or 
schedule performance expectations. The following are examples of data 
that could be compared: 

* actual cost data to planned cost data;

* the current number and scope of requirements to the original 
requirements established for the project;

* the current conditions and assumptions to the projects' initial 
assumptions and context; and: 

* the actual performance of the software development organization to 
its specified deliverables (e.g., schedule, costs, functionality, 
technical solutions).

Senior executives should ensure that there is a support and reward 
structure in place for identifying issues and raising them to the 
appropriate decision-making level and that there are no incentives for 
covering up significant problems. Go/no-go criteria can be a helpful 
tool in supporting management oversight.

Activity 4: The investment board regularly tracks the implementation of 
corrective actions for each underperforming project until the actions 
are completed.

The investment board ensures that: 

* corrective actions and related efforts are executed by the project 
management team and tracked by the investment board until the desired 
outcomes occur, and: 

* if the corrective actions are significant enough, an independent 
review is conducted before returning to the original project plan 
(i.e., reinstatement of funding) to ensure that all corrective actions 
have achieved their intended results and to determine whether 
additional changes or modifications are still needed.

Capturing Investment Information: 

To make good IT investment decisions, an organization must be able to 
acquire pertinent information about each investment and store that 
information in a retrievable format, to be used in future investment 
decisions. During this critical process the organization identifies its 
IT assets and creates a comprehensive repository of investment 
information. This repository of IT investment information is used to 
track the organization's IT resources to provide insights and trends 
about major IT cost and management drivers. The information in the 
repository serves to highlight lessons learned and to support current 
and future investment decisions. This critical process may be satisfied 
by the information contained in the current EA, augmented by additional 
information (e.g., financial information, risks, benefits, etc.) that 
the investment board may require to ensure that informed decisions are 
being made.

This repository can take many forms (e.g., a catalog, a list, IT system 
and software inventories, or a balance sheet), but regardless of form, 
the collection method should identify each IT investment and its 
associated components. An organization's "as-is" architecture, along 
with its sequencing plan, can provide a resource for developing a list 
of existing investments. In addition, the EA tool may provide an 
opportunity for gathering all of the necessary information in one 
place. This information does not have to be centrally located; it can 
be managed on a distributed basis. The guiding principle for developing 
the information source is that it should be accessible where it is of 
the most value to those making decisions about IT investments. The 
information is particularly important when executing the critical 
processes for Providing Investment Oversight, Selecting an Investment, 
Creating the Portfolio, and Managing the Succession of Information 
Systems. Additionally, beyond serving as a tool to aid in IT investment 
decision making, the IT information can also assist the organization 
with software licensing management, hardware life cycle management, and 
system architecture plans.

Figure 12: Capturing Investment Information: 

[See PDF for image]

[End of figure]

Purpose: 

To make available to decision makers information to evaluate the 
impacts and opportunities created by proposed (or continuing) IT 
investments.

Organizational Commitments: 

Commitment 1: The organization has documented policies and procedures 
for identifying and collecting information about IT projects and 
systems to support the investment management process.

These policies and procedures typically specify: 

* that responsibility for submitting, updating, and maintaining 
relevant inventory information for each project or asset is explicitly 
assigned;

* the process to be followed for the collection of information, access 
to the information, and support for maintaining the information; and: 

* the data elements required for each IT-related item, including: 

* the cost (e.g., history of actual development costs, annual operating 
and maintenance costs, and expected life cycle costs) of each item;

* the owner of each item;

* the physical location of each item; and: 

* the logical (e.g., architectural) location of each item.

For systems, specific IT data elements could be part of the 
organization's configuration management process. These data elements 
could include schedule data, such as dates of installation, last 
upgrade, last maintenance, and last security patch.

As in other critical processes of the ITIM, large and small 
organizations may implement this key practice differently. For example, 
the amount of administration and supporting infrastructure needed to 
collect information on projects and systems depends in large part on 
the size of the organization. A smaller organization that has a limited 
number of systems may be able to utilize systems that were created for 
other purposes, creating reports on an ad hoc basis. Larger 
organizations, however--in which IT-related information might be 
expected to be more extensive and decentralized--may require a 
dedicated system to acquire the relevant information and make it 
available to decision makers in a more structured manner. In a large, 
decentralized organization the collection and reporting of investment 
information on an ad hoc basis would likely be unmanageable.

Commitment 2: An official is assigned responsibility for ensuring that 
the information collected during project and systems identification 
meets the needs of the investment management process.

A designated official is needed to adequately manage this process. The 
official will ensure that a process is developed and maintained for 
collecting IT investment information so that assets can be accurately 
tracked. Staff or external advisors may be assigned to assist the 
official in conducting IT asset tracking and in verifying and 
validating IT investment data.

Prerequisites: 

Prerequisite 1: Adequate resources, including people, funding, and 
tools, are provided for identifying IT projects and systems and 
collecting relevant investment information about them.

These resources typically involve: 

* managerial attention to the process;

* staff support including, at a minimum, a designated official to 
manage the process; and: 

* supporting tools and equipment for tracking IT assets which may 
include: 

* an IT information database;

* IT data reporting, updating, and query tools; and: 

* a method for communicating changes in IT information to affected 
parties.

Activities: 

Activity 1: The organization's IT projects and systems are identified, 
and specific information is collected to support decisions about them.

A standard, documented procedure is used so that developing and 
maintaining the information is a repeatable event, producing IT data 
that are timely, sufficient, complete, and comparable. The information 
may be prepared by the information systems support component of an 
organization, and the verification and validation may be performed by a 
designated official or by another organizational unit, depending on the 
needs of the organization.

An IT project and system data repository typically includes an 
inventory listing of software licenses, planned IT projects, and 
existing systems with their own unique identifiers. The repository may 
also include information on: 

* how the project or system fits into the EA;

* the organizational unit that is responsible for the project;

* interfaces and dependencies with other projects and systems;

* the current life cycle phase of the project or system (e.g., being 
prototyped, under development, being operated and maintained, etc.) and 
associated life cycle events (e.g., current development, modernization, 
or enhancement efforts under way);

* the costs to date for the project or system and anticipated future 
costs;

* the general category of the project or system (e.g., infrastructure, 
software application, hardware replacement); and: 

* anything else that would be relevant to investment decision making 
about the project or system.

For example, a large project could be implemented using an incremental 
investment approach. Such an approach would require that the project's 
increments or useful segments be identified as part of the repository. 
This information would help decision makers select and prioritize the 
project's useful segments and align them with other projects and 
systems.

Activity 2: The information that has been collected is easily 
accessible and understandable to decision makers and others.

The repository of information about the IT investment is of value only 
to the extent that decision makers and stakeholders can and do use it. 
Knowledge of the contents of the repository by staff and managers 
throughout the organization can help them to avoid duplication of 
effort and reconcile overlapping resources. For example, a report in 
the repository can be used to better manage the licensing of an 
organization's application software by showing individually licensed 
applications that may be candidates for group licensing.

Activity 3: The information repository is used by investment decision 
makers and others to support investment management.

In order to continue to make informed investment decisions, it is 
important to maintain up-to-date information. Maintaining the integrity 
of the repository is important to ensuring that it remains a useful 
decision-making tool. As projects and systems change (i.e., additions, 
updates, and/or deletions), this information should be documented in 
the repository. An individual or organizational unit should be 
designated to maintain the repository.

Stage 3: Developing a Complete Investment Portfolio: 

Figure 13: The ITIM Stages of Maturity with Stage 3 Critical Processes: 

[See PDF for image]

[End of figure]

During Stage 3, the investment board enhances the IT investment 
management process by developing a complete investment portfolio. 
Taking a portfolio perspective enables the organization to consider its 
investments in a comprehensive manner, so that the investments address 
not only the strategic goals, objectives, and mission of the 
organization, but also the impact that projects have on one another. 
The organization develops its IT investment portfolio by combining all 
IT assets, resources, and investments that it owns, considering new 
proposals along with previously funded investments, and identifying the 
appropriate mix and synergies of IT investments that best meet its 
mission needs, organizational needs, technology needs, and priorities 
for improvement. This maturity stage is comprised of the following four 
critical processes: 

* Defining the Portfolio Criteria is the process of developing 
quantitative or qualitative factors such as cost, benefit, schedule, 
and risk in order to compare and select projects for inclusion in the 
investment portfolio(s).

Criteria: IT Assessment Guide (AIMD-10.1.13), 27-29, 45-46 (CCA); OMB 
IT Investment Guide, 7-9.

* Creating the Portfolio is the process of comparing worthwhile 
investments and then combining the investments selected into a funded 
portfolio.

Criteria: IT Assessment Guide (AIMD-10.1.13), 32-35 and 52, (CCA, OMB 
A-94, OMB A-130, OMB M-97-0 (2), Capital Programming Guide, 16-17; 
(CCA, OMB M-97-0 (2), OMB IT Investment Guide, 6-7.

* Portfolio Review is the process that builds upon the Providing 
Investment Oversight critical process from Stage 2 by adding the 
element of portfolio performance to the organization's control process 
activities.

Criteria: IT Assessment Guide (AIMD-10.1.13), 52-55, (CCA, PRA, FASA, 
EO 13011, OMB A-11, Part 3); Information Technology Investment (AIMD-
96-64), 65; IT Assessment Guide (AIMD-10.1.13), 61-62, (CCA, GPRA, CFO, 
OMB A-127, OMB A-123).

* Conducting Postimplementation Reviews (PIR) is the process for 
reviewing IT projects in order to learn from past investments and 
initiatives by comparing actual results to estimates. PIRs also serve 
as vehicles for evaluating the entire ITIM process.

Criteria: IT Assessment Guide (AIMD-10.1.13), 70-72 (CCA, PRA, EO 
13011, GPRA, CFO, OMB A-130); OMB IT Investment Guide, 12; Information 
Technology Investment (AIMD-96-64), 66.

Defining the Portfolio Criteria: 

Portfolio selection criteria are a necessary part of an IT investment 
management process. Developing an IT investment portfolio involves 
defining appropriate IT investment CBSR criteria to ensure that the 
organization's strategic goals, objectives, and mission will be 
satisfied by the selected investments. If an EA, including a sequencing 
plan, exists, it should be used as the foundation for developing and 
updating the portfolio selection criteria. Portfolio selection criteria 
reflect the strategic and enterprisewide focus of the organization and 
build on the criteria that are used to select individual IT projects. 
When IT projects are not considered in the context of a portfolio, 
criteria based on narrow, lower-level requirements may dominate 
enterprisewide selection criteria. IT projects sometimes are selected 
on the basis of an isolated business need, the type and availability of 
funds, or the receptivity of management to a project proposal. 
Portfolio selection criteria build on the criteria that are used to 
select individual projects. The portfolio criteria focus on alignment 
with the organization's mission, organizational strategy, and line-of-
business priorities. In Stage 3, portfolio selection criteria are used 
by the organization's investment board to select IT investments in the 
context of all other investments. These criteria should also be applied 
as uniformly as possible throughout the organization to ensure that 
decision making is consistent and that processes become 
institutionalized. When an organization's mission or business needs and 
strategies change, these criteria should be re-examined.

Figure 14: Defining the Portfolio Criteria: 

[See PDF for image]

[End of figure]

Purpose: 

To ensure that the organization develops and maintains IT portfolio 
selection criteria that support its mission, organizational strategies, 
and business priorities.

Organizational Commitments: 

Commitment 1: The organization has documented policies and procedures 
for creating and modifying IT portfolio selection criteria.

The organization has policies and procedures that outline a systematic 
process for creating and modifying the selection criteria. In smaller 
or highly centralized organizations, there may not be as critical a 
need to institute elaborate polices and procedures to manage portfolio 
criteria. If the investment decision-making process is managed for the 
entire organization by a compact group, and if the objectives for the 
organization's IT investments are well understood and stable, portfolio 
selection criteria might be established once and then modified 
incrementally year-to-year by this same small group. In large, 
decentralized organizations with diverse and evolving objectives, it is 
much more critical to solicit input to the development of portfolio 
selection criteria and to have a documented process for doing so.

For larger organizations, policies and procedures would typically 
specify: 

* the objectives for the portfolio management process;

* a link to the organization's strategic plans, budget processes, and 
enterprise IT architecture;

* the key information elements required to create or modify the 
selection criteria;

* a description of the roles and responsibilities for creating, 
modifying, and prioritizing the selection criteria;

* suggested investment and proposal selection criteria;

* a record of previous selection criteria, their weights and rankings, 
and how they were developed;

* triggers for initiating a change in the selection criteria; and: 

* a list of people to whom the selection criteria should be 
distributed.

Commitment 2: Responsibility is assigned to an individual or group for 
managing the development and modification of the IT portfolio selection 
criteria.

An individual or a working group shall be assigned the responsibility 
of developing the selection criteria and any subsequent modifications 
to those criteria. The assignment of responsibility is critical because 
it creates a point of focus for the successful implementation of this 
critical process. Those individuals who are assigned the task of 
developing and modifying the criteria should have a good working 
knowledge of investment management. Past experience in investment 
management can be beneficial when developing the selection criteria. 
Developing the right criteria with which to analyze a portfolio of 
projects is essential for making sound investment decisions.

Prerequisites: 

Prerequisite 1: Adequate resources, including people, funding, and 
tools, have been committed for portfolio selection criteria activities.

These resources typically involve: 

* the time and attention of the executives involved in the process,

* staff to support the activities within this process, and: 

* supporting tools and equipment.

Prerequisite 2: A working group has been designated to be responsible 
for developing and modifying the IT portfolio selection criteria.

A working group is designated to develop and modify the selection 
criteria. This group should incorporate the organization's mission, 
strategy, and priorities into the criteria. Thus, this group might be 
the IT investment board or a subset of the board that includes the CIO 
or some other member of the executive management team. While a working 
group may develop draft criteria, final approval should fall to the 
investment board or to an individual or group that has been designated 
by the board.

Activities: 

Activity 1: The enterprisewide investment board approves the core IT 
portfolio selection criteria, including cost, benefit, schedule and 
risk (CBSR) criteria, based on the organization's mission, goals, 
strategies, and priorities.

The selection criteria should be linked directly to the organization's 
broader mission, goals, strategies, and priorities. This ensures that 
the selected IT investments will support these larger organizational 
tenets and purposes. It is important that the criteria also take into 
account the organization's IT architecture in orders to (1) avoid 
unwarranted overlap across investments, (2) ensure maximum systems 
interoperability, and (3) increase the assurance that investments align 
with strategy as captured in the EA.

An organization often chooses to establish multiple portfolios to 
facilitate the investment process. This grouping of investments with 
similar characteristics can enable the organization to clarify the 
value of certain types of investments--such as infrastructure or e-
government systems--by developing criteria that focus on the 
contribution each type of investment makes to the organization. Also, 
the organization can determine beforehand how to distribute funding 
across the portfolios. Ultimately, the investment board should assess 
each investment as part of the single enterprise portfolio--that is, 
the aggregation of all of the smaller portfolios. The selection 
criteria used for assessing and ranking individual investments and 
proposals should generally include the four essential investment 
elements: cost, benefit, schedule, and risk. The assessment may also 
include other criteria, which serves to enhance the evaluation of each 
investment's strategic alignment and synergy with other projects. 
Organizations typically focus on these four areas and develop multiple 
measures under each broad element.

* Cost may include life cycle costs broken apart into initial costs, 
ongoing development costs, and indirect costs.

* Benefit may include tangible benefits and intangible benefits 
estimated using a variety of techniques (e.g., cost/benefit analyses 
using net present value, return on investment calculations).

* Schedule may include the life cycle schedule and the schedule of 
benefits.

* Risk may include investment, organizational, funding, and technical 
risks.

The organization must determine how these criteria are to be used to 
select IT investments for the portfolio. Costs and benefits are both 
affected by risks. A risk-adjusted return on investment could combine 
all of these categories. The selection criteria also may include a 
description of an investment's or proposal's minimum or maximum 
acceptable CBSR thresholds (e.g., a minimum acceptable return on 
investment hurdle rate or a maximum acceptable schedule length).

An organization could use a weighting schema when creating the 
selection criteria. The organization would then derive weights for each 
of the broad categories, as well as any subelements related to each 
category. This would help the organization prioritize those subelements 
that it considers the most significant (e.g., an organization that has 
limited experience developing systems may give technical risk a greater 
weight than projected cost). Alternatively, other risk analysis methods 
might incorporate the same "weighting" effect.

The mixture of weights among the ranking criteria will vary from 
organization to organization. The weighting schema used should take 
into account the organization's unique mission, capabilities, and 
limitations. The organization may also create different weighting 
schemas for different kinds of investments (e.g., operational, 
infrastructure, applications development investments, R&D). These 
weights may need to be refined over time as the organization gains more 
operational experience using the weighting schema. Additionally, as a 
starting point, the organization may want to borrow selection criteria 
used by other comparable organizations.

Ultimately, the criteria should reflect the priorities of the 
organization. Often, the most senior investment decision makers are 
involved in the development of these criteria.

Activity 2: Project management personnel and other stakeholders are 
aware of the portfolio selection criteria.

The criteria should be distributed to each IT investment board and all 
of the IT project managers, organizational planners, and any other 
interested parties. The selection criteria should be clearly addressed 
in funding submissions for IT projects.

In a large organization with multiple IT investment boards, a lower-
level board may add its own criteria that would deal with lower-level 
requirements, but the portfolio-level criteria would always take 
precedence.

Activity 3: The enterprisewide investment board regularly reviews the 
IT portfolio selection criteria, using cumulative experience and event-
driven data, and modifies the criteria as appropriate.

The IT criteria for selecting investments may be changed based on 
(1) historical experience; (2) changes in the organization's strategic 
direction, business goals, or priorities; or (3) other factors, such as 
increased IT management capabilities or technological changes. 
Ultimately, however, the task of modifying the criteria will be based 
on the experience and judgment of the enterprisewide investment board.

Creating the Portfolio: 

Individual IT investments vary in type and purpose. Some investments 
may involve purchasing hardware, others developing software, and still 
others operating or maintaining IT systems. The organization may choose 
to organize its investment process by considering investments within 
smaller portfolios (as described in Defining the Portfolio Criteria). 
These subordinate portfolios can help facilitate the prioritization of 
investments within business or service categories. The development of 
the portfolio is an ongoing process that includes decision making, 
prioritization, review, realignment, and reprioritization of projects 
that are competing for resources and funding. The process for creating 
the portfolios should ensure that each IT investment board manages 
investments according to an organizational, strategic-planning 
perspective. The boards should collectively analyze and compare all 
investments and proposals to select those that best fit with the 
strategic business direction, needs, and priorities of the entire 
organization. This is the fundamental process through which investments 
are selected into the portfolio.

Additionally, each organization has practical limits on funding, the 
risks it is willing to take, and the length of time for which it is 
willing to incur costs for a given investment before benefits are 
realized. To address these practical limits, the process of creating 
the portfolio primarily uses categorization to aid in investment 
comparability and CBSR oversight. Categorization involves grouping 
investments and proposals into predefined logical categories. Once this 
is accomplished, investments and proposals can be compared to one 
another within and across the portfolio categories, and the best 
overall portfolio can then be selected for funding.

Fundamental to the comparison of investments is an appropriate analysis 
of each investment. During Stage 2, the primary basis for comparison is 
CBSR, and each investment's performance is compared with those 
dimensions. However, in Stage 3 the basis for comparison expands to 
include more factors related to alignment, such as the degree of 
correlation to the organization's planning, market position, financial 
objectives, and business environment. Also, characteristics of each 
investment that could potentially influence the value of other 
investments in the portfolio--and at the same time be influenced by 
other investments--should be taken into consideration. This process may 
be greatly aided by establishing EA compliance as a fundamental 
requirement for selection and by ensuring that the final portfolio is 
consistent with the EA as a whole.

Figure 15: Creating the Portfolio: 

[See PDF for image]

[End of figure]

Purpose: 

To ensure that IT investments are analyzed according to the 
organization's portfolio selection criteria and to ensure that an 
optimal IT investment portfolio with manageable risks and returns is 
selected and funded.

Organizational Commitments: 

Commitment 1: The organization has documented policies and procedures 
for analyzing, selecting, and maintaining the investment portfolio.

As part of the process for selecting an investment portfolio, each IT 
investment board should have policies and procedures in place to help 
them select the most promising proposals and to ensure that the most 
feasible investments are considered. These policies should include 
specific screening criteria to help identify and expedite the selection 
of the most promising projects. To the extent possible, in order to 
help minimize risk, the organization should have a policy in place to 
ensure that projects are proposed in useful segments or "modules" that 
are short in duration, small in scope, and useful, even though the 
project may, at some point, be discontinued There should also be a 
documented process for reconciling differences between the IT 
investment portfolio and the organization's EA. Reconciliation may 
include an EA waiver or modifying the EA to include the delinquent 
investment. Also, as part of the process for selecting the portfolio, a 
structured and proven investment analysis (e.g. Return on Investment 
and Benefit/Cost Analysis) should be required. The results from the 
analysis should be used to help support portfolio decisions and ensure 
that the organization is aware of the financial as well as other 
internal and external effects.

The organization's policies and procedures for analyzing and developing 
IT investment portfolios typically: 

* provide common definitions for IT investment portfolio categories,

* apply to each IT investment board as each develops its comprehensive 
IT investment portfolio, and: 

* stipulate conditions that should be met for investment funding 
decisions where exceptions are made.

Prerequisites: 

Prerequisite 1: Adequate resources, including people, funding, and 
tools, are provided for the process of creating the portfolio.

These resources typically involve: 

* managerial time and attention to focus on creating the portfolio,

* staff support for carrying out activities within this critical 
process, and: 

* supporting tools and equipment to be used by the staff in creating 
the portfolio.

Prerequisite 2: Board members are knowledgeable about the process of 
creating a portfolio.

Understanding the principles behind the portfolio creation process is 
critical to successfully executing this process. Thus, it may be 
necessary to train board members to ensure that they are familiar with 
the goals of the process and can carry out their responsibilities 
competently.

Knowledge building and/or training may be provided ranging from: 

* in-depth courses for new members to: 

* a mandatory annual overview for all board members of the investment 
process, current process modifications, operational procedures for 
selecting investments, control, and evaluation.

Prerequisite 3: The investment board is provided with information 
comparing project and system performance with expectations.

The organization has defined the common portfolio categories that will 
be used across the organization when each IT board creates its 
portfolio of IT investments (if the organization has more than one 
board). The creation of these common categories (1) aids in the 
comparison of similar investments across the organization and (2) helps 
the boards create a common set of definitions.

Common portfolio categories should enhance decision making during the 
portfolio creation process. The organization should use categories that 
mirror its business strategy and goals. Organizations also need to 
consider their EA when developing their IT portfolio. By using the 
organization's EA framework to identify and establish the "as-is" 
environment, the "to-be" environment, and the transition plan, decision 
makers have an explicit and meaningful structural frame of reference 
for making better IT decisions. For example, the portfolio categories 
might be established by: 

* aligning IT spending with the strategic goals of the organization--
which types of projects, across which groups and which service lines;

* defining spending levels for the portfolio categories, for example, 
XX percent to technology development, XX percent to new services, XX 
percent to infrastructure projects, XX percent to technology 
enhancements and improvements; and: 

* prioritizing IT projects within the portfolio categories.

Establishing portfolio categorization allows projects to be prioritized 
within their own portfolio categories. Moreover, it keeps dissimilar 
projects from competing against each other (for example, O&M projects 
do not compete against new services projects). At the end of the budget 
cycle, resource spending should be more consistent with planned or 
desired IT budgets.

The organization may also want to define a set of thresholds for each 
common portfolio category. These thresholds should be meaningful to the 
organization and useful when making investment decisions, and they 
should differentiate the categories. A small organization with 
relatively few investments may want to use a simple set of portfolio 
categories. An organization using functional categories could define 
thresholds for each category, such as: 

* the maximum investment cost variances, both annually and in total;

* the minimum benefit that a given investment is expected to deliver, 
such as return on investment;

* the maximum length of time an investment should take; and: 

* a maximum risk score derived using an industry-accepted risk 
assessment tool.

Activities: 

Activity 1: Each IT investment board examines the mix of new and 
ongoing investments and their respective data and analyses and selects 
investments for funding.

After the investments have been assigned to portfolio categories, the 
investment board completes the selection process by examining the 
portfolio's mix of investments and making final investment decisions 
that are justified by sound management principles. To provide decision 
makers with an understanding of the relative costs, benefits, 
schedules, and risks of each investment and proposal compared to the 
others, the organization may use a scoring model or decision support 
tool. Typically, such a model or tool compares the costs, benefits, 
schedules, and risks of each investment or proposal against the 
organizational investment criteria and assigns each investment proposal 
a score. These scores may then be used to rank all investments. This 
ranked list of investments may then provide a starting point for the 
decision-makers to apply their judgment and knowledge of the 
organization's imperatives as they select investments for the 
portfolio. (See also GAO's Executive Guide: Measuring Performance and 
Demonstrating Results of Information Technology Investments (AIMD-98-
89, March 1998) for additional guidance on performance measurement.): 

The investment board may have to reconcile imbalances between total IT 
funding expectations and funds required for the qualified IT 
investments within each portfolio category. For example, the investment 
board may find that the funding requests for investments within the O&M 
category are higher than expected and that the funding requests for 
investments within the R&D category are lower than expected. The 
investment board can address this problem by (1) leaving the outcome as 
it is, (2) modifying the mix of investments, (3) modifying investment-
level funding, or (4) using some combination of these options.

The investment board can also use other applicable sources of 
information when comparing investments and determining each 
investment's funding. While the investment board should strongly 
consider the organizational priorities created by the selection 
criteria, it may also want to take into account: 

* the qualifications, abilities, and achievements of the project team;

* the unique or significant links between the investment and the 
organization's mission, strategies, and plans;

* the historical data, data on similar investments, or their own 
investment management experiences; or: 

* other organizational objectives.

Activity 2: Each investment board approves or modifies the performance 
expectations for its selected IT investments.

The board modifies or approves annual performance expectations, 
including CBSR for each investment. Because some investments may span 
multiple years, and most organizations select investments on an annual 
or biennial cycle, the investment board needs to approve each 
investment's performance expectations on an annual basis (e.g., 
performance expectations for a particular investment are to meet or 
exceed the performance goals by the end of the first year). Investment 
expectations should also take into account each investment's past 
performance, in addition to serving as the basis for future board 
reviews, control process activities, and postimplementation reviews.

Activity 3: Information used to select, control, and evaluate the 
portfolio is captured and maintained for future reference.

The organization creates a repository for capturing information (e.g., 
investment performance expectations and portfolio category thresholds) 
related to the portfolio creation process. This repository can be a 
part of a larger IT investment management information system or a 
component of the information collection and may be centrally located or 
distributed within the organization. Storing the information 
facilitates its use as part of control process activities, during 
investment evaluations, future selection decision making, and future 
training for board members.

The ability to effectively capture information on past and present IT 
decisions can enable an organization to make better decisions on IT 
investments during control process activities, as well as during the 
evaluation and selection process. In theory, IT investment decisions 
are only as good as the information that supports them. Data should be 
validated before they are used in the decision-making process. An IT 
information system that delivers information that is up to date, 
encompassing, and presented in a useful format enhances the decision 
process. Organizations sometimes base IT decisions more on judgment, 
intuition, partial data, and ad hoc studies than on objective, 
systematic, IT-related information that is routinely collected and 
analyzed. Organizations should focus on identifying effective 
approaches for collecting, analyzing, and utilizing IT information, as 
well as other strategic information.

Evaluating the Portfolio: 

This critical process builds upon the Stage 2 critical process, 
Providing Investment Oversight, by adding the elements of portfolio 
performance to an organization's investment control capacity. Compared 
to less mature organizations, Stage 3 organizations will have the 
foundation they need to control the risks faced by each investment and 
to deliver benefits that are linked to mission performance. In 
addition, a Stage 3 organization will have the benefit of good 
performance data that have been generated by Stage 2 processes. 
Executive-level oversight of risk management outcomes and incremental 
benefit accumulation provides the organization with increased assurance 
that each IT investment will achieve the desired results. Expanding 
this focus to the entire portfolio provides the organization with 
longer-term assurances that the IT investment portfolio will deliver 
mission value at acceptable cost.

The investment board's role is not to micromanage each investment, but 
instead to ensure appropriate executive-level involvement and 
participation in monitoring each investment's progress toward achieving 
performance expectations. As part of its oversight responsibilities, 
the board should also ensure that the investments in the portfolio 
continue to maintain alignment with the EA, where one exists. These 
investment (and portfolio) expectations are the baseline for periodic 
performance reviews that examine the costs incurred; the benefits 
attained; the current schedule; the accuracy of project reporting; and 
the risks that have been mitigated, eliminated, or accepted to date. 
For this reason, this critical process does not simply focus on, for 
example, the size and attributes of the benefits for a given investment 
because benefit expectations were defined during the process for 
selecting investments. Instead, this process focuses on how the 
investment board monitors and controls the investment portfolio to 
ensure that the overall portfolio provides the maximum benefits at a 
desired cost and at an acceptable level of risk. One way the investment 
board performs this executive-level involvement is by reviewing the 
adequacy of the risk management reviews conducted by the investment 
board's working group.

Organizations have different approaches for managing and controlling 
the performance expectations of projects. In some organizations the 
review board designates a working group to review and make decisions on 
projects based on their performance at specified project checkpoints or 
gates. At these checkpoints, funding and go/no-go decisions are made 
and projects are prioritized based on their investment categories. The 
checkpoints provide for an in-depth review of each project by the 
working group. This approach focuses on designated senior officials 
reviewing the projects throughout the year. The review board 
periodically evaluates the portfolio--once or twice a year--to ensure 
that the working group's decisions are sound. These reviews should be 
conducted from an enterprisewide portfolio perspective, not at the 
individual project level.

Figure 16: Evaluating the Portfolio: 

[See PDF for image]

[End of figure]

Purpose: 

To review the performance of the organization's investment portfolios 
at agreed upon intervals and to adjust the allocation of resources 
among investments as necessary.

Organizational Commitments: 

Commitment 1: The organization has documented policies and procedures 
for reviewing, evaluating, and improving the performance of its 
portfolio(s).

The documented policies and procedures typically specify that the IT 
investment board is responsible for reviewing, evaluating, and 
ultimately improving the performance of the investment portfolio. The 
policies and procedures could designate a working group, composed of 
the project manager, the executive sponsor, and members of the affected 
business unit, to first review performance and then report the results 
to the investment board. Actual investment data should be used as the 
basis for the review. Other potential policies and procedures might 
include having the project manager maintain information on the current 
status of the investment and its performance outcomes to date and the 
scope and frequency of portfolio performance reviews.

The investment board or designated working group should use a 
predetermined performance threshold when analyzing actual versus 
expected performance. This threshold is typically defined on the basis 
of the measures (e.g., more than XX percent over expected cost). 
However, it can include some other significant organization-specific 
factors (e.g., the scope of an investment has grown to reach mission-
critical importance). This predetermined threshold will be a major 
factor in defining the remedial action for underperforming investments. 
Changes to the investment's expectations and commitments are made with 
the involvement and agreement of the stakeholders and in concurrence 
with the investment board.

Prerequisites: 

Prerequisite 1: Adequate resources, including people, funding, and 
tools have been provided for reviewing the investment portfolio and its 
projects.

These resources typically include: 

* staff members for managing information associated with tracking 
investment performance and: 

* tools to support the staff members' activities.

Prerequisite 2: Board members are familiar with the process for 
evaluating and improving the portfolio's performance.

Board members should be familiar with how to evaluate and improve 
portfolio performance. This knowledge is essential to successfully 
executing this process. If board members do not possess prior 
experience in evaluating performance, then training may be necessary to 
ensure that they are familiar with the evaluation process and can carry 
out their responsibilities competently.

Training may be provided, ranging from: 

* in-depth courses for new board members to: 

* a mandatory annual overview for all board members of the investment 
process, current process modifications, and operational procedures for 
investment selection, control, and evaluation.

Prerequisite 3: Results of relevant Providing Investment Oversight 
reviews from Stage 2 are provided to the investment board.

Throughout the control phase of investment management, it is important 
that decision makers have all relevant data on current projects 
available to review. This is a tenet of good portfolio management. The 
data could include any results stemming from periodic project 
oversight. The investment board oversees project performance through 
routine reviews at predetermined checkpoints and/or major milestones. 
The board or a designated third party performs the review. The 
investment board review compares actual results with predetermined 
performance expectations for items such as project cost and adherence 
to schedule.

Prerequisite 4: Criteria for assessing portfolio performance are 
developed, reviewed, and modified at regular intervals to reflect 
current performance expectations.

During the control phase, projects are periodically reviewed to assess 
their performance. Criteria for assessing portfolio performance must 
also be reviewed at regular intervals to reflect current performance 
expectations. Criteria that were developed to assess the original 
investment portfolio might no longer reflect the organization's 
strategic objectives. The IT working group on portfolio selection 
criteria should assess the relevance of the selection criteria at least 
once a year; among other things, the performance of the portfolio 
selected should be assessed against the current criteria.

Activities: 

Activity 1: IT portfolio performance measurement data are defined and 
collected consistent with portfolio performance criteria.

The IT investment board is responsible for monitoring each investment's 
performance. The IT investment board or its designated third party 
examines actual investment performance to date with each investment's 
expectations using the collected investment data. The board is notified 
of and reviews any differences between actual outcomes and 
expectations. Guidelines for executing this activity include: 

* using exception reporting techniques to better manage the volume of 
data produced by this activity,

* designing performance management systems to collect information used 
in this process,

* conducting this review during a formal project review activity,

* documenting annual and life cycle expectations as a basis for the 
comparison, and: 

* using historical organizational performance data and industry 
baseline data as a basis for comparison.

(See Stage 3: Creating the Portfolio for a description of portfolio-
level CBSR expectation setting.): 

Activity 2: Adjustments to the IT investment portfolio are executed in 
response to actual portfolio performance.

It is quite common for adjustments to be made to the investment 
portfolio based on actual performance results. The portfolio should be 
reevaluated on a continuing basis and adjustments made if necessary. 
The investment board identifies IT investments in its portfolio that 
are underperforming by comparing each investment's actual performance 
to stated expectations. If an investment in the portfolio has been 
identified as underperforming, the investment board must take action to 
address its poor performance. If the board is unable to improve the 
performance of the individual investment, it must consider other 
options, including terminating the project early because the goal of 
the investment board is to maintain the optimal performance of the 
portfolio.

The "balanced scorecard" approach is a tool that the IT investment 
board can use to measure the performance of its IT investments. This 
approach creates a measurement balance across the overall performance 
management framework. A balanced approach to measuring the contribution 
of IT to mission outcomes and performance improvement recognizes the 
broad impact of IT's supporting role. By measuring IT performance 
across the four major goal areas (strategic, customer satisfaction, 
internal business performance, and IT innovation and learning), the 
scorecard forces managers to consider measurement within the context of 
the whole organization. This limits the possibility of overemphasizing 
one area of measurement at the expense of others. In addition, 
measuring IT performance from different perspectives helps strengthen 
the analysis of both the tangible and the intangible benefits that are 
attributable to technology. (For more information, see GAO's Executive 
Guide: Measuring Performance and Delivering Results of Information 
Technology Investments, 
[Hyperlink, http: //www.gao.gov/cgi-bin/getrpt?GAO/AIMD-97-163], 
September 1997.): 

Another tool to consider when measuring performance is the Earned Value 
Project Management[Footnote 27] method. This tool measures the actual 
work being performed against a detailed plan in order to accurately 
predict the final costs and schedule results for a given project. The 
tool requires that a plan for project performance measurement be 
created. The earned value, or work performed, is then measured against 
the actual costs of accomplishing the work, providing a measure of the 
project's true cost performance. The method provides project managers 
with a type of "early warning" system, allowing them to take corrective 
action should project spending outpace the physical work being 
accomplished. (Earned Value Project Management, Quentin W. Fleming, 
Project Management Institute, 2000.): 

As a communication tool, organizations may choose to use a spreadsheet 
or graphic illustration such as a management "stoplight," with green, 
yellow, and red identifiers, to summarize performance metrics for each 
investment. The stoplight should be backed up by rigorous controls and 
measures to ensure the reliability and validity of the project metrics. 
A graphic illustration provides a simple way for board members to 
quickly understand the status of each investment and any potential 
emerging problem areas.

Some investments that the board reviews may exceed performance 
expectations (e.g., at lower cost, in less time, and with better 
benefits than expected). In these cases, the board may wish to 
accelerate an investment's funding or schedule, reallocate resources 
within the overall portfolio, or make some other type of adjustment.

Conducting Postimplementation Reviews: 

The purpose of a postimplementation review (PIR) is to evaluate an 
investment after it has completed development (i.e., after its 
transition from the implementation phase to the O&M phase) in order to 
validate actual investment results. This review is conducted to 
(1) examine differences between estimated and actual investment costs 
and benefits and possible ramifications for unplanned funding needs in 
the future and (2) extract "lessons learned" about the investment 
selection and control processes that can be used as the basis for 
management improvements. Similarly, PIRs should be conducted for 
investment projects that were terminated before completion, to readily 
identify potential management and process improvements.

When conducting a PIR, an organization should measure a project's 
success not only against the CBSR presented in the business case, but 
also against the organizational objectives that are associated with the 
project. In certain instances, this is difficult because project 
objectives are not always defined clearly and expressed in quantifiable 
terms that are agreed to by all stakeholders. Even when the objectives 
are precise and are agreed to by all stakeholders, there still might 
not be a unified view of the value generated by undertaking the 
project. Uncontrollable variables can also affect the outcome of a 
project, affecting the criteria by which success is judged. Therefore, 
sound project and investment management are essential when assessing 
the results of a project. In order to judge the project's results 
fairly, the PIR team may have to consider what the objectives were at 
the time the project was initiated.

The timing of a PIR can be problematic--a PIR conducted too soon after 
an investment has been implemented may fail to capture the full 
benefits of the new system. In contrast, if a PIR is conducted too 
late, institutional knowledge about the investment can be lost. As a 
general guideline, PIRs should be conducted within a range of 6 to 18 
months after the investment begins its operational phase. However, this 
guideline should be adjusted depending upon the nature of the project 
and the organization's expectations for when the benefits that are 
documented in the project plans should be realized.

Figure 17: Conducting Postimplementation Reviews: 

[See PDF for image]

[End of figure]

Purpose: 

To compare the results of recently implemented investments with the 
expectations that were set for them and to develop a set of lessons 
learned from these reviews.

Organizational Commitments: 

Commitment 1: The organization has documented policies and procedures 
for conducting PIRs.

The postmplementation review is used to evaluate the overall 
effectiveness of the organization's capital planning and acquisition 
process. The PIR is used to evaluate an investment following its 
implementation in order to validate actual investment results. An 
organization will have difficulty performing an effective PIR unless it 
has established policies and procedures to assess the benefits and 
performance of its investments. Some of the more common policies and 
procedures governing an effective PIR include: 

* who conducts and participates in a PIR;

* types and sizes of investments for which a PIR is conducted;

* when it is appropriate to conduct a PIR;

* what information is presented in a PIR;

* the criteria and procedures for tailoring the standard PIR process;

* how conclusions, lessons learned, and recommended management action 
steps are to be disseminated to executives and others;

* where PIR information and documents are stored (electronically or 
otherwise) for later use; and: 

* when a PIR-like study should be conducted for other IT-related 
initiatives (such as a strategic shift in technology).

A PIR should generally cover: 

* investment expectations;

* actual investment results (e.g., end-user satisfaction, technical 
capability, mission and program impact, unanticipated benefits);

* cost and schedule deviations, such as additional "hidden" costs 
related to investments that have been made to enable the primary 
investment;

* environmental changes that affected the investment;

* a review of the assumptions that were made during the decision-making 
period;

* expected next steps for the investment;

* general conclusions and lessons learned; and: 

* recommendations to executives.

Prerequisites: 

Prerequisite 1: Adequate resources, including people, funding, and 
tools, have been provided for conducting PIRs.

These resources typically involve: 

* assigning a team to prepare and conduct each PIR,

* assigning one team member responsibility for leading the PIR, and: 

* tools to support each PIR, such as: 

* investment documentation in an asset library,

* spreadsheet programs and templates,

* investment planning and scheduling programs, and: 

* risk and benefit assessment methods and tools.

In most cases, the project team should actively assist the PIR team in 
conducting the PIR.

Prerequisite 2: Individuals assigned to the investment board to conduct 
PIRs should be familiar with both the policies and the procedures for 
conducting such reviews.

A PIR's success will largely depend on the credibility and competence 
of the individuals conducting the review. They should be familiar with 
established policies and procedures and be objective, well trained, and 
experienced in conducting PIRs. Training might include a class on 
assessing projects based on some developed criteria. Also, the team 
leader should have past experience conducting similar investment 
reviews.

Activities: 

Activity 1: The investment board identifies which projects will have a 
PIR conducted.

While it may not be necessary to conduct PIRs on all investments, an 
organization should review a mix of successful and less successful 
projects. By capturing this information and sharing it among those 
involved in the investment process, the organization can enhance the 
success of the portfolio as a whole.

In accordance with organizational policy, an IT investment board will 
identify and designate the projects for which a PIR will be conducted. 
Projects need to be selected based on some type of selection criteria. 
Then one or more examining teams will conduct the PIR(s) on the 
selected projects. The standard PIR process may be tailored to the 
specific investment being reviewed.

Often a centralized group (e.g., quality assurance, audit committee, 
etc.) will conduct PIRs under the direction of the enterprisewide IT 
investment board. Owners or users of the investment under review can 
participate in its PIR. This approach enhances the consistency of the 
data collected and ensures coverage of the appropriate projects. 
However, there are other acceptable approaches that may be used. The 
organization should employ the approach that best meets its needs as 
constrained by its resources.

Activity 2: Quantitative and qualitative investment data are collected, 
evaluated for reliability, and analyzed during the PIRs.

As part of the objective analysis of the investment, quantitative PIR 
data are collected. These data should largely arise from the selection 
and control process activities that have been conducted previously. 
Specific types of quantitative data can include: 

* performance expectations and actual outcomes;

* updated performance data and explanations for changes;

* measures of business or mission objectives, such as operating costs, 
schedule, and product cycle time;

* measurements of improved technical capability; and: 

* contribution of the investments toward achieving both the strategy 
and the objectives of the organization's IT strategic plan.

In addition to quantitative investment data, qualitative information, 
such as the perspectives and insights of project participants and end 
users, may serve to validate or raise questions about the quantitative 
information and the existing investment management processes used by 
the organization. Qualitative data can include: 

* surveys and interviews of end users, customers, project management, 
project staff, contractors, and developers;

* project management and staff interviews; and: 

* interviews with senior decision makers involved in investment 
oversight.

Some common techniques for performing analyses during a PIR can 
include: 

* conducting trend analysis using historical investment data,

* conducting a review of how benefits are realized,

* conducting means-end analysis to compare results with known causal 
factors, and: 

* performing force field analysis to understand the effects on the 
investment of major decisions that were made.

Activity 3: Lessons learned and recommendations for improving the 
investment process are developed during the PIR, documented, and then 
distributed to all stakeholders.

The investment board, with the assistance of the project team, conducts 
a review of the investment selection and control process in order to 
extract "lessons learned." Based on these lessons learned, 
recommendations can then be developed to (1) improve the investment 
process (i.e., select, control, and evaluate) and (2) improve the 
management of individual investments. Additional recommendations could 
include improving the development of the business case, refining the 
project/portfolio selection criteria, or determining whether or not the 
objectives of the project are being met.

Once the PIR has been documented, it should be made available to all 
stakeholders and then archived for future reference. A positive outcome 
of identifying lessons learned and developing recommendations is having 
the information available for future reference. Sharing experiences, 
both good and bad, promotes improvements to the overall investment 
process.

Using the collective results of PIRs from IT projects during different 
stages of their life cycles, organizations can learn valuable lessons 
and gain insights. The results from one project will often not provide 
enough information to allow significant modifications to be made to the 
investment decision-making process. PIRs help organizations focus their 
evaluation efforts on areas they deem important and avoid pitfalls and 
problems that others have experienced. Furthermore, PIRs help 
organizations address specific issues that may be impeding progress 
toward the effective management of an IT investment or the development 
of an assessment framework to fine-tune the IT investment process.

Stage 4: Improving the Investment Process: 

Figure 18: The ITIM Stages of Maturity With Stage 4 Critical Processes: 

[See PDF for image]

[End of figure]

The primary focus of Stage 4 is on improving the overall performance of 
an organization's IT portfolio. In addition, the critical processes 
associated with this stage help the organization to manage the 
succession of low-value operating IT systems to higher-value, follow-on 
investments. Thus, this stage comprises the following two critical 
processes: 

* Improving the Portfolio's Performance is the process for evaluating 
the performance of the portfolio and using this information gained from 
the portfolio to improve both current IT investment processes and the 
future performance of the investment portfolio.

Criteria: IT Assessment Guide (AIMD-10.1.13), 73, 78, 80 (CCA, GPRA, 
OMB A-130, OMB A-127, OMB A-123).

* Managing the Succession of Information Systems is the process for 
analyzing and managing the replacement of identified IT investments and 
assets with their higher-value successors.

Criteria: SIM Executive Guide (AIMD-94-115); Year 2000 Computing 
Crisis: An Assessment Guide (AIMD-10.1.14), 10; Capital Programming 
Guide, 54-55.

Improving the Portfolio's Performance: 

Ultimately, an organization needs to know how well its collected pool 
of investments in information management and technology is contributing 
to improvements in mission performance. Improving the Portfolio's 
Performance is, at the level of the investment portfolio, the 
equivalent of Stage 3's PIR for an investment. This critical process 
seeks to determine how well a portfolio of IT investments is 
(1) helping to achieve the strategic needs of the enterprise, 
(2) satisfying the needs of individual units and users with IT products 
and services, and (3) improving IT business performance for users and 
for the enterprise as a whole. To determine these things, performance 
information for an organization's entire portfolio of investments 
should be compiled and analyzed and trends examined.

Key input for these reviews includes PIRs, the IT investment board's 
experiences, and results to date for major investments, extracted from 
control process activities. These data are generally project or 
investment-specific and often are not aggregated for general trend 
analysis.

Figure 19: Improving the Portfolio's Performance: 

[See PDF for image]

[End of figure]

Purpose: 

To assess and improve the performance of the IT investment portfolio 
and the investment management process.

Organizational Commitments: 

Commitment 1: The organization has documented policies and procedures 
for evaluating and improving the performance of its portfolio(s).

These policies and procedures typically specify: 

* that each IT investment board be responsible for managing a 
comprehensive portfolio evaluation and improvement process,

* that access to portfolio data be provided and confidential/sensitive 
data be appropriately controlled,

* that each portfolio be evaluated at least annually to assess its 
performance,

* a mechanism for assembling and aggregating the investment performance 
data,

* the key measures and methods used to assess portfolio performance 
(e.g., a "balanced scorecard" approach),

* methods for analyzing the performance data,

* methods for comparing portfolio performance and portfolio 
expectations, and: 

* a mechanism for reporting the results of the analysis.

Prerequisites: 

Prerequisite 1: Adequate resources are provided for evaluating and 
improving the portfolio's performance.

These resources can include: 

* support staff for executing the activities in this critical process,

* methods and tools to aid the teams conducting the PIRs, and: 

* current and historical portfolio data.

Prerequisite 2: Board members who are responsible for evaluating and 
improving the investment processes and investment portfolio(s) exhibit 
core competencies in evaluating and improving portfolios.

These board members should be familiar with the IT investment 
management approach. Training for this critical process may also 
include familiarizing executives with techniques for economic and 
process management analysis. Training in quality management analysis 
and tools may also be helpful.

Knowledge building and/or training may be provided, ranging from: 

* in-depth courses for new members to: 

* an annual overview, for all board members, of the investment process, 
current process modifications, and operational procedures for 
investment selection, control, and evaluation.

Activities: 

Activity 1: Comprehensive performance measurement data for IT 
portfolios are defined and collected using agreed-upon methods.

The portfolio of investments should be evaluated on its ability to meet 
the strategic needs of the organization, provide general user 
satisfaction with product and service delivery and management, and 
deliver effective and efficient IT business functions (e.g., 
applications development, infrastructure availability, project 
performance). A combination of quantitative data and supporting 
qualitative information can be used to construct a picture of the 
organization's IT portfolio performance. This can be analogous to 
developing a balanced scorecard for IT investments. (For more 
information, see GAO's Executive Guide: Measuring Performance and 
Delivering Results of Information Technology Investments, 
[Hyperlink, http: //www.gao.gov/cgi-bin/getrpt?GAO/AIMD-97-163], 
September 1997.): 

Data collection and information synthesis should focus on answering key 
portfolio performance questions, such as the following: 

* Is IT spending in line with expectations?

* Are we consistently producing cost-effective results?

* How well is the portfolio being managed?

* Are users satisfied with the products and services that are being 
delivered?

* Are IT projects delivering their expected share of process 
improvements?

* How well are integrated project teams being used on major investment 
projects?

* Are quality IT products and services being delivered within general 
industry standards?

* Are accepted methods and tools being used on major systems investment 
projects?

* Is the IT infrastructure providing reliable and needed support for 
the organization?

Measures should be constructed to help objectively determine 
performance outcomes in these areas. In addition, the results of 
individual PIRs, as well as internal and external audits or reviews, 
should be examined. Other types of analyses, such as total cost of 
ownership, can also provide useful performance data on specific IT 
portfolio categories, such as infrastructure O&M.

Activity 2: Aggregate performance data and trends are analyzed.

Trend analysis and reports can help provide evidence that the 
investments in the IT portfolio have helped to achieve expected 
improvements in the effectiveness and efficiency of operations or 
service delivery. To make this a meaningful exercise, it is critical to 
develop baseline performance data.

Activity 3: Recommendations for improving the investment process and 
portfolio are developed and implemented.

Based on its analysis of the data, the organization develops 
recommendations for improving the investment process and the portfolio. 
After the board approves the recommendations, a plan should be put in 
place for implementing them. There are always problems in implementing 
change, but there are opportunities as well. Addressing problems or 
opportunities usually involves: 

* developing recommendations for the IT investment board;

* documenting the decision criteria, justification, and rationale for 
investment decisions;

* defining the expected benefits of the action recommended;

* making a decision on implementing each recommendation; and: 

* tracking the recommended action as it is implemented.

Managing the Succession of Information Systems: 

This critical process develops the capability for (1) planning and 
managing the migration of IT investments to their successors (i.e., 
replacement systems, software applications, and hardware) and 
(2) retiring low-value or high-cost IT investments. This critical 
process also enhances the organization's ability to forecast, plan, and 
manage the migration to new system investments. The target EA and 
sequencing plan can be useful guides in evaluating which investments 
should be phased out and which ones should be retained by the 
organization. Using these guides allows an organization to make 
investment decisions that align with its EA. Such a strategy ultimately 
helps the organization plan for the future by aligning its investment 
decisions with its mission goals and objectives.

This critical process is significant because some IT investments can 
outlive their usefulness and yet acquire organizational inertia or 
entrenchment, consuming resources that begin to outweigh their benefits 
and obscuring the full cost of operations and maintenance. This inertia 
or entrenchment can often occur because these assets (1) have created 
important constituencies within the organization, (2) have a number of 
popular user features, even though the total system cost exceeds the 
total system benefits, or (3) have not had an alternative IT analysis 
performed. Organizations at this maturity stage develop investment 
"exit criteria" such that investments can be "deselected" 
appropriately. This critical process supports a migration to a forward-
looking, solution-oriented view of IT investments.

This process differs from the reselection activity in Stages 2 and 3 in 
that it occurs solely in the O&M phase, whereas reselection can be 
considered during the implementation phase of the system development 
life cycle.

Figure 20: Managing the Succession of Information Systems: 

[See PDF for image]

[End of figure]

Purpose: 

To ensure that IT investments in operation are periodically evaluated 
to determine whether they should be retained, modified, replaced, or 
otherwise disposed of.

Organizational Commitments: 

Commitment 1: The organization has documented policies and procedures 
for managing the IT succession process.

The organization has documented policies and procedures that define how 
IT investments are identified, evaluated, and selected for succession. 
These policies and procedures typically specify: 

* that each investment board make IT replacement decisions within its 
business unit,

* that the enterprisewide investment board have final authority for 
making replacement decisions,

* the coordination of replacement decisions across multiple investment 
boards,

* the procedures for managing the migration of IT systems to their 
successors, and: 

* the procedures for disposing of retired IT systems.

Commitment 2: An official is designated to manage the IT succession 
process.

An official is designated to manage this process. While the IT 
investment board decides which investments to continue, change, 
replace, or retire, this official is responsible for managing the 
succession process and ensuring that the board's plans are executed.

Prerequisites: 

Prerequisite 1: Adequate resources are provided for conducting IT 
succession.

These resources typically involve: 

* the attention of executives involved in this process;

* staff, such as programmers and O&M, to support this process; and: 

* supporting tools and equipment for the staff to use.

Prerequisite 2: Investment board members exhibit core competencies in 
activities involving the succession of information systems.

To make competent replacement decisions, board members must have 
sufficient training to carry out their role. Because this critical 
process is similar in its core concepts to the project selection 
process, the IT succession training can be tied to selection-related 
training.

Knowledge building and/or training may be provided, ranging from: 

* in-depth courses for new members to: 

* an annual overview for all board members of the investment process, 
current process modifications, and operational procedures for 
investment selection, control, and evaluation.

Prerequisite 3: Investment information from the repository is used by 
the investment board.

A repository of investment information helps to ensure that each board 
is aware of all of the investments and resources for which it is 
responsible and of which system owner/manager(s) will be affected by 
system replacement decisions.

(See also Stage 2: Capturing Investment Information for a description 
of the activities associated with developing a collection of IT 
investment information.): 

Activities: 

Activity 1: The investment board develops criteria for identifying IT 
investments that may be ready for replacement.

Each investment board develops the criteria that determine which types 
of investments are candidates for replacement. In an organization with 
multiple boards, the enterprisewide board should formulate the criteria 
first. The criteria should then cascade down to the lower boards. A 
lower level board may have separate criteria for investments strictly 
within its domain.

These candidates for replacement might include investments: 

* at, near, or exceeding their planned life cycles;

* in their O&M phases;

* that have encountered significant data conversion problems;

* that are based significantly on assumptions that are no longer valid 
(e.g., investments that were based on a type of technology that is now 
obsolete); and: 

* for which a replacement application or hardware technology is 
imminent or planned.

(See also Stage 2: Instituting the Investment Board for a description 
of the manner in which multiple investment boards interact.): 

Activity 2: IT investments are periodically evaluated and appropriate 
investments are identified as candidates for replacement.

The defined criteria are applied to the IT portfolio to identify the 
replacement candidates. The analysis will generally be done case by 
case, looking at the continuing business case and mission benefits 
surrounding each candidate and the emerging technologies that are 
possible successors. The analysis should be based on the performance 
factors for each candidate under consideration (e.g., the ongoing costs 
of O&M, the risk of hardware loss due to unavailability of spare 
parts). This analysis may require managerial judgment to determine the 
merits of each particular case or the prospects for a particular 
candidate. Also, it is imperative that the investment's sponsor, 
manager, and/or owner be involved with this activity.

Beyond the normal process of retiring older systems, this activity may 
be triggered by a variety of other events. For example, after 
undergoing a significant strategic realignment or shift in its 
underlying IT architecture, the organization will probably want to 
engage in this activity to ensure that its IT resources are being 
utilized efficiently.

(See also Stage 2: Capturing Investment Information for a description 
of the activities associated with creating an inventory.): 

Activity 3: The interdependency of each investment with other 
investments in the IT portfolio is analyzed.

Some of the investments that are identified as candidates for 
replacement may be interdependent with other investments and projects. 
The purpose of this activity is to identify potential interdependencies 
among investments and to assess the effects that replacing any of these 
investments would have on the others. Contingency plans should be 
prepared to mitigate any negative impact that replacing an investment 
might have on remaining investments. The board may find it necessary to 
revise the replacement plans for some investments based on an analysis 
of the effects replacement could have on other investments.

Activity 4: For each IT investment that has been identified as a 
candidate for replacement, the investment board decides whether or not 
to replace it.

Decisions on whether to replace an investment will usually fall into 
the following categories: 

* Retain/continue. Take no replacement actions and continue to operate 
and maintain the current investment.

* Fix. Propose repairs to the investment so that it once again meets a 
predefined level of performance or business need.

* Enhance/improve. Propose modifications to the investment so that it 
provides greater functionality, lasts longer, or costs less.

* Replace. Propose replacing the investment with a new or different 
investment.

* Combine or disaggregate. Propose combining the functionality or 
technical attributes of one or more investments or break an investment 
apart and manage each piece individually.

* Retire/dispose. Terminate the investment and dispose of it.

Succession plans are implemented as needed to ensure timely and 
effective replacement of investments within the context of the IT 
investment management process.

Stage 5: Leveraging Information Technology for Strategic Outcomes: 

Figure 21: The ITIM Stages of Maturity with Stage 5 Critical Processes: 

[See PDF for image]

[End of figure]

At Stage 5, an organization leverages its IT investment capabilities 
both to anticipate the effects of next-generation information 
technologies and to significantly drive strategic business 
transformation. As organizations harness the capability to run 
effective management processes for constantly selecting, controlling, 
and evaluating IT investment, they can more effectively examine how 
best to institute major business transformations to better achieve 
their missions. These transformations no doubt will include fundamental 
changes made possible through the application of new information 
technologies to support major innovation in customer interaction, 
service delivery mechanisms, and more effective knowledge management. 
One essential success factor is to institute effective processes 
capable of analytically sorting through technology choices of 
increasing complexity.

Organizations at Stage 5 are focused on continuous improvement and 
strategic decision making aimed at anticipating and utilizing 
technology options to drive desired business transformation outcomes. 
Two critical processes are central to this stage.

* Optimizing the Investment Process is the process used to exploit IT 
decision making to improve the value of an IT investment management 
process. Best practices of other organizations are captured to improve 
the IT investment process--leading to world-class outcomes. The focus 
of these activities is cross functional, broad, and strategic in 
nature.

Criteria: CCA, Section 5123 (5); Benchmarking course material from CCI, 
Inc.; Best Practices in Information Technology: How Companies Get the 
Most Value From Exploring Their Digital Investment, James Cortada; The 
Information Paradox: Realizing the Business Benefits of Information 
Technology, John Thorpe; Business Process Improvement: The Breakthrough 
Strategy for Total Quality, Productivity, and Competitiveness, H. James 
Harrington; Better Change: Best Practices for Transforming Your 
Organization, PricewaterhouseCoopers.

* IT-Driven Strategic Business Change is the process for using 
information technology to strategically renovate and transform work 
processes and push the organization to explore new and better ways to 
execute its mission. The CIO should be directly involved in this 
process to ensure that IT strategy is tightly linked to the business 
strategy and that senior management is "on board" with the IT strategy 
so that funding does not become an issue.

Criteria: CCA, Section 5123 (5); Breakthrough Process Redesign: New 
Pathways to Building Customer Value, Charlene Adair and Bruce Murray; 
Transforming the Public Sector, David Osborne and Ted Gaebler; The 
Innovator's Dilemma, Clayton M. Christensen; Quality is Free: The Art 
of Making Quality Certain, Philip B. Crosby.

Optimizing the Investment Process: 

The purpose of this critical process is to measurably improve IT 
investment processes by learning from and adopting the tools, 
techniques, or methods used by best-in-class external organizations. 
Improvements can include using innovative investment oversight tools 
and techniques, changing the mechanics of investment management, or 
improving the "lessons learned" feedback mechanism. This process is 
part of an effort to continually improve the value of the 
organization's IT investments. Aspects of this process, such as 
measurement of the IT investment management process, can be implemented 
in earlier stages; at Stage 5, process measurement becomes an absolute 
necessity.

Process-based benchmarking--the first step in this critical process--is 
a structured technique for measuring an organization's IT investment 
management processes. It is different from traditional measurement-
based benchmarking where an organization compares its performance, 
cost, and cycle time to those of competitors, to industry averages, or 
to a consultant's proprietary data. Once they are benchmarked, an 
organization's IT investment management processes can be modified and 
improved using the tools, techniques, or methods learned from "best-in-
class" organizations. The performance gains resulting from implementing 
these process modifications can be measured and should result in IT 
investment management processes that meet or exceed those of the "best-
in-class" organizations.

Figure 22: Optimizing the Investment Process: 

[See PDF for image]

[End of figure]

Purpose: 

To identify and implement measurable improvements in the organization's 
IT investment management processes so that the processes meet or exceed 
those used by best-in-class organizations.

Organizational Commitments: 

Commitment 1: The organization has documented policies and procedures 
for improving its IT investment management process using benchmarking.

These policies and procedures typically specify the following: 

* As part of the benchmarking activity, performance measurements for 
the IT investment management process are collected and analyzed to form 
a process baseline. This baseline should include: 

* the current, documented IT investment management process,

* performance measurement definitions, and: 

* the expected performance measurement range.

* Historical data are used to analyze current performance.

* Leading practices are identified in external organizations to be used 
as benchmarks for process improvements.

* Significant changes to business processes are approved by senior 
management.

* The baselines and benchmarks are revisited and updated periodically.

Commitment 2: An official is designated to manage the benchmarking 
activities.

The organization designates an official to manage this process. This 
official is responsible for managing the benchmarking activities, 
ensuring that team members are well trained, and serving as the focal 
point for this critical process. Whether this official has benchmarking 
as his or her sole duty may depend on the specific circumstances of the 
organization. A smaller organization, with relatively simple investment 
processes, may not require extensive external benchmarking, and so the 
effort to carry out this critical process may be limited. In a larger 
organization, with complex and multifaceted investment activities, 
benchmarking activities may be extensive and require more focused 
attention by the responsible official.

Prerequisites: 

Prerequisite 1: Adequate resources are provided for conducting process 
benchmarking activities.

These resources can include: 

* external organizations or individuals who are responsible for 
measuring investment process performance and: 

* tools to support investment process measurement.

Prerequisite 2: Organizational managers and staff with responsibilities 
in this area are trained in process benchmarking techniques or are 
experienced in using these techniques.

For the benchmarking results to be valuable and useful, benchmarking 
team members must know how to conduct benchmarking studies. To ensure 
that they are competent, team members must either have recent 
benchmarking expertise or receive training.

Activities: 

Activity 1: Baseline data are collected for the organization's current 
IT investment management processes.

The board tasks a group with measuring the current state of the 
investment management process in order to provide a baseline for 
evaluating expected and actual process changes. Creating this baseline 
usually involves identifying and gathering process data on the 
components of the investment management process. These data typically 
include: 

* the level of resources an organization expends in conducting IT 
investment activities;

* quantitative process results, such as returns on investment and 
tangible benefits achieved;

* qualitative process results, such as measures of customer 
satisfaction and contributions to mission achievement; and: 

* the predefined range of values expected from the performance 
measurement.

Activity 2: External comparable best-in-class IT investment management 
processes are identified and benchmarked.

The purpose of this activity is to find and learn from organizations 
with more efficient and effective investment management processes. 
Tasks for doing this include: 

* identifying best-in-class organizations;

* collecting data from internal, private, and public sources about 
best-in-class organizations;

* visiting several best-in-class organizations;

* developing working relationships with one or more of these 
organizations; and: 

* benchmarking components of the best-in-class organizations' 
investment management processes.

Activity 3: Improvements are made to the organization's investment 
management processes.

Once an organization has learned from the best-in-class external 
organizations, it must apply this knowledge to its own processes. Thus, 
the organization should: 

* decide on improvement goals and expectations;

* develop appropriate target activities that will result in measurable 
process improvement; and: 

* analyze, rank, and choose process improvement activities.

The organization then creates and executes an improvement action plan. 
This plan will vary with the type and scope of the benchmarking 
studies. Executives should review and approve the action plan before 
implementing it so that (1) they are aware of the process changes and 
(2) other parties who may be interested in the research and process 
changes can learn from these initiatives.

Using IT to Drive Strategic Business Change: 

In the earlier ITIM maturity stages, the organization invested in 
information technologies, making certain that a good business case had 
been defined within the context of the IT investment management process 
and its enterprisewide investment portfolio. In Stage 5, the 
organization evolves its investment thinking toward managing IT-driven 
change of the overall business process. IT can provide the opportunity 
to change business processes and leverage the organization's human 
capital.

Information technologies can also provide opportunities for an 
organization to move dramatically in new directions in order to meet 
its goals.

Citizens and countries are using widely available computer encryption 
tools to secure their communications. These tools can be used for 
creating "digital signatures," which can support legally binding 
electronic transactions and help prevent fraud.

The Internet has created opportunities for (1) organizations to "move 
closer" to their customers; (2) business partners to reduce or 
eliminate the need for a third-party distribution network; and 
(3) government agencies to present one common and integrated service 
provider "face" for service requests and service delivery to the 
citizen (thus reducing the need for local offices despite the diversity 
of functions being executed at the agency). In addition, "smart 
munitions"--which can find their targets in any weather, be 
reprogrammed in flight, or be controlled in real time by a human far 
away from the target--are changing the way war is fought for some 
components of the military services.

Once an organization can competently manage its IT investments, it must 
proactively manage the potential of information technologies to 
profoundly influence the organization's strategic direction and 
outlook. The organization must develop mechanisms to actively scan its 
environment for new opportunities to utilize technology. This critical 
process describes the activities associated with employing IT 
investments strategically to change the organization's core processes.

Figure 23: Using IT to Drive Strategic Business Change: 

[See PDF for image]

[End of figure]

Purpose: 

To dramatically improve business outcomes by employing IT investments 
strategically.

Organizational Commitments: 

Commitment 1: The organization has documented policies and procedures 
for conducting IT-driven strategic business change activities.

The purpose of these policies and procedures is to define the 
activities and tasks to be carried out, the roles of the various 
parties when executing this critical process, and how these activities 
relate to the organization's ongoing business activities. Because 
business managers may be resistant to changing current business 
processes based on the promises of new technology, these policies 
should include incentives for management's participation in this 
critical process.

Commitment 2: An official is designated to manage the activities within 
this critical process.

An official is designated to initiate and manage a program to enhance 
awareness of state-of-the-technology and new information technologies, 
and to encourage the use of selected technologies to plan and manage 
changes to the organization's business processes.

Prerequisites: 

Prerequisite: Adequate resources are provided for conducting IT-driven 
strategic business change activities.

These resources typically include: 

* funding support for an IT state-of-the-technology laboratory, test 
center, or library;

* technical information and research;

* funding for employing external experts or reviewers;

* staff support for executing this critical process; and: 

* supporting tools and equipment.

Activities: 

Activity 1: The organization creates and maintains a knowledge base of 
state-of-the-technology IT products and processes.

The organization creates the capacity to follow and understand major 
technological events and trends. This capacity can be generated using 
one of several organizational structures (e.g., an advanced technology 
group, a cross-departmental group of experts, a group of external 
experts, or technology centers of excellence). A designated official is 
charged with managing this group, maintaining the knowledge base, and 
keeping the organization up to date on emerging technologies.

Activity 2: Information technologies with strategic business-changing 
capabilities are identified and evaluated.

Emerging trends, events, and technologies that have the potential to 
improve the organization's business are identified for further study 
(e.g., the growth of the Internet and the World Wide Web or the 
proliferation of wireless forms of communication). Particular attention 
should be paid to breakthrough technologies that have the capacity to 
radically improve the current working environment, business processes, 
products or services, or the organization's relationship with its 
customers (e.g., permitting staff to telecommute or to create "virtual 
communities" across the Internet). Also, to ensure that this activity 
focuses on applicable information technologies, the organization should 
ensure that individuals with business knowledge and experience are 
involved as stakeholders in this activity.

Activity 3: Strategic changes to the business processes are planned and 
implemented based on the capabilities of identified information 
technologies.

Once it has been concluded that a set of technologies offers a 
significant opportunity, senior managers must make the decision to plan 
for and engage in the change to the business processes. If the change 
is significant enough, managers might wish to create a separate 
organizational entity that is (1) uniquely positioned to take advantage 
of the set of technologies and (2) independent of the current way of 
doing business.

As part of planning these changes to the business processes, the 
organization should engage in risk-reducing activities such as pilots, 
simulations, or the development of prototypes. These risk-reducing 
activities are particularly important for large, complex, expensive, or 
important process change initiatives. The organization may also want to 
seek external review or expertise when conducting these process change 
activities. When planning the change, the organization should involve 
stakeholders from business, IT support, oversight, and customer groups.

[End of section]

Appendixes: 

Appendix I: Glossary: 

acquisition: The acquiring by contract, with appropriated funds, of 
supplies or services (including construction) by and for the use of the 
federal government through purchase or lease, whether the supplies or 
services are already in existence or must be created, developed, 
demonstrated, and evaluated. Acquisition begins at the point when 
agency needs are established and includes the description of 
requirements to satisfy agency needs, the solicitation and selection of 
sources, the awarding of contracts, contract financing, contract 
performance, contract administration, and those technical and 
management functions that are directly related to the process of 
fulfilling agency needs by contract.

action plan: A plan derived from recommendations that identifies the 
specific actions that will be taken to improve a process or a project 
and outlines a schedule for implementing those actions.

activity: An ITIM key practice that describes the actions necessary to 
implement a critical process. An activity occurs over time and has 
recognizable results. It typically involves establishing plans and 
procedures, performing the work, tracking it, and taking corrective 
actions as necessary.

alignment: The degree of agreement, conformance, and consistency among 
organizational purpose, vision, and values; structures, systems, and 
processes; and individual skills and behaviors.

assessment: An appraisal by a trained team of professionals to 
determine the state of an organization's current processes and to 
identify the high-priority process-related issues facing an 
organization. An assessment may also result in organizational support 
for process improvement.

asset: Property, funding, technical knowledge, or other valuable item 
owned by an organization. Investments typically create assets.

benchmarking: A structured approach for identifying the best practices 
from industry and government and comparing and adapting them to an 
organization's operations. Such an approach is aimed at identifying 
more efficient and effective processes for achieving intended results 
based on outstanding practices of other organizations.

benefit: A term used to indicate an advantage, profit, or gain attained 
by an individual or organization. Tangible benefits include benefits 
that can be explicitly quantified. Such benefits may include reductions 
in cost, increases in productivity, decreases in cycle time, or 
improvements in quality of service. Intangible benefits include 
benefits that may be easy to identify but that can be difficult to 
quantify. These benefits may include more efficient decision making, 
greater data accuracy, improved data security, reduced customer burden, 
or increased organizational knowledge.

business case: A structured method for organizing and presenting a 
business improvement proposal. Organizational decision makers compare 
business cases when they are deciding to expend resources. A business 
case typically includes an analysis of business process performance and 
associated needs or problems, proposed alternative solutions, 
assumptions, constraints, and a risk-adjusted cost/benefit analysis.

business process: A collection of related structured activities--a 
chain of events--that produces a specific service or product for a 
particular customer or customers.

business process improvement: A systematic disciplined approach that 
critically examines, rethinks, and redesigns mission-delivery 
processes and subprocesses within a process management approach.

Capability Maturity ModelSM: A descriptive model of the stages through 
which an organization progresses as it defines, implements evolves, and 
improves its organizational processes. This model serves as a guide for 
selecting process improvement strategies by facilitating the 
determination of current process capabilities and the identification of 
the issues that are the most critical to quality and process 
improvement.

change management: Those activities involved in (1) defining and 
instilling new values, attitudes, norms, and behaviors within an 
organization that support new ways of doing work and overcome 
resistance to change; (2) building consensus among customers and 
stakeholders on specific changes designed to better meet their needs; 
and (3) planning, testing, and implementing all aspects of the 
transition from one organizational structure or business process to 
another.

cost: A term used to indicate the expenditure of funds for a particular 
investment alternative over an expected time period. Cost may include 
direct and indirect initial costs plus any periodic or continuing costs 
for operation and maintenance.

cost/benefit analysis: A technique used to compare the various costs 
associated with an investment with the benefits that it proposes to 
return. Both tangible and intangible factors should be addressed and 
accounted for in the analysis.

critical process: A structured set of key practices that, when 
performed collectively, contributes to the attainment of a maturity 
stage.

customer: Individual or organizational entity for whom the product or 
service is rendered. The customer may also be the end user.

end users: The individuals or groups who will operate the system for 
its intended purpose when it is deployed.

enterprise architecture (EA): An integrated framework for evolving or 
maintaining existing IT and acquiring new IT to achieve the 
organization's strategic and business goals. A complete enterprise 
architecture should consist of both logical and technical components. 
The logical architecture provides the high-level description of the 
organization's mission, functional requirements, information 
requirements, system components, and information flows among the 
components. The technical architecture defines the specific IT 
standards and rules that will be used to implement the logical 
architecture.

executive sponsor: The executive who champions the investment, sets the 
terms of reference, guides the project team, and receives the progress 
reports and recommendations.

failure: The inability of a system or component to perform its required 
functions within specified performance parameters.

information system: The organized collection, processing, 
transmission, and dissemination of information in accordance with 
defined procedures, whether automated or manual.

information technology (IT): The computers, ancillary equipment, 
software, firmware, and related procedures, services (including support 
services), and other resources that are used by an organization to 
accomplish a function.

institutionalization: The building of corporate culture that supports 
methods, practices, and procedures so that they are the ongoing way of 
doing business.

inventory: An organized and itemized list of assets (e.g., IT products, 
services, or contracts).

IT investment: The expenditure of resources on selected information 
technology or IT-related initiatives with the expectation that the 
benefits from the expenditure will exceed the value of the resources 
expended.

IT investment board: A decision-making body, made up of senior program, 
financial, and information managers, that is responsible for making 
decisions about IT projects and systems based on comparisons and trade-
offs among competing projects, with an emphasis on meeting mission 
goals.

IT investment portfolio: The combination of all IT assets, resources, 
and investments owned or planned by an organization in order to achieve 
its strategic goals, objectives, and mission.

IT management: The processes and procedures used by IT project managers 
to direct, control, administer, and regulate a project team creating an 
IT asset in such a way that the resultant product meets its 
requirements upon delivery.

IT project: An organizational initiative that employs or produces IT or 
IT-related assets. Each project has or will incur costs, expects or 
will realize benefits, has a schedule of project activities and 
deadlines, and has or will incur risks.

key practices: The infrastructures and activities that contribute most 
to the effective implementation and institutionalization of a critical 
process.

succession management: An approach for determining when and how to 
replace current investments with other investments that provide greater 
benefits at lower costs.

maintenance: The process of modifying a system or component after 
delivery to correct faults, improve performance or other attributes, or 
adapt to a changed environment.

maturity model: A model of the stages through which organizations 
progress as they define, implement, evolve, and improve their 
processes. This model serves as a guide for selecting process 
improvement strategies by facilitating the determination of current 
process capabilities and the identification of the issues that are most 
critical to achieving quality and process improvement.

maturity stage: A well-defined evolutionary plateau toward achieving 
mature processes.

milestone: A scheduled event for which some individual is accountable. 
A milestone is typically used to measure progress.

mission: The enduring, chartered, long-term goal(s) of an organization.

modification: A change made to a system or component to improve 
performance or some other attribute or to adapt the system or component 
to function in a changed environment.

need: A capability shortfall such as those documented in a mission 
needs statement, deficiency report, or engineering change proposal. A 
new technology application or breakthrough may create a new expressed 
need by the customer.

organizational commitment: An ITIM key practice that describes the 
management actions that are necessary to ensure that the critical 
process is established and will endure. An organizational commitment 
typically involves establishing organizational policies and senior 
management sponsorship that drive the activities for the critical 
process.

outcome: The actual result, effect, or impact of a business initiative, 
program, or support function. Typically, actual outcomes are compared 
to expected outcomes.

performance measurement: The process of developing measurable 
indicators that can be systematically tracked to assess the progress 
that has been made in achieving predetermined goals and using such 
indicators to assess progress in achieving these goals.

policy: A guiding principle, typically established by senior 
management, that is adopted by an organization to influence and 
determine decisions.

portfolio: see IT investment portfolio.

portfolio management: The combination of practices, tools, and 
techniques that are used to measure, control, and increase the return 
on individual IT investments as well as on an aggregate enterprise 
level.

prerequisite: An ITIM key practice that describes the conditions that 
must exist within an organization to successfully implement a critical 
process. A prerequisite typically involves resources, organizational 
structures, and training. Along with organizational commitment, a 
prerequisite must be in place before a critical process's activities 
can be undertaken successfully.

procedure: A documented description of a sequence of actions to be 
taken in order to perform a given task.

process: A sequence of steps performed for a given purpose.

process maturity: The extent to which a specific process is explicitly 
defined, managed, measured, controlled, and implemented effectively. 
Maturity implies a potential for growth in capability and indicates the 
sophistication of an organization's processes and the consistency with 
which the organization conducts these processes.

project manager: The individual with business responsibility for an 
entire project. This individual typically directs, controls, 
administers, and regulates a project that involves developing or 
acquiring an information system.

project plan: A document that describes the technical and management 
approach to be followed for a project. The plan typically describes the 
work to be done, the resources required, the methods to be used, the 
procedures to be followed, the schedules to be met, and the way that 
the project will be organized.

project team: A group of people, each with assigned responsibilities, 
who work closely together to achieve the shared objective of 
delivering, operating, or maintaining an information system. The 
project team may work together on tasks that are highly interdependent. 
The project team may vary in size from a single individual assigned 
part time to a large organization assigned full time.

purpose: The desired outcome for a critical process.

Return on Investment (ROI): A financial management approach that is 
used to explain how well a project delivers benefits in relation to its 
cost. Several methods are commonly used to calculate a return on 
investment, including: Economic Value Added (EVA), Internal Rate of 
Return (IRR), Net Present Value (NPV), Payback, and the use of nominal 
qualitative measures.

risk: A term used to define the class of factors which (1) have a 
measurable probability of occurring during an investment's life cycle, 
(2) have an associated cost or effect on the investment's output or 
outcome (typically an adverse effect that jeopardizes the success of an 
investment), and (3) have alternatives from which the organization may 
chose.

risk management: An approach for addressing the risks associated with 
an investment. Risk management includes identification, analysis, 
prioritization, and control of risks. Especially critical are those 
techniques that (1) help define preventive measures to reduce the 
probability of negative events' occurring and (2) ensure that 
appropriate countermeasures are deployed to deal successfully with the 
consequences of these events.

schedule: A term used to define the time period corresponding to the 
life of the investment. The investment schedule typically contains 
associated phases and milestones that include: planning, proposal 
generation, acquisition or development, implementation, operations and 
maintenance, and succession.

selection criteria: Factors that are identified for use by an 
investment review board to identify and discriminate among investments 
for subsequent funding.

stakeholder: An individual or group with an interest in the success of 
an organization in delivering intended results and maintaining the 
viability of its products and services. Stakeholders influence 
programs, products, and services.

strategic plan: A document used by an organization to align its 
structure and budget with its priorities, missions, and objectives.

threshold: The limiting acceptable value of a measurement or technical 
parameter, typically a performance requirement.

validation: The process of determining whether or not the product 
delivered at the end of the development process satisfies predefined 
requirements.

verification: The process of determining whether or not the products of 
a given phase of development fulfill the requirements established at 
the start of the phase.

[End of section]

Appendix II: Conducting an ITIM Assessment: 

As described in the early sections of this guidance, ITIM may be used 
as a framework for the continuous improvement of an organization's IT 
investment management processes. It may also be used to evaluate the 
maturity of an organization's investment capabilities. This appendix 
describes the assessment process that individuals and teams may use 
when conducting an organizational review using the Information 
Technology Investment Management (ITIM) framework.

In the ITIM framework, maturity stages are achieved through 
implementation of critical processes. These critical processes were 
derived from our research on leading organizations, our reviews of 
federal agencies, and comments that we received from external reviewers 
during the ITIM development process.

Using ITIM to Assess IT Investment Decision-Making Processes: 

This framework is designed to complement either a formal audit 
methodology or an organization's self-assessment process. Using this 
assessment process, the review team completes the following phases: 

* prepares both itself and the organization for the assessment,

* collects evidence of organization activities,

* evaluates the organization based on the ITIM framework, and briefs 
the organization on its findings.

Using this assessment approach provides a widely accepted, repeatable, 
criteria-based assessment process for organizations to use when they 
are conducting self-assessments of their IT investment management 
capabilities. It also provides the organization with an understanding 
of any gaps in its investment management processes that are identified 
during the assessment. However, before engaging in an assessment, 
individuals and teams should do the following: 

* Become proficient with the ITIM framework.

* Review the related GAO and OMB IT investment guidance (see 
[Hyperlink, http: //www.gao.gov/cgi-bin/getrpt?GAO/AIMD-10.1.13]; 
AIMD-99-32; AIMD-98-89; AIMD-94-115 and OMB A-130; A-11, 
M-97-12; M-97-02). Understanding this past guidance provides greater 
insight into the developmental history, key issues, and critical 
success factors associated with the IT investment approach.

* Become familiar with generally accepted capital decision-making 
approaches and associated analytical tools.

* Gain an understanding, through training or experience, of the basic 
concepts behind the development, maturation, and evolution of 
organizational management skills and capabilities (i.e., maturity 
models).

* Have experience assessing organizations using standardized assessment 
process and tools.

Summary of ITIM Assessment Process: 

Figure 24 summarizes the three phases of the ITIM assessment process. 
Each phase is necessary to ensure that the assessment team and 
organization management have sufficient understanding of the process 
and the ITIM approach, that appropriate evidence is collected to 
support the assessment, and that the conclusions are founded on the 
ITIM framework.

Figure 24: Phases in an ITIM Assessment: 

[See PDF for image]

[End of figure]

Phase 1: Prepare for Assessment: 

Present ITIM Overview Briefing to the Organization The assessment 
process begins with the assessment team (hereafter referred to as the 
team) defining the scope of the assessment (i.e., department, agency, 
or bureau). The scope of the assessment will influence the location of 
the assessment (i.e., the physical place where most of the key people 
and activities are located), who will attend the team's briefing, and 
the extent of documentation required. After the scope of the assessment 
has been defined, the team conducts an overview briefing for the 
organization being assessed (hereafter referred to as the 
organization). This briefing covers the ITIM framework in general, the 
assessment process, and any organization-specific factors that are 
relevant to the job. The purpose of this briefing is to ensure that the 
organization understands: 

* ITIM and the assessment process (including some techniques for 
efficiently and effectively performing an ITIM assessment),

* the anticipated schedule of events,

* the importance of involving the right people,

* the general rules of data collection and evidence, and: 

* the expected reporting process.

In order for the assessment to return value to the organization, it is 
important that senior managers understand the ITIM framework and the 
assessment process. This understanding will facilitate the provision of 
appropriate information during the assessment and will demonstrate 
management commitment to the process. The key factors to consider are 
the role(s) of the participants in the organization's IT investment 
activities and its decision-making process.

The following organizational stakeholders should be considered: 

* the Chief Information Officer (CIO),

* the Deputy CIO,

* representatives from the organization's IT investment board(s),

* representatives from the office of the Chief Financial Officer (CFO),

* representatives from the organization's budget and planning offices, 
and: 

* various IT managers.

This overview briefing should take place sufficiently early in the 
assessment process to allow the organization to learn from the 
presentation and prepare for the assessment. This means that the 
briefing should occur at least a month before the start of on-site 
assessment activities. As a result of the briefing, the organization 
should be able to expedite the assessment by collecting the expected 
documentation, identifying the management processes for observation, 
and providing access to appropriate, relevant staff for interviews.

Evidence of an IT Investment Management Process A central component of 
any ITIM assessment is the team's collection of evidence about the 
organization's IT investment management process. The ITIM framework 
guides the team's collection efforts by listing examples of physical, 
documentary, and testimonial evidence for each ITIM critical process. 
The team should evaluate the variety of material with respect to the 
standards of evidence (sufficient, competent, and relevant) found in 
GAO's Government Auditing Standards, 2003 Revision ([Hyperlink, 
http: //www.gao.gov/cgi-bin/getrpt?GAO-03-673G])--also known 
as the "Yellow Book").

Besides collecting documents, a typical ITIM assessment may include 
interviews with as many as 25 to 50 people as well as numerous group 
discussions and briefings. Even more people can participate through the 
use of assessment instruments such as case studies, questionnaires, and 
surveys (see Conduct Case Study Reviews below). The number of 
participants will depend on the scope of the assessment and the 
organization's size.

Obtain IT Management Overview and Background Information In order for 
the assessment team to have a basis for understanding the existing 
investment processes, and to begin to understand the possible mapping 
of organizational processes on the framework, the organization's 
management should provide the team with an overview of their IT 
investment management process. It is incumbent upon the organization to 
ensure that its representatives have sufficient knowledge and 
experience in managing IT investments within the organization to 
accurately represent the organization and answer questions. The team 
should consider using an organizational liaison for the duration of the 
assessment to assist in identifying and gaining access to knowledgeable 
staff, providing access to and delivering copies of requested 
documentation, and facilitating access to physical evidence.

The organization's overview briefings should provide a high-level 
perspective on how the organization manages its IT investments. The 
briefings are intended to provide the team with the following: 

* An overview of the organization's IT investment management process 
(i.e., what the organization does--especially how it selects, controls, 
and evaluates its IT investments);

* An explanation of the organization's structure (who does what as 
documented in current organizational charts--especially any recent or 
anticipated changes);

* A description of how responsibility, accountability, and authority 
for the IT investment management process are distributed; and: 

* An index of relevant documents (the IT investment management 
processes contained in documented policies, procedures, and guidance, 
etc.). The index should describe how the organization's documents are 
laid out and how they relate to one another other.

The organization should also supply other documents and background 
information to the team to increase the team's efficiency and prevent 
misunderstandings during the assessment process. The following 
information may also be included: 

* a list of current IT investments (often referred to as the investment 
portfolio);

* examples of the data, information, and analyses upon which investment 
decisions are based;

* descriptions of the decisions that are made during the investment 
process;

* an overview of the organization's mission and business processes 
(this may be contained in the organization's current strategic plan);

* a list, with definitions, of terminology unique to the organization; 
and: 

* the organization's current investment performance and process 
improvement plan.

Refine the Assessment Plan Based on the initial information it receives 
from the organization, the team may refine its assessment plan. For 
instance, the team should reach consensus on which critical processes 
and/or maturity stages are applicable.

Phase 2: Collect Evidence: 

Attend Briefings, Conduct Interviews, and Collect Documentary Evidence 
The purpose of this set of activities is to obtain supporting evidence 
in greater depth regarding the organization's implementation of the key 
practices and critical processes and to follow up on issues or 
questions arising from other evidentiary sources to date. The amount of 
additional information to be collected, and the level within the 
organization from which it must be obtained, will depend upon many 
factors, including: 

* the evidence obtained to date,

* the maturity of the organization's management processes,

* the organization's size and complexity, and: 

* the scope of the assessment.

A detailed, revised data collection plan should be developed based on 
the information required and the information that was received in the 
initial overview and background briefings. The team should focus on the 
gaps that remain.

Rather than proceeding sequentially through the critical processes, the 
team may find it more effective and efficient in some situations to use 
other approaches to collect evidence. These alternative approaches can 
include: 

* collecting evidence from one organizational component at a time for 
multiple critical processes (e.g., collect and review all of the IT 
investment-related policies from a central policy review committee);

* collecting evidence for a single stage from multiple organizational 
components (e.g., collect and review all evidence for Stage 2 at one 
time); or: 

* collecting evidence for one ITIM component across all organizational 
components (e.g., collect and review all evidence relating to 
Organizational Commitments).

If the organization states that it is implementing a critical process 
using some set of practices other than the ones described in ITIM, then 
these practices should be: 

* clearly delineated,

* formally approved by the organization, and: 

* convincingly supportive of the intent of the critical process that it 
is supposed to support.

The organization may also provide for the team an in-depth walkthrough 
of specific key practices within a critical process. This would provide 
the team with evidence of a critical process and would also support the 
documentary evidence associated with an assessment of a critical 
process.

Obtain Information on the Investment Process Briefings at this point 
should be focused on those critical processes and key practices that 
were insufficiently documented following the initial background 
briefings. Processes and practices that are known to be missing in the 
organization may be skipped. Presenters should be encouraged to bring 
documentation to the briefings for distribution. In many instances the 
briefings may evolve into discussions as the team focuses on the 
supporting evidence of existing investment management processes.

Conduct Interviews The purpose of these interviews is to collect 
supporting evidence from organization staff who directly participate in 
the IT investment management process (e.g., executives, managers, 
support personnel). Interviewing a variety of organization staff 
assists the team in determining the extent to which the investment 
process policies and procedures have been communicated throughout the 
organization. These interviews should also point the team to other 
documentary evidence (probably located within investment projects) and 
guide the collection of evidence. (Also see Conduct Case Study Reviews 
below.): 

Collect and Review Documentary Evidence The purpose of this step is to 
review the documentary evidence of how investment management processes 
are actually implemented and determine how well the evidence correlates 
to the ITIM key practices. This activity is repeated for each key 
practice that is being carried out by the organization.

The team will typically begin by collecting broad, organization-level 
evidence (e.g., policy planning documents). This evidence will lead the 
team to lower-level, implementation-oriented documentation (e.g., 
meeting notes and working papers). In this step the team will: 

* determine what documentary evidence is available, based on 
information provided at briefings and interviews,

* request or collect documentary evidence,

* evaluate the documentary evidence, and: 

* organize the evidence according to the key practices within the ITIM 
framework.

Consolidate Evidence and Collect Follow-Up Evidence Before the team can 
make rating judgments of the key practices, critical processes, and 
maturity stages under consideration, it must complete the following: 

* determine whether or not the evidence provides a sufficient, 
competent, and relevant basis for making a rating judgment;

* assemble, organize, and analyze the collected evidence and 
consolidate it into a manageable summary of evidence according to the 
ITIM framework; and: 

* determine what follow-up evidence the team requires to make a rating 
judgment and develop a method to collect this evidence. The team must 
also decide how to proceed if (1) there is no other evidence available 
or (2) the available evidence is ambiguous and/or inadequate.

Invariably the team will identify the need for additional analyses or 
follow-up evidence to complete the assessment. To collect this 
evidence, the team can either send written questions or requests for 
specific evidence, or it can conduct follow-up interviews.

Determine the Sufficiency, Competency, and Relevance of the Evidence In 
order to achieve accurate and reliable ratings in the assessment 
process, the following evidence guidelines must be met: 

* There should be sufficient evidence collected from two or more 
(preferably independent) sources to support a rating.

* The evidence must be corroborative and directly relevant or logically 
linked to the key practice and the critical process.

* The evidence must provide adequate coverage and be competent. More 
specifically,

* testimonial evidence must be from interviews with or presentations by 
the staff who perform the related investment management process;

* original documentary evidence must be a direct result of executing 
the investment management process; and: 

* physical observations must be made by team members or other credible, 
unbiased third parties.

Under some circumstances, the team may decide that confirmation from 
three or more separate evidentiary sources is needed. For example, the 
team may realize that a particular individual's interview is 
significant enough that it may cause a critical process to be rated as 
"not implemented." In this case, the team may decide that this 
interview, as a single source of evidence, warrants corroboration from 
other interviews. As a general rule, if there is any doubt about 
whether a rating is valid, the team should initiate additional 
information collection efforts.

Consolidate the Evidence Consolidation helps the team to review and 
organize the large quantity of evidence that is typically acquired 
during an assessment. Evidence consolidation also provides an 
opportunity for the team to share interpretations of the collected 
evidence and enables the team members to develop a consensus on rating.

During evidence consolidation, the team assesses its progress toward 
its goals and reviews the evidence it has collected up to that point. 
While no particular format is mandatory, these steps are typically 
followed (often they are repeated multiple times): 

* Team members index, review, and assess the evidence collected to 
date.

* Team members identify key practices that require further 
clarification.

* Team members share opinions of the sufficiency of the evidence and 
develop preliminary ratings based on team consensus.

If the team cannot reach consensus, it must (1) identify the evidence 
needed to resolve the outstanding issues and (2) generate a plan for 
collecting the needed evidence.

Conduct Case Study Reviews The team may choose specific IT investment 
projects for in-depth reviews to validate organization-level evidence 
and to better understand the organization's IT investment management 
process. The decision to conduct case studies will depend on additional 
evidence is required to document investment processes. By evaluating 
the actual investment processes used with a variety of investment 
projects, the team obtains a clearer picture of: 

* the investment processes as they have actually have been implemented,

* the consistency with which the investment processes are executed,

* evidence of whether the organization's policies and procedures have 
been communicated to the project level,

* the commitment that the organization has to its investment processes, 
and: 

* the beneficial effects that improvements in these processes might 
have on the performance of the organization.

Select Investment Projects The team should select one or more 
investment projects in each major life cycle phase (e.g., R&D, full-
scale development, and O&M). At least one of the cases should include a 
high-cost and/or high-risk investment project. For each project, the 
team should follow the history of the investment project as it has 
cycled through the organization's IT investment process. Projects may 
be selected on the basis of whether the required documentation is 
available, though this approach may bias the conclusions that can be 
drawn from the evidence.

Select Participants The team also needs to determine whom they expect 
to participate in these project-level reviews. In all cases, 
participants should come from the investment projects selected and the 
organizational groups that support those investment projects. It may 
also be necessary to include people from other organizational 
components (e.g., IT investment oversight staff).

Execute the Review(s) These reviews will typically cover the following 
dimensions: 

Process conformance--the degree to which the project being reviewed has 
gone through the organization's IT investment decision-making process.

Data sufficiency, quality, and completeness--the type, accuracy, and 
value of the data used to make investment decisions about the project.

Decisions implemented--the type of decision made and the degree to 
which it has been implemented.

Reconcile Differences In some situations, the results of the case 
studies may contradict the preliminary ratings that the team has 
developed during the assessment of the organization. In this case, the 
team should investigate the contradiction(s), determine their root 
cause, and modify the preliminary rating(s) if necessary. As mentioned 
before, the purpose of the case studies or surveys is to provide 
additional corroborative evidence for the organizational ratings and 
conclusions that are reached during the organization-level ITIM 
assessment.

Phase 3: Final Ratings and Assessment: 

Determine the Final Ratings Once evidence collection is complete, the 
team must assess the consolidated evidence and decide whether each key 
practice, critical process, and maturity stage has been successfully 
executed. The team makes final rating judgments as a group. Developing 
a consensus, so that the majority agrees and no one is opposed, ensures 
that the decision is fair and that all the evidence has been 
considered.

ITIM is a hierarchical framework, so the rating of each higher-level 
component is entirely dependent on the components below it. That is, if 
any key practice is not executed satisfactorily, its corresponding 
critical process is not implemented satisfactorily, and the 
corresponding maturity stage cannot be considered complete. Because of 
this hierarchical prioritization, the team must begin by rating key 
practices and work its way up the hierarchy. The sequence of ratings is 
as follows: 

* First, key practices are rated.

* Next, critical processes are rated.

* Last, the ITIM maturity stage is determined.

The team members should devise a method and a mechanism for tracking 
and documenting the rating judgments as they are being made. Besides 
creating a reproducible "audit trail," these supporting documents are 
useful when delivering summary results.

Determine a Rating for Each Key Practice Key practices are rated as: 

* "executed" or: 

* "not executed.": 

An ITIM key practice is successfully "executed" if (1) the team judges 
that the key aspects of the practice are being carried out by the 
organization or (2) the organization presents the team with convincing 
evidence that an alternative practice achieves the same outcome. An 
ITIM key practice is "not executed" if there are significant weaknesses 
in the organization's execution of the practice and if no adequate 
alternative is in place. If the team has found no evidence of a 
practice during the assessment process, that result would support a key 
practice rating of "not executed.": 

If the team rates a key practice as "not executed," the organization 
should be given an opportunity to produce evidence that might mitigate 
or refute the evidence that indicated this rating. By double-checking, 
the team avoids making ratings based on incorrect information.

Determine a Rating for Each Critical Process Critical processes are 
rated as: 

* "implemented,": 

* "not applicable,": 

* "not implemented," or: 

* "not implemented, but improvements under way.": 

An ITIM critical process is "implemented" if all of its underlying key 
practices are successfully executed or if a satisfactory alternative is 
in place. An ITIM critical process is "not implemented" if there are 
significant weaknesses in the organization's execution of the 
underlying key practices and no adequate alternative is in place. An 
ITIM critical process is "not implemented, but improvements under way" 
if more than half, but not all, of its underlying key practices are 
rated as executed. For example, if well-defined policies and procedures 
have been developed, but no training has been established, the critical 
process would be rated as "not implemented, but improvements underway." 
This rating is intended to indicate that the organization has made 
progress in addressing the critical process, but the work has not been 
completed.

A critical process, like a key practice, can be implemented by 
alternative means. The crucial point in assessing an alternative 
approach is that the techniques used to fulfill the purpose of the 
critical process must be defined, implemented, and institutionalized. 
These are the same criteria that are described in the ITIM framework 
and used to assess the adequacy of an organization's execution of a key 
practice.

Determine the Investment Management Stage All of the critical processes 
within a particular investment management maturity stage--and within 
each lower stage--must be rated as "implemented" or "not applicable" in 
order for the organization to achieve that stage. For example, for an 
organization to be rated as an ITIM Stage 3 organization, all of the 
critical processes within both Stage 2 and Stage 3 must be rated as 
being "implemented" or "not applicable" by the team.

Deliver the Draft Summary Assessment The final step in the assessment 
process is the delivery of draft results to the organization. In 
addition, these draft results can form the basis for the development of 
a full audit report, if one is requested. The draft assessment, 
typically in the form of a briefing, contains: 

* an itemization of ITIM critical processes that have been assessed and 
rated;

* an identification of implemented critical processes, an 
identification of the achieved investment management stage, and 
graphical or summary representations of the above information;

* a rating of each key practice for each critical process that was 
assessed; and: 

* an evidence-based rationale for each rating determination.

The team can use the case study reviews to illustrate the ratings and 
conclusions that the team reached as a result of the assessment. In 
order to focus on the key practices needing improvement, the team 
typically will deliver draft ratings only for key practices that it 
judges to be "not executed." This approach optimizes time overall and 
ensures maximum time is spent corroborating investment management 
weaknesses and collecting additional evidence about them or about other 
areas.

[End of section]

Appendix III: Acknowledgments: 

A number of experts in the field of IT investment management commented 
on the draft revision of this framework. They provided us with a wide 
range of insights and ideas for improvement, and we have sought to 
acknowledge as many of these as possible. We wish to thank the 
reviewers for their encouragement and for sharing their time and 
expertise with us.

Thom Arnsperger: 
Director, Strategic Consulting Services: 
The Morgan-Franklin Corporation: 
Former consultant with MITRE at DOD; 

Dr. Scott Bernard: 
Director of Innovation and Strategy, DigitalNet Government Solutions: 
Assistant Professor and Director, Washington, D.C., Programs, School 
of Information Studies, Syracuse University; 

Dr. John Christian: 
Professor, Information Resources Management College: 
National Defense University: 
Author of ITIM Exposure Draft; 

Hank Conrad: 
Consultant, Eagle One: 
Former CIO of McDonald's Corporation; 

Michael Farber: 
Principal, Booz Allen & Hamilton; 

Jim Grant: 
President, C. G. James & Associates: 
Former Executive Vice President for Systems and Technology at the 
Royal Bank of Canada; 

Maj. Gen. Donald Lasher: 
Senior Executive and Consultant: 
U.S. Army (Ret.) and former CIO at USAA and Department of Interior; 

Bill McVay: 
Vice President for E-Government Solutions, DigitalNet Government 
Solutions: 
Former Deputy Branch Chief for Information Policy and Technology in 
the Office of Electronic Government at the Office of Management and 
Budget; 

Paul Rummell: 
Management Consultant: 
Former CIO of Canada; 

Debra Stouffer: 
Vice President for Strategic Consulting Practice, 
DigitalNet Government Solutions: 
Former Deputy CIO at HUD and former CTO at the Environmental 
Protection Agency; 

Bill Taylor: 
Director, Project Management and Evaluation: 
Office of the Chief Information Officer: 
Department of Housing and Urban Development:

(310447): 

FOOTNOTES

[1] 44 U.S.C. § 3506(h).

[2] The fiscal year 1997 Omnibus Consolidated Appropriations Act, Pub. 
L. 104-208, renamed both Division D (the Federal Acquisition Reform 
Act) and E (the Information Technology Management Reform Act) of the 
1996 DOD Authorization Act, Pub. L. 104-106, as the Clinger-Cohen Act 
of 1996.

[3] 40 U.S.C. §§ 11312-11313.

[4] 44 U.S.C. § 3506(h)(5); 40 U.S.C. §§ 11312-11313.

[5] E-Government Act of 2002, Public Law 107-347 (Dec. 17, 2002).

[6] U.S. General Accounting Office, Assessing Risk and Returns: A Guide 
for Evaluating Federal Agencies' IT Investment Decision-making, GAO-
AIMD-10.1.13 (Washington, D.C.: February 1997).

[7] Capability Maturity Model is a service mark of Carnegie Mellon 
University. 

[8] M. Paulk et al., Capability Maturity Model for Software (Version 
1.1), SEI-93-TR-024.

[9] Giga Information Group, Inc., Total Economic Impact, Part 2: 
Defining and Measuring IT Value (P-1297-009).

[10] U.S. General Accounting Office, Information Technology Investment 
Management: A Framework for Assessing and Improving Process Maturity, 
Exposure Draft, GAO-AIMD-10.1.23 (Washington, D.C.: May 2000).

[11] U.S. General Accounting Office, Information Technology: A 
Framework for Assessing and Improving Enterprise Architecture 
Management (Version 1.1), GAO-03-584G (Washington, D.C.: April 2003).

[12] Evaluating Information Technology Investments, A Practical Guide, 
Executive Office of the President, Office of Management and Budget, 
November 1995, and U.S. General Accounting Office, Assessing Risks and 
Returns: A Guide for Evaluating Federal Agencies' IT Investment 
Decision-making, GAO/AIMD-10.1.13 (Washington D.C.: February 1997).

[13] U.S. General Accounting Office, Information Technology: INS Needs 
to Strengthen Its Investment Management Capability, GAO-01-146 
(Washington, D.C.: Dec. 29, 2000); Information Technology: DLA Needs to 
Strengthen Its Investment Management Capability, GAO-02-314 
(Washington, D.C.: Mar. 15, 2002); United States Postal Service: 
Opportunities to Strengthen IT Investment Management Capabilities, GAO-
03-3 (Washington D.C.: Oct. 15, 2002); U.S. General Accounting Office, 
Bureau of Land Management: Plan Needed to Sustain Progress in 
Establishing IT Investment Management Capabilities, GAO-03-1025 
(Washington, D.C.: Sept. 12, 2003); U.S. General Accounting Office, 
Information Technology: Departmental Leadership Crucial to Success of 
Investment Reforms at Interior, GAO-03-1028 (Washington, D.C.: Sept. 
12, 2003).

[14] U.S. General Accounting Office, Executive Guide: Improving Mission 
Performance Through Strategic Information Management and Technology, 
GAO/AIMD-94-115 (Washington, D.C.: May 1994).

[15] Evaluating Information Technology Investments, A Practical Guide, 
Executive Office of the President, Office of Management and Budget, 
November 1995.

[16] U.S. General Accounting Office, Assessing Risks and Returns: A 
Guide for Evaluating Federal Agencies' IT Investment Decision-making, 
GAO/AIMD-10.1.13 (Washington, D.C.: February 1997).

[17] These principles were derived from the principles found in SEI's 
Software Acquisition Capability Maturity Model.SM

[18] For additional guidance on developing performance measures, see 
U.S. General Accounting Office, Executive Guide: Measuring Performance 
and Demonstrating Results of Information Technology Investment, GAO/
AIMD-98-89 (Washington D.C.: March 1998).

[19] For more information on procurement within the context of a 
capital budget, see OMB's Capital Programming Guide, Version 1.0 (July 
1997).

[20] U.S. General Accounting Office, Assessing Risk and Returns: A 
Guide for Evaluating Federal Agencies' IT Investment Decision-making, 
GAO-AIMD-10.1.13 (Washington, D.C.: February 1997); U.S. General 
Accounting Office, Assessing Risk and Returns: A Guide for Evaluating 
Federal Agencies' IT Investment Decision-making, GAO-AIMD-10.1.13 
(Washington, D.C.: February 1997); Evaluating Information Technology 
Investments, A Practical Guide, Executive Office of the President, 
Office of Management and Budget, November 1995.

[21] The organization head typically has ultimate responsibility for 
submitting a budget request to the organization's authorizing budget 
office (e.g., OMB in the federal government) and thus may rework or 
adjust the IT budget recommendations made by the investment board. An 
effective investment environment exists when the organization head with 
the senior executives and the CIO exhibit a corporate responsibility 
and serve as corporate officers on the investment board, instead of 
competing for their individual interests. 

[22] U.S. General Accounting Office, Assessing Risks and Returns: A 
Guide for Evaluating Federal Agencies' IT Investment Decision-making 
GAO/AIMD-10.1.13 (Washington D.C.: February 1997); U.S. General 
Accounting Office, Executive Guide: Improving Mission Performance 
Through Strategic Information Management and Technology (GAO/AIMD-94-
115, May 1994); Evaluating Information Technology Investments, A 
Practical Guide, Executive Office of the President, Office of 
Management and Budget, November 1995. Capital Programming Guide, 
version 1.0, Office of Management and Budget, (July 1997). E-Government 
Act of 2002, Public Law 107-347 (Dec. 17, 2002).

[23] "Steady state" means maintenance and operation costs at current 
capability and performance level, including costs for personnel, 
maintenance of existing information systems, corrective software 
maintenance, voice and data communications maintenance, and replacement 
of broken IT equipment.

[24] See IEEE 1058 Standard for Software Project Management Plans for 
an example of additional guidance on creating a project management 
plan.

[25] See U.S. General Accounting Office, Executive Guide: Measuring 
Performance and Demonstrating Results of Information Technology 
Investments, GAO/AIMD-98-89 (Washington D.C.: March 1998) for 
additional guidance on performance measurement.

[26] For additional guidance on earned value management, see the 
Defense Department's Earned Value Management Web site at http: //
www.acq.osd.mil/pm and Capital Programming Guide Supplement to Part 7 
of Circular No. A-11 (Office of Management and Budget Circular No. A-
11, July 2003).

[27] For additional guidance on earned value management, see the 
Defense Department's Earned Value Management Web site at http: //
www.acq.osd.mil/pm and Capital Programming Guide Supplement to Part 7 
of Circular No. A-11 (Office of Management and Budget Circular No. A-
11, July 2003).

GAO's Mission: 

The General Accounting Office, the investigative arm of Congress, 
exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics.

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading.

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. General Accounting Office

441 G Street NW,

Room LM Washington,

D.C. 20548: 

To order by Phone: 

 Voice: (202) 512-6000: 

 TDD: (202) 512-2537: 

 Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.

General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.

20548: