This is the accessible text file for GAO report number GAO-04-478T entitled 'Information Technology Management: Improvements Needed in Strategic Planning, Performance Measurement, and Investment Management Governmentwide' which was released on March 3, 2004. This text file was formatted by the U.S. General Accounting Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Testimony: Before the Subcommittee on Technology, Information Policy, Intergovernmental Relations and the Census, Committee on Government Reform, House of Representatives: United States General Accounting Office: GAO: For Release on Delivery Expected at 1:00 p.m. EST: Wednesday, March 3, 2004: Information Technology Management: Improvements Needed in Strategic Planning, Performance Measurement, and Investment Management Governmentwide: Statement of David A. Powner, Director, Information Technology Management Issues: GAO-04-478T: GAO Highlights: Highlights of GAO-04-478T, testimony before the Subcommittee on Technology, Information Policy, Intergovernmental Relations and the Census, Committee on Government Reform, House of Representatives Why GAO Did This Study: The federal government spends billions of dollars annually on information technology (IT) investments that are critical to the effective implementation of major government programs. To help agencies effectively manage their substantial IT investments, the Congress has established a statutory framework of requirements and roles and responsibilities relating to information and technology management, that addresses, for example, (1) IT strategic planning/ performance measurement (which defines what an organization seeks to accomplish, identifies the strategies it will use to achieve desired results, and then determines how well it is succeeding in reaching results-oriented goals and achieving objectives) and (2) IT investment management (which involves selecting, controlling, and evaluating investments). GAO was asked to summarize its January 2004 report on IT strategic planning/performance measurement and investment management (Information Technology Management: Governmentwide Strategic Planning, Performance Measurement, and Investment Management Can Be Further Improved, GAO-04-49, January 12, 2004) and to discuss how agencies can improve their performance in these areas. What GAO Found: GAO recently reported that the use of important IT strategic planning/ performance measurement and investment management practices by 26 major federal agencies was mixed (see figure below). For example, agencies generally had IT strategic plans and goals, but these goals were not always linked to specific performance measures that were tracked. Agencies also largely had IT investment management boards, but no agency had the practices associated with the oversight of IT investments fully in place. Although they could not always provide an explanation, agencies cited a variety of reasons for not having practices fully in place, including that the chief information officer position had been vacant and that the process was being revised. By improving their IT strategic planning, performance measurement, and investment management, agencies can better ensure that they are being responsible stewards of the billions of dollars for IT that they have been entrusted with through the wise investment of these monies. To help agencies improve in these areas, GAO has made numerous recommendations to agencies and issued guidance. For example, in the January 2004 report, GAO made recommendations to the 26 agencies regarding practices that were not fully in place. In addition, today GAO is releasing the latest version of its Information Technology Investment Management (ITIM) framework, which identifies critical processes for selecting, controlling, and evaluating IT investments and organizes them into a framework of increasingly mature stages; thereby providing agencies a road map for improving IT investment management processes in a systematic and organized manner. www.gao.gov/cgi-bin/getrpt?GAO-04-478T. To view the full product, click on the link above. For more information, contact David Powner at (202) 512-9286 or pownerd@gao.gov. [End of section] Mr. Chairman and Members of the Subcommittee: Thank you for the opportunity to join in today's hearing on the government's information technology (IT) management. This is a critical topic because, according to the President's most recent budget, the federal government spends billions of dollars annually on IT-- reportedly investing about $57 billion in fiscal year 2003.[Footnote 1] Yet these dollars are not always managed wisely. For example, the Administration reported that of the $60 billion in IT investments requested for fiscal year 2005, $22 billion--representing 621 major projects--are currently on its "Management Watch List."[Footnote 2] This list includes mission-critical projects that need improvement in the areas of performance measures, earned value management,[Footnote 3] and/or IT security. To help agencies effectively manage their substantial IT investments, the Congress has established a statutory framework of requirements and roles and responsibilities relating to information and technology management through laws such as the Paperwork Reduction Act of 1995[Footnote 4] and the Clinger-Cohen Act of 1996. This framework addresses, for example, IT strategic planning/performance measurement (which defines what an organization seeks to accomplish, identifies the strategies it will use to achieve desired results, and then determines how well it is succeeding in reaching results-oriented goals and achieving objectives), and investment management (which involves selecting, [Footnote 5] controlling,[Footnote 6] and evaluating [Footnote 7] investments). At your request, today I will summarize our recently issued report[Footnote 8] on the extent to which 26 agencies[Footnote 9] had in place 30 important practices associated with key legislative and other requirements for IT strategic planning/performance measurement and IT investment management (app. I lists the 30 practices). I will also discuss how agencies can improve their performance in these areas. Results in Brief: The use of important IT strategic planning/performance measurement and investment management practices--identified based on legislation, policy, and guidance--by the agencies in our review was mixed; collectively the agencies had less than 50 percent of the practices fully in place. For example, agencies generally had IT strategic plans and goals, but these goals were not always linked to specific performance measures that were tracked. Without enterprisewide performance measures that are tracked against actual results, agencies lack critical information about whether their overall IT activities are achieving expected goals. In the investment management area, the agencies largely had IT investment management boards, but no agency had the practices associated with the oversight of IT investments fully in place. Executive-level oversight of project-level management activities provides organizations with increased assurance that each investment will achieve the desired cost, benefit, and schedule results. Although they could not always provide an explanation, agencies cited a variety of reasons for not having practices fully in place, such as that the chief information officer (CIO) position had been vacant and that their process was being revised. Regardless of the reason, these practices are important ingredients for ensuring effective strategic planning, performance measurement, and investment management, which, in turn, make it more likely that the billions of dollars in government IT investments will not be wasted. To help agencies improve their performance in the IT strategic planning/performance measurement and IT investment management areas, we made numerous recommendations to each of the 26 agencies we reviewed. In addition, at today's hearing we are releasing the latest version of our Information Technology Investment Management (ITIM) framework.[Footnote 10] First issued as an exposure draft in May 2000, this version of the ITIM includes lessons learned from our use of the framework in our agency reviews and by users of the framework. The framework identifies critical processes for the successful selection, control, and evaluation of IT investments and organizes them into a framework of increasingly mature stages. ITIM offers organizations a road map for improving their IT investment management processes in a systematic and organized manner. Background: Advances in the use of IT and the Internet are continuing to change the way that federal agencies communicate, use, and disseminate information; deliver services; and conduct business. For example, electronic government (e-government) has the potential to help build better relationships between government and the public by facilitating timely and efficient interaction with citizens. To help agencies more effectively manage IT, the Congress has established a statutory framework of requirements and roles and responsibilities relating to information and technology management. In particular, the Paperwork Reduction Act of 1995 and the Clinger-Cohen Act of 1996 require agency heads, acting through agency CIOs to, among other things, * better link their IT planning and investment decisions to program missions and goals; * develop and maintain a strategic information resources management (IRM) plan that describes how IRM activities help to accomplish agency missions; * develop and maintain an ongoing process to establish goals for improving IRM's contribution to program productivity, efficiency, and effectiveness; methods for measuring progress toward these goals; and clear roles and responsibilities for achieving these goals; * develop and implement a sound IT architecture; * implement and enforce IT management policies, procedures, standards, and guidelines; * establish policies and procedures for ensuring that IT systems provide reliable, consistent, and timely financial or program performance data; and: * implement and enforce applicable policies, procedures, standards, and guidelines on privacy, security, disclosure, and information sharing. Nevertheless, the agencies face significant challenges in effectively planning for and managing their IT. Such challenges can be overcome through the use of a systematic and robust management approach that addresses critical elements such as IT strategic planning and investment management. Agencies Did Not Always Have Strategic Planning/Performance Measurement and Investment Management Practices in Place: Federal agencies did not always have in place important practices associated with IT laws, policies, and guidance related to strategic planning/performance measurement and investment management (see fig. 1). A well-defined strategic planning process helps to ensure that an agency's IT goals are aligned with its strategic goals. Moreover, establishing performance measures and monitoring actual-versus- expected performance using those measures can help to determine whether IT is making a difference in improving performance. Finally, an IT investment management process is an integrated approach to managing investments that provides for the continuous identification, selection, control, life-cycle management, and evaluation of IT investments. Figure 1: Percentage of Agencies' Use of 12 IT Strategic Planning/ Performance Measurement Practices (left) and 18 Investment Management Practices (right)A: [See PDF for image] [A] Percentages do not add to 100 percent due to rounding. Note: Yes--the practice was in place. Partially--the agency has some, but not all, aspects of the practice in place. Examples of circumstances in which the agency would receive this designation include when (1) some, but not all, of the elements of the practice were in place; (2) the agency documented that it has the information or process in place but it was not in the prescribed form (e.g., in a specific document as required by law or the Office of Management and Budget); (3) the agency's documentation was in draft form; or (4) the agency had a policy related to the practice, but evidence supported that it had not been completely or consistently implemented. No--the practice was not in place. Not applicable--the practice was not relevant to the agency's particular circumstances. [End of figure] Agency IT officials could not always identify why practices were not in place, but in those instances in which reasons were identified, a variety of explanations were provided; for example, that the CIO position had been vacant, that not including a requirement in the agency's guidance was an oversight, or that the process was being revised. Nevertheless, these practices are based on law, executive orders, Office of Management and Budget (OMB) policies, and our guidance, and are also important ingredients in ensuring effective strategic planning, performance measurement, and investment management that, in turn, make it more likely that the billions of dollars in government IT investments will be wisely spent. Agencies' Use of IT Strategic Planning/Performance Measurement Practices Was Uneven: Critical aspects of the strategic planning/performance measurement area include documenting the agency's IT strategic planning processes, developing IRM plans, establishing goals, and measuring performance to evaluate whether goals are being met. Although the agencies often had these practices, or elements of these practices, in place, additional work remains, as demonstrated by the following examples: * Strategic planning process. Strategic planning defines what an organization seeks to accomplish and identifies the strategies it will use to achieve desired results. A defined strategic planning process allows an agency to clearly articulate its strategic direction and to establish linkages among planning elements such as goals, objectives, and strategies. About half of the agencies had fully documented their strategic planning processes. Such processes are an essential foundation for ensuring that IT resources are effectively managed. * Strategic IRM plans. The Paperwork Reduction Act requires that agencies indicate in strategic IRM plans how they are applying information resources to improve the productivity, efficiency, and effectiveness of government programs. An important element of a strategic plan is that it presents an integrated system of high-level decisions that are reached through a formal, visible process. The Paperwork Reduction Act also requires agencies to develop IRM plans in accordance with OMB's guidance. However, OMB does not provide cohesive guidance on the specific contents of IRM plans. Accordingly, although agencies generally provided OMB with a variety of planning documents to meet its requirement that they submit an IRM plan, these plans were generally limited to IT strategic or e-government issues and did not address other elements of IRM, as defined by the Paperwork Reduction Act. In particular, these plans generally include individual IT projects and initiatives, security, and enterprise architecture elements but do not often address other information functions--such as information collection, records management, and privacy--or the coordinated management of all information functions. OMB IT staff agreed that the agency has not set forth guidance on the contents of agency IRM plans in a single place, stating that its focus has been on looking at agencies' cumulative results and not on planning documents. These staff also noted that agencies account for their IRM activities through multiple documents (e.g., Information Collection Budgets[Footnote 11] and Government Paperwork Elimination Act[Footnote 12] plans). Nevertheless, half the agencies indicated a need for OMB to provide additional guidance on the development and content of IRM plans. Accordingly, we recommended that OMB develop and disseminate to agencies guidance on developing IRM plans. * IT goals. The Paperwork Reduction Act and the Clinger-Cohen Act require agencies to establish goals that address how IT contributes to program productivity, efficiency, effectiveness, and service delivery to the public. We have previously reported that leading organizations define specific goals, objectives, and measures, use a diversity of measure types, and describe how IT outputs and outcomes impact operational customer and agency program delivery requirements.[Footnote 13] The agencies generally had the types of goals outlined in the Paperwork Reduction Act and the Clinger-Cohen Act. However, five agencies did not have one or more of the goals required by the Paperwork Reduction Act and the Clinger-Cohen Act. It is important that agencies specify clear goals and objectives to set the focus and direction for IT performance. * IT performance measures. The Paperwork Reduction Act, the Clinger- Cohen Act, and an executive order[Footnote 14] require agencies to establish a variety of IT performance measures--such as those related to how IT contributes to program productivity, efficiency, and effectiveness--and to monitor the actual-versus-expected performance using those measures. Although the agencies largely had one or more of the required performance measures in place, these measures were not always linked to the agencies' enterprisewide IT goals. Moreover, few agencies monitored actual-versus-expected performance for all of their enterprisewide IT goals. Specifically, although some agencies tracked actual-versus-expected outcomes for the IT performance measures in their performance plans or accountability reports and/or for specific IT projects, they generally did not track the performance measures that were specified in their IRM plans. As we have previously reported, an effective IT performance management system offers a variety of benefits, including serving as an early warning indicator of problems and the effectiveness of corrective actions; providing input to resource allocation and planning; and providing periodic feedback to employees, customers, stakeholders, and the general public about the quality, quantity, cost, and timeliness of products and services.[Footnote 15] Moreover, without enterprisewide performance measures that are tracked against actual results, agencies lack critical information about whether their overall IT activities are achieving expected goals. * Benchmarking. The Clinger-Cohen Act requires agencies to quantitatively benchmark agency process performance against public-and private-sector organizations, where comparable processes and organizations exist. Benchmarking is used because there may be external organizations that have more innovative or more efficient processes than their own processes. Seven agencies in our review had mechanisms in place--such as policies and strategies--related to benchmarking their IT processes. In general, however, agencies' benchmarking decisions were ad hoc. Few agencies had developed a mechanism to identify comparable external private-or public-sector organizations and processes and/or had policies related to benchmarking, although all but 10 of the agencies provided examples of benchmarking that they had performed. Our previous study of IT performance measurement at leading organizations found that they had spent considerable time and effort comparing their performance information with that of other organizations.[Footnote 16] Agency IT officials could not identify why strategic planning/ performance measurement practices were not in place in all cases, but in those instances in which reasons were identified, a variety of explanations were provided. For example, reasons cited by agency IT officials included that they lacked the support from agency leadership, that the agency had not been developing IRM plans until recently and recognized that the plan needed further refinement, that the process was being revised, and that requirements were evolving. Without strong strategic management practices, it is less likely that IT is being used to maximize improvement in mission performance. Moreover, without enterprisewide performance measures that are being tracked against actual results, agencies lack critical information about whether their overall IT activities, at a governmentwide cost of billions of dollars annually, are achieving expected goals. Agencies' Use of IT Investment Management Practices Was Mixed: Critical aspects of IT investment management include developing well- supported proposals, establishing investment management boards, and selecting and controlling IT investments. The agencies' use of practices associated with these aspects of investment management was wide-ranging, as follows: * IT investment proposals. Various legislative requirements, an executive order, and OMB policies provide minimum standards that govern agencies' consideration of IT investments. In addition, we have issued guidance to agencies for selecting, controlling, and evaluating IT investments.[Footnote 17] Such processes help ensure, for example, that investments are cost-beneficial and meet mission needs and that the most appropriate development or acquisition approach is chosen. The agencies in our review had mixed results when evaluated against these various criteria. For example, the agencies almost always required that proposed investments demonstrate that they support the agency's business needs, are cost-beneficial, address security issues, and consider alternatives. However, they were not as likely to have fully in place the Clinger-Cohen Act requirement that agencies follow, to the maximum extent practicable, a modular, or incremental, approach when investing in IT projects. Incremental investment helps to mitigate the risks inherent in large IT acquisitions/developments by breaking apart a single large project into smaller, independently useful components with known and defined relationships and dependencies. * Investment management boards. Our investment management guide states that establishing one or more IT investment board(s) is a key component of the investment management process. Such executive-level boards, made up of business-unit executives, concentrate management's attention on assessing and managing risks and regulating the trade-offs between continuing to fund existing operations and developing new performance capabilities. Almost all of the agencies in our review had one or more enterprise-level investment management board. However, the investment management boards for six agencies were not involved, or the agency did not document the boards' involvement, in the control phase. Maintaining responsibility for oversight with the same body that selected the investment is crucial to fostering a culture of accountability by holding the investment board that initially selected an investment responsible for its ongoing success. * Selection of IT investments. During the selection phase of an IT investment management process, the organization (1) selects projects that will best support its mission needs and (2) identifies and analyzes each project's risks and returns before committing significant funds. To achieve desired results, it is important that agencies have a selection process that, for example, uses selection criteria to choose the IT investments that best support the organization's mission and that prioritizes proposals. Twenty-two agencies used selection criteria in choosing their IT investments. In addition, about half the agencies used scoring models[Footnote 18] to help choose their investments. * Control over IT investments. During the control phase of the IT investment management process, the organization ensures that, as projects develop and as funds are spent, the project is continuing to meet mission needs at the expected levels of cost and risk. If the project is not meeting expectations or if problems have arisen, steps are quickly taken to address the deficiencies. In general, the agencies were weaker in the practices pertaining to the control phase of the investment management process than to the selection phase and no agency had the practices associated with the control phase fully in place. In particular, the agencies did not always have important mechanisms in place for agencywide investment management boards to effectively control investments, including decision-making rules for project oversight, early warning mechanisms, and/or requirements that corrective actions for under-performing projects be agreed upon and tracked. Executive level oversight of project-level management activities provides an organization with increased assurance that each investment will achieve the desired cost, benefit, and schedule results. Among the variety of reasons that agencies cited for not having IT investment management practices fully in place were that the CIO position had been vacant, that not including a requirement in the IT investment management guide was an oversight, and that the process was being revised. However, in some cases agencies could not identify why certain practices were not in place. It is important that agencies address their shortcomings, because only by effectively and efficiently managing their IT resources through a robust investment management process can they gain opportunities to make better allocation decisions among many investment alternatives and to further leverage their IT investments. Improving Agencies' IT Strategic Planning/Performance Measurement and Investment Management: To help agencies improve their IT strategic planning/performance measurement and investment management, we have made numerous recommendations to agencies and issued guidance. Specifically, in our January 2004 report we made recommendations to the 26 agencies in our review regarding practices that were not fully in place. These recommendations addressed issues such as IT strategic planning; establishing and linking enterprisewide goals and performance measures and tracking progress against these measures; and selecting, controlling, and evaluating investments. By implementing these recommendations, agencies can better ensure that they are using strategic planning, performance measurement, and investment management practices that are consistent with IT legislation, executive orders, OMB policies, and our guidance. Another mechanism that agencies can use to improve their IT management is to apply the management frameworks and guides that we have issued, which are based on our research into IT management best practices and our evaluations of agency IT management performance.[Footnote 19] In this vein, today we are releasing the latest version of our ITIM framework.[Footnote 20] This framework identifies and organizes critical processes for selecting, controlling, and evaluating IT investments into a framework of increasingly mature stages (see fig. 2). Figure 2: The ITIM Stages of Maturity with Critical Processes: [See PDF for image] [End of figure] First issued as an exposure draft in May 2000, this new version of the ITIM includes lessons learned from our use of the framework in our agency reviews and from lessons conveyed to us by users of the framework. In addition, in order to validate the appropriateness of our changes and to gain the advantage of their experience, we had the new version reviewed by several outside experts who are familiar with the ITIM exposure draft and with investment management in a broad array of public and private organizations. ITIM can be used to analyze an organization's investment management processes and to determine its level of maturity. The framework is useful to many federal agencies because it provides: (1) a rigorous, standardized tool for internal and external evaluations of an agency's IT investment management process; (2) a consistent and understandable mechanism for reporting the results of these assessments to agency executives, Congress, and other interested parties; and (3) a road map that agencies can use for improving their investment management processes. Regarding the first two points, we and selected agency Inspectors General have used the ITIM to evaluate and report on the investment management processes of several agencies.[Footnote 21] Concerning the third point, a number of agencies have recognized the usefulness of the ITIM framework and have used it to develop and enhance their investment management strategies. For example, one agency uses the framework to periodically review its IT investment management capabilities and has developed an action plan to move through the stages of maturity. In summary, our January 2004 report indicates that the federal government can significantly improve its IT strategic planning, performance measurement, and investment management. Such improvement would better ensure that agencies are being responsible stewards of the billions of dollars for IT with which they have been entrusted, by helping them to invest these monies wisely. This can be accomplished, in part, through the expeditious implementation of our recommendations and the adoption of best practices, which we have incorporated into our IT management frameworks and guides such as the ITIM. Mr. Chairman, this completes my prepared statement. I would be happy to respond to any questions that you or other Members of the Subcommittee may have at this time. Contacts: If you have any questions regarding this statement, please contact me at (202) 512-9286 or by e-mail at pownerd@gao.gov. Specific questions related to our January 2004 report may also be directed to Linda Lambert at (202) 512-9556 or via e-mail at lambertl@gao.gov or Mark Shaw at (202) 512-6251 or via e-mail at shawm@gao.gov. Questions related to the ITIM framework can be directed to Lester Diamond at (202) 512-7957 or via e-mail at diamondl@gao.gov. [End of section] Appendix I: Information Technology (IT) Strategic Planning/Performance Measurement and Investment Management Practices: Table 1 describes the 12 IT strategic planning/performance measurement and the 18 IT investment management practices that we used in our January 2004 report on the government's performance in these areas.[Footnote 22] We identified these 30 practices after reviewing major legislative requirements (e.g., the Paperwork Reduction Act of 1995 and the Clinger-Cohen Act of 1996), executive orders, Office of Management and Budget policies, and our own guidance. Table 1: IT Strategic Planning/Performance Measurement and Investment Management Practices: Practice Number: 1.1; Practice Description: IT Strategic Planning/Performance Measurement Practices: The agency has documented its IT strategic management process, including, at a minimum, * the responsibilities and accountability for IT resources across the agency, including the relationship between the chief information officer (CIO), chief financial officer (CFO), and mission/program officials; and; * the method by which the agency defines program information needs and develops strategies, systems, and capabilities to meet those needs. Practice Number: 1.2; Practice Description: IT Strategic Planning/Performance Measurement Practices: The agency has documented its process to integrate IT management operations and decisions with organizational planning, budget, financial management, human resources management, and program decisions. Practice Number: 1.3; Practice Description: IT Strategic Planning/Performance Measurement Practices: The agency requires that information security management processes be integrated with strategic and operational planning processes. Practice Number: 1.4; Practice Description: IT Strategic Planning/Performance Measurement Practices: The agency has a process that involves the CFO, or comparable official, to develop and maintain a full and accurate accounting of IT- related expenditures, expenses, and results. Practice Number: 1.5; Practice Description: IT Strategic Planning/Performance Measurement Practices: The agency prepares an enterprisewide strategic information resources management (IRM) plan that, at a minimum, * describes how IT activities will be used to help accomplish agency missions and operations, including related resources; and; * identifies major IT acquisition program(s) or any phase or increment of that program that has significantly deviated from the cost, performance, or schedule goals established for the program. Practice Number: 1.6; Practice Description: IT Strategic Planning/Performance Measurement Practices: The agency's performance plan required under GPRA includes; * a description of how IT supports strategic and program goals, * the resources and time periods required to implement the information security program plan required by the Federal Information Security Management Act (FISMA), and; * a description of major IT acquisitions contained in the capital asset plan that will bear significantly on the achievement of a performance goal. Practice Number: 1.7; Practice Description: IT Strategic Planning/Performance Measurement Practices: The agency has a documented process to; * develop IT goals in support of agency needs, * measure progress against these goals, and; * assign roles and responsibilities for achieving these goals. Practice Number: 1.8; Practice Description: IT Strategic Planning/Performance Measurement Practices: The agency has established goals that, at a minimum, address how IT contributes to; * program productivity, * efficiency, * effectiveness, and; * service delivery to the public (if applicable). Practice Number: 1.9; Practice Description: IT Strategic Planning/Performance Measurement Practices: The agency has established IT performance measures and monitors actual- versus-expected performance that at least addresses; * how IT contributes to program productivity, * how IT contributes to the efficiency of agency operations, * how IT contributes to the effectiveness of agency operations, * service delivery to the public (if applicable), * how electronic government initiatives enable progress toward agency goals and statutory mandates, * the performance of IT programs (e.g., system development and acquisition projects), and; * agency compliance with federal software piracy policy. Practice Number: 1.10; Practice Description: IT Strategic Planning/Performance Measurement Practices: The agency has developed IT performance measures that align with and support the goals in the GPRA performance plan. Practice Number: 1.11; Practice Description: IT Strategic Planning/Performance Measurement Practices: The agency developed an annual report, included as part of its budget submission, that describes progress in achieving goals for improving the efficiency and effectiveness of agency operations and, as appropriate, the delivery of services to the public through the effective use of IT. Practice Number: 1.12; Practice Description: IT Strategic Planning/Performance Measurement Practices: The agency requires that its IT management processes be benchmarked against appropriate processes and/ or organizations from the public and private sectors in terms of cost, speed, productivity, and quality of outputs and outcomes where comparable processes and organizations in the public or private sectors exist. Practice Number: IT Investment Management Practices: Practice Number: 2.1; Practice Description: IT Investment Management Practices: The agency has a documented IT investment management process that, at a minimum, * specifies the roles of key people (including the CIO) and groups within the IT investment management process, * outlines significant events and decision points, * identifies external and environmental factors that influence the process, * explains how the IT investment management process is coordinated with other organizational plans and processes, and; * describes the relationship between the investment management process and the agency's enterprise architecture. Practice Number: 2.2; Practice Description: IT Investment Management Practices: The agency established one or more agencywide IT investment management boards responsible for selecting, controlling, and evaluating IT investments that, at a minimum, * have final project funding decision authority (or provide recommendations) over projects within their scope of authority, and; * are composed of key business unit executives. Practice Number: 2.3; Practice Description: IT Investment Management Practices: The agencywide board(s) work processes and decision-making processes are described and documented. Practice Number: 2.4; Practice Description: IT Investment Management Practices: If more than one IT investment management board exists in the organization (e.g., at the component level), the organization has; * documented policies and procedures that describe the processes for aligning and coordinating IT investment decision making, * criteria for determining where in the organization different types of IT investment decisions are made, and; * processes that describe how cross-functional investments and decisions (e.g., common applications) are handled. Practice Number: 2.5; Practice Description: IT Investment Management Practices: As part of its investment management process, the agency has available an annually updated comprehensive inventory of its major information systems that includes major national security systems and interfaces. Practice Number: 2.6; Practice Description: IT Investment Management Practices: A standard, documented procedure is used so that developing and maintaining the inventory is a repeatable event, which produces inventory data that are timely, sufficient, complete, and compatible. Practice Number: 2.7; Practice Description: IT Investment Management Practices: The IT asset inventory is used as part of managerial decision making. Practice Number: 2.8; Practice Description: IT Investment Management Practices: Proposed IT investments are required to document that they have addressed the following items during project planning:; * that the project supports the organization's business and mission needs and meets users' needs, * whether the function should be performed by the public or private sector, * whether the function or project should be performed or is being performed by another agency, * that alternatives have been considered, and; * how security will be addressed. Practice Number: 2.9; Practice Description: IT Investment Management Practices: In considering a proposed IT project, the agency requires that the project demonstrate that it is economically beneficial through the development of a business case that at least addresses costs, benefits, schedule, and risks. Practice Number: 2.10; Practice Description: IT Investment Management Practices: In considering a proposed IT project, the agency requires that the project demonstrate that it is consistent with federal and agency enterprise architectures. Practice Number: 2.11; Practice Description: IT Investment Management Practices: The agency requires that the proposed IT investment, at a minimum, * support work processes that it has simplified or redesigned to reduce costs and improve effectiveness, and; * make maximum use of commercial-off-the-shelf (COTS) software. Practice Number: 2.12; Practice Description: IT Investment Management Practices: The agency has established project selection criteria distributed throughout the organization that include, at a minimum, * cost, benefit, schedule, and risk elements; * measures such as net benefits, net risks, and risk-adjusted return on investment; and; * qualitative criteria for comparing and prioritizing alternative information systems investment projects. Practice Number: 2.13; Practice Description: IT Investment Management Practices: The agency has established a structured selection process that, at a minimum, * selects IT proposals using selection criteria; * identifies and addresses possible IT investments and proposals that are conflicting, overlapping, strategically unlinked, or redundant; * prioritizes proposals; and; * is integrated with budget, financial, and program management decisions. Practice Number: 2.14; Practice Description: IT Investment Management Practices: Agency policy calls for investments to be modularized (e.g., managed and procured in well-defined useful segments or modules that are short in duration and small in scope) to the maximum extent achievable. Practice Number: 2.15; Practice Description: IT Investment Management Practices: The agencywide investment management board(s) has written policies and procedures for management oversight of IT projects that cover, at a minimum, * decision-making rules for project oversight that allow for terminating projects, when appropriate; * current project data, including expected and actual cost, schedule, and performance data, to be provided to senior management periodically and at major milestones; * criteria or thresholds related to deviations in cost, schedule, or system capability actuals versus expected project performance; and; * the generation of an action plan to address a project's problem(s) and track resolution. Practice Number: 2.16; Practice Description: IT Investment Management Practices: The agencywide investment management board(s) established an oversight mechanism of funded investments that, at a minimum, * determines whether mission requirements have changed; * determines whether the investment continues to fulfill ongoing and anticipated mission requirements; * determines whether the investment is proceeding in a timely manner toward agreed-upon milestones; * employs early warning mechanisms that enable it to take corrective action at the first sign of cost, schedule, or performance slippages; and; * includes the use of independent verification and validation (IV&V) reviews of under- performing projects, where appropriate. Practice Number: 2.17; Practice Description: IT Investment Management Practices: Corrective actions for under- performing projects are agreed upon, documented, and tracked by the agencywide investment management board(s). Practice Number: 2.18; Practice Description: IT Investment Management Practices: The agencywide investment management board(s) requires that postimplementation reviews be conducted to; * validate expected benefits and costs and; * document and disseminate lessons learned. Source: GAO. [End of table] [End of section] FOOTNOTES [1] Office of Management and Budget, Budget of the U.S. Government, Fiscal Year 2005, Report on IT Spending for the Federal Government for Fiscal Years 2003, 2004, and 2005. We did not verify this data. [2] Office of Management and Budget, Budget of the U.S. Government, Fiscal Year 2005, Analytical Perspectives. We did not verify these data. [3] Earned value management is a project management tool that integrates the investment scope of work with schedule and cost elements for optimum investment planning and control. [4] The Paperwork Reduction Act of 1995 revised the information resources management responsibilities established under the Paperwork Reduction Act of 1980, as amended in 1986. [5] During the selection phase the organization (1) identifies and analyzes each project's risks and returns before committing significant funds to any project and (2) selects those IT projects that will best support its mission needs. [6] During the control phase the organization ensures that, as projects develop and investment expenditures continue, the project is continuing to meet mission needs at the expected levels of cost and risk. If the project is not meeting expectations or if problems have arisen, steps are quickly taken to address the deficiencies. [7] During the evaluation phase, actual versus expected results are compared once projects have been fully implemented. This is done to (1) assess the project's impact on mission performance, (2) identify any changes or modifications to the project that may be needed, and (3) revise the investment management process based on lessons learned. [8] U.S. General Accounting Office, Information Technology Management: Governmentwide Strategic Planning, Performance Measurement, and Investment Management Can Be Further Improved, GAO-04-49 (Washington, D.C.: Jan. 12, 2004). [9] We reviewed 23 entities identified in 31 U.S.C. 901 and the 3 military services. These were the Departments of Agriculture, the Air Force, the Army, Commerce, Defense, Education, Energy, Health and Human Services, Housing and Urban Development, the Interior, Justice, Labor, the Navy, State, Transportation, the Treasury, and Veterans Affairs; and the Environmental Protection Agency, General Services Administration, National Aeronautics and Space Administration, National Science Foundation, Nuclear Regulatory Commission, Office of Personnel Management, Small Business Administration, Social Security Administration, and U.S. Agency for International Development. [10] U.S. General Accounting Office, Information Technology Investment Management: A Framework for Assessing and Improving Process Maturity, GAO-04-394G (Washington, D.C.: March 2004). [11] Each year, OMB's Office of Information and Regulatory Affairs publishes an Information Collection Budget by gathering data from executive branch agencies on the total number of burden hours it approved for collection of information at the end of the fiscal year and agency estimates of the burden for the coming fiscal year. [12] In fulfilling its responsibilities under this act, OMB requires agencies to report to OMB on their plans for providing the public with the option of submitting, maintaining, and disclosing required information electronically, instead of on paper. [13] U.S. General Accounting Office, Executive Guide: Measuring Performance and Demonstrating Results of Information Technology Investments, GAO/AIMD-98-89 (Washington, D.C.: March 1998). [14] Executive Order 13103, Computer Software Piracy (September 30, 1998). [15] GAO/AIMD-98-89. [16] GAO/AIMD-98-89. [17] For example, see GAO-04-394G. [18] With a scoring model, the assessment body typically attaches numerical scores and "relative value" weights to each of the individual selection criteria. Investments are then assessed relative to these scores and then against weights associated with each individual criterion. Finally, the weighted scores are summed to create a numerical value for each investment. [19] For example, see U.S. General Accounting Office, Information Technology: A Framework for Assessing and Improving Enterprise Architecture Management (Version 1.1), GAO-03-584G (Washington, D.C.: April 2003) and GAO/AIMD-98-89. [20] GAO-04-394G. [21] For example, see U.S. General Accounting Office, Information Technology: Departmental Leadership Crucial to Success of Investment Reforms at Interior, GAO-03-1028 (Washington, D.C.: Sept. 12, 2003); Bureau of Land Management: Plan Needed to Sustain Progress in Establishing IT Investment Management Capabilities, GAO-03-1025 (Washington, D.C.: Sept. 12, 2003); United States Postal Service: Opportunities to Strengthen IT Investment Management Capabilities, GAO-03-3 (Washington, D.C.: Oct. 15, 2002); Information Technology: DLA Needs to Strengthen Its Investment Management Capability, GAO-02-314 (Washington, D.C.: Mar. 15, 2002); and Information Technology: INS Needs to Strengthen Its Investment Management Capability, GAO-01-146 (Washington, D.C.: Dec. 29, 2000). [22] U.S. General Accounting Office, Information Technology Management: Governmentwide Strategic Planning, Performance Measurement, and Investment Management Can Be Further Improved, GAO-04-49 (Washington, D.C.: Jan. 12, 2004).