This is the accessible text file for GAO report number GAO-05-237 entitled 'Office of Personnel Management: Retirement Systems Modernization Program Faces Numerous Challenges' which was released on February 28, 2005. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Committees: February 2005: Office of Personnel Management: Retirement Systems Modernization Program Faces Numerous Challenges: [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-237]: GAO Highlights: Highlights of GAO-05-237, a report to congressional committees Why GAO Did This Study: The Office of Personnel Management (OPM) manages the systems that process retirement benefits for most federal civilian employees. In fiscal year 2003, over 198,000 claims were processed and over $50 billion in benefits was paid through OPM’s retirement systems. OPM is trying to modernize these systems through a program called Retirement Systems Modernization (RSM). The conference report accompanying the fiscal year 2004 OPM appropriations act directed us to review the management of and challenges facing RSM. Specifically, our objectives were to determine (1) the current status of and plans for OPM’s RSM program and (2) the challenges OPM faces in successfully managing the program. What GAO Found: The Office of Personnel Management (OPM) is in the midst of an effort known as Retirement Systems Modernization (RSM). OPM’s goals, through this program, are to improve customer service, reduce calculation errors, and reduce the amount of paper-based and manual processing. OPM expects RSM’s total cost to be about $294 million from fiscal year 1997 to fiscal year 2008, when the agency expects to have most of RSM implemented and integrated. As currently envisioned, RSM comprises four major components: licensed technology for pension benefits administration, data conversion of paper files and development of electronic processes for capture and storage of data, coverage determination and other applications, and foundation and infrastructure elements (see table). GAO’s experience with major systems acquisitions such as RSM has shown that having sound systems acquisition, change management, and investment management processes in place increases the likelihood of the acquisitions meeting cost and schedule estimates as well as performance requirements. However, GAO found that many of the processes in these areas for RSM are not sufficiently developed, are still under development, or are planned for future development. For example, OPM lacks needed processes for developing and managing requirements, planning and managing project activities, managing risks, and providing sound information to investment decision makers. Without these processes in place, RSM is at increased risk of not being developed and delivered on time and within budget and falling short of promised capabilities. Major Components of RSM and Current Status and Plans: RSM components and descriptions: Licensing technology–licensing a pension administration system from a private vendor to process claims and customer service inquiries; Status and plans: Issued request for proposals in late September. Contract award was expected by the end of January 2005. However, since our briefing, the award has been delayed pending review by Office of Management and Budget. Initial deployment planned for 2006. RSM components and descriptions: Data conversion–includes selecting a vendor to convert paper-based retirement information to an electronic format and developing processes to electronically capture and store incoming retirement information; Status and plans: Separate request for proposals for conversion of paper data to be issued before September 2005. RSM components and descriptions: Coverage determination and other RSM applications–includes developing an application to help determine benefit amounts; Status and plans: Pilot tested application in 2004. Plan is to fully deploy governmentwide by February 2005. RSM components and descriptions: Foundation and infrastructure elements–developing the infrastructure and shared services to support RSM; Status and plans: Maintaining and developing master database of retirement information to support licensed technology. Developing infrastructure to support data transmission between OPM and other agencies. Source: GAO. [End of table] What GAO Recommends: GAO is recommending that the director of OPM ensure that the RSM program office establish the management processes needed for effective oversight of the RSM program. In written comments on a draft of this briefing, the OPM director agreed that the management processes we identified are essential and described steps the agency is taking to strengthen these processes. www.gao.gov/cgi-bin/getrpt?GAO-05-237. To view the full product, including the scope and methodology, click on the link above. For more information, contact David Powner at (202) 512- 9286 or pownerd@gao.gov. [End of section] Contents: Letter: Recommendations: Agency Comments and Our Evaluation: Appendixes: Appendix I: Briefing Slides: Appendix II: Comments from the Office of Personnel Management: Abbreviations: OPM: Office of Personnel Management: RSM: Retirement Systems Modernization: Letter February 28, 2005: The Honorable Thad Cochran: Chairman: The Honorable Robert C. Byrd: Ranking Minority Member: Committee on Appropriations: United States Senate: The Honorable Jerry Lewis: Chairman: The Honorable David R. Obey: Ranking Minority Member: Committee on Appropriations: House of Representatives: The Office of Personnel Management (OPM) manages the systems that process retirement benefits for most federal civilian employees. In fiscal year 2003, over 198,000 claims were processed and over $50 billion in benefits was paid through OPM's retirement systems. OPM is trying to modernize these systems through a program called the Retirement Systems Modernization (RSM). OPM's goals, through this program, are to improve customer service, reduce calculation errors, and reduce the amount of paper-based and manual processing. The conference report accompanying the fiscal year 2004 OPM appropriations act[Footnote 1] directed us to review the management of and challenges facing RSM. Specifically, our objectives were to determine the following: (1) What is the current status of and plans for OPM's RSM program? (2) What challenges does OPM face in successfully managing the program? To determine program status and plans, we analyzed current and past program documents and interviewed program officials. To identify program challenges, we identified sound information technology systems acquisition and management processes from our prior work and from industry standards. We compared OPM's processes for managing RSM to sound management practices in key areas. We analyzed program documentation and interviewed program officials regarding their efforts and plans. We performed our work at OPM offices in Washington, D.C., from April through December 2004 in accordance with generally accepted government auditing standards. On December 8, 2004, we provided a detailed briefing to the relevant subcommittee offices on the results of this work. The briefing slides are included in appendix I. The purpose of this letter is to formally publish the briefing slides and officially transmit our recommendations to the director of OPM. In brief, we reported that OPM expects RSM's total cost to be about $294 million from fiscal year 1997 to fiscal year 2008. As currently envisioned, RSM comprises four major components: licensed technology for pension benefits administration, data conversion of paper files and development of electronic processes for capture and storage of data, coverage determination and other applications, and foundation and infrastructure elements. OPM planned to award the contract for licensed technology at the end of January 2005 and expects to have most of RSM implemented and integrated by the end of fiscal year 2008. After our briefing, the RSM project manager told us that the award of the licensed technology contract had been delayed because the Office of Management and Budget has asked to review a revised business case for RSM before OPM awards the licensed technology contract. OPM has implemented selected management processes; however, many processes are not sufficiently developed, are still under development, or are planned for future development. For example, OPM lacks needed processes for developing and managing requirements, planning and managing project activities, managing risks, and providing sound information to investment decision makers. Without these processes in place, the RSM program is at increased risk of not being developed and delivered on time and within budget. It is also at risk of falling short of promised capabilities. Recommendations: To address the challenges we identified, we are recommending that OPM establish the processes needed for effective oversight of the program. Before awarding a contract for the defined benefit technology portion of RSM, we recommend that the director of OPM ensures that the RSM program office performs the following: * Conducts a detailed validation of OPM's previously developed requirements, including examining each requirement to determine whether it is mandatory or optional for the system. OPM should then use this analysis during contract negotiations to ensure the best value in awarding the contract. * Develops project management processes for tracking cost, schedule, and performance progress on RSM components against planned objectives. We also recommend that the director of OPM ensure that the RSM program office expeditiously performs the following: * Develops an acquisition strategy that includes deliverables and milestones throughout the entire project life cycle, addresses issues related to the transition of the licensed technology to a fully operational state, and addresses all major elements of RSM. * Develops an acquisition strategy for the data conversion portion of RSM that addresses interdependencies with the licensed technology portion of RSM. * Develops a list of critical interdependencies among the tasks in different portions of RSM. * Implements a comprehensive risk management program including identification, tracking, and updating of programwide risks, and conducts routine discussions of risk mitigation activities with senior management. We are also recommending that the director of OPM ensure that the RSM program office expeditiously performs the following: * Updates change management plans to reflect the current acquisition approach. The plans should identify and address the tasks and associated milestones needed to prepare users for the impact of the system on their roles and responsibilities. * Develops a comprehensive governance structure that includes documented processes guiding the executive steering committee's activities. * Develops a detailed security plan and a set of security requirements for the licensed technology and data conversion portions of RSM. Agency Comments and Our Evaluation: OPM provided written comments on a draft of this report, which are included in appendix II. In the comments, the OPM director agreed that the management processes we identified are essential to ensure that RSM is accomplished on time, within budget, and meets its expectations of improved customer service and efficiency. The OPM director also listed several actions the agency has taken to strengthen management and oversight of RSM. For example, OPM is using its project management consultant to strengthen management processes, reporting and governance structures, and risk management. For the data conversion portion of RSM, OPM is also developing an acquisition strategy and stated that it has completed requirements analysis, security, and risk mitigation plans. OPM also stated it has recently hired an employee to lead efforts to develop a change management plan for each portion of RSM prior to implementation. However, OPM reiterated that it had included mandatory business requirements in the request for proposals for licensed technology and that the remaining business requirements would be reviewed after contract award to determine whether or not they should be adopted. As we state in our briefing slides, our review of the request for proposals and other program documentation does not indicate that OPM has clearly documented the mandatory versus optional requirements. A clear understanding of optional versus mandatory requirements is critical to ensuring that the government obtains the best value in its systems acquisitions. The licensed technology award has been delayed due to OMB's request to review a revised business case. Therefore, OPM has additional time to define its mandatory and optional requirements as recommended so that it can select the vendor that can provide the best solution at least cost. We are sending copies of this report to the director of the Office of Personnel Management and other interested parties. Copies will also be made available to others on request. Should you or your staffs have any questions concerning this report, please contact me at 202-512-9286 or by e-mail at [Hyperlink, pownerd@gao.gov]. Camille Chaires, Vijay D'Souza, Nancy Glover, Colleen Phillips, and Sylvia Shanks were key contributors to this report. Signed by: David A. Powner: Director, Information Technology Management: [End of section] Appendixes: Appendix I: Briefing Slides: Office of Personnel Management's Retirement Systems Modernization Program Faces Numerous Challenges: Briefing for the Subcommittee on Transportation, Treasury and General Government: Senate Appropriations Committee: and: The Subcommittee on Transportation, Treasury, and Independent Agencies: House Appropriations Committee: December 8, 2004: * Objectives: * Scope and Methodology: * Results in Brief: * Background: * Retirement Systems Modernization Status and Plans: * Retirement Systems Modernization Challenges: -Systems Acquisition Management: -Organizational Change Management: -IT Investment Management: -Information Security: * Conclusions: * Recommendations: * Agency Comments and Our Evaluation: Objectives: The Office of Personnel Management (OPM) is modernizing the systems and processes it uses to manage federal retirement benefits under a program called Retirement Systems Modernization (RSM). The conference report accompanying the fiscal year 2004 OPM appropriations act [NOTE 1] directs GAO to review the management of and challenges facing RSM. Specifically, our objectives are: * What is the current status of and plans for OPM's RSM program? * What challenges does OPM face in successfully managing the program? Scope and Methodology: To determine program status and plans, we analyzed current and past program documents including plans, acquisition documents, deliverables, and OPM analyses, and interviewed program officials. To identify program challenges, we identified sound IT systems acquisition and management processes from our prior work and industry standards, such as those developed by the Software Engineering Institute. [NOTE 2] Given RSM's current status and plans, we focused our review on the following key processes: Systems acquisition management: * Acquisition planning-preparation for key activities related to systems acquisition: * Requirements development and management-elicitation, analysis, validation, and management of customer and stakeholder needs: * Project management-planning and managing project-related activities, such as interrelation of components and tracking of cost and schedule: * Program risk management-identification and mitigation of potential project risks: We also evaluated three management areas critical to the success of IT modernization projects: * Organizational change management-preparing users for business process changes accompanying system implementation: * IT investment management-providing oversight for the selection, control, and evaluation of IT investments: * Information security-protecting the integrity, confidentiality, and availability of data by minimizing risks of unauthorized intrusions and serious disruptions of operations. We compared OPM's processes for managing RSM to sound management practices in each of these areas. We analyzed program documentation and interviewed program officials regarding their efforts and plans. We also reviewed relevant work by GAO and OPM's Inspector General's office. We performed our work at OPM offices in Washington, DC from April through December 2004 in accordance with generally accepted government auditing standards. Results in Brief: Status and Plans: OPM is in the midst of an effort, known as the Retirement System Modernization (RSM), to automate the processes and systems supporting federal retirement programs. From 1997 to 2001, OPM spent about $12 million developing planning documents, strategies, and analyses to build RSM. In 2001, the agency changed direction on RSM to rely heavily on licensing existing technology for administering pension benefits from an outside vendor that would host and maintain a large part of RSM. After the redirection, OPM estimated RSM's total cost to be about $282 million from fiscal year 2002 to 2008. As currently envisioned, RSM comprises four major components: licensed technology for pension benefits administration, data conversion of paper files, coverage determination and other applications, and foundation and infrastructure elements. Results in Brief: The major components of RSM and their current status and plans are: RSM components and descriptions: Licensing technology--licensing a pension administration system from a private vendor to process claims and customer service inquiries; Status and plans: Request for proposals issued in late September. Contract expected to be awarded by the end of January 2005 and initial deployment planned for 2006. RSM components and descriptions: Data conversion--includes selecting a vendor to convert paper-based retirement information to an electronic format and developing processes to electronically capture and store incoming retirement information; Status and plans: Separate request for proposals for conversion of paper data to be issued before September 2005. RSM components and descriptions: Coverage determination and other RSM applications--includes developing an application to help determine benefit amounts; Status and plans: Pilot tested the application in 2004. Plan is to fully deploy governmentwide by February 2005. RSM components and descriptions: Foundation and infrastructure elements--developing the infrastructure and shared services to support RSM; Status and plans: Maintaining and developing master database of retirement information to support licensed technology. Developing infrastructure to support data transmission between other agencies and OPM. [End of table] OPM expects to have most of RSM implemented and integrated by the end of fiscal year 2008. Challenges to RSM: Our experience with major systems acquisitions has shown that having sound systems acquisition, change management, and investment management processes in place increases the likelihood of the acquisitions meeting cost and schedule estimates as well as performance requirements. OPM has implemented selected processes in the areas of systems acquisition, change management, investment management, and information security; however, many processes are not sufficiently developed, are still under development, or are planned for future development. For example, OPM lacks needed processes for developing and managing requirements, planning and managing project activities, managing risks, and providing sound information to investment decision makers. Without these processes in place, RSM is at increased risk of not being developed and delivered on time and within budget. It is also at risk of falling short of promised capabilities. To address these challenges, we are recommending that OPM establish the processes needed for effective oversight of the program. In reviewing a draft of our briefing, RSM program management staff generally agreed with the facts and recommendations in our briefing and provided technical corrections that we incorporated as appropriate. Background: Office of Personnel Management Mission: OPM's mission is "to build a high quality and diverse federal workforce, based on merit system principles, that America needs to guarantee freedom, promote prosperity, and ensure the security of this great nation". [NOTE 3] One of OPM's major human resources tasks is to manage and administer the federal retirement program: * OPM's Center for Retirement and Insurance Services administers the retirement program. * In fiscal year 2003, OPM processed over 198,000 retirement annuity and survivor claims. * Federal agency staff, current federal employees, and those receiving benefits (annuitants) rely on OPM's existing information systems and processes to accurately provide services such as calculating benefit amounts, processing contributions, and paying benefits. Federal Retirement Systems: OPM administers two retirement plans for federal employees: The Civil Service Retirement System (CSRS)-CSRS provides retirement, disability, and survivor benefits and is a stand-alone pension system that covers most employees hired before 1984. The defined benefit [NOTE 4] portion of the Federal Employees Retirement System (FERS)-FERS is a 3-part retirement plan which includes: 1) a defined benefit portion administered by OPM; 2) Social Security; and 3) the Thrift Savings Plan, a defined contribution plan administered by the Federal Retirement Thrift Investment Board. FERS became effective January 1, 1987. [NOTE 5] For fiscal year 2003: * There were 2.6 million federal employees actively paying into the retirement system. * There were 2.4 million federal retirement annuitants. * Over $50 billion was paid to annuitants, survivors, and families. Over time, the number of FERS annuitants is expected to increase and the number of CSRS annuitants is expected to decrease. Figure 1: Annuitants and survivors in CSRS and FERS retirement systems: [See PDF for image] Source: GAO analysis based on OPM data. [End of figure] OPM has identified multiple factors that could limit its ability to provide high-quality retirement services, including: * the increasing number of FERS retirement claims, which are more difficult to process than CSRS claims because of the complexity of FERS calculations; * the existing high error rate associated with the CSRS and FERS benefit calculations, estimated by OPM at 11 percent of all claims; and: * the high cost, limited capabilities, and other problems with existing OPM information systems and processes. Retirement Systems Modernization Program: Through RSM, OPM expects to reengineer the various processes that provide services to retirement program participants and that support benefits counseling throughout an employee's career with the federal government. Specifically, OPM's objectives for RSM are to: * create a single, integrated data repository accessible to OPM and agency personnel across all retirement programs, * standardize applications for coverage and eligibility determinations and benefits calculations, making them specific to customer needs and accessible to federal agencies and program participants, * automate the process of entering data into its systems, * initiate electronic case management and workload planning, * create Web-accessible, self-service applications for the administration of retirement programs to members and appropriate on- demand changes to employee or annuitant records, and: * establish a comprehensive customer contact center. OPM expects RSM to change the systems and processes used by all agencies that have employees covered by retirement programs managed by OPM. This constitutes the vast majority of federal civilian employees. [NOTE 6] To represent federal agencies and provide input to OPM on RSM direction, priorities, and other topics, OPM created the Agency Advisory Committee, which consists of members from the departments of Agriculture, Defense, Labor, Treasury, and the U.S. Postal Service. Retirement Systems Modernization Program: RSM is not the first attempt to modernize OPM's retirement benefits systems. In 1987, OPM began an effort called the FERS Automated Processing System program. In September 1995, after 8 years of work and an investment of about $25 million, OPM stopped working on the program to conduct an internal progress review. [NOTE 7] The review found that: * Despite 8 years of work, the program was at a high risk of failure. * The program lacked important planning elements, such as a clear strategic vision and definition of the relationship of the project to overall agency information technology plans. * The development process did not define adequate performance measures and test and evaluation results. * The agency had not defined clear incentives for the contractor. * The program was unsuccessful in defining clear, traceable business requirements and in implementing a change control process to track changes in the requirements. * OPM lacked an effective, documented IT management process that could support such a large and complex program. In 1996, as a result of the review, OPM terminated the FERS Automated Processing System program. According to agency officials, OPM was able to reuse some of the investment in the FERS Automated Processing System. Specifically: * OPM is using the hardware it purchased for the FERS Automated Processing System program in its existing retirement systems. * OPM also used information gathered for this program for background and planning purposes for RSM. * The lessons identified in the internal progress review were used to establish management structures for RSM. OPM began plans for RSM in 1997 and originally intended to manage the system in house by purchasing and modifying hardware and existing software packages. From 1997 to 2001, OPM spent about $12 million developing planning documents, strategies, and analyses to deal with operational changes that RSM would cause. The agency also began developing security requirements and business requirements for RSM. In 2001, OPM decided to change the direction of RSM. Specifically, OPM examined the possibility of increasing the role of private vendors in hosting and maintaining key components of RSM. After an initial internal analysis, the agency issued a request for information in 2002 to private vendors regarding outsourcing key retirement program functions. The analysis of responses to the request for information showed: * Viable outsourcing alternatives existed for OPM's retirement system administration needs. * The proposed solutions could be both cost efficient and less risky to implement. * The proposed solutions were more likely to be completed on time and on budget. As a result of these findings, OPM decided to license technology for administering pension benefits (including functions such as claims processing and customer service inquiries) from a vendor. Given this change of direction, RSM now has four major components: * Licensing technology-Licensing an existing defined benefit system from a private company. Although OPM plans for the vendor to host and maintain the system, OPM employees will continue to manage the work of claims processing and customer service. * Data conversion-Awarding a contract to convert existing paper-based retirement information to an electronic format and developing processes to electronically capture and store incoming retirement information. * Coverage determination and other RSM applications-Developing applications including one that helps human resources specialists at federal agencies determine an employee's correct retirement coverage using a series of automated rules and steps. The coverage determination application is designed to reduce errors in benefit calculations. * Foundation and infrastructure elements-Implementing the network infrastructure, hardware, software, and other shared services to develop and run RSM applications and meet federal and OPM information technology standards. Examples include processes to support data transmission from federal agencies and a database to support the coverage determination application. The RSM program office is managing the integration of the four components. To assist in project management and technical support, OPM hired two support contractors and has recently hired a project management consultant. RSM Status and Plans Overview: Much of the work on RSM over the last 7 years has focused on planning and preparation. Tasks completed to date include developing and issuing a request for information to private vendors, developing a request for proposals, and some application development work. Between 1997 and September 2004, OPM has spent about $22 million for RSM, and plans to spend about $272 million more to acquire, deploy, and operate the system through 2008. OPM is currently working on the following efforts: * awarding a contract to license technology from a private company for defined benefit administration, * issuing a request for proposals for conversion of paper data, * enhancing and deploying the coverage determination application, and: * maintaining and enhancing the supporting hardware and network infrastructure. OPM expects to implement and integrate the majority of RSM components by the end of fiscal year 2007 and add other applications in 2008. RSM Status and Plans Costs: RSM is expected to cost about $294 million between 1997 and 2008. OPM spent about $12 million on planning for RSM between fiscal years 1997 and 2001. After deciding to license technology, OPM re-estimated RSM costs as about $282 million from fiscal year 2002 to 2008. As of September 2004, OPM has spent about $10 million of this amount. The $282 million comprises the following: Figure 2: Major costs for RSM components, fiscal years 2002-2008, rounded to nearest million: [See PDF for image] -graphic text: Foundation and infrastructure elements: $26. Coverage determination application and other RSM applications: $12. Licensing technology: $128. Data conversion and planning: $60. Project management and other costs: $56. Source: OPM. [End of figure] The status of major components of RSM as of October 2004 is as follows: Licensing technology: * OPM issued a draft request for proposals for licensing the defined benefit technology in July 2004 and planned to award the contract by December 2004. * The final request for proposals was scheduled to be issued in mid- August, but was not actually issued until late September. * As a result, the deadline for vendor proposal submission, which was originally September 10, was delayed to October 12. * Agency officials said that the contract award, originally planned for mid-December, is now planned for the end of January 2005. Figure 3: Planned versus actual schedule for licensed technology contract award: [See PDF for image] Source: OPM. [End of figure] * OPM still plans initial deployment of the licensed technology component in 2006. Data conversion: * In February 2004, OPM implemented electronic storage of retirement information on federal employees whose agencies were transitioning to the e-Payroll initiative. [NOTE 8] Previously, the agencies would have had to print a paper retirement record for each employee that would have then been entered into OPM's information systems. * OPM is planning to issue a separate request for proposals for conversion of paper data and select a vendor by September 2005. Historical paper retirement data is planned to be converted by the end of fiscal year 2007. Coverage determination and other RSM applications: * OPM piloted the coverage determination application earlier this year, initially releasing it to a limited number of agencies and users. OPM released the revised application in September 2004 and is planning to fully deploy the application governmentwide by February 2005. Agency officials estimated that there would be about 3,000 total application users and that about 1,000 were using the system as of December 2004. Foundation and infrastructure elements: * OPM is implementing changes in its retirement systems infrastructure to improve the ability of federal agencies to electronically transmit retirement data to OPM. The infrastructure currently supports parts of RSM, such as the coverage determination application and the transfer of e-Payroll-related data, that are already in use. OPM plans to implement additional changes and enhancements to its infrastructure to support other parts of RSM as they are developed. * OPM is also developing a master database to store all retirement data for current active employees. According to RSM program officials, this database supports the coverage determination application and certain other applications. Next year, OPM is planning to enhance the database to support the selected licensed technology solution. OPM expects to have most of RSM implemented and integrated by the end of fiscal year 2008. RSM Challenges: To effectively manage major IT programs, organizations can use sound acquisition and management processes to minimize risks and thereby maximize chances for success. Such processes have been identified by leading organizations such as the Software Engineering Institute, the CIO Council, and our prior work looking at best practices in industry and government. Key areas include: * Systems acquisition management-Management of cost, budget, and schedule goals for information systems acquisition projects. Key processes for systems acquisitions include acquisition planning, requirements development and management, project management, and program risk management. * Organizational change management-Preparing users for business process changes accompanying system implementation. * IT investment management-Providing oversight for the selection, control, and evaluation of IT investments. * Information security-Protecting the integrity, confidentiality, and availability of data by minimizing risks of unauthorized intrusions and serious disruptions of operations. Our work has shown that such processes are a significant factor in successful systems acquisitions and development programs, and they improve the likelihood of meeting cost and schedule estimates as well as performance requirements. RSM Challenges: Systems Acquisition Management: Acquisition planning: According to leading IT organizations, acquisition planning involves preparing for key activities, such as requirements definition and risk management, and items such as budgeting, scheduling, and developing an acquisition strategy. Effective acquisition planning includes having a written organizational policy guiding acquisitions and an acquisition strategy that includes objectives, projected costs and schedules, risks, and addresses the entire project life cycle. According to RSM program officials, OPM does not have an agency-wide policy specific to IT acquisitions. Agency officials stated that RSM is an unusually large program for OPM, and that they are developing plans and processes based on a combination of OPM's general procurement policy, IT development processes, and industry best practices. The officials said that approval processes for IT acquisitions are defined based on dollar amounts; however, they did not provide information detailing criteria for evaluation of investments or written processes guiding IT acquisitions. While OPM has developed an acquisition strategy for the licensed technology portion of RSM that provides objectives, estimated costs, and a high-level discussion of risks, this plan is not sufficiently detailed or comprehensive. Specifically, the strategy does not address issues related to supporting the acquisition later in its life cycle, including identifying the facilities and resources needed for support of the licensed technology, any plans to transition the software from the acquisition organization to a support organization, or planning for long-term growth and supportability of the system. According to agency officials, this information is contained in draft budget-related documentation, which is not yet completed. OPM also has not yet developed an acquisition strategy for the data conversion portion of RSM or defined which deliverables are needed for data conversion to test the defined benefit technology. Agency officials said they plan to develop the acquisition strategy for data conversion before issuing a request for proposals; however, they did not provide a specific date for completion. The lack of an agencywide policy for IT acquisitions increases the risk that OPM will not be able to effectively manage RSM to deliver promised capabilities on time and within cost estimates. This risk is more significant given that agency officials acknowledge RSM is a large and unique project for OPM. Further, the lack of a strategy to address longer-term issues relating to the licensed technology acquisition increases the risk that unforeseen issues could limit the ability of the system to become fully operational. Without a developed acquisition strategy that addresses all major elements of RSM, including data conversion, the risk is also increased that interdependencies among the different portions of RSM may not be identified and addressed early in the acquisition. Requirements development and management: Leading IT organizations recognize the importance of requirements development and management, including the elicitation, analysis, and validation of customer and stakeholder needs and expectations. An effective requirements management process involves establishing an agreed-upon set of requirements and managing any changes to the requirements in collaboration with stakeholders. In addition, requirements that are mandatory versus optional should be defined and used in deciding what requirements can be eliminated or postponed to meet other project goals, such as cost and schedule constraints. OPM created an extensive set of requirements when RSM development was originally planned. However, much of the work involved in establishing these requirements was performed four years ago and has not been updated. For the licensed technology portion of RSM, agency officials plan to reevaluate the previously developed system requirements relating to this portion of the project after contract award to see which requirements are essential and which can be eliminated. Accordingly, the winning vendor's solution may provide the essential requirements or it may need to be customized at additional cost. Agency officials told us that they believe that the functionality provided by vendors for the licensed technology portion of RSM is comparable and that similar amounts of customization would most likely be required of all vendors. They believe this is true because none of the potential vendors currently service federal customers and any unique requirements would most likely be due to specific rules and regulations of federal retirement programs. However, they did not provide an analysis to support this statement. If OPM does not determine its essential requirements before contract award, the agency may not be able to select the vendor that can provide the best solution at least cost. Based on its current approach, OPM will not know the actual cost of licensing technology until after contract award and the negotiation of customization costs with the selected vendor. Project management: Effective project management is the process for planning and managing all project-related activities, such as defining how components are interrelated, defining tasks, estimating and obtaining resources, and scheduling activities. Project management allows the performance, cost, and schedule of the overall program to be continually measured, compared with planned objectives, and controlled. OPM has defined major components of RSM but it has not yet defined the dependencies among tasks in the different portions of RSM. Specifically, OPM does not yet have a plan that identifies critical tasks in one area that will impact the completion of other tasks in the program. For example, OPM has not defined the tasks from the data conversion portion of the program that must be completed before the licensed technology portion of RSM can be tested. Additionally, OPM plans to develop key processes for managing and tracking progress on RSM after awarding the defined benefit technology contract. * Agency officials said they want to develop processes that meet their needs as well as the vendor's capabilities. * OPM's request for proposals requires that vendors submit a proposed project management plan and proposed performance standards, quality levels, and monitoring methods. Agency officials said its negotiations with vendors would include refining progress and quality metrics, planned objectives, and control mechanisms. Agency officials also stated that their recently hired project management consultant will help develop project management tools; however, they have not yet finalized the tasks the contractor will perform. By not identifying critical dependencies among tasks in RSM, OPM increases the risk that unforeseen delays in one area of RSM will hinder progress in other areas of RSM. Further, until it develops its own initial set of project management processes and metrics, OPM increases the risk that it will be unable to effectively oversee all RSM components. It also increases the risk that the project metrics ultimately negotiated with the vendor may not best meet the agency's needs. As a result, RSM is more likely to exceed expected costs and schedules. Program risk management: According to leading systems acquisition organizations, risk management is a process for identifying potential problems before they occur and adjusting the acquisition to decrease the chance of their occurrence. Risks should be identified as early as possible and a risk management process should be developed and put in place. Risks should be identified, analyzed, mitigated, and tracked to closure. RSM program staff develop an overall risk assessment annually as part of the budgeting process. In addition, an RSM support contractor maintains a database that tracks risks for each of the task orders they receive. However, OPM officials acknowledged that they do not yet have a process for identifying and tracking RSM programmatic risks, cross- cutting issues, and mitigation strategies at regular meetings of program officials. In its request for proposals for the licensed technology, OPM has asked each vendor to provide its own risk assessment and mitigation strategies. OPM has also said its project management consultant will assist it in implementing a program-wide risk management process; however, exact milestones and tasks have not been finalized. Until it establishes a formal program-wide risk management process to track and manage risks and cross-cutting issues on an ongoing basis, OPM does not have a mechanism to address unforeseen problems. Without such a process, OPM faces potential RSM cost, schedule, and quality control issues. RSM Challenges: Organizational Change Management: According to leading IT organizations, organizational change management is the process of preparing users for the business process changes that will accompany implementation of the system. An effective organizational change management process includes project plans that prepare users for impacts the new system might have on their roles and responsibilities and a process to manage those changes. OPM officials stated that change management issues pose one of the most substantial challenges to the success of RSM. These issues require action within OPM and at other agencies with which OPM exchanges information. To address change management, the agency has completed several planning documents for RSM. However, many of these documents were completed before OPM's 2002 decision to license technology from a private vendor. OPM has not updated these plans to reflect its new approach. Further, OPM has not yet developed a detailed transition plan to help prepare users for changes to their job responsibilities. Agency officials said that they will develop a more detailed change management plan after contract award for the licensed technology. However, they have not yet identified tasks and milestones for completing this plan. Our prior work has shown that addressing business process changes is critical to the success of system modernization efforts. [NOTE 9] If a comprehensive change management plan is not developed with user involvement before the licensed technology is implemented, confusion about roles and responsibilities could hinder effective implementation of the system and limit the cost and quality improvements that OPM is seeking. RSM Challenges: IT Investment Management: The Office of Management and Budget and GAO guidance call for agencies to establish IT investment management boards comprised of key executives to regularly track the progress of major systems acquisitions. These boards should have documented policies and procedures for management oversight of IT projects and systems, and should be able to adequately oversee projects' progress toward cost and schedule milestones and their risks. The board should also employ early warning systems that enable it to take corrective actions at the first sign of cost, schedule, and performance slippages. RSM has an executive steering committee chaired by the deputy associate director of the Center for Retirement and Insurance Services that acts as an IT investment management board for RSM. Currently, RSM program officials meet quarterly with the executive steering committee to discuss progress. However, RSM program officials were not aware of formal policies or procedures guiding the board's oversight responsibilities or activities. Agency officials stated that they will define such a governance structure for the licensed technology portion of RSM during the contract award process. An undefined governance structure increases the risk that the executive steering committee will not be able to effectively oversee and evaluate RSM and track program status successfully. For example, the committee may have difficulty monitoring accomplishment of mission requirements, compliance with key program dates, and closure of risk items. RSM Challenges: Information Security: Federal guidance and laws require that an agency's information systems security activities include a security management plan and detailed security requirements. In addition, systems should be certified and accredited. * The RSM project has an overall information security plan. In addition, the RSM program office has developed an overall security and privacy requirements plan that uses industry standard security controls to ensure the protection of the sensitive user data that will be processed by the systems. * The Coverage Determination Application portion of RSM also has an information security plan and has been certified and accredited, as has the portion of RSM dealing with capture of e-Payroll data. However, RSM is missing key required elements for certain portions of the program. Specifically: * The agency has not yet developed security plans for the licensed technology and data conversion portions of RSM. * In addition, agency officials said they do not currently have detailed security requirements for the licensed technology portion of RSM, although the request for proposals identified high-level security requirements. They plan to develop detailed security requirements after awarding the licensed technology contract to a vendor. Without fully developed security plans and security requirements for the licensed technology and data conversion portions of RSM, OPM increases the risk that both it and its vendors will not meet information security needs for these portions of the program. Conclusions: Billions of dollars of federal employee and annuitant retirement benefits are dependent on the accuracy and effectiveness of OPM's retirement systems, but longstanding attempts to modernize these systems have not materialized. OPM is currently in the midst of a new effort to modernize its retirement systems and processes through the acquisition and management of technology, systems, and services. OPM has made initial efforts to implement system management processes, including early requirements development, risk management of tasks in its support contracts, and plans for establishing effective project management metrics. It has plans to further define and implement these processes after it awards a contract for a key segment of RSM. However, OPM currently lacks many of the sound management processes it needs to adequately manage this program. Much work remains to be done to adequately manage requirements, risks, and contracts, and to ensure that effective security management, process change management, and executive oversight are in place. Until OPM establishes sound management processes in these areas, RSM is at increased risk of overrunning costs, missing milestones, and falling short of promised performance. Recommendations: Systems acquisition management: Before awarding a contract for the defined benefit technology portion of RSM, we recommend that the director of OPM ensure that the RSM program office: * Conducts a detailed validation of its previously developed requirements. This includes examining each requirement to determine whether it is mandatory or optional for the system. OPM should then use this analysis during contract negotiations to ensure the best value in awarding the contract. * Develops project management processes for tracking cost, schedule, and performance progress on RSM components against planned objectives. We recommend that the director of OPM ensure that the RSM program office expeditiously: * Develops an acquisition strategy that includes deliverables and milestones throughout the entire project life cycle, addresses issues related to the transition of the licensed technology to a fully operational state, and addresses all major elements of RSM. * Develops an acquisition strategy for the data conversion portion of RSM that addresses interdependencies with the licensed technology portion of RSM. * Develops a list of critical interdependencies among the tasks in different portions of RSM. * Implements a comprehensive risk management program including identification, tracking, and updating of program-wide risks, and conducts routine discussions of risk mitigation activities with senior management. Organizational change management: We recommend that the director of OPM ensure that the RSM program office expeditiously updates change management plans to reflect the current acquisition approach. The plans should identify and address the tasks and associated milestones needed to prepare users for the impact of the system on their roles and responsibilities. IT investment management: We recommend that the director of OPM ensure that the RSM program office expeditiously develops a comprehensive governance structure that includes documented processes guiding the executive steering committee's activities. Information security: We recommend that the director of OPM ensure that the RSM program office expeditiously develops a detailed security plan and a set of security requirements for the licensed technology and data conversion portions of RSM. Agency Comments and Our Evaluation: OPM officials, including the RSM program manager, provided us with oral comments on a draft of this briefing. In their comments, officials generally agreed with the facts and recommendations in our briefing. Agency officials stated that their recently hired project management consultant would help them address our recommendations in systems acquisition management and investment management. In addition, they provided suggestions for technical corrections, which we incorporated in this briefing as appropriate. Agency officials agreed that analyzing and validating requirements is important, but disagreed that this needed to be completed before awarding the licensed technology contract. They stated that they had previously performed some validation of requirements and revised the request for proposals for the licensed technology contract to reflect this. Agency officials also said they were using their understanding of mandatory and optional requirements during negotiations with potential vendors; however, they did not provide a clearly defined plan for doing this. A clear understanding of optional versus mandatory requirements is critical to ensuring that the government obtains the best value in its systems acquisitions. Our review of the request for proposals and other program documentation does not indicate that OPM has clearly documented the mandatory versus optional requirements. Since OPM has not fully determined its essential requirements, the agency may not be able to select the vendor that can provide the best solution at least cost. Based on its current approach, OPM will not know the actual cost of licensing technology until after contract award and the negotiation of customization costs with the selected vendor. NOTES: [1] H. R. Conf. Rep. No. 108-401, at 915 (2003); S. Rep. No. 108-146, at 162 (2003). [2] Carnegie Mellon University Software Engineering Institute, Software Acquisition Capability Maturity Model (Pittsburgh, PA, April 1999) and Capability Maturity Model-Integration (Pittsburgh, PA, August 2002). [3] Office of Personnel Management, Performance and Accountability Report Fiscal Year 2003, (Washington, D.C.: 2003). [4] Defined benefit plans calculate benefit amounts in advance based on factors such as salary level and years of service. They are different than defined contribution plans, in which employees and employers can contribute to a worker's individual account and benefits are based on the contributions to the account and the investment gains and losses that accumulate. [5] Due to an interim plan created under The Federal Employees Retirement Contribution Temporary Adjustment Act, almost all new employees hired after January 1, 1984 are covered by FERS. [6] Retirement benefits for members of the military and Foreign Service are not managed through these systems. [7] Office of Personnel Management Retirement Insurance Service, FERS Automated Process System, Information Technology Resources Board Review, (Washington, D.C.: July 16, 1996). [8] The e-Payroll initiative is a project managed by OPM to improve federal payroll operations by consolidating existing federal payroll system providers and simplifying and standardizing policies and procedures. [9] For example, see GAO, Information Technology: Architecture Needed to Guide NASA's Financial Management Modernization, GAO-04-43 (Washington, D.C.: November 21, 2003) and Department of Defense: Financial and Business Management Transformation Hindered by Long- standing Problems, GAO-04-941T (Washington, D.C.: July 8, 2004). [End of slide presentation] [End of section] Appendix II: Comments from the Office of Personnel Management: OFFICE OF THE DIRECTOR: UNITED STATES OFFICE OF PERSONNEL MANAGEMENT: WASHINGTON, DC 20415-0001: The Honorable David M. Walker: Comptroller General of the United States: Government Accountability Office: Washington DC 20548: Dear Mr. Walker: Thank you for providing us the opportunity to review and comment on the Government Accountability Office's draft report, Office of Personnel Management's Retirement Systems Modernization Program Faces Numerous Challenges (GAO-05-237). Our staffs met on December 1, 2004 to discuss OPM comments on the slides used to brief the Subcommittee on Transportation, Treasury and General Government, Senate Committee on Appropriation and the Subcommittee on Transportation, Treasury and Independent Agencies, House Committee on Appropriations on December 8, 2004. In the report, you make a number of recommendations to strengthen the processes to provide effective management and oversight of the Retirement Systems Modernization (RSM) effort. I agree that these processes are essential to ensure that our complex modernization effort be accomplished on time, within budget, and meet the expectations of improved customer service and promised efficiencies. As you know, we have taken a number of actions to make sure the necessary processes are in place to improve our management and oversight and will complete these activities prior to beginning implementation of the major modernization components. They include: Strengthen Project Management: OPM acquired a contract that will provide expert project management services to supplement current project leadership. We have assessed the current program management approach, and developed a plan to strengthen program management and oversight. The Program Management Plan addresses RSM project management, including the governance, program and project management roles, processes and reporting, baseline change control, risk and issue management, and quality assurance. An ANSI 748-A compliant Earned Value Management System (EVMS) will be implemented within the RSM program that integrates cost, schedule and resources using a web-based tool for capturing, managing, and reporting earned value data. EVMS will be used throughout the entire RSM program's life cycle. The EVMS will be implemented within 60 days of the vendor contract award for the Defined Benefits Technology, Paper Data Conversion, and Technology and Business Transformation Services contracts. OPM is committed to ensuring program management processes and appropriate oversight will be implemented prior to the major implementation activities of transitioning to the Defined Benefit Technology and Paper Data Conversion. Within the major components of the Paper Data Conversion and Defined Benefits Technology, OPM has used governance structures to provide sound investment information to decision makers. A copy has been provided of the OPM Information Technology (IT) Capital Planning and Investment Control Process. This process was established to meet the goals of ensuring that OPM's major IT initiatives are appropriately selected and delivered on time and within budget, and that the processes are consistent with the requirements specified in Office of Management and Budget Circulars A- 11 and A-1 30. This process will be followed for all IT implementations of the RSM program. License Defined Benefits Technology: A Defined Benefit Technology Solution Request for Proposal (RFP) has been issued and evaluations were completed on January 6, 2005. We used two governance structures consisting of decision makers and subject matter experts to thoroughly assess the proposals. The Source Selection Evaluation Team contains technical and functional stakeholders and subject matter experts. This team evaluated the technical rankings, price information, best technical solution and recommended a vendor. The Source Selection Advisory Council (SSAC) consists of OPM executives and external executives from other Federal agencies to bring full representation to the Retirement Systems Modernization program. The SSAC will make the final determination as to the best value in the vendor selection process. The contract award is dependent upon the Office of Management and Budget funding for this program. The Implementation, Transition and Business Process Plan will be completed in May 2005. With the future award of the Defined Benefits Technology contract, OPM can look forward to providing world class defined benefits services to all Federal employees and annuitants. Acquire Paper Data Conversion Contract: A Request for Proposal (RFP) has been developed to solicit vendors to image paper documents for active Federal employees and convert information into electronic data to be used with other retirement applications. The planned date for the solicitation of the RFP is March 2005, but its determination is dependent upon the successful contract award of the Defined Benefits Technology solution. The Paper Data Conversion major component also includes capturing recurring data feed(s) from agencies and payroll providers, and one-time feeds of agency historical electronic retirement data. The Paper Data Conversion Development Plan was completed November 2004, and the Acquisition Plan is currently being reviewed by key OPM staff. The Requirements Analysis, Security Architecture, Infrastructure Architecture, Risk and Risk Mitigation Plans were completed in December 2004. Acquire Information Technology and Business Transformation Services Contract: An Acquisition Plan for the Information Technology and Business Transformation has been developed and is currently being reviewed by key OPM staff. The Plan is to acquire vendor support to provide Business Transformation and Information Technology support services and products for the Retirement Systems Modernization's existing and future foundation elements and useful segments. The contract vehicle will be a single award Blanket Purchase Agreement that will enable OPM to interface and integrate the Defined Benefits Technology solution and Paper Data Conversion projects with existing OPM and other Federal agency applications and systems. The contract is estimated to be awarded in July 2005. The selected vendor will work to ensure that OPM's overall business processes and procedures are transformed and modernized in accordance with the directives mandated for OPM's Information Technology systems and applications. OPM included mandatory business requirements in the RFP for the Defined Benefit Technology and Paper Data Conversion solutions. Upon contract award, the remaining business requirements will be reviewed to determine whether the selected technology solutions should be customized or adopted without change. OPM's goal is to align Federal retirement administration functionality and goals with those used in administering private sector pension plans. Develop an RSM IT Security Plan: Information security is vital to all of the systems and applications within the OPM FEA, including RSM. RSM developed a detailed General Security System Requirements document which serves as the starting point for all RSM security implementation activities. This document, incorporating OPM IT Security policies and requirements as well as appropriate NIST and OMB guidance on information security, is reviewed, re-validated and updated as appropriate during the requirements gathering phase of each RSM segment or component. The detailed security requirements are then used to design the security-architecture for that segment or component. The three major acquisitions discussed above will include security requirements that the proposers must agree to and meet prior to award of the contracts. As RSM components are ready to be brought on-line, they will go through a rigorous Certification and Accreditation process, and will be included in OPM's disaster recovery and Continuation of Operations Plans. Improve Change Management: Change management is critical to the successful implementation of Retirement Systems Modernization. OPM developed a change management strategy early in the project. The strategy calls for the development of a detailed change management plan at the beginning of each implementation. This strategy reduces re-work that occurs when change management plans are developed too early in the implementation cycle. In preparation and recognition of the importance of change management. RSM recently hired an employee to lead this effort. In summary, we are committed to the successful implementation of the Retirement Systems Modernization initiative. We will strengthen our project management and oversight prior to beginning major implementation and will continue to look for opportunities to maximize our success as we implement the major components over the next several years. We appreciate your support in helping us attain these goals. Sincerely, Signed by: Kay Coles James: Director: [End of section] FOOTNOTES [1] H.R. Conf. Rep. No. 108-401, at 915 (2003); S. Rep. No. 108-146, at 162 (2003). GAO's Mission: The Government Accountability Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO's Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as "Today's Reports," on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select "Subscribe to e-mail alerts" under the "Order GAO Products" heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: