This is the accessible text file for GAO report number GAO-06-413 entitled 'Financial Management Systems: Lack of Disciplined Processes Puts Effective Implementation of Treasury's Governmentwide Financial Report System at Risk' which was released on April 21, 2005. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to the Secretary of the Treasury: April 2006: Financial Management Systems: Lack of Disciplined Processes Puts Effective Implementation of Treasury's Governmentwide Financial Report System at Risk: GAO-06-413: GAO Highlights: Highlights of GAO-06-413, a report to the Secretary of the Treasury: Why GAO Did This Study: For the past 9 years, since the first audit of the consolidated financial statements of the U.S. government (CFS), one of the major impediments to our ability to render an opinion on the CFS is that the federal government has not had adequate system, controls, and procedures to properly prepare the CFS. To address some of the internal control weaknesses identified in our audit report, Treasury began developing the Governmentwide Financial Report System (GFRS). The goal of this new system is to directly link information from federal agencies’ audited financial statements to amounts reported in the CFS, a concept that we strongly support. We reported internal control weaknesses and GAO recommendations regarding the preparation of the CFS, along with progress made in this area in a separate report. This report provides our assessment of Treasury’s ongoing effort to develop and implement GFRS and makes recommendations for reducing the risks associated with the development of GFRS. What GAO Found: Treasury’s development of GFRS is a positive initiative and Treasury has made progress towards preparing the CFS based on agencies’ audited financial statements, which has been one of our principal concerns. Treasury, though, has not yet effectively implemented the disciplined system development processes necessary to provide reasonable assurance that GFRS will meet all of its performance, schedule, and cost goals and result in the most efficient and effective means of preparing the CFS. Specifically, Treasury has not * developed a concept of operations or any other document that adequately defines or documents the expected performance of GFRS; * developed a detailed project plan and schedule through completion of GFRS; * developed a budget justification for GFRS in its Capital Asset Plan and Business Case (commonly referred to as the Exhibit 300), as called for in Office of Management and Budget (OMB) Circular No. A-11; and * implemented the disciplined processes necessary to effectively manage the GFRS project, which has contributed to usability problems encountered by its users. A disciplined software development and acquisition process can maximize the likelihood of achieving the intended results (performance) within established resources (costs) on schedule. Because of this relationship, these factors can be viewed as an equilateral triangle as shown below. [See PDF for Image] Source: GAO: [End of Figure] In our work at certain other federal agencies, we have found that project deficiencies such as those we have identified with the GFRS project have led to a range of problems, from increased cost and reduced functionality to system failure. Going forward, it will be important that Treasury better mitigate its risks so that long-standing internal control weaknesses regarding the preparation of the CFS can be eliminated and, more importantly, Treasury ends up with a system that fully meets its and agencies’ needs. What GAO Recommends: GAO is making three recommendations focused on reducing the risks associated with the development of GFRS. Treasury stated that it concurs with the recommendations in this report and is working to adopt them. To view the full product, including the scope and methodology, click on the link above. For more information, contact Gary Engel at (202) 512- 3406 or [Hyperlink engelg@gao.gov]. [End of Section] Contents: Letter: Results in Brief: Background: Scope and Methodology: Effective Implementation of the Disciplined Processes Are Key to Reducing Project Risks: Lack of Disciplined Processes Puts GFRS at Risk of Not Meeting Its Performance, Schedule, and Cost Goals: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Comments from the Department of Treasury: Figures: Figure 1: Percent of Effort Associated with Undisciplined Projects: Figure 2: Software Development Trade-off Triangle: Figure 3: Relationship between Requirements Development and Testing: [End of Section] United States Government Accountability Office: Washington, DC 20548: April 21, 2006: The Honorable John W. Snow: The Secretary of the Treasury: For the past 9 years, we have reported that the federal government did not have adequate systems, controls, and procedures to properly prepare its consolidated financial statements. Many of the weaknesses in internal control that contributed to our continuing disclaimers of opinion were identified by federal agency financial statement auditors during their audits of federal agencies' financial statements and were reported in detail with recommendations to the federal agencies in separate reports. However, some of the internal control weaknesses were identified during our tests of the Department of the Treasury's (Treasury) process for preparing the consolidated financial statements of the U.S. government (CFS). The ability to prepare the CFS has been a long-standing challenge for Treasury's Financial Management Service. To address some of the internal control weaknesses identified in our audit report,[Footnote 1] Treasury began developing the Governmentwide Financial Report System (GFRS). The goal of this new system is to directly link information from federal agencies' audited financial statements to amounts reported in the CFS, a concept that we strongly support and is a necessary prerequisite for our reliance on the individual agency audits in auditing the CFS. Once Treasury is able to achieve adequate systems, controls, and procedures to properly prepare its consolidated financial statements, a major impediment to our ability to audit the CFS would be eliminated. This report provides our assessment of Treasury's ongoing effort to develop and implement GFRS. Although the implementation of any information system, such as GFRS, is never risk-free, organizations that follow and effectively implement accepted best practices in systems development and implementation (commonly referred to as disciplined processes) have been shown to reduce these risks to acceptable levels. The term acceptable levels acknowledges the fact that any system acquisition has risks and will suffer the adverse consequences associated with defects. A disciplined software development and acquisition process can maximize the likelihood of achieving the intended results (performance) within established resources (costs) on schedule. Effective implementation of the disciplined processes reduces the potential for risks to occur and helps prevent those that do occur from having any significant adverse impact on the cost, timeliness, and performance of the project. Our reviews have found, like those performed by others, that the effectiveness of the processes used to manage the project provide valuable insight into whether a project will meet its cost, schedule, and performance objectives. We last briefed your staff on our initial assessment of this development effort in September 2004. This report updates our work and focuses on whether Treasury has effectively implemented disciplined processes necessary to reduce the risks to the GFRS project to acceptable levels and accordingly provide reasonable assurance that GFRS will meet its performance, schedule, and cost goals. We reported internal control weaknesses and our recommendations regarding the preparation of the CFS, along with progress made in this area in a separate report.[Footnote 2] Results in Brief: For fiscal years 2004 and 2005 Treasury's GFRS was able to capture certain agency financial information from agencies' audited financial statements. At the same time, the system was not yet at the stage of development that it could be used to compile the consolidated financial statements from the information that was captured. Therefore, for fiscal years 2004 and 2005 Treasury continued to have to primarily use manual procedures to prepare the CFS as GFRS continued to be developed. We identified some risks in Treasury's development of GFRS. We found that Treasury had not yet effectively implemented the disciplined processes necessary to provide reasonable assurance that GFRS will meet its performance, schedule, and cost goals. Specifically, Treasury had not: ˛ developed a concept of operations or any other document that adequately defines or documents the expected performance of GFRS; ˛ developed a detailed project plan and schedule through completion of GFRS; ˛ developed a budget justification for GFRS in its Capital Asset Plan and Business Case (commonly referred to as the Exhibit 300), as called for in Office of Management and Budget (OMB) Circular No. A- 11;[Footnote 3] and: ˛ implemented the disciplined processes necessary to effectively manage the GFRS project, which has contributed to usability problems encountered by its users. In our work at certain other federal agencies, we have found that not effectively implementing disciplined processes has led to a range of problems, from increased cost and reduced functionality to system failure. Going forward, it will be important that Treasury better mitigate its risks so that long-standing internal control weaknesses regarding the preparation of the CFS can be eliminated and, more importantly, Treasury ends up with a system that fully meets its and agencies' needs. This report includes three recommendations to the Secretary of the Treasury and the Fiscal Assistant Secretary focused on reducing the risks associated with the development of GFRS. In commenting on a draft of this report, Treasury stated that it concurs with our recommendations and is working to adopt them. Treasury also stated that it plans to develop a concept of operations for GFRS and recently started to develop a budget justification for GFRS in a Capital Asset Plan and Business Case. Further, Treasury stated that although it uses manual procedures to prepare the CFS, it does not agree that manual processes necessarily create internal control weaknesses. We agree. Our discussion of Treasury's manual processes was not intended to imply that using manual processes will by itself lead to internal control weaknesses; rather, it was to highlight that Treasury, as a user of GFRS, has to perform manual compilation procedures because GFRS was not yet at the stage of development that it could be used to compile the consolidated financial statements from the agency financial information that was captured. Background: Treasury began implementing a new process for preparing the CFS using GFRS during fiscal year 2004. GFRS is an application that is being developed to meet the business needs of Treasury to prepare the CFS. Treasury has stated that the goal of the new system is to be able to directly link information from federal agencies' audited financial statements to amounts reported in the CFS and to address our recommendations[Footnote 4] regarding the preparation of the CFS. For the fiscal year 2004 and 2005 reporting process, GFRS had enough functionality to capture certain federal agency financial information from federal agencies' audited financial statements. We also found that Treasury made progress in demonstrating that amounts in the Balance Sheet and the Statement of Net Cost were consistent with federal agencies' audited financial statements prior to eliminating intragovernmental activity and balances. GFRS, though, was not yet at the stage of development that it could be used to compile the CFS from the information that was captured. Treasury's plans had been to evaluate and consider the manual processes used during fiscal year 2004 to develop the necessary requirements needed for the development of the compilation module of GFRS. Although some enhancements to GFRS were implemented for fiscal year 2005, Treasury continued to primarily use manual procedures to prepare the most recently issued CFS. As part of the concept of directly linking information from agencies' audited financial statements to amounts reported in the CFS, Treasury has classified 35 federal agencies as verifying[Footnote 5] agencies and approximately 125 as nonverifying agencies and required these agencies to enter data in GFRS. Verifying agencies are required to reclassify certain of their audited financial statement line items to the appropriate CFS line items within GFRS. Both verifying and nonverifying agencies are required to enter certain of their financial statement notes and other requested data into GFRS. Treasury then uses this information to prepare the CFS. GFRS users at verifying and nonverifying agencies include financial statement preparers, Chief Financial Officers (CFOs), Inspectors General (IGs), and contracted independent public accountants. GFRS grants certain permissions to different user roles which limit each type of user to designated locations within the system. Only certain roles have permission to input, modify, and approve data. Scope and Methodology: As part of our annual audit of the CFS, we evaluate Treasury's financial reporting procedures and related internal control, including the development of GFRS. Our work was performed in accordance with U.S. generally accepted government auditing standards. In fiscal years 2004 and 2005, GFRS was used to collect federal agencies' audited financial statement data compiled in the CFS. To assess Treasury's implementation of disciplined processes in the development of GFRS, we reviewed industry standards and best practices from the Institute of Electrical and Electronics Engineers (IEEE), Software Engineering Institute (SEI), OMB Circular No. A-11, and prior GAO reports. Beginning in fiscal year 2004, we reviewed and analyzed available GFRS documentation and made inquiries of Treasury's GFRS project team and other Treasury personnel to determine whether Treasury has effectively implemented disciplined processes necessary to reduce the risks to the GFRS project to acceptable levels and accordingly provide reasonable assurance that GFRS will meet its performance, schedule, and cost goals. We briefed Treasury on our initial assessment of this development effort in September 2004. We reported internal control weaknesses related to GFRS in our fiscal year 2004 audit report[Footnote 6] and made a recommendation related to GFRS in our fiscal year 2004 internal control report.[Footnote 7] In fiscal year 2005, we reviewed documentation related to the planned enhancements for GFRS for 2005 and made inquiries of Treasury's GFRS project team and other Treasury personnel to further determine whether Treasury has effectively implemented disciplined processes. We again reported internal control weaknesses related to GFRS in our fiscal year 2005 audit report. We requested comments on a draft of this report from the Secretary of the Treasury or his designee. Treasury's comments are reprinted in appendix I and are also discussed in the Agency Comments and Our Evaluation section. Effective Implementation of the Disciplined Processes Are Key to Reducing Project Risks: Disciplined processes, which are fundamental to successful systems development and implementation efforts, have been shown to reduce the risks associated with software development and acquisition to acceptable levels. A disciplined software development and acquisition process can maximize the likelihood of achieving the intended results (performance) within established resources (costs) on schedule. Although there is no standard set of practices that will ever guarantee success, several organizations, such as SEI[Footnote 8] and IEEE,[Footnote 9] as well as individual experts, have identified and developed the types of policies, procedures, and practices that have been demonstrated to reduce development time and enhance effectiveness. The key to having a disciplined system development effort is to have disciplined processes in multiple areas, including project planning and management, requirements management, configuration management, risk management, quality assurance, and testing. Effective processes should be implemented in each of these throughout the project life cycle because change is constant. Effectively implementing the disciplined processes necessary to reduce project risks to acceptable levels is hard to achieve because a project must effectively implement several best practices, and inadequate implementation of any one may significantly reduce or even eliminate the positive benefits of the others. As part of the disciplined processes, acquiring and implementing a new financial management system such as GFRS requires a methodology that starts with a clear definition of the organization's mission and strategic objectives and ends with a system that meets specific information needs. We have seen many system efforts fail because federal agencies started with a general need, but did not define in precise terms (1) the specific problems they were trying to solve, (2) what their operational needs were, and (3) what specific information requirements flowed from these operational needs. Instead, they plunged into the acquisition and implementation process in the belief that these specifics would somehow be defined along the way. The typical result was that systems were delivered well past anticipated milestones; failed to perform as expected; and, accordingly, were over budget because of required costly modifications. Figure 1 shows how organizations that do not effectively implement the disciplined processes lose the productive benefits of their efforts as a project continues through its development and implementation cycle. Although undisciplined projects show a great deal of productive work at the beginning of the project, the rework associated with defects begins to consume more and more resources. In response, processes are adopted in the hopes of managing what later turns out, in reality, to have been unproductive work. Generally, these processes are "too little, too late" and rework begins to consume more and more resources because sufficient foundations for building the systems were not established or not established adequately. Experience has shown that projects for which disciplined processes are not implemented at the beginning are forced to implement them later when it takes more time and they are less effective.[Footnote 10] Figure 1: Percent of Effort Associated with Undisciplined Projects: [See PDF for image] Source: Reproduced by permission from Steve McConnell, Professional Software Development: Shorter Schedules, Higher Quality Products, More Successful Projects, Enhanced Careers(Boston, Mass: Pearson Education Inc., 2004). [End of figure] As shown in figure 1, a major consumer of project resources in undisciplined efforts is rework (also known as thrashing). Rework occurs when the original work has defects or is no longer needed because of changes in project direction. Disciplined organizations focus their efforts on reducing the amount of rework because it is expensive. Fixing a requirements defect after the system is released costs anywhere from 10 to 100 times as much as fixing it when the requirements are defined.[Footnote 11] As shown in figure 1, projects that are unable to successfully address their rework will eventually only be spending their efforts on rework and the associated processes rather than on productive work. In other words, the project will continually find itself reworking items. A disciplined software development and acquisition process can maximize the likelihood of achieving the intended results (performance) within established resources (costs) on schedule. Because of this relationship, these factors can be viewed as an equilateral triangle as shown in figure 2. Figure 2: Software Development Trade-off Triangle: [See PDF for image] Source: GAO. [End of figure] The performance corner of the triangle is also referred to as the product corner. This corner includes quality and all other product- related attributes including complexity, usability, maintainability, and defect rate. For example, beyond a certain point, it is difficult to increase performance without changing the cost and/or schedule attributes. Similarly, a schedule cannot be drastically compressed without degrading the quality of the software project or increasing the cost of development. Schedule, cost, and performance must be in balance for a project to succeed. The following sections of this report address our observations regarding the performance, schedule, and cost of Treasury's GFRS development effort. Lack of Disciplined Processes Puts GFRS at Risk of Not Meeting Its Performance, Schedule, and Cost Goals: Treasury's development of GFRS is a positive initiative and Treasury has made progress towards preparing the CFS based on agencies' audited financial statements, which is one of our principal concerns. While we support the goals of GFRS, we are concerned that Treasury has not yet effectively implemented a number of disciplined processes that have been shown to reduce project risks to acceptable levels. Specifically, we noted that Treasury has not yet implemented the types of project management processes necessary to provide the information required to assess the progress of GFRS and whether it is meeting its cost, schedule, and performance objectives. First, Treasury has not described its vision for the project in a manner that can be used to guide the development efforts. Although disciplined projects use a concept of operations document to serve this purpose, Treasury has not yet developed such a document. Treasury also has not developed a detailed project plan or schedule through the completion of GFRS or followed OMB's guidance for major investment planning, budgeting, and acquisition. Some adverse impacts associated with these process weaknesses have already been realized. Specifically, significant usability issues have arisen and little meaningful project management information is available to assess the project performance. Treasury Has Not Developed a Concept of Operations: Treasury has not adequately defined or documented the expected functionality for GFRS. One approach for describing the functionality that is expected from a system is to document the vision in a concept of operations. According to IEEE Standard 1362-1998, a concept of operations document is normally one of the first documents produced during a disciplined development effort because it describes system characteristics for a proposed system from the user's viewpoint. This is important because a good concept of operations document can be used to communicate overall quantitative and qualitative system characteristics to the user, developer, and other organizational elements typically involved in a systems development effort. This allows the reader to understand the user organizations, missions, and organizational objectives from an integrated systems point of view. Until such a document is developed, it is virtually impossible for the project to develop a firm foundation for its requirements and testing activities. Experience has shown that poor requirements[Footnote 12] are a major factor associated with projects that do not meet their cost, schedule, and performance objectives. Furthermore, without good requirements, it is impossible to develop a disciplined testing program. Figure 3 shows the relationship between requirements and testing and how the concept of operations provides the foundation for the requirements. Figure 3: Relationship between Requirements Development and Testing: [See PDF for image] Source: GAO. [End of figure] The Treasury project team stated they consider the November 2001 Working Group Report on the Financial Report of the United States Government Preparation Process to be their concept of operations. Our analysis noted at least two reasons why this document would not be acceptable for that purpose. First, the November 2001 report resembles an alternatives analysis where Treasury weighed several options to change its current process for preparing the CFS. The report does not address the objectives of a concept of operations because it does not outline the business processes that are expected to be implemented in the new system. For example, when we compared the November 2001 report to the current functionality of GFRS, we found that the November 2001 report stated that federal agencies would be required to submit an analysis of changes in net position and to split net position between intragovernmental and operations with the public. However, over 2 years ago Treasury stated to GAO that they would not be requiring federal agencies to split their net position and did not provide an alternative solution for addressing this key net position weakness affecting the preparation of the CFS. Second, the November 2001 report was issued prior to the Federal Accounting Standards Advisory Board's issuance of Statement of Federal Financial Accounting Standard No. 24, Selected Standards for the Consolidated Financial Report of the United States Government, which requires two additional financial statements to be included in the Financial Report--Reconciliation of Net Operating Cost and Unified Budget Deficit and Statement of Changes in Cash Balance from Unified Budget and Other Activities. Because the November 2001 report could not contemplate the preparation of these two statements, it is at least partially obsolete in terms of reporting requirements for two of the five primary consolidated financial statements. Going forward, an effective concept of operations could help Treasury reach its goal of having GFRS, when fully implemented, resolve several continuing weaknesses that we have identified during our CFS audit work, including providing additional functionality that would allow Treasury to eliminate some of the manual processes and provide improvements from its previous process. Treasury has not yet developed and documented in a concept of operations document or any other document (1) the corrective actions for financial reporting weaknesses that the system is expected to address and (2) the functionality it needs GFRS to provide when it is completed to implement these corrective actions. Accordingly, Treasury does not have adequate assurance that the resulting system will have the needed functionality to address the weaknesses that Treasury management may expect the system to eliminate or reduce the time it takes Treasury to produce the CFS. This basic information is critical for the project team to effectively develop a complete set of high-level requirements for GFRS and reach agreement with Treasury management on the functionality GFRS is expected to provide and how to determine when GFRS is complete and has ultimately met management's expectations. Treasury Has Not Developed a Detailed Project Plan and Schedule through Completion of GFRS: Treasury has not clearly defined what GFRS will encompass and when that functionality is expected to be delivered. This basic information should serve as the foundation to support realistic project scheduling efforts, so that Treasury could (1) determine the work needed to complete the project and (2) make informed decisions on when that work can be expected to be completed. While we understand that GFRS will take several years to develop, a "blueprint" guiding these efforts has not been developed. Rather than using best practices to schedule out the functionality to be developed and implemented over the entire development period, each year Treasury decides on the high-level requirements to be developed and implemented only during that given year. These requirements are not "placed in context" to the overall system goals and needs. This approach presents the following issues. ˛ It is unclear how the requirements selected for implementation each year move GFRS closer to providing the necessary functionality to address the material weaknesses Treasury expects the system to correct. ˛ Without a detailed project plan and schedule through completion of GFRS, it is unclear whether the project is capable of meeting its established time frames and goals, how to determine the progress made with a given year's efforts, or how to estimate the funding necessary to develop the system. These weaknesses have already had adverse project impacts. Specifically, even once the requirements for a given year have been "defined", we found that little meaningful project scheduling was used. For example, Treasury provided us with a list of its milestones for the fiscal year 2005 development of GFRS; however, this documentation did not provide information such as the detailed tasks to be performed, duration of the tasks, person responsible for completing each task, or how these development efforts link to the overall functionality required for GFRS. Accordingly, Treasury did not have the basic project management information necessary to determine whether its efforts were meeting the project's cost, schedule, and performance objectives. In our September 2004 briefing to Treasury on our initial assessment of the GFRS development effort, we noted that Treasury faced the risk of rework by using a single-year scheduling approach rather than developing a schedule for the entire development period. We stated that implementing any of management's high-level requirements for the current year of development could cause significant rework of the functionality that was implemented in the prior year. Disciplined organizations focus their efforts on reducing the amount of rework because it is expensive. Without a detailed project plan and schedule, the program does not have a process that is designed to avoid doing things twice and runs a much greater risk that it will cost additional time, money, and resources. Impacts of Not Following OMB Guidance: Treasury has not yet developed a budget justification for GFRS in a Capital Asset Plan and Business Case (commonly referred to as the Exhibit 300), called for in OMB Circular No. A-11. The OMB circular provides instructions on how to prepare the necessary budget justification and reporting requirements for major information technology investments,[Footnote 13] such as GFRS. The Exhibit 300 helps demonstrate to federal agency management and OMB that the federal agency has employed the disciplines of good project management, represented a strong business case for the investment, and met other administration priorities to define the proposed cost, schedule, and performance goals for the investment if funding approval is obtained. The business case should include security, privacy, and enterprise architecture, and provide the effectiveness and efficiency gains planned by the business lines and functional operations. According to Treasury personnel, it was a Treasury policy decision not to prepare an Exhibit 300 for submission to OMB. The circular states that an "exhibit 300 must be submitted for all major investments in accordance with this section." The circular also notes that "[g]ood budgeting requires appropriations for the full risk adjusted costs of asset acquisition be enacted in advance to help ensure that all costs and benefits are fully taken into account when decisions are made about providing resources." We found that Treasury has not yet determined the cost of GFRS to date or estimated its cost through completion. Instead, Treasury's project team stated that, to date, they have used funds remaining from other Treasury projects to pay for the development of GFRS.[Footnote 14] According to Treasury's project team, the development of GFRS is guided, limited, and constrained by the funds that they can "beg, borrow, and steal" from other Treasury development efforts. As noted in OMB Circular No. A-11, when "capital assets are funded in increments, without certainty if or when future funding will be available, it can and occasionally does result in poor planning, acquisition of assets not fully justified, higher acquisition costs, project (investment) delays, cancellation of major investments, the loss of sunk costs, or inadequate funding to maintain and operate the assets." GFRS Has Some Usability Issues: Effectively implementing disciplined processes helps ensure the system meets the users' needs. To date, GFRS has had some significant usability issues. In February 2005, Treasury held a forum with GFRS federal agency users, including agency CFO, IG, and independent public accountant staff, to discuss lessons learned from the fiscal year 2004 GFRS reporting process and planned changes to GFRS for fiscal year 2005. At the forum, the users noted the lack of user-friendly reporting capabilities in GFRS. As we noted in our September 2004 briefing to Treasury, user-friendly reporting capabilities were not available to federal agency users for fiscal year 2004 and such functionality was critical to the ultimate usefulness of the system. The process of viewing the information entered into GFRS is especially burdensome for the 32 federal agencies[Footnote 15] required to have this information audited. GFRS does not provide the ability for an agency to develop one report containing all of the information required to be entered into GFRS. Instead, users are required to run over 35 separate reports in GFRS, which has proven to be time-consuming. Compounding this is that agencies have only a very short period of time to enter the data into GFRS. Specifically, federal agencies are required to have their audited financial information in the GFRS closing package by November 18--just 3 days after their audited financial statements are due to OMB. We believe the lack of user-friendly reporting capabilities is a result of Treasury not effectively implementing disciplined processes. Treasury had not defined types of reporting that would be available to the users and when it would be available. Treasury's project team stated that it is now trying to address this issue and has acknowledged that a lack of an adequate understanding of users' reporting needs contributed to the problem. They also noted the lack of funding as another reason this issue has not been fully addressed. In our view, if Treasury had implemented disciplined processes during the planning of GFRS, such as developing a concept of operations and implementing an effective requirements management process, this issue (1) would have been identified and (2) actions could have been taken at the outset to develop a viable solution earlier in the process rather than waiting until significant amounts of development work had been performed. In our discussions with the Treasury project team over the reporting issue, we were told that they are concerned that providing more robust reporting capabilities to users could cause the system to fail and "crash" should all of the GFRS users run these reports at the same time. In our September 2004 briefing, we noted that one of the challenges to Treasury was adequately testing GFRS to ensure that it could function properly when all federal agencies use the system at the same time, which includes allowing users to run all needed GFRS reports. Treasury's project team told us that they were unable to test GFRS capacity because they had not identified or developed all the needed user-friendly reporting capabilities. We believe other usability issues exist that directly affect Treasury as a user of GFRS as well. For example, currently, Treasury cannot rely on the system to compile the CFS from the information that is captured. Therefore, for 2 years in a row, Treasury primarily used manual procedures to prepare the CFS. While Treasury's initial plans were to evaluate and consider the manual processes used during fiscal year 2004 to develop the necessary requirements needed for the development of the compilation module of GFRS, this capability has not been added and it is unclear when or if it will be available. Conclusions: The implementation of any major system, such as GFRS, is not a risk- free proposition. However, organizations that follow and effectively implement accepted best practices in systems development and implementation have been shown to reduce these risks to acceptable levels. Treasury has not yet done so. If it continues on this path, it runs an unnecessary risk of building a system that may be more costly and take longer to deploy, while not providing all of the intended system functionality. Going forward, it will be important that Treasury take the actions necessary to implement the disciplined processes that provide reasonable assurance that GFRS will achieve Treasury's ultimate goal of preparing the CFS with a direct link to federal agencies' audited financial statements and addressing long-standing internal control weaknesses. A key first step will be for Treasury to develop and document its view of how GFRS will operate, including how it will be used to address significant internal control weaknesses. Recommendations for Executive Action: We recommend that the Secretary of the Treasury direct the Treasury Fiscal Assistant Secretary to take the following three actions to reduce the risks associated with the development of GFRS: ˛ Follow the budget justification and reporting processes set out in OMB Circular No. A-11 and ensure that funding needs are fully addressed. ˛ Develop and adopt a concept of operations that fully describes the functionality expected to be provided by GFRS and the material internal control weaknesses that are expected to be addressed by the system. ˛ Develop and effectively implement the disciplined processes necessary to properly manage the development of GFRS, including the development of a detailed project plan and schedule through the completion of the system. Agency Comments and Our Evaluation: In written comments on a draft of this report, which are reprinted in appendix I, Treasury stated that our report clearly points out recommendations for reducing the risks associated with the continued development of GFRS. Treasury also stated that it concurs with our recommendations and is working to adopt them. Further, Treasury stated that it plans to develop a concept of operations for GFRS and recently started to develop a budget justification for GFRS in a Capital Asset Plan and Business Case. Treasury provided additional comments regarding Treasury's manual procedures used in preparing the CFS. In particular, Treasury stated that although it uses manual procedures to prepare the CFS, it does not agree that manual processes necessarily create internal control weaknesses. We agree. Our discussion of Treasury's manual processes was not intended to imply that using manual processes will lead to internal control weaknesses, but rather to highlight that Treasury, as a user of GFRS, has to perform manual compilation procedures because GFRS was not yet at the stage of development that it could be used to compile the consolidated financial statements from the agency financial information that was captured. Our understanding is that one of the automated functions Treasury expected GFRS to perform was the compilation of the CFS from the information captured rather than performing these functions manually. This capability has not yet been added to GFRS, and it is unclear when or if it will be available because, as we have reported, Treasury does not currently have a concept of operations that defines or documents the expected functionality for GFRS. This report contains recommendations to the Secretary of the Treasury. The head of a federal agency is required by 31 U.S.C. 720 to submit a written statement on actions taken on these recommendations. You should submit your statement to the Senate Committee on Homeland Security and Governmental Affairs and the House Committee on Government Reform within 60 days of the date of this report. A written statement must also be sent to the House and Senate Committees on Appropriations with the agency's first request for appropriations made more than 60 days after the date of the report. We are sending copies of this report to the Chairmen and Ranking Minority Members of the Senate Committee on Homeland Security and Governmental Affairs; the Subcommittee on Federal Financial Management, Government Information, and International Security, Senate Committee on Homeland Security and Governmental Affairs; the House Committee on Government Reform; and the Subcommittee on Government Management, Finance, and Accountability, House Committee on Government Reform. In addition, we are sending copies to the Fiscal Assistant Secretary of the Treasury. Copies will be made available to others upon request. This report is also available at no charge on GAO's Web site at [Hyperlink=http://www.gao.gov]. We appreciate the courtesy and cooperation extended to us by your staff throughout our work. We look forward to continuing to work with your offices to help improve systems development efforts in the federal government. If you have any questions about the contents of this report, please contact Gary Engel, Director, Financial Management and Assurance, who may be reached at (202) 512-3406 or by e-mail at: engelg@gao.gov, or Keith Rhodes, Chief Technologist, Applied Research and Methods, who may be reached at (202) 512-6412 or by e-mail at rhodesk@gao.gov. Key contributors to this letter were Chris Martin, Lynda Downing, and Katherine Schirano. :Signed By Gary T. Engel: Director: Financial Management and Assurance: [Signed By] Keith A. Rhodes: Chief Technologist: Applied Research and Methodology Center for Engineering and Technology: [End of section] Appendix I: Comments from the Department of Treasury: Department Of The Treasury: Financial Management Service: Washington, D.C. 20227: Commissioner: April 3, 2006: Mr. Gary T. Engel: Director, Financial Management and Assurance: Government Accountability Office: Washington, DC 20548: Dear Mr. Engel: Thank you for the opportunity to comment on GAO's draft report, GAO-06- 413, Lack of Disciplined Processes Puts Effective Implementation of Treasury's Governmentwide Financial Report System at Risk. Your report clearly pointed out 3 recommendations for reducing the risks associated with the continued development of the Governmentwide Financial Report System (GFRS). We concur with these recommendations and are working to adopt them. We plan to develop a Concept of Operations for GFRS and we recently started to develop a budget justification for GFRS in a Capital Asset Plan and Business Case. While we acknowledge that our system development process could be more disciplined and structured, the GFRS has been very successful in capturing the audited financial statement information from the agency for both FY04 and FY05. GFRS has also been very effective in reclassifying this agency data for compilation into the Financial Report (FR). In addition, the internal control security weaknesses identified by GAO in FY05 have been addressed by adopting and performing an Oracle Security Baseline Review. The Security Baseline details the requirements that FMS has set forth to protect the application. There are two other items we would like to call to your attention in the section titled "GFRS has some usability issues". 1. We have put in place a structured process to solicit, evaluate and prioritize agency requests for enhancements to GFRS and, to that extent, have strived to add and modify functionality as soon as reasonably possible within a limited development time frame and budget constraints. Our enhancements for GFRS are planned through FY07 and we will maintain a plan for the next several years. For FY 2006, we are accommodating two major agency requests; the roll-over of prior year reported data into the prior year fields and the ability to print multiple reports. In addition, every year, we conduct new user training during the summer and provide a training class on the new functionality for existing users, when applicable. In addition, we conducted a forum for the CFO and IG representatives to evaluate possible future improvements in GFRS. 2. We agree that some manual processes are used to prepare the FR. However, we do not agree with the notion that manual processes necessarily create internal control weaknesses. Instead it is necessary to look at the specific controls that are used to ensure the integrity of the entire process, both the automated and the manual processes. How we accomplish a task should not be judged only on the manual procedures involved but also by taking into account the internal controls that we have in place to mitigate the risks involved and ensure the integrity of the data. We have spoken to representatives from major corporations such as ALCOA and Martin Marietta regarding the processes they use to prepare their financial statements. Many of them use a combination of automated and manual processes and they agree that some manual procedures are often necessary and by themselves do not create internal control weaknesses. In conclusion, we will continue to work with the members of your team to resolve any security and system development concerns. Sincerely, [Signed By] Richard L. Gregg: Cc: Donald Hammond: Cc: Linda Combs, OMB: Cc: Danny Wersel, OMB: (198415): [End of section] FOOTNOTES: [1] Department of the Treasury, 2005 Financial Report of the United States Government (Washington, D.C.: December 2005). This report includes GAO's audit report. [2] GAO, Financial Audit: Significant Internal Control Weaknesses Remain in Preparing the Consolidated Financial Statements of the U.S. Government, GAO-06-415 (Washington, D.C.: Apr. 21, 2006). [3] Section 300 of OMB Circular No. A-11, Preparation, Submission, and Execution of the Budget (Nov. 2, 2005), sets forth requirements for federal agencies for planning, budgeting, acquiring, and managing information technology capital assets. [4] GAO, Financial Audit: Process for Preparing the Consolidated Financial Statements of the U.S. Government Continues to Need Improvement, GAO-05-407 (Washington, D.C.: May 4, 2005). [5] Treasury defines the verifying agencies as the 24 Chief Financial Officer (CFO) Act agencies, Export-Import Bank of the United States, Farm Credit System Insurance Corporation, Federal Communications Commission, Federal Deposit Insurance Corporation, National Credit Union Administration, U.S. Postal Service, Pension Benefit Guaranty Corporation, Railroad Retirement Board, Securities and Exchange Commission, Smithsonian Institution, and Tennessee Valley Authority. Treasury also refers to the verifying agencies as "significant" agencies. [6] Department of the Treasury, 2004 Financial Report of the United States Government (Washington, D.C.: December 2004). This report includes GAO's audit report. [7] GAO-05-407. [8] SEI is a federally funded research and development center operated by Carnegie Mellon University and sponsored by the Department of Defense. The SEI objective is to provide leadership in software engineering and in the transition of new software engineering technologies into practice. [9] IEEE is a nonprofit, technical professional association that develops standards for a broad range of global industries including the information technology and information assurance industries. The IEEE standards program serves the global needs of industry, government, and the public. It also works to assure the effectiveness and high visibility of the standards program both within IEEE and throughout the global community. [10] Steve McConnell, Rapid Development: Taming Wild Software Schedules (Redmond, Wash: Microsoft Press, 1996). [11] Steve McConnell, Code Complete, Second Edition (Redmond, Wash: Microsoft Press, 2004). [12] Requirements are the specifications that system developers and program managers use to design, develop, and acquire a system. They need to be unambiguous, consistent with one another, verifiable, and directly traceable to higher level business or functional requirements. It is critical that requirements flow directly from the organization's concept of operations. [13] OMB Circular No. A-11, section 300 defines a major investment as a system or project requiring special management attention because of its importance to the mission or function of the agency, a component of the agency or another organization; is for financial management and obligates more than $500,000 annually; has significant program or policy implications; has high executive visibility; has high development, operating, or maintenance costs; or is defined as major by the agency's capital planning and investment control process. [14] We did not review Treasury's alternative funding sources. [15] The Treasury Financial Manual states that the Inspector General or their contracted independent public accountant for each verifying agency, except those agencies with a year end other than September 30 (Federal Deposit Insurance Corporation, National Credit Union Administration, and Farm Credit System Insurance Corporation), must opine on the closing package data, entered by the Chief Financial Officer into GFRS, as to its consistency with the comparative, audited, consolidated, department-level financial statements. Of the 35 verifying agencies, 32 are currently required to have an audit of their closing package. GAO's Mission: The Government Accountability Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO's Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as "Today's Reports," on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select "Subscribe to e-mail alerts" under the "Order GAO Products" heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: