This is the accessible text file for GAO report number GAO-13-304 entitled 'California High-Speed Passenger Rail: Project Estimates Could Be Improved to Better Inform Future Decisions' which was released on March 29, 2013. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Requesters: March 2013: California High-Speed Passenger Rail: Project Estimates Could Be Improved to Better Inform Future Decisions: GAO-13-304: GAO Highlights: Highlights of GAO-13-304, a report to congressional requesters. Why GAO Did This Study: The planned 520-mile California high-speed rail project, which would link San Francisco to Los Angeles, would be designed to operate at speeds up to 220 miles per hour. At an estimated cost of $68.4 billion (in year-of-expenditure dollars), it is expected to be one of the most expensive transportation projects undertaken in the United States. The Authority is responsible for implementing the project and federal funding is being provided from the FRA’s High-Speed Intercity Passenger Rail program. GAO reviewed (1) the reliability of project cost estimates, (2) the reasonableness of revenue and passenger rail ridership forecasts, (3) the risks attendant with the project’s funding plan, and (4) the comprehensiveness with which the project’s economic impacts were identified. GAO obtained documents from and conducted interviews with federal officials and officials from the Authority related to cost, financing, ridership and revenue modeling and estimation, and business plans and analyses related to potential economic impacts. GAO also interviewed state and local officials as well as the project’s peer review group members. What GAO Found: The California High-Speed Rail Authority (Authority) met some, but not all of the best practices in GAO’s Cost Estimating and Assessment Guide (Cost Guide) for producing cost estimates that are accurate, comprehensive, well documented, and credible. By not following all best practices, there is increased risk of such things as cost overruns, missed deadlines, and unmet performance targets. The Authority substantially met the criteria for the accurate characteristic by, for example, the cost estimate’s reflecting the current scope of the project. However, the Authority partially met the criteria for the other three characteristics since the operating costs were not sufficiently detailed (comprehensive), the development of some cost elements were not sufficiently explained (well documented), and because no systematic assessment of risk was performed (credible). The Federal Railroad Administration (FRA) issued limited guidance for preparing cost estimates, and this guidance did not reflect best practices in the Cost Guide. The Authority plans to improve its cost estimates. GAO found the Authority’s ridership and revenue forecasts to be reasonable; however, additional updates are necessary to refine the ridership and revenue model for the 2014 business plan. GAO also found the travel-demand-modeling process used to generate these forecasts followed generally accepted travel-demand-modeling practices. For example, the Authority revised several assumptions, such as gasoline price forecasts, to reflect changes in current and anticipated future conditions. However, additional updates, such as the development of a new travel survey, will be necessary to further refine these forecasts and improve the model’s utility to make future decisions. External peer review groups have also recommended additional updates. The project’s funding, which relies on both public and private sources, faces uncertainty, especially in a tight federal and state budget environment. Obtaining $38.7 billion in federal funding over the construction period is one of the biggest challenges to completing this project. In the latter stages, the Authority will also rely on $13.1 billion in private-sector financing, but will require more reliable operating cost estimates and revenue forecasts to determine whether, or the extent to which, the system will be profitable. The Authority’s plan recognizes the uncertainty of the current funding environment and is building the project in phases. The Authority has also identified an alternative funding source. However, that funding source is also uncertain. The Authority did a comprehensive job in identifying the potential economic impacts of the high-speed rail project. This includes identification of user impacts, such as effects on travel time reliability, and non-user impacts, such as effects on highway congestion. However, the nature of specific economic impacts will depend on a number of factors, including future project decisions. GAO also found limitations in the Authority’s benefit-cost analysis of the project that could limit its usefulness to decision makers. Finally, GAO found that construction of the high-speed rail project will not eliminate the need for additional improvements to meet future statewide-travel demand, but current statewide-transportation assessments and planning have given little consideration to this issue. What GAO Recommends: To produce reliable cost estimates, FRA should improve its guidance so it is in line with the best practices in GAO’s Cost Guide. The Department of Transportation did not agree or disagree with the recommendation but said, with further analysis, applying the Cost Guide would be feasible. The Authority said it will incorporate many of the report’s findings into future cost and ridership estimates. View [hyperlink, http://www.gao.gov/products/GAO-13-304]. For more information, contact Susan Fleming at (202) 512-2834 or flemings@gao.gov. [End of section] Contents: Letter: Background: Authority Substantially or Partially Met GAO's Best Practices for Producing Reliable Cost Estimates, but Can Make Improvements: Ridership and Revenue Forecasts Are Reasonable for Current Purposes, but Will Require Updates: The Authority's Funding Plan Faces Uncertainty: The Authority Comprehensively Identified Potential Economic Impacts, but It Is Too Early to Determine Specific Impacts: Conclusion: Recommendation: Agency Comments and Our Evaluation: Appendix I: Objectives, Scope, and Methodology: Appendix II: GAO Analysis of California High-Speed Rail Authority's Cost Estimates: Appendix III: Description of Generally Accepted Travel-Demand-Modeling Practices and Authority's Methods for Developing Ridership and Revenue Forecasts: Appendix IV: Comments from the California High-Speed Rail Authority: Appendix V: GAO Contact and Staff Acknowledgment: Tables: Table 1: Summary of GAO's Assessment of California High-Speed Rail Authority's Cost Estimates: Table 2: Description of Updated California High-Speed Rail Travel- Demand-Model Assumptions: Table 3: Funding Secured for Constructing the California High-Speed Rail Project, as of February 2013: Table 4: The Authority's Funding Plan for Constructing Phase 1 of the California High-Speed Rail Project, as of April 2012: Table 5: Primary Economic Impacts Associated with High-Speed Rail Identified by the Authority in the April 2012 Revised Business Plan, BCA, or EIA: Table 6: Organizations and Individuals Interviewed: Table 7: Guidance and Reports Used to Identify Generally Accepted Travel-Demand-Modeling Practices: Figures: Figure 1: Map of Planned California High-Speed Rail System and Key Construction Phases, 2013: Figure 2: Time Line of California High-Speed Rail Project: Abbreviations: AAPOR: American Association for Public Opinion Research: ASC: Aviation System Consulting, LLC: BCA: benefit-cost analysis: CTC: California Transportation Commission: DOT OIG: Department of Transportation Office of Inspector General: EIA: economic impact analysis: FHWA: Federal Highway Administration: FRA: Federal Railroad Administration: FTA: Federal Transit Administration: HSIPR: High Speed Intercity Passenger Rail: ICE: independent cost estimate: IOS: initial operating segment: ITS: Institute of Transportation Studies: LAO: Legislative Analyst's Office: LOS: level of service: MPO: Metropolitan Planning Organization: MTC: Metropolitan Transportation Commission: NEC: Northeast Corridor: NEPA: National Environmental Policy Act: NOFA: Notice of Funding Availability: O&M: operations and maintenance: OMB: Office of Management and Budget: PRG: peer review group: PRIIA: Passenger Rail Investment and Improvement Act of 2008: SCC: Standard Cost Categories: TIGER: Transportation Investments Generating Economic Recovery: UIC: International Union of Railways: WBS: work breakdown structure: YOE: year of expenditure: [End of section] United States Government Accountability Office: Washington, DC 20548: March 28, 2013: Congressional Requesters: California's high-speed rail project is poised to be the first rail line in the United States designed to operate at speeds up to 220 miles per hour.[Footnote 1] At an estimated cost of $68.4 billion (in year-of-expenditure dollars), it is expected to be one of most expensive transportation projects undertaken in the United States. [Footnote 2] The Federal Railroad Administration (FRA) has obligated nearly $3.5 billion in grants to begin construction later this year. To complete the project, the state of California expects to rely on significant public-sector funding, in addition to private funding, through the project's anticipated completion date in 2028. Over the past two decades, there has been interest in developing high- speed rail between San Francisco and Los Angeles. Rail proponents argue that California faces significant transportation congestion and that high-speed rail can help alleviate the need to expand highway and air infrastructure. Proponents also argue that the corridor has characteristics that make high-speed rail viable: it connects highly populated metropolitan centers along a corridor that is currently highly traveled by air, rail, and automobile passengers. Opponents of the project argue that it is too expensive, especially with tight federal and state budgets, and has not sufficiently demonstrated that the project will be financially viable. As we reported in 2009, completing a high-speed rail project requires significant and sustained political, public, and financial support given the high costs and multiple stakeholders involved.[Footnote 3] In recent years, legislation has been enacted at the state and federal level that has allowed California's project to move forward by appropriating funding for high-speed rail. In 2008, California's voters approved a $9.95- billion bond measure, and in 2012, the state legislature appropriated over $4 billion from this bond to the project and connecting transit upgrades. At the federal level, the Passenger Rail Investment and Improvement Act of 2008 (PRIIA) authorized development of high-speed intercity passenger-rail corridors and the American Recovery and Reinvestment Act of 2009 (Recovery Act) appropriated $8 billion to fund development of these corridors and intercity passenger-rail projects.[Footnote 4] In June 2009, the FRA established the High Speed Intercity Passenger Rail (HSIPR) program that provides discretionary grants for high-speed or intercity passenger rail projects. In December 2009, the Fiscal Year 2010 Department of Transportation (DOT) appropriations act appropriated $2.5 billion for the HSIPR program ($400 million of which was later rescinded).[Footnote 5] The California High-Speed Rail Authority (Authority)--a state agency formed in 1996--has been tasked with implementing the project and has published four successive business plans detailing the project's cost estimates, financing plan, ridership projections, and anticipated economic impacts. Since 2009, the scope of the project has changed along with the project's estimated costs. In addition, questions have been raised about how the project will be funded beyond the current $11.5 billion in state and federal funding commitments, as well as the project's ability to attract sufficient ridership and bring economic benefits to the cities and communities along the corridor. This report assesses: (1) the reliability of the Authority's estimates of the project's costs, (2) the reasonableness of the Authority's passenger rail ridership and revenue forecasts, (3) the risks attendant with the Authority's funding plan for the project, and (4) the comprehensiveness with which the Authority identified potential economic impacts of the project. As such, we are assessing the quality of the available information used by policymakers. We are not evaluating the merits of the project itself, which should be considered in light of whether this project best meets the transportation needs of the estimated 51 million Californians in 2050. We obtained information from numerous sources to address our objectives, including the models underlying the various estimates and interviews with Authority officials and contractors that prepared them, the peer review group members that reviewed them, outside critics and supporters, high-speed rail experts and FRA officials. We based our analysis on estimates underlying the Authority's April 2012 revised business plan. To assess the reliability of the project's cost estimates, we compared the Authority's cost estimating approach to GAO's best practices found in the 2009 GAO Cost Estimating and Assessment Guide (Cost Guide). [Footnote 6] We used the Cost Guide to conduct an in-depth evaluation of the methods and assumptions used by the Authority to develop cost estimates of Phase 1 of the project; our goal was to determine whether best practices were employed that would help ensure that the cost estimates are well-documented, comprehensive, accurate, and credible. The Cost Guide was developed for cost estimators to assist them in preparing reliable estimates of capital program costs. While GAO's Cost Guide provides criteria to evaluate the Authority's methods for developing its cost estimates, we cannot use it to determine whether or not the Authority's cost estimate is precise. Nor can we use the Cost Guide to evaluate the benefits of this project in comparison to other transportation improvements. Such an evaluation would require a cost-benefit and other analyses, which we did not conduct as part of this work. To assess the reasonableness of the ridership and revenue estimates, we analyzed the extent to which the Authority's ridership model methodology adhered to FRA guidance and generally accepted practices and reviewed peer review reports assessing the model's methodology. We also interviewed Authority officials, their contractor, and peer review panel members. To assess the Authority's financing plan, we reviewed the plan, conducted interviews with Authority and other state and federal officials, and reviewed literature and other information on financing for high-speed rail projects in other countries as well as large transportation projects in the United States. To assess the comprehensiveness with which the Authority identified potential economic impacts of the project, we compared the impacts identified in the April 2012 revised business plan and related economic analyses prepared by the Authority with criteria for such assessments contained in DOT's program-funding notices, applicable legislation, and best practices found in academic literature and federal guidelines. For purposes of this report, potential economic impacts include those impacts on users of a transportation system such as travel time savings and those impacts to non-users of a system such as highway congestion or noise levels. (For more detailed information on our objectives, scope, and methodology, see appendix I.) We conducted this performance audit from February 2012 to March 2013 in accordance with generally accepted government auditing standards. These standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: In 2008, California voters approved Proposition 1A,[Footnote 7] which authorized $9.95 billion in state bond funding for construction of the California high-speed rail system and connection improvements to existing passenger rail systems. Proposition 1A established several requirements for this high-speed rail system, such as that the rail system must be capable of sustained operating speeds of no less than 200 miles per hour, and once built, must operate without a public subsidy. The planned 520-mile high-speed rail system will operate between San Francisco and Los Angeles at speeds up to 220 miles per hour (see figure 1). The Authority is the state entity charged with planning, designing, and constructing the California high-speed rail system. The Authority has a nine-member policy board appointed by the California legislature and Governor, and a staff of approximately 55 state employees who oversee, among other things, contracts for environmental review, preliminary engineering design, preliminary right-of-way acquisition tasks,[Footnote 8] contractor oversight and other activities.[Footnote 9] Figure 1: Map of Planned California High-Speed Rail System and Key Construction Phases, 2013: [Refer to PDF for image: illustrated map] First construction: North of Fresno to just north of Bakersfield: 130 miles; Estimated completion date: 2018. Initial operating segment (IOS): Merced to San Fernando Valley: 300 miles; Estimated completion date: 2021. Bay to Basin: San Jose to San Fernando Valley: 410 miles; Estimated completion date: 2026. Phase 1 blended: * San Francisco to Los Angeles: 520 miles; * Integrated with Caltrain system in San Jose; * Dedicated high-speed rail tracks to Los Angeles Union Station; * Multimodal connection with Metrolink at Palmdale; Estimated completion date: 2028. Sources: California High Speed Rail Authority and GAO. [End of figure] Construction of the California high-speed rail project is expected to occur in phases beginning with a 130-mile section from just north of Fresno, California, to just north of Bakersfield, California. Construction will begin in the Central Valley and proceed to other portions of the corridor as funding is available. The Central Valley is the furthest advanced in terms of design and engineering work, as well as environmental reviews. For example, FRA approved a preferred route alignment for the Merced to Fresno, California, portion of the corridor in September 2012. According to FRA, the federally funded portion of the project in the Central Valley has more complete cost estimates than subsequent segments given that preliminary engineering and environmental reviews are complete or nearly complete. Other project segments, however, are in different stages of development and have different levels of information from which to develop cost estimates. In July 2012, the California legislature appropriated $4.7 billion of the $9.95 billion in state bond funds, including $2.6 billion for construction of the high-speed rail project and $1.1 billion for upgrades in the San Francisco peninsula and in the Los Angeles basin (commonly referred to as the "bookends").[Footnote 10] The process of acquiring property for the right-of-way and construction has begun. Requests for proposals to select construction contractors and right-of-way acquisition were issued in March and September 2012, respectively. In addition, in January 2013, the Authority awarded a project and construction management contract for the initial phase of California's high-speed rail project. According to the Authority, a design-build contract for the first construction (covering approximately 30 miles) is expected to be awarded in June 2013 with construction planned to commence in summer 2013. (See figure 2). Figure 2: Time Line of California High-Speed Rail Project: [Refer to PDF for image: time line] 1996: The Authority was established by the state legislature to design and build the high-speed rail project. November 2008: California voters approved Proposition 1A, a bond referendum to provide $9.95 billion to fund initial design and construction of the project. 2009: FRA established the HSIPR program, and Congress appropriated $8 billion for high-speed rail projects as part of Recovery Act. 2010: Congress appropriated $2.5 billion to the HSIPR program. November 2011: The Authority issued a draft business plan that envisions a full-build high-speed rail project at a total year-of-expenditure cost of $98.5 billion. April 2012: Authority issued a revised business plan that envisions a blended high- speed rail system and project cost is reduced to $68.4 billion in year- of-expenditure dollars. July 2012: The California state legislature appropriated $4.7 billion to the project from Proposition 1A high-speed rail bonds. September 2017: In accordance with governing grant agreements, all Recovery Act funds must be expended. 2022: IOS expected to be operational[A] from Merced to the San Fernando Valley. 2026: Bay to Basin segment expected to be operational from San Jose to the San Fernando Valley. 2029: Phase 1 blended system expected to be operational from San Francisco to Los Angeles. Source: GAO. [A] The estimated construction date refers to the date for which construction of a segment is expected to be completed while the expected date of operations refers to the date for which a segment is expected to be operational. The estimated construction dates for the IOS and Phase 1 are 2021 and 2028, respectively, which differs from the estimated operational dates which are 2022 and 2029, respectively. [End of figure] The federal government has committed funding to the project. The FRA awarded the state approximately $3.3 billion in capital construction funds and $231 million for environmental review and preliminary engineering work under the HSIPR program for a total of approximately $3.5 billion.[Footnote 11] The California high-speed rail project is the largest recipient of HSIPR funds, with about 35 percent of program funds obligated. Most of the HSIPR money awarded to the project was appropriated by the Recovery Act and, in accordance with governing grant agreements, must be expended by September 30, 2017.[Footnote 12] In addition, approximately $945 million in fiscal year 2010 funding was awarded to the project by FRA and is to remain available until expended. While the funds remain available until expended under FRA's fiscal year 2010 appropriation, the governing grant agreements specify the schedule for expenditure of funds. Even though some funding has been committed, additional funding will be needed to complete the project. For example, according to the Authority's finance plan, over $38 billion in federal funds and over $4 billion from Proposition 1A proceeds will be needed to complete Phase 1 of the project. In addition, the Authority is also planning to obtain another $13.1 billion in private-sector capital to help defray the cost of construction after the initial operating segment is completed in 2028. As the federal agency responsible for awarding and overseeing grants to HSIPR applicants, the FRA established guidance outlining requirements and procedures and developed an oversight program to ensure that the project's goals, objectives, performance requirements, budgets, and other related program criteria are being met. Thus far, FRA's guidance to HSIPR grant applicants has been limited with respect to developing cost estimates and ridership and revenue forecasts. The Department of Transportation's Office of Inspector General (DOT OIG) noted that the lack of clear, detailed guidance allows for analyses of widely varying quality, making it difficult to accurately assess whether projects will be viable or require substantial financial support and has recommended FRA improve its guidance.[Footnote 13] In addition, we have previously reported that a clear definition of the federal role, goals, and objectives in conjunction with a robust grant oversight program, is critical to FRA making sound federal investments in high-speed rail projects.[Footnote 14] The Authority is required to prepare and periodically submit to the state legislature a business plan, which must identify, among other things, ridership estimates, operating and maintenance costs, and the source of project funding.[Footnote 15] The Authority's next business plan is expected to be released in 2014. Several groups have been established to review and comment on the estimates presented in the Authority's business plans. For example, an independent peer review group (PRG) was established in accordance with California law to evaluate the Authority's funding plans and report its judgment as to the feasibility and the reasonableness of the plans, appropriateness of assumptions, analyses and estimates, and any observations or evaluations it deems necessary.[Footnote 16] In addition, the Authority convened a Ridership and Revenue Peer Review Panel (Panel) to review the Authority's ridership and revenue-forecasting process and outcomes and conduct an in-depth review of the models used to estimate ridership and revenue and the forecasts derived from them. The California state auditor is required to periodically audit the Authority's use of bond proceeds.[Footnote 17] In response to the initial high estimated cost of building the San Francisco to Los Angeles route--about $98 billion--and other criticisms of the Authority's November 2011 draft business plan, the project underwent substantial revision for the April 2012 revised business plan. Most significantly, the Authority scaled back its plans to build dedicated high-speed rail lines over the project's entire length. Instead, the April 2012 revised business plan adopted a "blended" system in which high-speed rail service would be provided over a mix of dedicated high-speed lines and existing and upgraded local rail infrastructure (entirely at the bookends of the system on the San Francisco peninsula and in the Los Angeles basin). The estimated cost in the April 2012 revised business plan is $68.4 billion. The ridership and revenue forecasts in the April 2012 revised business plan also changed from the November 2011 plan. For example, in the November 2011 draft business plan, the Authority provided low and high estimates of ridership in 2030 of 14.4 million and 21.3 million passengers. In the April 2012 revised business plan, these estimates increased by 12 and 26 percent, respectively, to 16.1 million and 26.8 million. Revenues similarly increased between the two plans from a low and high estimate of $1.05 billion and $1.56 billion in the November 2011 plan[Footnote 18] to $1.06 billion and $1.81 billion in April 2012.[Footnote 19] The ridership and revenue estimates increased because among other things, a "one-seat" service from San Francisco to Los Angeles would begin sooner under the blended approach than the original solely dedicated lines approach. However, by 2040 ridership forecasts under the blended approach are less than the original full build approach. The range between the high and low estimates also increased between the November and April plans reflecting a greater degree of uncertainty in the estimates. We have previously reported that forecasting ridership and revenue is a complex and iterative process and that early stage estimates should be based on the best available data and what is initially known about the proposed project. [Footnote 20] As additional information becomes available, the Authority's model used to produce the forecasts is intended to be updated. Development of the high-speed rail system has been controversial with many strongly held beliefs among the numerous supporters and opponents of the project. Supporters have cited the need for high-speed rail to address growing congestion concerns, particularly in the metropolitan areas, and to address future transportation demands. Supporters have noted that California's expected population growth--which is expected to increase from 38 million in 2012 to an estimated 51 million Californians in 2050--and economic growth will continue to place more demands on California's transportation infrastructure requiring that significant new capacity be added to its transportation network. Further, supporters argue that the cost of expanding the current network of highways and airports to meet current and future transportation needs is cost prohibitive and would be detrimental to air quality and that high-speed rail will increase economic development in local communities to be served by high-speed rail and generate new jobs. In addition, supporters also note that several critical airport and highway expansions are infeasible due to land constraints, particularly at key airports and urban segments of highways. Opponents of the plan have argued, among other things, that the cost of the high-speed rail system is too great and that future funding for the system is too uncertain given the current fiscal environment. Opponents have also raised concerns about the credibility of the ridership and revenue forecasts presented in the Authority's business plans and specifically, the system's ability to attract the ridership levels needed to avoid public operating subsidies. Local communities and property owners in California's Central Valley have also raised concerns about the project and its potential impact to the agriculture sector in the region. As we reported in our December 2012 testimony, the Authority will face several challenges with acquiring rights-of-way in a timely manner, including potential construction delays as well as additional project costs.[Footnote 21] Timely right- of-way acquisition will be critical since some properties are in priority construction zones. Property to be acquired will include homes, businesses, and farmland. Not having the needed right-of-way could cause delays and add to project costs. There are a total of approximately 1,100 parcels to be acquired for the first construction segment; all of which are in California's Central Valley. According to Authority officials, although the Authority may face challenges in acquiring right-of-way, they have built contingencies for time and cost into their acquisition plan. Authority Substantially or Partially Met GAO's Best Practices for Producing Reliable Cost Estimates, but Can Make Improvements: Reliable Cost Estimates Are Critical to Successful Project Planning, Funding, and Oversight, but Little FRA Guidance Is Available: The Authority estimates that Phase 1 of the high-speed rail project in California will cost $68.4 billion to construct and hundreds of millions of dollars to operate and maintain annually.[Footnote 22] Since the project's financing plan, as articulated in the April 2012 revised business plan, will depend on an additional estimated nearly $38.7 billion in federal funds, it is vital that the Authority, FRA, and Congress be able to rely on these cost estimates for project planning, funding, and oversight. In addition, because the value of potential private investment depends on the cost of operating the system, it is vital that the Authority and the private sector be able to rely on the operating cost estimate. Given that our past work on high-speed rail projects around the world has shown that projects' cost estimates tend to be underestimated, ensuring the reliability of the estimates is critical to the success of this project.[Footnote 23] FRA provided limited guidance to grant applicants, including the Authority, about preparing cost estimates. FRA grant applicants were required to submit detailed capital cost estimates and high-level operating cost estimates, but FRA did not provide guidance on how applicants should produce these cost estimates to help ensure reliability. Moreover, the limited guidance that was provided did not reflect best practices included in our Cost Guide. FRA officials acknowledged that they specified the categories and types of costs to be estimated, not how applicants should prepare these cost estimates. FRA officials told us that they did not provide prescriptive guidance to grantees in preparing cost estimates because of the Recovery Act requirement to begin funding activities quickly following the enactment of the act in February 2009. In addition, FRA noted that the first two rounds of the HSIPR program were open to a wide range of project types and the level of detail necessary for an individual station is different from the level of detail necessary for a large, long-term corridor program like California. According to FRA officials, the Authority's application complied with the HSIPR grant application requirements. FRA found the cost estimates to be reasonable based on their comparison of the Authority's cost estimates (on a unit cost basis) to other rail projects in the United States and abroad. The Authority and its contractor told us that, in the absence of specific guidance on preparing cost estimates, they relied on their professional experience supplemented by available cost- estimating guidance from the Federal Transit Administration (FTA) which they thought was the most applicable guidance available. For example, since FRA's guidance did not require the Authority to perform an independent cost estimate (this involves a comparison of the Authority's original cost estimates to those performed by an independent entity), the Authority turned to FTA guidance to provide direction on how and when to conduct an independent cost estimate. We evaluated the Authority's cost estimates against GAO's Cost Guide, which details best practices for generating high-quality cost estimates at all levels of government. While not required by FRA, the best practices identified in our Cost Guide help estimators develop reliable cost estimates, which have the four following characteristics: * An accurate cost estimate is unbiased, not overly conservative or overly optimistic, and based on an assessment of most likely costs. * A credible cost estimate discusses any limitations of the analysis from uncertainty or biases surrounding data or assumptions. * A comprehensive cost estimate ensures that costs are neither omitted nor double counted. * A well-documented cost estimate is thoroughly documented, including source data and significance, clearly detailed calculations and results, and explanations for choosing a particular method or reference. Ensuring that cost estimates reflect these four characteristics helps minimize the risk of cost overruns, missed deadlines, and unmet performance targets. The Cost Guide also provides criteria for evaluating cost estimates to determine whether they exhibit these characteristics. We have previously applied the Cost Guide in reviewing several transportation and infrastructure projects and we applied it in our review of the Authority's cost estimates.[Footnote 24] Authority Substantially or Partially Met Best Practices in our Cost Guide for Producing Reliable Cost Estimates: The Authority substantially met best practices in our Cost Guide for producing accurate cost estimates, but only partially met our best practices for producing comprehensive, well documented, and credible estimates. By not following all best practices, there is increased risk of such things as cost overruns, missed deadlines, and unmet performance targets. Our assessment of the Authority's $68.4 billion construction and operating cost estimates for the high-speed rail project is summarized in table 1. Our assessment is discussed in more detail in appendix 2. Table 1: Summary of GAO's Assessment of California High-Speed Rail Authority's Cost Estimates: Characteristics of a reliable cost estimate: Accurate; Description of best practices: * Is based on most likely costs and is adjusted for inflation; * Is based on appropriate cost estimating techniques and contains minimal errors; * Reflects the current status of the project and has been updated with actual costs; * Is based on a historical record of cost estimating and comparable projects; GAO assessment: Substantially met. Characteristics of a reliable cost estimate: Comprehensive; Description of best practices: * Includes all project lifecycle costs such as construction, operation and maintenance; * Reflects the technical description of the project and is at an appropriate level of detail; * Reflects the ground-rules and assumptions of the project; GAO assessment: Partially met. Characteristics of a reliable cost estimate: Well documented; Description of best practices: Is supported by documentation that shows: * the source data and how they were adjusted; * the calculations and estimating methodology; * a description of how the estimate was created; * consistency with the technical baseline of the project; * review and approval from management; GAO assessment: Partially met. Characteristics of a reliable cost estimate: Credible; Description of best practices: Includes: * sensitivity analysis (assessing the impact of cost drivers); * risk and uncertainty analysis (identifying and quantifying risk); * cross-checks on major cost elements; * an independent cost estimate; GAO assessment: Partially met. Source: GAO. Note: GAO assessed the extent to which the Authority's project cost estimates met the best practices derived from the Cost Guide. The individual assessments are then used to determine an overall assessment for each characteristic. Assessments are made on a five- point scale: Not Met, Minimally Met, Partially Met, Substantially Met, and Fully Met. See Appendix 1 for a description of how we conducted our assessment and Appendix 2 for more detail on the complete assessment. [End of table] Accurate: We found that the Authority substantially met best practices for developing accurate cost estimates. Consistent with best practices, the estimates reflect the new "blended" system, which will rely, in part, on existing rail infrastructure; they contained few, if any, mathematical errors; and they have been adjusted for inflation. Furthermore, the Authority's contractor used a construction industry database of project costs supplemented with actual bid-price data from other transportation infrastructure projects. While the Authority generally complied with best practices for producing accurate cost estimates, we could not determine whether the estimates were unbiased. This was the only best practice related to accuracy where the Authority fell short. To help ensure an unbiased estimate, the Cost Guide recommends conducting a systematic analysis of the potential risks to the project and their likelihood of occurring--called a risk and uncertainty analysis. A risk and uncertainty analysis is also a best practice for developing a credible cost estimate as discussed below. Comprehensive: We found that the Authority partially met best practices for producing comprehensive cost estimates. For example, the Authority met the best practice for including in the cost estimates the major components of the project's construction and operating costs. The construction cost estimate is based on detailed construction unit costs that are, in certain cases, more detailed than the cost categories required by FRA in its HSIPR grant application. However, the operating cost estimate was not as detailed as its construction cost estimate, as over half of the operating costs are captured in a single category called Train Operations and Maintenance. Authority officials told us that they developed their cost estimates consistent with FRA's guidance which emphasized greater detail on the construction cost estimate and less detail on the operating cost estimate.[Footnote 25] FRA officials confirmed that they emphasize construction cost estimates because HSIPR grants are required by federal law to only fund the capital costs of a project, not its operating costs. However, sufficiently comprehensive operating cost estimates are necessary to determine the potential profitability of California's project, a key consideration to attracting private-sector investment in the project that the Authority is counting on to help complete construction of the project. In addition, the Authority did not follow the best practice that calls for clearly describing certain assumptions underlying the construction and operating cost estimates. For example, Authority officials told us that the California project will rely on proven high-speed rail technology from systems in foreign countries, but it is not clear how the cost estimates were adjusted for applying the foreign technology in California and how these adjustments are reflected in the complete project cost estimates. California requirements on the technology may differ in terms of speed or safety requirements from that of other systems. Authority officials said that they produced a thorough description of the technical requirements for high-speed rail, but we were unable to see how this document was linked to the cost estimates, that is, a description of how these technical requirements will impact the project cost estimates. Without comprehensive cost estimates, it is not possible to independently assure that all cost-influencing factors and assumptions were considered. Well Documented: We found that the Authority partially met best practices for producing well-documented cost estimates. In many cases, the methodologies used to derive the construction cost estimate were well documented, but in other cases, were more limited. For example, while track infrastructure costs ($23.6 billion in 2011 dollars) were thoroughly documented, costs for other elements, such as building new stations and acquiring trains ($3.2 billion in 2011 dollars), were not supported with sufficient documentation to identify how these costs were developed and what costs were included or excluded. Authority officials told us that since station locations and train technology are not yet finalized, they used a higher-level cost estimate. Additionally, we were unable to trace the estimates back to their source data and recreate the estimates using the stated methodology. For example, we were unable to identify the basis for how the operating costs from analogous foreign high-speed rail projects were adjusted for use in California. Authority officials said that the operating cost estimate was used at a high level to determine whether or not the California system will operate with an operating surplus. Authority officials plan to refine the operating cost estimate as the project progresses. However, without more detailed documentation, the Authority's cost estimates are more difficult to support and it may be harder to make changes to the estimates as they are revised since the basis of the original estimate may not be documented. In addition, without more thorough documentation, FRA and other oversight officials cannot replicate and evaluate what the Authority did to prepare its estimates and potentially exposes the project to possible cost overruns because the basis for costs are not known. Estimates that lack documentation are not useful for updates or information sharing and can hinder understanding and proper use. Credible: We found that the Authority partially met best practices to help ensure the credibility of its cost estimates. Those practices include: * testing such estimates with a sensitivity analysis, such as assessing the effect of changes in key cost inputs; * obtaining an independent cost estimate conducted by an unaffiliated party to see how outside estimates compare to the original estimates; and: * conducting risk and uncertainty analysis. In regard to the construction cost estimates, the Authority performed a sensitivity analysis for the approximately first 30 miles of construction and obtained an independent cost estimate for the first 185 miles of construction in the Central Valley but neither covered the entire Los Angeles to San Francisco project. And, as noted under the accuracy discussion, the Authority did not conduct a risk and uncertainty analysis on the cost estimates for any construction segment. Authority officials told us that in the absence of relevant FRA guidance, they followed FTA guidance for these types of evaluations. They noted that FTA guidance recommends conducting sensitivity tests once route alignments are selected and that, thus far, only 30 miles of the project meet those criteria. Similarly, Authority officials told us that they interpreted FTA guidance to require an independent cost estimate for those segments that have passed the 15 percent design milestone, which includes the first two construction segments from Merced to Fresno and Fresno to Bakersfield. Because the Authority's cost estimates cover both construction of the full system--from San Francisco to Los Angeles--and include operating costs, sensitivity analyses and independent cost estimates are more beneficial when they cover the entire project to help ensure greater credibility. The methodology of these tests can be altered to reflect the level of design, so a segment that has met a certain level of design can still be evaluated for credibility. Finally, as noted above, the Authority did not perform a risk and uncertainty analysis, which would improve the estimates' credibility by identifying a range of potential costs and indicating the degree of confidence decision- makers can place on the cost estimates. For example, the Authority faces the potential challenge of acquiring rights-of-way in a timely manner. Authority officials told us there about 400 parcels in the first construction package, about 100 parcels of which are considered to be potentially at-risk for timely delivery for construction. However, without a risk and uncertainty analysis it is not possible to determine how the cost estimates might be affected by such things as delays in acquiring necessary rights-of-way or having to pay more for property to keep the project on schedule. Authority officials said that they did not conduct a risk and uncertainty analysis yet because FRA did not require one, and the FTA guidance to which they turned, recommends such an analysis after a final route alignment is selected. In addition, Authority officials told us that they added contingencies to the cost estimates to account for the risk of cost overruns. However, according to our Cost Guide, risk and uncertainty analysis--as with sensitivity analysis and independent cost estimates--should cover the entire cost estimate and can be performed at varying levels of detail commensurate to the level of design. And, while contingencies are designed to cover potential cost overruns, based on our review, the Authority's contingencies, which range from 10 to 25 percent for various cost elements, were not calculated based on the results of a risk and uncertainty analysis (since this was not performed) but rather were based on professional judgment. Without a risk and uncertainty analysis, we cannot be assured that the contingencies are accurately calculated, and more importantly, what level of confidence we can have in the cost estimates. For the operating cost estimate, the Authority conducted sensitivity tests under various ridership scenarios as recommended by the Panel; however, these tests were designed to measure the ability of the initial operating section to cover operating costs with ticket revenues and not to determine the potential risk factors that may affect the operating cost estimate itself. The Authority also did not compare its operating cost estimate to an independent cost estimate or conduct a risk and uncertainty analysis. The Authority told us that it views the sensitivity test already conducted as well as a forthcoming evaluation of operating costs by the International Union of Railways (UIC) as sufficient to meet these requirements. To make its operating cost estimate more comprehensive and better documented, the Authority has contracted with the UIC to evaluate the existing methodology and data and help refine the Authority's estimates.[Footnote 26] While the UIC's evaluation will likely provide an expert review of the Authority's operating cost estimates, it may not address some of the key practices that ensure credibility. For example, the UIC's evaluation is not expected to result in new, independently-produced cost estimates that can be compared to the Authority's original estimates. Authority Can Make Improvements to Its Cost Estimates as the Project Progresses: The quality of any cost estimate can be improved as more information becomes available. And, based in part on evaluations from the PRG, the Authority is taking some steps to improve the cost estimates that will be provided in the 2014 business plan. As noted above, the Authority has contracted with the UIC to evaluate and provide recommendations on the Authority's operating cost estimates. While the study will provide additional analysis from a reputable source, it may not address all best practices from the Cost Guide that would help ensure that the operating cost estimate is comprehensive, accurate and credible. Cost estimates should also be updated with actual costs so that they are always relevant and current. Continual updating of a cost estimate as a program matures not only results in a higher-quality estimate but also provides an opportunity to incorporate lessons learned. While the Authority was not able to incorporate actual costs because construction had not yet begun, it will have the opportunity once contracts are awarded and actual costs begin to incur for the initial construction in the Central Valley, which is expected to begin in 2013. The bids for the first 30-mile construction package have been submitted to the Authority and will provide a check on how well the Authority has estimated the costs for this work, as well as provide more information on its cost estimates for other segments of the project.[Footnote 27] Ridership and Revenue Forecasts Are Reasonable for Current Purposes, but Will Require Updates: The Authority's ridership and revenue forecasts to date are reasonable and the methods used to develop them followed generally accepted travel-demand-modeling practices. In addition, the Authority completed several updates to its ridership-and-revenue forecasting model after the release of the April 2012 revised business plan and also completed several sensitivity analyses to test the reasonableness of its model. However, the Authority will need to complete several additional updates to improve its model and the resulting forecasts for the 2014 business plan. Authority officials stated that they have plans in place to complete several critical updates, including completing a new travel preference survey and developing a second generation travel demand model, but will not be able to complete these improvements in time for the 2014 business plan. Forecasts Are Reasonable Based on Use of Accepted Travel-Demand- Modeling Practices, but Ridership and Revenue Model Will Require Additional Updates: Based on our review, we found that the Authority's methods and model used to produce its ridership and revenue forecasts adhere to generally accepted travel-demand-modeling practices. However, the Authority will need to complete several updates to improve these forecasts for the 2014 business plan. In its April 2012 revised business plan, the Authority forecasts between 16.1 million and 26.8 million passengers per year and annual revenues of $1.06 billion to $1.81 billion in 2030. These forecasts were derived from a statewide ridership and revenue model, developed under contract to the Metropolitan Transportation Commission (MTC).[Footnote 28] Developing travel demand and revenue forecasts is difficult in almost every circumstance, across every mode and for a variety of reasons. As we have previously reported, limited data and information, especially early in a project before specific service characteristics are known, make developing reliable ridership and revenue forecasts difficult. [Footnote 29] To the extent early stage data and information are available they need to be updated to reflect changes in the economy, project scope, and consumer preferences. In addition, risks of inaccurate forecasts are a recurring challenge for project sponsors. Research on ridership and revenue forecasts for rail infrastructure projects around the world have shown that ridership forecasts are often overestimated. Although forecasting is inherently risky, reliable ridership and revenue forecasts are critical to accurately estimate the financial viability of a high-speed rail project and determine what project modifications, if any, may be needed. Such forecasts enable policymakers and private entities to make informed decisions about the proposed project and to determine the associated risks when making investment decisions. In addition, ridership forecasts are critical because they serve as the basis for revenue forecasts. If the California high-speed rail project is unable to generate the necessary ridership and revenue to cover the system's operating costs, the project may not be able to operate without a subsidy--as required by Proposition 1A. Conversely, if forecasts are overly conservative, it could lead to the state capturing less value from private investment than warranted. As such it is critical that the Authority's process for developing these forecasts is reliable and provides some assurance that the resulting forecasts provide a reasonable estimate of future demand for the system. Unlike our cost estimating criteria discussed earlier, there is no industry standard or established criteria for developing or evaluating intercity passenger high-speed rail ridership forecasts. FRA has not established guidance on acceptable approaches to the development of reliable ridership and revenue forecasts and has established only minimal requirements and guidance related to information HSIPR grant applicants must provide regarding forecasts. We previously reported that developing guidelines, methods, and analytical tools to develop credible and reliable ridership forecasts is necessary to ensure equitable consideration of high-speed rail as a potential option to address demands on the nation's transportation system.[Footnote 30] We recommended that the Secretary of Transportation develop guidance and methods for ensuring the reliability of ridership and other forecasts used to determine the viability of high-speed rail projects and support the need for federal grant assistance.[Footnote 31] The DOT OIG has also recommended that FRA develop specific and detailed guidance for the preparation of HSIPR ridership and revenue forecasts.[Footnote 32] According to FRA officials, they are in the process of developing an oversight plan that will include criteria for evaluating ridership forecasts. FRA officials indicated that they intend to use the DOT OIG's HSIPR Best Practices: Ridership and Revenue Forecasting guide as a starting point for developing this guidance.[Footnote 33] For the purposes of our assessment, we identified generally accepted travel-demand-modeling practices for high-speed rail projects from a variety of sources and developed criteria based on these practices to assess the reasonableness of the approach used to create the ridership and revenue models for the California high-speed rail project. [Footnote 34] In developing our criteria, we relied primarily on a 2011 report, prepared for the DOT OIG's office by the firm Steer Davies Gleave, on best practices related to travel demand modeling. [Footnote 35] In addition, we also examined other literature on developing rail ridership and revenue forecasts to supplement information in the Steer Davies Gleave report. Specifically, we reviewed, among other sources, our prior GAO reports, Federal Highway Administration (FHWA) and FRA guidance, and academic literature. (See appendix I for a list of guidance and reports we reviewed.) From our review of these reports and other sources, we identified common approaches to developing ridership and revenue forecast models and elements affecting the validity of those models. We identified seven key steps of the ridership-and-revenue forecasting process and then compared the Authority's process for completing these tasks to generally accepted travel-demand-modeling practices. We found that the Authority followed generally accepted travel-demand-modeling practices for each of the seven key steps: (1) developing trip tables, (2) determining and applying service characteristics, (3) developing mode choice models, (4) estimating induced travel, (5) estimating expected fare revenue, (6) conducting sensitivity analysis, and (7) conducting validation testing. (For a detailed description of generally accepted travel-demand-modeling practices and the Authority's process for completing each of these steps, see appendix III). Developing Trip Tables: The Authority's process for developing trip tables and collecting and compiling data on current travel patterns followed generally accepted practices. A central task of the ridership forecasting process involves collecting and compiling data on current travel patterns along the proposed high-speed rail route into trip tables.[Footnote 36] We found that the Authority followed generally accepted practices for developing trip tables. For example, trips were distinguished by mode of travel (auto, air, rail), trip purpose (commute, business, recreation, and other), and trip length (over and under 100 miles) as is general practice. Various data sources were used to develop the base year trip tables, including, among other sources, 2000 Census Bureau data, survey data from a 2005 travel survey[Footnote 37] and a 2011 long-distance travel survey conducted by Harris Interactive, [Footnote 38] and existing regional models used by metropolitan planning organizations (MPOs).[Footnote 39] Overall, the data sources used for developing the trip tables were consistent with generally accepted standards. One potential limitation, however, is that the 2011 survey sample was not selected at random from among California residents but rather was limited to individuals who had opted to join an online survey panel. The American Association for Public Opinion Research (AAPOR)[Footnote 40] has recommended against the use of such panels, often called "opt-in" survey panels, when accurate population estimates are needed, due to concerns about data quality and the possibility that a panel may differ from the intended target population in unknown ways. Data were weighted to adjust for differences between the survey sample and the California population on four characteristics--geographic location, age, wealth, and employment status--but, the data may still not be representative of the California population in other characteristics related to travel behavior. Determining and Applying Service Characteristics: The Authority's process for determining and applying service characteristics followed generally accepted practices; however, as detailed service plans are finalized or scenarios are changed, the model will need to be updated to reflect revised service characteristics. Ridership forecasts require information on the service characteristics (such as travel time and fares) of competing modes of travel, such as automobile and air travel along the proposed route. We found that the Authority collected and considered relevant service characteristics and used appropriate data sources. These included information on service characteristics of the interregional transportation modes in the area including information on time, cost, and other service characteristics for each mode--auto, conventional rail, and air--based on the most currently available published or observed data. High-speed rail service characteristics were defined based on the initial service plans and fare structure because published or observed data do not exist. Some or all of the high-speed rail characteristics will likely change as service plans are finalized and engineering decisions are made, and those changes can significantly affect ridership and revenue forecasts. This was illustrated in a 2012 sensitivity analysis completed by the Authority in which service characteristics for the proposed high-speed rail system were adjusted to reflect reduced service on the San Francisco Peninsula, which is likely under the current "blended" approach whereby the high-speed rail system will share tracks with Caltrain. [Footnote 41] This adjustment reduced high-speed rail ridership and revenue forecasts by 11 percent and 13 percent, respectively, compared with forecasts in the April 2012 revised business plan. Updated representations of the base and forecast year level of service characteristics will be important for producing realistic ridership forecasts in the future. Developing a Mode Choice Model: While the Authority mostly followed generally accepted practices as well as used some more advanced techniques in its mode choice model development, peer and academic reviewers such as the California High- Speed Rail PRG, the Panel, and academic experts from the University of California Berkeley's Institute of Transportation Studies (ITS), [Footnote 42] have identified limitations with the survey data used to develop the mode choice model. "Mode choice models"--or travel behavior models--illustrate how different variables such as income, travel time, and travel costs affect travel choices, including decisions about which mode of transportation to use. The reliability of the mode choice model estimates depends heavily on the quality of data used, which in turn depends on methods used to collect the data. [Footnote 43] These models can be based on traveler behavior observed in actual travel situations or self reported via a survey (revealed preference data), or on hypothetical situations presented to travelers in a survey (stated preference), or both. There are advantages and disadvantages to using revealed preference and stated preference data. Revealed preference data provides information on travelers' actual choices made in a specific market. However, according to the Steer Davies Gleave report, when collected from travel surveys, biases may exist in the respondents' responses due to a desire of respondents to justify their chosen mode. In addition, since true high-speed rail does not exist in the U.S., it is not possible to use revealed preference alone to determine how American travelers would actually use high-speed rail. To address this problem, stated preference data have been used in high-speed rail studies to asses likely traveler responses to a new service. According to the Steer Davies Gleave report, while stated preference data can provide detailed information about a traveler's likely responses to different modes or services that do not currently exist, this type of data may also exhibit bias; specifically, survey respondents may respond favorably to a hypothetical new mode, when in reality it may be more difficult to change habitual behavior. According to the Steer Davies Gleave report, limitations with each of these types of data can be mitigated by taking various steps such as by combining both revealed preference and stated preference data. The primary source of data for the Authority's mode choice model was a revealed preference and stated preference survey, of air, rail, and auto trip passengers, conducted at airports, rail stations, and by telephone from August to November 2005. However, in its July 2011 and May 2012 reports to the Authority, the Panel reported that the Authority's main mode choice model was based solely on stated preference responses, and recommended in its May 2012 report that the Authority collect new survey data and use both revealed and stated preference data in developing a new mode choice model. Authority officials stated that they are currently developing a new revealed preference and stated preference survey, which they plan to begin administering in early 2013. We discuss this survey in further detail in the next section. In addition, academic experts from the University of California Berkeley's ITS[Footnote 44] have previously reviewed the ridership and revenue model and have also identified limitations in the Authority's method of applying a statistical model to the survey data. These data came from a choice-based sample, meaning that survey respondents were selected for having already chosen a mode of transport (air, rail, or auto). The Authority used a conventional method of applying a statistical model to such data, but a newer method has been identified in a recent research paper.[Footnote 45] According to some academic experts, the use of the latest method could be an improvement. The Panel also reviewed this issue in its May 2012 report and stated that while they do not see the non-use of this new method in the Version 1 model as an important defect, it is worth investigating this issue as the Authority continues to refine the travel demand model. Estimating Induced Travel: The Authority followed generally accepted practices when producing induced travel estimates. Induced travel refers to trips that occur as a result of the high-speed rail project and that might not otherwise have been made using existing modes. In general, these are new trips that are generated because a new travel mode exists. The Authority estimated induced travel for the California high-speed rail project to be on average 2 percent of total high-speed rail trips. According to the Steer Davies Gleave report, based on its review of forecasts of proposed U.S. high-speed rail systems, an upper limit on induced travel of approximately 10 percent of total high-speed rail trips is widely accepted for proposed U.S. high-speed rail systems. Steer Davies Gleave also reviewed actual induced travel for high-speed rail systems outside the United States and found that it ranged from 6 to 27 percent. The Authority forecasted induced travel to be on average 2 percent. This estimate appears to be conservative and reasonable when compared to proposed U.S. high-speed rail systems and actual induced travel for high-speed rail systems outside the United States. Estimating Expected Fare Revenue: The Authority followed generally accepted practices when estimating expected fare revenue for the California high-speed rail project. Expected fare revenue is a product of forecast ridership and average fares. High-speed rail fares are based on a boarding fare and a per mile fare for interregional trips. For example, travel from Los Angeles to San Francisco would mean an average, one-way, high-speed rail fare of $81 in 2010 dollars or 83 percent of average 2009 airfares from Los Angeles to San Francisco. Forecast ridership is impacted by high-speed rail fares and revenue is an output of the ridership and revenue model. According to the Authority, the annual number of riders needed to break-even when the Phase 1 blended system is opened in 2029 is 6.1 million or 23 percent of the high forecast. Authority officials stated that they did not produce a revenue optimization forecast--that is, a ridership forecast that would maximize revenue--to produce these high-speed-rail fare estimates and acknowledged that they will need to do so in the future when meeting with potential private high-speed rail operators, which will establish their own revenue-maximizing fares for the system. According to the Authority, the Authority will not operate the high-speed rail system and a private operator is expected to serve as a contract operator of the system. As such, the private operator will be expected to assume all revenue risks of the project (including setting fares). The Authority's process for calculating expected fare revenue adheres to generally accepted practices; however, as other factors and inputs change, total expected fare revenue will likely also change. For example, over time, it will be important for the Authority to monitor changes in airfares, gasoline prices, and other key assumptions and incorporate these changes, as necessary, into future revenue forecasts. Conducting Sensitivity Analysis: The Authority followed generally accepted practices when conducting sensitivity analysis of key model assumptions for the California high- speed rail project. A sensitivity analysis is typically conducted by varying key model assumptions such as socioeconomic data, type of trips taken, gasoline and auto fleet efficiency, and airfares, or parameter values to determine how the model behaves in response to changes to these assumptions. The Authority conducted several sensitivity analyses on its ridership and revenue model, most in 2011. For example, the Authority conducted sensitivity analyses that tested key factors, such as changes in fuel economy, air and auto travel time, air and auto travel costs, and high-speed rail travel time assumptions. In one analysis, the Authority tested the overall effect of a higher auto fuel economy and found that this change reduced ridership and revenue forecasts by 16 and 19 percent, respectively, from the Phase 1 high ridership and revenue forecasts presented in the November 2011 draft business plan. According to the Panel that reviewed these analyses, the results of the various sensitivity analyses that the Authority has conducted show that the model is appropriately sensitive across the range of variables tested. In addition, the Authority performed a sensitivity analysis of an extreme downside scenario to test the ridership and revenue implications of a series of downside events, such as increased average rail travel time from Merced to the San Fernando Valley (140 minutes instead of 126 minutes), decreased train frequency (3 trains per hour instead of 4 trains per hour during peak times), lower auto-operating costs, and lower air fares (10 percent below actual 2009 average air fares). Based on this analysis, the Authority determined that an extreme downside scenario would be expected to reduce ridership and revenue forecasts by 27 percent and 28 percent, respectively, below that shown for the low IOS forecasts in the April 2012 revised business plan. According to the Authority, they tested these events using the IOS phase because the financial viability will be the most fragile during the early stages of ridership. According to the Authority, these forecasts are still sufficient to cover the Authority's estimated operating costs and not require a public operating subsidy. Authority officials stated that they intend to conduct additional sensitivity analyses going forward. Conducting Validation Testing: The Authority mostly followed generally accepted practices to validate the ridership and revenue model. Model validation generally involves verifying that the model reflects observed traveler behavior including total travel, region-to-region travel flows, and observed market shares by mode. In the United States, without another high-speed rail system to use for comparison, model validation is a difficult task. Furthermore, validating the proposed California project with foreign high-speed rail systems is difficult because some of the travel market characteristics in other countries with high-speed rail, such as the cost of driving, may not be comparable. The Authority has taken some steps to validate the model through tests performed using data on Amtrak's premium Acela service in the Northeast Corridor (NEC) as input to the California high-speed rail model and compared the output with 2008 actual ridership and 2030 NEC forecasts.[Footnote 46] The results of the Authority's comparison of the California high-speed rail model with NEC input to actual 2008 ridership data showed that ridership in the California high-speed rail model with NEC-like conditions in 2008 is 79 percent of actual 2008 NEC ridership. Similarly, the Authority's comparison of the California high-speed rail system ridership forecasts from the model run with "NEC-like" service is about the same as projected 2030 ridership on the Acela service. Authority officials told us they believe the results from these tests demonstrate that the ridership and revenue model is reasonably sensitive to speed, frequency, and fares. While the NEC corridor has some comparable characteristics to the proposed California high-speed rail corridor--such that it is capable of reaching top speeds up to 150 miles per hour and covers a distance of over 400 miles--the proposed California high-speed rail corridor and the NEC differ in important ways. For example, population density, [Footnote 47] congestion, and travel behavior in the two corridors differ, and as such, forecast comparisons should be interpreted with caution. In developing the forecasts for the April 2012 revised business plan, the Authority also revised several model assumptions used in the initial ridership and revenue forecasts presented in the November 2011 draft business plan. Specifically, the Authority revised model assumptions to reflect changes in current and anticipated future conditions for airfares and airline service frequencies, decreases in gasoline price forecasts, and anticipated declines in the growth rates for population, number of households, and employment. Some of the initial assumptions were largely based on pre-2007 data and did not reflect potential effects of the 2007 to 2009 recession. (See table 2 for a summary of updates that have thus far been completed.) According to Authority officials, this was done to build in additional conservatism in the ridership forecasts. Table 2: Description of Updated California High-Speed Rail Travel- Demand-Model Assumptions: Model input assumptions: Airfares and airline service frequencies; Description of revised assumptions: * Authority updated the model to reflect changes in current and anticipated future airfares and airline service frequencies. Initially, the model reflected 2005 average fares and airline service frequencies--it was updated to reflect 2009 average fares and airline service frequencies; * The Authority engaged Aviation System Consulting, LLC (ASC), a California-based expert firm to address these issues. ASC analyzed the past decade of U.S. Department of Transportation data on airline service and fare levels, explained the economic factors affecting airline responses to changes in competition and capacity, and helped determine scenarios of potential airline competitive response to the introduction of high-speed rail service. Model input assumptions: Conventional rail service and fare assumptions; Description of revised assumptions: * Conventional rail service and fare assumptions for 2030 were updated to reflect 2011 current and forecasted conditions; * Authority updated the fare assumptions for all lines to on-line published fares in 2011. Model input assumptions: Revised automobile operating costs assumptions; Description of revised assumptions: * Gasoline price forecasts increased: The Authority adjusted its assumptions related to automobile operating costs to reflect the most recent forecasts from the U.S. Energy Information Administration, adjusted to reflect the historical difference between California prices and the national average; * Vehicle fuel economy forecasts: To align with Corporate Average Fuel Economy standards, forecasts were revised to show an average of 3 percent to 6 percent fuel economy growth; * 2030 auto operating forecasts: The Authority estimated a range for auto operating costs incorporating both fuel and nonfuel components. The Authority revised 2030 operating costs for the 2012 revised business plan. The Authority used $0.20 per mile in 2011 dollars for the low model scenario and $0.28 per mile for the high model scenario. Model input assumptions: Socioeconomic forecast updates; Description of revised assumptions: The 2030 population, household, and employment forecasts used for the original ridership and revenue model were developed in 2006-2007 from local agency socioeconomic projections. The Authority developed two alternate forecasts of population, households, and employment to account for decreased expectations regarding future socioeconomic growth resulting from the 2007-2009 recession: * High forecast: To develop the 2030 forecasts, the Authority purchased forecast data of county-level population, households and total employment for the state, for two years--2008 (pre-recession data) and 2011 (post-recession data). A combination of these data were used to account for pre-and post-recession impacts to factor 2030 pre- recession forecasts; * Low forecast: 2011 county-level forecasts of population, households, and employment by economic sector for the state were purchased and used to develop 2030 forecasts. Model input assumptions: Alternative trip frequency assumptions; Description of revised assumptions: * The Authority developed a long-distance travel survey to collect current long-distance travel data to help provide perspective for the 2012 revised business plan; * The Authority contracted with Harris Interactive for the data collection, and the survey was fielded in May 2011. Source: California High-Speed Rail Authority information. [End of table] Updating model assumptions can help mitigate of the risks of overestimating ridership and revenue forecasts--referred to as optimism bias. Biased ridership forecasts are a recurring problem with rail infrastructure projects and we have previously reported that forecasting ridership and revenue is a complex and uncertain process and ridership forecasts of high-speed rail projects are often overestimated.[Footnote 48] Other research on ridership and revenue forecasts for rail infrastructure projects have confirmed that actual ridership is likely to be lower than forecasted unless steps are taken to incorporate more conservative assumptions into the model. For example, a recent study examined a sample of 62 rail projects and found the ridership forecasts of 53 of them were overstated; actual ridership was, on average, 41 percent lower than forecasted.[Footnote 49] Updates to model inputs, such as fuel prices and other projections, are important for updating ridership forecasts for any project; in this instance, updates to model inputs resulted in more conservative ridership forecasts. Next Business Plan Will Not Reflect All Planned Improvements to Ridership Forecasts: The Authority has plans to complete future improvements to its ridership and revenue forecasts, including completing a new travel preference survey and developing a second generation travel demand model. However, the Authority will not be able to complete these critical improvements in time for the 2014 business plan. According to Authority officials, the 2014 plan is expected to include, among other updates, updated ridership and revenue forecasts. Two critical updates to the ridership forecasts that peer reviewers and academic experts have recommended are the development of a new 2013 revealed and stated preference travel survey and a second generation travel demand model-- which will make use of the new survey. Although the Authority has begun taking steps to complete both of these tasks, neither the survey nor the second generation model will be completed in time for the 2014 business plan. The Authority began work developing a new 2013 revealed and stated preference survey in late 2012. According to Authority officials, a survey sample and survey questionnaire design was initiated in December 2012. Data collection for the 2013 revealed preference and stated preference survey will not begin until 2013. Full data collection, cleaning, and preliminary analysis of results are expected to be completed by mid-April 2013. The new survey will include a larger sample of 4,500 respondents compared with the 3,172 respondents to the 2005 survey. In addition, the revealed preference portion of the data set will be designed and coded to facilitate estimation using revealed and stated preference responses simultaneously, which was not done in the first version of the model. According to the Panel, development of a new survey is critical as it will address several long-term issues that can only be overcome with the collection and analysis of new survey data. The Authority also has plans to develop a second-generation travel demand model; however, the second generation model will not be completed in time for the 2014 business plan. According to Authority officials, work on the second-generation model will not begin until the ridership and revenue analysis for the 2014 business plan is completed. According to the Authority, the second generation model will use data from the new 2013 revealed and stated preference survey to supplement data from the 2005 survey. In addition, the Authority plans to replace the 2011 Harris Interactive long-distance travel survey data with data from the 2012 California Household Travel Survey being conducted by the California Department of Transportation. According to Authority officials, both surveys will be needed for developing its second-generation travel demand model. The Panel, which has released five reports assessing the Authority's ridership and revenue forecasts, has reported that the Authority's ridership and forecasts to date are reasonable for planning purposes but has also stated that additional updates and enhancements, particularly the development of a new model, will be critical for future project decision making. For example, in its most recent October 2012 report, the Panel stated that a second-generation model will be required to meet the Authority's long-term goals of completing detailed planning studies and make key planning and operational decisions on issues such as specific rail alignments, station design requirements, and pricing strategies. While the Authority will not be able to complete a second-generation travel demand model in time for the 2014 business plan, it has begun work on developing an enhanced model that will be used to produce the ridership and revenue forecasts for the 2014 business plan. The enhanced model will retain the same structure of the original model, but some of the individual model components will be updated. For example, the main mode choice model will use both revealed preference and stated preference results from the 2005 travel survey. According to Authority officials, the enhanced travel demand model will be completed by May 31, 2013. Even if the Authority is not able to complete the major ridership and revenue forecast improvements in time for the 2014 business plan, ongoing disclosure of interim results from model improvements both before and after the business plan are published will be important to outside reviewers and the public. Peer reviewers and other groups that have examined the Authority's ridership and revenue forecasts have reported the need for greater transparency in the Authority's analyses. For example, in its January 2012 report, the California State Auditor reported that the Authority's November 2011 draft business plan lacked detail in its presentation of some of the revenue forecasts. Similarly, in its October 2012 report, the Panel advised the Authority to provide summaries on the Authority's website, describing, among other things, recent forecasts, key input assumptions used to develop the forecasts (e.g., fuel price trends, socioeconomic growth rates, and changes in household size and structure), and updated service characteristic information. According to Authority officials, documentation supporting analysis in its 2014 draft business plan will be available on the Authority's website when the plan is made public. In addition, Authority officials stated that the Authority has generally posted key technical documents as they are made public. The Authority also has developed a work plan for other travel demand model improvements. This includes, among other things, plans to complete additional validation testing of model results using data from the NEC. For the 2014 business plan, the Authority is planning to conduct further testing and sensitivity analysis of the ridership and revenue forecasts to examine the sensitivity of the forecasts to reduced frequencies of services; changes in alternative fare structures (for example, premium fares for intra-regional trips in the San Francisco Bay Area and Los Angeles Basin); changes to service plans (destinations and schedules); and other sensitivity analyses aimed at quantifying risks. In its October 2012 report, the Panel also recommended additional sensitivity analyses to be completed on the second generation travel demand model, including analyses examining the impact of pricing strategies on revenue, impact of local transit feeder systems on station choice, and impact of major changes in the roadway network on highway congestion and subsequent mode choice decisions. In addition, the Authority is planning to conduct Monte Carlo simulations to test numerous potential combinations of assumptions on the forecasts that will be part of the 2014 Business Plan during fiscal years 2013 through 2014, provided the foundational information to construct, test, and analyze the simulation and its results is sufficiently developed at that stage of the program. All of these will be important as the ridership and revenue forecasts continue to evolve with development of the high-speed rail project. The Authority's Funding Plan Faces Uncertainty: The project's funding, which relies on both public and private sources of financing, faces uncertainty about whether those funds can be obtained in a tight federal and state budget environment. The Secretary of Transportation and the Governor of California have committed to funding this project, but obtaining sustained congressional and public support for appropriating additional funds is one of the biggest challenges to completing this project. In the latter stages, the Authority will also rely on private-sector financing, but will require more reliable operating cost estimates and revenue forecasts to determine whether, and the extent to which, the system will be profitable, as well as the value of any private investment. The Authority's financing plan recognizes the uncertainty of the current funding environment so the Authority is building the project in phases and has identified an alternative funding source that is also uncertain. However, delays in obtaining funds as planned will likely lead to project delays and higher costs for construction. A summary of funding already committed to date can be found in table 3. Table 3: Funding Secured for Constructing the California High-Speed Rail Project, as of February 2013: State high-speed rail bonds; $8.2 billion[A]. Federal HSIPR grants; $3.3 billion[B]. Total secured funding; $11.5 billion. Source: GAO analysis of FRA grant information and the California High- Speed Rail Authority's April 2012 Revised Business Plan. [A] The Authority expects approximately $8.2 billion in proceeds from the $9.95 billion authorized in Proposition 1A high-speed rail bonds to be available for construction of high-speed rail. The remainder is for connectivity projects and engineering and environmental work. [B] Approximately $3.3 billion of $3.5 billion in obligated HSIPR grants is available for construction of high-speed rail project. The remainder is for engineering and environmental work. [End of table] Additional Public Sector Funding Is Needed to Complete the Project, but Securing It Will Be Challenging: The Authority is relying primarily on public-sector funding to complete construction of Phase 1 of the project with $55 billion, or 81 percent of the total construction cost, expected to come from state and federal sources. Heavy reliance on public-sector funding is not unusual for a project of this size. For example, France built its high- speed train system primarily with public-sector funding. In the United States, federal-aid highway system continues to be funded by the federal government through the gas tax and, more recently, with transfers from the general fund.[Footnote 50] The Authority expects to obtain public sector funds over the life of the project as individual segments are ready for construction. This type of "phased" funding is typical for major transportation infrastructure projects. Table 4 provides a summary of the Authority's funding plan for Phase 1 the high-speed rail project. Table 4: The Authority's Funding Plan for Constructing Phase 1 of the California High-Speed Rail Project, as of April 2012: Funding source: Federal; First construction: $3.3 billion; Initial operating segment: $20.3 billion; Bay-to-Basin: $8.4 billion; Phase 1 blended: $10.0 billion; Total: $42.0 billion; (61%). Funding source: State high-speed rail bond; First construction: $2.7 billion; Initial operating segment: $4.4 billion; Bay-to-Basin: 0.0; Phase 1 blended: $1.1 billion; Total: $8.2 billion; (12%). Funding source: Locally generated; First construction: 0.0; Initial operating segment: $0.7 billion; Bay-to-Basin: $1.2 billion; Phase 1 blended: $3.1 billion; Total: $5.0 billion; (7%). Funding source: Subtotal public; First construction: $6.0 billion; Initial operating segment: $25.4 billion; Bay-to-Basin: $9.6 billion; Phase 1 blended: $14.2 billion; Total: $55.2 billion; (81%). Funding source: Private investment; First construction: 0.0; Initial operating segment: 0.0; Bay-to-Basin: $10.1 billion; Phase 1 blended: $3.0 billion; Total: $13.1 billion; (19%). Funding source: Operating cash flow; First construction: 0.0; Initial operating segment: 0.0; Bay-to-Basin: $0.2 billion; Phase 1 blended: 0.0; Total: $0.2 billion; (0). Funding source: Subtotal private investment and operating cash flow; First construction: 0.0; Initial operating segment: 0.0; Bay-to-Basin: $10.3 billion; Phase 1 blended: $3.0 billion; Total: $13.3 billion; (19%). Funding source: Total; First construction: $6.0 billion; Initial operating segment: $25.4 billion; Bay-to-Basin: $19.9 billion; Phase 1 blended: $17.2 billion; Total: $68.5 billion (100%). Source: GAO analysis of California High-Speed Rail Authority's April 2012 revised business plan. [End of table] Federal Funding: The Authority's April 2012 revised business plan relies on approximately $42 billion in federal funding for the project's construction, which includes the $3.3 billion that has already been obligated. The remaining $38.7 billion in federal funds have not been identified in federal budgets or appropriations but would amount to an average of more than $2.5 billion annually over the life of the project's construction. This exceeds the average annual funding made available under DOT's New Starts transit-funding grant program since 2008 (about $1.6 billion per year). Moreover, it exceeds the federal government's average annual appropriations to Amtrak since 2008 (about $1.5 billion per year). Largely as a result of the funding challenge, the Authority is taking a phased approach--planning to build segments as funding is available. Thus, according to the Authority's 2012 revised business plan, no additional funding will be needed until 2015 when it hopes to begin construction beyond the first construction segment. Based on our past work on high-speed rail, successful projects require significant and sustained financial commitments from the public sector before private investors will participate, and the Authority's plan reflects this funding model.[Footnote 51] For example, in Japan, private investment is contingent on substantial government investment. Other federally-supported transportation programs--like those for highway and certain transit infrastructure--rely on a dedicated revenue source for their funding and allow for multi-year funding agreements for eligible projects. In contrast, the HSIPR program has not been funded with a dedicated revenue stream, but from the general fund, a process that means that the program has to compete for appropriations with other discretionary programs.[Footnote 52] In addition, the HSIPR program has provided one-time grants and, as currently structured has not awarded multi-year agreements for grantees. Our 2013 High-Risk Series report identified the use of general funds for high-speed rail projects as a challenge to project completion.[Footnote 53] Given that the HSIPR grant program has not received funding since 2010 and that future funding proposals will likely be met with continued concern about the general level of federal spending, the largest block of expected funding for the California project is uncertain. State and Local Funding: The Authority is also relying on a total of about $8.2 billion in state high-speed rail bond proceeds (which includes the $3.7 billion that has been appropriated to date) and another $5 billion in locally- generated funds for the project's construction.[Footnote 54] The proceeds from the high-speed rail bonds are dedicated, in that they can only be used for this project and do not have to compete with other budgetary priorities. However, the remaining $4.5 billion will have to be appropriated to the project. The $5 billion in local funds-- most of which are expected at the end of the project's construction timeline--have not been committed by local entities yet. Authority officials told us that these local funds could include revenues derived from property development in and around high-speed rail stations or improved service on existing transit corridors. For example, planned improvements on the Caltrain corridor may result in increased ridership; if so, a portion of increased revenues could be earmarked for high-speed rail construction. Cap-and-Trade Funding: The Authority acknowledges the risk in relying on a significant amount of federal funding (and to a lesser extent, local funding) for the project. To help mitigate this risk, the California Department of Finance has identified California's newly implemented cap-and-trade program as a potential source of revenue in the event other public funding is not available.[Footnote 55] However, there are a number of challenges with using this program as a potential source of revenue for the high-speed rail project. First, while the state of California anticipates this program to generate tens of billions of dollars in revenues, amounts are not certain since the auctions have only recently begun. The first auction of greenhouse gas emissions allowances was held in November 2012 and resulted in revenues at the low-end of the state's initial estimates.[Footnote 56] The state anticipates revenue will increase significantly by fiscal year 2015 when more bidders are required to participate. Second, in order for any program revenue to benefit the high-speed rail project, appropriations must be enacted to provide funding for the project. This means that high-speed rail will have to compete with other state funding priorities, and the Authority cannot be assured what portion of cap-and-trade revenues, if any, will be dedicated for high-speed rail. Finally, concerns have been raised about whether revenues from the cap-and-trade program can be used for the high-speed rail project. An analysis by California's Legislative Analyst's Office (LAO) indicated that, among other things, the project might not comply with the legislative goal of the cap-and-trade program of reducing greenhouse gas emissions by 2020, which raises concerns about the potential use of cap-and-trade revenues for the high-speed rail project. The LAO suggested that the California Legislature obtain legal advice to address any potential legal issues. However, according to the California Air Resources Board's 2008 plan to implement the cap- and-trade program, high-speed rail was identified as an eligible project to meet the state's greenhouse gas reduction goals. Securing Private-Sector Investment Will Depend on Reliable Operating Cost and Ridership Forecasts: In addition to the challenges of obtaining public-sector funding, the Authority may face challenges in attracting private-sector funding if its' operating cost estimate and ridership forecasts prove to be optimistic. The Authority expects that once the initial operating segment is operational in 2023, it will generate a profit that would be attractive to private investors. The Authority is planning to raise approximately $13.1 billion by selling an operating concession to a private firm or consortium of firms.[Footnote 57] The Authority plans to use the proceeds of this sale to help complete construction of the system. Our past work on high-speed rail systems has shown that private sector investment is easier to attract only after the public sector has made a substantial capital investment in the system.[Footnote 58] The Authority's plan is consistent with this funding approach; however, to successfully attract private investment in this project, the Authority will have to meet two significant milestones: * complete construction of the IOS (this will require at least an additional $25 billion of investment from public sources), and: * demonstrate that the IOS can operate at a profit.[Footnote 59] Public-private partnerships for intercity passenger rail, such as what the Authority is planning for, have been proposed but not implemented in the United States.[Footnote 60] However, in other transportation sectors, public-sector infrastructure owners have used public-private partnerships to incorporate private-sector operating expertise and encourage private investment. For example, the state of Indiana raised $3.8 billion in 2006 by arranging a private operating concession for its existing Indiana Toll Road. In addition, according to DOT, the city of Denver, Colorado, raised nearly $100 million from private sources to help finance a $2 billion expansion to its transit and commuter rail project. While the private-sector can provide needed funding and management expertise to a transportation project, this approach is not risk-free. As we have previously reported for highway projects, public-private partnership can also present trade-offs, such as the risk that the private operator will demand more revenue from users (e.g., tolls) than initially expected.[Footnote 61] Other nations' experiences with high-speed rail indicate that under certain circumstances the private sector can operate these systems and that they could potentially be profitable on an operating basis. For example, Japan sold certain high-speed rail lines to private operators and does not provide operating subsidies to these firms. And, in 2010, Britain sold an operating concession for its High-Speed 1 line to a consortium of private investors for approximately Ł2 billion (approximately $3.2 billion).[Footnote 62] Private firms have also expressed interest in operating high-speed rail projects in the United States. For example, according to FRA, several private consortia were preparing to submit bids on a HSIPR-funded project between Tampa and Orlando, Florida. However, public-sector support was withdrawn when the governor canceled the project which precluded private-sector investment. And, in Texas, a privately-financed high-speed rail project failed when the investors encountered financial difficulties. Authority officials told us that they have met with a number of private firms and high-speed rail operators that have expressed interest in California's project but have not entered into any agreements since the project has not yet been built. Attracting private investment may not only require up-front public investment but may also require the use of revenue guarantees, or public guarantees of a minimal level of income to the project regardless of ridership levels. Such guarantees will reduce the risk for private operators, and therefore their cost of raising capital, but according to the Authority, a revenue guarantee is considered to be a type of operating subsidy that is barred by the legislation that authorized the state high-speed rail bonds (Proposition 1A bonds). Private-sector investment in the California high-speed rail project, if any, will ultimately be determined by the profitability of the system--that is, the extent to which operating revenues exceed operating costs. The Authority currently estimates an operating profit in the first and all subsequent years of operation.[Footnote 63] However, this estimate is only as reliable as the underlying operating cost and revenue forecasts. As discussed earlier, the Authority's current ridership and revenue forecasts are reasonable for planning purposes, however, further refinements will be required as the project continues to evolve. The Authority's current operating cost estimates will also need to be improved in the future. Accordingly, both cost and ridership forecasts will change before the initial operating segment is completed in 2022, making the future value of potential private funding uncertain at this time. The Authority Comprehensively Identified Potential Economic Impacts, but It Is Too Early to Determine Specific Impacts: Potential Economic Impacts Were Comprehensively Identified, but Specific Impacts Will Depend on Project Decisions, Economic Conditions, and Other Factors: The Authority comprehensively identified key economic impacts that could result from the high-speed rail project, including user and non- user impacts, as required by FRA and other federal requirements. FRA guidance, as contained in the program Notices of Funding Availability (NOFA), requires HSIPR applicants to identify the potential benefits and costs of proposed projects with a focus on a public return on investment.[Footnote 64] A public return on investment includes a project's potential to deliver transportation, economic recovery, and other benefits. To assist in project evaluation, FRA encouraged HSIPR applicants to provide an economic analysis that quantified the monetary value of user benefits and, if available, public benefits. [Footnote 65] However, according to FRA, program NOFAs did not explicitly require either a formal benefit-cost analysis (BCA) or preparation of an economic impact analysis (EIA).[Footnote 66] FRA officials said their review of economic impacts was based on a reasonableness test--that is, were economic impacts identified and were the assumptions behind the impacts reasonable. The officials said that FRA did not have the time or resources to conduct an in-depth analysis and that a reasonableness test provided increased assurance as to potential economic impacts of project proposals. Projects awarded funding must also comply with the National Environmental Policy Act (NEPA) and its implementing regulations. NEPA requires that government agencies undertaking a major federal action (such as providing grant funding) with significant effects on the environment prepare an analysis of the environmental impacts of the proposed action, including a discussion of alternatives to the proposed action.[Footnote 67] Under FRA's guidelines for considering environmental impacts, among the impacts to be considered in a NEPA environmental assessment are such things as land use and potential economic effects on existing business districts and metropolitan areas. The program NOFAs required HSIPR applicants seeking funds to develop new high-speed rail corridors and intercity passenger rail services to complete a NEPA review and the June 2009 program NOFA required HSIPR applicants to present information that provided a business and investment justification that contained project cost and benefit estimates. To analyze the extent to which economic impacts were comprehensively identified by the Authority, the key components of economic impacts on users and non-users were identified from a variety of sources, including NEPA, NOFAs for the HSIPR and Transportation Investment Generating Economic Recovery (TIGER) programs as well as a report on best practices for assessments of public benefits prepared for the DOT OIG for the DOT OIG's March 2012 report on FRA's HSIPR program. [Footnote 68] The best practices report identified user impacts such as travel costs and travel quality and non-user impacts such as highway and airway congestion and noise or air quality levels. Non- user impacts may also include economic impacts that are difficult to quantify, such as the effects improved transportation connectivity may have on allowing firms to access larger labor or product markets or increasing the labor supply because people can more easily access jobs. The latter are included in what is termed "wider economic impacts." FRA's requirement for a public return on investment from HSIPR projects included aspects of both user and non-user impacts. The best practices report noted the importance of both ridership and revenue forecasts and cost estimates in determining public benefits. In particular, it stated that public benefits assessment depends heavily on ridership and revenue forecasts and the implications these have on project impacts on travelers and the general population. Similarly, operating, maintenance, and capital cost estimates were also identified as important elements of public benefits assessments. The Authority's April 2012 revised business plan identified the primary user and non-user economic impacts of the California high- speed rail project (see table 5). For example, the plan identified potential user impacts such as travel time reliability for high-speed rail users and non-user impacts such as the effects on highway congestion and economic development around stations. In addition to the business plan, the Authority prepared a BCA that provided a more detailed analysis of both user and non-user impacts included in the revised business plan. The Authority also prepared an EIA that focused on those other economic impacts of the system that do not fall into the BCA framework. According to the Authority, the EIA presented longer-term impacts on California's economy from building the high- speed rail system. Table 5: Primary Economic Impacts Associated with High-Speed Rail Identified by the Authority in the April 2012 Revised Business Plan, BCA, or EIA: User impacts: Primary economic impacts associated with high-speed rail: Generalized travel costs to users. Includes impacts on the following: * In-vehicle travel time; * Waiting, access, egress time; * Out-of-pocket cost; Economic impacts identified by the Authority: * In-vehicle travel time; * Access and egress times; * Vehicle operating costs (fuel and non-fuel such as operating and maintenance costs); Where economic impact discussed: Revised Business Plan; BCA. Primary economic impacts associated with high-speed rail: Travel quality. Includes crowding, comfort, travel time variance and reliability of service; Economic impacts identified by the Authority: * Comfort of high-speed rail; * Reliability of travel times; Where economic impact discussed: Revised Business Plan; BCA. Non-user impacts: Primary economic impacts associated with high-speed rail: Congestion. Includes reduced congestion on the following: * Highway; * Air; * Rail; Economic impacts identified by the Authority: * Congestion impacts on highway vehicle miles traveled; * Congestion impacts on aviation (flight and passenger delays); * Analysis excluded potential impacts in urban rail corridors near Los Angeles and the San Francisco Bay Area; Where economic impact discussed: Revised Business Plan; BCA. Primary economic impacts associated with high-speed rail: Environment. Includes impacts on the following: * Noise; * Air quality; * Carbon emissions; * Landscape/townscape; Economic impacts identified by the Authority: * Automobile and truck emissions impacts; * Automobile noise pollution; * Aviation emissions impacts; * BCA analysis specifically excluded impacts on land use and land values; Where economic impact discussed: BCA. Primary economic impacts associated with high-speed rail: Travel safety; Economic impacts identified by the Authority: * Automobile accident cost impacts; Where economic impact discussed: BCA. Primary economic impacts associated with high-speed rail: Financial impacts, including impacts on such things as: * Operator revenues; * Federal, state, and local tax revenues; Economic impacts identified by the Authority: * Potential impacts on state gross domestic product from investing in high-speed rail; * Potential impacts on local tax revenue; Where economic impact discussed: Revised Business Plan; EIA. Source: GAO analysis of Steer Davies Gleave and Authority documents. [End of table] Although the Authority comprehensively identified the primary potential economic impacts, it is too early to determine specific economic impacts since these will depend on a number of factors. These include the following: * Future project decisions. The California high-speed rail project is in its early stages of development, and a number of project decisions have yet to be made, including final alignment of train routes, some station locations, and the type and frequency of service. Decisions such as these can be expected to have a bearing on potential economic impacts. For example, route alignments and station locations can affect economic development. While acknowledging that the extent to which high-speed rail would change California's economic landscape was not fully understood, the EIA suggested that based on studies in other countries, the main economic impacts from high-speed rail in California will likely occur in areas within 2 hours of major economic centers, such as the San Francisco Bay Area and Los Angeles. However, the EIA also concluded that the greatest volume of redevelopment attributable to high-speed rail will likely occur in major metropolitan areas and that the Central Valley could see moderate clustering of development around stations. Although economic development around stations offers the potential for economic impacts, achieving such development may be subject to a number of factors, and certain impacts may not be easy to identify. In July 2012, we reported on the potential for economic development associated with bus rapid transit projects.[Footnote 69] We found that in the five case study locations we examined, although the bus rapid-transit project was having some positive effect on economic development, individuals associated with these projects were unsure about how much economic activity could be attributed to the presence of bus rapid transit compared with other factors or circumstances. In addition, the project sponsors and experts we spoke with told us that transit-supportive policies and development incentives can play a crucial role in helping to attract and spur economic development associated with bus rapid transit. In September 2009 we reported the characteristics of transit- oriented development around stations can increase nearby land and housing values, but we also found that determining transit-oriented effects on the availability of affordable housing in these developments are complicated by lack of direct research and data. [Footnote 70] * Economic conditions over the life of the project. As we reported in March 2012, the closer the economy is to full employment, the smaller net effect a project will have on total economic activity.[Footnote 71] The speed with which the nation and California recover from the 2007 to 2009 recession, which cannot be known in advance, will affect the net employment from any new infrastructure projects. If the economy achieves full employment, such projects would affect the composition of employment but not its level or rate of growth. [Footnote 72] The high-speed rail project will be constructed and operated over a period of many decades and likely over many economic cycles. The Authority's April 2012 BCA used a 67-year period (from 2013 to 2080) to estimate potential economic benefits and costs of the project, and, for purposes of analyzing potential operating and maintenance costs, the April 2012 revised business plan assumed a 38- year operating period (2022 to 2060).[Footnote 73] Over such an extended period, economic conditions can be expected to change, as will potential economic impacts. The Authority's April 2012 EIA recognizes that the project's economic impacts will be affected by California's economy and unemployment rates. According to the Authority's EIA, as of February 2012 California's 10.9 percent unemployment rate was the nation's third highest.[Footnote 74] It goes on to estimate that the high-speed rail project has the potential to create about 1 million direct and indirect job-years through construction of the Phase 1 blended system, based on the assumption that 20,000 job-years would be created for each $1 billion in capital investment.[Footnote 75] The accuracy of this estimate will depend on the economic conditions and unemployment rates at the time the jobs are created. The EIA acknowledged this uncertainty when it stated that multipliers used to estimate indirect and induced jobs are snapshots in time of an economy and represent only current or recent economic relationships and technologies. They do not capture structural changes in the economy, new technologies, or changes in wages that occurred since the multiplier data were produced or that might occur in the future. * Uncertainty about some impacts. Uncertainty may particularly affect non-user impacts, some of which are difficult to measure or quantify. The Authority's April 2012 EIA discusses the potential for wider economic impacts from high-speed rail such as the benefits of bringing California's economic activities and markets closer together by reducing travel times. The EIA states, by improving transportation connectivity and reducing congestion, the high-speed rail system could make California's economy more efficient, productive, and competitive by such things as bringing businesses closer to labor and other markets and providing workers with greater access to jobs. However, the best practices report prepared for the DOT OIG found that additional data on things like the relationship between economic density and productivity, labor supply elasticity, and price-cost margins were needed to assess wider economic impacts of high speed rail projects.[Footnote 76] The study went on to note that no research was currently available regarding wider economic impacts of U.S. high- speed rail projects since such projects do not yet exist. The Authority's April 2012 EIA recognized this difficulty and states that the extent to which the high-speed rail project will affect the economic landscape of California is not well understood, though transportation infrastructure investments have historically created fundamental shifts in the spatial relationship between places. These limitations do not imply deficiencies in Authority or DOT performance but rather the inherent analytical complexities of large infrastructure investments. * Uncertainty regarding local or regional impacts. The April 2012 EIA discusses the potential for economic impacts along the high-speed rail line, including direct and indirect employment opportunities, increased efficiencies and productivity from bringing labor and other markets closer together, and transit-oriented development around stations. The specific impacts to regions or localities will depend on a number of factors, including project-related factors and factors associated with local policies and decisions. Among the project- related factors are both the rate of project spending over time as well as where project funds are spent. The high-speed rail project is expected to be constructed over a long period of time and in phases when funding becomes available. The rate of spending and its timing will influence when and to what extent regions and localities may experience economic impacts associated with the project. Which specific regions or localities may experience economic impacts will be influenced by such project decisions as route alignments and station locations. Local policies and decisions will also affect regional and local economic impacts. Studies conducted for the Authority by the University of California at Berkeley suggested there are opportunities for economic development from the high-speed rail system in a variety of locations, including Fresno and Bakersfield in California's Central Valley. However, the studies cautioned that the extent of economic development will depend on cities establishing a framework of planning and development policies that encourage development. Some cities have begun taking actions to promote economic development related to the high-speed rail system. For example, in June 2012 Fresno issued a solicitation for consultants to prepare a master plan for its planned downtown rail station that would enable the city to maximize local economic benefits from the high-speed rail system. Other cities have not yet acted. The Authority's April 2012 EIA found that of the 13 potential stations on the Phase I blended corridor, 7 (53.8 percent) did not have station-specific development plans. We also found some limitations in the specific economic analyses prepared by the Authority. In particular, the April 2012 BCA has shortcomings that could limit its usefulness to decision makers. * Identification of negative impacts of the project and their effects on the BCA analysis. The BCA lacked detail regarding the handling of negative impacts associated with the high-speed rail project. DOT guidance on benefit-cost analysis suggests that negative (or adverse) impacts, such as non-user (or highway) delays associated with rail construction, be included in BCA analyses to facilitate consistent project comparisons.[Footnote 77] The guidance recommends that negative impacts, such as highway delays associated with rail construction, be shown as a negative benefit and not included in project investment costs to better facilitate comparisons between projects. The April 2012 BCA recognized that during the period of project construction there would be roadway delays in urban areas that would offset some travel time savings. However, the BCA excluded such impacts from the analysis since the impacts were expected to be (1) localized, (2) minimal since the high-speed rail project minimizes urban grade crossings, and (3) negligible in proportion to overall travel time savings once the project is complete. Aside from roadway delays, the Authority acknowledged there were additional categories of economic impacts that may be negative. For example, the April 2012 revised business plan discusses how the high- speed rail system could limit access to parts of farmland in the Central Valley potentially reducing the output of affected farmlands. The BCA contained little discussion of such impacts and states that the BCA did not incorporate or monetize land use and land value impacts the high-speed rail project may cause (positive or negative). The Authority said negative impacts were assumed to be part of the mitigation measures that would be conducted as part of the environmental review process and right-of-way acquisition and that the costs of these measures were included in the cost side of the benefit- cost calculation. According to the Authority, including negative impacts as a negative benefit would lead to double counting them as both a negative benefit as well as a cost in the benefit-cost calculation. We agree that negative impacts of the high-speed rail project should not be double counted, but the BCA should include discussion of potential negative impacts and how they are treated in the analysis. Such information would better inform decision makers about the existence of negative impacts and their potential effect on project benefits or costs. * Identification of the risks and uncertainties associated with the BCA analysis. The BCA did not discuss the potential risks and uncertainties associated with either the benefits or costs used in the analysis. Forecasts are inherently uncertain, including those for ridership and economic projections. Recognition and analysis of risks is an important part of project evaluation. As we reported in February 2011, Executive Order 12893 and Office of Management and Budget (OMB) Circulars Nos. A-94 and A-4 indicate that benefit and cost information shall be used in agency decision making and that the level of uncertainty in estimates of benefits and costs shall be disclosed. [Footnote 78] In particular, Executive Order 12893 requires that uncertainties about the amount and timing of important benefits and costs associated with an infrastructure investment be recognized and addressed through appropriate quantitative and qualitative assessments. Similarly, DOT's TIGER guidance, which the Authority used to prepare its BCA, requires applicants to assess the reliability of any forecasts used to generate benefit estimates but does not specifically require a discussion of risks and uncertainties in a BCA. We reported in February 2011 that the majority of the applications to the TIGER and HSIPR programs we reviewed did not provide information related to uncertainties in projections, data limitations, or the assumptions underlying their models.[Footnote 79] Even when such information was provided, we found that it was not always comprehensive. We recommended that DOT require, among other things, grant applicants to clearly communicate the level of uncertainty in estimates of project benefits and costs. The Authority did not conduct a risk analysis beyond examining the potential effects of high and low cost scenarios. While the Authority may have used credible sources for variables in its analysis (e.g., fuel prices), this does not eliminate all forecasting risks, and those risks should be identified. In addition, as we have noted, decisions on route alignments and other aspects of the project have yet to be made; decisions that could add to project risk. High-Speed Rail's Impact on Meeting State Travel Demand Has Not Been Fully Assessed: Although the Authority comprehensively identified the project's potential economic impacts, additional analysis is needed at the state level of how high-speed rail will affect other transportation modes and their ability to meet future travel demand. This includes the potential cost of additional improvements that may be needed or conversely planned projects that may not be needed. An important aspect of non-user impacts is a project's potential effect on other transportation modes, including highways, aviation, and local transit systems. This is an important issue since the Authority has estimated that, as a result of population growth and other factors, overall interregional trips in California will increase from about 500 million in 2000 to about 900 million in 2030. In addition, under the blended approach adopted by the Authority, success of the high-speed rail system will depend in part on local transit improvements. As part of the planning process, the Authority considered high-speed rail's impact on the capacity of other transport modes. For example, in April 2012 the Authority issued an analysis of the highway and airport improvements that would be needed to provide an equivalent capacity to the high-speed rail system envisioned in the April 2012 revised business plan.[Footnote 80] The analysis found the total cost (in 2011 dollars) of equivalent capacity investment in highways and airports would range between $123 billion and $138 billion to build up to 4,600 highway lane-miles, 115 airport gates, and 4 airport runways. [Footnote 81] However, the analysis did not focus on potential additional highway or other transportation improvements that may be required even with construction of the high-speed rail system. Rather, the analysis identified the potential highway and airport improvements that would be required to provide an equivalent capacity to that of the high-speed rail, not an assessment of additional improvements required to meet future intercity travel demand. Identifying such improvements was not the task of the Authority since its task is to develop a high-speed rail system. Rather, this task would fall to the state as part of its overall planning responsibilities under federal transportation-planning requirements. Constructing a high-speed rail system is not expected to meet all of California's future intercity travel demand. Among other things, development of the phase I blended approach will affect the need for additional improvements to local transportation systems to support high-speed rail. For example, officials with the Orange County Transportation Authority told us they have bus projects and a street car project in the planning phases that will be linked to both local commuter rail and the high-speed rail system. Similarly, officials from the Southern California Association of Governments told us the cost of integrating high-speed rail with local transit was still being developed and that the cost of an initial list of prioritized projects to facilitate this integration exceeded $3 billion. Of this $3 billion in projects, about $1 billion in projects were categorized as high priority. The officials said that although they have got agreement from the Authority to help fund some of the high priority projects and some funding is expected to come from Proposition 1A, a funding source for the list of potential projects had not been determined. Even though high-speed rail is not expected to meet all of California's future intercity travel demand, statewide transportation planning has not yet fully assessed the impact of the high-speed rail system in meeting this demand. In November 2011, the California Transportation Commission (CTC) issued a Statewide Transportation System Needs Assessment[Footnote 82] that identified the preservation, management, and expansion projects required over the 2011 to 2020 period. The assessment identified a total cost for the projects of about $540 billion and a nearly $300 billion funding gap in meeting the project needs identified. According to CTC officials, the needs assessment included information about high-speed rail that was readily available from Authority documents. However, high-speed rail was not assessed in terms of project needs, costs, or the funding gap because, according to CTC officials, Regional Transportation Plans that formed the basis of the assessment did not include high-speed rail in identifying project needs and costs. Similarly, California Department of Transportation officials told us current highway transportation planning has not looked at this issue, and the department's long range transportation plans have not included consideration of high-speed rail. According to the officials, the department did not see an immediate need to do an assessment since the high-speed rail system is not expected to be operational for another 10 years or more. The officials agreed that the high-speed rail system will affect highways, and its impact will need to be considered in future transportation plans. Conclusion: As currently conceived, California's high-speed rail project is expected to be among the most expensive infrastructure projects that has been undertaken in this country. Therefore, concerns about potential cost escalations and optimistic ridership forecasts, as well as the potential burden this could place on public budgets are well placed. Cost and revenue estimates for large projects are, by their nature, imprecise; these estimates endeavor to predict many years into the future within the confines of what is known today. According to our past work reviewing high-speed rail projects in other countries, cost and ridership estimates tend to be overly optimistic. However, experts agree that taking steps to anticipate project risks and improve the credibility of such estimates will lower the risk of cost overruns and missed revenue forecasts. Improving the reliability of cost and revenue forecasts is critical to providing project sponsors, FRA, the Congress, and ultimately, the public with greater confidence that this project can be viable. This confidence is of particular importance as the Authority will seek significant and sustained funding from federal, state, and private sources. We found that the Authority did not fully employ best practices for producing reliable cost estimates as expressed in GAO's Cost Guide, which are recommended practices but not required. The cost estimates can be improved as the project progresses from design to construction, and ultimately, to operation. The Authority was not required to follow the Cost Guide; instead, it was required to follow FRA's guidance which we found to be limited. That guidance identified the cost categories that applicants should include in its cost analyses, but did not specify how cost estimates should be generated. The Authority told us that it looked to FTA's cost-estimating guidance to help inform the Authority's cost-estimating methodology. The Authority can be commended for supplementing its analyses using FTA's guidance, but this does not necessarily ensure a fully reliable cost estimate. Our past work, as well as that of the DOT OIG, has shown that FRA has yet to develop sufficient program guidance for project evaluation and oversight under its additional grant-making responsibilities under the HSIPR program. Developing guidance for HSIPR applicants and grantees that incorporates best practices from the Cost Guide would allow cost estimators to improve the reliability of cost estimates for expensive projects like high-speed rail. Such guidance would help ensure project costs reflect the four characteristics required for developing reliable cost estimates and minimize the risk of cost overruns, missed deadlines, and unmet performance targets. The Authority is in the process of updating its ridership and revenue model in response to recommendations provided by experts and peer review groups, such as the Ridership and Revenue Peer Review Panel. We believe that these steps will have the potential to improve the Authority's ridership forecasts, and we encourage continued refinement, as more information becomes available and continued review by peer review groups. Improved forecasts will be particularly important as the Authority seeks to secure private investment in the project. The potential project revenues--which are primarily dependent on ridership--will help determine how much the Authority may be able to obtain from private sources. Similar to the cost estimates, FRA has developed minimal guidance for applicants to develop reliable ridership and revenue forecasts. We, along with the DOT OIG, have previously made recommendations that FRA improve this guidance to ensure reliability of ridership and revenue forecasts that are used to determine the viability of high-speed rail projects. According to FRA officials, the agency is currently in the process of implementing these recommendations. Since we have already made recommendations to FRA on this issue in our prior work and FRA is taking actions on these recommendations, we are not at this time making additional recommendations related to improving FRA's ridership and revenue forecasting guidance. Recommendation: We recommend that the Secretary of Transportation direct the Administrator of FRA to improve its guidance for high-speed rail project sponsors to better ensure that cost estimates that are submitted by applicants seeking federal funding are accurate, comprehensive, well-documented, and credible according to the best practices detailed in GAO's Cost Guide. Agency Comments and Our Evaluation: We provided a draft of this report to DOT and the Authority for review and comment. DOT neither agreed nor disagreed with our recommendation. In an e-mailed response, DOT said it was pleased that the Authority met many of the criteria in the Cost Guide for producing accurate, comprehensive, well-documented, and credible cost estimates; that ridership and revenue forecasts were reasonable; and that the Authority did a comprehensive job in identifying potential economic impacts. However, DOT noted (1) that the currently funded project has sound cost estimates while future, currently unfunded phases will continue to be refined as the project progresses and data improves, (2) that FRA's cost-estimating guidance was the best available at the time, and (3) that GAO's Cost Guide focuses on federally managed acquisitions. DOT's response noted that the project is a multi-decade effort consisting of many segments and phases, each in a different stage of development. DOT's response also noted that the Cost Guide was issued in March 2009, one month after passage of the Recovery Act and a few months before deadlines for the HSIPR guidance, and, therefore, it was not feasible for FRA to incorporate Cost Guide best practices into guidance. Finally, DOT said applying the Cost Guide principles to future FRA capital cost guidance, while feasible, would require analysis and adaptation to accommodate unique aspects of long- term grantee-managed transportation projects. DOT noted that the Cost Guide is focused primarily on federally managed acquisitions and programs, not infrastructure projects that non-federal parties will develop and build. DOT also provided technical comments that we incorporated as appropriate. We recognize DOT' concerns; however, our charge was to assess the reliability of the cost estimates and not just whether they complied with FRA's guidance, which we found to be less than best practices. While the Cost Guide was released in 2009, it is a culmination of cost- estimating best practices that have previously been published and have been available to federal agencies for many years. Therefore, these practices could have been considered when preparing HSIPR program guidance. Finally, the best practices contained in the guide are applicable to developing cost estimates for a wide variety of programs and projects, whether federally managed or not. The Authority provided a letter summarizing its comments about the report (see appendix IV). In general, the Authority believes that the report highlights its efforts to produce cost estimates that reflect the scope of the project, that methods and models used to develop ridership and revenue forecasts adhered to applicable best practices, and that comprehensively identifying potential economic impacts demonstrates a strong economic case for the project. However, the Authority noted that different components of its program (such as implementation phases and construction packages) are at different stages of development and that it would not be practicable to apply the full complement of tools in the Cost Guide at the program level at this time. This is because the environmental review is under way on a number of sections and alignments and other choices are still to be made. The Authority also stated it plans to improve its cost estimates and ridership forecasts. For example, the Authority stated that the updated ridership model being developed for the 2014 business plan will incorporate many of the changes we suggested and that the Authority will improve the quantification of project risks. We commend the Authority for planning to improve its cost estimates and forecasts. Regarding not applying the Cost Guide at the program level, we note that a program does not need to be in an advanced stage of planning in order to complete a sensitivity or cost risk and uncertainty analysis. In fact, such analyses are most valuable when performed early in a program's life cycle. Single point estimates are more uncertain at the beginning of a program because less is known about its detailed requirements and the opportunity for change is greater. For example, undefined or unknown technical information, uncertain economic conditions, and political issues are often encountered during a program's acquisition. For management to make good decisions, the program estimate must reflect the degree of uncertainty, so that a level of confidence can be given about the estimate. Therefore, it is important to conduct a risk and uncertainty analysis at all stages of a project so cost estimates reflect the risk and uncertainty that exist. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution of this report until 30 days from the report date. At that time, we will send copies of this report to the appropriate congressional committees, the Secretary of Transportation, the Administrator of FRA, and the Director of OMB. The report will also be available at no charge on the GAO website at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report, please contact me at (202) 512-2834 or flemings@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V. Signed by: Susan A. Fleming: Director: Physical Infrastructure Team: List of Addressees: The Honorable Bill Shuster: Chairman: The Honorable Nick Rahall: Ranking Member: Committee on Transportation and Infrastructure: House of Representatives: The Honorable Jeff Denham: Chairman: The Honorable Corrine Brown: Ranking Member: Subcommittee on Railroads, Pipelines and Hazardous Materials: Committee on Transportation and Infrastructure: House of Representatives: The Honorable Darrell Issa: Chairman: House Committee on Oversight and Government Reform: House of Representatives: The Honorable Karen Bass: House of Representatives: The Honorable John Campbell: House of Representatives: The Honorable Jim Costa: House of Representatives: The Honorable Mike Honda: House of Representatives: The Honorable Duncan Hunter: House of Representatives: The Honorable Barbara Lee: House of Representatives: The Honorable Zoe Lofgren: House of Representatives: The Honorable Kevin McCarthy: House of Representatives: The Honorable Tom McClintock: House of Representatives: The Honorable Howard P. "Buck" McKeon: House of Representatives: The Honorable John Mica: House of Representatives: The Honorable Gary Miller: House of Representatives: The Honorable Devin Nunes: House of Representatives: The Honorable Adam Schiff: House of Representatives: [End of section] Appendix I: Objectives, Scope, and Methodology: This report assesses (1) the reliability of the California High-Speed Rail Authority's (Authority) estimates of the project's costs; (2) the reasonableness of the Authority's passenger rail ridership and revenue forecasts; (3) the risks to the Authority's plan to fund the project; and (4) the comprehensiveness with which the Authority identified potential economic impacts of the project. Our analysis focused on the Authority's cost estimates, ridership and revenue forecasts, and economic estimates presented in the April 2012 revised business plan. To address these objectives, we reviewed numerous documents, including Federal Railroad Administration (FRA) guidance, Department of Transportation Office of Inspector General (DOT OIG) reports, prior GAO reports, and pertinent legislation. In addition, we obtained documents from and conducted interviews with Authority officials to obtain information about the Authority's process for developing its various cost estimates, ridership and revenue forecasts, and economic impact estimates. Specifically, we reviewed the November 2011 draft business plan and April 2012 revised business plans and documentation used to develop the analysis presented in those plans. We conducted interviews with the Authority's contractors--Parsons Brinckerhoff and Cambridge Systematics--to obtain additional information about the Authority's processes for developing these estimates and to clarify information in their written documentation. In addition, we also conducted interviews with officials from various federal and state agencies, peer review groups, academic experts, advocacy groups, and transit and local government groups to obtain information on, among other things, their role with the California high-speed rail project and their views on the Authority's cost estimates, financing plans, ridership and revenue forecasts, and potential economic impacts. (See table 6 for a list of organizations and individuals we interviewed for this study.) Table 6: Organizations and Individuals Interviewed: Federal agencies: Department of Transportation, Federal Aviation Administration; Department of Transportation, Federal Highway Administration; Department of Transportation, Federal Railroad Administration; Department of Transportation, Federal Transit Administration; Department of Transportation, Office of Inspector General. State agencies: California Air Resources Board. California Bureau of State Audits; California High-Speed Rail Authority; California Legislative Analyst's Office; California State Treasurer; California Transportation Commission; Caltrans. Peer review groups: California High-Speed Rail Peer Review Group; Ridership and Revenue Peer Review Panel. Advocacy groups and experts: Californians for High-Speed Rail; Citizens Advocating Responsible Rail Design; Samer Madanat, Mark Hansen, and David Brownstone, University of California Berkeley Institute of Transportation Studies; Arpad Horvath and Mikhail Chester, University of California Berkeley Civil and Environmental Engineering; Wendell Cox; Julien Dehornoy; Bent Flyvbjerg; Joseph Vranich. Transit and local government groups: Amtrak; Bay Area Rapid Transit; Caltrain; Los Angeles County Metropolitan Transportation Authority; Metropolitan Transportation Commission; Orange County Transportation Authority; Peninsula Corridor Joint Powers Board; San Francisco County Transportation Authority; Southern California Association of Governments; Southern California Regional Railroad Authority; Transbay Transit Center Joint Powers Authority. Source: GAO. [End of table] To assess the reliability of the project cost estimates, we analyzed the Authority's cost estimating approach against GAO's best practices found in the 2009 GAO Cost Estimating and Assessment Guide (Cost Guide).[Footnote 83] GAO designed the Cost Guide to be used by federal agencies to assist them in developing reliable cost estimates and also as an evaluation tool for existing cost estimates. To develop the Cost Guide, GAO cost experts assessed measures applied by cost-estimating organizations throughout the federal government and industry and considered best-practices for the development of reliable cost- estimates. We analyzed the cost estimating practices used by the Authority against these best practices. For our reporting needs, we collapsed these best practices into four general categories representing practices that help ensure that a cost estimate is (1) accurate, (2) well documented, (3) comprehensive, and (4) credible. After a review of all source data, including but not limited to electronic cost models for both capital investment and operating and maintenance phases, all supporting documentation, personal interviews, and independent research, we assessed the extent to which the Authority met these best practices on a five-point scale: * Not Met--Authority provided no evidence that satisfies any of the criteria. * Minimally Met--Authority provided evidence that satisfies a small portion of the criterion. * Partially Met--Authority provided evidence that satisfies about half of the criterion. * Substantially Met--Authority provided evidence that satisfies a large portion of the criterion. * Fully Met--Authority provided complete evidence that satisfies the entire criterion. We determined the overall assessment rating by assigning each individual rating a number: Not Met = 1; Minimally Met = 2; Partially Met = 3; Substantially Met = 4; and Fully Met = 5. For the purposes of this assessment we have also included a Not Applicable (N/A) assessment category. Then, we took the average of the individual assessment ratings to determine the overall rating for each of the four characteristics. The resulting average becomes the Overall Assessment as follows: Not Met = 0 to 1.4; Minimally Met = 1.5 to 2.4; Partially Met = 2.5 to 3.4; Substantially Met = 3.5 to 4.4; and Fully Met = 4.5 to 5.0. To assess the reasonableness of the Authority's ridership and revenue forecasts, we analyzed the extent to which the Authority's methods for developing the ridership model and resulting ridership and revenue forecast adhered to federal guidance and generally accepted travel demand modeling practices for high-speed rail projects. Unlike with GAO's cost-estimating criteria discussed earlier, there is no single industry standard for developing or evaluating intercity passenger high-speed rail ridership forecasts. As such, for the purposes of our assessment, we reviewed a variety of sources that identify generally accepted travel demand modeling practices and developed criteria based on these practices to assess the reasonableness of the approach used to create the ridership and revenue models for the California high- speed rail project.[Footnote 84] In developing our criteria, we relied primarily on a 2011 report prepared for the DOT OIG's office by the firm Steer Davies Gleave, on best practices related to developing high- speed rail ridership and revenue forecasts.[Footnote 85] The report provides a description of current standard practices in high-speed rail ridership and revenue forecasting, key steps typically involved in completing these forecasts, and a description on the range of data and methods used in the forecasting process. The intent of this guidance is to provide information that will assist reviewers to understand and evaluate forecasting studies. In addition, we also examined other literature on developing rail ridership and revenue forecasts to corroborate information in the Steer Davies and Gleave report. Specifically, we reviewed, among other sources, forecasting guidance from the FRA, Federal Highway Administration (FHWA), prior GAO reports and other ridership and revenue guidance in academic research. (See table 7 for a list of sources used to develop criteria). Table 7: Guidance and Reports Used to Identify Generally Accepted Travel-Demand-Modeling Practices: Federal agency guidance: Source: Prepared by Steer Davies Gleave for the Department of Transportation, Office of Inspector General; Guidance or report: HSIPR Best Practices: Ridership and Revenue Forecasting (April 2011). Source: Prepared by Cambridge Systematics for the Federal Highway Administration; Guidance or report: Travel Model Validation and Reasonableness Checking Manual, Second edition (September 2010). Source: Department of Transportation, Federal Railroad Administration; Guidance or report: High-Speed Intercity Passenger Rail (HSIPR) Program; Notice of Funding Availability; issuance of interim program guidance,"74 Federal Register 119" (23 June 2009), pp. 29900-29929. Guidance or report: High-Speed Intercity Passenger Rail Program, Notice of Funding Availability of Individual Projects (NOFA); issuance of interim program guidance, "75 Federal Register 126" (1 July 2010), 38365-38383. Prior GAO reports: Guidance or report: Intercity Passenger and Freight Rail: Better Data and Communication of Uncertainties Can Help Decision Makers Understand Benefits and Trade-offs of Programs and Policies, [hyperlink, http://www.gao.gov/products/GAO-11-290]; (Washington D.C.: Feb. 24, 2011). Guidance or report: High Speed Passenger Rail: Future Development Will Depend on Addressing Financial and Other Challenges and Establishing a Clear Federal Role, [hyperlink, http://www.gao.gov/products/GAO-09-317]; (Washington D.C.: Mar. 19, 2009). Academic Literature: Guidance or report: Bent Flyvbjerg, "Quality Control and Due Diligence in Project Management: Getting Decisions Right by Taking the Outside View," International Journal of Project Management (November 2012). Guidance or report: Transportation Research Board, "Metropolitan Travel Forecasting: Current Practice and Future Direction, Special Report 288", (2007). Guidance or report: McFadden, Bolduc, Bierlaire, "The Estimation of Generalized Extreme Value Models from Choice-Based Samples", School of Architecture, Civil and Environmental Engineering, Ecole Polytechnique Federale de Lausanne (August, 2006). Guidance or report: The British Department for Transport, "Procedures for Dealing with Optimism Bias in Transport Planning", (June 2004). Guidance or report: Flyvberg, Bruzelius, Rothengatter, "Megaprojects and Risk: An Anatomy of Ambition," Cambridge University Press (2003); SourceFlyvberg, Bruzelius, Rothengatter, "Megaprojects and Risk: An Anatomy of Ambition," Cambridge University Press (2003). Source: GAO. [End of table] From these sources we identified recommended practices related to seven key steps relating to: (1) developing trip tables, (2) determining and applying service characteristics, (3) developing mode choice models, (4) estimating induced travel, (5) estimating expected fare revenue, (6) conducting sensitivity analysis, and (7) conducting validation testing. We compared generally accepted practices for each of these steps to the Authority's process for developing the ridership and revenue forecast as outlined in the April 2012 revised business plan and in supporting technical documentation. We could not evaluate each of the many detailed design decisions, assumptions, and model inputs used by the Authority, but rather focused on the seven key steps and whether they were implemented in accordance with generally accepted practices. We reviewed documents from and conducted interviews with Authority officials and their contractor--Cambridge Systematics--to obtain information about the Authority's process for developing the ridership and revenue forecasts. Specifically, we examined the Authority's process for developing the models used to produce the various forecasts, the assumptions and data sources used to develop the models, the survey instruments used to collect data, and the Authority's process for model estimation, calibration, and validation. We focused our analysis on identifying key steps in developing ridership forecast models for high-speed rail projects, elements affecting validity and reliability of models, common limitations of models and pitfalls, and recommended approaches for external review. In addition, we interviewed organizations that had conducted reviews of the Authority's ridership and revenue forecasts, such as academic experts from University of California Berkeley's Institute of Transportation Studies (ITS) and members of the Ridership and Revenue Peer Review Panel and the California High-Speed Rail Peer Review Group. From these interviews, we obtained additional information about (1) generally accepted methods used for project ridership and revenue for high-speed projects and elements of these approaches that have the greatest potential risk, (2) general assumptions underlying demand forecast models, elements impacting validity and reliability of models, and existing data limitations, and (3) the extent to which the Authority's approach follows generally accepted practices for developing valid and reliable ridership and revenue estimates. To assess the Authority's financing plan, we reviewed the plan, conducted interviews with Authority officials and other state and federal officials, and reviewed literature and other information on financing for high-speed rail projects in other countries as well as large transportation projects in the United States. To assess how well the Authority identified economic impacts associated with the high-speed rail project, we reviewed the April 2012 revised business plan as well as the April 2012 benefit-cost analysis and April 2012 economic impact analysis. In addition, to establish criteria for the various components of economic impact analysis, including user and non-user impacts, we reviewed pertinent legislation, such as the Passenger Rail Investment and Improvement Act of 2008 and the National Environmental Policy Act, and the NOFAs associated with the HSIPR and Transportation Investment Generating Economic Recovery (TIGER) grant programs. FRA officials told us the NOFA's outlined the type of information HSIPR grant applicants were to provide regarding project benefits and costs and how this information would be reviewed by FRA in reviewing grant applications. We also reviewed reports from the DOT OIG regarding HSIPR project viability assessments. In particular, we reviewed the June 2011 report prepared for the DOT OIG's office on best practices related to public benefit assessments of high-speed rail projects produced by the firm Steer Davies Gleave. The Steer Davies Gleave report identified important components of user and non-user impacts associated with public benefits assessments. To gain a better understanding of economic impact analysis, we reviewed the Economic Impact Analysis Primer prepared by the FHWA's Office of Asset Management. The Economic Analysis Primer identified the basic process of identifying and analyzing economic impacts, including benefit-cost analyses. The document also identified the similarities and differences between benefit-cost analyses and economic impact analyses. We also interviewed officials from FRA, the Authority, FHWA, the Federal Transit Administration, DOT OIG, and Steer Davies Gleave about economic impact issues. To assess issues related to high-speed rail and future travel demand, we reviewed the Authority's revised business plan and April 2012 equivalent capacity study as well as the November 2011 Statewide Transportation Systems Needs Assessment prepared by the California Transportation Commission (CTC). We also discussed these issues with officials from CTC and the California Department of Transportation as well as officials with local transportation agencies in California about potential improvement projects they had planned that were associated with the high-speed rail project. The proposed high-speed rail project is a very large public works project with costs expected to be spread over more than a decade. Depending on how cost figures are presented, different impressions of the magnitude and funding requirements of the program could be given. Whether or not the effects of inflation are included in the estimate is a source of significant differences. Year of expenditure (YOE) dollars include inflation in out-year costs, a convention adopted to facilitate budgeting over time but not necessarily a good representation of the true economic costs of the project. Removing the increase in cost attributable solely to inflation in the price level provides a better picture of burden on taxpayers and other funders because the tax base, including incomes, property values, and retail sales, would have increased with inflation as well. In the case of the high-speed rail project, the YOE cost total is 25 percent greater than that when inflation effects are removed. An estimate of the cost in present value terms, which accounts for inflation and the time value of money, would be smaller still. We conducted this performance audit from February 2012 to March 2013 in accordance with generally accepted government auditing standards. These standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix II: GAO Analysis of California High-Speed Rail Authority's Cost Estimates: Characteristic: Accurate; Overall assessment[A]: Substantially Met; Best practice: The cost estimate results are unbiased, not overly conservative or optimistic, and based on an assessment of most likely costs; Individual assessment: Partially Met; While the Authority has attempted to ensure accuracy and eliminate bias in their estimate by conducting sensitivity analysis, parametric checks, and the use of peer review, these have all been on subsets of the total program. No risk or sensitivity analysis has been developed at the program level or between the low and high estimates. Alternative high and low estimates do not create a range of estimates, but rather point estimates evolving around potential options. In the absence of cost risk and uncertainty analysis it is not possible to determine if the estimate is unbiased. Unless the estimate is based on an assessment of the most likely costs and reflects the degree of uncertainty given all of the risks considered, management will not be able to make informed decisions. Best practice: The estimate has been adjusted properly for inflation; Individual assessment: Substantially Met; Both capital investment and operations and maintenance (O&M) costs are inflated to YOE dollars using sound data and methodologies. Source data used for cost estimating are normalized to appropriate base years, although in some instances the normalizing processes were not clear. Best practice: The estimate contains few, if any, minor mistakes; Individual assessment: Fully Met. Best practice: The cost estimate is regularly updated to reflect significant changes in the program so that it is always reflecting current status; Individual assessment: Fully Met. Best practice: Variances between planned and actual costs are documented, explained, and reviewed; Individual assessment: Not applicable. Best practice: The estimate is based on a historical record of cost estimating and actual experiences from other comparable programs; Individual assessment: Substantially Met; The estimate relies on construction cost data from commercial databases heavily supplemented with local construction bids from analogous construction projects. The Authority collects technical and summary-level cost data on existing and future high-speed trainsets, but there is no documentation that explains how these data were adjusted for use in the cost estimate. The O&M estimate relies on applicable historical data. However, the extent of applicability is unknown because adjustments are not thoroughly documented. Best practice: The estimating technique for each cost element was used appropriately; Individual assessment: Substantially Met; The estimating techniques are reasonable for those Standard Cost Categories (SCC) elements discretely estimated, where a unit price estimating methodology was employed. The 2012 O&M model is a simplified version of the 2009 model, appropriately suited to the cost model's stated purpose of establishing program viability. However, the simplification results in an unnecessary loss of fidelity in some cost elements. Characteristic: Comprehensive; Overall assessment[A]: Partially Met; Best practice: The cost estimate includes all life cycle costs; Individual assessment: Substantially Met; The Authority has included all relevant costs with the relatively minor exclusion of disposition costs in the capital investment estimate. Best practice: The cost estimate completely defines the program, reflects the current schedule, and is technically reasonable; Individual assessment: Partially Met; The technical baseline description for the capital investment cost estimate resides in multiple documents that collectively comprise the technical baseline of the program. However, there is no distinct technical baseline description for the O&M estimate. Officials stated that later versions of the O&M estimate will align with the Concept of Operations plan, which was approved in February 2012. Best practice: The cost estimate work breakdown structure (WBS) is product-oriented, traceable to the statement of work/objective, and at an appropriate level of detail to ensure that cost elements are neither omitted nor double-counted; Individual assessment: Partially Met; The program utilized the FRA (SCC) and associated definitions for the capital investment costs. The cost estimate expands upon this structure to provide detailed identification of infrastructure work, but has reduced insight into common support costs. The standardized O&M FRA SCC elements were not used for capturing O&M costs because the O&M estimate was not required to comply with the SCC elements. While the O&M estimate includes common elements for administration and support costs, the O&M WBS is greatly simplified. As a consequence, up to two-thirds of O&M costs are collected in a single cost element. Best practice: The estimate documents all cost-influencing ground rules and assumptions; Individual assessment: Partially Met; Ground rules and assumptions are imbedded in much of the documentation for both the capital investment and O&M estimates as well as in the cost models, but not all assumptions have supporting rationale or sources. As the design for a specific section advances, risks are quantified and assigned to specific WBS elements. At the program level, contingency factors are used to capture less-defined risks. However, O&M risks are not specifically related to O&M WBS elements, and the impact of budget constraints on specific WBS elements has not been clearly defined. In addition, the impacts of technology maturity on cost are not entirely defendable. Unless ground rules and assumptions are clearly documented, the cost estimate will not have a basis for areas of potential risk to be resolved. Characteristic: Well documented; Overall assessment[A]: Partially Met; Best practice: The documentation should capture the source data used, the reliability of the data, and how the data were normalized; Individual assessment: Partially Met; The documentation provides some insight into the development of the cost estimates; however, much of our analysis was based on information derived from interviews and discussions with Authority representatives, not from readily available information in the documentation. The O&M model includes relevant data, but sources and variables can only be described as somewhat documented. For the most part, documentation relates how inputs are adjusted from past O&M models but fails to account for how earlier values were derived. Without sufficient background knowledge about the source and reliability of the data, the cost estimator cannot know with any confidence whether the data collected can be used directly or need to be modified. Best practice: The documentation describes in sufficient detail the calculations performed and the estimating methodology used to derive each element's cost; Individual assessment: Substantially Met; The documentation provided varying degrees of insight into the estimating methodology. The majority of costs--that is, infrastructure and site work--are described at a detailed level by unit cost, quantities, labor rates, equipment, and material costs, and the like. However, some cost elements had little or no supporting documentation. Best practice: The documentation describes step-by-step how the estimate was developed so that a cost analyst unfamiliar with the program could understand what was done and replicate it; Individual assessment: Partially Met; Details of the estimating process and methodology were provided for the track structure and track and site work elements of the capital investment model, but supporting data and details of how other elements are estimated were not available. No comprehensive document exists that explains the O&M model element by element. Without good documentation, management and oversight will not be convinced that the estimate is credible. In addition, analysts unfamiliar with the program will not be able to replicate the estimate because they will not understand the logic behind it. Best practice: The documentation discusses the technical baseline description and that the data in the baseline are consistent with the estimate; Individual assessment: Partially Met; The documentation of the capital investment cost model and the technical baseline are consistent with one another. The primary emphasis and underlying data sources are for the infrastructure and site work, but little definition or supporting data are provided for the remaining cost elements. In addition, the O&M cost estimate is not based on an approved technical baseline document, although officials state that later versions will be aligned to the Concept of Operations plan. Because the technical baseline is intended to serve as the basis for developing a cost estimate, it should be discussed in the cost estimate documentation. Best practice: The documentation provides evidence that the cost estimate was reviewed and accepted by management; Individual assessment: Partially Met; Documents indicate that the Authority's management team was engaged in reviewing the cost estimates and there are multiple indications that management reviewed pieces of the cost estimate. However, many of these reviews appear to be for subsets of the total program, either by construction package or phase, and focusing more on the financing rather than the detailed estimating methodology or underlying assumptions. While specific subsets of the estimate appeared to be reviewed by or discussed with management, we found no specific instance where the total program estimate, including supporting source data and estimating methodologies, was provided to senior management for review, discussion, and subsequent approval; Because a cost estimate should form the basis for establishing the budget, it is imperative that management understands how the estimate was developed, including the risks associated with source data and estimating methodologies. Characteristic: Credible; Overall assessment[A]: Partially Met; Best practice: The cost estimate includes a sensitivity analysis that identifies a range of possible costs based on varying major assumptions, parameters, and data inputs; Individual assessment: Partially Met; A formal sensitivity analysis has been performed for Contract Package 1, design and construction of the first 26-33 miles of trackway infrastructure between the counties of Madera and Fresno. In addition, the Authority conducted limited sensitivity analysis on summary-level variables in the O&M model. However, sensitivity analysis of the entire program estimate has not been done. The capital investment estimate includes low and high cost alternative alignments, and the O&M estimate provides three alternative scenarios driven by ridership options. However, without a complete sensitivity analysis that reveals how the cost estimate is affected by a change in a single assumption, the cost estimator will not fully understand which variable most affects the cost estimate. Best practice: A risk and uncertainty analysis was conducted that quantified the imperfectly understood risks and identified the effects of changing key cost driver assumptions and factors; Individual assessment: Partially Met; The Authority utilized FRA guidance in developing its estimates, guidance that does not require risk or uncertainty analysis at the program level this early in the design stage. Authority officials stated that more advanced engineering designs are being developed to support the process and that risk and uncertainty analysis has been undertaken on Contract Package 1. Authority officials acknowledge the existence of risk and have tried to accommodate expected risk through the application of contingency factors. While the capital investment and O&M models include a contingency element, the factors used do not appear to be based on historical data or analogous sources; Lacking risk and uncertainty analysis, management cannot determine a defensible level of contingency reserves that are necessary to cover increased costs resulting from unexpected design complexity, incomplete requirements, technology uncertainty, and other uncertainties. Best practice: Major cost elements were crossed checked to see whether results were similar; Individual assessment: Partially Met; The Authority recognizes the importance of crosschecks and identified a series of crosschecks to verify and validate the results of the data. Authority officials stated there are several stages of crosschecks and quality control which are described in their cost estimating procedures. Yet little documentation has been provided that would allow us to verify that crosschecks and alternative methodologies have been developed. For example, estimators have crosschecked major cost factors in the O&M model with cost data from foreign systems, but there is no evidence that costs have been estimated using different methodologies. The main purpose of cross-checking is to determine whether alternative methods produce similar results. If so, then confidence in the estimate increases, leading to greater credibility. The Authority has contracted with the International Union of Railways (UIC) for a study that intends to verify and validate the capital investment cost model. Authority officials stated that the UIC panel of experts will also provide a set of international cost comparisons for infrastructure maintenance and rolling stock maintenance. Best practice: An independent cost estimate was conducted by a group outside the acquiring organization to determine whether other estimating methods produce similar results; Individual assessment: Minimally Met; An independent cost estimate (ICE) was performed on the Merced-Fresno and Fresno-Bakersfield segments for infrastructure costs. However, while the segments cover 35 percent of the planned system rail length, they make up less than 10 percent of the overall estimated program cost. An ICE should be performed on the entire program, including O&M costs. ICEs can provide decision makers with additional insights into a program's potential costs because they frequently use different methods and are less burdened with organizational bias. Source: GAO. [A] Not Met--The Authority provided no evidence that satisfies any of the criterion, Minimally Met--The Authority provided evidence that satisfies a small portion of the criterion, Partially Met--The Authority provided evidence that satisfies about half of the criterion, Substantially Met--The Authority provided evidence that satisfies a large portion of the criterion, and Fully Met--The Authority provided complete evidence that satisfies the entire criterion. [End of table] [End of section] Appendix III: Description of Generally Accepted Travel-Demand-Modeling Practices and Authority's Methods for Developing Ridership and Revenue Forecasts: Travel-demand-modeling tasks: Developing trip tables; Generally accepted travel-demand-modeling practices: A central task of the ridership forecasting process involves collecting and compiling data on current travel patterns into trip tables. This process involves the following: * Collection of trip table data: In order to calculate the number of trips that divert to the new or improved high-speed rail mode, the level of trip making without the new or improved high-speed rail mode must first be established. Reliable information on modal travel volumes is a prerequisite for valid ridership and revenue forecasts. The competing modes from which high-speed rail draws its share are mainly auto, rail, bus and air. There are various methods for collecting trip table data for these modes, including, obtaining publicly available data collected through travel surveys as well as from special-purpose surveys developed and undertaken for the purpose of a particular transportation study. Data can also be obtained from trip distribution models and ticket sales; * Base and forecast year trip table development: The base and forecast year input trip tables are the basis for a study's ridership estimates and revenue forecasts. Any overestimate or underestimate of the trip tables will translate to high or low forecasts of ridership. Base trip tables generally summarize the current total number of trips by mode for each city pair along the route and are generally prepared by using a variety of sources of data on actual trip making patterns. Growth factors--which determine the rate of increase over time--can then be applied to the base trip tables to develop forecast year trip tables, which contain estimates of future travel on various modes in the absence of a proposed high-speed rail alternative. Forecast year trip tables may also be prepared by estimating future-year trips directly; * Trip segmentation: Trip tables are generally segmented by mode of travel, trip purpose, and other traveler characteristics. Criteria frequently used in defining market segments include trip purpose, trip length, traveler income, travel party size, and others; Authority's methods for developing ridership and revenue forecasts: * The Authority collected data from a variety of sources including, among others, socioeconomic data from local agencies, U.S. Census Bureau, and the California Department of Finance; travel data from various travel surveys; and highway, air, conventional rail, and urban transit network data from local agencies; * The high-speed ridership and revenue model for inter-regional travel was developed utilizing surveys and other statewide travel information. Intra-regional travel models from Metropolitan Planning Organizations (MPOs) in the San Francisco and Los Angeles regions were adapted for use in the high-speed rail ridership and revenue model from the models maintained by the MPOs for those regions. A factoring process was used to estimate ridership in the San Diego region; * Base year trip tables were developed from existing California regional models used by local authorities including the Metropolitan Transportation Commission (MTC) and the Southern California Association of Governments as well as interregional trip tables developed from travel survey data. Forecast year trip tables were developed by projecting base year forecast data to forecast year 2030,; * and then the models were run on 2030 projections; * Trip were segmented by long versus short trips (over and under 100 miles), and trip purposes (commute, business, recreation, other). Travel-demand-modeling tasks: Determining and applying service characteristics; Generally accepted travel-demand-modeling practices: Travel demand forecasts require information on the level of service (LOS) characteristics such as travel time and fares, of competing modes of travel, such as travel time for automobile and air travel, along the proposed route and of the high-speed rail alternative. LOS data are required for all modes in the study (i.e., air, rail, auto) and for base and forecast years. This information is used in the mode choice modeling process, whereby service characteristics of the various available modes are compared to the service characteristics of the proposed high-speed rail mode. This allows forecasters to estimate the likelihood that travelers will divert from their existing travel mode to high-speed rail; Determining and applying service characteristics for a proposed network involves: * Developing a network representation: Network representation can be detailed (i.e., with detailed representations of street and transit networks that include location, alignment, connections, and service characteristics) or can be less explicit and instead focus directly on zone to zone level of service data. A less explicit network representation can be used if the structure of the network is very simple); * Preparing skim tables: Skim tables contain data on the time, cost, and other service characteristics of the various modes that are available for a trip; Accurate and realistic representation of the base and forecast year LOS characteristics is of paramount importance for realistic high- speed ridership forecasting. Rail LOS information may be approximately derived from the service plan but may not represent it in complete detail; Authority's methods for developing ridership and revenue forecasts: * The Authority developed a detailed network representation for the entire state to forecast travel between regions. Data were obtained from the existing statewide highway network and details were added using data from local regional models, from the MTC, the Southern California Association of Governments, the San Diego Association of Governments, and data from the Kern County region; * LOS characteristics were defined for the four inter-regional travel modes: auto, conventional rail, high-speed rail, and air. LOS characteristics covered three broad categories: costs, times, and reliability, which were summarized in travel skim tables. Several of these characteristics were varied during model application to see how ridership and revenue would be impacted. Characteristics were collected from published or observed data from various sources including, the MTC and the Federal Aviation Administration. The high- speed rail characteristics were based on the initial service plan and fare structure. Travel-demand-modeling tasks: Developing a mode choice model; Generally accepted travel-demand-modeling practices: Mode choice modeling predicts the outcome of the decision process by which travelers choose the mode(s) to take from origin to destination. This step predicts the fraction of trips that will divert to the high-speed rail project from existing modes. Mode choice models are typically developed via a statistical analysis of the behavior of travelers in different situations. The statistical analysis may be based on behavior observed in actual travel situations (revealed preference data) or in hypothetical situations presented to travelers in a survey (stated preference data), or both; Selection of the mode choice model structure is important to obtain reliable and credible ridership and revenue estimates. Two types of mode choice models are commonly used in high-speed rail demand forecasting: * Choice models: These models are used to predict the decisions of travelers considering alternative transportation modes. Multinomial logit models and nested logit model are types of choice model that can be used; * Diversion choice model: A diversion choice model considers only two modes-the one in use in the base situation and the high-speed rail alternative; Authority's methods for developing ridership and revenue forecasts: The Authority developed two choice models: intra-regional urban model (models behavior associated with shorter distance and more frequent trip making) and an inter-regional model (models traveler behavior associated with longer-distance travel); * Intra-regional (Urban) Models: For both the San Francisco Bay Area and the greater Los Angeles regions, mode choice models were adapted from existing models to include the high-speed rail mode. The updated mode choice models were applied using the MPO trip tables for each region as input. San Diego is the only other region that contains the possibility of intra-regional high-speed rail trips, but the estimate of these riders was very low relative to the other regions. Because the level of effort to develop, calibrate, and apply the regional mode choice model was very high, intra-regional ridership for San Diego was developed using a population-based estimate rather than a traditional mode choice model; * Inter-regional models: The Authority developed four sets of models which included trip frequency, destination choice, primary mode choice, and access/egress mode choice. The destination choice component predicts the destinations of the trips generated in the trip frequency component based on zonal characteristics and travel impedances. The mode choice components (main mode choice, access mode choice and egress mode choice) predict the modes that the travelers would choose based upon the modal service levels as well as characteristics of the travelers and trips being made; * Data were derived from, among other sources, the California Department of Transportation Statewide Model, existing regional mode choice models, and revealed preference and stated preference survey data. The economic and household characteristics were forecast for each zone in the year 2030 based on data and forecasts from state, regional, and local government agencies. The primary main mode choice model relied primarily on stated preference data. Travel-demand-modeling tasks: Estimating induced travel; Generally accepted travel-demand-modeling practices: A high-speed rail project will improve the overall level of service for intercity travel within a given corridor. This improvement will make conditions more favorable for travel. Trips will therefore be taken on high-speed rail that might not otherwise have been made using any of the current modes. The new trips are commonly referred to as induced travel; An upper limit on induced travel of approximately 10 percent of total high-speed rail trips is widely accepted for proposed high-speed rail systems in the U.S; Authority's methods for developing ridership and revenue forecasts: The Authority forecasted 2.05 percent induced travel for the blended Phase 1 low scenario. Travel-demand-modeling tasks: Estimate expected fare revenue; Generally accepted travel-demand-modeling practices: Expected fare revenue is determined by a calculation using the ridership estimates generated by the model and the average fares. The total ridership for the system is generally calculated by adding the diverted trips calculated from the mode choice models and the induced trips to produce the total ridership for the system; Authority's methods for developing ridership and revenue forecasts: The Authority calculated fare revenue by multiplying the ridership estimates generated from the ridership model by the average high-speed rail fares forecasted for each region-to-region pair. Travel-demand-modeling tasks: Conduct sensitivity testing; Generally accepted travel-demand-modeling practices: All ridership and revenue forecasting studies should incorporate an analysis of the sensitivity of forecast results to key inputs and modeling assumptions including fare, running time, service frequency, station locations, and assumptions about socio-economic and travel growth in forecast years. Sensitivity analysis typically is conducted by varying, more or less systematically, selected forecasting model inputs, parameters, or assumptions (e.g., inflation rate or fuel cost) around their "standard" value, running the model, and examining the variation in outputs. Sensitivity analysis can help determine the reliability associated with the model output forecasts and can help identify the factors that have greatest impact on project ridership and revenue; Authority's methods for developing ridership and revenue forecasts: Several sensitivity tests were done to determine how the model reacted to different sets of assumptions such as changes to fuel costs, travel times, and fares. In addition, the Authority developed an extreme case scenario to test the sensitivity of the model to a series of downside events, such as increased average rail travel time from Merced to the San Fernando Valley (140 min. instead of 126 min), decreased train frequency (3 trains per hour instead of 4 trains per hour during peak times), lower auto-operating costs and lower air fares (10 percent below actual 2009 average air fares). Travel-demand-modeling tasks: Conduct model validation; Generally accepted travel-demand-modeling practices: Model validation is a key component of ridership and revenue forecasting and generally consists of testing the validity of the model using data other than (and usually newer than) the data from which it was estimated, to assess how well the model predicts actual ridership; There are two superior (but not often performed) ways of checking model performance: (1) the historical method, in which a prior-year model is used to forecast current travel, which is then compared with actual current travel; and (2) "backcasting," in which a current year model is used to estimate travel for a prior year, which is then compared with actual travel in the prior year. Backcasting is used by 5 percent of all and 13 percent of large MPOs; Authority's methods for developing ridership and revenue forecasts: The Authority validated the model through tests performed using Amtrak's Acela service in the Northeast Corridor (NEC) as input to the California high-speed rail model and compared the output with 2008 actual ridership and 2030 NEC forecasts. Efforts to validate the model by comparing to the NEC appear reasonable. The NEC is not an ideal test for the model but it is the only one available in the U.S. Use of foreign systems would raise difficult issues of comparability. Source: GAO, DOT OIG, FHWA, FTA, and academic studies of ridership and revenue best practices and Authority information. [End of table] [End of section] Appendix IV: Comments from the California High-Speed Rail Authority: California High-Speed Rail Authority: 770 L Street, Suite 800: Sacramento, CA 95814: 916-324-1541: [hyperlink, http://www.cahighspeedrail.cs.gov] March 8, 2013: Board Members: Dan Richard, Chairperson; Lynn Schenk Vice-Chairperson; Thomas Richards, Vice Chairperson; Jim Hartnett; Michael Rossi; Thomas J. Umberg; Jeff Morales, Chief Executive Officer. Ms. Susan Fleming: Director, Physical Infrastructure: U.S. Government Accountability Office: 441 G Street, NW: Washington D.C. 20548: Re: Response to Government Accountability Office Draft Report 13-304, "California High-Speed Passenger Rail: Project Estimates Could be Improved to Better Inform Future Decisions" (GAO Code 541090). Dear Ms. Fleming: The California High-Speed Rail Authority (Authority) appreciates the Government Accountability Office's (GAO) professionalism and thoroughness in conducting the most extensive external review to date of the program's cost estimates and other analyses. This letter highlights some of the findings that we found particularly important and discusses the areas where we have begun to take steps to incorporate your findings into our on-going efforts to improve our estimates. As the GAO states in the Draft Report, the Authority was not required to follow the GAO Cost Guide and did follow all applicable guidance from the Federal Railroad Administration (FRA), which is administering the Authority's federal grants. Where no guidance from the FRA existed, the Authority followed Federal Transit Administration (FTA) guidance. As guidance from the FRA and FTA continues to evolve, we will continue to meet all applicable requirements. As noted by the GAO, different components of the Authority's program (such as the implementation phases and construction packages) are at different stages of development and analysis. Given this, it would not be practicable to conduct every analysis in the Cost Guide at the program level at this time. That is because the environmental review is underway on a number of sections with alignment and other choices still to be made. As the program advances, the individual "projects" that comprise the overall program will all undergo each of the steps described in the Cost Guide. The GAO's finding that "the Authority substantially met best practices for developing accurate cost estimates," highlights the Authority's efforts to produce cost estimates that correctly reflect the program's scope. Additionally, the Draft Report states that the "GAO found the Authority's ridership and revenue forecasts to be reasonable" and that the methods and model used to develop them adhered to applicable best practices. This finding reinforces conclusions from the Independent Peer Review Group and Ridership Peer Review Panel that the ridership forecasts are reliable. Further, the Draft Report found that "The Authority comprehensively identified key economic impacts that could result from the high speed rail project," demonstrating the strong economic case for the project, including its ability to create jobs in the short-term and its potential to change the state's economic landscape in the long-term through improved connections between its metropolitan areas. While these findings are encouraging validation of the work that we have done to date, we fully acknowledge that any estimate can be refined and improved. As part of our effort to continue improving our estimates, we have already begun to incorporate many of the suggestions included in the GAO report. One such area is in the continued improvement of our ridership estimates. The updated ridership model being developed in time for the 2014 Business Plan will include many of the changes suggested by the GAO. The model will incorporate updated socioeconomic inputs and new data from survey work that is being conducted right now. These changes will continue to fine- tune and refine the estimates based on the most up-to-date information available on characteristics of the travel market in California. At the same time, we continue to improve the quantification of the risks associated with each project that makes up the overall program. As alignments are selected and individual pieces of the program continue to advance through various stages of design and development, we will be able to conduct many of the risk and uncertainty analyses that are recommended in the GAO's Cost Guide. The risk quantification analysis will be integral to the procurement and business model decisions that will form the framework for private investment in the program. Finally, we are also working to upgrade our Operations and Maintenance (O&M) cost forecasting. For the 2014 Business Plan, we are developing an improved O&M cost model that includes a much more detailed bottom- up cost estimate. That update is addressing issues raised in the GAO review, as well as detailed review and analysis by international high speed rail experts convened through the International Union of Railways, or MC. This updated model will be able to help validate the top-down estimate from the 2012 Business Plan and to test the financial impacts of different operating scenarios going forward. The process undertaken by the GAO over the last year has helped identify many of the areas where our estimates are strongest as well as areas of focus for continued improvement in the coming months and years. We will continue to incorporate GAO's input as we work to employ best practices in all aspects of the planning and delivery of our program. I thank you and your staff for your hard work and collaborative approach and we look forward to the release of the Final Report later this month. Please feel free to contact me with any questions at (916)384-1487. Sincerely, Signed by: Jeff Morales: Chief Executive Officer: [End of section] Appendix V: GAO Contact and Staff Acknowledgment: GAO Contact: Susan A. Fleming, (202) 512-2834 or flemings@gao.gov: Staff Acknowledgments: In addition to the individual named above, Paul Aussendorf, Assistant Director; Russell Burnett; Jason Lee; Delwen Jones; Richard Jorgenson; James Manzo (Technomics, Inc.); Maria Mercado; Susan Offutt; Paul Revesz; Max Sawicky; Maria Wallace; and Crystal Wesco made key contributions to this report. [End of section] Footnotes: [1] The National Railroad Passenger Corporation's (Amtrak) Acela service is currently the fastest passenger train service in the United States--it is capable of traveling at a maximum of 150 miles per hour but the average speed on the Washington, D.C., to Boston, Massachusetts, corridor is below 80 miles per-hour. [2] Estimates of the costs and revenues associated with the high-speed rail project appear in this report in two different formats: year of expenditure (YOE) and inflation adjusted dollars. YOE means the cost is presented in nominal dollars that reflect anticipated inflation over the period from the base year to the year in which a specific component of spending would take place. These estimates are also expressed in terms of current dollars in the base year of 2011, reversing the inflation adjustment used to calculate YOE dollars. The current dollar estimate will be lower than that expressed in YOE terms. Revenue estimates are expressed in current dollars in a base year. See appendix 1 for a discussion of the choice of methods for representing the costs of a project that spends out over a long period. Unless otherwise noted, all costs are presented in YOE dollars. [3] GAO, High-Speed Rail: Future Development Will Depend on Addressing Financial and Other Challenges and Establishing a Clear Federal Role, [hyperlink, http://www.gao.gov/products/GAO-09-317], (Washington, D.C.: Mar. 19, 2009). [4] Pub. L. No. 110-432, div. B, 122 Stat. 4848, 4960 (Oct. 16, 2008); Pub. L. No. 111-5, 123 Stat. 115, 208 (Feb 17, 2009). [5] Pub. L. No. 111-117, div. A, 123 Stat. 3034, 3056 (Dec. 16, 2009); Pub. L. No. 112-110, § 2222 (Apr. 15, 2011). [6] GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs, [hyperlink, http://www.gao.gov/products/GAO-09-3SP], (Washington, D.C.: March 2009). [7] Proposition 1A was codified in California law. See Cal. Sts. & High. Code § 2704 et seq. (2012). [8] This project will construct new rail right-of-way to provide service, some of which may require acquisition of privately owned land. [9] Program management services and ridership-and-revenue-modeling services for the California high-speed rail project are generally carried out by consultants under contract with the Authority. Parsons Brinckerhoff is the consulting firm responsible for program management of the California high-speed rail system. In addition, the Authority retained Cambridge Systematics--a transportation-consulting firm that provides ridership and modeling forecast services--to develop a ridership and revenue model for the project. [10] An additional $819.3 million was appropriated by the state legislature for connectivity projects and $252.6 million for environmental, system design, and preliminary engineering work. [11] In addition, $400 million was awarded to the Transbay Joint Powers Board for construction at the Transbay Transit Center in San Francisco. Pursuant to Proposition 1A, the Transbay Transit Center is the northern terminus of the California high-speed rail line. [12] The grant agreement between FRA and the Authority was amended in December 2012. The amendment revised the schedule of federal and state cash flows so that federal Recovery Act funds would be expended in advance of state funds. According to FRA, the amendment would ensure Recovery Act funds are expended by September 2017 and lead to reduced construction costs through reduced design-build contract bids. The original grant agreement called for a 50/50 match between federal and state funds. FRA officials said the grant amendment maintains this match but on a different time frame. [13] Department of Transportation Office of Inspector General, FRA Needs to Expand Its Guidance on High-Speed Rail Project Viability Assessments, CR-2012-083 (Washington, D.C.: Mar. 28, 2012). [14] GAO, High Speed Passenger Rail: Learning from Service Start-Ups, Prospects for Increased Industry Investment, and Federal Oversight Plans, [hyperlink, http://www.gao.gov/products/GAO-10-625] (Washington, D.C.: June 17, 2010). [15] Cal. Pub. Util. Code § 185033(a). [16] Cal. Pub. Util. Code § 185035(a), (c). [17] Cal. Sts. & High. Code § 2704.04(e). In April 2010 and January 2012, the California state auditor released two reports identifying several risks with the project including funding uncertainties and concerns about the Authority's level of oversight of the project. [18] These revenue forecasts are in 2010 dollars. [19] These revenue forecasts are in 2011 dollars. [20] GAO, High-Speed Passenger Rail: Preliminary Assessment of California's Cost Estimates and Other Challenges, [hyperlink, http://www.gao.gov/products/GAO-13-163T] (Washington, D.C.: Dec. 6, 2012). [21] [hyperlink, http://www.gao.gov/products/GAO-13-163T]. [22] The Authority developed a low cost estimate and a high cost estimate based on various potential routes for the train within the seven construction sections between San Francisco and Los Angeles. The Authority developed the low cost estimate based on a sum of the lowest cost route alignments in each construction section and, similarly, the high cost estimate reflects a sum of the highest cost route alignments in each construction section. The Authority then adjusted the 2011 cost estimates to calculate year-of-expenditure, or nominal dollars, using assumptions on expected inflation rates. [23] [hyperlink, http://www.gao.gov/products/GAO-09-317]. [24] Examples of where we have applied the Cost Guide can be found in the following: GAO, Air Traffic Control Modernization: Management Challenges Associated with Program Costs and Schedules Could Hinder NextGen Implementation, [hyperlink, http://www.gao.gov/products/GAO-12-223] (Washington, D.C.: Feb. 16, 2012); GAO, St. Lawrence Seaway: Estimates for the Asset Renewal Program Will Change, and Implementing Best Practices May Improve the Estimates' Reliability, [hyperlink, http://www.gao.gov/products/GAO-10-541R] (Washington, D.C.: May 13, 2010); and, GAO, VA Construction: VA Is Working to Improve Initial Project Cost Estimates, but Should Analyze Cost and Schedule Risks, [hyperlink, http://www.gao.gov/products/GAO-10-189], (Washington, D.C.: Dec. 14, 2009). [25] The DOT OIG has since developed best practices for developing operating cost estimates for high-speed rail projects and has recommended that FRA update its guidance to HSIPR grantees. See DOT OIG, FRA Needs to Expand its Guidance on High-speed Rail Project Viability Assessments, CR-2012-083 (Washington, D.C.: Mar. 28, 2012). [26] The Authority is mandated by state law to include in its 2014 business plan the results of a study by the UIC, examining how the estimated operating costs compares to rail systems in other countries. [27] The bids for the first construction project were not yet made public at the time this report was prepared. [28] The Authority was a project participant with MTC. The primary objective of this work was to provide information for the development of the Bay Area Regional Rail Plan, and to provide information to update environmental analyses to be conducted by the Authority. [29] [hyperlink, http://www.gao.gov/products/GAO-13-163T]. [30] [hyperlink, http://www.gao.gov/products/GAO-09-317]. [31] We further recommended that the methods could include such things as independent, third-party reviews of applicable ridership and other forecasts, identifying and implementing ways to structure incentives to improve the precision of ridership and cost estimates received from grant applications, or other methods that can ensure a high degree of reliability of such forecasts. [32] Department of Transportation OIG, FRA Needs to Expand Its Guidance on High-Speed Rail Project Viability Assessments, CR-2012-083 (Washington, D.C.: Mar. 28, 2012). [33] Steer Davies Gleave, Ltd., prepared for the U.S. Department of Transportation, Office of Inspector General, High Speed Intercity Passenger Rail Best Practices: Ridership and Revenue Forecasting (April 2011). [34] Developing a ridership model involves many design decisions, assumptions, and inputs with varying degrees of impact on the resulting forecasts. Of necessity, we did not evaluate every design decision, assumption, and input, so our analysis was limited to a high- level evaluation of the major steps taken. For the purposes of our analysis, we assessed "reasonableness" of the Authority's ridership and revenue forecasts by the extent to which the methodology used to generate these forecasts follows generally accepted practices for high- speed rail ridership modeling, and as such should be expected to produce generally plausible and defensible model results. A reasonable assessment indicates a level of confidence in the methods used to generate the ridership forecast and the resulting forecasts for decision-making purposes. [35] Steer Davies Gleave, High Speed Intercity Passenger Rail Best Practices, April 2011. [36] Trip tables summarize data on the number of trips by mode for each city pair along the proposed route. [37] The 2005 survey was performed under contract to and under the direction of the MTC. The Authority was a "project participant" with the MTC. [38] The Authority subcontracted with Harris Interactive--a market research firm--to conduct the May 2011 long-distance travel survey. The survey data collected were used to verify whether travel characteristics had changed since the earlier survey. [39] The Authority used existing data from the Metropolitan Transportation Commission and Southern California Association of Governments. [40] R. Baker, et. al., Public Opinion Quarterly, Prepared for the American Association for Public Opinion Research, AAPOR Report on Online Panels, October 20, 2010, 74(4): 711-781. AAPOR is an association comprised of professionals working in the survey and public opinion research field on various topics such as election polling, market research, statistics, and research methodology. AAPOR produces work focused on developing, among other resources, best practices in collecting, analyzing, and interpreting survey data. [41] Caltrain is a commuter rail service on the San Francisco Peninsula. According to Authority officials, the purpose of this sensitivity analysis was to understand the impact of running less frequent services in the Peninsula in order to better address the capacity issues on the Caltrain corridor and potentially save capital costs. Two primary changes were made to service characteristics in this analysis. The first was a reduction in Caltrain service from 12 trains per hour in the peak periods to 6 trains per hour, and only 1 train per hour in the off-peak period. The second change was a modification to the high-speed rail operating plan whereby service from San Francisco Transbay Terminal to Los Angeles Union Station was reduced from 4 trains per hour to 2 trains per hour in the peak and off-peak periods. Service from San Jose to Los Angeles Union Station was increased from 2 trains per hour to 3 trains per hour in the peak period. According to Authority officials, a high-speed rail service frequency of 4 trains per hour in the Peninsula is the service alternative currently being considered by the Authority and is identified in the April 2012 revised business plan. A 2 trains per hour alternative has not been approved. [42] Academic experts from the University of California Berkeley's ITS conducted a review of ridership and revenue forecast models used to develop forecasts in June 2010 and produced a report summarizing their findings. See D. Brownstone, M. Hansen, and S. Madanat, Review of Bay Area/California High-Speed Rail Ridership and Revenue Forecasting Study, University of California Berkeley's Institute of Transportation Studies, UCB-ITS-RR-2010-1, June 2010. [43] Some decisions, such as parameter adjustments, are made based on professional judgment and experience of forecast modelers and were not evaluated as part of our review. [44] D. Brownstone, M. Hansen, and S. Madanat, Review of Bay Area/ California High-Speed Rail Ridership and Revenue Forecasting Study, University of California Berkeley's Institute of Transportation Studies, UCB-ITS-RR-2010-1, June 2010. [45] M. Bierlaire, D. Bolduc, and D. McFadden, The Estimation of Generalized Extreme Value Models from Choice-based Samples. Transportation Research Board (2008) 42. [46] In 2012, ridership on Amtrak's premium Acela service was nearly 3.4 million passengers and total ridership on the regional NEC service (Acela and non-Acela) was more than 8.0 million passengers. [47] For example, the populations of cities along the NEC range from 0.3 to 18.8 million while the populations of cities along the Los Angeles to San Francisco corridor range from 0.1 to 12.9 million. [48] [hyperlink, http://www.gao.gov/products/GAO-09-317]. [49] Bent Flyvbjerg, "Quality Control and Due Diligence in Project Management: Getting Decisions Right by Taking the Outside View," International Journal of Project Management (Nov. 2012). [50] Federal funds for construction or improvement of federal-aid highways generally must be matched with funds from other sources. For example, unless otherwise specified in authorizing legislation, most projects that are part of the Interstate System have an 80 percent federal share. 23 U.S.C. § 120(b). [51] [hyperlink, http://www.gao.gov/products/GAO-09-317]. [52] The President's fiscal year 2013 budget request proposed providing multi-year funding ($47 billion over 6 years) for high-speed rail. [53] GAO, High-Risk Series: An Update, [hyperlink, http://www.gao.gov/products/GAO-13-283], (Washington, D.C.: February 2013) [54] According to their April 2012 revised business plan, the Authority is planning on applying $8.2 billion of the $9.95 billion in state high-speed rail bond proceeds to project construction. Some $950 million of the remaining $1.75 billion will be used for transportation projects that will connect to high-speed rail. And, the other $800 million will be used for environmental, planning, and support costs. [55] According to the California Air Resources Board--the state agency in charge of implementing the cap-and-trade program--California law requires that the volume of greenhouse gas emissions generated in the state be lowered to 1990 levels by 2020. To meet this goal, California established a limit on annual greenhouse gas emissions, or, a cap. The cap on pollution is divided into allowances that have to be acquired by polluting entities, such as coal-burning power plants. The state conducts periodic auctions to sell these allowances and also allows them to be traded (bought and sold) on a secondary market. [56] According to a report from California's Legislative Analyst's Office, the November 2012 auction was expected to result in a price of $10 to $50 per credit. According to the California Air Resources Board, the November 2012 auction resulted in a price of about $10 and raised approximately $290 million in auction proceeds. [57] As we reported in February 2008, an operating concession is a type of public-private partnership that offers rights to a private party to operate in return for a fixed-fee or percentage of revenues. See GAO, Highway Public-Private Partnerships: More Rigorous Up-front Analysis Could Better Secure Potential Benefits and Protect the Public Interest, [hyperlink, http://www.gao.gov/products/GAO-08-44] (Washington, D.C.: Feb. 8, 2008). For the California high-speed rail project, an operating concession would be an agreement between the Authority and a private firm, or consortium of firms, to retain ticket revenues with the responsibility to operate and maintain the service, in exchange for an up-front payment to the Authority. [58] [hyperlink, http://www.gao.gov/products/GAO-09-317]. [59] The Authority is planning for approximately $38.7 billion in additional federal funds for completing the entire Phase 1 of the project. Of this, approximately $25 billion will be used to complete the Initial Operating Segment. Operating profits, which are total operating revenues less operating expenses, are necessary to make an operating concession viable without other guarantees of income to the private investor. [60] Intercity passenger rail service in the United States is provided by Amtrak mostly over tracks owned by private freight railroads. There have been recent proposals to use private operators other than Amtrak for new intercity passenger rail services, such as the Desert Xpress and All Aboard Florida. [61] [hyperlink, http://www.gao.gov/products/GAO-08-44]. [62] British pounds converted to U.S. dollars based on the November 15, 2010 exchange rate according to historical data published by the Board of Governors of the Federal Reserve System. [63] The Authority's 2012 revised business plan estimates an average of $937 million in annual operating profits, from 2022 through 2034. [64] 74 Fed. Reg. 29900 (June. 23, 2009). [65] User impacts (or benefits) are the impacts on passengers who actually use a transportation improvement. That is, travelers must use the improvement to benefit from it. Non-user impacts (or benefits) are those impacts on those who do not directly use an improvement. [66] A benefit-cost analysis is designed to identify the alternative with the greatest net benefit by comparing the monetary value of benefits and costs of each alternative with a baseline. Benefit-cost analysis provides for a comparison of alternatives based on economic efficiency, that is, which investment or policy would provide the greatest net benefit (i.e., greater than costs). Economic impact analysis is a tool for assessing how the benefits and costs of transportation alternatives would be distributed throughout the economy and for identifying groups in society (for example, by region, income, or race) that are likely to gain from, or bear the costs of, a policy. [67] 42 U.S.C § 4321 et seq. [68] DOT OIG, FRA Needs to Expand Its Guidance on High-Speed Rail Project Viability Assessments, CR-2012-083 (March 28, 2012). In particular, we focused on best practices contained in the report HSIPR Best Practices: Public Benefits Assessment, Steer Davies Gleave, June 2011 that was prepared for the Office of Inspector General in conjunction with the March 2012 report. The Steer Davies Gleave report included a list of components that would be included in a public benefits assessment of high-speed rail projects. The components of public benefit assessments were similar to economic impacts identified through a review of the HSIPR NOFAs. [69] GAO, Bus Rapid Transit: Projects Improve Transit Service and Can Contribute to Economic Development, [hyperlink, http://www.gao.gov/products/GAO-12-811] (Washington, D.C.: July 25, 2012). Bus rapid transit projects vary in design but generally include service enhancements designed to attract riders and provide similar transit-related benefits to rail transit. For the purposes of the report the term economic development was used to refer to components of transit-oriented development, such as high-density, mixed-use developments and pedestrian-friendly environments and streetscapes. [70] GAO, Affordable Housing in Transit-Oriented Development: Key Practices Could Enhance Recent Collaboration Efforts between DOT-FTA and HUD, [hyperlink, http://www.gao.gov/products/GAO-09-871] (Washington, D.C.: Sept. 9, 2009). [71] GAO, Commuter Rail: Potential Impacts and Cost Estimates for the Cancelled Hudson River Tunnel Project, [hyperlink, http://www.gao.gov/products/GAO-12-344] (Washington, D.C.: Mar. 9, 2012). [72] According to the Bureau of Labor Statistics, a full employment economy in 2020 would have a 5.2 percent unemployment rate. See Bureau of Labor Statistics, Monthly Labor Review, January 2012. [73] The BCA states that different scenarios were considered in developing the analysis period and, depending on the scenario, the construction period varied. The analysis period used was 47 years beyond project completion for the scenario with the longest construction period. The assumed operating period is not intended to indicate the system will stop operating in 2060. [74] According to the EIA, the unemployment rate over the San Francisco Bay Area to Los Angeles corridor varied. The EIA states that through the recent recession, the San Francisco Bay Area unemployment rate was 1.0 to 1.5 percentage points less than the state as a whole. At the time of the EIA, the Central Valley unemployment rate about 4 percent higher than the statewide average and the Southern California unemployment rate was within 1 percent of the statewide rate. [75] The Authority's EIA states that a "job-year" refers to the idea that one person working one job for 20 years represents 20 job-years. It is not the number of individuals working. Also, the employment factor used to develop the estimate is for construction and capital spending and not employment that might be associated with spending for operation and maintenance of the high-speed rail system. [76] HSIPR Best Practices: Public Benefits Assessment, Steer Davies Gleave, June 2011, prepared for Office of Inspector General, U.S. Department of Transportation. [77] U.S. Department of Transportation, Federal Highway Administration, Economic Analysis Primer (August 2003). [78] GAO, Intercity Passenger and Freight Rail: Better Data and Communication of Uncertainties Can Help Decision Makers Understand Benefits and Trade-offs of Programs and Policies, [hyperlink, http://www.gao.gov/products/GAO-11-290] (Washington, D.C.: Feb. 24, 2011); Executive Order 12893, Principles for Federal Infrastructure Investments(Jan. 26, 1994); OMB Circular A-94 Revised, Guidelines and Discount Rates for Benefit-Cost Analysis of Federal Programs (Oct. 29, 1992); and OMB Circular A-4, Regulatory Analysis (Sept. 17, 2003). [79] [hyperlink, http://www.gao.gov/products/GAO-11-290]. [80] California High-Speed Rail Authority, Comparison of Providing Equivalent Capacity to High-Speed Rail through Other Modes (April 2012). [81] The scope of the analysis was the 520-mile Phase 1 system. Comparisons between high-speed rail and highways and aviation were based on system capacity, or "people carrying" capacity, not ridership estimates. According to the analysis, system capacity was used, in part, to recognize that capacity provides a steady state comparison between transport modes since it is tied to physical infrastructure, not the number of people using that infrastructure. The Authority assumed 12 trains per hour would be operated in each direction, 1,000 seats per train, and an average load factor of 70 percent. Load factor is the number of average passengers on a train divided by the number of available seats. [82] California Transportation Commission, 2011 Statewide Transportation System Needs Assessment, Final Report (November 2011). [83] GAO, Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs, [hyperlink, http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009). [84] For the purposes of our analysis, we assessed "reasonableness" of the Authority's ridership and revenue forecasts by the extent to which the methods used to generate these forecasts followed generally accepted practices for high-speed rail demand modeling, and as such should be expected to produce generally plausible and defensible model results. A reasonable assessment indicates a level of confidence in the methods used to generate the ridership forecast and the resulting forecasts for decision-making purposes. [85] Steer Davies Gleave, prepared for the U.S. Department of Transportation, Office of Inspector General, High-Speed Intercity Passenger Rail Best Practices: Ridership and Revenue Forecasting (April 2011). [End of section] GAO’s Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select “E-mail Updates.” Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]