Skip to main content

Transportation Research: Actions Needed to Improve Coordination and Evaluation of Research

GAO-03-500 Published: May 01, 2003. Publicly Released: May 01, 2003.
Jump To:
Skip to Highlights

Highlights

The Research and Special Programs Administration (RSPA) within the Department of Transportation (DOT) is responsible for coordinating and ensuring the evaluation of DOT research programs to promote the efficient use of departmental research funds, which in fiscal year 2002 totaled over $1 billion. RSPA is also responsible for conducting multimodal research that cuts across different modes of transportation. The House Committee on Appropriations directed GAO to examine RSPA's coordination and evaluation of research within DOT and the status of its own multimodal research.

Recommendations

Recommendations for Executive Action

Agency Affected Recommendation Status
Department of Transportation To better meet legislation and DOT requirements for coordinating and evaluating transportation research within the department, the Secretary of Transportation should, in conjunction with the RSPA Administrator, work with DOT operating administrations to develop a strategy for reviewing all DOT research projects to identify areas of unnecessary research duplication, overlap, and opportunities for joint efforts. The strategy should address time frames for implementing this review as well as discuss the extent to which finalizing the development and implementation of a DOT-wide research tracking system database could serve to facilitate this process. Once this strategy has been developed and implemented, the results of this effort should be incorporated in the department's annual research plan and reported to the Congress on an annual basis.
Closed – Implemented
RITA (formerly RSPA) has developed strategy to review all of DOT research programs to identify areas of unnecessary research duplication overlap and opportunities for joint efforts. RITA's strategy consists of on-going internal reviews of all of DOT's research programs, conducted by the Program Review Working Group. The objectives of these internal reviews are to continuously improve transportation research management and performance; identify, share, and learn best practices; identify opportunities for leveraging resources and for crossmodal research initiatives; and prevent unnecessary research duplication. RITA's Program Review Working Group conducts these reviews by (1) convening meetings where officials from each of the operating administrations share information about areas of on-going or planned research, looking for opportunities for joint effort, and (2) conducting annual reviews of each operating administrations research plans, looking for research duplication, among other things. Plans for a database, which could assist in identifying areas of research duplication and opportunities for joint efforts are progressing, in that according to a RITA official, funding has been identified and the testing of a preliminary system is anticipated for Spring 2008, pending final approval. The results of RITA's coordination activities are discussed during Planning Team and Planning Council meetings and have been reported in the 2006-2010 Research, Development, and Technology Strategic Plan. According to the Strategic Plan, in fiscal year 2005 and again in fiscal year 2006, the Program Review Working Group found no research duplication and identified a number of areas for crossmodal collaboration, including geospatial technologies, freight capacity, security, alternative energy technologies, and advanced materials and sensors. A RITA official told us that eight working groups have been formed with representation from the relevant operating administrations to foster collaboration in these different areas. In addition to the Strategic Plan, the results of the coordination efforts are also included in the Research, Development and Technology Annual Funding Fiscal Years 2006-2008, A Report to Congress, expected to be delivered to Congress in mid-September 2007.
Department of Transportation To better meet legislation and DOT requirements for coordinating and evaluating transportation research within the department, the Secretary of Transportation should, in conjunction with the RSPA Administrator, work with DOT operating administrations to develop and apply quantifiable performance measures to assess the effectiveness of research coordination efforts (once a strategy for review has been developed and implemented), and document the results of these efforts in the department's annual research plan. These measures could include the number or research projects identified as possible candidates for joint effort or elimination and/or the associated reduction in the department's research spending.
Closed – Not Implemented
DOT and RSPA did not concur with this recommendation in 2003, citing their views that: (1) the most useful and effective performance measures focus on results, while coordination is a process; (2) there are no performance measures capable of quantifying effective coordination; and (3) existing coordination processes have been effective in preventing unnecessary research duplication. RITA (formerly RSPA) officials also told us that they did not develop performance measures to assess the effectiveness of research coordination effort because they do not believe that a metric approach is well suited for assessing the effects of the coordination process.
Department of Transportation To better meet legislation and DOT requirements for coordinating and evaluating transportation research within the department, the Secretary of Transportation should, in conjunction with the RSPA Administrator, work with DOT operating administrations to develop a strategy to ensure that the results of all DOT's transportation research activities are evaluated according to established best practices. This strategy should include estimates of the costs for ensuring that evaluations are completed. Once the strategy has been developed and implemented, the results of these efforts should be incorporated in the department's annual research plan and reported to the Congress on an annual basis.
Closed – Implemented
RITA (formerly RSPA) has developed strategy to ensure that the results of all DOT's research activities are evaluated according to established best practices. RITA's strategy includes three primary mechanisms, including (1) ensuring systematic application of the Office of Management and Budget's Research and Development Investment Criteria (relevance, quality, and performance) and the Program Assessment Rating Tool by the operating administrations; (2) annual internal program reviews with self-reporting by the operating administrations; and (3) documenting the operating administration's external stakeholder coordination and review. This strategy, documented in the agency's 2006-2010 Research, Development, and Technology Strategic Plan, upholds the technical quality of DOT's research activities and helps to ensure that research activities are addressing critical needs. The results of the fiscal year 2005 and 2006 reviews are included in the "Research, Development and Technology Annual Funding Fiscal Years 2006-2008: A Report to Congress", expected to be delivered to Congress in mid-September 2007. According to RITA, the fiscal year 2007 reviews have been focused on how well the operating administrations are implementing best practices, including external stakeholder involvement, merit review of competitive proposals, independent expert review, research performance measures, and external research coordination. RITA will report the results of its fiscal year 2007 reviews to the Planning Council and will publish the results in the next Research, Development and Technology Annual Funding Report to Congress. RITA has not estimated costs for these evaluations and a RITA official told us they do not have plans to do so since many of the evaluations have already been completed.
Department of Transportation To better meet legislation and DOT requirements for coordinating and evaluating transportation research within the department, the Secretary of Transportation should, in conjunction with the RSPA Administrator, work with DOT operating administrations to include in the department's annual research plan a summary of all research program evaluations conducted and a schedule of future evaluations.
Closed – Implemented
In 2005, RSPA published a summary of all research program evaluations in that it published the results of its review of the operating administrations' application of the Office of Management and Budget's Research and Development Investment Criteria in its Fiscal Year 2005 Annual Research, Development, and Technology Plan. The results of RITA's fiscal year 2005 and 2006 reviews, and a schedule of RITA's planned fiscal year 2007 reviews, is included in the "Research, Development and Technology Annual Funding Fiscal Years 2006-2008, A Report to Congress", expected to be delivered to Congress in mid-September 2007. This report to Congress also includes summaries of research program evaluations conducted by modal research advisory committees, the Transportation Research Board, and key modal stakeholders in fiscal years 2006 and 2007.
Department of Transportation The Secretary of Transportation should direct RSPA's Administrator to document RSPA's process for systematically evaluating the results of its own multimodal research programs, and apply this process to any future multimodal research programs that RSPA conducts.
Closed – Implemented
RITA has established a systematic process for evaluating all of DOT's research, including RITA's multi-modal research programs. This process consists of reviews to identify, share, and apply best practices, identify opportunities for leveraging resources, and advance cross-modal research. RITA oversees DOT's Intermodal Advanced Research Working Groups which have identified 8 cross-modal research areas for collaboration with operating administrations including: congestion reduction; energy efficiency/alternative fuels; enhanced safety data; human factors; nanotechnology; positioning, navigation and timing; safe transport in an aging society, and system resilience/global logistics. RITA has developed an evaluation strategy to ensure that these cross-modal activities, as well as other DOT research activities, are evaluated according to best practices, including systematic application of OMB investment criteria, external stakeholder involvement, merit review of competitive proposals, independent expert review, development of research performance measures, and external research coordination. RITA officials told us that they are continuing to pursue the development and implementation of a more consistent process to be applied across all program areas, which they expect to complete by the end of fiscal year 2008.

Full Report

Office of Public Affairs

Topics

Federal aid for transportationFinancial managementFunds managementTraffic surveillanceInternal controlsProgram evaluationResearch program managementTransportation researchResearch programsElectric vehicles