This is the accessible text file for GAO report number GAO-13-611 entitled 'National Transportation Safety Board: Management and Operational Improvements Found but Strategy Needed to utilize Cost Accounting System' which was released on July 24, 2013. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Committees: July 2013: National Transportation Safety Board: Management and Operational Improvements Found but Strategy Needed to utilize Cost Accounting System: GAO-13-611: GAO Highlights: Highlights of GAO-13-611, a report to congressional committees. Why GAO Did This Study: The NTSB plays a vital role in transportation safety. It is charged with investigating all civil aviation accidents in the United States and selected accidents in other transportation modes, determining the probable cause of these accidents, and making appropriate recommendations, as well as performing safety studies. In 2006, NTSB’s reauthorization legislation mandated GAO to annually evaluate its programs. From 2006 to 2008, GAO made 21 recommendations to NTSB aimed at improving management and operations across several areas. Since that time, NTSB has taken action to address all 21 recommendations. Some of these were completed by requiring only a single action, whereas others required continuing effort to achieve operational improvement. For this review, GAO examined the extent to which desired outcomes are being achieved in five areas where continuing effort was necessary. GAO analyzed workforce, financial, and program data, and interviewed agency officials about actions NTSB has taken. What GAO Found: GAO’s analysis found varying degrees of improvement associated with the National Transportation Safety Board’s (NTSB) actions in areas selected for review. * Training Center utilization. NTSB increased utilization of its Training Center—both non-classroom and classroom space—since 2006. NTSB has also set and achieved its cost recovery goal at the Training Center in the last 2 fiscal years, allowing NTSB to recover half of its operating costs. * Recommendation close-out process. By automating the recommendation follow-up process, NTSB has reduced by about 3 months the amount of time it takes to respond to agencies on whether planned actions to implement NTSB recommendations are acceptable; this allows agencies to move forward with approved actions sooner than under NTSB’s former paper-driven process. * Communication. NTSB employees’ responses on federal employee surveys from 2004 to 2012 indicated an increase from 49 to 57 percent in employees’ positive responses regarding managers’ communication about agency goals, and from 44 to 49 percent regarding the amount of information received. We compared NTSB employees’ responses to those of employees from a group of small agencies and found that NTSB employees’ satisfaction level was about the same or more positive depending on the question. NTSB officials continue to monitor employees’ views about communication to address any remaining concerns. * Diversity management. NTSB employees’ positive responses to the federal employee survey questions about managers’ commitment to diversity and NTSB’s diversity policies and programs increased from about 54 percent to over 70 percent from 2004 to 2012. However, employees’ positive responses to the question about managers’ ability to work well with employees with different backgrounds declined 6 percentage points over the same period. In addition, the proportion of minority and women employees in NTSB’s workforce, including in its investigator staff, showed little appreciable change over the period 2008 to 2012. NTSB’s workforce had a smaller proportion of some minority groups than the civilian labor force. NTSB officials are using results from their recent diversity survey to identify gaps in their diversity management efforts and to benchmark future progress. It is too soon to tell whether NTSB’s actions will lead to additional changes in its workforce diversity profile. * Financial management. To improve operational effectiveness, NTSB has implemented a cost accounting system that includes a time and attendance program to track staff hours and costs related to accident investigations. NTSB is currently focused on ensuring the quality of the time and attendance data, but has not yet developed a strategy to maximize the utility of its cost accounting system for making resource and operational decisions. Thus, NTSB has not yet fully achieved its vision of using the data to improve labor productivity and mission effectiveness. What GAO Recommends: In each of the five areas NTSB needs to continue its improvement efforts. Further, GAO recommends that NTSB senior managers develop a strategy for maximizing the utility of NTSB’s cost accounting system. GAO provided a draft of this report to officials at NTSB. NTSB officials concurred with the recommendation and provided technical comments, which GAO incorporated as appropriate. View [hyperlink, http://www.gao.gov/products/GAO-13-611]. For more information, contact Gerald L.Dillingham, Ph.D at (202) 512-3834 or dillinghamg@gao.gov. [End of section] Contents: Letter: Background: Results across NTSB Management Areas: Conclusion: Recommendation: Agency Comments: Appendix I: Objective, Scope, and Methodology: Appendix II: GAO Contact and Staff Acknowledgments: Related GAO Products: Table: Table 1: Profile of the National Transportation Safety Board Workforce by Race, Gender, and National Origin, 2008 through 2012: Figures: Figure 1: Total Expenses, Revenue and Deficit, National Transportation Safety Board Training Center, Fiscal Years 2004 through 2012. Figure 2: National Transportation Safety Board's Average Response Time and Number of Responses to Agencies' Recommendation Correspondence, by Quarter, Fiscal Year 2009 through First Quarter, Fiscal Year 2013: Figure 3: National Transportation Safety Board Employee Responses about Communication on Selected Office of Personnel Management Survey Questions, 2004 through 2012: Figure 4: National Transportation Safety Board Employee Responses about Diversity and Inclusiveness on Selected Office of Personnel Management Survey Questions, 2004 through 2012: Abbreviations list: BLS: Bureau of Labor Statistics CNS: Correspondence, Notation, and Safety Recommendation system DOT: Department of Transportation NTSB: National Transportation Safety Board OPM: Office of Personnel Management: [End of section] GAO: United States Government Accountability Office: 441 G St. N.W. Washington, DC 20548: July 25, 2013: The Honorable John D. Rockefeller IV: Chairman: The Honorable John Thune: Ranking Member: Committee on Commerce, Science, and Transportation: United States Senate: The Honorable Bill Shuster: Chairman: The Honorable Nick J. Rahall, II: Ranking Member: Committee on Transportation and Infrastructure: House of Representatives: The National Transportation Safety Board (NTSB) plays a vital role in advancing transportation safety by investigating every civil aviation accident in the United States and selected accidents involving other transportation modes, determining the probable causes of these accidents, and making subsequent recommendations. NTSB also develops studies that raise awareness of emerging safety issues. Since 2008, NTSB has investigated more than 1,100 aviation accidents and dozens of accidents in other transportation modes, and issued over 1,000 recommendations to different agencies for improving safety. Most recently, in 2012 NTSB launched 279 accident investigations and made 207 recommendations to improve transportation safety. NTSB is a relatively small agency with a staff of about 400 employees and a fiscal year 2013 budget of $104 million. In light of NTSB's expansive mission to investigate accidents across all transportation modes, it is critical that NTSB manage its resources and operations as efficiently as possible. In 2006, NTSB's reauthorization legislation mandated GAO to annually evaluate its programs.[Footnote 1] GAO conducted a series of annual reviews covering NTSB's management and operations, including examining the extent to which NTSB (1) followed leading practices in selected management areas, (2) addressed challenges in completing investigations, and (3) generated sufficient revenues to cover its Training Center costs.[Footnote 2] GAO's reviews resulted in 21 recommendations to NTSB aimed at improving management and operations across 12 areas.[Footnote 3] In 2012, we reported that all 21 of these recommendations had been implemented and closed. [Footnote 4] Since then, NTSB has undertaken numerous actions and continues to plan and implement changes in an effort to improve its management and operations. However, measuring the outcomes of NTSB's actions is necessary to determine whether those actions have led to measurable improvements and whether NTSB is efficiently marshaling these efforts and using resources effectively. Our objective in this review was to assess whether there has been management and operational improvements associated with NTSB's actions. We considered recommendations within management and operational areas of key importance to NTSB's current priorities, as stated in its 2013-2016 Strategic Plan, areas that required continuing effort to achieve desired outcomes. This process yielded five recommendations for our review. We then identified outcome measures to assess changes by comparing current conditions with the initial conditions at the time we performed our previous work, or earlier in some instances, to establish a baseline. We used NTSB's financial and program data, employee survey data from the Office of Personnel Management (OPM), and workforce data from NTSB and the Bureau of Labor and Statistics (BLS). To ensure the data used were of sufficient reliability for our analysis, we examined program reporting procedures and quality assurance controls, and discussed various data elements with knowledgeable agency officials. We also spoke with NTSB officials who were knowledgeable about management and operations related to the 5 selected recommendations. We conducted this performance audit from August 2012 through July 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. See Appendix I for a more detailed explanation of our scope and methodology. Background: NTSB was initially established within the newly formed Department of Transportation (DOT) in 1966,[Footnote 5] but was made independent from DOT in 1974.[Footnote 6] NTSB's principal responsibility is to promote transportation safety by investigating transportation accidents,[Footnote 7] determining probable cause, and issuing recommendations to federal and state agencies, as well as industry, to address safety issues identified during accident investigations. NTSB also leads teams of investigators to assist with selected international aviation accident investigations. Unlike regulatory transportation agencies, such as the Federal Aviation Administration, NTSB does not have the authority to promulgate regulations to promote safety, but makes recommendations in its accident reports and safety studies to agencies that have such regulatory authority. In addition, NTSB conducts safety studies resulting from identified safety concerns rather than as the result of a specific accident. NTSB is comprised of a five-person board--a chairman, vice chairman, and three members--appointed by the President with the advice and consent of the Senate. The chairman is the NTSB's chief executive and administrative officer. The agency is headquartered in Washington, D.C., and maintains 9 field offices and a training center located in Ashburn, Virginia. In fiscal year 2013, the board was supported by a staff of about 400, which includes nearly 140 investigators assigned to its modal offices--aviation; highway; marine; and rail, pipeline, and hazardous materials--as well as 73 investigation-related employees, such as engineers and meteorologists. NTSB's modal offices vary in size in relation to the number of investigators, with the Aviation Safety office being the largest. In addition, the Office of Research and Engineering provides technical, laboratory, analytical, and engineering support for the modal offices. Staff from this office interpret information from flight data recorders, create accident computer simulations, and publish general safety studies. This review focuses on the extent to which NTSB has achieved measurable improvements from actions the agency has taken in five management and operational areas based on prior GAO recommendations. * Training Center utilization. Making efficient and effective use of resources provided by Congress is a key responsibility of federal agencies. NTSB's Training Center, which opened in August 2003 in Ashburn, Virginia, consists of classrooms, offices, and laboratory facilities used for instructional purposes and active investigations. [Footnote 8] NTSB uses this center to train its own staff and others from the transportation community to improve accident investigation techniques. NTSB charges tuition for those outside NTSB to take its courses, and generates additional revenue from space rentals to other organizations for events such as conferences on a cost reimbursable basis. Although there is no statutory requirement that NTSB cover the cost of its Training Center through the revenues generated from the facility, a 2007 review we conducted found that NTSB wasn't capitalizing on its lease flexibility to generate additional revenues and classrooms were significantly underutilized.[Footnote 9] For example, we found that less than 10 percent of the available classroom capacity was used in fiscal years 2005 and 2006. Furthermore, NTSB was encouraged in a Senate report accompanying the 2006 appropriations bill for DOT to be more aggressive in imposing and collecting fees to cover the costs of the Training Center.[Footnote 10] Since then, NTSB leased a large portion of the Training Center's non-classroom space to the Federal Air Marshall Service and provided short-term leases of classroom space to other organizations. In addition, NTSB increased the amount of training it delivered at the Training Center. * Recommendation close-out process. Efficiently managing the recommendation tracking process is a key function, according to NTSB officials. The recommendation close-out process is managed by the Safety Recommendations and Quality Assurance Division, which has responsibility for tracking the status of its recommendations. When NTSB receives correspondence from an agency about an NTSB recommendation, this division ensures it is properly routed and reviewed and contacts the agency about whether the response is acceptable. If NTSB is delayed in communicating with agencies about whether NTSB considers actions to address recommendations acceptable, that agency could delay implementing a course of action pending approval. In fiscal year 2010, NTSB replaced a lengthy, paper-based process with an automated system--the Correspondence, Notation, and Safety Recommendation system (CNS)--intended to facilitate the recommendation close out process by electronically storing and automatically routing agencies' proposals to the appropriate NTSB reviewers, allowing for concurrent reviews by multiple parties within NTSB, and more accurately tracking responses. It is important to note that an agency is not necessarily restricted from implementing action prior to formal NTSB approval of that action. Depending on the complexity of the issue, agencies may begin to address issues prior to NTSB's providing formal approval. In other circumstances, NTSB addresses safety deficiencies immediately before the completion of an investigation. For example, during the course of the TWA flight 800 investigation, NTSB issued an urgent safety recommendation once it was determined that an explosion in a fuel tank caused the breakup of the aircraft. * Communication. Useful management practices include seeking and monitoring employee attitudes, encouraging two-way communication between employees and management, and incorporating employee feedback into new policies and procedures. This type of communication and collaboration across offices at all levels can improve an agency's ability to carry out its mission by providing opportunities to share best practices and helping to ensure that any needed input is provided in a timely manner. To this end, NTSB managers and board members began holding periodic meetings with staff, conducting outreach to regional offices, and surveying staff about the effectiveness of communication techniques. * Diversity management. Implementing a diversity management strategy and a more diverse workforce helps foster a work environment that not only empowers and motivates people to contribute to mission but also provides accountability and fairness for all employees. Diversity management helps an organization create and maintain a positive work environment where the similarities and differences of individuals are valued, so that all can reach their potential and maximize their contributions to the organization's strategic goals. NTSB has developed diversity training courses and held events to educate staff on diversity and inclusiveness issues, created career development and mentoring programs to create upward mobility, targeted its recruitment program to reach a more diverse pool of applicants, and surveyed staff to assess the effectiveness of its efforts. * Financial management. Sound financial management is crucial for responsible stewardship of federal resources.[Footnote 11] Traditionally, government financial systems and government managers have focused on tracking how agencies spend their budgets but did not focus on assessing the costs of activities to achieve efficiencies. More recently however, some government agencies have adopted cost accounting systems that track the cost of providing a service--in NTSB's case, an accident investigation. In 2006, GAO recommended that NTSB develop a cost accounting system to track the amount of time employees spend on each investigation and other activities. This approach allows management to link the cost of providing a service directly with the budget and allocate resources based on those costs. To determine the costs associated with conducting accident investigations, NTSB launched a time and attendance program tied to its cost accounting platform that allows the agency to collect and analyze labor and certain other costs associated with individual investigations. Investigators account for their time on investigations through the time and attendance system using specific codes that identify different investigations. Results across NTSB Management Areas: Our analysis found varying degrees of improvement associated with NTSB's actions in each of the management and operational areas we selected for review. Training Center Utilization: Our analysis showed that NTSB improved the utilization of the Training Center, which allowed it to recover a larger portion of its operating costs. NTSB increased utilization of both classroom and non-classroom space at the Training Center since we conducted our work in 2006. NTSB subleased all available office space at the Training Center to the Federal Air Marshal Service in 2007, and utilization of non-classroom spaces has been at 95 percent since then. At the same time, NTSB increased utilization of the classroom space, increasing its own use of classrooms, subleasing approximately one-third of the classroom space to the Department of Homeland Security in 2008, and providing short term leases to other outside parties for classroom use. Subsequently, NTSB reported classroom utilization rose from less than 10 percent in 2005 to 18 percent in fiscal year 2007. By fiscal year 2009, it had increased to over 60 percent--the target we identified in our 2008 report as the appropriate minimum level.[Footnote 12] Classroom utilization has remained above 60 percent through fiscal year 2012. We also found that improved Training Center utilization generated additional revenue over time, which allowed NTSB to recover a larger portion of the facility's operating costs. When the Training Center first opened in fiscal year 2004, NTSB recovered about 4 percent of its operating costs, resulting in a deficit of nearly $6.3 million. Portions of the Training Center's costs that are not covered by revenues from tuition and other sources such as facility rentals are offset by general appropriations to the agency; therefore, generating additional revenue makes those appropriated funds available for other uses. In 2011, NTSB indicated that it was committed to improving cost recovery at the Training Center. That year the agency set a goal to recover costs of the Training Center within 10 percent of the previous fiscal year. For example, in fiscal year 2010, NTSB recovered $2 million in operating costs, making the fiscal year 2011 goal to recover at least $1.8 million of the Training Center's costs. NTSB achieved its goal in fiscal years 2011 and 2012 by which time the agency was recovering about half of the Training Center's operating costs, reducing the operating deficit at the Training Center to $2.1 million, one-third of what it was in 2004. (See figure 1 for changes in the Training Center's expenses and revenues.) Figure 1: Total Expenses, Revenue and Deficit, National Transportation Safety Board Training Center, Fiscal Years 2004 through 2012. [Refer to PDF for image: multiple line graph] Fiscal year: 2004; Total Revenue: $258,760; Total Expenses: $6.5 million; Deficit: $6.3 million. Fiscal year: 2005; Total Revenue: $634,800; Total Expenses: $4.5 million; Deficit: $3.9 million. Fiscal year: 2006; Total Revenue: $651,191; Total Expenses: $3.3 million; Deficit: $ Fiscal year: 2007; Total Revenue: $817,555; Total Expenses: $3.1 million; Deficit: $2.3 million. Fiscal year: 2008; Total Revenue: $1.6 million; Total Expenses: $3.8 million; Deficit: $2.2 million. Fiscal year: 2009; Total Revenue: $2.1 million; Total Expenses: $3.8 million; Deficit: $1.7 million. Fiscal year: 2010; Total Revenue: $2.0 million; Total Expenses: $3.9 million; Deficit: $1.9 million. Fiscal year: 2011; Total Revenue: $1.9 million; Total Expenses: $4.0 million; Deficit: $2.1 million. Fiscal year: 2012; Total Revenue: $2.0 million; Total Expenses: $4.1 million; Deficit: $2.1 million. Source: NTSB. [End of figure] Recommendation Close-Out Process: The automation of NTSB's recommendation follow-up process has reduced the amount of time it takes to formally respond to agencies about whether planned actions to implement an NTSB recommendation are acceptable.[Footnote 13] In fiscal year 2010, NTSB deployed the previously described CNS to manage the Board's correspondence, including accident reports, safety studies, recommendation transmittals, and public notice responses. CNS allows for the relevant modal offices and the Research and Engineering Office to simultaneously review and assess planned actions to address NTSB recommendations. According to NTSB officials, prior to the implementation of CNS, the average time NTSB took to respond to an agency's proposals to address an NTSB recommendation was 216 days. After CNS was implemented, that figure dropped to 115 days--a reduction of 47 percent. At the same time, the number of responses the agency put out each quarter also increased. (See figure 2.) NTSB officials have indicated that they have an internal goal to further reduce the response time to 90 days on average. Figure 2: National Transportation Safety Board's Average Response Time and Number of Responses to Agencies' Recommendation Correspondence, by Quarter, Fiscal Year 2009 through First Quarter, Fiscal Year 2013: [Refer to PDF for image: vertical bar and line graph] Fiscal year and quarter: 2009 Q1; Responses: 41; Average response time: 263 days. Fiscal year and quarter: 2009 Q2; Responses: 25; Average response time: 210 days. Fiscal year and quarter: 2009 Q3; Responses: 36; Average response time: 193 days. Fiscal year and quarter: 2009 Q4; Responses: 45; Average response time: 199 days. Fiscal year and quarter: 2010 Q1; Responses: 53; Average response time: 172 days. Fiscal year and quarter: 2010 Q2; Responses: 44; Average response time: 109 days. Fiscal year and quarter: 2010 Q3; Responses: 71; Average response time: 79 days. Fiscal year and quarter: 2010 Q4; Responses: 42; Average response time: 66 days. Fiscal year and quarter: 2011 Q1; Responses: 49; Average response time: 159 days. Fiscal year and quarter: 2011 Q2; Responses: 119; Average response time: 153 days. Fiscal year and quarter: 2011 Q3; Responses: 82; Average response time: 131 days. Fiscal year and quarter: 2011 Q4; Responses: 59; Average response time: 111 days. Fiscal year and quarter: 2012 Q1; Responses: 54; Average response time: 87 days. Fiscal year and quarter: 2012 Q2; Responses: 82; Average response time: 85 days. Fiscal year and quarter: 2012 Q3; Responses: 72; Average response time: 118 days. Fiscal year and quarter: 2012 Q4; Responses: 119; Average response time: 139 days. Fiscal year and quarter: 2013 Q1; Responses: 84; Average response time: 89 days. Source: NTSB. Note: According to NTSB officials, the rise in response times in the first quarter of 2011 is associated with a significant increase in the number of letters prepared, most of which were part of an existing backlog. In addition, in fiscal year 2012, a temporary 20 percent decrease in staff in the Safety Recommendation and Quality Assurance Division contributed to longer response times, which improved in the first quarter of fiscal year 2013 with a full complement of staff. [End of figure] Communication: Our analysis of improvements in NTSB's employee and management communication related to NTSB's efforts indicated uneven results; specifically we observed improvements in some but not all measures. We reviewed NTSB employees' responses to the three federal survey questions we determined related to employees' perceptions about managers' communication, as described below: * Managers communicate the goals and priorities of the organization. NTSB respondents increased their positive responses to this question, from about 49 percent in 2004 to 57 percent in 2012. (See fig. 3.) We compared NTSB employees' responses with employees in a group of small federal and independent agencies[Footnote 14] and found that NTSB employees' satisfaction level increased while the proportion of employees from small agencies responding positively to this question during the same period was relatively unchanged from 57 percent in 2004 to 59 percent in 2012. * How satisfied are you with the information you receive from management on what's going on in your organization? Responses to this question indicated an increase in the level of satisfaction, from 44 percent in 2004 to 49 percent in 2012. NTSB employees' responses were similar to the positive responses by employees from small agencies that showed an increase from 44 percent to 50 percent. * Managers promote communication among different work units (for example, about projects, goals, needed resources). The proportion of respondents reporting positive responses on this survey question from 2004 to 2012 was relatively unchanged from 48 percent to 50 percent. Similarly, there was little change in the proportion of positive responses reported by federal employees from small agencies from 50 percent in 2004 and 49 percent in 2012. Figure 3: National Transportation Safety Board Employee Responses about Communication on Selected Office of Personnel Management Survey Questions, 2004 through 2012: [Refer to PDF for image: horizontal bar graph] Managers communicate the goals and priorities of the organization: Year: 2004; Negative response: 29 Positive response[A]: 49 Year: 2006; Negative response: 27 Positive response[A]: 50 Year: 2007; Negative response: 19 Positive response[A]: 62 Year: 2008; Negative response: 13 Positive response[A]: 69 Year: 2009; Negative response: 15 Positive response[A]: 62 Year: 2010; Negative response: 15 Positive response[A]: 65 Year: 2011; Negative response: 19 Positive response[A]: 64 Year: 2012; Negative response: 24 Positive response[A]: 57 How satisfied are you with the information you receive from management on what’s going on in your organization? Year: 2004; Negative response: 33%; Positive response[A]: 44%. Year: 2006; Negative response: 31%; Positive response[A]: 43%. Year: 2007; Negative response: 22%; Positive response[A]: 59%. Year: 2008; Negative response: 24%; Positive response[A]: 53%. Year: 2009; Negative response: 22%; Positive response[A]: 55%. Year: 2010; Negative response: 24%; Positive response[A]: 52%. Year: 2011; Negative response: 22%; Positive response[A]: 56%. Year: 2012; Negative response: 23%; Positive response[A]: 49%. Managers promote communication among different work units (for example, about projects, goals, needed resources): Year: 2004; Negative response: 29%; Positive response[A]: 48%. Year: 2006; Negative response: 29%; Positive response[A]: 50%. Year: 2007[B]; Not asked. Year: 2008; Negative response: 21%; Positive response[A]: 57%. Year: 2009[B]; Not asked. Year: 2010; Negative response: 26%; Positive response[A]: 52%. Year: 2011; Negative response: 25%; Positive response[A]: 56%. Year: 2012; Negative response: 29%; Positive response[A]: 50%. Source: GAO analysis of annual survey data. [A] Positive responses are aggregates of the top two most positive responses, such as strongly agree and agree, and negative responses are aggregates of the bottom two most negative responses, such as strongly disagree and disagree. [B] OPM administered an annual employee survey in 2007 and 2009 that excluded this communication question. This survey was similar to its federal employee survey, but included fewer questions. [End of figure] NTSB officials stated that their internal communication surveys, which the agency administered 2009 through 2011, provided information that helped them identify continuing barriers to employee and management communication. In 2012, NTSB developed an action plan in this area that included detailed activities, target dates, and regular status reports. Furthermore, because of lingering concerns, NTSB continues to monitor employees' views about employee and management communication to address any remaining weaknesses. Diversity Management: Our analysis of outcomes associated with NTSB's efforts to improve its diversity management program indicated uneven results, with indications of improvements in some measures but not all. We reviewed NTSB employees' responses to the three federal survey questions that we determined related to employees' perceptions about managers' diversity and inclusiveness efforts, as described below: * My supervisor/team leader is committed to a workforce representative of all segments of society. The federal survey of NTSB employees indicated an increase in the positive responses, from 54 percent to about 71 percent. (See fig. 4.) We compared NTSB employees' responses to those of employees in a group of small federal and independent agencies and found that NTSB employees were more positive in 2012 than the employees from small agencies, whose positive responses rose from 57 percent in 2004 to 69 percent in 2012. * Policies and programs promote diversity in the workplace. Again, NTSB employees' responses to this question indicated an increase from 2004 to 2012, from 55 percent to 73 percent. NTSB employees' satisfaction level exceeded that of employees from small agencies, whose positive responses to this question stayed about the same, from 56 percent in 2004 to 57 percent in 2012. * Managers/supervisors/team leaders work well with employees of different backgrounds. The proportion of NTSB employees reporting positive responses on this survey question from 2004 to 2012 declined from 66 percent to 60 percent. The decline in NTSB employees' level of satisfaction was greater than that shown by employees from small agencies, whose positive responses also declined from 66 percent in 2004 to 63 percent in 2012. Figure 4: National Transportation Safety Board Employee Responses about Diversity and Inclusiveness on Selected Office of Personnel Management Survey Questions, 2004 through 2012: [Refer to PDF for image: horizontal bar graph] My supervisor/team leader is committed to a workforce representative of all segments of society: Year: 2004; Negative response: 12%; Positive response[A]: 54%. Year: 2006; Negative response: 12%; Positive response[A]: 52%. Year: 2007[B]; Not asked. Year: 2008; Negative response: 9%; Positive response[A]: 57%. Year: 2009[B]; Not asked. Year: 2010; Negative response: 8%; Positive response[A]: 65%. Year: 2011; Negative response: 11%; Positive response[A]: 66%. Year: 2012; Negative response: 10%; Positive response[A]: 71%. Policies and programs promote diversity in the workplace: Year: 2004; Negative response: 15%; Positive response[A]: 55%. Year: 2006; Negative response: 14%; Positive response[A]: 43%. Year: 2007[B]; Not asked. Year: 2008; Negative response: 9%; Positive response[A]: 56%. Year: 2009[B]; Not asked. Year: 2010; Negative response: 19%; Positive response[A]: 52%. Year: 2011; Negative response: 16%; Positive response[A]: 64%. Year: 2012; Negative response: 11%; Positive response[A]: 73%. Managers/supervisors/team leaders work well with employees of different backgrounds: Year: 2004; Negative response: 14%; Positive response[A]: 66%. Year: 2006; Negative response: 15%; Positive response[A]: 61%. Year: 2007[B]; Negative response: 13%; Positive response[A]: 63%. Year: 2008; Negative response: 10%; Positive response[A]: 66%. Year: 2009[B]; Negative response: 12%; Positive response[A]: 67%. Year: 2010; Negative response: 13%; Positive response[A]: 60%. Year: 2011; Negative response: 16%; Positive response[A]: 62%. Year: 2012; Negative response: 16%; Positive response[A]: 60%. Source: GAO analysis of annual survey data. [A] Positive responses are aggregates of the top two most positive responses, such as strongly agree and agree, and negative responses are aggregates of the bottom two most negative responses, such as strongly disagree and disagree. [B] OPM administered an annual employee survey in 2007 and 2009 that excluded two of the three diversity questions. The annual employee survey is similar to OPM's federal employee survey, but included fewer questions. [End of figure] One of the potential outcomes of a robust diversity management program is an increase in the diversity of the workforce. Based on our analysis of NTSB's workforce diversity data, we found that the proportion of white employees in NTSB's workforce declined from 77 percent in 2008 (289 employees) to 73 percent (293 employees) in 2012. (See table 1.) NTSB's total workforce increased 6 percent over the same period, from 378 in 2008 to 402 in fiscal year 2012. The proportion of women remained roughly the same at about 40 percent, as did the proportion of African American employees at 17 percent, and Hispanic employees at 2 percent of the total NTSB workforce. Conversely, NTSB increased the number of employees who are Native American although these employees represent only 2 percent of the overall workforce. We compared these figures to those representing comparative groups in the civilian labor force and found that NTSB's labor force had a larger proportion of some minority groups (e.g., African American) and smaller proportion of other groups (e.g., Hispanic) than the civilian labor force. Roughly half of NTSB's workforce performs investigations and investigation-related work--work directly related to NTSB's core mission. In 2008 there were 190 investigators and related staff; in 2010, there were 198, and in 2012 there were 206. Based on our analysis, we found that the proportion of investigator and investigation-related staff that were white was about 90 percent over the period of 2008 to 2012 and the proportion of women was about 19 percent. We compared these figures to those representing comparative groups in the civilian labor force and found that NTSB's investigator and investigation-related workforce had a smaller proportion of minority and other groups, including African American, Asian, Hispanic, and women than the civilian labor force. In addition, NTSB reported that from fiscal years 2008 to 2012, it had no minority group members among its 16 senior executives although the number of women increased from 3 to 4.[Footnote 15] Table 1: Profile of the National Transportation Safety Board Workforce by Race, Gender, and National Origin, 2008 through 2012: Year: 2008: All NTSB[B]: African American: 17%; Native American[A]:1%; Asian: 4%; Hispanic: 2%; White: 77%; Women: 39%; Men: 61%. Investigator and related staff: African American: 3%; Native American[A]: 1%; Asian: 3%; Hispanic: 3%; White: 90%; Women: 17%; Men: 83%. Civilian labor force [C]: African American: 12%; Native American[A]: NA; Asian: 5%; Hispanic: 14%; White: 81%; Women: 47%; Men: 53%. Year: 2010. All NTSB: African American: 17%; Native American[A]: 1%; Asian: 4%; Hispanic: 2%; White: 75%; Women: 41%; Men: 59%. Investigator and related staff: African American: 3%; Native American[A]: 1%; Asian: 3%; Hispanic: 3%; White: 90%; Women: 19%; Men: 81%. Civilian labor force[C]: African American: 12%; Native American[A]: NA; Asian: 5%; Hispanic: 15%; White: 81%; Women: 47%; Men: 53%. Year: 2012: All NTSB: African American: 17%; Native American[A]: 2%; Asian: 5%; Hispanic: 2%; White: 73%; Women: 40%; Men: 60%. Investigator and related staff: African American: 2%; Native American[A]: 3%; Asian: 3%; Hispanic: <1%; White: 89%; Women: 18%; Men: 82%. Civilian labor force[C]: African American: 12%; Native American[A]: NA; Asian: 5%; Hispanic: 16%; White: 80%; Women: 47%; Men: 53%. Source: NTSB and BLS data. [A] Beginning in 2003, BLS stopped reporting separately on persons in selected race categories, such as Native Americans or those describing themselves as 2 or more races, because the number was too small to develop sufficient reporting estimates. [B] In 2010, NTSB began reporting the proportion of its workforce identifying themselves as 2 or more races. However, to be consistent with the BLS data, we did not include this information in this table. [C] These data represent the civilian labor force as reported by BLS in its annual survey. According to OPM, the relevant civilian labor force is the benchmark used to measure federal agencies' progress toward eliminating underrepresentation of minority groups and women. [End of table] Despite its efforts, NTSB has not been able to appreciably change its diversity profile for minority group members and women. However, as mentioned previously, NTSB has taken steps to implement initiatives as a result of its diversity management strategy, including its recently completed diversity and inclusiveness survey, which the agency plans to use to identify gaps in its diversity and inclusiveness efforts and to benchmark future progress. It is too soon to tell whether initiatives, such as its recruitment strategies, will lead to additional changes in its workforce diversity profile. Financial Management: We were unable to determine if NTSB's cost accounting system had improved the agency's ability to make operational decisions because it has not yet fully utilized the system for its intended purposes. For the implementation of a cost accounting system to be effective, it must be tailored to the needs of the organization, be a tool managers can use to make everyday decisions, and be based on sound data that captures time spent on all activities, such as investigations and training. In 2011, NTSB implemented a cost accounting system that includes a time and attendance program in response to a GAO recommendation. This program allows an investigator to assign his or her time to specific investigations through a series of codes, allowing NTSB to assess the cost of conducting investigations rather than simply tracking and managing a budget. NTSB officials stated that the time and attendance system has allowed it to obtain information about the cost of investigations more efficiently than its prior method. In a May 2011 advisory, NTSB management envisioned that the cost accounting system would enable NTSB to measure and compare performance with other organizations, and the data from the system would help the agency monitor and improve productivity and mission effectiveness by better utilizing personnel resources. However, officials provided no time frame for when the data might be used by management for making resource and operational decisions. NTSB officials stated that they are currently focused on ensuring the quality of the time and attendance data before developing goals, targets, and management tools or using such information to make resource or operational decisions. While ensuring the quality of data is a necessary step in fully implementing a cost accounting system, it has been over 2 years since NTSB first began collecting time and attendance data to establish the cost of conducting investigations. As a result, NTSB is not using the system's full capabilities. Thus, NTSB has not yet fully achieved its vision of using the data to improve labor productivity and mission effectiveness. Conclusion: NTSB has implemented a cost accounting system, but effective utilization is required to achieve the long-term rewards of those efforts. Although NTSB has been collecting data from this system to account for costs of investigations, it has not yet developed a management strategy that would allow the agency to maximize the utility of the cost accounting system. This has prevented NTSB from using that information to make the decisions necessary to better manage its labor resources. Without fully utilizing the cost accounting system, NTSB will not achieve the intended benefits of improving labor productivity and mission effectiveness. Recommendation: NTSB needs to continue its improvement efforts in each of the five areas discussed in this report. Further, to improve financial management and provide information to managers for operational decisions, we recommend that the Chairman of the NTSB direct senior management to develop a strategy for maximizing the utility of NTSB's cost accounting system. Agency Comments: We are sending copies of this report to the appropriate Congressional Committees and to the Chairman of the National Transportation Safety Board. In addition, the report is available at no charge on the GAO website at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report, please contact me at (202) 512-2834. Contact points for Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix II. Signed by: Gerald L. Dillingham, Ph.D. Director, Physical Infrastructure Issues: [End of section] Appendix I: Objective, Scope, and Methodology: Our objective in this review was to assess whether there have been management and operational improvements associated with NTSB's actions in areas where GAO had conducted previous work and made recommendations. In that previous work, GAO made 21 recommendations within 12 management and operational areas. [Footnote 16] Some areas have more than one recommendation. For example, the communication area has two related recommendations; the first called for NTSB to develop mechanisms to facilitate communication from staff to management and the other called for NTSB to report to Congress on the status of the GAO recommendations. To determine whether there were any associated improvements based on NTSB's actions, we first identified a subset of the 12 original management and operational areas that were included in NTSB's 2013- 2016 Strategic Plan. Being highlighted in NTSB's current strategic plan, indicates that these areas were relevant and important areas for the agency. We identified 7 of the 12 original management and operational areas in NTSB's current strategic plan. These 7 areas contained 13 of the 21 original recommendations. Many of the 13 recommendations within these areas required only a single action by NTSB to implement (e.g. develop a strategic plan that follows performance based practices, articulate risk-based criteria for selecting which accidents to investigate, or obtain authority to use appropriations to make lease payments in order to correct its violation of the Anti-Deficiency Act) while other of those recommendations involved strategies for improvement that lend themselves to continual action and monitoring overtime in order to achieve desired improvements (e.g. increase utilization of the training center, and improve the recommendation close out process). It is these latter types of recommendations that we focused on in this review, and based on these criteria, we identified 5 for review: * increase the utilization of the Training Center, * improve the process for changing the status of recommendations through computerization and concurrent review, * develop mechanisms to facilitate communication from staff to management, * develop strategies for diversity management as part of the human capital program, and: * develop a full cost accounting system to track time employees spend on each investigation in training. We identified a sixth recommendation--maximize the delivery of core curriculum for each mode at the Training Center--based on our selection criteria. However, we did not include this recommendation in our review because NTSB lacked information about employee training and we could not develop an outcome measure without such data. To identify the outcome measures to assess changes associated with NTSB's actions, we used information from prior GAO reports, information from NTSB including strategic plans, workforce reports, and financial reports as well as information gathered from interviews with NTSB officials. The measures we identified for each recommendation are: (1) Training Center utilization--utilization of classroom and non-classroom space and operating deficit; (2) recommendation close-out process--average time to respond to agency proposals; (3) employee and management communication--employee responses to Office of Personnel Management's (OPM) federal employee surveys;[Footnote 17] (4) diversity management--employee responses to OPM's federal employee surveys and NTSB employment levels of women and members of racial and ethnic groups; and (5) financial management-- cost accounting reports used to measure performance. To measure whether there was improvement in the outcomes and results associated with NTSB's actions, we compared current conditions with the initial conditions at the time we performed our previous work or earlier in some instances in order to establish a baseline before actions occurred. We used NTSB's financial and program data, employee survey data from OPM, and workforce data from NTSB and the Bureau of Labor and Statistics (BLS). To ensure the data used were of sufficient reliability for our analysis, we examined program reporting procedures and quality assurance controls, and discussed various data elements with knowledgeable agency officials. We also spoke with NTSB officials who were knowledgeable about operations and management in these five selected areas. [End of section] Appendix II: GAO Contact and Staff Acknowledgments: National Transportation Safety Board: Office of the Chairman: Washington, DC 20594: July 10, 2013: The Honorable Gene L. Dodaro: Comptroller General of the United States: Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Mr. Dodaro: Thank you for the opportunity to respond to your draft report, entitled National Transportation Safety Board Management and Operational Improvements Found, but Strategy Needed to Utilize Cost Accounting System (GAO-13-611). We thank you for acknowledging the management improvements that have been made by the NTSB. We believe that the Government Accountability Office's (GAO) recommendation that the "Chairman of the NTSB direct senior management to develop a strategy for maximizing the utility of NTSB's cost accounting system" will assist in our ongoing efforts to improve our labor cost accounting system, and to help ensure that we execute our important mission efficiently and effectively. We value our ongoing constructive relationship with the GAO and we request an informal meeting with appropriate GAO staff to discuss this recommendation and learn about any cost accounting best practices that GAO uses or is aware that other agencies use. In addition, we would like to thank Dr. Gerald Dillingham and the GAO Physical Infrastructure Team who have worked diligently and constructively on our engagements. If you have any questions, please contact Dr. David Mayer, NTSB Managing Director, at (202) 314-6060. Should you should have questions or if we can assist in the future, please contact us. Sincerely, Signed by: Deborah A. P. Hersman: Chairman: [End of section] Appendix III: GAO Contact and Staff Acknowledgments: GAO Contact: Gerald L. Dillingham, Ph.D. (202) 512-2834 or dillinghamg@gao.gov: Staff Acknowledgments: In addition to the contact named above, the following individuals made key contributions to this report: H. Brandon Haller (Assistant Director), Christopher Jones, Gail Marnik, Josh Ormond, and Amy Rosewarne. [End of section] Related GAO Products: National Transportation Safety Board: Implementation of GAO Recommendations. [hyperlink, http://www.gao.gov/products/GAO-12-306R]. Washington, D.C.: January 6, 2012: National Transportation Safety Board: Issues related to the 2010 Reauthorization. [hyperlink, http://www.gao.gov/products/GAO-10-366T]. Washington, D.C.: January 27, 2010: National Transportation Safety Board: Reauthorization Provides an Opportunity to Focus on Implementing Leading Management Practices and Addressing Human Capital and Training Center Issues. [hyperlink, http://www.gao.gov/products/GAO-10-183T]. Washington, D.C.: October 29, 2009: National Transportation Safety Board: Progress Made in Management Practices, Investigation Priorities, Training Center Use, and Information Security, But These Areas Continue to Need Improvement. [hyperlink, http://www.gao.gov/products/GAO-08-652T]. Washington, D.C.: April 23, 2008: National Transportation Safety Board: Observations on the Draft Business Plan for NTSB's Training Center. [hyperlink, http://www.gao.gov/products/GAO-07-886R]. Washington, D.C.: June 14, 2007: National Transportation Safety Board: Progress Made, Yet Management Practices, Investigation Priorities, and Training Center Use Should Be Improved. [hyperlink, http://www.gao.gov/products/GAO-07-118]. Washington D.C.: November 22, 2006. National Transportation Safety Board: Preliminary Observations on the Value of Comprehensive Planning, and Greater Use of Leading Practices and the Training Academy. [hyperlink, http://www.gao.gov/products/GAO-06-801T]. Washington, D.C.: May 24, 2006: [End of section] Footnotes: [1] The National Transportation Safety Board Reauthorization Act of 2006, Pub. L. No. 109-443, § 5, 120 Stat. 3297, 3299, codified at 49 U.S.C. § 1138. [2] GAO, National Transportation Safety Board: Preliminary Observations on the Value of Comprehensive Planning, and Greater Use of Leading Practices and the Training Academy. [hyperlink, http://www.gao.gov/products/GAO-06-801T] (Washington, D.C.: May 24, 2006). [3] GAO, National Transportation Safety Board: Preliminary Observations on the Value of Comprehensive Planning, and Greater Use of Leading Practices and the Training Academy. [hyperlink, http://www.gao.gov/products/GAO-06-801T] (Washington, D.C.: May 24, 2006); GAO, National Transportation Safety Board: Progress Made, Yet Management Practices, Investigation Priorities, and Training Center Use Should Be Improved. [hyperlink, http://www.gao.gov/products/GAO-07-118] (Washington, D.C.: Nov. 22, 2006); and GAO, National Transportation Safety Board: Progress Made in Management Practices, Investigation Priorities, Training Center Use, and Information Security, But These Areas Continue to Need Improvement. [hyperlink, http://www.gao.gov/products/GAO-08-652T] (Washington, D.C.: April 23, 2008). [4] GAO, National Transportation Safety Board's Implementation of GAO Recommendations. [hyperlink, http://www.gao.gov/products/GAO-12-306R] (Washington, D.C.: Jan. 6, 2012). [5] Department of Transportation Act, Pub.L.No. 89-670, § 5, 80 Stat. 931 (1966). [6] Independent Safety Board Act of 1974, Pub.L.No. 93-633, Title III, 88 Stat. 2156 (1974). [7] NTSB's regulations state that NTSB is responsible for the organization, conduct, and control of all aviation accident and incident investigations in the United States involving any civilian or certain public aircraft. 49 C.F.R. § 831.2. [8] A regional office for aviation safety is also located at the Training Center. [9] Pursuant to 49 U.S.C. 1113(b)(1)(I), NTSB has statutory authority to enter into agreements for the provision of facilities, accident- related and technical services, or training in accident investigation theory and techniques, and require such entities to provide appropriate consideration for the reasonable costs of any facilities, goods, services, or training provided by the NTSB. [10] S. Rpt. 109-109, at 235 (2005), accompanying the Senate Transportation, Treasury, Housing and Urban Development, and Agencies Appropriations bill for fiscal year 2006. [11] GAO, Executive Guide: Creating Value through World-Class Financial Management, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-134] (Washington, D.C.: April 2000). [12] GAO, National Transportation Safety Board: Progress Made in Management Practices, Investigation Priorities, Training Center Use, and Information Security, But These Areas Continue to Need Improvement, [hyperlink, http://www.gao.gov/products/GAO-08-652T] (Washington, D.C.: Apr. 23, 2008). [13] As noted above, NTSB also issues recommendations to immediately address safety deficiencies found during an investigation before a report is issued. [14] As part of its federal employee survey report, OPM reports the results of 45 small and independent agencies. This group would be comparable to NTSB in terms of size since the agencies that fall within this group have less than 800 employees. [15] NTSB's senior executives went from 16 to 15 from fiscal years 2008 to 2012. NTSB board members are not included in this group since, as political appointees, they are part of another pay plan. [16] GAO, National Transportation Safety Board: Preliminary Observations on the Value of Comprehensive Planning, and Greater Use of Leading Practices and the Training Academy. [hyperlink, http://www.gao.gov/products/GAO-06-801T] (Washington, D.C.: May 24, 2006); GAO, National Transportation Safety Board: Progress Made, Yet Management Practices, Investigation Priorities, and Training Center Use Should Be Improved. [hyperlink, http://www.gao.gov/products/GAO-07-118] (Washington, D.C.: Nov. 22, 2006); and GAO, National Transportation Safety Board: Progress Made in Management Practices, Investigation Priorities, Training Center Use, and Information Security, But These Areas Continue to Need Improvement. [hyperlink, http://www.gao.gov/products/GAO-08-652T] (Washington, D.C.: Apr. 23, 2008) [17] OPM weights the data it collects from the federal employee viewpoint survey respondents to produce survey estimates that accurately represent the survey population. For the period 2004 to 2012, NTSB employees' response rate varied year to year from 70 percent in 2004 to 66 percent in 2012. In comparison, the government- wide response rate to OPM's federal survey was 46.1 percent in 2012. [End of section] GAO’s Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select “E-mail Updates.” Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]