Military Education: Additional DOD Guidance Is Needed to Enhance Oversight of the Service Academies and Their Preparatory Schools

What GAO Found

While DOD, collectively, has partially implemented each of our four recommendations regarding the service academies and their preparatory schools, the actions taken by the services vary significantly. In addition, OUSD (P&R) has not yet updated its guidance for the academies and preparatory schools to address our 2003 recommendations. The text below summarizes the four recommendations we made in September 2003 regarding the academies and their preparatory schools and the status of DOD’s implementation.

Recommendation regarding the service academies:

Recommendation: Further enhance performance goals and measures whereby the information required in annual assessment reports can be better evaluated;

Status of implementation and actions taken: Partially implemented: The Air Force and the Navy have established performance goals and measures for their respective academies, but the Army has not.

Performance goals and quantifiable measures are key components of a complete oversight framework; they allow an agency to track its progress in achieving intended results. Assessing performance against stated goals and quantifiable measures can help inform management decisions about such issues as the need to redirect resources or shift priorities. In our 2003 report on the academies, we noted that although OUSD (P&R) and the services used the number and type of commissioned officers as the primary means of measuring the military academies’ performance, there were few stated goals against which to assess that performance. We noted that OUSD (P&R) required the services to provide it with information on current and past performance for academy operations, but we also observed that apart from officer accession goals—that is, the number and types of commissioned officers—neither OUSD (P&R) nor the services had specific, stated performance goals against which to judge the adequacy of the academies’ performance. Consequently, we recommended that DOD improve its oversight of the operations and performance of the academies through the enhancement of performance goals and measures.

Since we issued our report in 2003, the Air Force and the Navy have established performance goals and measures for their academies, while the Army has not done so. In its concurrence with our recommendation in 2003, OUSD (P&R) stated that it would monitor the development of improved goals and measures by the academies. In its written response to our questions for the present review, OUSD (P&R) stated that following issuance of our reports in 2003, it had convened a working group that included representatives from the services to begin addressing our recommendations, and that the services had agreed to develop ideas to implement our recommendation regarding the academies. In 2005, the Air Force issued guidance establishing a performance measurement program for the Air Force Academy designed to provide insight into the academy’s admissions and general governance. As part of its program the Air Force identified numerous performance measurement indicators, such as those for attrition and disenrollments, and the Air Force has collected, analyzed, and reported data on these performance measures each subsequent academic year. In 2007, the Navy established as a goal for midshipmen entering the Naval Academy as members of the class of 2013 that a minimum of 65 percent of midshipmen choosing a Navy commission would complete a technical degree program—that is, major in science, technology, engineering, or mathematics—before receiving their commission (graduating). However, apart from officer accession goals on which we reported in 2003, the Army has not established performance goals for the U.S. Military Academy that meet the intent of our recommendation.

Recommendations regarding the preparatory schools:

Recommendation: Clarify the preparatory schools' mission statements by aligning these statements with the department's guidance and the academies' expectations;

Status of implementation and actions taken: Partially implemented: The Air Force has clarified the mission statement of its preparatory school, but the Army and Navy have not.

Mission statements should define an organization’s purpose in language that states desired outcomes. In our 2003 report on the preparatory schools, we noted that the preparatory schools’ mission statements did not clearly articulate the purpose for which the schools were being used by the academies. As we observed in that report, the academies expected that the preparatory schools would enhance diversity at the academies and that the preparatory schools existed to help the academies meet their diversity needs. The preparatory schools’ mission statements, however, did not explicitly state this purpose and referred instead to preparing “selected personnel who meet special needs,” “selected candidates,” or “candidates” for admission to and success at the academies. This omission represents a continuing issue, which we first reported in 1992. Without transparency in their mission statements, the academies and their respective preparatory schools were unable to establish goals that fully reflected the preparatory schools’ intended purpose. Therefore, we recommended that OUSD (P&R) clarify the preparatory schools’ mission statements by aligning these statements with the department’s guidance and the academies’ expectations. DOD concurred with this recommendation in its response to our report.

However, only the Air Force has clarified the mission statement for its preparatory school. In 2006, the Air Force revised its statement to indicate that one of the school’s missions was to enhance diversity at the Air Force Academy. Although both the Army and the Navy revised the mission statements for their preparatory schools since our 2003 review, these revisions simply capture what students who attend them can expect of the schools. The Army and the Navy have not clarified the mission statements of their preparatory schools, as our recommendation intended, in a manner that articulates the purpose for which these two schools are being used by their respective academies.

Recommendation: Establish performance goals and quantifiable measures, linked with the schools' mission statements;

Status of implementation and actions taken: Partially implemented: OUSD (P&R) has not established performance goals and quantifiable measures across the services; the Air Force, however, has taken independent action that is consistent with the intent our recommendation.

A complete oversight framework includes tracking achievements in comparison with plans, goals, and objectives and analyzing the differences between planned results and actual performance. In our 2003 report on the preparatory schools, we noted that OUSD (P&R) and the services received annual reports from the academies on preparatory school performance, but that without performance goals and measures, the reports were limited in the information they were able to provide OUSD (P&R), the services, or the academies on the preparatory schools’ performance. Without specific performance goals, there was no objective yardstick against which to gauge preparatory school effectiveness. We consequently recommended that DOD establish performance goals and quantifiable measures linked with the schools’ mission statements. DOD concurred with this recommendation in its response to our report.

However, OUSD (P&R) has not established performance goals and quantifiable measures across the services, though we note that the Air Force has taken independent action that is consistent with the intent of our recommendation. In 2004, the DOD working group we discussed earlier met and developed two performance goals and quantifiable measures for the preparatory schools. These measures and goals are as follows:

• Preparatory school-to-academy entrance ratio: The ratio of the number of preparatory school students entering an academy to the number that entered the preparatory school for that academy should be greater than 70 percent.
• Preparatory student versus direct appointee graduation rate: Preparatory school students’ academy graduation rate will not lag behind the graduation rate for those students who were directly appointed to the academy by more than 5 percent.

Recommendation: Enhance the existing oversight framework by using performance goals and quantifiable measures to objectively evaluate the performance of the preparatory schools;

Status of implementation and actions taken: Partially implemented: The Air Force is using performance goals and quantifiable measures to evaluate the performance of its preparatory school, but the Army and Navy are not.

Performance goals and quantifiable measures constitute key components of a complete oversight framework. In our 2003 report on the preparatory schools, we noted that the academies exercised direct oversight of their respective preparatory schools and monitored the schools’ performance. We also observed, however, that it was difficult to evaluate how effective the preparatory schools had been in accomplishing their missions because the academies were not using performance goals and quantifiable measures to evaluate the performance of the schools. Without using goals that were linked to clear mission statements, OUSD (P&R), the service headquarters, and the academies did not have an objective basis by which to judge the effectiveness of the preparatory schools’ performance. We consequently recommended that OUSD (P&R) enhance the existing oversight framework by using performance goals and quantifiable measures to objectively evaluate the performance of the preparatory schools. DOD concurred with this recommendation in its response to our report and stated that it would revise DOD Directive 1322.22.

However, only the Air Force has used the performance goals and quantifiable measures that the DOD working group developed to evaluate the performance of its preparatory school—the Army and Navy have not. In 2004, DOD’s working group agreed that each of the services would evaluate the performance of its respective preparatory school using the two performance measures that the group developed. The Air Force has collected, analyzed, and reported information on these goals and measures as part of its performance measurement program since 2005. The Air Force has also issued semiannual reports to Air Force leadership that includes information for these quantified measures, as well as evaluations of the performance of its preparatory school against the performance goals. In contrast, while the Army and the Navy monitor data for the quantifiable measures—for example, the Army and the Navy have provided information regarding these measures in their reports to OUSD (P&R)—neither evaluates the performance of its preparatory school against the performance goals that the working group established. OUSD (P&R) stated in its response to our 2003 report that it would revise DOD guidance to address this recommendation. However, OUSD (P&R)’s current draft guidance does not require the services to use performance goals and quantifiable measures to objectively evaluate the performance of the preparatory schools. Without specifying its expectations and requirements for using performance goals and quantifiable measures to assess the performance of the preparatory schools, the department will continue to lack an objective basis with which to judge the effectiveness of the preparatory schools in accomplishing their mission.

Why GAO Did This Study

The Departments of the Army, the Navy, and the Air Force each have their own service academies that produce a portion of the officer corps for the nation’s armed forces. Each academy also operates its own preparatory school, which exists to prepare select students for admission to the academies. The Department of Defense (DOD) invests a significant amount of taxpayer dollars in these institutions. According to the most recently available DOD information, the academies and preparatory schools cost more than $1.6 billion to operate during fiscal year 2010. To ensure the best value for the taxpayers’ investment in the academies and preparatory schools, effective management principles are critical. These principles include establishing a complete oversight framework, which includes performance goals and quantifiable measures that are linked to clear statements of the organization’s mission.

Over the past two decades, we have issued many products to help inform congressional oversight of the academies and preparatory schools in response to congressional inquiries regarding these institutions. In 2002, the House of Representatives report on defense appropriations for fiscal year 2003 directed us to review DOD’s oversight of the academies and the preparatory schools, and in September 2003 we issued two reports, containing four recommendations. In our report on the academies, we recommended that DOD enhance its performance goals and measures to improve oversight of the academies’ operations and performance. In our report on the preparatory schools, we recommended that DOD

(1) clarify the mission statements for the preparatory schools, (2) establish performance goals and quantifiable measures for the schools, and (3) use performance goals and quantifiable measures to objectively evaluate the performance of the schools. DOD concurred with each of the recommendations we made in these two reports. In 2011, the Senate Armed Services Committee directed us to assess the extent to which DOD has implemented the recommendations contained in our September 2003 reports.

To determine the extent to which DOD has implemented our September 2003 recommendations, we assessed whether each recommendation was implemented, partially implemented, or not implemented. To arrive at these assessments we used the following criteria:

• Implemented: All actions necessary to implement the recommendation were completed.

• Partially implemented: Some, but not all, actions necessary to implement the recommendation were completed.

• Not implemented: Minimal or no actions necessary to implement the recommendation were completed.

For more information, please contact Brenda Farrell at (202) 512-3604 or farrellb@gao.gov.