This is the accessible text file for GAO report number GAO-10-412 entitled 'Foreign Medical Schools: Education Should Improve Monitoring of Schools That Participate in the Federal Student Loan Program' which was released on June 28, 2010. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Committees: United States Government Accountability Office: GAO: June 2010: Foreign Medical Schools: Education Should Improve Monitoring of Schools That Participate in the Federal Student Loan Program: GAO-10-412: Contents: Letter: Agency Comments and Our Evaluation: Appendix I: Briefing Slides: Appendix II: Objectives, Scope, and Methodology: Appendix III: Comparison of Average Cost of Attendance of U.S. and Selected Foreign Medical Schools: Appendix IV: Performance on Licensing Exam by Number of Attempts on Each Exam Step, and by Nationality of IMG: Appendix V: International Medical Graduates and Residencies: Appendix VI: Summary of Responses by Focus Group Participants at the Five Foreign Medical Schools We Visited: Appendix VII: Comments from the U.S. Department of Education: Appendix VIII: GAO Contact and Staff Acknowledgments: Bibliography: Related GAO Products: Tables: Table 1: Residency Programs by State, Academic Year 2008-2009: Table 2: Number of Participants in Student Focus Groups, by School: Table 3: Decision to Attend a Foreign Medical School: Figures: Figure 1: Licensing Exam Pass Rates by Number of Exam Attempts on Each Exam Step, 1998 through 2008: Figure 2: Comparison of First-Time Pass Rates for U.S. Citizen IMGs and Foreign IMGs on Step 1 (Basic Science) and Step 2 Clinical Knowledge, 1998 through 2008: Figure 3: Comparison of First-Time Pass Rates for Both U.S. Citizen IMGs and Foreign IMGs on Step 2 Clinical Skills, 2004 through 2008, and Step 3 (on Delivery of General Medical Care), 1998 through 2008: Figure 4: Percentage of IMGs in Residencies as a Percentage of All Residents by State, Academic Year 2008-2009: Abbreviations: AAMC: Association of American Medical Colleges: ACGME: Accreditation Council for Graduate Medical Education: AMA: American Medical Association: ECFMG: Educational Commission for Foreign Medical Graduates: FFEL: Federal Family Education Loans: FSMB: Federation of State Medical Boards: HEAL: Health Education Assistance Loan: HHS: Health and Human Services: IMG: International Medical Graduate: MCAT: Medical College Admission Test: NBME: National Board of Medical Examiners: OB/GYN: Obstetrics and Gynecology: USMLE: United States Medical Licensing Exam: [End of section] United States Government Accountability Office: Washington, DC 20548: June 28, 2010: The Honorable Tom Harkin: Chairman: The Honorable Michael B. Enzi: Ranking Member: Committee on Health, Education, Labor, and Pensions: United States Senate: The Honorable George Miller: Chairman: The Honorable John P. Kline: Ranking Member: Committee on Education and Labor: House of Representatives: Each year, the federal government makes a significant financial investment in the education and training of the U.S. physician workforce. A quarter of that physician workforce is composed of international medical graduates (IMG) and they include both U.S. citizens and foreign nationals. In fiscal year 2008, the federal government loaned $633 million to U.S. students enrolled in foreign institutions--including medical students--through the Federal Family Education Loan (FFEL) program.[Footnote 1] The government also makes a substantial domestic investment in the graduate training of the physician workforce. For example, in fiscal year 2008, federal support for residency training in the United States amounted to nearly $9 billion. As with medical students educated in the United States, this training is required of all IMGs--U.S. citizens and foreign nationals alike--who seek to practice medicine without supervision in the United States. The Department of Education (Education), which administers the federal student loan program, must also monitor foreign schools that seek to participate in the program with respect to specific statutory requirements. Among these is the statutory requirement that at least 60 percent of their students who take the U.S. medical licensing exam must pass the exam. Most recently, Congress increased the pass rate to 75 percent, effective July 2010. Little is known about IMGs with respect to how much they borrow overall, or the outcome of their medical studies, leading some policy makers to question the federal return on investment in IMGs. Therefore, Congress mandated that GAO study the performance of IMGs educated at these schools and other aspects of a foreign medical education, including the potential effect of the new 75 percent pass rate requirement on school participation in the federal loan program. [Footnote 2] This report examines the following questions: 1. What amount of federal student aid loan dollars has been awarded to U.S. students attending foreign medical schools? 2. What do the data show about the pass rates of international medical graduates on license examinations? 3. To what extent does Education monitor foreign medical schools' compliance with the pass rate required to participate in the federal student loan program? 4. What is known about schools' performance with regard to the institutional pass rate requirement? 5. What is known about where international medical graduates have obtained residencies in the United States and the types of medicine they practice? 6. What is known about discipline and malpractice involving foreign- educated physicians? On May 26 and 27, 2010, we briefed your staff on the final results of our analysis in addition to providing interim updates in February and March 2010. This report formally conveys the information provided during the briefing. (See app. I for the briefing slides.) In summary, we found the following: * From 1998 to 2008, U.S. students enrolled at foreign medical schools borrowed $1.5 billion in FFEL loans to attend free-standing medical schools.[Footnote 3] Although this amount represents less than 1 percent of all federal student loans borrowed during this period, borrowing has grown significantly, in part because of increases in tuition, student enrollments, and the availability of additional loan funds for graduate and professional students.[Footnote 4] Although our results are not generalizable, some students who participated in our focus groups estimated that their student loan debt would range from about $90,000 to $250,000 for their medical degree alone.[Footnote 5] In addition, some student borrowers stated that they lack reliable cost and performance information about foreign medical schools. * IMGs, as a group, have consistently passed their medical licensing exam at lower rates over the past decade than their U.S.-educated peers, but have narrowed this performance gap for most of the exam steps. In 1998, for example, average IMG pass rates on the clinical knowledge exam were 55 percent compared with 95 percent for U.S.- educated graduates. By 2008, however, IMG rates had increased to 82 percent while they remained about the same for U.S. graduates. IMGs still lag behind on the exam step for clinical skills--which involves interaction with patients--with about a 26 percentage point difference in 2008. They also required more attempts to pass the exam than their U.S.-educated counterparts. However, pass rates on the additional attempts were lower for both IMGs and U.S.-educated students compared with pass rates on initial attempts. Many factors are likely to have affected IMG pass rates, according to experts and others we interviewed, including students' proficiency in English and the extent to which foreign schools may or may not focus on preparing students for the exam. * Education has not been able to fully enforce the institutional pass rate requirement needed for continued federal student loan eligibility. The three private organizations that administer each step of the exam have declined to release student scores on grounds that the data are proprietary in nature and should not be used for marketing purposes. As a result, Education reviews pass rates only when a school applies for the program, when it periodically seeks recertification, or when there is a change in ownership. More recently, however, two of the three testing organizations have begun negotiating with individual schools for the release of aggregate student performance data. On the basis of this development, Education officials told us that the department now plans to require pass rate data annually from all foreign medical schools participating in the federal loan program. * Our own analysis of 2008 pass rate data of institutions located in countries that participate in the federal loan program indicates that while a majority of foreign medical schools in these countries met the current 60 percent student pass rate requirement, very few--11 percent- -would likely meet the newly required 75 percent pass rate. Meanwhile, officials from the three testing organizations cautioned against associating student performance on the U.S. medical licensing exam with institutional quality, given the variability among students, the fact that some schools restrict who may sit for the exam, and that other schools may encourage practice runs. * IMGs have entered into residency programs in all states, though they are concentrated in the eastern United States, and a larger proportion tend to practice in primary care than do U.S.-educated graduates. Nationwide, in academic year 2008-2009 there were 109,482 medical residents, over 30,000 of whom were IMGs (about 27 percent). Of this group of IMGs, about 78 percent were located in states east of the Mississippi, compared with 69 percent of all residents. The distribution of IMG residents by region shows the largest percentage in the Mid-Atlantic and the smallest percentages in the Mountain states and Puerto Rico.[Footnote 6] Several factors can affect the location of IMGs whether in residencies or in subsequent practice, such as the geographical distribution of residency programs, which are largely concentrated in the Mid-Atlantic region and in urban areas. With regard to medical practice, a larger proportion of IMGs go into residencies in primary care fields (68 percent compared to 37 percent of U.S. medical graduates in academic year 2008-2009). Moreover, IMGs increased as a percentage of all residents in core primary care fields (from 31 to 39 percent) between academic years 2001-2002 and 2008- 2009.[Footnote 7] Research shows that IMGs are also more likely than U.S.-educated graduates to practice as primary care physicians after finishing their residency training. * Overall, few significant differences exist between all IMGs and U.S.- educated physicians with regard to either disciplinary actions that would revoke or suspend their licenses or with regard to malpractice payments[Footnote 8]--and rates of disciplinary actions are low for physicians as a whole. Our analysis of national data from 2004 to 2008 on license revocation and suspension showed that IMGs accounted for a somewhat larger proportion of these actions than would be expected based on their share of the physician workforce overall, but it was not a statistically significant difference. With regard to malpractice, which research suggests is a weak indicator of physician competence; data on the whole suggest little difference between IMGs and domestically educated graduates. GAO is making several recommendations to the Department of Education concerning the lack of student consumer data on foreign medical institutions and also the department's monitoring of pass rates for foreign medical schools whose students take the U.S. medical licensing exam. Specifically, GAO recommends that the Secretary of Education: * collect consumer information, such as aggregate student debt level and graduation rates, from foreign medical schools participating in the federal student loan program and make it publicly available to students and their families; * require foreign medical schools to submit aggregate institutional pass rate data to the department annually; * verify data submitted by schools, for example, by entering into a data-sharing agreement with the testing organizations; and: * evaluate the potential impact of the 75 percent pass rate requirement on school participation in the federal student loan program and advise Congress on any needed revisions to the requirement. For this report, we analyzed the Department of Education's loan data for all foreign medical schools participating in the FFEL program between academic years 1998 and 2008.[Footnote 9] To assess the performance of international medical graduates on licensing examinations, we analyzed trends in exam data from 1998 to 2008. We evaluated Education's monitoring of foreign medical schools' compliance with the minimum licensing exam pass rate requirement through interviews with agency officials and analysis of exam data at an institutional level. We also analyzed graduate medical education data and interviewed cognizant officials to ascertain where graduates obtained residencies and to identify their medical specialties. With regard to discipline and malpractice, we analyzed data from the Department of Health and Human Services, the Federation of State Medical Boards, and two states--California and Florida--with high populations of international medical graduates.[Footnote 10] We also interviewed experts about the relevance and availability of these data. Because external data were significant to each of our research objectives, we assessed the reliability of the publicly and privately held data we obtained. We determined the data to be sufficiently reliable for the purposes of this report. Finally, we visited five stand-alone foreign medical schools in the Caribbean and Europe selected based on federal student loan volume and other institutional characteristics. At each school, we interviewed school officials and conducted student focus groups with a nongeneralizable sample of current students. We conducted this performance audit from June 2009 to June 2010 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Our scope and methodology are discussed in greater detail in appendix II. Additional information related to our principal findings can be found in appendixes III-VI. Agency Comments and Our Evaluation: We provided a draft of this report to the Departments of Education and Health and Human Services (HHS). We also shared relevant excerpts of the draft report with the private organizations that administer the licensing exams. Each of them provided technical comments which were incorporated into the report as appropriate. Education also provided additional comments which are reprinted in appendix VII. Education agreed with our recommendations and plans to collect consumer information on foreign medical schools. Education will also ask these schools to submit pass rate information starting with exams taken during the award year ending June 30, 2010, and will try to establish a mechanism to verify pass rates in cooperation with the private organizations that administer the exams. In addition, Education said that it has already begun evaluating the potential impact of the 75 percent pass rate requirement through proposed regulations, and that a notice of proposed rulemaking inviting public comment is scheduled to be published in summer 2010. In its comments, HHS noted that increasing the pass rate requirement will adversely affect federal student loan availability for future students attending foreign medical schools, adding that IMGs contribute a significant percentage of primary care residents in the United States. If you or your staff have questions about this report, please contact George A. Scott at (202) 512-7215 or ScottG@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff members who made key contributions to this report are listed in appendix VIII. Signed by: George A. Scott: Director, Education, Workforce, and Income Security: [End of section] Appendix I: Briefing Slides: Foreign Medical Schools: Education Should Improve Monitoring of Schools That Participate in the Federal Student Loan Program: Briefing to Congressional Committees: Education and Labor: House of Representatives: Health, Education, Labor, and Pensions: United States Senate: May 2010: Overview: * Introduction; * Objectives; * Scope and Methodology; * Summary of Findings; * Background; * Findings; * Conclusions and Recommendations; Introduction: Each year, the federal government makes a significant financial investment in the education and training of the U.S. physician workforce. A quarter of that physician workforce is comprised of international medical graduates (IMG) and they include both U.S. citizens and foreign nationals. * In fiscal year 2008, the federal government provided $633 million in federally guaranteed loans to U.S. citizens enrolled in foreign institutions, including medical students. These loans were made through the Federal Family Education Loan (FFEL) program.[Footnote 1] * That same year, it spent another nearly $9 billion to sponsor the hundreds of residency programs that train graduates from foreign and U.S. medical schools alike, and that are required to practice medicine in the United States. Foreign medical schools can differ substantially from U.S. schools in their requirements for admission and length of study. * For this and other reasons, the Department of Education, which administers the federal student loan program, imposes requirements for foreign schools to participate in the program.[Footnote 2] * Among these is a statutory requirement that at least 60 percent of IMGs who take the U.S. medical licensing exam must pass. Recently, Congress increased the pass rate requirement to 75 percent, effective July 2010. Little is known about U.S. citizens who study medicine abroad with respect to how much they borrow in federal loans overall, or the outcomes of their medical studies. GAO undertook this study pursuant to a mandate enacted in section 1101 of the Higher Education Opportunity Act.[Footnote 3] Objectives: 1. What amount of federal student aid loan dollars has been awarded to U.S. students attending foreign medical schools? 2. What do the data show about the pass rates of international medical graduates on license examinations? 3. To what extent does Education monitor foreign medical schools' compliance with the pass rate required to participate in the federal student loan program? 4. What is known about schools' performance with regard to the institutional pass rate requirement? 5. What is known about where international medical graduates have obtained residencies in the United States and the types of medicine they practice? 6. What is known about discipline and malpractice involving foreign- educated physicians? Scope and Methodology: To conduct our work, we: * reviewed and analyzed federal student loan volume and privately held data on U.S. licensing exams, residency surveys, and disciplinary actions and malpractice payments; * interviewed officials and experts and reviewed information and documents from: - Department of Education; - Department of Health and Human Services, Health Resources and Services Administration; - Educational Commission for Foreign Medical Graduates (ECFMG), the National Board of Medical Examiners, and the Federation of State Medical Boards; - Selected schools participating in the federal student loan program; - Agencies responsible for licensing and disciplining physicians in four states with high concentrations of IMGs, and found the data from two of these states to be reliable for our purposes; * reviewed relevant literature on IMGs published from 1990 to 2009 by various associations and individual researchers; and; * reviewed relevant federal laws and regulations; * conducted site visits to five foreign medical schools: - American University of the Caribbean (St. Maarten), - Ross University (Dominica), - St. George's University (Grenada), - Royal College of Surgeons (Ireland), - Poznan University of Medical Sciences (Poland); and, * conducted a non-generalizable sample of student focus groups— consisting primarily of U.S. citizens who were receiving federal student loans—at each school we visited. We determined that the data used to address our research objectives were sufficiently reliable for the purposes of this report. Summary of Findings: 1. From 1998 to 2008, U.S. students borrowed $1.5 billion in federally guaranteed student loans to study at foreign medical schools. Meanwhile, some student borrowers stated that they lack reliable cost and performance information about these schools. 2. Pass rates on the U.S. Medical Licensing Exam have improved for all international medical graduates, but still lag behind those of U.S.- educated graduates. 3. Education lacks the data necessary to fully enforce foreign medical school compliance with the pass rate requirement. 4. While a majority of foreign medical schools met the current 60 percent pass rate requirement in 2008, very few would likely meet the newly enacted 75 percent rate. 5. International medical graduates in residency programs are located primarily in the eastern U.S. and tend to practice primary care fields. 6. Available data on disciplinary proceedings and malpractice reveal few differences between international medical graduates and U.S.- educated physicians. Background: Who Are International Medical Graduates (IMGs)? * They are physicians who have graduated from medical schools in countries other than the U.S. and Canada.[Footnote 4] * IMGs comprise over one quarter (about 244,000 physicians) of the current physician workforce in the U.S. * They include both citizens of the U.S. and other countries. * Of IMGs who practice in the U.S., nearly 20 percent are U.S. citizens or permanent residents. Federal Student Loan Program Requirements for Foreign Medical Schools: To participate in the program, foreign medical schools must: * Be located in one of 26 countries[Footnote 5] whose standards of accreditation are comparable to those in the U.S. as determined by the National Committee on Foreign Medical Education and Accreditation; [Footnote 6] * Meet the statutorily mandated institutional pass rate on certain medical licensing exams[Footnote 7] and other specific statutory and regulatory eligibility requirements; * A few schools have been statutorily exempted from the pass rate requirement in view of their existing approved U.S. clinical training programs in operation since 1992: - St. George's University, Grenada; - Our Lady of Fatima University, the Philippines; - American University of the Caribbean, St. Maarten; - Tel Aviv University, Israel; - Ross University, Dominica. Federal Student Loan Program Requirements for U.S. Medical Students Abroad: According to Department of Education guidance, students who are enrolled in eligible foreign medical schools: * Must generally be U.S. citizens, nationals, or eligible permanent residents to qualify for federally guaranteed student loans. * May borrow up to $20,500 in subsidized and unsubsidized loans [Footnote 8] annually. Total borrowing through the program for students enrolled in foreign medical schools is limited to $138,500. * May borrow additional loan funds up to the cost of attendance minus other federal assistance in loans for graduate and professional students, available as of 2006, if they have met the federal student loan program limit. By contrast, students enrolled in U.S. medical schools may borrow up to $224,000 in federal guaranteed student loans to replace loan funds no longer available through the Department of Health and Human Services' Health Education Assistance Loan (HEAL) program.[Footnote 9] How Foreign Medical Schools Differ from U.S. Medical Schools: Foreign medical schools can vary considerably from U.S. medical schools. * Most students must receive a competitive score on the Medical College Admission Test (MCAT) to be admitted to a U.S. medical school because of educational standards and limited slots. Unlike U.S. medical schools, foreign medical schools do not necessarily require an undergraduate degree or an admissions test; and, if an admissions test is required, the score need not be as competitive as for U.S. schools. * Some foreign medical schools can be highly selective in their admissions though they do not necessarily require the MCAT. * Some foreign medical schools are for-profit institutions with shortened academic timelines. Whereas U.S. medical school students typically complete degrees in four years, students at some schools in the Caribbean can do so in under three. Requirements for IMGs to Enter a Residency Program in the United States: Because medical schools outside the United States and Canada vary in their educational standards and curricula, ECFMG certification is designed to assure residency program directors and the public that IMGs have met the minimum standards of eligibility to enter such programs.[Footnote 10] To enter a U.S. residency training program, IMGs must first become certified by the ECFMG whose requirements include: * Passing the first two (i.e. Step 1, Step 2CK and Step 2 CS) of the three steps of the U.S. medical licensing exam). * Showing evidence of a medical degree from a recognized medical school.[Footnote 11] The Steps of the U.S. Medical Licensing Exam: The following exam steps can be taken while attending medical school abroad: * Step 1 uses a multiple-choice format to assess knowledge and application of basic science concepts in the practice of medicine. * Step 2 Clinical Knowledge (CK) uses a multiple-choice format to assess knowledge of clinical science principles. * Step 2 Clinical Skills (CS) tests students ability to examine and interact with patients and colleagues. The final step of the licensing exam is often taken during residency. [Footnote 12] * Step 3 uses multiple-choice items and computer-based simulations and provides a final assessment of a physician's ability to assume independent delivery of general medical care. Discipline and Malpractice: Physicians—whether educated in the U.S. or abroad—who are licensed to practice, may be subject to various disciplinary actions taken through state medical boards, or hospitals, and other medical providers. * License actions: revocations, suspensions, or other restrictions on a physician's license based on a review by state medical boards. * Malpractice:[Footnote 13] a finding of negligent care, related to services that a physician provided or failed to provide. Finding 1: Student Loans at Foreign Medical Schools: Loans to U.S. Medical Students Abroad Were Concentrated at Free- standing Medical Institutions: Between 1998 and 2008, a total of 137 foreign medical institutions in 31 countries participated in the FFEL program during at least one academic year. * School participation has remained relatively constant. * As of 2008, there were 113 schools. * Of these, 21 were free-standing medical institutions and the rest were component schools.[Footnote 14] From 1998 to 2008, U.S. students borrowed $1.5 billion to study medicine at the free-standing institutions.[Footnote 15] * Of that amount, about $1.3 billion (about 90 percent) went to students at three Caribbean schools (American University of the Caribbean, Ross University and St. George's University). Figure: Borrowing at Foreign Medical Schools Has Steadily Increased Since 1998: [Refer to PDF for image: vertical bar graph] Since 1998, loans at freestanding schools grew by 338 percent compared with a 154 percent increase in the FFEL program overall. School officials we met with attributed loan growth to increases in tuition, enrollment, and the introduction of graduate and professional loans in 2006. Award years: 1997-1998; FFEL loans at free-standing medical schools: $72.0 million. Award years: 1998-1999; FFEL loans at free-standing medical schools:: $82.3 million. Award years: 19999-2000; FFEL loans at free-standing medical schools:: $92.5 million. Award years: 2000-2001; FFEL loans at free-standing medical schools:: $92.8 million. Award years: 2001-2002; FFEL loans at free-standing medical schools:: $99.1 million. Award years: 2002-2003; FFEL loans at free-standing medical schools:: $103.4 million. Award years: 2003-2004; FFEL loans at free-standing medical schools:: $112.6 million. Award years: 2004-2005; FFEL loans at free-standing medical schools:: $115.9 million. Award years: 2005-2006; FFEL loans at free-standing medical schools:: $127.2 million. Award years: 2006-2007; FFEL loans at free-standing medical schools:: $252.1 million. Award years: 2007-2008; FFEL loans at free-standing medical schools:: $315.1 million. Source: GAO analysis of Department of Education data. [End of figure] Importance of Federally Guaranteed Student Loans: Of the students who participated in our focus groups, many reported that they relied heavily on federal student loans to fund their medical education abroad. The majority were most likely to categorize federal loans as very important in contrast to other forms of financial assistance, both private and public.[Footnote 16] * While little is known about the debt burden accumulated by IMGs, focus group participants we interviewed projected their student loan debt would range from $90,000 to $250,000 for their medical degree alone.[Footnote 17, 18] * Officials at the foreign medical schools we visited reported total student costs ranging from just over $30,000 to about $90,000 per year. Most of the tuition and fees reported by schools fell within the range of average cost for U.S. medical schools. Some Focus Group Participants Reported That They Lacked Reliable Information on Likely Loan Debt and Institutional Performance: Some students who participated in our focus groups said that when making their initial selection of schools, they lacked a centralized source of reliable information about their likely medical student loan debt or institutional performance such as graduation rates. Many focus group participants said they relied on student-driven websites to guide their decision-making. According to ECFMG officials, IMG oriented websites can vary in both accuracy and sophistication. Students lack sources that are objective and factual. Table: Comparison of Sources Available for Decision-making by U.S.- educated students and IMGs: Admission Requirements: U.S. Medical Students: Centralized medical college association website which allows comparison of several school websites; IMGs: Multiple school websites and student-led electronic bulletin boards. Graduation Rates: U.S. Medical Students: Medical college association; IMGs: No known reliable sources. Medical School Loan Debt: U.S. Medical Students: Medical college association; IMGs: No known reliable sources. Source: GAO analysis of U.S. Department of Education, association, and independent websites. [End of table] Finding 2: U.S. Medical Licensing Exam Pass Rates: Figure: Pass Rates for All IMGs Have Improved on Most Licensing Exam Steps But Still Lag Behind Those of U.S.-Educated Graduates: [Refer to PDF for image: 3 multiple line graphs] Step 1 (on basic science), 1998-2008: 1998 First-time pass rate, US-Educated: 95%; 1998 First-time pass rate, IMGs: 62%. 2008 First-time pass rate, US-Educated: 94%; 2008 First-time pass rate, IMGs: 73%. 32,114 graduates tested in 2008: US-Educated: 17,222 (54%); IMGs: 14,892 (46%). Step 2 Clinical Knowledge, 1998-2008: 1998 First-time pass rate, US-Educated: 95%; 1998 First-time pass rate, IMGs: 55%. 2008 First-time pass rate, US-Educated: 97%; 2008 First-time pass rate, IMGs: 82%. 28,977 graduates tested in 2008: US-Educated: 16,262 (57%); IMGs: 12,351 (43%). Step 3 (on delivery of general medical care), 1998-2008: 1998 First-time pass rate, US-Educated: 95%; 1998 First-time pass rate, IMGs: 55%. 2008 First-time pass rate, US-Educated: 95%; 2008 First-time pass rate, IMGs: 78%. 26,450 graduates tested in 2008: US-Educated: 17,077 (65%); IMGs: 9,373 (35%). Source: GAO analysis of Educational Commission for Foreign Medical Graduates and National Board of Medical Examiners data. Notes: IMGs include both U.S. citizens and citizens of other countries. As such, IMG pass rates reflect performance for individuals who may or may not have received federal student loans. All test- takers are referred to as "graduates"; however, students are eligible to take the Step 1, Step 2 clinical knowledge, and Step 2 clinical skills exams while they are still enrolled in medical school. [End of figure] Despite Improvements, IMG First-Time Pass Rates on the Step 2 Clinical Skills Exam Have Declined: Figure: Step 2 Clinical Skills, 2004-2008: [Refer to PDF for image: multiple line graph] 2004 First-time pass rate, US-Educated: 95%; 2004 First-time pass rate, IMGs: 84%. 2008 First-time pass rate, US-Educated: 97%; 2008 First-time pass rate, IMGs: 71%. 30,593 graduates tested in 2008: US-Educated: 16,916 (55%); IMGs: 13,677 (45%). Source: GAO analysis of Educational Commission for Foreign Medical Graduates and National Board of Medical Examiners data. Notes: IMGs include both U.S. citizens and citizens of other countries. As such, IMG pass rates reflect performance for individuals who may or may not have received federal student loans. Test-takers are referred to as 'graduates"; however, students are also eligible to take the Step 2 clinical skills exam. The Step 2 clinical skills exam became a certification requirement on June 14, 2004. Thus, pass rate data for this exam are available only for exam year 2004 and onward. Prior to that, ECFMG administered the Clinical Skills Exam, starting in July 1998, as a requirement for certification. [End of table] Officials from the two organizations that maintain licensing exam data told us that IMG pass rates on the clinical skills exam, offered since 2004, may have declined in recent years due, in part, to increases in the minimum requirements for passing the exam. IMGs Needed More Attempts to Pass Licensing Exams than U.S.-Educated Graduates: IMGs have needed more attempts to pass the licensing exams than their U.S.educated counterparts. However, pass rates on additional attempts were lower than initial attempts for both IMGs and U.S.-educated students. About 27 percent of IMG test-takers repeated exams during the last ten years, compared to about 6 percent of U.S.-educated test- takers. * In 2008, 4,833 IMGs repeated the Step 1 exam at least once and 42 percent passed on the repeated attempt. * In the same year, 1,182 U.S.-educated graduates repeated the Step 1 exam at least once and 69 percent passed on subsequent attempts. According to researchers, test takers who require multiple attempts to pass the licensing exams will likely have difficulty being selected for a residency because they are viewed by some as less qualified as those passing the exams on their first try. Factors That May Affect IMG Pass Rates: Limited English Proficiency and Institutional Focus on U.S. Licensing Exams: Research indicates that English language proficiency may affect IMGs' exam performance on the clinical skills exam, especially in gathering data, sharing information, and establishing rapport with patients. [Footnote 19] Research and experts suggest that other factors are the extent to which exam preparation is incorporated into a medical school's curriculum and the proportion of the school's students taking the exam. * Although passing the licensing exam is critical for those seeking to practice in the U.S., not all schools focus on this as a part of their program. * Focus group participants at the two European schools we visited told us that faculty provided limited assistance in exam preparation. Officials at both schools told us they would provide additional test- preparation support for their students. * At the three Caribbean schools we visited, there was significant focus on preparing students for the exam. Institutional policies at two of the three schools precluded students from taking the exam if they were not ready and required successful performance on the exam in order to graduate. Other Factors May Also Explain Lower Pass Rates for IMGs Who Were U.S. Citizens: Experts we interviewed suggested that some U.S. citizen IMGs may lack the test-taking skills of their U.S.-educated counterparts. Several focus group participants we spoke with told us that their lower MCAT scores made them less competitive when applying to U.S. medical schools. * According to data from the Association of American Medical Colleges, fewer than one third of 2005-2007 applicants with MCAT scores below 30 were accepted to U.S. medical schools.[Footnote 20] Several focus group participants we spoke with told us that they entered foreign medical school from other disciplines or applied to medical school years after finishing college, suggesting that they did not follow the traditional academic pathway to medical school and may not have had the same exposure to medical concepts as other students. Finding 3: Monitoring of Licensing Exam Pass Rates: Education Has Not Been Able to Fully Enforce the Pass Rate Requirement: The three private organizations that administer the licensing exams have not released their data to schools or to Education on grounds that the information is proprietary and should not be used for marketing purposes. * As a result, Education has not been able to collect such data from the schools. * Education officials said that since federal student loans to foreign medical schools comprise less than 1 percent of all student loans, they use a risk-based approach to monitor compliance with the pass rate requirement. They request the data from schools[Footnote 21] only when schools: - apply to participate in the loan program, - seek recertification for participation—at intervals between 1 to 6 years, or, - change institutional ownership. * When schools did provide the data, however Education officials noted that they could not independently verify it. Schools Are Beginning to Negotiate for the Data and Education Has Recently Reconsidered Its Approach to Enforcement: Recently two of the three testing organizations have begun to negotiate, on a school-by-school basis, to provide the data to schools, which may allow schools to systematically prove that they are satisfying the pass rate requirement. * These organizations administer and/or govern Step 1, Step 2CS, and Step 2CK of the medical licensing exam. Education told us that the department will soon fully enforce the pass rate requirement by requesting schools to provide exam data on an annual basis, but did not elaborate on how and-when it would implement this new approach. However, a spokesperson for the testing organization that administers Step 3 said that the organization would not be entering into such agreements with schools because it shares exam data only with state medical boards. While the National Committee for Foreign Medical Education and Accreditation recommended the inclusion of Step 3 as part of this requirement, Education officials indicated that they would not request these data because the testing organization that administers Step 3 is not specifically covered by the pass rate requirement.[Footnote 22] Finding 4: Performance on Institutional Pass Rates: A Majority of Foreign Medical Schools Met the Pass Rate at 60 Percent in 2008, But Very Few Would Do So at 75 Percent: According to our analysis[Footnote 23] of 2008 institutional pass rate data: At 60%: A majority (58 percent) of foreign medical schools met or exceeded the current pass rate requirement. * For 6 countries, all 30 schools met the pass rate. * For 3 countries, less than one-third of the 105 schools met the pass rate. At 75 percent: Only 24 of 218 schools with test-taking students (or 11 percent) would meet the newly enacted pass rate were they to post the same performance as in 2008. * Two of the five schools statutorily exempt from the requirement, but which account for over 50 percent of federal student loans, would not meet the new standard. Institutional Pass Rates May Be Limited as a Measure of School Quality: Testing officials cautioned against associating students' performance on medical licensing exams with institutional quality for the following reasons: * Test-takers could be a school's best or worst students. * Some schools restrict who may sit for the exam based on student readiness—thus inflating an institution's score. * Other schools may encourage "practice runs" for students who are less prepared or have no restrictions on who sits for the exams— possibly lowering institutional scores. * Performance on Step 3 of the exam—which may occur during residency training in the U.S.—has little to do with attending school abroad. [Footnote 24] Additionally, according to some school officials, institutions with small numbers of test-takers may find it hard to meet the pass rate requirement.[Footnote 25] Some Schools Have Expressed Concerns About the 75 Percent Rate: Some school and medical association officials we interviewed expressed the view that the new pass rate could discourage loan program participation: * They noted that because some non-profit institutions are "highly selective" and only admit small numbers of U.S. students each year, these schools would find it burdensome and not cost effective to meet the terms of the provision. * They also expressed concern that their pass rate performance, given small numbers of test-takers, would misrepresent the quality of their program if the school lost eligibility to participate.[Footnote 26] Finding 5: Residencies and Practice: In 2008, about 30,000 IMGs Were in Residencies in the U.S., with 78 Percent of All IMGs Located East of the Mississippi River and 24 Percent in New York Alone: Figure: Illustrated U.S. map: [Refer to PDF for image: Geographic location: New England; IMG Residencies: 2,264 (7.5%). Geographic location: Middle Atlantic; IMG Residencies: 10,732 (35.8%). Geographic location: New York; IMG Residencies: 7,062 (23.5%). Geographic location: South Atlantic; IMG Residencies: 3,702 (12.3%). Geographic location: Puerto Rico; IMG Residencies: 252 (0.8%). Geographic location: Central East North; IMG Residencies: 5,650 (18.8%). Geographic location: Central East South; IMG Residencies: 1,077 (3.6%). Geographic location: Central West North; IMG Residencies: 1,667 (5.6%). Geographic location: Central West South; IMG Residencies: 2,557 (8.5%). Geographic location: Mountain; IMG Residencies: 689 (2.3%). Geographic location: Pacific; IMG Residencies: 1,423 (4.7%). Source: GAO analysis of Accreditation Council for Graduate Medical Education (ACGME) academic year 2008-2009 data. Note: The geographic location of IMGs in residencies is partly determined by the availability of residency slots in the United States each year. The map depicts geographic areas as defined by ACGME. [End of figure] More IMGs than U.S.-Educated Graduates Go into Primary Care-Related Residencies, And This Pattern Continues into Practice: Figure: Number and percentage distribution of U.S.-educated graduates and IMGs in entry-level residencies, academic year 2008-09: [Refer to PDF for image: 2 pie-charts] U.S.-educated Graduates: 59,060: Surgical: 27.1% (15,983); Primary Care: 37.2% (21,944); Other: 35.8% (21,133). International Medical G2,75015,983); Primary Care: 67.9% (16,141); Other: 20.5% (4,875). Source: GAO analysis of ACGME data. Note: The number of primary care residency slots is determined, in part, by funds available through Medicare payments to hospitals and other teaching institutions. [End of figure] Some IMGs Practice in Underserved Areas: After completing residencies, some IMGs practice medicine in medically underserved areas, but it is not known to what extent they do so or whether they are more likely than U.S.-educated graduates to serve in these areas. There are some programs designed to channel foreign physicians into geographic areas with a shortage of health care professionals. * The "Conrad 30" J-1 visa waiver allows physicians on Exchange Visitor (J-1) visas to remain in the U.S. after completing residency, if they agree to practice for at least three years in an area that is federally designated as having a healthcare professional shortage. * These waivers must be requested by a state or federal agency and states can request such waivers in order to retain foreign physicians for such designated areas. Although J-1 visa waivers for foreign IMGs have been a federal and state strategy to encourage physicians to practice in underserved areas, HHS, which defines these areas and coordinates programs for addressing physician shortages, does not officially collect data on waivers granted or on placements for physicians who are granted waivers. [Footnote 27] * According to one estimate, more than 700 foreign IMGs were awarded J- 1 visas waivers in 2008 to practice in underserved areas.[Footnote 28] Figure: Rates of Disciplinary Action Are Low for Physicians Overall, Both for IMGs As A Group And for U.S.-Educated Graduates: [Refer to PDF for image: horizontal bar graph] Percent of all physicians 2004-2008: U.S. Graduates[A]: 75%; International Graduates: 25%. Disciplinary action of all kinds[B] 1992-2007: U.S. Graduates[A]: 1.3%; International Graduates: 1.1%. Source: Data on percentage of all physicians by IMG and U.S.-educated status: American Medical Association (AMA) Physician Master File data as analyzed and reported by the Association of American Medical Colleges (AAMC) for 2004 to 2008; discipline data: Federation of State Medical Boards (FSMB) data prepared for the U.S. Department of Education for 1992-2007. Notes: [A] Canadian medical graduates are counted as U.S. medical graduates for all analyses. [B] "Disciplinary actions" includes license revocations, suspensions, restrictions, and other actions, though not malpractice cases. [End of figure] Finding 6: Discipline and Malpractice: Figure: There Are No Significant Differences Nationwide with Regard to License Revocation and Suspension: [Refer to PDF: Illustration] Nationwide (2004 to 2008): IMGs made up: 25 percent of all physicians in the U.S. from 2004 to 2008; yet accounted for: 29 percent of license revocations and: 27 percent of license suspensions. National data on license revocations and suspensions for the most recent five years with complete data show that, while all IMGs represented about one-quarter of all physicians, they accounted for a somewhat larger share of these disciplinary actions. These differences were not statistically significant when compared to the same disciplinary actions experienced by U.S.-educated physicians. Source: Data on percentage of all physicians by IMG and U.S.-educated status were obtained from AMA Physician Master File data as analyzed and reported by the AAMC. License revocation and suspension data were obtained through GAO analysis of FSMB data. Note: Canadian medical graduates are treated as U.S. medical graduates for all analyses. [End of figure] Finding 6: Discipline and Malpractice: Figure: Similarly, for Two Key States, There Are Few Significant Differences: California (2004-2008): IMGs made up: 25 percent of physicians from 2004 to 2008, which is also the US average, yet accounted for: 28 percent of state license revocations and: 26 percent of the state's license suspensions. Florida (2004-2008): IMGs made up: 38 percent of physicians, which is higher than the U.S. average yet accounted for: 59 percent[Footnote 29] of state license revocations and: 41 percent of the state's license suspensions. Source: GAO analysis of state license revocation and suspension data, AMA Physician Master File data analyzed and reported by the AAMC. 36 Note: Canadian medical graduates are treated as U.S. medical graduates for all analyses. [End of figure] Data from two states with large numbers of IMGs generally reflect the national pattern. However, our analysis found significant differences between all IMGs and U.S.educated physicians in Florida with respect to license revocations. State officials told us they did not know the reasons for these differences. Figure: Nationwide, IMGs Did Not Account for A Disproportionate Share of Malpractice Payments in 2004-2008[Footnote 30]: [Refer to PDF for image: illustration] All Physicians: IMGs made up 25 percent of all U.S. physicians from 2004 to 2008 and: IMGs accounted for 25 percent of U.S. physicians with reported multiple malpractice payments from 2004 to 2008. Surgeons: IMGs made up 22 percent of all surgeons working in the U.S. in 2007 and: IMGs accounted for 25 percent of U.S. surgeons with reported malpractice payments from 2004 to 2008. Obstetricians/Gynecologists: IMGs made up 17 percent of all OB/GYNs working in the U.S. in 2007 and: IMGs accounted for 18 percent[Footnote 31] of U.S. OB/GYNs with reported malpractice payments from 2004 to 2008. Source: GAO analysis of data reported to the National Practitioner Data Bank. Note: Percentages have been rounded to nearest whole percentage point. Canadian medical graduates are treated as U.S. medical graduates for all analyses. Malpractice payments are payments made as a result of a settlement or judgment based on the provision or failure to provide health care services. [End of figure] Research on IMG Performance Is Limited: Few research studies have compared performance of IMGs with that of U.S.-educated graduates beyond disciplinary actions, such as on their clinical processes and patient outcomes.[Footnote 32] * A 2004 study found IMGs were as likely as other physicians to follow professional standards. * A 1991 study found little difference in outcomes for mortality or length of hospital stay for 16 different medical and surgical conditions. * A 1990 study found that certain IMGs had higher rates of complications for a particular surgical procedure. In addition, a 2004 California study found that of three characteristics—age, gender, and IMG status—IMG status had the weakest relationship to the likelihood of professional discipline. Conclusion: While only a very small proportion of all federally guaranteed student loans goes to U.S. students to study medicine at foreign schools, it does not diminish the valuable role this funding plays. * For individual Americans, the loans represent the single most important avenue available to finance their medical education—without which, they would not become physicians. * For the nation as a whole, these loans help assure a steady supply of the U.S. physician workforce. The fact that foreign educated doctors, including U.S. citizens, are more likely than their domestically educated peers to practice primary care medicine fulfills an ever-increasing demand for such physicians. However, in contrast to those pursuing a domestic medical education, U.S. students seeking overseas study are at a considerable disadvantage as consumers, given the absence of reliable information about foreign medical schools—such as their graduation rates or cost. Yet, the Higher Education Opportunity Act emphasizes the importance of consumer-based choices. For policymakers, meanwhile, Education has yet to fully enforce compliance with the pass rate required for foreign medical schools to participate in the federal loan program since verifiable data have not been available. Moreover, while foreign medical schools may soon be able to negotiate access to their students' scores, it still remains to be seen whether a specific pass rate is a useful measure of quality, and, until further evaluation, whether a pass rate of 75 percent is appropriate. * Our own analysis of exam scores by country suggests that the new pass rate requirement may dissuade or even disqualify many schools from participating in the loan program. * Such an outcome, would, in turn, severely narrow the choices for U.S. students who are prepared to undertake the long and difficult road to medical practice. Recommendations: To enhance information available for prospective students of foreign medical schools and strengthen monitoring of foreign medical schools participating in the federal student loan program, we recommend that the Secretary of Education: * Collect consumer information, such as aggregate student debt level and graduation rates, from foreign medical schools participating in the federal student loan program as recommended by the National Committee on Foreign Medical Education and Accreditation and make it publicly available to students and their families; * Require foreign medical schools to submit aggregate institutional pass rate data to the Department annually; * Verify data submitted by schools, for example by entering into a data sharing agreement with the testing organizations; * Evaluate the potential impact of the newly enacted 75 percent pass rate requirement on school participation in the federal student loan program and advise Congress on any needed revisions to the requirement. [End of section] Briefing slides footnotes: [1] To date, the FFEL program has been the only federal student financial aid program in which foreign schools participated. Under newly enacted legislation, the SAFRA Act, which was included in the Health Care and Education Reconciliation Act of 2010, (Pub. L. No. 111- 152 (2010)), the FFEL program will terminate June 30, 2010, after which no new loans will be made under the FFEL program. The SAFRA Act extends the availability of loans under the William D. Ford Federal Direct Loan (Direct Loan) program to students at eligible foreign institutions. Thus, beginning in July 2010, students at foreign medical schools will receive new loans through the Direct Loan program, instead of the FFEL program. Throughout this report we refer to the federal student loan program in our findings. Where our findings are specific to the FFEL program, however, we refer to that program by name. [2] To receive federal loan funds, students must be enrolled at eligible institutions located in countries with standards of accreditation of medical schools that are comparable to the standards of accreditation of medical schools in the United States. [3] Pub. L No.110-315 (2008). [4] The definition of IMG may vary. Health workforce experts do not consider Canadian graduates to be IMGs because the accreditor and medical education standards for Canada are similar to the U.S. Alternatively, the Department of Education's definition of IMGs includes U.S. students who were educated in Canadian medical schools. For the purposes of our report, our definition of IMGs excludes graduates of U.S. and Canadian medical schools. [5] According to the Foundation for the Advancement of International Medical Education and Research's International Medical Education Directory, there were 251 foreign medical schools located in comparable countries, some of which participate in the federal loan program. At various places in this report, the data presented may pertain to schools that either participate in the federal student loan program or to all foreign medical schools regardless of their participation. [6] 20 U.S.C. § 1002(a)(1)(C), (a)(2)(B); 34 C.F.R. § 600.55(a)(4). [7] 20 U.S.C. § 1002(a)(2)(A)(i)(1)(bb); 34 C.F.R. § 600.55(a)(5)(i)(B). [8] Federally subsidized student loans are based on financial need. The interest on subsidized federal loans is paid by the federal government while students are enrolled at least half-time in college and during periods of authorized deferment. The interest for these loans is paid by the student upon graduation. Interest on unsubsidized federal loans is paid by the student and accrues from the date the loan is disbursed to a student until it is paid in full. [9] U.S. medical students abroad were not eligible to participate in the HEAL program and thus are ineligible to receive the additional loan funds made available to students attending U.S. medical schools. [10] While there is no limit on the number of times a student may take an exam step, test-takers have seven years from the time they first pass a step to pass all other steps required for certification. [11] The Foundation for Advancement of International Medical Education and Research currently lists more than 2,000 such institutions. [12] According to ECFMG officials, a number of states allow physicians to take Step 3 prior to entering a residency program and a significant number of IMGs do so. [13] Research indicates that malpractice data are of limited value as an indicator of competence or negligence. Possibly no more than 3 percent of medically adverse events result in malpractice claims. Moreover, of those that do, an estimated 40 percent are found not to have involved error or injury. [14] Free-standing institutions are schools whose principal offering is medical education; whereas, component medical institutions are medical schools within a larger university system. [15] About another $1 billion was borrowed by students to study at component schools. However, because Education does not capture loan data by academic discipline, it is unclear what portion of that amount was used to study medicine as opposed to other academic disciplines. [16] The findings obtained from our student focus groups cannot be generalized to the student population of international medical graduates. [17] Students participating in the student focus groups did not always indicate whether the loan funds borrowed were federal student loans or private loans. [18] According to the Association of American Medical Colleges, students enrolled in U.S. medical schools incur about $155,000 in student loan debt for their medical degrees. [19] According to one study, although research indicates that IMGs who are U.S. citizens are more likely than foreign IMGs to claim English as a native language and to have received medical school instruction in English, nearly 30 percent of U.S. citizen IMGs are non-native English speakers. See Boulet, John R., et al, "U.S. Citizens Who Obtain Their Medical Degrees Abroad: An Overview, 1992-2006," Health Affairs, vol. 28, no.1 (2009). [20] The highest possible score for all students is 45. [21] Department officials told us that in 2004, when they requested pass rate data from schools, they received data back only sporadically. Between 1998 and 2008, only one school has lost eligibility based on the pass rate data it provided to Education. Fourteen other schools have lost eligibility due to not meeting other program rules. Education officials stated that foreign medical schools have been able to obtain student consent agreements for pass rate data on Steps 1 and 2. [22] The statutory and regulatory pass rate requirement applies only to examinations administered by ECFMG, which are Step 1 and Step 2. 20 U.S.C. § 1002(a)(2)(A)(i)(I)(bb); 34 C.F.R. § 600.55(a)(5)(B). Because Step 3 is administered by the Federation of State Medical Boards, it is excluded from the pass rate requirement. [23] The data we obtained from ECFMG and NBME did not allow for the identification of individual schools, but allowed us to look at institutional performance by country. Because school identities were masked, we could not differentiate between federal loan participants and others. To the extent that not all schools participated in the federal loan program, our results may be overstated. Our analysis was based on individuals who took the exam. Because we could not estimate the proportion of students who were allowed to take the exam, we could not identify the extent to which scores may be overstated. [24] According to ECFMG officials, a number of states allow graduates to take Step 3 prior to entering a residency program and a significant number of IMGs do so. [25] Education officials agreed with this assessment and reported to us that they are proposing regulations that would make special provisions for foreign medical schools that enroll small numbers of U.S. citizens by asking them to provide the data only when the pass rate is based on at least 8 or more students who took the exam. [26] Education officials reported to us that they are proposing regulations that would make special provisions for foreign medical schools that enroll small numbers of U.S. citizens. [27] GAO recommended in 2006 that HHS collect and maintain data on waiver physicians in order to better address physician shortages. According to an official at HHS' Health Resources and Services Administration, as of September 2008, no action had been taken by HHS regarding this recommendation. See GAO, Foreign Physicians: Data on Use of J-1 Visa Waivers Needed to Better Address Physician Shortages, GAO-07-52 (Washington, D.C.: Nov. 30, 2006). [28] The Texas Primary Care Office surveys all states annually on their requests for J-1 visa waivers. [29] Denotes a statistically significant difference. [30] Our analysis of malpractice payments is generally consistent with prior work by GAO and others. See Medical Malpractice: Characteristics of Claims Closed in 1984, HRD-87-55, (Washington, D.C.: April 1987), and S. Mick and M. Comfort, The Quality of Care of International Medical Graduates: How Does It Compare to That of U.S. Medical Graduates?" Medical Care Research and Review, Dec. 1997. [31] U.S.-educated OB/GYNs were significantly more likely to have had a malpractice payment during this period than IMG OB/GYNs. [32] These studies were among the most recently available research studies conducted on this subject. [End of section] Appendix II: Objectives, Scope, and Methodology: Objectives: Our review focused on the following questions: (1) What amount of federal student aid loan dollars has been awarded to U.S. students attending foreign medical schools? (2) What do the data show about the pass rates of international medical graduates on license examinations? (3) To what extent does Education monitor foreign medical schools' compliance with the pass rate required to participate in the federal student loan program? (4) What is known about schools' performance with regard to the institutional pass rate requirement? (5) What is known about where international medical graduates have obtained residencies in the United States and the types of medicine they practice? (6) What is known about discipline and malpractice involving foreign educated physicians? Defining International Medical Graduates: With regard to the Federal Family Education Loan (FFEL) program, program requirements allow only U.S. citizens, nationals, permanent residents, and certain other eligible noncitizens to obtain these loans. In view of this fact, our findings for the first objective were limited to U.S. citizens, nationals, permanent residents, and certain other eligible noncitizens who were also IMGs. By contrast, our findings pertaining to the remaining objectives were based on all IMGs educated in medical schools abroad, including those students who are U.S. citizens or residents as well as students who are foreign nationals. For these objectives, we included graduates of Canadian medical schools in the total population of U.S. medical graduates. According to the body of literature we reviewed, Canadian schools are generally not considered to be foreign medical schools since their medical education system is closely comparable to that of the United States. Similarly, we included graduates of Puerto Rican medical schools in the total population of U.S. medical graduates because medical schools in Puerto Rico are subject to the same accreditation standards as other U.S. medical schools. In addition, our analyses excluded students in and graduates of osteopathic programs because only graduates of U.S. osteopathic schools are eligible to become licensed physicians in the United States. Data Analysis by Objective: To determine the amount of federal student loan dollars borrowed abroad, we analyzed the Department of Education's loan data for all foreign medical schools participating in the FFEL program between academic years 1997-1998 and 2007-2008.[Footnote 11] We also interviewed Education officials about participation in the FFEL program as it relates to foreign medical schools and their graduates. To aid our discussion on this matter, we adopted the use of the department's nomenclature (i.e., free-standing institutions and component institutions).[Footnote 12] We reviewed relevant federal laws and regulations that specify the conditions for compliance with the FFEL program. To understand how determinations of medical comparability are made for foreign countries and the impacts on institutional eligibility, we also observed a meeting of the Department of Education's National Committee on Foreign Medical Education and Accreditation during September 2009. This committee is charged with reviewing the standards that foreign countries use to accredit medical schools to determine whether those standards are comparable to those used to accredit medical schools in the United States. If a country is determined to have comparable medical accreditation standards (i.e., comparable countries), then accredited medical schools in that country may apply to participate in the FFEL program. Throughout this report we refer to the federal student loan program in our findings. Where our findings are specific to the FFEL program, however, we refer to that program by name. To determine the performance of IMGs on licensing examinations, we analyzed medical licensing examination trend data covering the period 1998 through 2008. We interviewed officials of the Educational Commission for Foreign Medical Graduates (ECFMG) and the National Board of Medical Examiners (NBME) to better understand issues related to licensing examination pass rates and international medical graduates. We entered into an agreement with both of these institutions to obtain their medical licensing examination data. To analyze student achievement on medical licensing exams, we calculated pass rates by dividing the number of test takers who passed an exam step by the number of test takers who attempted this exam step in any given calendar year. We excluded data from our analysis when information that identified country and school location was missing. In addition, not all students who attend foreign medical schools are allowed to sit for the exam. To the extent that these students are not reflected in the total number of exam takers, our findings may be overstated. We also interviewed external stakeholders about factors that may affect IMGs' pass rates on the licensing exam. In addition, we reviewed literature on IMGs' performance on the U.S. medical licensing exam. We also obtained information at selected foreign medical schools from administrators and students about efforts to prepare students for the medical licensing exam. To assess Education's monitoring of foreign medical schools' compliance with the licensing examination pass rate requirement, we interviewed Education officials about their monitoring activities and reviewed proposed rule changes to these activities. Additionally, we interviewed ECFMG and NBME officials about their efforts to share aggregate pass rate data with schools. On the basis of pass rate data we obtained from these organizations, we calculated the number of schools in comparable countries whose pass rates met or exceeded the 60 percent requirement for the 3-year period 2006 through 2008. Because school identities were masked in the data we received, we could not differentiate those schools that were FFEL participants from non-FFEL schools. We also looked theoretically at schools' performance on the medical licensing exam with regard to a 75 percent pass rate using 2006 through 2008 data. To determine where international medical graduates obtained residencies and what medical specialties were practiced, we analyzed residency data collected by the Accreditation Council for Graduate Medical Education. We also interviewed HHS officials about geographic areas experiencing physician shortages, and what is known about the extent to which IMGs are practicing in these areas. We interviewed representatives of organizations and a small number of private, for- profit businesses that provide clinical and residency placement services to IMGs for a fee, as well as officials in North Dakota and Texas, who were identified as key information sources in the tracking of IMG residents in rural areas. We identified, obtained, and reviewed previous research on IMGs, including literature on geographic patterns of their field of practice, factors influencing specialty choice, participation in certain visa programs, and the extent to which IMGs practice in primary care and in underserved areas as compared with U. S. medical graduates. To determine what is known about discipline and malpractice involving IMGs (i.e., physician discipline or license actions, as well as malpractice claims), we obtained and analyzed data for the years 2004 to 2008 from the Federation of State Medical Boards and HHS's National Practitioner Data Bank, as well as two states with high populations of IMGs (California and Florida). In addition, we calculated odds ratios and conducted tests to determine whether differences in discipline and malpractice rates between IMGs and U.S. medical graduates were statistically significant. We considered such differences to be significant if they had less than a 5 percent chance of occurring from chance or random fluctuations. Disciplinary action or a malpractice payment[Footnote 13] may occur throughout a physician's career. Because we did not analyze IMGs' or U.S. medical graduates' experience of disciplinary action or malpractice payments by age, date of medical school graduation, or date of licensure, the age distribution of the physicians reflected in our analyses is unknown. Our analysis of malpractice payments is limited to data reported to the National Practitioner Data Bank. While a wide variety of entities are required to report to the National Practitioner Data Bank--including state medical boards, health care facilities, and insurance companies--our analysis may nevertheless not reflect the full extent of medically adverse events. In addition, while we analyzed malpractice payments, we did not analyze the underlying cases to determine the extent to which they involved error or injury. We interviewed officials from three states about the licensing and discipline of physicians (California, Florida, and New York). We also interviewed several experts knowledgeable about physician discipline and malpractice litigation to determine what is known about IMGs' experience of these actions, and to better understand the value and limitations of discipline and malpractice data. We identified, obtained, and reviewed research on discipline and malpractice involving IMGs, as well as research that examined the relationship between malpractice and negligence, and research that compared IMGs with other physicians on selected measures of performance, such as compliance with professional standards. Data Reliability: Because external data were significant to each of our research objectives, we assessed the reliability of the publicly and privately held data obtained from federal departments, agencies, and associations.[Footnote 14] To assess the reliability of each data set, we administered an automated survey form to each data manager or individual assigned primary oversight of the data. Each survey was specifically tailored to the system in question and addressed data uses, internal controls, and data entry practices. Once each survey was completed, we reviewed responses to assess the adequacy of the internal controls and processes in place. In addition, we also tested the integrity of the electronic data we received by searching for possible outliers in the data, invalid variable values, and duplicate records. We determined that each data set was sufficiently reliable for the analytical purposes of this report. Institutional Site Visits: To supplement our data collection and analysis, we conducted site visits to five foreign medical schools: American University of the Caribbean (St. Maarten), Ross University (Dominica), St. George's University (Grenada), Royal College of Surgeons (Ireland), and Poznan University of Medical Sciences (Poland). The schools were selected according to the following criteria: (1) countries deemed to be comparable by the National Committee on Foreign Medical Education and Accreditation with free-standing medical schools, (2) number of free- standing medical schools eligible to participate in the Federal Family Education Loan Program, as defined by the Department of Education, and (3) 2008 loan volume for these institutions, by country. For the site visits, we considered visits only to comparable countries with free-standing medical institutions eligible to participate in the FFEL program. The reason is that Education was not able to provide student enrollment data that distinguishes medical school students from all other academic disciplines at component medical schools. Therefore, by considering only free-standing institutions, we were able to determine a minimum number of medical school students who take out loans to attend a given institution. According to data provided by Education, there are a total of 21 free-standing institutions in 10 countries. The schools visited in the Caribbean represented 3 of the 4 largest institutions in terms of loan volume and students enrolled, and all were for-profit schools. To provide institutional diversity, we also selected two schools in Europe. While both institutions were not-for- profit, one school was a public institution and the other was a private institution. Both schools had smaller student loan volumes and enrollments than the schools in the Caribbean. At each selected school, we met with administrators and asked them to solicit participation of students to participate in focus groups. We conducted a total of eight focus groups that ranged in size from 4 to 14 participants. For each group, we specifically asked for U.S. citizens who also were federal student loan borrowers. We also allowed a few non-federal loan recipients as well as non-U.S. citizens to participate to provide additional perspective. A summary of focus group participants' responses captured during the focus groups can be found in appendix VI. Literature Review: We also reviewed literature published from 1990 to 2009 that included research articles identified through 11 databases such as ERIC, PsycINFO, and Wilson Social Sciences Abstracts. To examine trends in students pursuing medical school abroad, we searched "foreign, medical school," "foreign medical graduates," "trends, physician supply," and other such terms. To examine the factors that influence international medical students' performance on medical licensing exams, we searched terms such as "U.S. medical licensing exam," and "licensing." We also reviewed relevant articles from the annual medical education issue of the Journal of the American Medical Association. We also included in our review articles that were identified by government officials and representatives of professional associations we interviewed. Our bibliography can be found at the end of the report. [End of section] Appendix III: Comparison of Average Cost of Attendance at U.S. and Selected Foreign Medical Schools: [Refer to PDF for image: stacked vertical bar graph] Average U.S Medical School, Private: Tuition/fees: $39,530; Expenses: $57,486; Total: $97,016. Average U.S Medical School, Public: Tuition/fees: $22,068; Expenses: $40,586; Total: $62,654. School A, Europe: Tuition/fees: $62,217; Expenses: $21,579; Total: $83,796. School B, Caribbean: Tuition/fees: $39,600; Expenses: $51,200; Total: $90,800. School C, Caribbean: Tuition/fees: $32,166; Expenses: $28,206; Total: $60,372. School D, Caribbean: Tuition/fees: $42,108; Expenses: $30,762; Total: $72,870. School E, Europe: Tuition/fees: $15,925; Expenses: $14,909; Total: $30,834. Source: GAO analysis of Institutional data and published American Association of Medical Colleges, based on academic year 2008-2009 data. Notes: Expenses include such cost as room and board, books, transportation, and other miscellaneous costs identified by institutions, which may vary. Cost expressed in United States dollars. [End of figure] [End of section] Appendix IV: Performance on Licensing Exam by Number of Attempts on Each Exam Step, and by Nationality of IMG: Figure 1: Licensing Exam Pass Rates by Number of Attempts on Each Exam Step, 1998 through 2008: [Refer to PDF for image: horizontal bar graph] Step 1: Attempt: 1; U.S.-Educated graduates: Passed: 97%; Attempted: 178,744; IMGs: Passed: 64%; Attempted: 168,800. Attempt: 2; U.S.-Educated graduates: Passed: 66%; Attempted: 8,583; IMGs: Passed: 42%; Attempted: 13,524. Attempt: 3; U.S.-Educated graduates: Passed: 66%; Attempted: 314; IMGs: Passed: 41%; Attempted: 649. Attempt: 4; U.S.-Educated graduates: Passed: 100%; Attempted: 1; IMGs: Passed: 100%; Attempted: 1. Step 2: Clinical Knowledge; Attempt: 1; U.S.-Educated graduates: Passed: 97%; Attempted: 181,201; IMGs: Passed: 72%; Attempted: 140,254. Attempt: 2; U.S.-Educated graduates: Passed: 75%; Attempted: 4,425; IMGs: Passed: 48%; Attempted: 11,330. Attempt: 3; U.S.-Educated graduates: Passed: 71%; Attempted: 302; IMGs: Passed: 52%; Attempted: 455. Attempt: 4; U.S.-Educated graduates: Passed: 80%; Attempted: 5; IMGs: Passed: 29%; Attempted: 7. Step 2: Clinical Skills; Attempt: 1; U.S.-Educated graduates: Passed: 98%; Attempted: 74,974; IMGs: Passed: 82%; Attempted: 62,575. Attempt: 2; U.S.-Educated graduates: Passed: 95%; Attempted: 665; IMGs: Passed: 77%; Attempted: 4,027. Attempt: 3; U.S.-Educated graduates: Passed: 100%; Attempted: 7; IMGs: Passed: 63%; Attempted: 225. Attempt: 4; U.S.-Educated graduates: Passed: 0; Attempted: 0; IMGs: Passed: 0; Attempted: 0. Step 3: Attempt: 1; U.S.-Educated graduates: Passed: 96%; Attempted: 168,280; IMGs: Passed: 71%; Attempted: 101,464. Attempt: 2; U.S.-Educated graduates: Passed: 75%; Attempted: 4,069; IMGs: Passed: 54%; Attempted: 12,174. Attempt: 3; U.S.-Educated graduates: Passed: 68%; Attempted: 333; IMGs: Passed: 56%; Attempted: 1,111. Attempt: 4; U.S.-Educated graduates: Passed: 75%; Attempted: 12; IMGs: Passed: 23%; Attempted: 23. Sources: GAO analysis of Educational Commission for Foreign Medical Graduates and National Board of Medical Examiners data. Note: All test takers are referred to as "graduates"; however, students are also eligible to take the Step 1, Step 2 clinical knowledge, and Step 2 clinical skills exams while they are still enrolled in medical school. Percentages were not indicated for attempts with small numbers of test-takers. [End of figure] Among IMGs themselves, pass rates for foreign IMGs have generally exceeded those for U.S. citizen IMGs on Step 1 and Step 2, Clinical Knowledge, although U.S. citizens have achieved higher pass rates on the Step 2 Clinical Skills exam and their lead on the Step 3 exam has steadily been decreasing. (See figures 2 and 3) Figure 2: Comparison of First-Time Pass Rates for U.S. Citizen IMGs and Foreign IMGs on Step 1 (Basic Science) and Step 2, Clinical Knowledge, 1998 through 2008: [Refer to PDF for image: 2 multiple line graphs] Step 1 (Basic Science): First-time pass rate, 1998: Foreign IMGs: 62%; U.S. citizen IMGs: 63%. First-time pass rate, 2008: Foreign IMGs: 71%; U.S. citizen IMGs: 68%. 14,892 graduates tested in 2008: U.S. citizen IMGs: 21% (3,123); Foreign IMGs: 79% (11,769). Step 2, Clinical Knowledge: First-time pass rate, 1998: Foreign IMGs: 54%; U.S. citizen IMGs: 57%. First-time pass rate, 2008: Foreign IMGs: 84%; U.S. citizen IMGs: 75%. 12,351 graduates tested in 2008: U.S. citizen IMGs: 20% (2,493); Foreign IMGs: 80% (9,858). Sources: GAO analysis of Educational Commission for Foreign Medical Graduates and National Board of Medical Examiners data. Note: All test takers are referred to as "graduates"; however, students are eligible to take the Step 1 and Step 2 clinical knowledge exams while they are still enrolled in medical school. [End of figure] Figure 3: Comparison of First-Time Pass Rates for U.S. Citizen IMGs and Foreign IMGs on Step 2, Clinical Skills, 2004 through 2008, and Step 3 (on Delivery of General Medical Care), 1998 through 2008: [Refer to PDF for image: 2 multiple line graphs] Step 2, Clinical Skills: First-time pass rate, 1998: Foreign IMGs: 82%; U.S. citizen IMGs: 88%. First-time pass rate, 2008: Foreign IMGs: 69%; U.S. citizen IMGs: 81%. 13,677 graduates tested in 2008: U.S. citizen IMGs: 21% (2,811); Foreign IMGs: 79% (10,866). Step 3, Delivery of General Medical Care: First-time pass rate, 1998: Foreign IMGs: 54%; U.S. citizen IMGs: 72%. First-time pass rate, 2008: Foreign IMGs: 80%; U.S. citizen IMGs: 78%. 9,373 graduates tested in 2008: U.S. citizen IMGs: 27% (2,554); Foreign IMGs: 73% (6,819). Sources: GAO analysis of Educational Commission for Foreign Medical Graduates and National Board of Medical Examiners data. [End of figure] Notes: Test takers are referred to as "graduates"; however, students are also eligible to take the Step 2 clinical skills exam. This exam became a certification requirement on June 14, 2004, so pass rate data for the Step 2 clinical skills exam are available only for exam year 2004 and onward. According to ECFMG and NBME officials, there has been a clinical skills exam requirement for ECFMG certification since July 1998. Prior to then, ECFMG administered the ECFMG Clinical Skills Assessment (CSA), starting in July 1998, as a requirement for ECFMG certification. [End of figure] [End of section] Appendix V: International Medical Graduates and Residencies: Table 1: Residency Programs by State, Academic Year 2008-2009: State: Alabama; Number of programs: 110; Percentage of all programs: 1.3. State: Alaska; Number of programs: 1; Percentage of all programs: 0.0. State: Arizona; Number of programs: 110; Percentage of all programs: 1.3. State: Arkansas; Number of programs: 63; Percentage of all programs: 0.7. State: California; Number of programs: 753; Percentage of all programs: 8.6. State: Colorado; Number of programs: 94; Percentage of all programs: 1.1. State: Connecticut; Number of programs: 159; Percentage of all programs: 1.8. State: District of Columbia; Number of programs: 159; Percentage of all programs: 1.8. State: Delaware; Number of programs: 27; Percentage of all programs: 0.3. State: Florida; Number of programs: 293; Percentage of all programs: 3.4. State: Georgia; Number of programs: 164; Percentage of all programs: 1.9. State: Hawaii; Number of programs: 30; Percentage of all programs: 0.3. State: Idaho; Number of programs: 4; Percentage of all programs: 0.0. State: Illinois; Number of programs: 411; Percentage of all programs: 4.7. State: Indiana; Number of programs: 100; Percentage of all programs: 1.1. State: Iowa; Number of programs: 80; Percentage of all programs: 0.9. State: Kansas; Number of programs: 58; Percentage of all programs: 0.7. State: Kentucky; Number of programs: 99; Percentage of all programs: 1.1. State: Louisiana; Number of programs: 143; Percentage of all programs: 1.6. State: Maine; Number of programs: 21; Percentage of all programs: 0.2. State: Maryland; Number of programs: 214; Percentage of all programs: 2.5. State: Massachusetts; Number of programs: 387; Percentage of all programs: 4.4. State: Michigan; Number of programs: 347; Percentage of all programs: 4.0. State: Minnesota; Number of programs: 174; Percentage of all programs: 2.0. State: Mississippi; Number of programs: 43; Percentage of all programs: 0.5. State: Missouri; Number of programs: 212; Percentage of all programs: 2.4. State: Montana; Number of programs: 2; Percentage of all programs: 0.0. State: Nebraska; Number of programs: 55; Percentage of all programs: 0.6. State: Nevada; Number of programs: 18; Percentage of all programs: 0.2. State: New Hampshire; Number of programs: 45; Percentage of all programs: 0.5. State: New Jersey; Number of programs: 196; Percentage of all programs: 2.2. State: New Mexico; Number of programs: 53; Percentage of all programs: 0.6. State: New York; Number of programs: 1,105; Percentage of all programs: 12.7. State: North Carolina; Number of programs: 247; Percentage of all programs: 2.8. State: North Dakota; Number of programs: 7; Percentage of all programs: 0.1. State: Ohio; Number of programs: 447; Percentage of all programs: 5.1. State: Oklahoma; Number of programs: 65; Percentage of all programs: 0.7. State: Oregon; Number of programs: 74; Percentage of all programs: 0.8. State: Pennsylvania; Number of programs: 569; Percentage of all programs: 6.5. State: Puerto Rico; Number of programs: 67; Percentage of all programs: 0.8. State: Rhode Island; Number of programs: 59; Percentage of all programs: 0.7. State: South Carolina; Number of programs: 95; Percentage of all programs: 1.1. State: South Dakota; Number of programs: 7; Percentage of all programs: 0.1. State: Tennessee; Number of programs: 180; Percentage of all programs: 2.1. State: Texas; Number of programs: 544; Percentage of all programs: 6.2. State: Utah; Number of programs: 66; Percentage of all programs: 0.8. State: Vermont; Number of programs: 35; Percentage of all programs: 0.4. State: Virginia; Number of programs: 187; Percentage of all programs: 2.1. State: Washington; Number of programs: 131; Percentage of all programs: 1.5. State: West Virginia; Number of programs: 59; Percentage of all programs: 0.7. State: Wisconsin; Number of programs: 163; Percentage of all programs: 1.9. State: Wyoming; Number of programs: 2; Percentage of all programs: 0.0. State: Total; Number of programs: 8,734; Percentage of all programs: 100. Source: GAO analysis of Accreditation Council for Graduate Medical Education (ACGME) data. [End of table] Figure 4: Percentage of IMGs in Residencies as a Percentage of All Residents by State, Academic Year 2008-2009: [Refer to PDF for image: illustrated U.S. map] State average: 24.7%; Alabama: 19.8%; Alaska: 17.1%; Arizona: 23.3%; Arkansas: 30.9%; California: 12.0%; Colorado: 6.2%; Connecticut: 41.8%; Delaware: 16.0%; District of Columbia: 23.7%; Florida: 29.6%; Georgia: 25.7%; Hawaii: 12.8%; Idaho: 4.0%; Illinois: 27.9%; Indiana: 19.2%; Iowa: 21.9%; Kansas: 22.8%; Kentucky: 20.9%; Louisiana: 37.5%; Maine: 20.7%; Maryland: 23.5%; Massachusetts: 21.3%; Michigan: 40.6%; Minnesota: 22.0%; Mississippi: 20.0%; Missouri: 23.8%; Montana: 5.0%; Nebraska: 25.9%; Nevada: 47.9%; New Hampshire: 15.2%; New Jersey: 58.1%; New Mexico: 21.0%; New York: 44.6%; North Carolina: 11.1%; North Dakota: 62.6%; Ohio: 29.5%; Oklahoma: 33.2%; Oregon: 7.1%; Pennsylvania: 28.8%; Puerto Rico: 31.5%; Rhode Island: 20.3%; South Carolina: 13.1%; South Dakota: 27.8%; Tennessee: 24.2%; Texas: 21.0%; Utah: 9.3%; Vermont: 20.5%; Virginia: 17.9%; Washington: 8.8%; West Virginia: 37.3%; Wisconsin: 23.3%; Wyoming: 55.0%. Sources: GAO analysis of Accreditation Council for Graduate Medical Education (ACGME) data; The National Atlas of the United States (map). [End of figure] [End of section] Appendix VI: Summary of Responses by Focus Group Participants at the Five Foreign Medical Schools We Visited: Although findings are not generalizable to all U.S. students studying in foreign medical schools, we conducted focus groups with 82 enrolled U.S. citizens or permanent residents at five schools we visited to gain a better understanding of and perspective on students who attend foreign medical schools. Table 2 shows a summary of the number of students who participated in our focus groups by school. Table 2: Number of Participants in Student Focus Groups, by School: School: School A; Number of focus groups conducted: 2; Number of participants: 21. School: School B; Number of focus groups conducted: 2; Number of participants: 19. School: School C; Number of focus groups conducted: 2; Number of participants: 25. School: School D; Number of focus groups conducted: 1; Number of participants: 7. School: School E; Number of focus groups conducted: 1; Number of participants: 10. School: Total; Number of focus groups conducted: 8; Number of participants: 82. Source: GAO analysis of focus group survey responses. [End of table] Demographics: We obtained basic demographic information on students who participated in our focus groups to learn more about their year of medical school, gender, age, and citizenship status. This information has been included to provide some perspective on who participated in our focus groups; however, this information is not generalizable to all IMG students. * Sixty-eight percent of the students who participated in our focus groups were enrolled in their second year of medical school abroad. Another 16 percent were in the first year of study. * Sixty-six percent of participants were male and the remaining 39 percent were female. * Almost half the participants were between the ages of 25 and 34. * Over two-thirds of participants were U.S. citizens and 13 percent had dual citizenship with the United States and usually the country where the medical school was located. Reasons for Attending a Foreign Medical School: We asked participants in our focus groups to tell us about some of the reasons they decided to pursue a medical education outside the United States and whether they had applied to any other medical schools. Table 3 summarizes our questions and the most frequent responses provided by participants in our focus groups. Table 3: Decision to Attend a Foreign Medical School: Question: What are some of the reasons you decided to pursue your medical education outside the U.S.?; Most frequent responses by focus group participants: * Noncompetitive grade point average or Medical College Admission Test; * Perceived as too old for U.S. medical schools; * Not accepted into U.S. medical schools; * Did not want to wait another year to apply or retake admission test; * Location of school/living abroad; * Only opportunity to achieve goal of becoming a doctor. Question: What are some of the reasons you chose this medical school?; Most frequent responses by focus group participants: * Opportunity for clinical rotations in U.S with hospitals affiliated with the medical school; * School's reputation; * Accelerated program or other admissions requirements enable student to finish school quickly; * Availability of federal financial aid or cost of attendance. Question: How did you learn or find out about this school?; Most frequent responses by focus group participants: Internet (student- led bulletin boards); Word of mouth through friends, family, other doctors. Question: Did you apply to U.S. medical schools?; If so, how many U. S. medical schools did you apply to?; Most frequent responses by focus group participants: Fifty-nine of the 82 students participating in the focus groups responded that they had initially applied to U.S. medical schools; * Those attending School A: 6.6 schools; * Those attending School B: 7.4 schools; * Those attending School C: 12.6 schools; * Those attending School D: 14.5 schools; * Those attending School E: 4.2 schools. Question: Were you accepted by a U.S. medical school?; Most frequent responses by focus group participants: Ten of the 82 participants who participated in our focus groups indicated that they had also been accepted by a U.S. medical school. Question: Other than the medical school you are enrolled in, did you apply to any other medical schools outside the U.S.?; Most frequent responses by focus group participants: Forty-four of the 82 focus group participants responded that they applied by another foreign medical school in addition to the school where they were enrolled. Source: GAO analysis of focus group survey responses. [End of table] Financing Medical Education: We asked focus group participants to assess the relative importance of various forms of financial aid to help finance their medical education. In addition, we asked them about the extent to which cost of attendance was a factor in their application decision. For those focus group participants who have received federal student loans, we also asked them about their experience in working with the school's financial aid office. Participants in our focus groups ranked the various financial aid sources in the following order, from most to least important: * federal student aid, * personal savings/family resources, * private loans, * scholarships, and: * Veterans Affairs Benefits/GI Bill: While the cost of medical school was a factor in the application decision, the availability of federal financial aid was considered essential and without which students would not have been able to afford medical school. Some notable quotes included the following: * "My dream would have died without Department of Education loans." * "Federal assistance makes this possible. I am worried that the program will go away." * "Without federal loans, most of the students could not attend [school name]." * "It is vital to maintain this program that has not only been proven by the long history of graduates who have returned to the U.S., but also been shown to be necessary to help relieve the shortage of physicians in the United States." Several participants said that they lack consumer information on expected debt level, and that there is no reliable source of such information to help decide which medical school to attend. * One participant explicitly asked about the school's pass rate on the licensing exam but got a vague answer from school officials. "[Name of school's] marketing statement says that the school's pass rate is 98 percent on Step 1. Even U.S. schools don't hit this mark. But [school name] only counts those students who actually sit for the exam and don't drop out. They warn you but only after you get here and sign for the loan. [I] would have no problem if they said it was cutthroat environment. [Your] eyes are open and you know what to expect. People drop left and right. You can be 25 years old, $75,000 in debt, and [have to leave] in the 3rd semester with nothing. There should be truth in advertising … schools should show their numbers. This is ethical. [I] knew a couple of students with families who failed out. [They] uprooted [their] life only to get creamed." Focus group participants reported that, in general, obtaining financial aid was a smooth process and that the school's financial aid office was very helpful in disbursing the aid on time, and in responding to students' inquiries. One participant noted that the school even requires students to obtain financial counseling before getting any loans. Residency and Practice: We asked focus group participants about their plans to pursue a residency and also their long-term career plans. Almost all participants in our focus groups--69 of the 82--indicated that they would eventually return to the United States to either pursue a residency or practice medicine. While 54 of these 69 participants who said that they would eventually return to the United States indicated that they preferred to pursue a specialty during their residency, 41 indicated that they would also consider primary care as either their first choice or one of their choices. Three of the 69 participants stated they planned to go into research. Participants cited a number of factors to consider when deciding which field of medicine to practice, including scores on medical licensing exams, earnings potential (which affected ability to pay off student loans), physician workload, flexible schedule, and having to deal with health insurance companies. While most participants in our focus groups said they would like to go into a specialty, several acknowledged "the reality" that most graduates of foreign medical schools end up in primary care because residency slots in specialties are extremely competitive and limited. One participant stated, "At a time when there is an increased need for primary care physicians in the U.S., it should be noted that a substantial number of our graduates obtain primary care residencies." When talking specifically about where they hope to obtain a residency or eventually practice, most participants--69 of the 82--indicated they preferred to return to the South Atlantic states, and another 31 participants indicated that New England or the Mid-Atlantic states were also a possibility. Ten of the 82 participants told us that they had plans to serve in medically underserved communities or help specific ethnic populations. Among the factors influencing their residency decision were being close to family, desire to help underserved populations, whether the residency program accepted international medical graduates, and availability of loan forgiveness programs. Quality of Instruction: We obtained focus group participants' perspectives on the quality of instruction at the school, including the benefits of attending a foreign medical school. Many participants said that, overall, the permanent professors were knowledgeable and accomplished, but a few students at some schools said that instruction provided by visiting professors varied in quality. A few notable quotes included the following: * "We do an evaluation at the end of the semester. I think they [i.e., the school] pay attention to that. If they [i.e., professors] are not good at teaching, you don't see them anymore." * "Quality is very high here. The rotations--doctors you're paired with, take the effort to teach. [That is] not the case in the U.S. [I] feel I'm learning more." * "All of the faculty are non-practicing. They are available all the time. They hold your hand more." * "In the U.S., many of the professors are practicing doctors or do research. It's different at [name of school]...The emphasis is on teaching so that makes them more accessible." * "They [i.e., visiting professors] may really know a particular subject. They may really know the research, but when asked 'how,' they can't explain it." * "Teaching quality is highly variable. The awful ones were all visiting professors. They're here on vacation for 2-4 weeks from the U.S. [They were] brought only because [they] happened to be available, not because [they were] good. Visiting professors make up 20 percent of the faculty. Most have been coming here for years." Focus group participants at one school told us that they valued the campus' proximity to a teaching hospital and the resulting opportunities to gain early exposure to patients. In contrast, a focus group participant at another school commented that the school did not have a relationship with surrounding hospitals, which meant that students would have to wait for a clinical rotation to gain exposure to patients. Some focus group participants at for-profit institutions said that the instructional materials were covered quickly or that to prepare for medical licensing exams students needed to undertake self-study. Several focus group participants at one school said they would have liked to have information on student performance on the medical licensing exam to help them select which foreign medical school to attend. * Participants discussed various benefits of attending a foreign medical school that included interacting with a wide spectrum of patients, developing cultural sensitivity, and obtaining insights into different health care systems. Quality of Educational Facilities: In addition to asking them to evaluate the quality of instruction, we asked focus group participants to comment on the quality of the educational facilities such as classrooms, laboratories, libraries, and equipment. We heard mixed comments regarding educational facilities and whether they adequately met students' needs. Some notable quotes included the following: * "Comparatively speaking to U.S. medical schools, classrooms are good with flat screens and projectors." * "The university has invested in some new buildings and these are equipped with state-of-the-art technology." * "You can get video streaming of classes and online learning...but the system's been getting crowded." * "Cadavers are an advantage. The number of cadavers available allows for very small groups of four students. Other schools often assign 12 students to a cadaver." * "Great that they have access to cadavers. Many colleges do not have these." * "The library is small but has all the books we need." * "Library is horrible. Very poor electronic resources but large journal collection. Library is small given large student population." * "[I'm] angry [that] pathology is important, [but] the microscopes are from 40 years ago." Several focus group participants noted that class size was an issue and that the infrastructure was not keeping up with enrollment growth. * "Classes [are] expanding faster than infrastructure. They try to keep up with it, but bigger classes affect individual attention, divert resources from other things, stretches resources." * "Average class size is 500 students and increasing toward 600." * "It may not take away from me to be in a big class. I don't go to class. Lectures are recorded and placed online for people to watch." [Student makes the choice to stay home and watch lectures and considers it a "waste of time to go to class".] * "Teachers don't agree with the administration's decision to expand classes. [They think it is] unwieldy ... [and that there is] too much of a focus on business." * "Feeling resentful of increase in enrollment without adequate infrastructure. [Name of school] is constantly expanding because it's a business. But people get what they need from the facilities." Another participant provided the following contrasting perspective: * "While it is certainly true that there has been recent expansion in class size, the quality of education has not been affected. I can honestly say that I have never felt that I did not have access to my instructors or clinical tutors. It makes no difference to me if there are 100, 200, or 500 people in the lecture hall. What does matter is how many people are in my lab groups. For instance, my anatomy dissection group had four people in it [similar to or better than U.S. schools]…These sessions provide all students with ample opportunity for discussion and questions." [End of section] Appendix VII: Comments from the U.S. Department of Education: United States Department Of Education: Office Of Postsecondary Education: 1990 K St. NW: Washington, Dc 20006: [hyperlink, http://www.ed.gov] The Department of Education's mission is to promote student achievement and preparation for global competitiveness by fostering educational excellence and ensuring equal access. June 22, 2010: Mr. George A. Scott: Director: Education, Workforce, and Income Security Issues: United States Government Accountability Office: Washington, DC 20548: Dear Mr. Scott: Thank you for providing the Department of Education with a draft copy of the U.S. Government Accountability Office's (GAO's) report entitled, "Foreign Medical Schools: Education Should Improve Monitoring of Schools That Participate in the Federal Student Loan Program" (GAO-10-412). This study looks at the performance of international medical graduates (IMGs) to evaluate the return on investment for the provision of federal student financial aid funds to students attending foreign medical schools. Specifically, the study looks at the amount of federal loan dollars awarded to students attending foreign medical schools, data on U.S. licensing examination pass rates for foreign medical schools, and the Department's enforcement of the statutory pass rate threshold for foreign medical schools participating in the federal student loan program; IMGs' success in obtaining residencies in the U.S.; and discipline and malpractice involving IMGs. Following are GAO's recommendations and the Department of Education, Office of Postsecondary Education's response: Recommendation 1: Collect consumer information, such as aggregate student debt level and graduation rates, from foreign medical schools participating in the federal student loan program and make it publicly available to students and their families. Response: We agree with the recommendation. We will begin making plans, consistent with the Paperwork Reduction Act, to initiate such a data collection for foreign medical schools. This will also require data systems enhancements to capture the education program of each student within those institutions that offer more than a medical program. Recommendation 2: Require foreign medical schools to submit aggregate institutional pass rate data to the Department annually. Response: We agree with the recommendation. The Department will send a letter to foreign medical schools asking for United States Medical Licensing Examination (USMLE) pass rate information starting with exams taken during the award year ending June 30, 2010. Recommendation 3: Verify data submitted by schools, for example by entering into a data sharing agreement with the testing organizations. Response: We agree with the recommendation. The Department should have a mechanism to verify the USMLE pass rate data received from foreign medical schools, although we note this will require the cooperation of the private organizations that administer the exams. To this end, we will again attempt to establish a data sharing agreement with those organizations. Recommendation 4: Evaluate the potential impact of the 75 percent pass rate requirement on school participation in the federal student loan program and advise Congress of any needed revisions to the requirement. Response: We agree with the recommendation. We have already begun to evaluate the potential impact of the 75 percent pass rate requirement on foreign medical schools through the development of regulations to implement the statutory requirement. The forthcoming Notice of Proposed Rulemaking (NPRM), scheduled for publication this summer, was developed through the negotiated rulemaking process. This process involved gathering input from the public through three public hearings and forming a negotiated rulemaking committee comprising individuals representing key stakeholder constituencies for issues related to foreign institutions participating in the federal student financial aid programs. The 75 percent pass rate provision received a significant amount of discussion, and the resulting NPRM will propose regulatory language to properly implement the statute, while addressing concerns raised by the negotiating committee. Public comment on the NPRM will provide the Department with additional information necessary for its continued evaluation, as will the collection and verification of pass rate data. I appreciate your examination of this important issue. The Department of Education is committed to the continued evaluation of foreign medical schools to effect the best use of federal student financial aid funds. Sincerely, Signed by: David A. Bergeron: Acting Deputy Assistant Secretary for Policy, Planning and Innovation: Office of Postsecondary Education: [End of section] Appendix VIII: GAO Contact and Staff Acknowledgments: GAO Contact: George A. Scott, Director, (202) 512-7215 or scottg@gao.gov: Staff Acknowledgments: In addition to the contact named above, Meeta Engle, Assistant Director; Carla Craddock, Analyst-in-Charge; James Bennett; Susan Bernstein; Alexander Galuten; Lauren Gilbertson; Kristen Jones; Mitchell Karpman; Kathleen King; Ruben Montes de Oca; Christopher Morehouse; Terry Richardson; and Cynthia Saunders and made key contributions to this report. [End of section] Bibliography: Overview of IMGs: Boulet, John R., Stephen Seeling, John Norcini, Danette McKinley, and Richard Cooper. "The Contribution of US Citizen International Medical Graduates to the US Physician Workforce." Presented at the 2009 Physician Workforce Research Conference, Washington, D.C., April 30 - May 1, 2009. Johnson, Karin, Amy Hagopian, Catherine Veninga, and L. Gary Hart. "The Changing Geography of Americans Graduating from Foreign Medical Schools." Academic Medicine, vol. 81, no. 2 (2006). Licensing Exam Performance: Bates, Joanna, and Rodney Andrew. "Untangling the Roots of Some IMGs' Poor Academic Performance." Academic Medicine, vol. 76, no. 1 (2001). Boulet, John R., Richard A. Cooper, Stephen S. Seeling, John J. Norcini, and Danette W. McKinley. "U.S. Citizens Who Obtain Their Medical Degrees Abroad: An Overview, 1992 - 2006." Health Affairs, vol. 28, no. 1 (2009). Harik, Polina, Brian E. Clauser, Irina Grabovsky, Melissa J. Margolis, Gerald F. Dillion, and John R. Boulet. "Relationships among Subcomponents of the USMLE Step 2 Clinical Skills Examination, The Step 1, and the Step 2 Clinical Knowledge Examinations." Academic Medicine, vol. 81, no. 10 (2006). Norcini, John, M. Brownell Anderson, and Danette W. McKinley. "The Medical Education of United States Citizens Who Train Abroad." Surgery, vol. 140, no. 3 (2006). Van Zanten, Marta and John R. Boulet. "Medical Education in the Caribbean: Variability in Educational Commission for Foreign Medical Graduate Certification Rates and United States Medical Licensing Examination Attempts." Academic Medicine, vol. 84, no. 10 (2009). Van Zanten, Marta, John R. Boulet, Danette W. McKinley, Andre DeChamplain, and Ann C. Jobe. "Assessing the Communication and Interpersonal Skills of Graduates of International Medical Schools as Part of the United States Medical Licensing Exam (USMLE) Step 2 Clinical Skills (CS) Exam." Academic Medicine, vol. 82, no. 10 (2007). Residency and Practice: American Medical Association. "International Medical Graduates in the U.S. Workforce: A Discussion Paper." (2008). Fordyce, Meredith A., Frederick M. Chen, Mark P. Doescher, and L. Gary Hart. "2005 Physician Supply and Distribution in Rural Areas of the United States." WWAMI Center for Health and Workforce Studies at the University of Washington Department of Family Medicine, Final Report #116 (2007). Hagopian, Amy, Matthew J. Thompson, Emily Kaltenbach, and L. Gary Hart. "The Role of International Medical Graduates in America's Small Rural Critical Access Hospitals." The Journal of Rural Health, vol. 20, no. 1 (2004). Hagopian, Amy, Matthew J. Thompson, Karin E. Johnson, and Denise M. Lishner. "International Medical Graduates in the United States: A Review of the Literature 1995-2003." WWAMI Center for Health and Workforce Studies at the University of Washington Department of Family Medicine, Working Paper #83 (2003). Hing, Esther, and Susan Lin. "Role of International Medical Graduates Providing Office-based Medical Care: United States, 2005-06." National Center for Health Statistics Data Brief (2008). Ogunyemi, Dotun, and Ron Edelstein. "Career Intentions of U.S. Medical Graduates and International Medical Graduates." Journal of the National Medical Association, vol. 99, no. 10 (2007). Salsberg, Edward, and Joseph Nolan. "The Posttraining Plans of International Medical Graduates and US Medical Graduates in New York State." Journal of the American Medical Association, vol. 283, no. 13 (2000). Discipline and Malpractice: Kohatsu, Neal D., Dawn Gould, Leslie K. Ross, and Patrick J. Fox. "Characteristics Associated with Physician Discipline: A Case-Control Study." Archives of Internal Medicine, vol. 164 (2004). Lee, Shoou-Yih D., William H. Dow, Virginia Wang, and Jonathan B. VanGeest. "Use of Deceptive Tactics in Physician Practices: Are There Differences between International and U.S. Medical Graduates?" Health Policy, vol. 67 (2004). Localio, A. Russell, Ann G. Lawthers, Troyen A. Brennan, Nan M. Laird, Liesi E. Hebert, Lynn M. Peterson, Joseph P. Newhouse, Paul C. Weiler, and Howard H. Hyatt. "Relation Between Malpractice Claims and Adverse Events Due to Negligence: Results of the Harvard Medical Practice Study III." The New England Journal of Medicine, vol. 325, no. 4 (1991). Mick, Stephen S., and Maureen E. Comfort. "The Quality of Care of International Medical Graduates: How Does It Compare to That of U.S. Medical Graduates?" Medical Care Research and Review, vol. 54, no. 4 (1997). Studdert, David M., Michelle M. Mello, Atul A. Gawande, Tejal K. Gandhi, Allen Kachalia, Catherine Yoon, Ann Louise Puopolo, and Troyen A. Brennan. "Claims, Errors, and Compensation Payments in Medical Malpractice Litigation." The New England Journal of Medicine, vol. 354, no. 19 (2006). Studdert, David M., Eric J. Thomas, Helen R. Burstin, Brett I.W. Zbar, E. John Orav, and Troyen A. Brennan. "Negligent Care and Malpractice Claiming Behavior in Utah and Colorado." Medical Care, vol. 38, no. 3 (2000). U.S. Department of Education, National Committee on Foreign Medical Education and Accreditation. Report to the U.S. Congress by the National Committee on Foreign Medical Education and Accreditation Recommending Institutional Eligibility Criteria for Participation by Certain Foreign Medical Schools in the Federal Family Education Loan Program. 2009. Wolfe, Sidney M., and Kate Resnevic. "Public Citizen's Health Research Group Ranking of the Rate of State Medical Boards' Serious Disciplinary Actions, 2006-2008 (HRG Publication#1868)." (2009). [hyperlink, http://www.citizen.org/publications/publicationredirect.cfm?ID=7652] (accessed Jan. 6, 2010). [End of section] Related GAO Products: Graduate Medical Education: Trends in Training and Student Debt. [hyperlink, http://www.gao.gov/products/GAO-09-438R]. Washington, D.C.: May 4, 2009. Primary Care Professionals: Recent Supply Trends, Projections, and Valuation of Services. [hyperlink, http://www.gao.gov/products/GAO-08-472T]. Washington, D.C.: February 12, 2008. Foreign Physicians: Data on Use of J-1 Visa Waivers Needed to Better Address Physician Shortages. [hyperlink, http://www.gao.gov/products/GAO-07-52]. Washington, D.C.: November 30, 2006. Foreign Physicians: Preliminary Findings on the Use of J-1 Visa Waivers to Practice in Underserved Areas. [hyperlink, http://www.gao.gov/products/GAO-06-773T]. Washington, D.C.: May 18, 2006. Student Loans and Foreign Schools: Assessing Risks Could Help Education Reduce Program Vulnerability. [hyperlink, http://www.gao.gov/products/GAO-03-647]. Washington, D.C.: July 25, 2003. Foreign Physicians: Exchange Visitor Program Becoming Major Route to Practicing in U.S. Underserved Areas. [hyperlink, http://www.gao.gov/products/GAO/HEHS-97-26]. Washington, D.C.: December 30, 1996. Student Loans: Millions Loaned Inappropriately to U.S. Nationals at Foreign Medical Schools. [hyperlink, http://www.gao.gov/products/GAO/HEHS-94-28]. Washington, D.C.: January 21, 1994. Medical Licensing By Endorsement: Requirements Differ for Graduates of Foreign and U.S. Medical Schools. [hyperlink, http://www.gao.gov/products/GAO/HRD-90-120]. Washington, D.C.: May 17, 1990. Medical Malpractice: Characteristics of Claims Closed in 1984. [hyperlink, http://www.gao.gov/products/GAO/HRD-87-55]. Washington, D.C.: April 22, 1987. Federal, State, and Private Activities Pertaining to U.S. Graduates of Foreign Medical Schools. [hyperlink, http://www.gao.gov/products/GAO/HRD-85-112]. Washington, D.C.: September 27, 1985. [End of section] Footnotes: [1] Until recently, these student loans were made through the Federal Family Education Loan program, which was the only federal student financial aid program in which foreign schools could participate. Under newly enacted legislation, the SAFRA Act, which was included in the Health Care and Education Reconciliation Act of 2010, (Pub. L. No. 111-152 (2010)), the FFEL program will terminate June 30, 2010, after which no new loans will be made under the FFEL program. The SAFRA Act extends the availability of loans under the William D. Ford Federal Direct Loan (Direct Loan) program to students at eligible foreign institutions. Thus, beginning in July 2010, students at foreign medical schools will receive new loans through the Direct Loan program, instead of the FFEL program. Throughout this report we refer to the federal student loan program in our findings. Where our findings are specific to the FFEL program, however, we refer to that program by name. [2] Pub. L. No. 110-315, § 1101 (2008). A similar mandate was directed at the Department of Education's National Committee on Foreign Medical Education and Accreditation. The committee issued its report to Congress in 2009. [3] Another $1 billion went to students attending foreign schools with a medical program; however, it is not known what portion of this amount was used to pursue a medical degree as opposed to some other discipline because the Department of Education does not track loan volume according to academic discipline. [4] GRADPlus loans are part of the federal student loan program and are non-need-based loans that graduate and professional students may borrow irrespective of their expected financial contributions to paying educational expenses. Funds borrowed are limited by other financial assistance received, such as other FFEL loans, and a student's cost of attendance. [5] By comparison, the median loan debt for students pursuing medical degrees in the United States was $155,000 in 2008. [6] The Mid-Atlantic states are New Jersey, New York, and Pennsylvania. The Mountain states are Arizona, Colorado, Idaho, Montana, Nevada, New Mexico, Utah, and Wyoming. [7] Core primary care specialties are internal medicine, family medicine, pediatrics, or internal medicine/pediatrics. [8] Malpractice payments are a monetary exchange as a result of a settlement or judgment of a written complaint or claim demanding payment based on a physician's provision of or failure to provide health care services, and may include, but is not limited to, the filing of a cause of action, based on the law of tort, brought in any State or Federal Court or other adjudicative body. [9] These academic years were chosen because they coincide with reauthorizations to the Higher Education Act of 1965. [10] Overall, we interviewed officials from four states--California, Florida, New Jersey, and New York--and conducted data reliability assessments of their disciplinary data. On the basis of the outcome of these assessments, we included data from California and Florida in this report. [11] These academic years were chosen because they coincide with reauthorizations of the Higher Education Act of 1965. [12] According to Education's definition, free-standing institutions are schools whose principal offering is medical education in contrast to component medical schools that are part of a larger university system. [13] Malpractice payments are a monetary exchange as a result of a settlement or judgment of a written complaint or claim demanding payment based on a physician's provision of or failure to provide health care services, and may include, but is not limited to, the filing of a cause of action, based on the law of tort, brought in any State or Federal Court or other adjudicative body. [14] GAO, Assessing the Reliability of Computer-Processed Data, [hyperlink, http://www.gao.gov/products/GAO-09-680G] (Washington, D.C.: July, 2009). [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: