This is the accessible text file for GAO report number GAO-05-861T entitled 'Federal Research: Observations on the Small Business Innovation Research Program' which was released on June 29, 2005. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Testimony: Before the Subcommittee on Environment, Technology, and Standards, Committee on Science, House of Representatives: United States Government Accountability Office: GAO: For Release on Delivery Expected at 3:00 p.m. EDT: Tuesday, June 28, 2005: Federal Research: Observations on the Small Business Innovation Research Program: Statement for the Record of Anu K. Mittal, Director: Natural Resources and Environment Team: GAO-05-861T: GAO Highlights: Highlights of GAO-05-861T, a report to Subcommittee on Environment, Technology and Standards, Committee on Science, House of Representatives: Why GAO Did This Study: Since it was established in 1982, GAO has consistently reported on the success of the Small Business Innovation Research (SBIR) program in benefiting small, innovative companies, strengthening their role in federal research and development (R&D), and helping federal agencies achieve their R&D goals. However, through these reviews GAO has also identified areas where action by participating agencies or the Congress could build on the program’s successes and improve its operations. This statement for the record summarizes the program’s successes and improvements over time, as well as the continuing challenge of assessing the long term results of the program. What GAO Found: Between July 1985 and June 1999, GAO reviewed, reported, and testified on the SBIR program many times at the request of the Congress. While GAO’s work focused on many different aspects of the program, it generally found that SBIR is achieving its goals to enhance the role of small businesses in federal R&D, stimulate commercialization of research results, and support the participation of small businesses owned by women and/or disadvantaged persons. Participating agencies and companies that GAO surveyed during the course of its reviews generally rated the program highly. GAO also identified areas of weaknesses and made recommendations that, if addressed, could strengthen the program further. Some of these concerns related to (1) duplicate funding for similar, or even identical, research projects by more than one agency, (2) inconsistent interpretations of extramural research budgets by participating agencies, (3) geographical concentration of awards in a small number of states, and (4) lack of clarification on the emphasis that agencies should give to a company’s commercialization record when assessing its proposals. Most of GAO’s recommendations for program improvement have been either fully or partially addressed by the Congress in various reauthorizations of the program or by the agencies themselves. One issue that continues to remain somewhat unresolved after almost two decades of program implementation is how to assess the performance of the SBIR program. As the program has matured, the Congress has emphasized the potential for commercialization as an important criterion in awarding funds and the commercialization of a product as a measure of success for the program. However, in 1999, GAO reported that the program’s other goals also remain important to the agencies. By itself, according to some program managers, limited commercialization may not signal “failure” because a company may have achieved other goals, such as innovation or responsiveness to an agency’s research needs. GAO identified a variety of reasons why assessing the performance of the SBIR program has remained a challenge. First, because the authorizing legislation and the Small Business Administration’s (SBA) policy directives do not define the role of the company’s commercialization record in determining commercial potential and the relative importance of the program’s goals, different approaches have emerged in agencies’ evaluations of proposals. Second, GAO found that it has been difficult to find practical ways to define and measure the SBIR program’s goals in order to evaluate proposals. For example, the authorizing legislation lacks a clear definition of “commercialization,” and agencies sometimes differed on its meaning. Finally, GAO reported that as the emphasis on commercialization had grown, so had concerns that noncommercial successes may not be adequately recognized. For example, program managers identified various projects that met special military or medical equipment needs but that had limited sales potential. www.gao.gov/cgi-bin/getrpt?GAO-05-861T. To view the full product, including the scope and methodology, click on the link above. For more information, contact Anu K. Mittal, (202) 512- 3841, mittala@gao.gov. [End of section] Mr. Chairman and Members of the Subcommittee: We are pleased to have the opportunity to comment on the Small Business Innovation Research (SBIR) program. Since the program's inception, we have consistently reported on its success in benefiting small, innovative companies, strengthening their role in federal research and development (R&D), and helping federal agencies achieve their R&D goals. However, through these reviews we have also identified areas where action by participating agencies or the Congress could build on the program's successes and improve its operations. Over the life of the program these recommendations have largely been implemented. This statement will discuss the program's successes as well as the continuing challenge of assessing the long term results of the SBIR program. As a competitor in the global economy, the United States relies heavily on innovation through research and development. The potential of small businesses to be sources of significant innovation led the Congress to increase government funding for R&D projects with commercial potential that are conducted by small high-technology companies. In this context, the Small Business Innovation Development Act of 1982 established the SBIR program to stimulate mission-related technological innovation, use small businesses to meet federal R&D needs, foster participation by minority and disadvantaged persons in technological innovation, and increase private sector commercialization of innovations derived from federal R&D[Footnote 1]. The act provided for a three-phased program: phase I to determine the feasibility and scientific and technical merit of a proposed research idea; phase II to further develop the idea, taking into account its commercial potential; and phase III to commercialize the resulting product or process with no further SBIR funding. The original program was reauthorized in 1986, extending the program's expiration date from 1988 to 1993.[Footnote 2] In 1992, it was reauthorized by the Small Business Research and Development Enhancement Act to expand and improve the program, to emphasize its goal of increasing private sector commercialization, to increase participation by small businesses, and to improve the government's dissemination of program-related information.[Footnote 3] In addition, the act increased funding for phase-I and phase-II proposals to $100,000 and $750,000, respectively, with adjustments once every 5 years for inflation and changes in the program. The program was again reauthorized in 2000 by the Consolidated Appropriations Act of 2001,[Footnote 4] which directed the Small Business Administration (SBA) and participating agencies to, among other things, expand the scope of publicly available information on specific grants and to annually report on their SBIR programs. In addition, the act requires award recipients to provide information to help SBA evaluate the program. The SBIR program is currently scheduled to expire on September 30, 2008. Current law requires every federal department with an R&D budget of $100 million or more to establish and operate a SBIR program funded by a set percentage of that agency's extramural R&D budget--originally 1.25 percent and now 2.5 percent. In addition, agencies with R&D spending above $20 million are directed to establish goals for financing small business R&D projects at levels higher than the previous year. As of fiscal year 2004, 12 federal agencies participated in the SBIR program, including the departments of Agriculture, Commerce, Defense (DOD), Education, Energy, Health and Human Services (HHS), Homeland Security, Housing and Urban Development, and Transportation; the Environmental Protection Agency; the National Aeronautics and Space Administration (NASA); and the National Science Foundation (NSF). Each agency manages its own program, while SBA plays a central administrative role, such as issuing policy directives and annual reports for the program. Awards from three agencies--DOD, National Institutes of Health, and NASA--account for the majority of SBIR funds. From its inception in fiscal year 1983 through fiscal year 2003, federal agencies have awarded over $15 billion for more than 76,000 projects. SBIR Program Has Generally Met Its Goals: Between July 1985 and June 1999, we reviewed, reported, and testified on the SBIR program many times at the request of the Congress. While our work focused on many different aspects of the program, we generally found that SBIR is achieving its goals to enhance the role of small businesses in federal R&D, stimulate commercialization of research results, and support the participation of small businesses owned by women and/or disadvantaged persons. Participating agencies and companies that we surveyed during the course of our reviews generally rated the program highly. Specific examples of program success that we identified include the following: * High-quality research. Throughout the life of the program, awards have been based on technical merit and are generally of good quality. For example, in 1989 we reported that according to agency officials, more than three-quarters of the research conducted with SBIR funding was as good as or better than other agency-funded research. Agency officials also rated the research as more likely than other research they oversaw to result in the invention and commercialization of new products. When we again looked at the quality of research proposals in 1995, we found that while it was too early to make a conclusive judgment about the long-term quality of the research, the quality of proposals remained good, according to agency officials. * Widespread competition. The SBIR program successfully attracts many qualified companies, has had a high level of competition, and consistently has had a high number of first-time participants. Specifically, we reported that the number of proposals that agencies received each year had been increasing. In addition, as we reported in 1998, agencies rarely received only a single proposal in response to a solicitation, indicating a sustained level of competition for the awards. We also found that the agencies deemed many more proposals worthy of awards than they were able to fund. For example, the Air Force deemed 1,174 proposals worthy of awards in fiscal year 1993 but funded only 470. Moreover, from fiscal years 1993 through 1997, one third of the companies that received awards were first-time participants. This suggests that the program attracts hundreds of new companies annually. * Effective outreach. SBIR agencies consistently reach out to foster participation by women-owned or socially and economically disadvantaged small businesses. For example, we found that DOD's SBIR managers participated in a number of regional small business conferences and workshops that are specifically designed to foster increased participation by women-owned and socially and economically disadvantaged small businesses. * Successful commercialization. SBIR successfully fosters commercialization of research results. At various points in the life of the program we have reported that SBIR has been successful in increasing private sector commercialization of innovations. For example, past GAO and DOD surveys of companies that received SBIR Phase II funding have determined that approximately 35 percent of the projects resulted in the sales of products or services, and approximately 45 percent of the projects received additional developmental funding. We have also reported that agencies were using various techniques to foster commercialization. For example, in an attempt to get those companies with the greatest potential for commercial success to the marketplace sooner, DOD instituted a Fast Track Program, whereby companies that are able to attract outside commitments/capital for their research during phase I are given higher priority in receiving a phase II award. * Helping to serve mission needs. SBIR has helped serve agencies' missions and R&D needs. Agencies differ in the emphasis they place on funding research to support their mission and to support more generalized research. Specifically, we found that DOD links its projects more closely to its mission. In comparison, other agencies emphasize research that will be commercialized by the private sector. Many of the projects DOD funded have specialized military applications while NIH projects have access to the biomedical market in the private sector. Moreover, we found that SBIR promotes research on the critical technologies identified in lists developed by DOD and/or the National Critical Technologies Panel. Generally agencies reviewed these listings of critical technologies to develop research topics or conducted research that fell within one of the two lists. Improvements Made to the SBIR Program Over Time: We have also identified areas of weaknesses and made recommendations that, if addressed, could strengthen the program further. Many of our recommendations for program improvement have been either fully or partially addressed by the Congress in various reauthorizations of the program or by the agencies themselves. For example, * Duplicate funding. In 1995, we identified duplicate funding for similar, or even identical, research projects by more than one agency. A few companies received funding for the same proposals two, three, and even five times before agencies became aware of the duplication. Contributing factors included the fraudulent evasion of disclosure by companies applying for awards, the lack of a consistent definition for key terms such as "similar research," and the lack of interagency sharing of data on awards. In response to our recommendations, SBA strengthened the language agencies use in their application packages to clearly warn applicants about the illegality of entering into multiple agreements for essentially the same effort and developed Internet capabilities to access SBIR data for all of the agencies. In SBA's view, the stronger language regarding the illegality of seeking funding for similar or identical projects addresses the need to develop consistent definitions to help agencies determine when projects are "similar." * Inconsistent interpretations of extramural research budgets. In 1998, we found that while agency officials adhered to SBIR's program and statutory funding requirements, they used differing interpretations of how to calculate their "extramural research budgets." As a result some agencies were inappropriately including or excluding some types of expenses. To address our recommendation that SBA provide additional guidance on how participating agencies were to calculate their extramural research budgets, the Congress in 2000 required that the agencies report annually to SBA on the methods used to calculate their extramural research budgets. * Geographical concentration of awards. In 1999, in response to congressional concerns about the geographical concentration of SBIR awards, we reported that companies in a small number of states, especially California and Massachusetts, have submitted the most proposals and won the majority of awards. The distribution of awards generally followed the pattern of distribution of non-SBIR expenditures for R&D, venture capital investments, and academic research funds. We reported that some agencies had undertaken efforts to broaden the geographic distribution of awards and that the program implemented by the National Science Foundation had been particularly effective. Although we did not make any recommendations on how to improve the program's outreach to states receiving fewer awards, in the 2000 reauthorization of the program, Congress established the Federal and State Technology Partnership Program to help strengthen the technological competitiveness of small businesses, especially in those states that receive fewer SBIR grants. * Clarification on commercialization and other SBIR goals. Finally, in response to our continuing concern that clarification was needed on the relative emphasis that agencies should give to a company's commercialization record and SBIR's other goals when evaluating proposals, in 2000 the Congress required companies applying for a second phase award to include a commercialization plan with their SBIR proposals. This requirement partially addressed our concern. Moreover, in the spring of 2001, SBA initiated efforts to respond to our recommendation to develop standard criteria for measuring commercial and other outcomes of the SBIR program, such as uniform measures of sales and developmental funding, and incorporate these criteria into its Tech-Net database. Specifically, SBA began implementing a reporting system to measure the program's commercialization success. In fiscal year 2002, SBA further enhanced the reporting system to include commercialization results that would help establish an initial baseline rate of commercialization. In addition, small business firms participating in the SBIR program are required to provide information annually on sales and investments associated with their SBIR projects. Assessing the Performance of the SBIR Program Remains a Challenge: One issue that continues to remain somewhat unresolved after almost two decades of program implementation is how to assess the performance of the SBIR program. As the program has matured, the Congress has emphasized the potential for commercialization as an important criterion in awarding funds and the commercialization of a product as a measure of success for the program. However, in 1999, we reported that the program's other goals also remain important to the agencies. By itself, according to some program managers, limited commercialization may not signal "failure" because a company may have achieved other goals, such as innovation or responsiveness to an agency's research needs. We identified a variety of reasons why assessing the performance of the SBIR program has remained a challenge. * First, because the authorizing legislation and SBA's policy directives do not define the role of the company's commercialization record in determining commercial potential and the relative importance of the program's goals, different approaches have emerged in agencies' evaluations of proposals. As a result, the relative weight that should be given to the program's goals when evaluating proposals remains unclear. Innovation and responsiveness to an agency's needs, for example, may compete with the achievement of commercialization. In the view of many program managers, innovation involves a willingness to undertake R&D with a higher element of risk and a greater chance that it may not lead to a commercial product; responsiveness to an agency's needs involves R&D that may be aimed at special niches with limited commercial potential. Striking the right balance between achieving commercial sales and encouraging new, unproven technologies is, according to the program managers, one of the key ingredients in the program's overall success. * Second, we found that it has been difficult to find practical ways to define and measure the SBIR program's goals in order to evaluate proposals. For example, the authorizing legislation lacks a clear definition of "commercialization," and agencies sometimes differed on its meaning. This absence of a definition makes it more difficult to determine when a frequent winner is "failing" to achieve a sufficient level of commercialization and how to include this information in an agency's review of the company's proposal. Similarly, efforts to define and measure technological innovation, which was one of the program's original goals, have posed a challenge. Although definitions vary, there is widespread agreement that technological innovation is a complex process, particularly in the development of sophisticated modern technologies. * Finally, we reported that as the emphasis on commercialization had grown, so had concerns that noncommercial successes may not be adequately recognized. For example, program managers identified various projects that met special military or medical equipment needs but that had limited sales potential. These projects would be helpful in reducing the agency's expenditures and meeting the mission of the agency but may not be appropriately captured in typical measurements of commercialization. In general, we found that program managers valued both noncommercial and commercial successes and feared that the former might be ignored in emphasizing the latter. To help evaluate the performance of the program, in the 2000 reauthorization of SBIR, Congress required SBA to develop a database that would help the agency collect and maintain in common format necessary program output and outcome information. The database is to include the following information on all phase II awards: (1) revenue from the sale of new products or services resulting from the SBIR funded research, (2) additional investment from any non-SBIR source for further research and development, and (3) any other description of outputs and outcomes of the awards. In addition, the database is to include general information for all applicants not receiving an award including an abstract of the project. In conclusion, Mr. Chairman, our work has shown that, overall, the SBIR program has been successful in meeting its goals and that the Congress and the agencies have implemented actions to strengthen the program over time. However, an assessment of the program's results remains a challenge because of the lack of clarity on how much emphasis the program should place on commercialization versus other goals. For further information, please contact Anu Mittal at (202) 512-3841 or mittala@gao.gov. FOOTNOTES [1] Pub.L. No. 97-219 (1982). [2] Pub.L. No. 99-443 (1986). [3] Pub.L. No. 102-564 (1992). [4] Pub. L. No. 106-554 (2000).