This is the accessible text file for GAO report number GAO-14-96 entitled 'Small Business Innovation Research: DOD's Program Supports Weapon Systems, but Lacks Comprehensive Data on Technology Transition Outcomes' which was released on December 20, 2013. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Committees: December 2013: Small Business Innovation Research: DOD's Program Supports Weapon Systems, but Lacks Comprehensive Data on Technology Transition Outcomes: GAO-14-96: GAO Highlights: Highlights of GAO-14-96, a report to congressional committees. Why GAO Did This Study: To compete in the global economy, the United States relies heavily on innovation through research and development. The Small Business Innovation Development Act of 1982 initiated SBIR programs across federal agencies in an effort to stimulate innovation through small businesses. DOD spends over $1 billion annually to support SBIR awards. The Conference Report accompanying the National Defense Authorization Act for Fiscal Year 2013 mandated that GAO assess the transition of technologies developed through the DOD SBIR program. This report examines (1) practices the military department SBIR programs use to facilitate the transition of SBIR technologies, and (2) the extent to which SBIR technologies are transitioning to DOD users, including major weapon system acquisition programs. GAO reviewed SBIR program documentation and data. GAO also interviewed officials from DOD’s Office of Small Business Programs and the military departments to determine the practices used to facilitate technology transition and assess SBIR transition outcome data. What GAO Found: The Small Business Innovation Research (SBIR) programs within the military departments use a variety of practices and tools to facilitate technology transition—-the act of passing technologies developed in the science and technology environment on to users such as weapon system acquisition programs or warfighters in the field. GAO identified some common transition practices and tools across SBIR programs. For example, specific initiatives, such as the Commercialization Readiness Program, are used by each SBIR program and focus resources on enhancing technology transition opportunities. Transition facilitators are also used by each program to provide a network of personnel who manage SBIR activities that support technology transition. GAO also found some different practices and tools used to support technology transition efforts, such as the Navy Transition Assistance Program, which provides consulting services and helps showcase SBIR projects in an effort to improve small businesses’ abilities to transition their projects. Transition facilitation efforts are supported by administrative funds provided through each program’s SBIR budget and from other funds received from their respective military department. A recent increase in the amount of administrative funding that can come from SBIR budgets is expected to help the programs enhance their transition facilitation efforts. GAO was unable to assess the extent of technology transition associated with the military department SBIR programs because comprehensive and reliable technology transition data for SBIR projects are not collected. Transition data systems used by DOD provide some transition information but have significant gaps in coverage and data reliability concerns. The military departments have additional measures through which they have identified a number of successful technology transitions, but these efforts capture a limited amount of transition results. SBIR transition reporting requirements recently established by Congress have led DOD to evaluate its options for providing transition data. GAO identified several challenges to attaining complete and accurate technology transition data. For instance, the lack of a common definition for technology transition across SBIR programs could cause reporting inconsistencies. Additionally, tracking transition can be challenging because of the sometimes lengthy period between SBIR project completion and transition to a DOD user. DOD initially plans to use transition data from Company Commercialization Reports—viewed by DOD as the best available source—to meet the new transition reporting requirements. However, SBIR officials indicated that addressing transition reporting requirements is a longer-term effort, and there is no specific plan including a time line for when DOD will be able to support those requirements. Without a plan that establishes a time line, it is unclear how and when DOD will begin to provide the technology transition information expected by Congress. Although Congress did not specify when reporting was to begin, it expects DOD to report new transition-related information to the Administrator of the Small Business Administration to meet the new reporting requirements. However, unless DOD communicates its plan and accompanying time line, the congressional committees to whom the Small Business Administration reports may be unaware of the data quality issues with the transition- related information DOD plans to use to support reporting in the near term. What GAO Recommends: GAO recommends that DOD establish a common definition of technology transition for SBIR projects, develop a plan to track transition that will improve the completeness, quality, and reliability of transition data, and report to Congress its plan for meeting new SBIR technology transition reporting requirements. DOD partially concurred with these recommendations, but cited challenges to improving transition data. GAO believes options are available to address the challenges. View [hyperlink, http://www.gao.gov/products/GAO-14-96]. For more information, contact Michael J. Sullivan at (202) 512-4841 or sullivanm@gao.gov. [End of section] Contents: Letter: Background: Military Department SBIR Programs Use a Variety of Practices, Tools, and Administrative Funds to Support Technology Transition Efforts: DOD Lacks Comprehensive Data on SBIR Technology Transition Outcomes, but Some Transition Successes Have Been Identified: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Scope and Methodology: Appendix II: Comments from the Department of Defense: Appendix III: GAO Contact and Staff Acknowledgments: Tables: Table 1: DOD Small Business Innovation Research (SBIR) Program Framework: Table 2: Fiscal Year 2012 Budget and Awards for Military Department Small Business Innovation Research (SBIR) Programs: Table 3: DOD Small Business Innovation Research (SBIR) Program Transition Data Sources Overview: Table 4: Examples of Transition Outcomes Reported by Military Department Small Business Innovation Research (SBIR) Programs: Abbreviations: CCR: Company Commercialization Report: DOD: Department of Defense: FPDS-NG: Federal Procurement Data System-Next Generation: SBIR: Small Business Innovation Research: [End of section] GAO: United States Government Accountability Office: 441 G St. N.W. Washington, DC 20548: December 20, 2013: Congressional Committees: To compete in the global economy, the United States relies heavily on innovation through research and development. Recognizing the potential for small businesses to be a source of significant innovation, Congress enacted the Small Business Innovation Development Act of 1982.[Footnote 1] The act established the Small Business Innovation Research (SBIR) Program for use by defense and civilian agencies to stimulate technological innovation and utilize small businesses to meet federal technology needs.[Footnote 2] The Department of Defense (DOD) has 13 SBIR program components (programs) and spends over $1 billion annually on SBIR contract awards.[Footnote 3] These programs provide DOD with opportunities to acquire technological innovations that can improve military capabilities and ensure superiority over adversaries. The conference report accompanying the National Defense Authorization Act for Fiscal Year 2013 mandated that GAO assess the transition of technologies developed through the DOD SBIR program. This report examines (1) practices the military department SBIR programs use to facilitate the transition of technologies, and (2) the extent to which SBIR technologies are transitioning to DOD users, including major weapon system acquisition programs. In conducting this review, we focused on the SBIR programs within the three military departments--Air Force, Army, and Navy--which in recent years have comprised about 75 percent of the total DOD SBIR program budget. We evaluated documentation and interviewed officials from DOD's Office of Small Business Programs and the three military departments on processes and practices used to facilitate technology transition--the act of passing technologies developed in the science and technology environment on to users--and track transition outcomes. We reviewed information on the SBIR program structure and processes for selecting and managing technology projects, and the practices used to support the transition of SBIR technologies to DOD acquisition programs and other military users. In addition, we reviewed practices and data systems that DOD uses to track SBIR projects and their outcomes. We also reviewed available documentation on these practices and systems, and interviewed DOD officials to assess whether they provide accurate, reliable, and comprehensive data on the transition of technologies to military users. Any limitations that were identified for these data collection practices and data systems are discussed later in this report. We conducted this performance audit from April 2013 to December 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Please see Appendix I for further details on the scope and methodology used for this work. Background: DOD relies on its science and technology community--DOD research laboratories, test facilities, industry, and academia--to identify, pursue, and develop new technologies that address military needs. The DOD SBIR program is one mechanism through which DOD attempts to accomplish its science and technology goals and develop technologies that contribute to weapon systems or transition directly to warfighters for use in the field. Within DOD, the Office of Small Business Programs oversees the department's SBIR program activities, develops policy, and manages program reporting. This office generally relies on the agencies, such as the Army, Air Force, and Navy, to oversee and execute their own SBIR program activities. Each agency has flexibility to tailor its SBIR program to meet its needs, including determining what type of research to pursue, which projects to fund, and how to monitor ongoing projects. To initiate the project award process, SBIR programs work with the science and technology and acquisition communities to generate and prioritize research and development topics. These topics describe technical areas of interest and capability needs, which the programs use in their solicitations for proposals from small businesses. DOD conducts three solicitations each year where small businesses compete for Phase I contract awards that are expected to respond to the needs identified in each topic. Once awarded, SBIR projects are managed through a three-phase program structure, which is outlined in table 1. Table 1: DOD Small Business Innovation Research (SBIR) Program Framework: Phases of SBIR: Phase I: Projects selected competitively based on scientific and technical merit, applicants' past SBIR performance, and potential for commercial application. Focus of work conducted in this phase is on determining project feasibility and merit; Typical project duration and funding: 6 months, up to $150,000; Sources of funding: SBIR program funding. Phases of SBIR: Phase II: Expands on efforts of Phase I projects, focusing on technology development to prototype. In general, project should have confirmed interest in transition from a user; Typical project duration and funding: 2 years, up to $1 million; Sources of funding: SBIR program funding, can include external funding. Phases of SBIR: Phase III: Referred to as commercialization; continued project development with goal to transition a technology into a commercial product or process for sale to government or private- sector customers. A primary goal for DOD commercialization is to transition technologies to weapon system acquisition programs and the warfighters in the field; Typical project duration and funding: Unlimited; Sources of funding: Non-SBIR government or private-sector funding. Source: DOD. [End of table] The number of Phase I and Phase II projects varies from year to year based on technology needs and funding availability. Table 2 shows the budgets and project awards reported for the military department SBIR programs in fiscal year 2012. Table 2: Fiscal Year 2012 Budget and Awards for Military Department Small Business Innovation Research (SBIR) Programs: Military department SBIR program: Air Force; Budget (in millions): $304.1; Number of Phase I awards: 531; Number of Phase II awards: 207. Military department SBIR program: Army; Budget (in millions): $183.6; Number of Phase I awards: 322; Number of Phase II awards: 200. Military department SBIR program: Navy; Budget (in millions): $271.3; Number of Phase I awards: 503; Number of Phase II awards: 279. Source: DOD Office of Small Business Programs. [End of table] We and others have previously found that DOD and its technology development programs, such as the SBIR program, have encountered challenges to transitioning their technologies to acquisition programs or directly to the warfighter for use in the field.[Footnote 4] For instance, in our past work we found several reasons why technologies may not transition, including insufficient maturity, inadequate demonstration or recognition by users of a technology's potential, and unwillingness or inability of acquisition programs to fund final stages of development. To address SBIR technology transition challenges, DOD, the Small Business Administration, and Congress have established additional program provisions, incentives, and reporting requirements. For example, the Commercialization Readiness Program was initiated to accelerate the transition of SBIR funded technologies to Phase III, especially those that lead to acquisition programs and high priority military requirements, such as fielded systems.[Footnote 5] As part of this initiative, Congress authorized the military departments to use up to 1 percent of SBIR funding for administrative activities that facilitate transition. This funding is used to support program staff and contractors who provide assistance to SBIR awardees, including efforts to enhance networking and build relationships among small businesses, prime contractors, and DOD science and technology and acquisition communities. More recently, Congress's reauthorization of the SBIR program in the National Defense Authorization Act for Fiscal Year 2012 included a number of measures related to technology transition efforts, including the following: * Requires DOD to set a goal to increase the number of Phase II contracts awarded that lead to technology transition into acquisition programs or fielded systems, and use incentives to encourage program managers and prime contractors to meet the goal. * Requires that DOD report specific transition-related information to the Administrator of the Small Business Administration for inclusion in an annual report to designated congressional committees. This includes reporting the number and percentage of Phase II contracts that led to technology transition into acquisition programs or fielded systems, information on the status of each project that received funding through the Commercialization Readiness Program and efforts to transition those projects, and a description of each incentive used to meet the department's transition goal.[Footnote 6] * Authorizes DOD to establish goals for the transition of Phase III technologies in subcontracting plans for contracts of $100 million or more, and to require prime contractors on such contracts to report the number and dollar amount of contracts entered into by prime contractors for Phase III projects. * Sets the ceiling for discretionary technical assistance that can be provided annually for all Phase I and Phase II projects at $5,000 per project. Programs can use this funding to assist awardees in making technical decisions on projects, solving technical problems, minimizing technical risks, and commercializing projects. * Establishes a pilot effort to allow DOD SBIR programs to use not more than 3 percent of their SBIR budgets for, among other things, program administration, technical assistance, and the implementation of commercialization and outreach initiatives.[Footnote 7] Military Department SBIR Programs Use a Variety of Practices, Tools, and Administrative Funds to Support Technology Transition Efforts: The military department SBIR programs use several management practices and tools to support technology transition efforts. We identified some common transition elements across the programs, but also found some differences in how each program approaches its technology transition efforts. The programs' technology transition efforts are supported through use of administrative funds coming from their SBIR budgets and other funds provided by their respective military department. The transition facilitation practices, tools, and funds used to promote the transition of SBIR technologies include the following: Early focus on transition through topic generation and project selection: Technology transition efforts begin with topic generation and project selection processes that emphasize the pursuit of projects for which there is a demonstrated military need and potential transition opportunities. To do this, the military department programs formally engage stakeholders from the science and technology and acquisition communities in generating and endorsing topics for SBIR solicitations. In proposing topics and selecting projects, programs have to balance their desire for technological innovation with meeting pressing warfighter needs. SBIR officials stated that projects that pursue incremental improvements generally are more likely to deliver the technical capability expected for technology transition to occur. In contrast, they noted that projects that focus more on "leap- forward" technology innovations that can support future warfighting needs tend to require more long-term development and have greater technical and transition risks. DOD policy requires that at least 50 percent of military department topics are endorsed by the acquisition community, such as program executive offices. This helps ensure that the acquisition community is engaged with the SBIR programs and that a significant portion of projects are dedicated to addressing specific needs identified by military users. Phase II transition initiatives: Transition-focused activities increase as Phase II projects progress, commensurate with an increasing technology maturity and understanding of a project's potential opportunities for use. In particular, the military department SBIR programs target transition opportunities through their Commercialization Readiness Programs and other initiatives that provide additional support to select Phase II projects. In some cases, the SBIR programs require formal technology transition agreements or matching funding as a condition to receiving additional Phase II funding. Technology transition agreements, which Air Force and Navy officials reported using, help manage project expectations and formalize stakeholder commitments by outlining cost, schedule, and performance expectations for transition to occur. Matching funds from intended users, which are required by the Navy for some projects, can help create greater buy-in for transition because the intended users have a monetary stake in the project. Transition facilitators: Each military department SBIR program has a network of transition facilitators who manage the Commercialization Readiness Program and other enhancement efforts, as well as broader SBIR activities that support technology transition. The facilitators are located at military labs, acquisition centers, and program executive offices to work directly with government stakeholders and help ensure projects are responsive to warfighter needs. They also help small businesses identify and position themselves for opportunities to transition their SBIR technologies. Although the roles and responsibilities vary somewhat across the programs, in general, transition facilitators assist with topic generation and prioritization; foster communication among small businesses, research laboratories, and the acquisition community in support of transition opportunities; and monitor project progress, including outcomes. Navy Transition Assistance Program: The Navy established an additional program over a decade ago to prepare its SBIR participants for technology transition opportunities. The Transition Assistance Program is a voluntary 11-month program with, on average, about two-thirds of Phase II recipients participating each year. It provides consulting services focused on improving the small businesses' abilities to transition their SBIR products, including assistance in transition planning and developing marketing tools. Under the program, profiles are used to describe the expected capability, level of technology maturity, and potential technology transition opportunities for each project. These profiles are available in electronic form through a web- based portal called the Navy Virtual Acquisition Showcase, and support the annual Navy Opportunity Forum conference. The conference provides Transition Assistance Program participants with direct exposure and one-on-one opportunities to interact with prospective transition partners in the government and industry. Other transition facilitation tools: SBIR programs also use technology roadmaps and formal relationship-building activities, such as conferences and workshops, to support transition efforts. Technology roadmaps are schedule-based planning documents used to identify opportunities for SBIR technology insertion into acquisition programs or direct use by the warfighter. Conferences and workshops, such as the annual Beyond Phase II conference hosted by the Office of Small Business Programs, are used by the programs to provide opportunities for SBIR Phase II companies to interact directly with prospective government and industry users and showcase their projects. Administrative funds: The technology transition practices and tools used by the programs are supported by administrative funds provided through their SBIR budgets as well as non-SBIR sources from their respective agencies. The Commercialization Readiness Program and discretionary technical assistance provisions enable programs to use portions of their SBIR budgets to fund administrative activities, including transition support. For fiscal year 2012, this funding totaled about $12 million across the three military departments. The fiscal year 2012 SBIR reauthorization included a new provision--which DOD officials advocated--that allows the programs to use up to 3 percent of their funds to support administrative activities, which is in addition to funds available through the Commercialization Readiness Program and discretionary technical assistance. SBIR officials stated that although this additional funding allowance is in the initial stages of being used, they believe these funds will help enhance transition facilitation measures for their programs going forward. Additional agency funding outside of the SBIR budget is also used to manage programs and support transition activities, but the amount of such funding is not readily identifiable because the military departments do not all require that the amount of funding used to support associated administrative efforts be documented. DOD Lacks Comprehensive Data on SBIR Technology Transition Outcomes, but Some Transition Successes Have Been Identified: We were unable to assess the extent of technology transition associated with the military department SBIR programs because comprehensive and reliable technology transition data are not collected. Tracking mechanisms used by DOD--Company Commercialization Reports (CCR) and the Federal Procurement Data System-Next Generation (FPDS-NG)--provide some information on SBIR Phase III activities, but these mechanisms have significant gaps in coverage and data reliability concerns that limit their transition tracking capabilities. The military departments have additional measures through which they have identified a number of successful SBIR transitions to DOD acquisition programs and directly to fielded systems, but these efforts capture a limited amount of transition information. DOD is assessing how to comply with the new transition reporting requirements directed by Congress, but has yet to develop a plan that will support identification and annual reporting of the extent to which SBIR technologies transition to DOD acquisition programs or to fielded systems. SBIR Programs Use Three Resources to Identify Transition Results, but Each Has Inherent Data Limitations: The military department SBIR programs rely, to varying degrees, on two data systems--CCR and FPDS-NG--as well as their own agency-specific data collection activities to identify transition results. Table 3 more fully describes the data sources used and their limitations. Table 3: DOD Small Business Innovation Research (SBIR) Program Transition Data Sources Overview: Transition data source: Company Commercialization Reports (CCR); Description: * DOD SBIR database used to capture sales resulting from, and investments associated with, Phase III awards (non-SBIR funds), including DOD-specific commercialization; * Reported commercialization results used by SBIR program management when evaluating future awards to previous SBIR participants (i.e., as a gauge of a firm's ability to commercialize products); Data limitations: * Do not capture all commercialization data. Only small businesses seeking additional SBIR awards are requested to report Phase III commercialization; data for past SBIR participants that do not pursue new awards is limited; * Self-reported data poses reliability and completeness challenges because of the potential for misreporting; * Collection of data on the specific military user of the technology is inconsistent, at best. Specific users include acquisition programs, such as the F-35 Joint Strike Fighter. Transition data source: Federal Procurement Data System-Next Generation (FPDS-NG); Description: * FPDS-NG is the primary government-wide contracting database that provides information on all government contracting actions; * Includes a data field through which contracting officers can identify contracts with SBIR associations; * Can be used to identify Phase III commercialization awards; Data limitations: * Does not capture all commercialization data; limited to government contracts; * Not designed to provide commercialization data for subcontracting between a prime contractor and a SBIR recipient; DOD officials indicated this type of commercialization is prevalent; * Contract miscoding of SBIR lineage can cause over-or under-reported commercialization results; DOD officials indicated that contracting officers have challenges in correctly coding contracts, including contracts sometimes being wrongly associated with SBIR as well as contracts failing to be acknowledged as SBIR-related; * Does not directly collect data on the specific military user of the technology. Specific users include acquisition programs, such as the F- 35 Joint Strike Fighter. Transition data source: Agency-specific SBIR transition documentation activities; Description: * To varying degrees, military department programs track status and completion of SBIR projects via internal management systems and input from transition agents, users, and small businesses; * Programs also collect success stories for a select amount of projects and make them available using tools such as annual reports and web-accessible databases; Data limitations: * Do not capture all commercialization data; SBIR programs tend to track subsets of Phase II projects, such as Commercialization Readiness Program and Phase II Enhancement projects; * Used to corroborate results from CCR or FPDS-NG, and potentially identify commercialization not captured through data systems (e.g., subcontract awards); * Tracing SBIR lineage to technologies is a stated challenge because it is resource-intensive and technologies evolve over time; * Data collection is somewhat ad hoc and internal tracking tool use is varied. Source: GAO analysis; DOD SBIR program documentation. [End of table] Although the CCR and FPDS-NG data systems do not capture complete data on the transition of SBIR technologies, they do provide high-level commercialization information that the SBIR programs use to track progress in achieving program goals. Because the data help support program management efforts, the Office of Small Business Programs and the military departments, to varying degrees, take steps to verify the quality of CCR and FPDS-NG data. For example, the Army assesses and validates CCR data for its projects on an ongoing basis. This process involves comparing recent updates to the database with FPDS-NG contract data and internal Army tracking data to confirm the accuracy of commercialization funding reported by the small businesses. The Navy SBIR program uses FPDS-NG as its primary source of commercialization data and employs similar validation techniques to improve the accuracy of commercialization data tracked through this system. By comparing contracts in FPDS-NG flagged as SBIR-related to DOD contract management systems, the Navy is able to verify the accuracy of Phase III awards data tied to government contracts. Both the Army and Navy officials acknowledged, however, that even with their data validation efforts, problems persist because of the limitations of the Company Commercialization Reports and FPDS-NG. The military department programs have developed some internal capabilities to track certain projects and provide insight into the types of capabilities enabled by them. Like Company Commercialization Reports and FPDS-NG, these capabilities do not provide comprehensive transition information, but may help the departments to gain more insight into transition outcomes for some technologies developed within SBIR programs and to respond to DOD and congressional inquiries about program results. In particular, the programs identify transition success stories for a limited number of projects, ranging from Phase III awards for additional research and development to transition to major acquisition programs or fielded systems. Information on these success stories can come from SBIR program officials, acquisition program officials, prime contractors, or directly from the small businesses. The Air Force's database of identified transition successes includes 95 transition stories dating back to 2004. The Army's program produces an annual report describing transition outcomes for 20-30 successful projects. The Navy's program maintains a searchable database of SBIR projects that includes profiles on select transitioned projects as well. Table 4 provides examples of transition outcomes for projects identified through our review of these reporting mechanisms. Table 4: Examples of Transition Outcomes Reported by Military Department Small Business Innovation Research (SBIR) Programs: SBIR program: Air Force; Project descriptions: * Wideband low-profile antenna transitioned to the Sand Dragon unmanned air system to provide detection and identification capabilities for various concealed targets (e.g., foliage canopies or camouflage nets). A Phase III contract was received to develop various other radio frequency systems using technology developed under this SBIR project; * Web-based software application toolkit that efficiently collects, assesses, and analyzes text-based intelligence information, thereby reducing the burden placed on an analyst who reviews intelligence information. A Phase III award was provided by the Air Force through the Rapid Innovation Fund to develop this application further. SBIR program: Army; Project descriptions: * Structural health monitoring system received Phase III award for further testing with Black Hawk helicopters and other rotorcraft applications. System uses sensors to assess structural damage, such as fatigue cracks, identifies damage in inaccessible areas of structures, characterizes damage growth, and provides an image of the damage size and orientation; * Highly responsive and maneuverable space observation satellite system received Phase III funding from the National Reconnaissance Office for system advancement of sensors and information processes. System can provide Army and other agencies with control of space resources and immediate, real-time access to the collected data at low- cost and low risk. SBIR program: Navy; Project descriptions: * Portable machine that uses a hydraulic system to attach arresting gear cable terminals on aircraft carriers transitioned to a Navy acquisition program office for further shipboard testing. This machine significantly reduces required labor and eliminates inconsistencies when compared to the current process used to attach wire rope terminals on aircraft carriers; * Wireless handheld maintenance technology that provides on-demand logistics and maintenance information to shipboard and supporting personnel transitioned to both the Navy and Army. The device enhances productivity, reduces manning requirements, and supports combat readiness and operational effectiveness. Source: GAO analysis; military department SBIR programs documentation. [End of table] SBIR program officials within the military departments emphasized that, in addition to their broader program efforts to identify transition outcomes, some acquisition organizations have implemented their own practices to track transition. For example, the Navy Program Executive Office for Submarines tracks the transition of SBIR technologies to its acquisition programs by managing a list of companies, the value of contract awards, the specific program office associated with each contract award, and the SBIR technology associated with the award. The office indicated that 20 active Phase III awards associated with its acquisition program efforts are being tracked. DOD Is in the Early Stages of Developing a Plan to Improve Technology Transition Tracking and Reporting: The National Defense Authorization Act for Fiscal Year 2012 mandated that DOD report new transition-related information to the Administrator of the Small Business Administration who will report this information annually to designated congressional committees. This reporting will include information on the number and percentage of Phase II projects that transition into acquisition programs or to fielded systems, the efficacy of steps taken by DOD to increase the number of transitioned projects, and additional information specific to the transition of projects funded through Commercialization Readiness Programs. In order to provide more complete and accurate transition data to support the new reporting requirements, DOD recognizes it may need to modify its existing data systems or develop new tools to better capture the transition results for SBIR projects. According to the Office of Small Business Programs, DOD's response to the new reporting requirements is still being evaluated, in part because there are several challenges to compiling complete and accurate technology transition data. One such challenge we found was variation across the military departments in their definitions of technology transition. Specifically, transition definitions ranged from any commercialization dollars applied to a project, to only when a technology is actually incorporated into a weapon system or in direct use by the warfighter. The Office of Small Business Programs acknowledged that a standard DOD SBIR definition of technology transition must be ensured before the congressionally-required reporting begins. Standards for internal control state that management should establish procedures to ensure that it is able to achieve its objectives, such as being able to compile and report consistent, complete, and accurate data.[Footnote 8] Additionally, according to SBIR officials, tracking transition outcomes can be challenging because the sometimes lengthy period between SBIR project completion and transition to a DOD user can obscure a project's SBIR linkages. Time lags can occur because of delays in transition funding availability, additional development or testing needs before transition, or schedule delays encountered by intended users. During the time between project completion and transition, personnel associated with projects may change and technologies may evolve. This increases the likelihood that transitions associated with SBIR technologies go unacknowledged. SBIR officials within the military departments also stated that limited resources for administrative activities constrain their ability to effectively follow up on the transition outcomes for completed projects. Although the Office of Small Business Programs acknowledges the limitations of CCR data, the initial plan is to use this data source-- viewed by DOD as the best available--as the primary means for beginning to address the new transition reporting requirements. Additionally, in an effort to improve DOD's future technology transition reporting and its understanding of transition results in general, the Office of Small Business Programs has initiated an assessment of different options for enhancing transition data. For example, as part of this assessment, DOD is examining whether CCR could be modified to improve reporting. Additionally, existing DOD reporting mechanisms, such as Selected Acquisition Reports--annually required for major defense acquisition programs--are being considered as potential vehicles for supporting SBIR technology transition reporting. Opportunities to build more SBIR awareness directly into acquisition activities are being considered as well, such as including provisions in acquisition strategy documents or formal program reviews. According to the Office of Small Business Programs, DOD intends to issue a policy directive in fiscal year 2014 that will provide guidance for implementing overall SBIR program requirements. However, SBIR officials indicated that addressing technology transition reporting requirements is viewed as a longer-term effort because of the challenges we have discussed, and no specific plan including a time line has been established for when DOD will be able to support those requirements. Without a plan that establishes a time line, it is unclear how and when DOD will begin to provide the technology transition information expected by Congress. Although Congress did not specify when reporting was to begin, it expects DOD to report new transition-related information to the Administrator of the Small Business Administration to meet the National Defense Authorization Act for Fiscal Year 2012 requirement. However, as stated above, DOD expects this to be a longer-term effort and designated congressional committees may not be aware of when DOD will likely have developed the capability to provide comprehensive and accurate data. Further, unless DOD communicates its plan and accompanying time line, these committees may be unaware that the transition-related information DOD plans to provide in the near-term to address the National Defense Authorization Act for Fiscal Year 2012 requirements has data quality issues. Standards for internal control emphasize the need for federal agencies to establish plans to help ensure goals and objectives can be met, including compliance with applicable laws and regulations. Further, communicating internal control efforts on a timely basis to external stakeholders, such as congressional committees, helps ensure that effective oversight can take place. [Footnote 9] Conclusions: The SBIR program efforts within DOD provide opportunities for small businesses to develop new technologies that may improve current U.S. military capabilities and provide innovative solutions to address future needs of the warfighter. However, information on technology transition outcomes for SBIR projects is limited. Consequently, DOD cannot identify the extent to which the program is supporting military users. The Office of Small Business Programs is taking steps to respond to new technology transition reporting requirements, but has not yet determined how and when it will more completely and reliably track and report on the extent of transition for SBIR technologies. While initial reporting efforts are expected to use existing data systems, such as CCR, DOD will need to overcome the inherent limitations of data collected through those systems if it expects to provide a comprehensive picture of transition outcomes. Recommendations for Executive Action: To improve tracking and reporting of technology transition outcomes for SBIR projects, we recommend that the Secretary of Defense direct the Office of Small Business Programs to take the following three actions: 1. Establish a common definition of technology transition for all SBIR projects to support annual reporting requirements; 2. Develop a plan to meet new technology transition reporting requirements that will improve the completeness, quality, and reliability of SBIR transition data; and: 3. Report to Congress on the department's plan for meeting the new SBIR reporting requirements set forth in the program's fiscal year 2012 reauthorization, including the specific steps for improving the technology transition data. Agency Comments and Our Evaluation: We provided a copy of a draft of this report to DOD for review and comment. Written comments from the department are included in appendix II of this report. DOD partially concurred with our recommendations. In its response, DOD stated that it has established a working group that is currently working with all stakeholders to develop a common definition of technology transition for all SBIR projects. DOD also agreed that it is important to improve the completeness, quality, and reliability of SBIR transition data, but noted that it has significant concerns related to the difficulty in actually capturing the data. The department indicated that the full scope of data collection challenges and associated resource needs is unknown at this time. While we recognize there are challenges to improving transition data, we believe there are avenues already available that DOD could pursue to improve transition data that may not require extensive resource commitments. For example, DOD's SBIR program could work more closely with its acquisition community to track transition outcomes. As outlined in this report, some acquisition organizations have developed their own practices to track transition outcomes, which the program may be able to leverage for use on a broader scale. In addition, DOD could consider greater use of contracting provisions to require contractors to report on SBIR project activities, or use existing program reporting mechanisms, such as Selected Acquisition Reports, to capture additional transition information. We believe that collection of better data is not only needed to support the congressional reporting requirements, but also to help DOD assess the efficacy of existing transition efforts and the benefits the program yields for the warfighter. DOD stated it will continue with initiatives that seek to improve the collection of SBIR technology transition data. However, it did not specify if or when it intends to develop a plan for meeting the transition reporting requirements. We continue to believe a plan that includes a time line for when DOD will begin to support reporting requirements should be provided to the designated congressional committees in the near term to make clear the limitations of reported transition data and the department's approach to improving the data over time. We are sending copies of this report to appropriate congressional committees and the Secretary of Defense. In addition, this report will be available at no charge on the GAO website at [hyperlink, http://www.gao.gov]. If you or your staff have any questions concerning this report, please contact me at (202) 512-4841 or by email at sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III. Signed by: Michael J. Sullivan: Director, Acquisition and Sourcing Management: List of Committees: The Honorable Carl Levin: Chairman: The Honorable James Inhofe: Ranking Member: Committee on Armed Services: United States Senate: The Honorable Richard Durbin: Chairman: The Honorable Thad Cochran: Ranking Member: Subcommittee on Defense: Committee on Appropriations: United States Senate: The Honorable Howard McKeon: Chairman: The Honorable Adam Smith: Ranking Member: Committee on Armed Services: House of Representatives: The Honorable Rodney Frelinghuysen: Chairman: The Honorable Pete Visclosky: Ranking Member: Subcommittee on Defense: Committee on Appropriations: House of Representatives: [End of section] Appendix I: Scope and Methodology: To identify what processes are used by the Department of Defense (DOD) to facilitate transition for Small Business Innovation Research (SBIR) technologies, we reviewed prior reports by GAO, DOD and other organizations, such as the National Research Council and the Rand Corporation, as well as reviewed DOD policies, procedures, and funding information. Using this information, we scoped our work to focus on the SBIR activities conducted by the Air Force, Army, and Navy. These three organizations typically receive about three-fourths of the annual SBIR funding that supports the 13 participating DOD organizations. With the military department SBIR programs as our focus, we interviewed DOD officials from the Office of Small Business Programs and the SBIR program offices at the Air Force, Army, and Navy on practices and tools used to facilitate technology transition. In addition, we interviewed and collected documentation from DOD officials within the acquisition community concerning their use of and interactions with the SBIR program. Specifically, we interacted with officials at the Air Force Life Cycle Management Center and Air Force Research Laboratory; the Army Aviation and Missile, Research, Development, and Engineering Center; the Naval Sea Systems Command; and the F-35 Joint Program Office. This included interviewing SBIR program management and transition facilitation personnel at each location, as applicable. Similarly, to assess the extent to which SBIR technologies are transitioning to DOD users, we met with officials in the Office of Small Business Programs, military department SBIR program offices, and the aforementioned military acquisition organizations to discuss what data are available to measure transition of SBIR technologies to acquisition programs, or directly to warfighters in the field. We determined that DOD uses two primary data systems--Company Commercialization Reports and the Federal Procurement Data System-Next Generation. We discussed with DOD officials what data are collected by these systems, how the data are validated and used, and whether there are limitations to the data collected. We also reviewed available documentation on the systems. In assessing data limitations, we discussed with SBIR officials whether the systems provide accurate, reliable, and comprehensive data on SBIR projects that transition to military users. In addition, we interviewed military department officials about other data collection practices they may have implemented to track SBIR projects and results. Any limitations that were identified for the data collection practices and data systems used to identify technology transition outcomes for SBIR projects are discussed in this report. We conducted this performance audit from April 2013 to December 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix II: Comments from the Department of Defense: Office of The Under Secretary Of Defense: Acquisition, Technology and Logistics: 3000 Defense Pentagon: Washington, DC 20301-3000: December 17, 2013: Mr. Michael J. Sullivan: Director: Defense Capabilities and Management: U.S. Government Accountability Office: 441 G Street, N.W. Washington, DC 20548: Dear Mr. Sullivan: This is the Department of Defense (DoD) response to the Government Accountability Office (GAO) Draft Report, GAO-14-96, "Small Business Innovation Research: DOD's Program Supports Weapon Systems, But Lacks Comprehensive Data on Technology Transition Outcomes," dated November 18, 2013 (GAO Code 121126). Detailed comments on the report recommendations are enclosed. Sincerely, Signed by: Andre J. Gudger: Director: Office of Small Business Programs: Enclosure: As stated: GAO Draft Report Dated November 18, 2013: GAO-14-96 (GAO CODE 121126): “Small Business Innovation Research: DOD’s Program Supports Weapon Systems, But Lacks Comprehensive Data On Technology Transition Outcomes” Department Of Defense Comments To The GAO Recommendation: Recommendation 1: To improve tracking and reporting of technology transition outcomes for Small Business Innovation Research (SBIR) projects, the Government Accountability Office (GAO) recommends that the Secretary of Defense direct the Office of Small Business Programs to take the following three actions: * Establish a common definition of technology transition for all SBIR projects to support annual reporting requirements; * Develop a plan to meet new reporting requirements that will improve the completeness, quality, and reliability of SBIR technology transition data; and; * Report to Congress on the Department’s plan for meeting the new SBIR reporting requirements set forth in the program’s fiscal year 2012 reauthorization, including the specific steps for improving technology transition data. DoD Response: Partially concur. The DoD Office of Small Business Programs (OSBP), with support from the DoD Service and Component SBIR Program Managers, have established a SBIR/STTR commercialization working group that is currently working with all stakeholders to develop a common definition of technology transition for all SBIR projects. However, with regard to the subsequent recommendations, the Department agrees that it is important to improve the completeness, quality, and reliability of SBIR technology transition data, but has significant concerns related to the difficulty in actually capturing the data. As GAO notes in the report, “tracking transition can be challenging because of the sometimes lengthy period between SBIR project completion and the transition to a DoD user.” While the Department is working towards benchmarking methods that will improve our ability to capture the data, the full scope of challenges and associated resources is unknown at this time. Nevertheless, the Department will continue with initiatives that seek to improve the collection of SBIR technology transition data and to fulfill the SBIR reporting requirements set forth in the program’s fiscal year 2012 reauthorization. [End of section] Appendix III: GAO Contact and Staff Acknowledgments: GAO Contact: Michael J. Sullivan, (202) 512-4841, sullivanm@gao.gov: Staff Acknowledgments: In addition to the contact named above, John Oppenheim, Assistant Director; Danielle Greene; Victoria Klepacz; Sean Merrill; Scott Purdy; and Sylvia Schatz also made key contributions to the report. [End of section] Footnotes: [1] Pub. L. No. 97-219. [2] Every federal agency with a budget of $100 million or more for extramural research and development is required to use a portion of its budget--not less than 2.7 percent in fiscal year 2013--to establish and operate SBIR programs. [3] The 13 DOD agencies with SBIR programs are the Army, Navy, Air Force, Missile Defense Agency, Defense Advanced Research Projects Agency, Chemical Biological Defense, Special Operations Command, Defense Threat Reduction Agency, National Geospatial-Intelligence Agency, Defense Logistics Agency, Defense Microelectronics Activity, Defense Health Program, and the Office of Secretary of Defense. [4] GAO, Defense Technology Development: Technology Transition Programs Support Military Users, but Opportunities Exist to Improve Measurement of Outcomes. [hyperlink, http://www.gao.gov/products/GAO-13-286] (Washington, D.C.: Mar. 7, 2013); and Defense Technology Development: Management Process Can Be Strengthened for New Technology Transition Programs. [hyperlink, http://www.gao.gov/products/GAO-05-480] (Washington, D.C.: June 17, 2005). RAND Corporation, Improving the Department of Defense's Small Business Innovation Research Program Testimony, April 26, 2007; National Research Council, An Assessment of the SBIR Program at the Department of Defense (Washington, D.C.: National Academies Press, 2009). [5] The National Defense Authorization Act for Fiscal Year 2006 authorized the Commercialization Pilot Program under the Secretary of Defense and the Secretary of each military department. Pub. L. No. 109- 163, § 252. The National Defense Authorization Act for Fiscal Year 2012 continued the program and renamed it the Commercialization Readiness Program. Although the program may support any Phase III awards, such as technology transition to commercial products, DOD is required to provide goals to increase the number of Phase II SBIR contracts that lead to technology transition into programs of record or fielded systems and to use incentives to meet those goals. Pub. L. No. 112-81, § 5122(a). [6] The National Defense Authorization Act for Fiscal Year 2012 requires DOD to report transition information to the Small Business Administration's Administrator for inclusion in an annual report to the Committee on Small Business of the Senate, and to the Committee on Science and the Committee on Small Business of the House of Representatives. [7] The percent authorized under the pilot to allow funding for administrative, oversight and contract processing activities--not more than 3 percent--is for commercialization and outreach initiatives that were not in effect until December 31, 2011; it is not associated with the 1 percent available for use under the Commercialization Readiness Program. [8] GAO, Standards for Internal Control in the Federal Government.[hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: Nov. 1, 1999). [9] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [End of section] GAO’s Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select “E-mail Updates.” Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]