This is the accessible text file for GAO report number GAO-13-292 entitled 'National Science Foundation: Steps Taken to Improve Contracting Practices, but Opportunities Exist to Do More' which was released on March 28, 2013. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to the Chairman, House Appropriations Subcommittee on Commerce, Justice, Science, and Related Agencies, Committee on Appropriations, House of Representatives: March 2013: National Science Foundation: Steps Taken to Improve Contracting Practices, but Opportunities Exist to Do More: GAO-13-292: GAO Highlights: Highlights of GAO-13-292, a report to the Chairman, Subcommittee on Commerce, Justice, Science, and Related Agencies, Committee on Appropriations, House of Representatives. Why GAO Did This Study: The NSF spends more than $400 million of its $7 billion annual budget acquiring goods and services in support of its mission to promote science and engineering. Much of this spending involves exploration activities in remote locations throughout the world, such as the Arctic and Antarctic. GAO examined the extent to which NSF uses key contracting practices in three phases of the acquisition process: (a) acquisition planning, (b) contract award, and (c) post-award contract monitoring. GAO selected and reviewed a nongeneralizable sample of 11 contracts or orders with at least $3 million in funding obligations for fiscal year 2011, which accounted for about 70 percent of NSF’s total contract obligations for that year. Although all 11 contracts and orders received funding during fiscal year 2011, some were awarded more than 7 years ago. Some were awarded more recently. We reviewed each of the 11 contracts to determine the extent to which they reflected the use of key contracting practices based on the Federal Acquisition Regulation, our prior work, and NSF-OIG findings. GAO also reviewed NSF contracting policies and met with NSF contracting and program officials. What GAO Found: For the contracts GAO reviewed, the National Science Foundation (NSF) generally used key contracting practices in each of the three phases of the acquisition process, but the agency needs additional guidance on early acquisition planning as well as arrangements for contract audits. The three phases of the process and key practices are shown in the figure below: Figure: Key Practices in the Acquisition Process: [Refer to PDF for image: illustration] Acquisition planning: * Acquisition plan development; * Market research; * Contract type. Contract award: * Price; * Cost accounting system reviews; * Cost accounting disclosure. Contract monitoring: * Monitoring plans; * Monitoring activities; * Incurred cost audits. Source: GAO analysis of acquisition process. [End of figure] The contracts GAO reviewed all involved some degree of acquisition planning, but NSF’s guidance does not address appropriate time frames for early planning activities. Without such guidance, NSF contract and program officials said they could not convince their colleagues of the need to initiate early planning activities. Delays in these activities can lead to further delays later. For example, NSF had to extend one order on a non-competitive basis for more than a year to complete planning tasks for the follow-on order. In another case, the delayed award of an order compressed the data collection period for a report with firm deadlines, which could lead to higher overall costs. Further, having sufficient time for early planning may facilitate an increased use of lower risk contracting approaches. Contract documentation showed that NSF generally followed key practices in the award phase. An NSF corrective action plan, in response to NSF’s Office of Inspector General’s (NSF-OIG) 2009 financial statement audits, clarifies the agency’s procedures for reviewing contractors’ accounting practices and financial disclosure statements to better align with key practices. Contract file documentation shows NSF improved in this area, with most of the negotiated contracts having documentation of accounting system reviews. Further, NSF generally documents price reasonableness determinations. NSF updated its guidance and took steps to incorporate key contract monitoring practices. NSF-OIG’s 2009 financial statement audits recommended that NSF obtain incurred cost submissions and audits for its largest cost-reimbursable contracts to ensure the validity of costs billed to NSF. Around the same time, the NSF-OIG and the NSF Office of the Director signed a memorandum of understanding (MOU) that provides a process for arranging for contract audits. Audits for one of the ocean drilling contracts completed in 2012 resulted in $1.5 million in recovered funds. The NSF Director and NSF-OIG have both identified additional audits of this contract as a top priority. However, despite the terms of the MOU, and the agreement between NSF and the NSF-OIG on the need for further audits, arrangements have not been made to conduct additional audits of this contract for more recent fiscal years, according to officials. Similarly, despite requests from the contracting officer, NSF has not made arrangements for incurred cost audits for another large contract GAO reviewed. What GAO Recommends: GAO recommends that the Director of NSF (1) supplement existing guidance on acquisition planning to address the time needed for the early stages of the process, and (2) arrange for audits to be performed on major contracts, consistent with the terms of the memorandum of understanding with NSF-OIG. NSF agreed with the recommendations. View GAO-13-292. For more information, contact William T. Woods at (202) 512-8214 or woodsw@gao.gov. [End of section] Contents: Letter: Background: Acquisition Planning Practices Conducted to Varying Degrees, but NSF Lacks Guidance on Time Needed for Early Planning Activities: NSF Generally Followed Key Practices in the Award Phase: NSF Followed Plans for Monitoring Performance, but Incurred Costs Audits Were Not Completed Regularly: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Scope and Methodology: Appendix II: Comments from the National Science Foundation: Appendix III: GAO Contact and Staff Acknowledgments: Tables: Table 1: Use of Key Practices in the Acquisition Planning Phase for the 11 Contracts/Orders Reviewed: Table 2: Use of Key Practices in Contract Award Phase for the 11 Contracts/Orders Reviewed: Table 3: Use of Key Practices in Contract Monitoring Phase for the 11 Contracts/Orders Reviewed: Figures: Figure 1: Key Activities in the Acquisition Process: Figure 2: Contracting Obligations by Acquisition Method, Fiscal Year 2011: Figure 3: Acquisition Planning Phases: Abbreviations: CAS: Cost Accounting Standards: DACS: Division of Acquisition and Cooperative Support: DCAA: Defense Contract Audit Agency: FAR: Federal Acquisition Regulation: IT: Information Technology: MOU: Memorandum of Understanding: NSF: National Science Foundation: NSF-OIG: National Science Foundation Office of Inspector General: T&M: Time and Materials: [End of section] United States Government Accountability Office: Washington, DC 20548: March 28, 2013: The Honorable Frank Wolf: Chairman, Subcommittee on Commerce, Justice, Science, and Related Agencies: Committee on Appropriations: House of Representatives: Dear Mr. Chairman, The National Science Foundation (NSF) spends more than $400 million annually acquiring the goods and services it needs to carry out its mission to promote science and engineering. Much of NSF's acquisition spending involves exploration activities in remote and austere locations throughout the world. Our work at several federal agencies has highlighted the importance of sound contracting practices--such as adequate time for planning, sufficient market research, and effective contract monitoring--as critical to a strong foundation for successful acquisition outcomes. But in recent years, we and others, including the NSF Office of the Inspector General (NSF-OIG), have identified various challenges regarding the contracting practices at NSF, particularly involving the agency's use of cost-type contracts. You requested that we assess NSF's contracting practices. Specifically, our objective was to assess the extent to which NSF incorporates key contracting practices in the three major phases of the acquisition process: (a) acquisition planning, (b) contract award, and (c) post-award contract monitoring. To do so, we reviewed a nongeneralizable sample of 11 contracts and contract orders with funding obligations of at least $3 million each in fiscal year 2011, the latest year for which data were available when we began our work. We selected contracts and orders to reflect a mix of program offices and a variety of contract types, such as fixed-price and cost- reimbursement. The sample we used is not generalizable to the universe of all contracts at NSF, but the 11 contracts and orders represent 70 percent of the total dollars obligated by NSF on all its contracts and orders during 2011.[Footnote 1] The 11 contracts and orders consisted of 4 contracts for which NSF used the negotiation process set forth in Part 15 of the Federal Acquisition Regulation (FAR) and 7 orders under existing contracts, which involved the use of streamlined procedures described in other parts of the FAR. Although all 11 contracts were active during the time of our review, some of the selected contracts were awarded more than 7 years ago--before NSF updated its contracting manual--and some more recently. As more fully described in appendix I, we determined the extent to which each of the contracts and orders in our sample reflected the use of selected key contracting practices. We selected the key practices based on requirements and principles in the FAR, practices identified in our prior work, or matters raised in NSF- OIG recommendations. In our view, each of the selected practices is critical to the foundation of a successful acquisition. We conducted this performance audit from February 2012 to March 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: The acquisition process at federal agencies generally consists of three phases: (1) acquisition planning; (2) contract award; and (3) contract monitoring. Each phase involves a number of key activities, as shown in figure 1: Figure 1: Key Activities in the Acquisition Process: [Refer to PDF for image: illustration] Acquisition planning: * Acquisition plan development; * Market research; * Contract type. Contract award: * Price; * Cost accounting system reviews; * Cost accounting disclosure. Contract monitoring: * Monitoring plans; * Monitoring activities; * Incurred cost audits. Source: GAO analysis of acquisition process. [End of figure] In the acquisition planning phase, agencies establish their requirements and develop a plan to meet those requirements. Both program and contracting officials participate in acquisition planning activities.[Footnote 2] During this phase, agencies conduct market research to determine what products or services are available and on what terms. They select a contracting approach best suited to the nature of the acquisition, addressing among other things, the availability of existing contracts, extent of competition required, and the most appropriate contract type, such as cost-reimbursable or fixed-price. In the award phase, agencies solicit bids, quotes, or proposals from prospective vendors, depending on the contracting method selected. In negotiated acquisitions, they evaluate the submissions from vendors under established evaluation criteria in the solicitation and award a contract to the vendor representing the best value to the government, based on a combination of technical and cost factors. Agencies follow a similar process when ordering from the Federal Supply Schedule, where quotes from contractors are evaluated using stated evaluation criteria and orders are awarded to the contractor that would provide the best value and offers the lowest overall cost alternative. In the contract monitoring phase, agencies engage in a range of activities intended to ensure that the contractor delivers according to the terms of the contract. These activities often are described in detail in a contract surveillance plan, sometimes called a quality assurance surveillance plan. For cost-reimbursement contracts, agencies may arrange for an audit of costs incurred by the contractor. These audits may be performed by entities such as the agency inspector general or the Defense Contract Audit Agency (DCAA). NSF spends most of its annual budget of about $7 billion to fund grants to universities and other research entities, but the agency also spent more than $446 million in fiscal year 2011 acquiring goods and services in support of its mission. The largest of these acquisitions involved contracts for logistics support of scientific missions in the Arctic and Antarctica, as well as ocean-drilling projects in various locations. For these types of large-scale projects, NSF uses the negotiated contracting procedures of Part 15 of the FAR. NSF uses negotiated contracting methods for about 66 percent of its contract spending, as shown in figure 2. For another 32 percent of its contract spending, NSF uses a variety of more streamlined contracting methods allowed under the FAR. These include placing orders under Federal Supply Schedule contracts awarded by the General Services Administration or other pre-existing contracts.[Footnote 3] Placing orders under existing contracts is often a more simplified approach than awarding a new contract. The remaining 1 percent or so of NSF contract spending is through various other methods, such as interagency agreements with the U.S. Navy for deep sea research vessel certification. Figure 2: Contracting Obligations by Acquisition Method, Fiscal Year 2011: [Refer to PDF for image: pie-chart] Negotiated acquisitions following FAR Part 15 procedures: 66.2%; Streamlined acquisitions such as those using the Federal Supply Schedule: 32.5%; Other: 1.3%. Source: GAO analysis of FPDC data. [End of figure] The Division of Acquisition and Cooperative Support (DACS) at NSF is responsible for the solicitation, negotiation, award, and administration of the agency's contracts for NSF's research facilities and major programs. DACS oversees NSF procurement systems, contracts policy, processes and guidance. This Division is under the Office of Budget, Finance, and Award Management which reports to the Office of the Director. The Office of Inspector General provides independent oversight of the agency's programs and operations, including contracts. The NSF-OIG is responsible for promoting efficiency and effectiveness in agency programs and for preventing and detecting fraud, waste, and abuse. By statute, the NSF-OIG is under the general supervision of the National Science Board and reports to the Board and Congress. Much of NSF's contracting activity is for recurring needs, such as logistics support for its facilities in the polar regions, data collection, or surveys. For example, the National Survey of Recent College Graduates began in 1973 and continues today. In our prior work on acquisition planning practices, we found that documenting decisions, particularly when there is frequent staff turnover, is key to providing insight for subsequent contracts. Specifically, we found that documenting cost estimates is particularly important to help ensure the information is available when planning for follow-on contracts.[Footnote 4] Incorporating lessons learned from prior acquisitions can help further refine requirements and strategies when planning for future acquisitions. NSF officials must decide on a contract pricing arrangement for every contract or order. The major categories of pricing arrangements NSF uses are fixed-price, time-and-materials, and cost-reimbursement. Under a fixed-price contract, the government generally pays a firm price and may also pay an award or incentive fee related to performance. In a time-and-materials contract or order, the government pays a set amount for every hour of service the contractor provides, plus the cost of any materials used. Because the number of hours to be provided is dependent on a number of factors, this type of contract requires an enhanced level of government oversight. When using a cost- reimbursement contract, the government agrees to reimburse all the allowable costs incurred by the contractor as prescribed in the contract. These types of contracts can be risky because the government agrees to pay for costs incurred regardless of the outcome achieved. Cost-type contracts that exceed certain dollar thresholds generally are subject to the cost allocation rules of the government's Cost Accounting Standards (CAS), and in these cases the contractor generally is required to disclose its cost accounting practices in a CAS Disclosure Statement. We previously reported on the use of cost- reimbursement contracts at several agencies, including NSF, finding that agencies frequently did not document why they selected this type of contract.[Footnote 5] Financial statement audits performed by an independent accounting firm on behalf of the NSF-OIG for fiscal years 2009 and 2010 identified significant deficiencies related to the use and monitoring of cost- reimbursement contracts at NSF. Specifically, the audits found that NSF did not ensure the adequacy of contractor accounting systems prior to award or the validity of costs incurred on the contract. In 2011, however, the same firm concluded that the concern had been addressed through the adoption of new policies and procedures. While we were conducting our audit work, NSF was in the process of conducting a self- assessment of its acquisition function in accordance with the Office of Management and Budget (OMB) Circular A-123. The agency also retained a consulting firm to review its self-assessment. In July 2012, the firm issued a report summarizing its findings. We did not assess the methodology, findings, or conclusions of either the NSF self-assessment or the consulting firm's review. In October 2012, NSF updated its contracting manual to incorporate a number of changes. For example, NSF reorganized the manual to align it with the FAR and added additional guidance to address the deficiencies identified in the financial audits. All of the contract activities in our review were subject to prior versions of the contracting manual. Acquisition Planning Practices Conducted to Varying Degrees, but NSF Lacks Guidance on Time Needed for Early Planning Activities: The NSF contract files we reviewed reflected the use of selected key acquisition planning practices to varying degrees, but the agency has not provided guidance on the time needed to complete early planning phase activities. Allowing sufficient time to plan procurements may facilitate an increased use of lower risk contracting vehicles by providing time for the contracting officer to consider including more fixed-priced elements. Our observations of the use of some of the key practices for acquisition planning activities are summarized in table 1, and explained in more detail below. Table 1: Use of Key Practices in the Acquisition Planning Phase for the 11 Contracts/Orders Reviewed: Acquisition planning: Key practices: Acquisition Plan. Comprehensive plan for fulfilling the agency need; Negotiated acquisitions: Used to some degree; Streamlined acquisitions: Used to some degree. Key practices: Market Research. Collecting and analyzing information about capabilities within the market; Negotiated acquisitions: Always or almost always used; Streamlined acquisitions: Always or almost always used. Key practices: Contract Type Rationale. Documents the agency's choice of contract type; Negotiated acquisitions: Used to some degree; Streamlined acquisitions: Used to some degree. Source: GAO analysis of NSF contract files. [End of table] The acquisitions we reviewed all involved some degree of acquisition planning, but the time spent planning and the content of planning documents varied. Planning for the negotiated acquisitions ranged from a few months to more than 6 years, while many of the streamlined acquisitions in our sample had more abbreviated planning periods. Contracting and program officials responsible for one program office told us they often copy planning documents from predecessor orders to compensate for abbreviated planning periods. This practice, however, does not allow for incorporation of new guidance or changing contract requirements. In addition, some of the individual contract acquisition plans for the earlier contracts in our sample did not include details on how the agency planned to evaluate the proposals from competing vendors. Documenting a decision regarding the plan for proposal evaluation is an important component of the acquisition planning phase. Contracting guidance at NSF does not identify the range of time needed to conduct acquisition planning activities for the types of acquisitions methods it employs. Currently, the guidance states that the process of acquisition planning should begin as soon as a program need is identified and it is determined that the need must be met through the use of resources from outside the government. The guidance does not provide any detail, however, on the expected range of time needed to conduct planning activities in the earliest stages of an acquisition when key documents are prepared, such as the statement of work and a cost estimate.[Footnote 6] Acquisition planning usually occurs in three phases, and while NSF has established expected time frames for the latter stages of acquisition planning, the agency has not established such expectations for the earliest planning phase. Figure 3 depicts what we found at NSF. Figure 3: Acquisition Planning Phases: [Refer to PDF for image: illustration] Acquisition Planning Process: [No lead times established] Pre-solicitation; Procurement request; Solicitation (has contracting lead time). Award (has contracting lead time). Source: GAO analysis of agency acquisition planning processes. [End of figure] Allowing sufficient time to plan procurements may provide agencies a better opportunity to clearly define contract requirements, outline source selection procedures, conduct market research to support competition, estimate costs, and consider opportunities for increased use of lower-risk contracting vehicles containing more fixed-priced elements. Conversely, the lack of sufficient time for planning may have adverse effects, such as unplanned delays. For example, NSF had to extend one streamlined order in our review on a non-competitive basis for more than a year and a half in order to complete planning tasks for the follow-on order. The contracting officer used the additional planning time to conduct the analysis needed to incorporate more fixed-priced elements into the new order. Planning for the earlier order did not include documentation of a price history analysis, which, according to contracting officials, may have helped expedite the follow-on planning and was likely due to short planning time frames for the earlier order. In another case, the delayed award of one of the orders in our review caused a compressed period for data collection for a report with firm deadlines. The schedule risk from these delays could lead to higher overall costs. Further, officials from two program offices told us that they would benefit from knowing an expected time range to complete early planning activities. For example, in the absence of guidance on the time needed to complete early planning activities, program and contracting officials responsible for NSF's largest contract told us they had difficulty convincing their colleagues of the appropriate time to initiate contract planning. They added that this acquisition required a number of changes before a follow-on contract could be awarded--some based on updates to the FAR and some based on internal decisions, including the use of a different source selection strategy. Market research is a key element in the acquisition planning phase that provides insight into available sources for the acquisition and may provide information on estimated costs. We found evidence of market research in each of the acquisition plans we reviewed, though the link between the research conducted and the impact on the acquisition strategy was not always clear. For example, the acquisition plan for one streamlined acquisition noted concerns about the lack of offerors for past solicitations. The acquisition plan stated that NSF would use the Federal Supply Schedule and release the request for quotations to six potential offerors, but it did not address how market research impacted this decision. By contrast, NSF engaged in extensive planning for its Integrated Ocean-Drilling Program, including requirements development and market research to identify potential sources to support its mission. According to officials, this planning, which occurred over about a 5-year period, consisted of soliciting interest from more than 30 international institutions using various techniques such as market surveys and sources sought notices. NSF used this multi-year planning period to set up the funding and organizational infrastructure requirements of this complex international program. All of the files we reviewed showed that during the planning phase agency officials had addressed how the contract would be priced. However, the planning documentation for the cost-reimbursable acquisitions in our review did not consistently include assessments of the additional risk and burden these high-risk contracts place on the agency or an assessment of the potential for firmer pricing in future acquisitions.[Footnote 7] Knowing the risk of using a cost- reimbursable contract and identifying opportunities to use a less risky contract type after experience provides a basis for firmer pricing is a sound practice identified by our prior work, by the Department of Defense, and, more recently, in federal regulation. [Footnote 8] Despite the risk associated with cost-type contracts, NSF contracting officials did not document their acknowledgment of this risk for an early contract for the ocean drilling program or whether they would attempt to minimize the future use of a cost-type contracts. Further, in a prior report we noted that NSF's procurements of data collection and analysis services for mandated surveys did not consider pricing history and whether there was a basis to transition to firmer pricing.[Footnote 9] According to NSF officials, when re- awarding these types of survey procurements, staff will make an effort to identify tasks to convert to firmer pricing. In fact, a contracting officer responsible for the survey-related orders in our sample told us he has been conducting analysis to determine what tasks could be transitioned to a fixed-price contract type rather than a time-and- materials contract type. He stressed that some tasks are less suitable for fixed-pricing due to the unknowns and "what ifs" inherent in the work, but his goal is to incorporate fixed-pricing into 70 to 80 percent of each survey order. We identified examples of this transition to firm fixed-price elements in some of NSF's more recently awarded streamlined acquisitions. NSF Generally Followed Key Practices in the Award Phase: Contract documentation for negotiated and streamlined acquisitions showed that NSF generally followed key practices in the award phase. Table 2 summarizes our findings based on the contracts and orders we reviewed. Table 2: Use of Key Practices in Contract Award Phase for the 11 Contracts/Orders Reviewed: Award: Key practices: Price reasonableness. Determination that the price or cost is fair and reasonable; Negotiated acquisitions: Always or almost always used; Streamlined acquisitions: Always or almost always used. Key practices: Cost Accounting System Review and pre-award audits. Help ensure contractor's cost accounting systems are adequate; Negotiated acquisitions: Always or almost always used; Streamlined acquisitions: n/a[A]. Key practices: CAS Disclosure Statement. Describes contractor's cost accounting practices and procedures; Negotiated acquisitions: Always or almost always used; Streamlined acquisitions: n/a. Source: GAO analysis of NSF contract files. [A] Cost accounting system reviews, pre-award audits, and CAS disclosure statement reviews are not applicable to streamlined acquisitions we reviewed. [End of table] Most of the contracts in our sample included price reasonableness determinations, as outlined in both federal regulation and NSF guidance current at the time of our review. For most of the streamlined acquisitions we reviewed, NSF documented reasonable price determinations, including an analysis of the contractor's proposed labor hours and the level of effort. In one case, contracting staff worked with an offeror to obtain lower labor rates that were more in line with the government cost estimate. These actions decreased the cost of the order by approximately 8 percent ($1.2 million). In recent years, NSF has taken steps to address deficiencies related to accounting system and disclosure statement reviews identified in its fiscal year 2009 financial statement audits. Specifically, NSF clarified its CAS disclosure statement and accounting system review procedures to better align with sound practices identified by the NSF- OIG and in federal regulation.[Footnote 10] Contract file documentation indicates that NSF has improved in this area, with most of the negotiated contracts we reviewed having documentation of more recent accounting system and CAS disclosure statement reviews, and the most recent contract having documentation of pre-award audits of all contractors in the competitive range. One of the earlier contracts did not have pre-award audits on file or an accounting system review prior to award. NSF officials told us that they did not think this requirement applied. In another earlier case, the contracting officer waived the requirement for a CAS disclosure statement adequacy determination prior to award with the expectation that the determination would be made shortly after award. However, NSF did not have documentation of the final disclosure statement adequacy determination. NSF Followed Plans for Monitoring Performance, but Incurred Costs Audits Were Not Completed Regularly: NSF updated its guidance and took steps to incorporate sound practices related to contract monitoring, but the agency has not made arrangements for audits of some of the larger contracts we sampled. Our findings are summarized in table 3. Table 3: Use of Key Practices in Contract Monitoring Phase for the 11 Contracts/Orders Reviewed: Contract monitoring: Key practices: Monitoring Plans. Describe how the contract will be monitored; Negotiated acquisitions: Always or almost always used; Streamlined acquisitions: Always or almost always used. Key practices: Monitoring Activities. Help ensure the supplies and services acquired under the contract conform to requirements; Negotiated acquisitions: Always or almost always used; Streamlined acquisitions: Always or almost always used. Key practices: Incurred Cost Audits. Help ensure direct and indirect costs are allowable; Negotiated acquisitions: Used to some degree; Streamlined acquisitions: n/a[A]. Source: GAO analysis of NSF contract files. [A] Incurred cost audits are not required for the type of streamlined acquisitions we reviewed. [End of table] Most of the contracts we reviewed included documentation of surveillance plans outlining how NSF would monitor contractor performance and costs, although one of the streamlined acquisitions did not have the surveillance documents called for in the acquisition plan. Further, we found evidence that at least some monitoring activities occurred for all the procurements we reviewed, though not always as specified in the monitoring plans or using deliverables described in the contract or order. For example, the acquisition plan for a large, information technology (IT) order states that the contractor shall provide "daily, weekly, and monthly progress reports" as well as an IT Management Plan and other ad hoc reports as required, with similar requirements reflected in the order. The contracting officer for this order was not aware of any daily progress reports for this order, and added that the monitoring process for these types of acquisitions depends on the quality of the contractor, noting that for some contracts with few performance issues, the monitoring is less rigorous. In another case, the contracting officer noted that despite the statement of work calling for a Quality Assurance Plan, such a plan would be too restrictive for an IT support contract due to the frequent changes in IT systems. Our prior reports state that without consistent cost surveillance, such as through incurred cost audits, an agency may be exposed to the unnecessary risk of overpaying the contractor.[Footnote 11] Further, NSF-OIG's fiscal year 2009 financial statement audits recommended that NSF obtain incurred cost submissions and audits for its largest cost- reimbursable contracts, depending on materiality and risk, to assure the validity of costs billed to NSF. In response, NSF updated its guidance on incurred cost audits and took the necessary steps to obtain incurred cost audits for its largest contract. Around the same time, in August 2009, NSF-OIG and the NSF Office of the Director signed a memorandum of understanding (MOU) that provides procedures to ensure appropriate coordination between the NSF-OIG and NSF for the performance and funding of contract audits. The MOU indicates that the NSF-OIG will provide, within its resources, appropriated funds necessary to perform contract audits selected for its annual audit plan. The NSF-OIG solicits recommendations from NSF per the MOU and prioritizes its annual audit plan based on this input, its own needs, and a variety of risk factors. The MOU identifies the following factors the NSF-OIG uses to prioritize contract audits: type of contract, materiality, whether NSF is the cognizant agency responsible for contractor oversight, known prior audit concerns, contract administration at other federal agencies, and whether NSF expects to continue to have a relationship with the contractor. For audits that NSF determines necessary that are not in the NSF-OIG audit plan, the MOU states that "NSF will obtain and fund the services of an outside auditor." The contracts branch officials told us that their first option is to ask the NSF-OIG to obtain an audit, and if the NSF- OIG does not complete the contract audit, the branch tries to obtain alternative funding. NSF officials told us, however, that alternate funding requires approval at senior management levels, and contracting staff continue relying on the NSF-OIG as the primary means for obtaining contract audits. The NSF Director and NSF-OIG identified the need for incurred costs audits of an ocean drilling contract in our sample.[Footnote 12] Despite the MOU, the agency has not made arrangements for these audits of the contract. Officials stated for earlier years of this contract, the Contracts Branch identified and provided funds for the contracting officer to initiate audits for this contract through DCAA. According to officials, an audit of a prime subcontractor for this contract resulted in $1.5 million in recovered funds. But at the time of our review, despite agreement on the importance of additional audits, the findings from the prior year's audits, and NSF's continued relationship with the contractor, the agency had yet to make arrangements to plan and fund incurred cost audits for more recent fiscal years for this contract, according to officials. Similarly, despite the contracting officer requesting incurred cost audits for another major contract in our review, the audit did not meet the NSF- OIG priorities. According to officials, NSF has not conducted or planned for audits on this contract. In addition, audits for another major contract we reviewed are not scheduled to be completed until fiscal year 2015, which is about two years after the contract expires. In a recent report, we pointed out that timely closing of contracts, including completing any necessary incurred cost audits, can help the government limit its financial risk and possibly recover improper payments.[Footnote 13] Conclusions: Sound acquisition planning, including cost estimation and identification of the most cost-effective contract type, is important to establishing a strong foundation for successful outcomes for the millions of dollars NSF spends annually on acquisitions. Without sufficient planning time frames to develop acquisition plans that align with sound acquisition practices NSF may have a limited ability to develop a strong foundation for its acquisitions. How long the early acquisition planning activities should take is not covered in existing NSF guidance and will vary based on the complexity of the acquisition. However, without a clear understanding of the time frames needed for the early acquisition planning process, program officials may not know when to start planning or how long the planning will take, potentially increasing the likelihood of poorly prepared documents and contract delays. Better insights into when acquisition planning should begin would help ensure sufficient time to carry out the important acquisition planning activities that are designed to facilitate more successful outcomes. When an acquisition involves substantial uncertainties and the agency deems a cost-type contract as the most appropriate vehicle, contract and program staff need to provide additional oversight to protect the government's interests. NSF has taken steps to address NSF-OIG recommendations to increase contract oversight. NSF has a management responsibility to ensure that adequate resources are available to enable contracting officers to determine that costs billed by contractors are allowable, through incurred cost audits or similar assessments. The process in place to ensure the necessary audits occur requires coordination between the NSF-OIG and the NSF Office of the Director; however, the process has not worked for some of the contracts we reviewed. Further, the Contracts Branch continues to place a strong reliance on the NSF-OIG to provide the resources to obtain the audits. Without a process to ensure audits are conducted in cases when NSF-OIG resources are not available, NSF exposes itself to unnecessary risk and cannot assure the validity of costs billed. Recommendations for Executive Action: We recommend that the Director of NSF take the following two actions: * To help ensure good acquisition outcomes through comprehensive acquisition planning, direct DACS to supplement existing guidance on the time frames for acquisition planning to include a focus on the early stages. * Consistent with the terms of the existing MOU with the Office of the Inspector General, take steps to arrange, and fund as necessary, timely audits of major contracts. Agency Comments and Our Evaluation: We provided a draft of this report to NSF for review and comment. In written comments, NSF agreed with our recommendations. NSF also provided technical comments, which we incorporated as appropriate. NSF comments are reprinted in appendix II. We are sending a copy of this report to the Director of the National Science Foundation. In addition, the report is also available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-4841 or WoodsW@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III. Sincerely yours, Signed by: William T. Woods: Director, Acquisition and Sourcing Management: [End of section] Appendix I: Scope and Methodology: As requested by the Subcommittee on Commerce, Justice, Science, and Related Agencies, House Committee on Appropriations, we reviewed the National Science Foundation's (NSF) contracting practices. Specifically, we assessed the extent to which the NSF incorporates key contracting practices in the three major phases of the contracting process: (a) acquisition planning, (b) contract award, and (c) post- award contract monitoring. Within each contracting phase, we focused our work on selected elements: * Acquisition planning. We focused on the completeness and review of written acquisition plans, market research, contract type determinations, and time frames for planning. We selected these elements because they are critical to the successful planning of a contract and, in one case, had been identified in the past by the NSF Office of the Inspector General (NSF-OIG) as a potential concern. * Contract award. We focused on cost and price analyses, cost accounting system reviews and pre-award audits, and Cost Accounting Standards (CAS) Statement reviews. We selected these elements because they were identified by the NSF-OIG as deficiencies in the past and are essential to determining that the contractor has the ability to complete the contract cost requirements. * Contract monitoring. We focused on the development of monitoring or surveillance plans, monitoring activities, and incurred cost audits. These activities were previously identified by the NSF-OIG as deficiencies and are key to determining if the contractor is performing as expected and within allowable costs. To determine key practices in each of these areas, we relied on prior reports and findings from the GAO, NSF-OIG, and other agencies. Below is the list of GAO reports we relied on: * GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999); * GAO, Contract Management: Trends and Challenges in Acquiring Services, [hyperlink, http://www.gao.gov/products/GAO-01-753T] (Washington, D.C. May 22, 2001); * GAO, Defense Contracting: Improved Insight and Controls needed Over DOD's Time-and-Materials Contracts, [hyperlink, http://www.gao.gov/products/GAO-07-273] (Washington, D.C.: June 29, 2007); * GAO, Contract Management: Extent of Federal Spending under Cost- Reimbursement Contracts Unclear and Key Controls Not Always Used, [hyperlink, http://www.gao.gov/products/GAO-09-921] (Washington, D.C.: Sept. 30, 2009); and: * GAO, Acquisition Planning: Opportunities to Build Strong Foundations for Better Services Contracts, [hyperlink, http://www.gao.gov/products/GAO-11-672] (Washington, D.C.: Aug. 9, 2011). We also reviewed internal NSF guidance and the Federal Acquisition Regulation (FAR) for additional key practices. To determine the extent to which NSF's contracting practices incorporate key practices and address prior NSF-OIG recommendations, we reviewed a nongeneralizable sample of 11 contracts and orders with funding obligations over $3 million in fiscal year 2011, the latest year for which data were available when we began our work.[Footnote 14] We used a risk-based approach to select our sample to ensure it included NSF acquisitions with the highest obligation dollar amount. The 11 contracts and orders selected for review represent 70 percent of total contract obligations in fiscal year 2011 and reflect a mix of program offices, a range of obligation amounts, and a variety of contract types, such as fixed-price and cost-reimbursement. We selected four contracts for which NSF used the negotiation process set forth in Part 15 of the Federal Acquisition Regulation and seven orders on existing contracts for which NSF used streamlined procedures described in other parts of the FAR. The four negotiated acquisitions in our sample are cost-reimbursement contracts and represent about 56 percent of NSF's total fiscal year 2011 contract obligations and about 80 percent of the obligations in our sample. The seven streamlined acquisitions represent about 14 percent of NSF's fiscal year 2011 contract obligations and 20 percent of the obligations in our sample. One of the seven streamlined orders is a hybrid contract type using fixed-price and time-and-materials (T&M) elements; one is a cost- reimbursable order; and the other five are T&M orders. Although the 11 contracts were active during the time of our review , some of the selected contracts were awarded more than 7 years ago--before NSF updated its contracting manual to provide more procedural guidance-- and some more recently. We reviewed the files for the selected contracts and used practices identified in the FAR, NSF internal guidance, and prior GAO reports to assess NSF's use of key practices and procedures for the acquisition planning, award, and contract monitoring phases. In addition to contract file review, we met with contract and program officials to confirm our understanding of information in the contract files and of NSF's practices and procedures as evidenced by the contract files. We also reviewed and considered additional documentations provided by the program and contract officials that were not maintained in the contract files. To assess progress NSF made in response to prior NSF-OIG findings, we reviewed prior NSF-OIG recommendations and corrective action plans. We met with NSF-OIG officials to better understand their recommendations related to our review and used this information to provide assessments of progress made in response to these findings. NSF was in the process of a full acquisition system assessment when we initiated our review. While we were completing our audit work, NSF issued a review of its acquisition function in July 2012. While we met with the internal controls officials involved in this review to understand their process, we did not assess the NSF review as part of this review. We conducted this performance audit from February 2012 to March 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix II: Comments from the National Science Foundation: National Science Foundation: Office of the Director: 4201 Wilson Boulevard: Arlington, Virginia 22230: March 22, 2013: Mr. William T. Woods: Director, Acquisition and Sourcing Management: United States Government Accountability Office: Washington, DC 20548: Dear Mr. Woods: The National Science Foundation (NSF) appreciates the opportunity to comment on the draft report Steps Taken to Improve Contracting Practices but Opportunities Exist to do More (GAO-13-292). NSF is committed to implementing a sound and effective acquisition system. We are specifically appreciative of GAO's recognition that NSF has made substantial improvements in our contracting practices. Consistent with NSF's efforts to continuously enhance business procedures and operations, NSF agrees that the report's recommendations can lead to further improvements. In closing, NSF wishes to acknowledge the dedication and professionalism of the GAO team responsible for this review. We look forward to receiving the final report from GAO. Sincerely, Signed by: Cora B. Marrett: Deputy Director: [End of section] Appendix III: GAO Contact and Staff Acknowledgments: GAO Contact: William T. Woods, (202) 512-4841 or woodsw@gao.gov: Staff Acknowledgments: In addition to the contact named above, Penny Berrier, Assistant Director, Caryn Kuebler, Margaret Childs, Danielle Greene, Jeffrey Hartnett, Julia Kennon, Jean McSween, Emily Owens, Ken Patton, Erin Schoening, Roxanna Sun, and Alyssa Weir also made key contributions to this report. [End of section] Footnotes: [1] One of the contracts we selected based on 2011 obligations had expired and was replaced with a new contract awarded in December 2011. To provide a more current assessment we reviewed the contract awarded in fiscal year 2012. [2] GAO, Acquisition Planning: Opportunities to Build Strong Foundations for Better Services Contracts, [hyperlink, http://www.gao.gov/products/GAO-11-672] (Washington, D.C.: Aug. 9, 2011); and FAR 7.102. [3] The Federal Supply Schedules program consists of contracts awarded by GSA or the Department of Veterans Affairs for similar or comparable goods or services, established with more than one supplier, at varying prices. Federal Acquisition Regulation (FAR) § 8.401 and § 8.402. The program offers a large group of commercial products and services ranging from office supplies to information technology services. [4] GAO, Contract Management: Extent of Federal Spending under Cost- Reimbursement Contracts Unclear and Key Controls Not Always Used, [hyperlink, http://www.gao.gov/products/GAO-09-921] (Washington, D.C.: Sept. 30, 2009). [5] [hyperlink, http://www.gao.gov/products/GAO-09-921]. [6] Prior to the October 2012 Acquisition Manual, the "requirements package" was known as the "request for contract" package. [7] While assessing risk and identifying opportunities to use a less risky contract is a sound practice, there was no requirement to document this analysis prior to FAR changes effective March 16, 2011. 76 Fed. Reg. 14,543 (interim rule). 77 Fed. Reg. 12,925 (March 2, 2012) (final rule, effective April 2, 2012)). FAR § 16.103(d). [8] [hyperlink, http://www.gao.gov/products/GAO-09-921]; GAO, Defense Contracting: Improved Insight and Controls needed Over DOD's Time-and- Materials Contracts, [hyperlink, http://www.gao.gov/products/GAO-07-273] (Washington, D.C.: June 29, 2007); DOD, September 2004 Defense Procurement and Acquisition Policy Memo, Requirements for Service Contracts; and FAR Part 16. [9] [hyperlink, http://www.gao.gov/products/GAO-09-921]. [10] Cost Accounting Standards generally apply to cost-reimbursable contracts above a certain dollar threshold. [11] [hyperlink, http://www.gao.gov/products/GAO-09-921]; GAO, DCAA Audits: Widespread Problems with Audit Quality Require Significant Reform, [hyperlink, http://www.gao.gov/products/GAO-09-468] (Washington, D.C.: Sept. 23, 2009); Defense Contracting: DOD Initiative to Address Audit Backlog Shows Promise, but Additional Management Attention Needed to Close Aging Contracts, [hyperlink, http://www.gao.gov/products/GAO-13-131] (Washington, D.C.: Dec. 18, 2012). [12] Incurred cost audits are an important tool that enables management to assess a contractor's compliance with the financial terms and conditions of a contract. [13] [hyperlink, http://www.gao.gov/products/GAO-13-131]. [14] One of the contracts we selected based on 2011 obligations had expired and was replaced with a new contract awarded in December 2011. To provide a more current assessment we reviewed the contract awarded in fiscal year 2012. [End of section] GAO’s Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select “E-mail Updates.” Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]