This is the accessible text file for GAO report number GAO-14-14 entitled 'Federal Real Property: GSA Should Clarify Savings Goals for the National Broker Contract Program' which was released on December 2, 2013. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Federal Real Property: GSA Should Clarify Savings Goals for the National Broker Contract Program: Federal Real Property: Report to Congressional Requesters: October 2013: GAO-14-14: GAO Highlights: Highlights of GAO-14-14, a report to congressional requesters. Why GAO Did This Study: Approximately half of the federal government’s 1.2 million employees work in space leased from the private sector. For fiscal year 2013, GSA expected to lease approximately 197 million square feet at a cost of about $5.2 billion. Since 2005, GSA has acquired some of this leased space using its NBC program to enlist commercial real estate brokerage firms to negotiate with building owners on behalf of the government. In 2012, GSA relied on this contract to replace approximately 33 percent of its expiring leases. GAO was asked to review GSA’s NBC program. This report examines (1) NBC’s program goals and the extent to which they are being realized and (2) the changes GSA has made to the program and what challenges and suggestions for improvement, if any, key stakeholders identified. GAO reviewed NBC contract documentation, agency documents, relevant legislation and regulations, and available data on leasing tasks assigned to brokers. GAO interviewed officials from GSA headquarters, all 11 GSA regions, and representatives of all 4 brokers currently participating in the program. What GAO Found: While the General Services Administration (GSA) has used the National Broker Contract (NBC) program to assist with the agency’s lease portfolio, it is unclear whether the program has resulted in the rental rate cost savings that GSA anticipated when proposing the program. GSA officials have stated that brokers should be able to obtain lower rental rates than in-house staff because brokers have greater market expertise and in addition are able to credit a portion of the broker’s commission to the rental rate. In 2012, when GSA attempted to compare rental rates negotiated by brokers with those negotiated by in-house staff, the agency not only found little difference between the two, but also stated that the data were insufficient to conduct a meaningful comparison. In April 2013, GSA began requiring the use of a different market rent data report-— “Bullseye”—-which includes market information, analysis, and insight regarding the local submarket. Officials said this new data would improve their ability to compare rental rates negotiated by brokers to market rental rates, but will likely not allow officials to determine whether brokers are negotiating better deals than in-house staff. Beginning in fiscal year 2013, GSA also deemphasized previous annual goals for broker use and began identifying and assigning leases to brokers based on where agency officials believe the brokers provide the greatest value. GSA’s goals and metrics for evaluating the NBC program have not been linked to the anticipated cost savings in rental rates. As a result, GSA has no means of evaluating and reporting on this aspect of the program and the value of the NBC program in terms of cost savings continues to be unclear. Clarifying goals and linking them to cost savings would also serve as a way to be transparent with Congress and other stakeholders about the program’s purpose and how GSA plans to monitor and achieve its objectives. Clear, linked goals are especially important given that GSA is planning to seek a third generation of broker contracts in 2015. Officials from all 11 GSA regional offices and representatives from all 4 broker firms identified challenges and made suggestions to improve the program. Two areas frequently cited are: the broker evaluation process and greater flexibility using brokers. * Broker evaluation process: Both GSA regional officials and brokers expressed dissatisfaction with the current broker performance evaluation system and brokers suggested additional guidance and training would help. Both groups also suggested that GSA officials be allowed to use region-specific performance evaluation data to award tasks. * Greater flexibility using brokers: Regional officials suggested that greater flexibility in applying the NBC program would allow them to make better use of brokers. Regional officials described a preferred “ menu of services” that would allow them to use the brokers more selectively for the specific tasks where GSA wants assistance as opposed to working with the broker throughout the entire lease procurement. GSA has begun identifying potential changes to the structure and administration of the NBC program as the agency develops a strategy for the third generation of NBCs. For example, officials told GAO they are examining alternatives to the current commission and rent structure. What GAO Recommends: GAO recommends that GSA ensure the program’s goals are linked to cost savings achieved through the use of NBC brokers, and develop and implement a means of evaluating and reporting results. GSA concurred with the recommendation. View [hyperlink, http://www.gao.gov/products/GAO-14-14]. For more information, contact David Wise at (202) 512-2834 or wised@gao.gov. [End of section] Contents: Letter: Background: Achievement of Goals Is Mixed, and Cost Savings from the NBC Program Are Unclear: GSA Has Made Changes to the NBC Program, and Stakeholders Have Suggested Additional Actions That Can Improve the Program: Conclusion: Recommendation for Executive Action: Agency Comments: Appendix I: Objectives, Scope, and Methodology: Appendix II: Comments from General Services Administration: Appendix III: GAO Contact and Staff Acknowledgments: Figures: Figure 1: Description of Typical Steps for GSA's Real Property Lease Acquisition: Figure 2: GSA Data on Cost of Leased Space Relative to Industry Market Rates, 2006 to 2012: Figure 3: Comparison of One Broker Firm's 2012 Single Regional and Overall National Broker Performance Evaluation Scores: Abbreviations: COTR: Contracting Officer's Technical Representative: FAR: Federal Acquisition Regulation: GSA: General Services Administration: IG: Inspector General: MSA: metropolitan statistical area: NBC: National Broker Contract: [End of section] GAO: United States Government Accountability Office: 441 G St. N.W. Washington, DC 20548: October 31, 2013: The Honorable Thomas R. Carper: Chairman: Committee on Homeland Security and Governmental Affairs: United States Senate: The Honorable Claire McCaskill: Chairman: Subcommittee on Financial and Contracting Oversight: Committee on Homeland Security and Governmental Affairs: United States Senate: Approximately half of the federal government's 1.2-million employees work in space leased from the private sector. As the federal government's principal landlord, the General Services Administration (GSA) is responsible for acquiring and managing a significant amount of this leased space for federal agency customers.[Footnote 1] GSA reports that for fiscal year 2013, the agency expected to lease approximately 197 million square feet at an annual cost of about $5.2 billion. In some cases, GSA uses its National Broker Contract (NBC) to enlist commercial real-estate brokerage firms (i.e., brokers) to negotiate with building owners on behalf of the federal government and acquire leased space. In 2012, it relied on this contract to acquire approximately 33 percent of its expiring leases.[Footnote 2] The NBC program began in 2004, when GSA awarded contracts to four private broker firms that have a nationwide presence, and GSA began enlisting these firms to negotiate with building owners on behalf of the federal government in 2005. The agency expected to realize a range of benefits that would lower costs. The initial contracts expired in 2010, at which point GSA awarded four more contracts.[Footnote 3] Contracts were awarded for one year with four one-year extension options. This program is different from many service contracts held by the government because GSA does not directly pay for broker services. Instead, brokers are compensated by commission--a percentage of the cost of the lease--paid by building owners. Thus, Congress does not appropriate funds for the work conducted by brokers.[Footnote 4] Members of Congress have raised questions about why GSA chose to use contractors to acquire leases for GSA, rather than performing this service itself, the extent to which the program has resulted in benefits, and whether the brokers are providing the best value for the government. Furthermore, there are concerns that these brokers--whose commission is a percentage of the lease price--may have an inherent conflict of interest in negotiating the best deal for the government. [Footnote 5] GSA intends to continue using private sector brokers and is currently developing the next generation of broker contracts with the intent to award the next round of contracts in fiscal year 2015. You asked us to review GSA's NBC program. This report examines (1) NBC program goals and the extent to which they are being realized, and (2) the changes GSA has made to the program and what challenges and suggestions for improvement, if any, key stakeholders identified. To meet these objectives, we analyzed GSA's NBC contract documentation and reviewed available data on leasing tasks assigned to brokers. These data were available beginning in 2005, the first year after the contract was signed, through 2013. We analyzed GSA's in-house and broker-negotiated rental rates, and discussed with GSA headquarters officials the methodology used and the limitations of their analysis and findings. In consultation with an economist and social science analyst at GAO, we determined that performing our own analysis would be limited by the same factors identified by GSA. We reviewed internal and external reports on the NBC program, which provided information about program outcomes. In addition, we obtained data from GSA on the NBC program including information on commissions and credits and on use of brokers. To determine the reliability of this data, we reviewed relevant documentation including internally published reports, and we interviewed agency officials about their processes for reviewing the data and ensuring their accuracy. We found the broker-related leasing data generally reliable for the purposes of this report. We interviewed GSA headquarters officials, officials from all 11 GSA regions concerning the NBC program, and individuals representing all four brokers currently part of NBC. We asked officials to identify challenges and suggestions that could help improve the program. Therefore, we could not always determine whether these challenges or suggestions were applicable in other regions or with other broker firms unless other officials brought them to our attention. We reviewed relevant legislation, regulations, prior GAO work, GSA guidance, and industry reports. We also identified GAO guidance on standards for internal controls--plans, methods, and procedures used to meet missions, goals, and objectives--that are key to helping agencies better achieve their missions and desired program results and assessed GSA's efforts to set goals and objectives to achieve the desired results of the NBC program.[Footnote 6] We conducted this performance audit from October 2012 to October 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. For more information on our scope and methodology, see appendix I. Background: GSA increased its use of brokers in the years prior to beginning the NBC. Before 1997, all leasing acquisition work was performed in-house. However, downsizing initiatives in the 1990s reduced GSA's in-house capacity to acquire leases, and in 1997 GSA began to sign contracts with private broker firms to assist with its leasing portfolio. From 1997 to 2003, GSA had multiple, separate regional contracts for broker services, and GSA paid brokers a fee from appropriated funds in exchange for a variety of lease acquisition and other services. By 2003, approximately 20 percent of the leasing work was being performed by brokers under the regional contracts, although the regional contracts were found to be administratively burdensome and inconsistent, because of variations in the contracts' terms, conditions, and pricing structures. In 2003, GSA conducted a business analysis comparing the advantages, disadvantages, and costs of different types of contracting options on a national, zonal and local level.[Footnote 7] Based on this analysis, GSA concluded that the national contracts through the NBC program represented the best option available, and GSA awarded four contracts in October 2004, and contract performance began on April 1, 2005.[Footnote 8] GSA identified three expected savings from the NBC program: reduced rental rates, reduced administrative expenses and fees, and reduced personnel expenses. GSA officials believed that rental rates would be reduced as a result of a broker's expert knowledge of the commercial real estate market, which would assist in negotiating lower rental rates. In addition, brokers agreed to forgo a portion of the commission they received--in the form of a commission credit reducing the government's initial rental cost. GSA officials also stated that the national contract would reduce its administrative expenses because the previous regional contracts had differing terms, conditions, and pricing structures. In 2007,[Footnote 9] we recommended, among other things, that GSA quantify savings associated with reduced fees and administration expenses using the NBC. In response to our recommendation, GSA reported that it had identified cost savings as a result of moving from regional contracts to the NBC. Finally, GSA anticipated achieving further savings by hiring fewer realty specialists--the position responsible for handling leasing transactions--over time. GSA stated that, as a result of government downsizing, the number of in-house realty specialist staff had decreased from 811 in 1995 to 450 in 2003. GSA estimated that approximately 140 realty specialists were eligible to retire by the end of 2005, and that the agency would need to hire approximately 300 additional staff to handle the leasing workload. In our 2007 report, we found that, after implementing the NBC program, officials subsequently determined that there were additional tasks for realty specialist staff and no longer viewed the NBC program as a way to avoid hiring additional staff. According to GSA officials, in October 2012 they employed approximately 476 realty specialists. The NBC program differed from the previous regional contracts in three major ways. First, the contracts are national contracts, which means that while the program is implemented and used at the regional level by GSA regional officials, all regions are subject to the same contract language and rules. Second, unlike the previous contracts, wherein brokers were paid for providing a variety of lease acquisition and other services to support GSA in acquiring leases, under the NBC program brokers are to perform the full lease acquisition, described in figure 1[Footnote 10], as well as lease expansion[Footnote 11] or extension and market data collection. Third, under the NBC program, brokers are compensated through commissions paid by the lessor (the entity leasing space to GSA), and no payments are made directly by the government as was the case with the regional contracts.[Footnote 12] Although commission payments are factored by lessors into rental rates, these contracts for the NBC program were referred to by GSA as "no cost" contracts.[Footnote 13] Figure 1: Description of Typical Steps for GSA's Real Property Lease Acquisition: [Refer to PDF for image: process illustration] Advertisement: GSA advertises requirements for space on the Federal Business Opportunities website at [hyperlink, http://www.fedbizopps.gov], or in the local newspaper where appropriate, in order to obtain the maximum competition from the private sector. Market survey: Based on agency requirements, GSA conducts a market survey with agency representatives to identify properties that meet the agency’s requirements. Request for lease proposal (RLP): GSA develops the RLP package, which is a standard document tailored to the requirements of each particular requirement, and sends it to all prospective offerors identified during the market survey. Negotiation and evaluation: Once offers are received and evaluated GSA begins negotiations. GSA internally establishes negotiation objectives (acceptable ranges for rental rates, cost for tenant improvements, and cost ranges for additional requirements) and conducts discussions with potential lessors in the competitive range. Final proposal revision: Once negotiations are completed, GSA requests final proposal revisions, where offerors are requested to submit their “best and final offer” to the government. Final evaluation and award: After submission of final revisions, GSA reviews and evaluates offers and makes an award determination. Contract execution: GSA compiles and sends an executable lease document with all negotiated terms and conditions to the apparent successful offeror for signature. Build-out and acceptance: The lessor completes the build-out of the space in accordance with the requirements of the lease, and GSA inspects and accepts the space when complete. Source: GAO. [End of figure] GSA's stated goals for the program are to provide consistent, high- quality service nationwide to federal agencies that rely on GSA for lease acquisition services, as well as leverage the expertise of private-sector brokers. GSA set two more specific goals that the agency uses to determine the effectiveness of the NBC program. First, GSA determined that brokers should be accountable for achieving rental rates in line with the overall GSA goal. For example, in 2012 the overall goal for the leasing portfolio was to have rental rates at 9.5 percent below market average. Secondly, GSA set yearly goals for using brokers to acquire leased space. For example in 2012, GSA set the goal of 55 percent of expiring leases to be handled by brokers and 45 percent handled by GSA in-house. Achievement of Goals Is Mixed, and Cost Savings from the NBC Program Are Unclear: Rental Rate Goals Met, but Unclear If Cost Savings Result from NBC Program: GSA officials report that they are meeting their overall rental rate goals for the entire lease portfolio. As mentioned previously, GSA sets a yearly goal that overall rental rates for space leased from the private sector fall a certain percentage below the average industry- market rental rate. These rental rate goals do not differentiate between broker-negotiated leases and those done in-house. In fiscal year 2012, the goal was for rental rates to be 9.5 percent below market; GSA reported that its overall portfolio, which includes leases negotiated by both brokers and GSA staff, was on average 11.45 percent below market.[Footnote 14] From 2006 to date, GSA has, to varying degrees, reported that it exceeded its targets for paying lower than average rental rates. See figure 2 below. GSA headquarters officials have used this data as an indication of the NBC program's success, because they say the data show that the use of brokers has not driven up the rental rates. The officials also noted that they consider this an indication that brokers are not driving up lease rates to obtain higher commissions. Figure 2: GSA Data on Cost of Leased Space Relative to Industry Market Rates, 2006 to 2012: [Refer to PDF for image: horizontal bar graph] Fiscal year: 2006; Actual percentage below market rate: 9.2%; Target percentage below market rate: 8.5%; Exceeded target by: 0.7%. Fiscal year: 2007; Actual percentage below market rate: 10.6%; Target percentage below market rate: 8.8%; Exceeded target by: 1.8%. Fiscal year: 2008; Actual percentage below market rate: 9.4%; Target percentage below market rate: 9.0%; Exceeded target by: 0.4%. Fiscal year: 2009; Actual percentage below market rate: 9.7%; Target percentage below market rate: 9.3%; Exceeded target by: 0.4%. Fiscal year: 2010; Actual percentage below market rate: 9.7%; Target percentage below market rate: 7.5%; Exceeded target by: 2.2%. Fiscal year: 2011; Actual percentage below market rate: 12.9%; Target percentage below market rate: 8.5%; Exceeded target by: 4.4%. Fiscal year: 2012; Actual percentage below market rate: 11.5%; Target percentage below market rate: 9.5%; Exceeded target by: 2.0%. Source: GAO presentation of GSA data. Note: The table above demonstrates the extent to which GSA exceeded its goals with respect to obtaining below market rates; thus, all percentages pertain to below market rates. [End of figure] GSA is reportedly meeting its overall rental goals, but it is not clear that the NBC program results in rental rates lower than those negotiated by in-house staff. In 2012, GSA officials attempted to determine if brokers negotiated lower rental rates than in-house staff using data from fiscal years 2011 and 2012. GSA's own analysis did not show that brokers were negotiating significantly lower rental rates, and in fact overall it showed that brokers were negotiating rental rates that were similar to in-house staff.[Footnote 15] Furthermore, GSA officials cautioned us that they found this available data insufficient to determine if brokers were negotiating lower rental rates than in-house staff. Officials stated that the market rate data they were using were not specific enough to provide the level of detail necessary to discern the potential difference between the rates. They said that the current market data do not take into account the unique circumstances for each individual lease, and how these circumstances affect the rental rate. For example, a lease for a law enforcement agency, with higher security requirements and unique types of space (e.g., a shooting range), may have to be negotiated in the higher end of the market rate. However, the market rate data GSA has does not consider these requirements, and therefore may underestimate the value of the rental rate negotiated. The converse is also true, and it is possible that the market rate data can overstate a rental rate negotiated for an office space with few special requirements. The officials said that this makes comparison of negotiated rental rates difficult because even if both of the example leases were in the same market, one rate may be overvalued while the other is undervalued based on its unique requirements. This limitation notwithstanding, officials told us they still believe it is probable that brokers are negotiating lower rates. In April 2013, GSA began requiring regional officials to use different market rental-rate data, which they expect will allow them to better compare the rental rates achieved by brokers to market rates in the future. For all broker projects greater than 10,000 square feet, regional officials are to obtain an in-house research report at the beginning of the leasing process referred to as "Bullseye."[Footnote 16] A Bullseye report is specifically tailored for each lease transaction and includes market information, analysis, and insight regarding the local submarket. The officials stated that the new reports would provide more specific data to gauge the effectiveness of the broker program by comparing negotiated rental rates to the average for the submarket rate at the time of lease award. According to GSA officials, this new reporting will improve their comparison of rental rates negotiated by brokers to market rental rates but will probably still not allow them to definitively determine whether brokers are negotiating better deals than in-house staff. Because this effort is relatively new, data were not available at the time of our review to conduct further analysis. GSA headquarters officials noted the only way to ideally compare one lease to another is to acquire space for similar agencies, in the same market at the same time. In previous attempts by GSA to perform comparison analysis, GSA determined there were not enough data points with these characteristics to draw a meaningful conclusion. GSA Recently Changed Goals for Use of Brokers: GSA initially set goals under NBC to have brokers conduct a high percentage of its leases, but beginning in fiscal year 2013 began deemphasizing the importance of annual percentage goals for broker usage. GSA set yearly goals based on the number of leases in the portfolio they determined could be handled by brokers and originally anticipated that 80 percent of the leasing workload would be assigned to brokers by fiscal year 2009. GSA lowered its broker usage goals for fiscal years 2011 and 2012 from 80 percent to 55 percent, and reported that in fiscal year 2012 it used brokers for 33 percent, or 220, of its expiring leases.[Footnote 17] GSA officials told us that they began lowering usage goals after 2010 because more leases needed to be handled in-house for training purposes. GSA officials told us they did not meet their broker usage goal for a variety of reasons. First, there are some leases with sensitivities such as internal conflict within an agency or the space is for a high- profile public official that they prefer to handle in-house. Second, some work is purposefully kept in-house for training purposes and to maintain staff expertise. (This was also a reason the officials gave for why they began lowering usage goals in 2011.) Third, some GSA regional officials have not always found it advantageous to send leases to brokers because it can be time consuming. Specifically, officials from 6 of the 11 regions expressed concern that the NBC program is administratively burdensome and adds time to leasing procurement. Officials in one region stated that they essentially run "shadow procurements" for all NBC leases. That is, even though the broker is doing a significant amount of paperwork and finding lessors, the work completed by brokers must be closely monitored and reviewed to ensure that the process is in accordance with government standards. [Footnote 18] In addition, GSA officials are required to conduct performance evaluations at multiple points throughout the process, which officials told us is more time consuming than the evaluation process for in-house personnel.[Footnote 19] In 2010, the GSA Office of the Inspector General (IG) reported [Footnote 20] that due to a number of steps unique to NBC projects, there was an average of 1.6 months added to the total time for the leasing process for the data in its sample. Representatives from one broker firm we spoke with told us that NBC leases can take longer because GSA personnel can be less responsive on a broker-negotiated lease than leases they are personally negotiating. They noted that there are many approval points that the broker cannot pass without approval from a GSA official and that significant time can elapse before GSA officials review the paperwork.[Footnote 21] In fiscal year 2013, GSA deemphasized the importance of annual percentage goals for broker usage. Instead, GSA headquarters officials told us they reviewed each regional portfolio for fiscal year 2013 through fiscal year 2015 procurements and developed a list of leases that they determined should be sent to brokers. The list consists of leases that are viewed by the officials as having a "high to moderate value" and having the greatest potential for brokers to negotiate lower rental rates than in-house personnel.[Footnote 22] GSA officials stated that they believe NBC provides the greatest value with large complex leases in large metropolitan areas, where brokers have a strong commission incentive. According to GSA headquarters officials, these areas have the highest potential for cost savings if brokers are able to negotiate lower rental rates as a result of private-sector market knowledge and negotiation skills. The officials told us that, after they compiled the list, regional officials were given an opportunity to comment, resulting in some changes, and that the resulting list of leases is what GSA plans to have brokers handle. For example, regional officials said that they removed some of the leases on the list because they had already been sent to a broker or begun in- house, or as stated previously, officials wanted to keep some larger projects in-house for training and maintaining in-house expertise. According to GSA headquarters officials, they plan to move forward with these new goals. They noted that the focus is now on managing the workload and not on meeting a specified utilization number. GSA headquarters officials told us that they believe they are now using the NBC program for its intended purpose, i.e., workload management and driving savings by using broker expertise in more difficult markets and working the more complex, high-to-moderate value leases. Cost Savings from Using the NBC Program Are Unclear: The two goals GSA uses to evaluate the NBC program are not closely linked to the anticipated cost savings used to justify the program. GSA headquarters officials told us that when they attempted to determine if brokers were negotiating lower rates than in-house staff, they found that the data they had were insufficient for this purpose. The program has evolved over the years, and reduced rental rates from using brokers are the only expected savings that remain. However, GSA lacks data on whether using brokers results in cost savings. Furthermore, GSA officials stated that they do not believe the Bullseye data will necessarily allow for comparison of broker-and in- house-negotiated rental rates. Without data on the cost benefits of using brokers relative to using in-house staff, the value of the NBC program will continue to be unclear. Federal internal control standards state that a key factor in helping agencies achieve their missions better and program results is the use of appropriate internal controls. Among other things, agencies need to compare actual performance to planned or expected results, monitor performance measures, and collect operational and financial data to ensure that agencies are meeting their goals for effective and efficient use of resources.[Footnote 23] This is particularly important because as programs change, as the NBC program has, and agencies strive to improve operational processes, agencies would be better equipped to assess results by continually assessing and evaluating whether the control activities being used are effective and updated when necessary. By not having adequate data, GSA cannot assess the effectiveness of activities conducted under the NBC program or know if adjustments are needed to the program. GSA Has Made Changes to the NBC Program, and Stakeholders Have Suggested Additional Actions That Can Improve the Program: GSA's Changes to the NBC Program Implementation: According to GSA officials, they have made changes to the NBC program since the first contracts were signed in 2004. Based on experiences with the first contracts (NBC1), GSA made changes to the current contracts (NBC2) in three key areas, as described below. Implementation of Commission Cap: According to GSA officials, NBC2 brokers were required to submit caps on the amount of commission they would retain for any given transaction.[Footnote 24] Under NBC1, brokers were required to submit the percentage of any commission they received that they would give back to GSA in the form of a rent credit. In our 2007 report we found that allowing brokers to represent the government while negotiating their commissions creates an inherent conflict of interest between the brokers' interest in promoting and negotiating higher commissions and their responsibility to effectively represent GSA's interest in selecting properties that best meet the government's needs, including its cost needs.[Footnote 25] Under NBC2, GSA believes this issue has been addressed, because the brokers have committed to a cap on the amount of commission the brokers retain. Using commissions paid by building owners to compensate brokers is typical in the commercial real estate industry. However, brokers could still have an incentive to seek government approval of a higher rent in order to increase the overall amount of their commission.[Footnote 26] Changes in Broker Performance Evaluations: According to GSA officials, GSA made two major changes to broker performance evaluations for NBC2. First, under NBC2, GSA reduced the number of mandatory broker performance evaluations from seven to four for each lease. In addition to receiving a final overall evaluation, brokers are now evaluated at the market-survey, lease-award, and post-award-services steps. GSA officials stated these changes were made to streamline the process and decrease the administrative burden on the in-house staff overseeing the brokers. Second, under NBC2, GSA awards tasks to brokers based on price and performance data.[Footnote 27] For NBC1 task orders were based on equitable distribution--i.e., brokers received equitable square-footage distributions in each of GSA's 11 regions. Under both contracts, brokers are evaluated and scored on a scale from 1 to 5 on document quality, timeliness, cost control, business relations, and personnel technical quality.[Footnote 28] These evaluations are used to derive a score for each task that is compiled into a report and sent to GSA headquarters officials on a quarterly basis. GSA officials use these scores to determine a national-level rating for each broker. For NBC2, GSA regional officials[Footnote 29] determine which factors (document quality, timeliness, cost control, business relations, and personnel technical quality) are the most important for a lease and then use the national-level performance rating, along with cost, to determine which broker will be awarded a task order.[Footnote 30] Earlier Broker Involvement: According to GSA officials, under NBC2 GSA included the option of bringing brokers into the process earlier, at the requirements development process. "Requirements development"[Footnote 31] is the process by which GSA works with an agency to obtain the specific requirements for a lease--such as the amount of space needed and the preferred location--at the beginning of the leasing process. Under NBC1, the contract allowed for the broker to assist with requirements development, but GSA policy was to have in- house staff assist the agency. GSA officials stated that brokers could add value by being involved in requirements development because they can bring in market data information and other expertise, which can better inform GSA on how much space might be needed and options for where it can be located. GSA officials told us that they are considering additional changes as they plan for the third generation of contracts. According to the officials, they are reconsidering the structure and administration of the NBC program with the goal of improving its effectiveness. At the time of our review, we were told GSA was assessing potential changes using a range of tools, including those below: * Collecting lessons learned, such as using brokers for new leases, which may provide better value, and working projects with short timelines in-house to allow for greater control over project timelines. * Incorporating new approaches and strategies, such as examining alternatives to the current commission and rent structure and developing new methods for evaluating broker performance. * Creating and gathering input via a dedicated members only discussion group for real estate stakeholders on the LinkedIn networking website. * Exploring the feasibility of conducting a pilot program prior to full contract launch. * Using the previously mentioned contractor review of the NBC program's effectiveness and changes the officials and industry stakeholders thought would improve the program. GSA plans to issue a request for proposals in 2014, and to award new contracts in fiscal year 2015. All Brokers and Several GSA Regions Report That Changes to Performance Evaluation Are Needed: Officials in 7 of the 11 GSA regions expressed dissatisfaction with the current broker performance-evaluation system. As mentioned previously, past performance is one of the factors regional officials use to select a broker. However, GSA regional officials noted some challenges associated with the process for evaluating broker performance. For example, GSA regional officials told us that under the current system, the same level and quality of work can receive very different ratings across regions. The officials noted that the ratings are largely at the discretion of regional officials and that the standards for scores are not necessarily applied uniformly. [Footnote 32] Regional officials suggested additional training or guidance for officials who do the ratings could help address these problems. GSA headquarters officials indicated that in addition to training, the changes to the internal structure used to support the contract that are being considered under NBC3 will help address these concerns. However, officials in four regions told us the broker performance-evaluation process worked well and allowed them to accurately represent each brokers' performance. GSA headquarters officials said they are looking at ways to make the performance evaluation process more consistent. They noted that in a performance- based contract, they are required to evaluate the contractors' performance and need regional support to ensure this practice is enforced. According to the headquarters officials, in some instances, milestones are evaluated months or up to a year after the completion date and that evaluations deemed "untimely" are discarded. Officials emphasized that in order to accurately reflect a contractor's performance, the evaluations must be completed once all tasks associated with that milestone are completed. Representatives from all four national broker firms participating in the NBC program expressed dissatisfaction or concern with the broker performance-evaluation system. Representatives from one broker firm told us they believed the evaluations are subjective because the standards for numerical performance evaluation scores are ambiguous. For example, the same work could receive a rating of 3 in one region and a rating of 5 in another region, presenting a challenge when attempting to determine the better value to the government. Brokers from another firm said there needs to be greater consistency between regions on how they conduct broker evaluations. All four brokers also suggested that providing more detailed guidance for GSA officials who do the rating would make the ratings less subjective and more consistent across regions. GSA regional officials and brokers also suggested using region- specific broker performance-evaluation data to award tasks. Officials from 10 of 11 GSA regions told us that this change would help them award the task order to the broker that performed the best in the region. As officials in one region explained, the brokers' national scores--which combine the scores from all regions--are not helpful, because these scores do not always reflect the performance of a specific broker that works well in a region. The officials in this region told us that, with 11 regions, the combined national score loses effectiveness, because brokers have different strengths and weaknesses in various markets. For illustrative purposes, in figure 3, we present actual data from a 2012 quarterly performance report to show how using national scores may result in assigning a task to a broker that does not perform as well in a specific region as the broker firm's overall national performance score.[Footnote 33] In this region, officials conducted two evaluations and rated this broker firm as "marginal" in every category, while the overall national scores for the same broker firm are primarily "very good" and "exceptional," except for cost control. Figure 3: Comparison of One Broker Firm's 2012 Single Regional and Overall National Broker Performance Evaluation Scores: [Refer to PDF for image: illustrated table] Regional rating: Document quality: Marginal; Timeliness: Marginal; Cost control: Marginal; Business relations: Marginal; Personnel technical quality: Marginal. National rating: Document quality: Very good; Timeliness: Exceptional; Cost control: Marginal; Business relations: Very good; Personnel technical quality: Very good. Key: Exceptional; Very good; Satisfactory; Marginal; Unsatisfactory. Source: GAO. [End of figure] In the example shown in figure 3, if officials in this region identified "Timeliness" as the most important factor when awarding a task order, under the current contract, they would have to use the national scores to determine the broker with the best performance in timeliness. Thus, the task order could be awarded to this broker whose past performance was rated as marginal by regional officials. According to GSA headquarters officials, they are considering using regional past performance for task order selection. They noted that they would make this decision after they are confident that GSA staff are adequately trained and have a clear understanding of the contract requirements. Additionally, they said that they place a priority on making sure the contract is administered fairly and properly before they begin using regional past performance. Of the 10 regions where officials suggested using region-specific broker performance evaluation data to award tasks, officials in 3 regions suggested developing a hybrid approach for performance evaluation data, wherein they could have the option of combining national scores with region-specific scores and using the combined score to choose a broker. Officials in one region told us this would allow for assignments to be made based on work completed in that region which would help the officials make better decisions. Similarly, officials in another region told us they would prefer to use a combination of both the regional and national level scores because each region has different needs, and brokers have different strengths and weaknesses. GSA headquarters officials told us they are considering using regional performance evaluation scores, but want to determine if the performance evaluation processes are working properly before they make a change. These officials also stated that there is concern that if regions with less attractive portfolios (fewer and less valuable leases) are not factored into a national score used to determine assignments, brokers may have less incentive to perform well in smaller markets. Representatives from two broker firms suggested using region-specific broker-performance-evaluation data to award tasks. For example, one broker told us that since work is being distributed using the national average performance-evaluation scores, this overlooks the benefit that some regions may have developed strong working relationships not captured under a national score. They also noted that the way work is assigned does not seem to take into account "logical follow-on" or unique experience.[Footnote 34] All Brokers and Some GSA Regions Report Challenges with the Commission Structure: GSA officials in three regions expressed concerns about the current commission structure and made suggestions for improvement. These regional officials were generally concerned that the commission-based system did not guarantee that brokers had an incentive to negotiate a lower rental rate for the government. Officials in one region told us that under the current system, brokers benefit from higher rental rates because their commission is a percentage of the total rental rate. They suggested that if commission rates were a specified amount instead of a variable percentage, GSA would be better assured that the broker was not trying to get a larger commission than is appropriate in that market. GSA headquarters officials told us that they are reexamining the commissions and are actively looking at alternatives to the current commission and rent structure. All four brokers participating in the NBC program told us that the change to the commission structure GSA implemented in 2011 has decreased commissions and potentially could reduce the quality and quantity of brokers' staff. In 2011, GSA modified the contracts to incorporate changes to the Federal Acquisition Regulation (FAR)[Footnote 35] as a result of the Duncan Hunter National Defense Authorization Act for Fiscal Year 2009 (Duncan Hunter Act).[Footnote 36] As explained by GSA, these changes require that every task order or group of task orders estimated to yield a commission in excess of $150,000 be sent to all brokers, who then have an opportunity to be awarded the task order. Brokers were unanimous in their concern that this requirement, which they call "re-bidding" or "re-competing," would ultimately result in lower value for the government. They told us that their commissions or initial contract pricing were based on the assumption that the profitability of the larger leases (including those with commissions in excess of $150,000) would help offset the potential losses on smaller leases, which they say are generally not profitable to the brokers. The brokers also told us that although this may result in short-term savings for the government by encouraging broker bidding and reducing commissions, ultimately it is not financially sustainable for them. Two brokers provided examples in which decreased revenue negatively affected staffing. For example, one broker told us these requirements were forcing commissions lower and that they would eventually have to compensate by laying-off staff or hiring staff with less experience; and another broker told us that decreased revenues resulted in a loss of employees and difficulty replacing them. Representatives from 3 of the 4 broker firms stated the changes to the FAR did not require contract modifications. These brokers stated that in their opinion, the changes to the FAR do not apply to the NBC program and recommended that changes be made that reflect this.[Footnote 37] Most GSA Regions Suggested More Flexibility in Using Brokers: Officials in the majority of regional offices (8 of the 11) suggested that greater flexibility in applying the NBC program would allow them to make better use of the brokers. Currently, when a broker is awarded a task order, as we described previously, it is a full service lease acquisition, which means the broker handles the majority of the work for a task. Regional officials described a preferred "menu of services" that would allow them to use the brokers more selectively for the specific tasks where GSA wants assistance as opposed to working with the broker throughout the entire lease procurement. For example, regional officials told us this would allow officials to involve brokers at crucial points--such as by conducting market analysis--and then allow GSA to handle the other parts of the procurement process in-house. The officials said that this would benefit GSA, because it would allow GSA to harness the brokers' market expertise, which, in their opinion, is how brokers add the most value. According to GSA headquarters officials, they are reviewing the contract on an on-going basis and making the changes that will offer greater flexibility. They said that they have enhanced the requirements development, allowing the broker to perform most of the duties associated with requirements development process and are removing unnecessary requirements that delay task-order issuance. Brokers Suggested Earlier Involvement and Increased Access: The current NBC program allows for brokers to have only a limited role in working with the agency requesting space during the earliest part of a leasing procurement--the requirements development phase --but according to both GSA regional officials and brokers early broker involvement seldom occurs.[Footnote 38] In some cases, it does not occur because a region has a group dedicated to the requirements development phase, so the region uses in-house expertise. However, all four brokers suggested that earlier involvement in the procurement process would improve the program's effectiveness by allowing the brokers to assist in determining space needs or identifying other means of cost savings. Brokers from one firm noted that they have considerable experience in reducing costs for private sector clients by helping them plan and implement space-saving arrangements, and that they could provide added benefit to GSA if they were involved earlier in the process. Brokers from another firm said that GSA and the agency requesting space will sometimes develop requirements that will result in higher costs, but that changes could result in significant savings. They said that they would like to start with a blank slate and help determine how to save money for multiple sets of circumstances, including broader space planning for a single agency, planning across agencies, multiple leases in the same area, or negotiating multiple leases at the same time in the same building. In addition to earlier involvement in the leasing process, all four brokers participating in the NBC program suggested that allowing them increased access to agencies requesting space would expedite the process. Brokers indicated that the procurement process generally slows down when they have limited direct interaction with tenant agencies because the extra layer of communication can cause miscommunications. This situation can result in the broker's trying to secure a lease with bad information, which can delay the process until the broker determines exactly what the agency wanted. Two brokers said that if they had more access to the agencies, the process would move more quickly and they would have better information on which to make decisions. Conclusion: GSA's goals and metrics for evaluating the NBC program have not been linked to the cost savings in rental rates GSA anticipated when proposing the program. As a result, GSA does not have a means of evaluating and reporting on this aspect of the program and the value of the NBC program in terms of cost savings continue to be unclear. While GSA has taken steps to better assess how its rental rates compare to market rates, these changes will not necessarily be sufficient for determining the overall costs and benefits of the NBC program. Accordingly, GSA will continue to lack the data needed to assess whether the use of brokers' results in the expected cost savings through overall rental rate reduction. Clarifying its goals for the program and linking them to cost savings would also serve as a way to be transparent to the Congress and other stakeholders about the purpose of the program and how GSA plans to monitor and achieve the program's goals. Such transparency is especially important given that GSA is planning to seek a third generation of broker contracts in fiscal year 2015. Recommendation for Executive Action: To promote transparency and fully reflect the expectations GSA used to justify the NBC program, we recommend that the Administrator of GSA ensure the program's goals are linked to cost savings achieved through the use of NBC brokers and develop and implement a means of evaluating and reporting results. Agency Comments: We provided a draft of this report for review and comment to GSA. GSA concurred with the report's recommendation and provided technical clarifications, which we incorporated as appropriate. GSA's comments are discussed in more detail below. GSA's letter is reprinted in appendix II.[Footnote 39] GSA stated that it will take action to implement the recommendation as well as address the challenges and suggested improvements noted in the report. GSA stated, with respect to the use of "shadow procurements" reported, that this practice was an extraordinary and unnecessary scrutiny of broker work that was limited to one GSA region and is not reflective of the NBC program in its totality. This point raised by GSA is reflected in the report, and the report does not imply that this was indicative of the program as a whole. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Director of the Office of Management and Budget and the Administrator of General Services. In addition, the report will be available at no charge on GAO's website at hyperlink, http://www.gao.gov]. If you or your staff have any questions, please contact me at (202) 512-2834 or wised@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III. Signed by: David J. Wise: Director, Physical Infrastructure Issues: [End of section] Appendix I: Objectives, Scope, and Methodology: Our objectives were to examine (1) the General Services Administration's (GSA) National Broker Contract (NBC) program goals and the extent to which they are being realized; and (2) the changes GSA has made to the program and what challenges and suggestions for improvement, if any, key stakeholders identified. To identify the goals of the NBC program, we reviewed GSA's 2003 analysis used to recommend the program as well as the subsequent goals and policies used to implement the program. From these sources and interviewing GSA officials, we identified expected savings and goals for the NBC program. The expected savings of the program include reducing rental rates for GSA leases of private sector space and lower administrative expenses as compared to the previous, mainly regionally based, contracts with private brokers. The goals that we identified for the project are for brokers to negotiate rental rates in line with overall GSA lease portfolio goals, and a yearly goal for using the broker program. We also conducted interviews with GSA headquarters officials managing the NBC program concerning the expected savings and goals of the program. To examine the extent to which benefits and goals have been realized we reviewed available GSA data and data analysis, GSA internal reports, GSA Office of Inspector General reports, and prior GAO work. We reviewed GSA broker data from the first year of the contract in 2005 through 2013. We analyzed GSA's in- house and broker-negotiated rental rates, and discussed with GSA headquarters officials the methodology used and the limitations of their analysis and findings. In consultation with an economist and social science analyst at GAO, we determined that performing our own analysis would be limited by the same factors identified by GSA. We reviewed internal and external reports on the NBC program, which provided information about program outcomes. In addition, we obtained data from GSA on the NBC program including information on commissions and credits and on use of brokers. To determine the reliability of this data we reviewed relevant documentation including internally published reports, and we interviewed agency officials about their processes for reviewing the data and ensuring their accuracy. We found the broker-related leasing data generally reliable for the purposes of this report. We interviewed officials from all 11 GSA regions and representatives from all four broker firms currently on contract about the use and perceived benefits of the program. We asked officials to identify challenges and suggestions that could help improve the program. Therefore, we could not always determine whether these challenges or suggestions were applicable in other regions or with other broker firms unless other officials brought them to our attention. We also identified federal guidance on standards for internal controls--plans, methods, and procedures used to meet missions, goals, and objectives--that are key to helping agencies better achieve their missions and desired program results.[Footnote 40] To identify stakeholder suggestions for improving the NBC program we conducted interviews with the major stakeholders of the program including: GSA headquarters officials, all 11 GSA regional office officials, and representatives from all four broker firms. Each of the stakeholders was asked to speak about the changes between the first and second broker contract, and whether they felt that benefits had resulted because of these changes, and what further improvements, if any, they would suggest for the program. We reviewed modifications GSA made to the contract in response to the Duncan Hunter National Defense Authorization Act for Fiscal Year 2009 [Footnote 41] and to the Federal Acquisition Regulation. [Footnote 42] We compared the first and second signed national broker-contract documentation, and analyzed the reasons for the changes between the two contracts. We reviewed the project plan for reviewing the NBC program and spoke with officials concerning their plans for making changes for the next generation of contracts. We also identified federal guidance on standards for internal controls--plans, methods, and procedures used to meet missions, goals, and objectives--that are key to helping agencies better achieve their missions and desired program results and assessed GSA's efforts to set goals and objectives to achieve the desired results of the NBC program.[Footnote 43] We conducted this performance audit from September 2012 to October 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix II: Comments from General Services Administration: Note: Page numbers in the draft report reviewed by GSA may differ and have moved in the publication process. GSA: U.S. General Services Administration: The Administrator: 1800 F Street, NW: Washington, DC 20405: Telephone: (202) 501-0800: Fax: (202) 219-1243: September 30,2013: The Honorable Gene L. Dodaro: Comptroller General of the United States: U.S. Government Accountability Office: Washington, DC 20548: Dear Mr. Dodaro: The U.S. General Services Administration (GSA) appreciates the opportunity to review and comment on the draft report entitled "Federal Real Property: GSA Should Clarify Savings Goals for the National Broker Contract Program," (GAO-14-14). The Government Accountability Office (GAO) recommends that the Administrator of GSA ensure the National Broker Contract (NBC) program goals are linked to cost savings achieved through the use of NBC brokers, and develop and implement a means of evaluating and reporting results. GSA agrees with GAO's recommendation and will take action to implement the recommendation as well as address the challenges and suggested improvements noted in the report. However, GSA would like to caution that the discussion of regional officials conducting "shadow procurements" (page 9) is isolated to one GSA region and is not reflective of the NBC program in its totality. The term "shadow procurement" is typically used to describe work that is being duplicated. In this instance, what the region described is an extraordinary and unnecessary scrutiny of broker work. The level of oversight practiced is not only contrary to National Office guidance but also contradicts the intended purpose of the NBC2 contract. The stringent training requirements outlined in Section H.7 of the NBC2 contract, as well as the consistently satisfactory performance evaluations of the broker firms, clearly demonstrate a "shadow procurement" is unnecessary. As GAO acknowledges, GSA has made changes to improve the NBC program. GSA believes that changes to the in-house staffing structure GSA uses to support the NBC will allow for a more accurate reflection of program costs and associated savings. As noted by the brokerage firms in the report, GSA's current internal structure can cause unnecessary delays and confusion (page 20). Having a dedicated staff of lease contracting officers and contracting officer technical representatives who work only on broker tasked leases will eliminate most of the issues described, and allow for more accurate data collection and comparison of staffing efficiencies. GSA is taking steps to include this streamlined staffing structure into the NBC3 contract. Technical comments that update and clarify statements in the draft report are enclosed. If you have any additional questions or concerns, please do not hesitate to contact me at (202) 501-0800, or Ms. Dorothy Robyn, Commissioner, Public Buildings Service, at (202) 501-1100. Staff inquiries may be directed to Mr. Chris Wisner, Assistant Commissioner, Office of Leasing, at (202) 208-4947. Sincerely, Signed by: Dan Tangherlini: Administrator: Enclosure: cc: David J. Wise, Director, Physical Infrastructure Issues, GAO: [End of section] Appendix III: GAO Contact and Staff Acknowledgments: GAO Contact: David Wise, (202) 512-2834 or wised@gao.gov: Staff Acknowledgments: In addition to the contact named above, the following individuals made important contributions to this report: David Sausville (Assistant Director), Kenneth Bombara, Lorraine Ettaro, Bert Japikse, Aaron Kaminsky, Josh Ormond, Amy Rosewarne, and Jade Winfree. [End of section] Footnotes: [1] Some agencies have independent statutory authority, which authorizes them to acquire leased space. [2] This percentage was calculated by GSA based on the average of leases expiring in fiscal years 2013, 2014, 2015, and 2016 minus non- commissionable leases, which include leases for things such as border stations, renewals and parking. [3] GSA awarded contracts to CB Richard Ellis (CBRE); Jones Lang LaSalle, Inc. (JLL); Studley Inc.; and UGL Equis Corporation. [4] Building owners typically factor the cost of the commission into the rent charged to a tenant. Tenants indirectly pay the commission through their rental payments. [5] While this conflict is inherent to this type of commission based payment for services, it is typical in the commercial real estate industry. [6] GAO, Internal Control: Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999). [7] GSA, Business Analysis and Case for National Contracts for Brokerage Services in GSA (Aug. 25, 2003). [8] The starting date was delayed by bid protests. [9] GAO, GSA Leasing: Initial Implementation of the National Broker Services Contracts Demonstrates Need for Improvements, [hyperlink, http://www.gao.gov/products/GAO-07-17] (Washington, D.C.: Jan. 31, 2007). [10] GSA headquarters officials noted that requirements development is also considered part of the leasing process. [11] Use of brokers for expansions was included in NBC1, but removed from NBC2. [12] Compensating brokers in this way is typical in the commercial real estate industry as we reported in our initial review of the NBC program, [hyperlink, http://www.gao.gov/products/GAO-07-17]. [13] The term "no cost" is somewhat of a misnomer, since there would be no enforceable contract without a tender of consideration by each party. In this case, the consideration benefiting the broker stems from its right to collect a share of the lease brokerage fees (commission payments) paid by prospective building owners. [14] GSA's metric, used to compare the cost of leased space relative to industry market rates, is referred to as "LCRM--Lease Cost Relative to Market." This measure compares GSA leasing rates to third-party data on rates paid by industry. [15] GSA officials told us that commission credits were factored into the rental rate prior to the comparison with in-house negotiated rates. This means any commission credit given by a broker to GSA was taken into consideration and lowered the rental rate before it was compared to the in-house negotiated rental rates. [16] These reports are already used for all leasing projects run by in- house staff. GSA regional officials are strongly encouraged to obtain a "Bullseye" report for projects above 2,000 square feet. [17] NBC1 goals were based on the average number of leases expiring annually over the next two fiscal years, minus non-commissionable leases. NBC2 goals were based on the average number of leases expiring annually over the next four fiscal years, minus non-commissionable leases. Fiscal year 2012 goals are based on the average of leases expiring in fiscal year 2013 through fiscal year 2016 minus non- commissionable leases for Transportation Security Administration or other leases located on-site at airports, Indian Health Services, Bureau of Indian Affairs, United States Postal Service, Border Stations, Renewals and Parking. [18] GSA headquarters officials told us that running a "shadow procurement" is a choice made by regional officials. These officials stated that this level of review is unnecessary and that the training requirements outlined in Section H.7 of the NBC2 contract, as well as the performance evaluation results of the firms clearly demonstrate a "shadow procurement" is unnecessary. [19] According to GSA headquarters officials, as with any performance- based contract, regional officials are required to evaluate contractors' performance. They noted that if the contractor's work is not correct upon initial submission, if properly documented, officials have the ability to request the removal of personnel. [20] Office of Inspector General, General Services Administration, Audit of the PBS National Broker Contract, Report Number: A0700228/P/W/R11004 (Dec. 22, 2010). This report was originally issued as for Official Use Only, but was subsequently released to the public on July 3, 2012, pursuant to a Freedom of Information Act request. [21] GSA headquarters officials indicated they believe a new online system they are implementing will add transparency to the review process, and will allow headquarters officials to identify bottlenecks. According to these officials, anticipated changes to the internal staffing structure used to support the broker contract in NBC3 will also help to address this issue by increasing transparency. [22] GSA does not have a standard dollar amount to define "high-to- moderate value" and instead bases this classification on the leases type, location, and size. GSA defines as "high value" those new leases that are in the top 100 metropolitan statistical areas (MSA) and are in excess of 2,000 square feet. GSA defines as moderate those (1) new leases in the top 100 MSAs and less than 2,000 square feet, (2) new leases beyond the top 100 MSA in excess of 10,000 square feet, and (3) those succeeding leases in the top 100 MSA in excess of 5,000 square feet. [23] GAO, Internal Control: Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C. November 1999). [24] Commission caps vary based on the type of lease and the size of the procurement, and brokers submitted different rates for full lease acquisition, extensions, competitive, and non-competitive. GSA requires brokers to negotiate a "market rate commission," and tenant agencies are to receive a credit to the rent for any difference between the market commission and the cap. [25] [hyperlink, http://www.gao.gov/products/GAO-07-17]. [26] GSA headquarters officials stated that negotiation objectives are agreed to between the Lease Contracting Officer and the broker prior to initiating negotiations with the landlord. The performance evaluation of the brokers includes a "cost control" on every transaction. As all tasking is based on performance evaluations, if brokers perform badly on this measure, they are at risk for not receiving future task orders. [27] The National Broker Services Contract signed in 2010 state that: for task orders or groups of task orders estimated to yield a net commission to the contractor in excess of $3,000 but less than or equal to $5,000,000, all contractors will be provided a fair opportunity to be considered based on price or price and one or more, or all of the following: (1) price, based on the contractor's pricing structure; (2) past performance; (3) workload capacity; and (4) workload distribution. [28] According to GSA headquarters officials, the Contracting Officer's Technical Representative (COTR) is responsible for evaluating the brokers for each task they complete and giving them a numerical score ranging from 1 to 5. A score of 1 indicates "unsatisfactory" performance and a score of 5 represents "exceptional" performance. The contract defines the minimum level of acceptable performance, and performance exceeding the definitions is to be evaluated "appropriately." For example, for business relations the broker is evaluated on how they represent GSA at all times when dealing with all parties and complies with GSA standards to achieve an acceptable solution for all parties. [29] The Contracting Officer's Representative (COR) or the COTR assigned to work the project selects the criteria GSA deems to be important given the nature of the procurement, and forward this criteria to the Regional Contracting Officer to be used as a basis for award. [30] Regional task order evaluations that are completed in a given quarter are reviewed by the broker prior to being finalized. These evaluations are then consolidated at the National level to be used for the following quarter's task order selections. [31] During the requirements development phase, the tenant agency determines, among other things, the necessary square footage, circulation requirements, parking requirements, and the number of personnel to be housed. [32] GSA headquarters officials noted that both the brokers and the regional officials have two opportunities in the process to review and provide comments on each evaluation. Additionally, the brokers have opportunity to appeal their rating. [33] GSA headquarters officials stated that GSA considers this a positive reason for using the national scores. They noted that the region with a marginal rating had only two milestone evaluations for that broker and that this is not enough volume to make a reasonable determination of broker performance. The officials also noted that if regions award all work to the best regional performer, other firms would never be awarded tasks and therefore never have an opportunity to improve performance in that region. [34] GSA headquarters officials stated that Section C.4.3. (1) of the contract does allow logical follow-on work. The COTR or COR can provide a justification to the RCO if they desire to task the follow- on project to the broker. [35] 48 C.F.R. § 16.505. [36] Pub. L. No. 110-417, § 863, 122 Stat. 4356, 4547-4548 (2008). [37] It is too early to determine whether GSA's application of this provision affects GSA's ability to use the NBC program, or will affect the satisfaction with the work being performed under this program. [38] GSA modified the NBC2 contract in July 2013 to significantly expand the brokers' role in requirements development. [39] GSA's letter also included an enclosure that contained the agency's technical comments. The enclosure is not reprinted in appendix II. [40] GAO, Internal Control: Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999). [41] Pub. L. No. 110-417, § 863, 122 Stat. 4356, 4547-4548 (2008). [42] 48 C.F.R. § 16.505. [43] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [End of section] GAO’s Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select “E-mail Updates.” Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]