Embassy Construction: State Needs to Better Measure Performance of Its New Approach
Fast Facts
Will State's new approach produce better embassies, and how will we know?
In 2011, the State Department began to shift from using a standard design for the construction of its embassies to customizing each embassy's design. This new approach (called Excellence) was meant to improve the appearance and functionality, as well as reduce the operating costs of these embassies.
However, we found that State does not have performance measures or sufficient data to evaluate these and other goals of Excellence. We recommended that State strengthen its performance measures, as well as its monitoring and reporting tools, for the Excellence approach.
U.S. Embassy in Panama Constructed under Standard Embassy Design and Rendering of U.S. Consulate General in India to Be Delivered under the Excellence Approach
Highlights
What GAO Found
In 2011, the U.S. Department of State's (State) Bureau of Overseas Buildings Operations (OBO) established the Excellence approach in response to concerns regarding the aesthetics, quality, location, and functionality of embassies built using its Standard Embassy Design (SED). The SED utilized a standard prototypical design for new embassies and consulates along with a streamlined delivery method combining responsibility for design and construction under a single contract. Under the Excellence approach, OBO now directly contracts with design firms to develop customized embassy designs before contracting for construction. OBO officials believe that greater design control under Excellence will improve embassies' appearance in representing the United States, functionality, quality, and operating costs.
Excellence consists of several key elements and involves trade-offs. For example, OBO now allots time and funding to develop customized designs and hires leading design firms to produce them. These design firms have faced initial adjustment challenges designing U.S. embassies, and OBO only recently began evaluating their performance as required by federal regulation. OBO's new approach poses cost and schedule trade-offs since, for example, OBO now has greater design control but may also be responsible if design problems are identified during construction. GAO's survey found that OBO staff who responded held split or conflicting opinions on Excellence compared with SED.
U.S. Embassy in Panama Constructed under Standard Embassy Design and Rendering of U.S. Consulate General in India to Be Delivered under the Excellence Approach
While OBO has established guidance to implement Excellence, it lacks tools to fully evaluate the performance of this new approach. Performance measures are essential tools for managers to evaluate progress toward a program's goals, as noted in Standards for Internal Control in the Federal Government. However, OBO has not established performance measures to specifically evaluate and communicate the effectiveness of Excellence in delivering embassies. Moreover, OBO's bureau-wide strategic measures do not address Excellence priorities, such as greater adaptability to individual locations, functionality, or sustainability. OBO also lacks a reliable system to monitor operating performance, such as building energy usage, and a centralized database to broadly manage the Excellence program, to include effectively reporting on projects' design and construction costs and schedules. Without performance measures and reliable systems to collect and analyze relevant data, OBO cannot fully assess the value of shifting to the Excellence approach and away from the SED.
Why GAO Did This Study
In 1998, terrorists bombed the U.S. embassies in Nairobi, Kenya, and Dar es Salaam, Tanzania, killing over 220 people and injuring 4,000. In 1999, State began a new embassy construction program, administered by OBO, which to date has received $21 billion, according to State. OBO's primary goal was to provide secure, safe, and functional workplaces, and it adopted SED with a streamlined, standard design for all embassies. In 2011, OBO replaced the SED with the Excellence approach, which makes use of customized designs for each embassy.
GAO was asked to review the implementation of Excellence. This report examines (1) the reasons for State's shift to the Excellence approach, (2) key elements and tradeoffs of the new approach, and (3) the extent to which State has established guidance and tools to implement and evaluate its Excellence approach. GAO analyzed information from State policy, planning, funding, and reporting documents and interviewed State and industry officials. GAO also surveyed OBO staff about, among other things, the sufficiency of OBO's policies, procedures, and technical guidance for the Excellence approach. GAO will examine project cost and schedule issues in a subsequent report.
Recommendations
GAO is making four recommendations that State take several steps to strengthen performance measures and reporting, monitoring mechanisms, and data systems for the Excellence approach. State concurred with all four recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of State | To better assess OBO's performance, the Secretary of State should determine whether the existing OBO program performance measure and annual target of moving 1,500 people into safe, secure, and functional facilities is still appropriate or needs to be revised. |
State concurred with our recommendation, and in subsequent review of the metric determined the target of 1,500 employees per fiscal year was too low. According to State officials, OBO determined that a new target of relocating 3,000 employees per fiscal year into secure, safe, and functional facilities was more appropriate for achieving OBO's overall mission goals. As a result, State increased the target to 3,000 staff per year as part of its Joint Strategic Plan with USAID for fiscal years 2018-2022.
|
| Department of State | To better assess OBO's performance, the Secretary of State should establish additional performance measures applicable to the new goals of the Excellence approach in support of the Capital Security Construction Program. |
In February 2017, State concurred with GAO's recommendation and committed to addressing it through an update to OBO's Bureau Strategic Plan. Before this could happen, the Excellence initiative was overtaken by the Embassy After Next initiative-a series of in-house process improvements encompassing Standardization, Life-Cycle Asset Management, Building Information Modeling, Digital Design Review, and Project Performance Management. OBO has included new performance measures and milestones directly mapped to the Embassy After Next initiative's priorities in its FY2018-2022 Functional Bureau Strategy. OBO has also developed reference sheets that describe the process and algorithms used to calculate the performance measures, as well as implementation guides outlining how various milestones are to be achieved.
|
| Department of State | To better assess OBO's performance, the Secretary of State should finalize the mechanisms OBO will use to better track and evaluate the actual operations and maintenance performance of its buildings--whether Excellence or SED--and document through appropriate policies, procedures, or guidance. |
State concurred with GAO's recommendation and in response, initiated a life cycle asset management program within OBO. As part of this program, in September 2019, OBO finalized a methodology for conducting life cycle cost analyses during the planning, design, and construction phases of new embassy facilities, and for existing facilities. Additionally, in 2019 OBO began conducting Facility Performance Evaluations (FPE)-with the general intent that an FPE will be performed for every post every 5 years. These evaluations directly inform 5 high-level Key Performance Indicators (KPIs): 1) Building Condition Index - a performance-based index based on the current replacement value of the facility and the normalized sum of the condition ratings for each building system (e.g., heating/ventilation/air-condition; electrical; plumbing, etc.) and structure within the facility. It represents the physical condition of a facility along with its components as compared to its replacement value. 2) Facility Condition Needs Index - a monetary-based condition metric allowing for relative comparisons among different buildings. A measurement lower than 40 on a 0 to 100 scale indicates that a facility's renewal and maintenance costs are approaching current replacement value of the facility, and that facility replacement should be considered. 3) Normalized Maintenance Cost - the maintenance cost for an embassy on a square meter basis. 4) Energy Use Intensity - kilowatt hours, per square meter, per year. 5) Facility Management Operations and Maintenance - a staffing assessment that can include such metrics as Non-Residential Maintenance Staff per 1000 square meters, ratio of Non-Residential Maintenance Staff to FTE positions at post, and the ratio of in-house performed maintenance costs to outsourced (contracted) expenditures. According to OBO officials, these performance indicators are aligned with industry standards and will allow for continuous monitoring of life-cycle cost performance across OBO's facility portfolio. They also stated that these indicators will directly inform management decisions on whether to replace facilities near the end of their life cycle (i.e., construct a new embassy), or to recapitalize through modernization and rehabilitation. According to OBO officials, seven older facilities have undergone FPE pilots, and OBO plans to conduct eight more on newer facilities such as London, Jakarta, and The Hague in fiscal year 2020.
|
| Department of State | To better assess OBO's performance, the Secretary of State should finalize the mechanisms OBO will use to centrally manage project management data (to include project cost and schedule information), currently termed the Ideal Operational State, and document through appropriate policies, procedures, or guidance. |
State concurred with GAO's recommendation, and in response, initiated a series of data system improvements. Through its Building Information Management (BIM) initiative, OBO has developed internal data standards aligned with International Organization for Standardization information management criteria. OBO also updated its strategic plan to track the percentage of new data requirements and standards met by contractors. Additionally, OBO initiated the development of an Integrated Master Schedule to serve as a single source for project schedule information for new capital projects from inception to occupancy. It is intended to reduce inefficiencies and errors resulting from data residing in multiple offices by linking and consolidating workflows as a project progresses, and all new major capital projects began using it in December 2019. In November 2020, OBO reported that data improvements stemming from its BIM and IMS efforts had enabled the automation of Program Performance Reviews. These reviews are monthly status reports to senior leadership on the scope, schedule, and budget of all ongoing major projects. According to OBO, its technical staff mapped new standardized data and system sources, including the newly-developed Integrated Master Schedule, to store in an enterprise data warehouse that can automatically generate a monthly report, as well as provide user access to real-time project information. By creating an effective, centralized data system capturing essential and reliable project management data including cost and schedule performance, OBO management now has a critical tool for consolidating key project data, assessing performance, guiding strategic oversight and internal control, as well as responding to oversight inquiries.
|