The Boeing Company
Highlights
The Boeing Company protests the award of a contract to Northrop Grumman Systems Corporation under request for proposals (RFP) No. FA8625-07-R-6470, issued by the Department of the Air Force, for aerial refueling tankers. Boeing challenges the Air Force's technical and cost evaluations, conduct of discussions, and source selection decision.
B-311344; B-311344.3; B-311344.4; B-311344.6; B-311344.7; B-311344.8; B-311344.10; B-311344.11, The Boeing Company, June 18, 2008
DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.
Decision
File: B-311344; B-311344.3; B-311344.4; B-311344.6; B-311344.7; B-311344.8; B-311344.10; B-311344.11
Rand L. Allen, Esq., Paul F. Khoury, Esq., Scott M. McCaleb, Esq., Martin P. Willard, Esq., Nicole J. Owren-Wiest, Esq., Kara M. Sacilotto, Esq., Nicole P. Wishart, Esq., Jon W. Burd, Esq., Stephen J. Obermeier, Esq., and Heidi L. Bourgeois, Esq., Wiley Rein LLP; and Charles J. Cooper, Esq., Michael W. Kirk, Esq., and Howard C. Neilson, Esq., Cooper & Kirk; Lynda Guild Simpson, Esq., and Stephen J. Curran, Esq., The Boeing Company, for the protester.
Neil H. O'Donnell, Esq., Allan J. Joseph, Esq., David F. Innis, Esq., Thomas D. Blanford, Esq., Aaron P. Silberman, Esq., Tyson Arbuthnot, Esq., Michelle L. Baker, Esq., James Robert Maxwell, Esq., and Suzanne M. Mellard, Esq., Rogers Joseph O'Donnell; and Joseph O. Costello, Esq., Northrop Grumman Systems Corporation, for the intervenor.
Bryan R. O'Boyle, Esq., Col. Neil S. Whiteman, James A. Hughes, Esq., Col. Timothy Cothrel, Robert Balcerek, Esq., Maj. Christopher L. McMahon, W. Michael Rose, Esq., Stewart L. Noel, Esq., Gerald L. Trepkowski, Esq., Lynda Troutman O'Sullivan, Esq., John J. Thrasher III, Esq., Lt. Col. Thomas F. Doyon, Anthony P. Dattilo, Esq., Bridget E. Lyons, Esq., John R. Hart, Esq., Ronald G. Schumann, Esq., Maj. Steven M. Sollinger, Maj. Sandra M. DeBalzo, and John M. Taffany, Esq., Department of the Air Force, for the agency.
Guy R. Pietrovito, Esq., and James A. Spangenberg, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.
DIGEST
1. Protest is sustained, where the agency, in making the award decision, did not assess the relative merits of the proposals in accordance with the evaluation criteria identified in the solicitation, which provided for a relative order of importance for the various technical requirements, and where the agency did not take into account the fact that one of the proposals offered to satisfy more trade space technical requirements than the other proposal, even though the solicitation expressly requested offerors to satisfy as many of these technical requirements as possible.
2. Protest is sustained, where the agency violated the solicitation's evaluation provision that no consideration will be provided for exceeding [key performance parameter] KPP objectives when it recognized as a key discriminator the fact that the awardee proposed to exceed a KPP objective relating to aerial refueling to a greater degree than the protester.
3. Protest is sustained, where the record does not demonstrate the reasonableness of the agency's determination that the awardee's proposed aerial refueling tanker could refuel all current Air Force fixed'wing tanker'compatible receiver aircraft in accordance with current Air Force procedures, as required by the solicitation.
4. Protest is sustained, where the agency conducted misleading and unequal discussions with the protester, where the agency informed the protester that it had fully satisfied a KPP objective relating to operational utility, but later determined that the protester only partially met this objective, without advising the offeror of this change in its assessment and while continuing to conduct discussions with the awardee relating to its satisfaction of the same KPP objective.
5. Protest is sustained, where the agency unreasonably determined that the awardee's refusal to agree to the specific solicitation requirement that it plan and support the agency to achieve initial organic depot-level maintenance within 2 years after delivery of the first full-rate production aircraft was an administrative oversight, and improperly made award, despite this clear exception to a material solicitation requirement.
6. Protest is sustained, where the agency's evaluation of military construction costs in calculating the offerors' most probable life cycle costs for their proposed aircraft was unreasonable, where the evaluation did not account for the offerors' specific proposals, and where the calculation of military construction costs based on a notional (hypothetical) plan was not reasonably supported.
7. Protest is sustained, where the agency improperly added costs to an element of cost (non-recurring engineering costs) in calculating the protester's most probable life cycle costs to account for risk associated with the protester's failure to satisfactorily explain the basis for how it priced this cost element, where the agency did not determine that the protester's proposed costs for that element were unrealistically low.
8. Protest is sustained, where the agency's use of a Monte Carlo simulation model to determine the protester's probable cost of non-recurring engineering associated with the system demonstration and development portion of the acquisition was unreasonable, where the model's inputs concerned total weapons systems at an overall program level and there is no indication that this is a reliable predictor of anticipated growth of the protester's non'recurring engineering costs.
9. Protester is not required to file a defensive protest when during the procurement it is apprised of an agency's evaluation judgments with which it disagrees or where it believes the evaluation is inconsistent with the solicitation's evaluation scheme, because GAO's Bid Protest Regulations, 4 C.F.R. sect. 21.2(a)(2) (2008), require that where the protest involves a procurement conducted on the basis of competitive proposals under which a debriefing is requested and, when requested, is required, these protest grounds can only be raised after the offered debriefing.
10. While an agency, in an appropriate case, may request under GAO's Bid Protest Regulations, 4 C.F.R. sect. 21.3(d) (2008), that a protester provide specific relevant documents, of which the agency is aware and does not itself possess, this does not allow for wide-open document requests by an agency of broad categories of documents.
DECISION
The Boeing Company protests the award of a contract to Northrop Grumman Systems Corporation under request for proposals (RFP) No. FA8625-07-R-6470, issued by the Department of the Air Force, for aerial refueling tankers.[1] Boeing challenges the Air Force's technical and cost evaluations, conduct of discussions, and source selection decision.[2]
As explained below, we find that the agency's selection of Northrop Grumman's proposal as reflecting the best value to the government was undermined by a number of prejudicial errors that call into question the Air Force's decision that Northrop Grumman's proposal was technically acceptable and its judgment concerning the comparative technical advantages accorded Northrop Grumman's proposal. In addition, we find a number of errors in the agency's cost evaluation that result in Boeing displacing Northrop Grumman as the offeror with the lowest evaluated most probable life cycle costs to the government. Although we sustain Boeing's protest on grounds related to these errors, we also deny many of Boeing's challenges to the award.
Specifically, we sustain the protest, because we find that (1) the Air Force did not evaluate the offerors' technical proposals under the key system requirements subfactor of the mission capability factor in accordance with the weighting established in the RFP's evaluation criteria; (2) a key technical discriminator relied upon in the selection decision in favor of Northrop Grumman relating to the aerial refueling area of the key system requirements subfactor, was contrary to the RFP; (3) the Air Force did not reasonably evaluate the capability of Northrop Grumman's proposed aircraft to refuel all current Air Force fixed-wing, tanker'compatible aircraft using current Air Force procedures, as required by the RFP; (4) the Air Force conducted misleading and unequal discussions with Boeing with respect to whether it had satisfied an RFP objective under the operational utility area of the key system requirements subfactor; (5) Northrop Grumman's proposal took exception to a material solicitation requirement related to the product support subfactor; (6) the Air Force did not reasonably evaluate military construction (MILCON) costs associated with the offerors' proposed aircraft consistent with the RFP; and (7) the Air Force unreasonably evaluated Boeing's estimated non'recurring engineering costs associated with its proposed system development and demonstration (SDD).
BACKGROUND
Aerial refueling is a key element supporting the effectiveness of the Department of Defense's (DoD) air power in military operations and is, as such, an important component of national security. See AR, Tab 333, Capability Development Document, Dec. 27, 2006, at 2, 7; see also Air Force Refueling: The KC'X Aircraft Acquisition Program, Congressional Research Service (CRS) Report for Congress, No. RL34398,
To begin replacing the aging refueling tanker fleet, the Air Force established a three'pronged approach under which it intended to first conduct a procurement to replace the older KC-135 tankers, while maintaining the remaining KC-135 and KC-10 tankers; the first procurement, which is the acquisition protested here, was identified by the Air Force as the KC-X procurement or program. See AR, Tab 4, Acquisition Strategy Plan Briefing, at 9-10. The Air Force intends to replace the remaining KC'135 and KC-10 aircraft in later procurements under programs the agency identified as the KC-Y and KC-Z.
Solicitation
The RFP, issued
A detailed system requirements document (SRD) was provided in the RFP that presented the technical performance requirements for the KC-X aircraft. In this regard, the SRD stated that
[t]he primary mission of the KC-X is to provide world'wide, day/night, adverse weather aerial refueling . . . on the same sortie to receiver capable United States (U.S.), allied, and coalition military aircraft (including unoccupied aircraft). [The KC'X aircraft will] provide robust, sustained [aerial refueling] capability to support strategic operations, global attack, air-bridge, deployment, sustainment, employment, redeployment, homeland defense, theater operations, and special operations. Secondary missions for KC-X include emergency aerial refueling, airlift, communications gateway, aeromedical evacuation (AE), forward area refueling point (FARP), combat search and rescue, and treaty compliance.
RFP, SRD sect. 1.2.1. The SRD identified the minimum and desired performance/capability requirements for the aircraft. The minimum performance capabilities of the aircraft were identified in nine key performance parameters (KPP), which the Air Force summarized as follows:
| KPP | Parameter | Required Performance |
| 1 | Tanker Air Refueling Capability | Air refueling of all current and programmed fixed wing receiver aircraft |
| 2 | Fuel Offload and Range | Fuel, offload, range chart equivalent to KC'135 |
| 3 | Communications, Navigation, Surveillance/Air Traffic Management | Worldwide flight operations at all times in all civil and military airspace |
| 4 | Airlift Capability | Carry passengers, palletized cargo, and/or aeromedical patients on entire main deck |
| 5 | Receiver Air Refueling Capability | Refueled in flight from any boom equipped tanker aircraft |
| 6 | Force Protection | Operate in chemical/biological environments |
| 7 | Net-Ready | Meet enterprise-level joint critical integrated architecture requirements |
| 8 | Survivability | Operate in hostile environments (night vision and imaging systems, electromagnetic pulse, defensive systems: infrared detect and counter, radio frequency detect, no counter) |
| 9 | Multi-point Refueling | Multi-point drogue[5] refueling |
AR, Tab 46, Source Selection Evaluation Team (SSET) Final Briefing to Source Selection Advisory Council (SSAC) and Source Selection Authority (SSA), at 18.
The RFP provided for award on a best value basis and stated the following evaluation factors and subfactors:
| | |
| Key System Requirements | |
| System Integration and Software | |
| Product Support | |
| Program Management | |
| Technology Maturity and Demonstration | |
| Proposal Risk | |
| Past Performance | |
| Cost/Price | |
| Integrated Fleet Aerial Refueling Assessment (IFARA) | |
Offerors were informed that the mission capability, proposal risk, and past performance factors were of equal importance and individually more important than the cost/price or IFARA factors, and that the cost/price and IFARA factors were of equal importance. The subfactors within the mission capability factor were stated to be of descending order of importance. RFP sect. M.2.1.
The RFP stated that the agency, in its evaluation of proposals under the mission capability subfactors, would assign one of the color ratings identified in the solicitation,[6] and one of the proposal risk ratings that were also identified.[7] RFP sections M.2.2, M.2.3. In this regard, offerors were informed that proposal risk would only be assessed at the mission capability subfactor level and for only the first four subfactors. RFP sect. M.2.3.
With respect to the key system requirements subfactor, the most important mission capability subfactor, offerors were informed that the agency would assess the offerors' understanding of, and substantiation of their ability to meet, the requirements of the SRD (with the exception of the logistics requirements that were to be evaluated under the product support subfactor). The RFP provided that the offerors' approaches to meeting the SRD requirements would be evaluated under the key system requirements subfactor in the following five areas: aerial refueling, airlift, operational utility, survivability, and other system requirements. RFP sect. M.2.2.1.2.
In order for a proposal to be found acceptable under this subfactor (and overall), an offeror was required to meet the various identified minimum, mandatory KPP thresholds identified in the SRD for each of the nine KPPs. The SRD also identified KPP objectives relating to some, but not all of, the identified KPP thresholds. In this regard, the RFP stated that
[a]ll KPP thresholds [relating to the aerial refueling, airlift, operational utility, and survivability areas] must be met. Depending on substantiating rationale, positive consideration will be provided for performance above the stated KPP thresholds up to the KPP objective level. No consideration will be provided for exceeding KPP objectives. If there is no stated objective and, depending on substantiating rationale, positive consideration will be provided when the specified capability above the KPP threshold is viewed as advantageous to the Government.
RFP sect. M.2.2.1.1.a.
Among the minimum requirements identified in the SRD was a KPP No. 1 threshold that required the offeror's proposed aircraft to be capable of aerial refueling all current [Air Force] tanker compatible fixed wing receiver aircraft using current [Air Force] procedures . . . . RFP, SRD sect. 3.2.10.1.1.9. Another minimum requirement was a KPP No. 2 threshold that required the offeror's aircraft to be capable of satisfying the fuel offload versus unrefueled radius range as depicted in a linear graph contained in the RFP; this threshold charted the minimum pounds of fuel an aircraft must be capable of offloading to a receiver aircraft at a given distance of unrefueled flight by the tanker.[8] See RFP, SRD sect. 3.2.1.1.1.1. Also identified under KPP No. 2, as an objective, was that the aircraft should be capable of exceeding the fuel offload versus unrefueled radius range as depicted in this chart. RFP, SRD sect. 3.2.1.1.1.2.
In addition, the SRD identified numerous key system attributes (KSA) for the aerial refueling, airlift, operational utility, survivability, and other system requirements areas, as well as numerous other non'KPP/KSA requirements for these areas that were desired but not required.[9] The RFP provided that these requirements did not have to be satisfied by the offerors, but were desired and considered part of the offerors' design trade space.[10] RFP sect. M.2.2.1.1.b. With respect to these aspects of the evaluation of the key system requirements subfactor, offerors were informed that
[f]or non-KPP requirements, the Government may give consideration for alternate proposed solutions or capabilities below the stated SRD requirement, depending on substantiating rationale. The Government may give additional consideration if the offeror proposes to meet (or exceed if there is an objective) the SRD threshold or requirement, depending on substantiating rationale.
RFP sect. M.2.2.1.1.b. The RFP further stated that the Air Force sought an affordable KC'X system that not only met all of the KPP threshold requirements, but as many KSA and other SRD requirements as possible. RFP, SOO for KC-X SDD, at 2.
Finally, with regard to the overall evaluation of the key system requirements subfactor, the RFP stated that evaluation of the offeror's proposed capabilities and approaches against the SRD requirements will be made in the following descending order of relative importance: KPPs, KSAs, and all other non'KPP/KSA requirements. RFP sect. M.2.2.1.1.c.
With respect to the aerial refueling area of the key system requirements subfactor, offerors were informed that the agency's evaluation would include tanker aerial refueling, receiver aerial refueling, fuel offload versus radius range, drogue refueling systems (including simultaneous multi-point refueling), the operationally effective size of the boom envelope, the aerial refueling operator station and aircraft fuel efficiency. RFP sect. M.2.2.1.2.a. With respect to airlift area, the RFP provided that the agency's evaluation would include airlift efficiency, cargo, passengers, aero-medical evacuation, ground turn time, and cargo bay re-configuration. RFP sect. M.2.2.1.2.b. Offerors were instructed with regard to this area to provide an airlift efficiency calculation, based upon a calculation procedure stated in the solicitation, that would result in a payload pounds - nautical miles per pound fuel used calculation (in other words, the weight of cargo per pound of fuel burned). RFP sect. L.4.2.2.4.1. Under the operational utility area, the agency's evaluation would include aircraft maneuverability, worldwide airspace operations, communications/information systems (including Net-Ready capability), treaty compliance support, formation flight, intercontinental range, 7,000-foot runway operations, bare base airfield operations, and growth provisions for upgrades. RFP sect. M.2.2.1.2.c. The survivability area evaluation would include situational awareness, defensive systems against threats, chemical/biological capability, [electromagnetic pulse] protection, fuel tank fire/explosion protection, and night vision capability. RFP sect. M.2.2.1.2.d. The remaining other system requirements area evaluated SRD requirements were not included in any of the other areas. RFP sect. M.2.2.1.2.e.
Under the system integration and software subfactor, the evaluation was to consider the offeror's ability to implement a systems engineering approach and software development capability to satisfy the KC-X performance requirements, considering a number of listed attributes. RFP sect. M.2.2.2.
Under the product support subfactor, the evaluation was to consider the offeror's product support approach that includes logistics planning and analysis; interim contractor support; transition to organic two-level maintenance support; approach and rationale for proposed operational availability, reliability and maintainability and mission capable rates; logistics footprint; site activation/beddown; and training. RFP sect. M.2.2.3.
With respect to the program management subfactor, offerors were informed that the agency would assess whether the offeror's proposal demonstrates a capability to effectively and efficiently implement and manage the KC-X Program. RFP sect. M.2.2.4. Included in this evaluation was whether the offeror demonstrated a sound approach to achieving FAA Certification/Validation and a feasible, effective, low risk manufacturing and quality assurance approach to integrating military capability into the commercial baseline aircraft and transition to full rate production. RFP sections M.2.2.4.C, M.2.2.4.F.
With respect to the past performance factor, the RFP informed offerors that the agency's performance confidence assessment group (PCAG) would conduct an in'depth review and evaluation of all performance data to determine how closely the work performed under those efforts related to the effort solicited under the RFP. The RFP provided that for this factor the agency would assess the degree of confidence that the agency had in an offeror's ability to perform the tanker contract, based upon an assessment of the offeror's demonstrated record of performance, and focusing on performance in five areas: the four mission capability subfactors and the cost/price factor.[11] RFP sect. 2.4.1. In this regard, the RFP stated that the agency would consider each offeror's, and its major/critical subcontractor's, demonstrated record of performance. Offerors were also informed that, in assessing an offeror's past performance, the agency would consider the relevance of an offeror's (and its subcontractor's, joint venture's, and teaming partner's) present and past performance, and that [m]ore recent and more relevant performance by the same division/organization may have a greater impact on the performance confidence assessment than less recent or less relevant effort.
Where relevant performance records indicate performance problems, the Government will consider the number and severity of the problems and the appropriateness and effectiveness of any corrective actions taken (not just planned or promised). The Government may review more recent contracts or performance evaluations to ensure corrective actions have been implemented and to evaluate their effectiveness.
RFP sect. M.2.4.4.
With respect to the IFARA evaluation factor, the RFP provided that the agency would assess the utility and flexibility of a fleet of the offeror's proposed aircraft by evaluating the number of aircraft required to fulfill the peak demand of the aerial refueling elements evaluated in the 2005 Mobility Capabilities Study.[12] Specifically, offerors were informed that the Air Force would analyze offeror'provided data in the evaluation scenario primarily using the Combined Mating and Ranging Planning System (CMARPS) modeling and simulation tool to calculate a fleet effectiveness value, and would report this finding to the source selection authority (SSA), along with any major insights and observations gleaned from the evaluation.[13] To calculate the fleet effectiveness value, the agency, using the CMARPS modeling tool, would calculate the number of KC-135R aircraft and the number of the offeror's proposed aircraft needed to satisfy the scenario, and then divide the number of KC'135R aircraft required by the number of the offeror's aircraft. The RFP stated that, with respect to this ratio, a fleet effectiveness value of 1.0 would be equal in effectiveness to the KC-135R, while a value in excess of 1.0 would be viewed as more advantageous to the agency. RFP sect. M.2.6.
Under the cost/price factor, the RFP provided that offerors' proposed costs and prices would be evaluated for realism and reasonableness, respectively. RFP sect. M.2.5. Offerors were also informed that the agency would calculate a most probable life cycle cost (MPLCC) estimate for each offeror, which was described by the solicitation to be an independent government estimate, adjusted for technical, cost, and schedule risk, to include all contract, budgetary and other government costs associated with all phases of the entire weapon system life cycle (SDD, [Production and Deployment], and Operations and Support (O&S)). RFP sect. 2.5.2. The RFP provided that, as part of the other government costs, the agency would evaluate anticipated MILCON costs associated with the offerors' proposed aircraft. RFP sect. 2.5.2.4. The RFP also provided that the agency would assess technical, cost, and schedule risk for the entire most probable life cycle cost estimate based upon the offeror's proposed approach, and that the impact of technical, schedule, and/or cost risk will be quantified (dollarized), where applicable, and included in the MPLCC. RFP sect. M.2.5.2.5.
The RFP instructed the offerors to provide detailed cost information supported by a basis of estimate. Offerors were informed that the basis of estimate must
completely describe the cost element content . . . philosophy, and methodology used to develop the estimate including appropriate references to any historical supporting cost date.
RFP sect. L.6.4.7. The basis of estimate was required to include a narrative with supporting data explaining how the proposed cost estimates (SDD, [production and deployment], O&S) were created. RFP sect. L.6.2. With respect to proposed O&S costs, which include fuel costs, offerors were informed that they should assume a 25'year system life from the date each aircraft is delivered and calculate their O&S costs for 2 years beyond the date of their final production delivery; to support their O&S cost projections, offerors were required to provide all assumptions, ground rules, methodology, and supporting data.[14] RFP sections L.6.1.1.13, L.6.4.9. In this regard, the offerors were informed that if the historical data did not support the proposed prices, the cost documentation would be considered adequate only if the agency could understand the technical content, estimating methodology, and the build-up of the offerors' costs. RFP sect. L.6.4.7.
Proposals
The Air Force received proposals from Boeing and Northrop Grumman in response to the RFP. Boeing proposed as its KC-X aircraft the KC-767 Advanced Tanker, a derivative of its commercial 767-200 LRF (long range freighter) aircraft.[15] The KC-767 was composed of elements of a number of Boeing commercial aircraft, including the 767-200ER, 767-300F, 767-400 ER, 737, and 777 models. AR, Tab 61, Boeing Initial Technical Proposal, Executive Summary, at V1-ES-1. Boeing's proposed production plan for its SDD and production KC-X aircraft was to build the 767-200 LRF baseline aircraft at the
Northrop Grumman proposed the KC-30 aircraft, which was a derivative of the Airbus A330'200 commercial aircraft.[16] AR, Tab 140, Northrop Grumman Initial Technical Proposal, Executive Summary, at I-1. Northrop Grumman proposed a production plan that provided for a number of changed locations for the production, assembly, and modification of its SDD and LRIP aircraft. For the first SDD aircraft, Northrop Grumman proposed to build the commercial A330 aircraft in sections in various European locations, then assemble the aircraft in
SSET Evaluation
The proposals were evaluated by the agency's SSET, which initiated discussions with the offerors by issuing evaluation notices (EN).[18] After evaluating the offerors' EN responses, the SSET provided a mid-term evaluation briefing to the SSAC and SSA. Because there were concerns regarding how to properly show that all SRD requirements had been evaluated, the SSET prepared and provided another briefing to the SSA that detailed how each offeror's proposal was evaluated against each SRD requirement.
Following the offerors' mid-term briefings, the SSET provided a MPLCC/schedule risk assessment briefing to the SSAC and SSA, and subsequently the SSET provided MPLCC/schedule risk assessment briefings to the offerors. AR, Tab 133, Boeing's MPLCC/Schedule Risk Assessment Briefing; Tab 203, Northrop Grumman's MPLCC/Schedule Risk Assessment Briefing.
Extensive discussions were conducted with each offeror, after which a pre-final proposal revision briefing was provided to the SSAC and SSA by the SSET that presented updated evaluation ratings of Boeing's and Northrop Grumman's proposals and discussion responses. Following approval of this briefing by the SSA, the SSET again provided to each offeror the agency's evaluation ratings of their respective proposals. AR, Tab 135, Boeing's Pre-Final Proposal Revision Briefing; Tab 205, Northrop Grumman's Pre-Final Proposal Revision Briefing.
Final revised proposals were received from the offerors. Although intended by the agency to be the final proposal revisions, shortly after receipt of these proposals, the Air Force reopened discussions with the offerors in response to the enactment of the National Defense Authorization Act for Fiscal Year 2008, Pub. L. No. 110-181, 122 Stat. 3, 208-12, 222-24 (2008).[20] As a part of these discussions, the Air Force provided offerors with additional information concerning the firms' respective IFARA evaluations and with a clarified chart on Airlift Efficiency.
The protester's and awardee's final proposal revisions were evaluated by the SSET as follows:
| Boeing | Northrop Grumman | ||
| | |||
| Key System Requirements | Blue/Low | Blue/Low | |
| System Integration/Software | Green/Moderate | Green/Moderate | |
| Product Support | Blue/Low | Blue/Low | |
| Program Management | Green/Low | Green/Low | |
| Technology Maturity/Demonstration | Green | Green | |
| Past Performance | Satisfactory Confidence | Satisfactory Confidence | |
| Cost/Price (MPLCC) | $108.044 Billion | $108.010 Billion | |
| Cost Risk SDD Phase/Production & Deployment Phase | Moderate/Low | Low/Low | |
| IFARA Fleet Effectiveness Value | 1.79 | 1.9 | |
AR, Tab 46, SSET Final Briefing to SSAC and SSA, at 508, 532.
As indicated by the nearly identical evaluation ratings received by both firms' technical proposals and the nearly identical evaluated MPLCCs, the competition was very close, and, as evaluated, both firms' proposals were found to be advantageous to the government. Ultimately, the SSAC concluded, however, that Northrop Grumman's proposal was more advantageous to the agency than Boeing's under the mission capability, past performance, cost/price, and IFARA factors; the two firms were found to be essentially equal under the proposal risk factor. AR, Tab 55, Proposal Analysis Report (PAR), at 46-48.
SSAC's Mission Capability Factor Evaluation
Northrop Grumman's evaluated advantage under the mission capability factor was largely based upon the firm's perceived superiority under the key system requirements and program management subfactors; the two firms were found essentially equal under the remaining three subfactors.
The SSAC assigned both firms' proposals, under the key system requirements subfactor (the most important mission capability subfactor), blue, low risk ratings, noting:
Both Offerors proposed to meet all KPP Thresholds. Both Offerors proposed capability beyond KPP Thresholds and offered significant trade space KSA capability. Additionally, both offered numerous non-KPP/KSA trade space capabilities deemed beneficial to the Government.
In the aerial refueling area, the SSAC noted major discriminators in favor of Boeing under several KPP No. 1 objectives, including its capability to [Deleted] and [Deleted], and for a noteworthy non'KPP/KSA capability to [Deleted].
The SSAC also noted a number of major discriminators in favor of Northrop Grumman in the aerial refueling area, including one under the KPP No. 2 objective for Northrop Grumman's proposal to exceed the RFP's fuel offload versus unrefueled radius range (Boeing's aircraft was also evaluated as exceeding this KPP objective but to a lesser degree),[22] and for a number of non-KPP/KSA requirements, including the proposal of a better aerial refueling efficiency (more pounds of fuel offload per pound of fuel used) than Boeing's; a boom envelope that was [Deleted] times greater than that defined by the Allied Technical Publication (ATP)-56[23] (Boeing proposed a boom envelope that was [Deleted] times greater than that defined by the publication); and a higher offload and receive fuel rate than Boeing.
In the aerial refueling area, the SSAC also identified five discriminators offering less benefit for Boeing that were assessed under 14 different SRD requirements and one such discriminator for Northrop Grumman that was assessed under 2 SRD requirements.
The SSAC found that Boeing's proposal had no weaknesses in the aerial refueling area, but identified the following two weaknesses in Northrop Grumman's proposal:
The first weakness is related to the specified lighting around the fuel receptacle of the KC-30. The specified lighting for refueling as a receiver may provide [Deleted]. The second weakness is related to Northrop Grumman's boom approach. The [Deleted].
In the airlift area, the SSAC found that both offerors met all threshold requirements for the airlift KPP (there was only one KPP in this area), and that both offerors exceeded the threshold requirement for efficiently transporting equipment and personnel. AR, Tab 55, PAR, at 16. There were no KPP objectives identified by the SRD in the airlift area.
The SSAC identified one major discriminator in favor of Boeing in the airlift area: Boeing satisfied the non'KPP/KSA requirement for the capability to [Deleted].
Three discriminators offering less benefit were identified for Boeing in the airlift area and one such discriminator identified for Northrop Grumman. No proposal weaknesses were identified for either offeror in the airlift area.
In the operational utility area, the SSAC found that both offerors satisfied the three KPP thresholds identified in this area, and partially met the one KPP objective identified.[27] The SSAC also found that both offerors met the KSA thresholds and objectives in this area.
Ultimately, the SSAC concluded, largely based upon Northrop Grumman's evaluated advantages in the aerial refueling and airlift areas, that Northrop Grumman's proposal was superior to Boeing's under the key system requirements subfactor.[31] Specifically, the SSAC noted:
While [the] KC-767 offers significant capabilities, the overall tanker/airlift mission is best supported by the KC-30. [The] KC-30 solution is superior in the core capabilities of fuel capacity/offload, airlift efficiency, and cargo/passenger/aeromedical carriage. These advantages in core capabilities outweigh the flexibility advantages of the attributes which Boeing offered (e.g. [Deleted], etc.)
Under the program management subfactor, the SSAC assigned both offerors green, low risk ratings, identifying no strengths, deficiencies, or uncertainties in either firm's proposal.
Northrop Grumman's approach of providing four green aircraft for use early in SDD, by leveraging the existing A330 commercial production line, is deemed to be of benefit to the Government by reducing program risk. Northrop Grumman's approach adds value for the Government through increased confidence in overall program management.
Past Performance Factor Evaluation
The SSAC found that both offerors had equal confidence ratings in four of the five past performance areas; the only difference in ratings was with respect to the program management area, under which Northrop Grumman's past performance was assessed as satisfactory confidence but Boeing's proposal was assessed as little confidence.[32]
IFARA Factor Evaluation
The SSET also calculated a fleet effectiveness value for each proposed aircraft based upon offeror-provided data, which was analyzed under a variety of scenarios using the CMARPS modeling and simulation tool.[33] As noted above, the fleet effectiveness value reflected the quantity of an offeror's aircraft that would be required to perform the scenarios in relation to the number of KC-135R aircraft that would have been required. See RFP sect. M.2.6. The agency concluded that, whereas [Deleted] KC-135R aircraft would be required to perform the identified scenarios, the offerors' aircraft could perform the scenarios with fewer aircraft, that is, [Deleted] KC-30 aircraft and [Deleted] KC-767 aircraft. AR, Tab 55, PAR, at 45. The SSET calculated a fleet effectiveness value of 1.79 for the KC-767, and a higher (superior) value of 1.90 for the KC-30.
The SSAC also noted a number of insights and observations concerning the IFARA evaluation of the offerors' aircraft. With respect to Boeing's proposed aircraft, the agency stated that, as compared to the KC-135R in the peak demand scenario:
[the] KC-767 used [Deleted]% more ramp space (without requiring additional bases), burned [Deleted]% more fuel and was able to accomplish the scenarios with [Deleted] fewer aircraft when taking the aerial refueling receptacle into account. Additional aircraft were needed if every runway in the scenario were interdicted to 7,000 feet. In the base denial scenarios, when a base was closed, [Deleted]% of the Air Tasking Order (ATO) could be completed by basing KC-767s within the remaining bases' ramp space. Within the scenarios, [the] KC-767 offloaded between [Deleted]% and [Deleted]% of its fuel.
[the] KC-30 used [Deleted]% more ramp space (needing some additional bases), burned [Deleted]% more fuel and was able to accomplish the scenarios with [Deleted] fewer aircraft when taking the aerial refueling receptacle into account. In the base denial sensitivity assessment, in some cases when a base was closed, the [Deleted]. [The] KC-30 has exceptional short field capability if the runway is interdicted to 7,000 feet (as noted in Subfactor 1.1). Within the scenarios, [the] KC-30 offloaded between [Deleted]% and [Deleted]% of its fuel.
Cost/Price Evaluation
The Air Force calculated a MPLCC for each offeror, which, as noted above, was intended to be an independent government estimate of each proposal, adjusted for technical, cost and schedule risk and including all contract, budgetary and other government costs associated with all phases of the aircraft's entire life cycle (SDD, production and deployment, and O&S). See RFP sect. 2.5.2;
With respect to Boeing's proposal, the Air Force made a number of adjustments in Boeing's proposed costs in calculating its MPLCC. For example, the agency added an additional $[Deleted] million to Boeing's proposed costs of $[Deleted] billion for SDD because the agency concluded that the firm had not adequately supported its basis of estimate for these costs, despite repeated discussions on this issue. Most of this adjustment ($[Deleted] million) was associated with a moderate risk rating that was assigned to Boeing's cost proposal to account for the agency's concern that Boeing had not adequately supported its proposed $[Deleted] billion for non'recurring engineering costs that Boeing estimated it would incur in the development of its proposed aircraft. As another example, the Air Force added $[Deleted] billion to Boeing's proposed costs for the production and deployment lots 6 through 13 (the budgetary aircraft) because the agency concluded that Boeing had not substantiated an approximately [Deleted]'percent decrease in proposed costs for these lots following the fixed-price production lots (lots 1 through 5). The Air Force also upwardly adjusted Boeing's MPLCC by $[Deleted] billion for other government costs, the bulk of which ($[Deleted] billion) reflected additional O&S repair costs because the Air Force did not accept Boeing's estimating methodology of these costs. The agency also added additional costs to Boeing's MPLCC to account for the agency's estimated MILCON costs of $[Deleted] billion. AR, Tab 46, SSET Final Briefing to SSAC and SSA, at 451-76; Tab 55, PAR, at 40-42.
The Air Force also made a number of adjustments in Northrop Grumman's proposed costs, including upwardly adjusting the proposed SDD costs by $[Deleted] million and the firm's estimated costs for lots 6 through 13 (budgetary aircraft) by $[Deleted] million. In addition, the Air Force added additional costs to Northrop Grumman's MPLCC to account for the agency's estimated MILCON costs of $[Deleted] billion. AR, Tab 46, SSET Final Briefing to SSAC and SSA, at 479-502; Tab 55, PAR, at 42-43.
The Air Force calculated a MPLCC for Boeing of $108.044 billion and a MPLCC for Northrop Grumman of $108.010 billion.
In comparing the firms' evaluated costs, the SSAC noted that Northrop Grumman had a lower evaluated MPLCC, but that the firms' evaluated MPLCCs were within $34 million of each other (approximately a .03-percent difference). The SSAC noted, however, that Boeing's slightly higher evaluated MPLCC was driven primarily by the firm's much higher SDD costs, which reflected Boeing's more complex design, development, and integration activities. AR, Tab 55, PAR, at 43. In addition, the SSAC accepted the SSET's evaluation that Boeing's proposal presented a moderate cost risk for SDD. Northrop Grumman's proposal was assessed as a low cost risk for SDD costs. The SSAC viewed this difference in cost risk for the SDD phase to be the discriminator under this factor.
SSAC Recommendation
Ultimately, the SSAC recommended to the SSA that [the SSA] select Northrop Grumman's proposal for award, because the SSAC concluded that Northrop Grumman's proposal was more advantageous under the mission capability, past performance, cost/price, and IFARA evaluation factors. With respect to cost/price, the SSAC noted that, although the difference between the two proposals' MPLCC was negligible, Northrop Grumman's risk rating under this factor (low risk) for the SDD phase was lower than that assigned to Boeing's proposal (moderate cost/price risk) for the SDD phase.
Selection Decision
As noted above, the SSA was presented with the SSET's evaluation results in a number of briefings at various stages in the procurement. In addition, the SSA was briefed by the SSAC with respect to that council's recommendation for award and was presented with the SSAC's detailed PAR, which documented the SSAC's weighing of the offerors' respective strengths and weaknesses and the SSAC's award recommendation.
The SSA agreed with the SSAC's recommendation that Northrop Grumman's proposal reflected the best value to the agency, and [the SSA] identified Northrop Grumman's evaluated superiority under the mission capability, past performance, cost/price, and IFARA factors as supporting this conclusion; [the SSA] also concluded that neither offeror had an advantage under the proposal risk factor. With respect to the mission capability factor, the SSA emphasized that Northrop Grumman's evaluated superiority in the aerial refueling and airlift areas of the key system requirements subfactor were key factors in [the SSA's] decision.[34] AR, Tab 54, Source Selection Decision, at 9. Although not key to [the SSA's] determination that Northrop Grumman's proposal was more advantageous than Boeing's under the key system requirements subfactor, the SSA noted Boeing's evaluated superiority in the survivability area; [the SSA] also noted that neither offeror had an advantage in the operational utility area.
With respect to the aerial refueling area, the SSA noted that Northrop Grumman exceeded the KPP objective for fuel offload capability for the unrefueled radius range to a greater degree than did Boeing; this, the SSA found, demonstrated that a single KC-30 can refuel more receivers or provide more fuel per receiver than a single KC-767. AR, Tab 54, Source Selection Decision, at 5-6. In addition, the SSA noted that Northrop Grumman offered a larger boom envelope than Boeing, and proposed a superior fuel offload and receive rate than Boeing.
With respect to the airlift area, the SSA noted Northrop Grumman's superior airlift efficiency, dual cargo deck configuration, and ability to carry more passengers and aeromedical litters and patients. The SSA concluded that the KC-30's airlift capability was compelling to my decision.
In sum, the SSA selected Northrop Grumman's proposal for award, finding
Northrop Grumman's proposal was better than Boeing's proposal in four of the five factors evaluated and equal in one. Northrop Grumman's offer was clearly superior to that of Boeing's for two areas within KC-X's Mission Capability factor: aerial refueling and airlift. Additionally, Northrop Grumman's KC-30's superior aerial refueling capability enables it to execute the IFARA scenario described in the RFP with [Deleted] fewer aircraft than Boeing's KC'767 -- an efficiency of significant value to the Government. I am confident that Northrop Grumman will deliver within the cost, schedule, and performance requirements of the contract because of their past performance and the lower risk of their cost/price proposal.
Award was made to Northrop Grumman on
DISCUSSION
In its protest, which was supplemented numerous times as evaluation documents were provided during the development of the case,[36] Boeing challenges the Air Force's evaluation of technical and cost proposals, conduct of discussions, and source selection decision. In this regard, the protester identifies what it alleges are prejudicial errors under each of the RFP's evaluation factors and subfactors, and contends that, if the proposals had been evaluated in accordance with the RFP, its proposal would have been selected for award. As discussed below, we find a number of significant errors in the Air Force's evaluation under the key system requirements and product support subfactors of the mission capability evaluation factor and in its cost evaluation, and that the agency conducted misleading and unequal discussions with Boeing.[37]
Document Production
During the development of the record, Boeing requested that the Air Force provide various documents pursuant to our Bid Protest Regulations, 4 C.F.R. sect. 21.3(g). We granted Boeing's requests in part where we were persuaded that the requested documents were relevant to the protest issues.
The Air Force also requested that Boeing produce certain broad categories of documents bearing upon, among other things, Boeing's interpretation of the solicitation and several of its protest allegations. Boeing objected to that request, asserting that the documents sought were not relevant. The agency responded that its request was reasonable and limited, and sought relevant documents, which would be necessary to allow GAO to perform a complete and accurate review of the issues in Boeing's protests. Air Force's Response to Boeing's Objection to Air Force's Document Production Request (Apr. 11, 2008) at 1.
Our Bid Protest Regulations provide, in pertinent part, that [i]n appropriate cases, the contracting agency may request that the protester produce relevant documents, or portions of documents, that are not in the agency's possession. 4 C.F.R. sect. 21.3(d).
We denied the Air Force's request, because our regulations do not provide for broad agency request for documents whose existence and relevance are not at all apparent.[38] Rather, our regulations are intended to permit a contracting agency, in an appropriate case, to request a specific relevant document or documents, of which the agency is aware and does not itself possess. See 60 Fed.
Occasionally, the agency may be aware of the existence of relevant documents that only the protester possesses. In appropriate cases, the agency may request that the protester produce those documents.[39]
Dismissal Requests
Prior to the submission of the agency's report, the Air Force and Northrop Grumman requested that we summarily dismiss a substantial portion of Boeing's protest as untimely. The agency and intervenor argued that some of Boeing's protest grounds were untimely challenges to alleged, apparent solicitation improprieties. They also argued that some of Boeing's challenges to the agency's evaluation of proposals were untimely because Boeing was allegedly aware of the bases of these protest grounds during the competition, but did not protest until after award and the firm's receipt of a post'award debriefing.
Our Bid Protest Regulations contain strict rules for the timely submission of protests. These timeliness rules reflect the dual requirements of giving parties a fair opportunity to present their cases and resolving protests expeditiously without disrupting or delaying the procurement process. Peacock, Myers & Adams, B'279327,
We did not, and do not now, agree with the Air Force and Northrop Grumman that Boeing's protest is a challenge to the ground rules established by the RFP for this procurement. We find that Boeing, rather than objecting to any of the RFP's requirements or evaluation criteria, is instead protesting that the Air Force failed to reasonably evaluate proposals in accordance with the RFP's identified requirements and evaluation criteria.[40] We also do not agree with the agency and intervenor that, because Boeing was informed during the competition of the agency's view of the merits of its proposal and/or how the proposals were being evaluated, Boeing was required to protest the agency's evaluation or evaluation methodology prior to award and to the protester's receipt of its required debriefing. Even where the protester is apprised of agency evaluation judgments with which it disagrees or where it believes the evaluation is inconsistent with the solicitation's evaluation scheme, our Bid Protest Regulations require that these protest grounds be filed after the receipt of the required debriefing.[41] See 4 C.F.R. sect. 21.2(a)(2); see also 61 Fed. Reg. 39039, 39040 (July 26, 1996) (to address concerns regarding strategic or defensive protests, and to encourage early and meaningful debriefings, a protester shall not file an initial protest prior to its required debriefing); Rhonda Podojil--Agency Tender Official, B'311310, May 9, 2008, 2008 CPD para. 94 at 3 (application of debriefing exception to A'76 competitions conducted on the basis of competitive proposals).
Key System Requirements Subfactor Evaluation
Boeing complains that the Air Force failed to evaluate the firms' proposals under the key system requirements subfactor--the most important subfactor of the mission capability factor--in accordance with the RFP's identified evaluation scheme. As noted above, under this subfactor, the agency was to assess the offerors' understanding of, and ability to meet, the various SRD requirements. Boeing argues that the Air Force did not reasonably consider the weighting assigned to the various SRD requirements by the RFP in making its source selection,[42] even though they involve the major discriminators referenced in justifying the award to Northrop Grumman.[43] Boeing also asserts that the evaluation did not account for the fact that the RFP specifically requested offerors to propose as many of these trade space requirements as possible. In this regard, Boeing complains that the agency assigned no credit for the fact that Boeing's aircraft satisfied significantly more trade space SRD requirements than did Northrop Grumman's under the key system requirements subfactor. See Boeing's Post'Hearing Comments at 18.
The Air Force and Northrop Grumman deny that the agency failed to evaluate the firms' proposals in accordance with the solicitation criteria. They contend that the SSET performed an elaborate evaluation, identifying specifically how Boeing and [Northrop Grumman] met or exceeded KPP thresholds, and how each traded, partially met or met desired requirements (trade space). See, e.g., Air Force's Memorandum of Law at 62. The Air Force notes that the SSET identified potential strengths, which the SSAC categorized, as relevant here, as major discriminators or discriminators offering less benefit.
Because Boeing and [Northrop Grumman] offered significant trade space and the benefits for each [SRD] reference line capability were not of equal value, the SSAC gave positive consideration for additional capability beyond the applicable threshold based upon the magnitude of benefit to the Air Force. The offerors' approaches, their relative benefits, advantages, and operational contributions for the five areas within [the] Key System Requirements [subfactor] were evaluated by the SSAC for accomplishing the comparative analysis. The SSAC deliberated extensively, using expert technical, engineering, and operational judgment to carefully evaluate the capabilities offered, consistent with the RFP Measures of Merit and the priorities of KPP, KSA, and non-KPP/KSAs. Both offerors proposed highly capable solutions to the requirements that offered tremendous benefit above current Air Force tanker capability.
An agency is obligated to conduct an evaluation consistent with the evaluation scheme set forth in the RFP. FAR sect. 15.305(a); Serco, Inc., B-298266,
Here, we agree that the SSET's evaluation identified and documented the SRD requirements under which the firms' evaluated strengths and weaknesses were assessed. Nevertheless, the record does not establish that the SSAC and SSA, in considering those strengths and weaknesses, applied the relative weights identified in the RFP for the various SRD requirements (under which the KPPs were most important). Moreover, the record does not show any consideration by the SSAC or SSA of the fact that Boeing's proposal was evaluated as satisfying significantly more SRD requirements than Northrop Grumman's.
For example, the record shows that most of Boeing's evaluated major discriminators in the aerial refueling area were assessed under KPP requirements, and conversely most of Northrop Grumman's evaluated major discriminators were assessed under less important non-KPP/KSA requirements. Specifically, the SSAC identified as major discriminators the following requirements that Boeing's aircraft satisfied but Northrop Grumman's aircraft did not: (1) the capability to [Deleted] (a KPP No. 1 objective); (2) the capability, [Deleted] (another KPP No. 1 objective); (3) the capability to [Deleted] (another KPP No. 1 objective); and (4) the capability to [Deleted] (a noteworthy non'KPP/KSA requirement).[44] See AR, Tab 55, PAR, at 13. The SSAC identified as a major discriminator for Northrop Grumman that firm's satisfaction of one KPP objective (KPP No. 2 objective for exceeding the fuel offload unrefueled range), where Boeing also satisfied this objective but to a lesser degree. The Air Force also identified as major discriminators for Northrop Grumman under this area the firm's better air refueling efficiency, larger boom envelope, and better offload and receive rates, all of which were non-KPP/KSA requirements.
Although the record thus evidences that most of Boeing's evaluated major discriminators were assessed under KPP requirements, and conversely most of Northrop Grumman's evaluated major discriminators were assessed under less important non-KPP/KSA requirements,[45] we have found no document in the contemporaneous evaluation record that shows that the SSAC or SSA gave any meaningful consideration to the weights that were to be assigned to the various KPP, KSA, and other requirements. That is, the SSAC's briefing slides to the SSA and its PAR do no more than identify the SRD requirements for which the evaluated discriminators were assessed, but do not evidence any consideration of the descending order of importance assigned to these various SRD requirements.
In its briefing to the SSA, the SSAC merely reports each of the firms' advantages without any analysis of whether or not Boeing's advantages (which as indicated above are mostly derived from KPP objectives) were entitled to greater weight than Northrop Grumman's advantages (which are mostly derived from less important non'KPP/KSA requirements). See, e.g., AR, Tab 55, SSAC Recommendation Briefing to SSA, at 6-7 (aerial refueling discriminators). Similarly, in the PAR, the SSAC duly reports the relative order of importance that was to be assigned to the KPP, KSA and other requirements, see AR, Tab 55, PAR, at 4, but there is no suggestion that the assigned weights to these requirements were applied in any of the SSAC's comparative analyses of the firms' evaluated discriminators. See, e.g., id. at 13'14 (aerial refueling discriminators). Thus, although it is true that the SSAC reported in the PAR that it considered the priorities of KPP, KSA, and non'KPP/KSAs, see id. at 29, the record does not provide any evidence of such a weighing.
Similarly, in [the SSA's] selection decision, the SSA reports that the evaluation of the offerors' proposed capabilities and approaches against the SRD requirements were made in the following order of importance: KPPs, KSAs, and all other non-KPP/KSA requirements. See AR, Tab 54, Source Selection Decision, at 5. Despite this reported recognition of the varying weights assigned to the different SRD requirements, the SSA's decision document does not evidence any consideration of the fact that Boeing's assessed major discriminators were derived from requirements that were identified as being more important than most of the requirements from which Northrop Grumman's discriminators were derived. See id. at 5-7.
We agree with the Air Force that it is permissible to identify relative strengths found under less important evaluation factors to be discriminators for selection purposes, where there are lesser relative differences favoring another proposal under more important evaluation factors. However, we find no evidence in this record that any such analysis, which considered the relative weight of the KPPs, KSAs and non'KPP/KSA requirements, was performed here.
The Air Force also identified more discriminators offering less benefit for Boeing's proposal than for Northrop Grumman's proposal in the aerial refueling area. Specifically, the SSAC identified five such discriminators for Boeing that were assessed under 13 different SRD requirements, and only one such discriminator for Northrop Grumman that was assessed under 2 SRD requirements. As noted above, the RFP requested that offerors satisfy as many of the trade space SRD requirements as possible. See RFP, SOO for KC-X SDD, at 2. Despite having solicited proposals that satisfy as many SRD requirements as possible, there is no evidence in the record showing that either the SSAC or the SSA accounted for the fact that Boeing's proposal was evaluated as satisfying significantly more SRD requirements in the aerial refueling area than did Northrop Grumman's proposal. In fact, in deciding that Northrop Grumman had a significant advantage in the aerial refueling area, the SSA did not even discuss the fact that Boeing had more discriminators offering less benefit than did Northrop Grumman, much less that Boeing satisfied far more SRD requirements than did Northrop Grumman in this area.
As noted by the Air Force, the assignment of adjectival ratings and the source selection should generally not be based upon a simple count of strengths and weaknesses, but upon a qualitative assessment of the proposals. See Kellogg Brown & Root Servs., Inc., B'298694.7,
In short, our review of the record indicates that, as illustrated by the aerial refueling area examples above, the Air Force failed to evaluate proposals in accordance with the RFP's evaluation criteria.[46] That is, the record evidences that the Air Force failed to assess the relative merits of the offerors' proposals based upon the importance assigned to the various SRD requirements by the RFP or to account for the fact that Boeing proposed to satisfy far more SRD requirements than did Northrop Grumman.
Fuel Offload versus Unrefueled
Boeing protests that one of the key discriminators relied upon by the SSA in [the SSA's] selection decision was contrary to the RFP's evaluation criteria. This contention concerns the significant discriminator assessed by the Air Force under the aerial refueling area of the key system requirements subfactor. The assessed significant discriminator reflects the conclusion that Northrop Grumman's proposed aircraft exceeded to a greater degree than Boeing's aircraft a KPP objective to exceed the RFP's identified fuel offload to the receiver aircraft versus the unrefueled radius range of the tanker. The SSA noted in this regard that Northrop Grumman's aircraft exceeded the threshold by [Deleted] percent at 1,000 nautical miles and by [Deleted] percent at 2,000 nautical miles, whereas Boeing's aircraft exceeded the threshold by [Deleted] percent at 1,000 nautical miles and by [Deleted] percent at 2,000 nautical miles. AR, Tab 54, Source Selection Decision, at 5. This was a key reason supporting the SSA's determination that Northrop Grumman's proposed aircraft was more advantageous than Boeing's aircraft in the aerial refueling area and was superior overall to Boeing's. See id. at 6'7, 9, 19.
The RFP informed offerors that the agency would evaluate the offerors' approach to meeting the SRD requirements related to aerial refueling, which would include the fuel offload versus radius range. RFP sect. M.2.2.1.2.a. With respect to fuel offload versus unrefueled range, the RFP identified as a KPP threshold (a mandatory minimum requirement) the range that offerors must satisfy to be found acceptable. See RFP, SRD sect. 3.2.1.1.1.1. The RFP also identified as a KPP objective that the offerors' aircraft should be capable of exceeding the threshold. See RFP, SRD sect. 3.2.1.1.1.2. Finally, the RFP specifically informed offerors that [n]o consideration will be provided for exceeding KPP objectives. RFP sect. M.2.2.1.1.a.
Boeing argues that section M.2.2.1.1.a. of RFP unambiguously prohibited crediting Northrop Grumman for exceeding the fuel offload versus unrefueled range objective to a greater extent than Boeing. Boeing asserts that this limitation on providing credit for exceeding KPP objectives played an important role in shaping . . . how offerors designed and selected the aircraft that was proposed to meet the stated SRD requirements, see Protester's Comments at 14, and states that, had Boeing known of the agency's desire for a larger aircraft which can carry more fuel, it likely would have offered the agency an aircraft based upon the 777 aircraft platform.[47] See Protest at 2, 40.
The Air Force and Northrop Grumman respond that the agency appropriately found [Northrop Grumman's] superior ability to offload fuel at radius to be a major discriminator and of operational benefit to the Air Force. Air Force's Memorandum of Law at 70; see Northrop Grumman's Comments at 18-19. In this regard, the agency and intervenor argue, despite the plain solicitation language cited above by the protester, that the RFP, read as a whole, indicated to offerors that the agency would consider, and award credit for, the amount by which offerors proposed to exceed the fuel offload versus unrefueled radius range chart identified in the KPP. In this regard, the Air Force and Northrop Grumman argue that this KPP objective did not identify an objective level, and therefore this particular objective was unbounded, such that unlimited credit could be provided for exceeding this KPP objective. See, e.g., Air Force's Request for Partial Dismissal at 19; Northrop Grumman's Post'Hearing Comments at 102. The Air Force argues that:
[t]he RFP made clear that the Air Force desired maximum fuel offload at radius because it described the objective in qualitative rather than quantitative terms, thereby placing both offerors on notice that the extent to which each offeror's proposed solution exceeded the threshold could become a potential discriminator between the offerors.
Air Force's Memorandum of Law at 70. The agency also argues that, reading this KPP objective to exceed the fuel offload versus radius range threshold, see RFP, SRD sect. 3.2.1.1.1.2, in conjunction with the non-KPP/KSA trade space requirement that the aircraft should operate with maximum fuel efficiency, see RFP, SRD sect. 3.2.1.1.1.3, offerors should have known that the agency would be giving credit under this KPP objective for the degree to which the offerors would exceed the charted KPP threshold with no upward limits. See Air Force's Request for Partial Dismissal at 17. Northrop Grumman contends that Boeing's reading of this provision is inconsistent with the general nature of what the Air Force sought, which Northrop Grumman argues was a greater refueling capacity, including the possibility of reducing the number of airplanes required to complete a mission. Northrop Grumman's Comments at 27.
Where, as here, a dispute exists as to the actual meaning of a particular solicitation provision, our Office will resolve the matter by reading the solicitation as a whole and in a manner that gives effect to all its provisions; to be reasonable, an interpretation of a solicitation must be consistent with such a reading. Stabro Labs., Inc., B-256921,
We find from our review of the solicitation that the offerors were unambiguously informed that their proposals would not receive additional consideration or credit for exceeding a KPP objective. This is true whether we look to the express provision itself, the meaning of which is plain, or whether we view this restriction within the context of the whole solicitation. The only reasonable interpretation of the KPP objective here is that an offeror would be credited for meeting the fuel offload versus unrefueled radius range objective if its aircraft exceeded the charted KPP threshold, and that no additional credit would be provided for exceeding the charted threshold amount to a greater degree than other proposed aircraft.
Contrary to the Air Force's and Northrop Grumman's positions that this KPP objective was unbounded because no finite number or level is stated as part of the objective, the plain language of section M.2.2.1.2.a. of the RFP unequivocally prohibited any consideration for exceeding the stated KPP objective and the RFP did not suggest that the stated objective must be finite or be at an objective level in order for this section to be applicable. To read this provision as suggested by the intervenor and agency would render meaningless section M.2.2.1.2.a, and be inconsistent with identification of an objective for this KPP threshold. See Brown & Root, Inc. and Perini Corp., a joint venture, B-270505.2, B-270505.3,
The Air Force, as the drafter of the RFP, could have provided for unbounded consideration of the degree to which offerors exceeded the fuel offload versus unrefueled range, but did not. In fact, the last sentence in section M.2.2.1.1.a. states that [i]f there is no objective and, depending on substantiating rationale, positive consideration will be provided when the specified capability above the KPP threshold is viewed as advantageous to the Government. Thus, according to the RFP, unbounded credit could be given for exceeding the KPP where no KPP objective is stated (depending on the substantiating rationale and when advantageous to the government).[48] Indeed, the solicitation contained a number of KPP thresholds that did not have corresponding KPP objectives, see, e.g., RFP, SRD sect. 3.2.1.6.1. (KPP No. 4, Airlift Capability); sect. 3.2.8 (KPP No. 8, Survivability), but that is not the case with respect to this KPP threshold.
We also note that the RFP elsewhere specifically informed offerors of other objectives for which their proposals could receive additional consideration for exceeding objectives; that is, with respect to non-KPP requirements, the RFP stated that the agency may give additional consideration if the offeror proposes to meet (or exceed if there is an objective) the SRD threshold or requirement, depending on the substantiating rationale. See RFP sect. M.2.2.1.1.b. In addition, offerors were informed with regard to certain non-KPP objectives that they should try to exceed the requirement by as much as possible. See, e.g., RFP, SRD sect. 3.2.10.1.5.2.2 (The boom envelope should exceed the ATP-56 envelope as much as possible (OBJECTIVE).)
We also agree with Boeing that the RFP, read as whole, established a complex set of trade-offs for offerors to consider in determining what aircraft to propose to the agency, and we do not agree that common sense mandates that unbounded refueling capabilities were being sought by the RFP. [49] Although it is apparent that a larger aircraft could provide greater refueling capabilities, there could be associated disadvantages with respect to costs and space constraints. Thus, given that the RFP did not establish any size requirements or limitations upon the aircraft that could be proposed, the restriction on credit for exceeding this KPP objective provided offerors with a key consideration in determining what sort of aircraft to offer, as well as how to best structure their proposals.
As indicated above, the Air Force and Northrop Grumman argued that Boeing's protest of the agency's evaluation of the firms' proposal under this KPP objective is untimely because it is actually a challenge to the terms of the solicitation.[50] They base this argument upon their contention that Boeing learned of the agency's interpretation from the agency's briefings during the competition. However, we agree with Boeing's contention that the agency's briefings supported Boeing's understanding that no credit would be given for exceeding this KPP objective. For example, in Boeing's mid-term briefing, the Air Force reported to Boeing with regard to the aerial refueling area of the key system requirements that, although its aircraft exceeded the fuel offload versus unrefueled range and the agency identified by how much Boeing's aircraft exceeded the range, its proposal was evaluated to have met the objective.[51] See AR, Tab 129, Mid-term Briefing to Boeing, at 26. Similarly, in its pre-Final Proposal Revision Briefing, Boeing was informed that its offer to exceed the KPP threshold for this requirement was evaluated as having met the objective.[52] See AR, Tab 135, Pre-Final Proposal Revision Briefing to Boeing, at 30. Based on our review of the record, Boeing was not informed in its briefings of the SSA's and SSAC's interpretation that the RFP allowed unbounded credit to be given for exceeding the fuel offload versus unrefueled radius range KPP objective, and only became aware of the agency's interpretation from the redacted source selection decision that was provided to Boeing at its post-award required debriefing.[53]
In sum, we find that a key discriminator relied upon by the SSA in making [the SSA's] selection decision--that is, the assessment related to the KPP objective to exceed the fuel offload versus unrefueled range--was not consistent with the RFP. It is a fundamental principle of competitive procurements that competitors be treated fairly, and fairness in competitions for federal procurements is largely defined by an evaluation that is reasonable and consistent with the terms of the solicitation. For that reason, agencies are required to identify the bases upon which offerors' proposals will be evaluated and to evaluate offers in accordance with the stated evaluation criteria. See Competition in Contracting Act of 1984, 10 U.S.C. sect. 2305(a)(2)(A), (b)(1) (2000); FAR sections 15.304(d), 15.305(a); Sikorsky Aircraft Co.; Lockheed Martin Sys. Integration-Owego, B-299145 et al.,
KC-30 Overrun and Breakaway Capability
Boeing also complains that the Air Force did not reasonably assess the capability of Northrop Grumman's proposed aircraft to refuel all current Air Force fixed-wing tanker-compatible aircraft using current Air Force procedures, as required by a KPP No. 1 threshold under the aerial refueling area of the key system requirements subfactor.[54] See RFP, SRD sect. 3.2.10.1.1.9. Specifically, Boeing notes that current Air Force refueling procedures require that the tanker aircraft be capable of overrun and breakaway procedures when necessary, which would require the tanker aircraft to fly faster than the receiver aircraft or quickly accelerate during refueling.[55] Boeing's Second Supplemental Protest at 29. Boeing contends that the Air Force unreasonably determined that Northrop Grumman's proposed aircraft would meet these requirements.
With regard to the overrun issue, the record shows that Northrop Grumman was twice informed by the Air Force during discussions that the firm's initially identified maximum operational airspeed of [Deleted] Knots Indicated Air Speed (KIAS) would not be sufficient under current Air Force overrun procedures to achieve required overrun speeds of [Deleted] KIAS for various fighter aircraft, including the [Deleted], or [Deleted] KIAS for the [Deleted].[56] See AR, Tab 184, EN NPG-MC1-003, at 2; EN NPG-MC1-003a, at 1-2. Ultimately, Northrop Grumman informed the Air Force that a [Deleted] limited the aircraft's operational speed, but that Northrop Grumman proposed to include a [Deleted] to achieve the necessary overrun speed.[57] See id., Northrop Grumman Response to EN NPG'MC1'003a, at 2-7. The Air Force accepted Northrop Grumman's proposed solution as satisfying this KPP threshold. HT at 628.
Boeing complains that Northrop Grumman's proposed solution of [Deleted] to achieve overrun speed requires [Deleted], which is not consistent with the Air Force's current procedures as is required by the KPP.[58] See Boeing's Second Supplemental Protest at 29-32; Boeing's Comments at 64. Boeing also argues that the agency did not note that Northrop Grumman qualified its promise to increase its maximum operational airspeed in its EN response. Specifically, Boeing points out that Northrop Grumman stated that, [Deleted], the KC'30 had a maximum airspeed of [Deleted] KIAS, and not the [Deleted] KIAS evaluated by the Air Force. See AR, Tab 184, Northrop Grumman Response to EN NPG-MC1-003a, at 9.
At the hearing that our Office conducted in this protest, the Air Force produced its SSET mission capability factor team chief to testify regarding the agency's evaluation of the capability of Northrop Grumman's aircraft to satisfy this KPP threshold.[59] This witness, in response to direct examination, stated that the SSET found that [Deleted] would allow the KC-30 to achieve the necessary airspeed to perform the required overrun and breakaway procedures. Specifically, he testified that the SSET was convinced that, by [Deleted], the KC-30 could achieve an operational airspeed of [Deleted] KIAS, because Northrop Grumman had informed the agency in its EN response that the commercial A330 aircraft had a maximum dive velocity[60] of 365 KIAS and had been flight tested to a dive velocity of [Deleted] KIAS, and that analysis had been done showing that the A330 could even achieve a dive velocity of [Deleted] KIAS. HT at 626-27. The mission capability factor team chief testified that the SSET evaluated Northrop Grumman's response to indicate that the [Deleted], see HT at 637-38, and that in any event Air Force current procedures did not require the use of the [Deleted] during aerial refueling operations. HT at 638-39.
From this record, we cannot conclude that the Air Force reasonably evaluated the capability of Northrop Grumman's proposed aircraft to satisfy the KPP threshold requirement to refuel all current Air Force fixed-wing tanker-compatible aircraft using current Air Force procedures. The contemporaneous record, as explained by the hearing testimony, does not establish that the Air Force understood Northrop Grumman's response in discussions concerning its ability to satisfy the solicitation requirements, nor does it demonstrate that the agency had a reasonable basis upon which to accept Northrop Grumman's promises of compliance.
First, we agree with Boeing that the SSET erred in concluding that the [Deleted] in tanker refueling operations was not a current Air Force procedure.[61] See HT at 638, 735; Air Force's Post-Hearing Comments at 19. As noted above, the contemporaneous evaluation record shows that the agency interpreted the solicitation requirement to comply with current [Air Force] procedures to mean compliance with the procedures set forth in the agency's flight manuals for the KC'135 and KC-10 tanker aircraft, and expressly informed Northrop Grumman during discussions that the flight manuals for the KC-135 and KC-10 established the current Air Force procedures for refueling operations. See AR, Tab 184, EN NPG'MC1-003a, at 1, wherein the agency stated [a]erial refueling procedures were contained in T.O. 1-1C-1-3 and 1-1C-1-33 for the KC-135 and KC-10 respectively when the RFP was released.[62] These manuals show that current Air Force procedures provide that tanker pilots [Deleted] in refueling operations. For example, the KC-135 manual under Section IV, Air Refueling Procedures, warns tanker pilots that they must be prepared to assume aircraft control [Deleted], and under Section V, Emergency Air Refueling Procedures, instructs tanker pilots that in a breakaway situation, if a climb is required, they must [Deleted]. See AR, Tab 289, Flight Manual KC-135 (Tanker) Flight Crew Air Refueling Procedures, Supp. III, T.O. 1-1C-1-3,
Refueling is more demanding and difficult for both tanker and receiver aircraft if the tanker [Deleted]. For the tanker pilot, [Deleted]. For the receiver pilot, [Deleted]. Due to these realities, existing refueling guidelines dictate that [Deleted] should be used for refueling under normal circumstances. [Citations omitted.] Beginning aerial refueling [Deleted] should it become necessary, violates this policy. As previously noted, [Deleted].
Boeing's Comments, attach. 14, Declaration of Retired Air Force Pilot, at 3-4. Although the Air Force and Northrop Grumman generally disagree with Boeing's consultant that the Air Force's current procedures provide for the [Deleted], neither the agency or intervenor have directed our attention to anything in the KC'135 or KC'10 flight manuals or to any other source that would establish that Boeing's view, which appears to be reasonable on its face, is in error.
We also find unsupported the agency's conclusion that Northrop Grumman's proposed solution of [Deleted] did not also involve [Deleted]. In its EN response, Northrop Grumman informed the Air Force that 330 KIAS was the normal design maximum operating velocity of the commercial A330 aircraft, and that selection of a [maximum operating velocity] drives overall design characteristics of the aircraft, specifically aerodynamic and structural design limits, handling quality definition, and thrust. See AR, Tab 184, Northrop Grumman Response to EN NPG'MC1-003a, at 2. Northrop Grumman explained that its [Deleted] limited the aircraft to its maximum operating velocity, but that the firm could [Deleted] to exceed the maximum operating velocity. The awardee then stated three cases . . . to illustrate the performance of the KC-30 with and without [Deleted].
The SSET read, as described by the testimony of its mission capability factor team chief, Northrop Grumman's EN response to describe a fourth case (although not identified as such) under the third case heading, but located at the end of that section, where, the agency contends, the KC'30's [Deleted] but the [Deleted]. See HT at 664. However, we are unable to accept such a reading of Northrop Grumman's EN response. It ignores the logical structure of Northrop Grumman's response to the agency, which only identified and described three cases. Moreover, nowhere in its response to the agency's EN does Northrop Grumman suggest a fourth case where the [Deleted]; rather, the only reference to both the [Deleted] in the third case expressly states that the [Deleted] (Case 3: KC-30 [Deleted]).[63] See AR, Tab 184, Northrop Grumman Response to EN NPG'MC1-003a, at 6. In any event, given the uncertainty surrounding the agency's interpretation of Northrop Grumman's solution to a matter the agency believed could render the firm's proposal unacceptable, see HT at 625, 649, this is something the agency should have continued to clarify and resolve during discussions with the firm.[64]
Even apart from the agency's apparent misreading of Northrop Grumman's EN response and disregard of the current Air Force procedure to [Deleted], the record does not establish that the agency had a reasonable basis for concluding that Northrop Grumman's proposed solution would allow its aircraft to obtain the requisite overrun airspeeds to satisfy this KPP threshold. The witness that the Air Force produced to support its arguments on this point testified that the SSET had concluded that the KC-30 had the inherent capability of reaching airspeeds greater than [Deleted] KIAS (the aircraft's certified maximum operational airspeed) based upon the far greater airspeed ([Deleted] KIAS) identified by the firm for its certified dive velocity.[65] See HT at 624-28; Air Force's Post-Hearing Comments at 17'18. In this regard, the SSET apparently believed that simply [Deleted] would enable the aircraft to achieve its indicated dive velocity airspeed as its operational airspeed.
Although the SSET mission capability factor team chief repeatedly testified that the dive speed indicated that the aircraft would have the structural ability to fly at the dive speed limitation, see, e.g., HT at 674, he also admitted under cross examination that he did not know what the relationship was between maximum operating airspeed and design dive speed:
Q: What's your understanding of what the general margin is between maximum operational velocity and dive velocity?
A: I'm not aware.
Q: Was there somebody on your team that was advising you about what the general margin is or difference is between maximum operational velocity and dive velocity?
A: There could have been. We had advisors for handling qualities.
Q: I know you had advisors. I'm asking you, were there any advisors who actually helped you with understanding the difference between dive velocity and maximum operational velocity?
A: They did not help me, no.
Q: Did they help the team?
A: Not that I'm aware of.
HT at 669-70. The SSET mission capability factor team chief's (and presumably the SSET's) lack of knowledge concerning the relationship between maximum operating airspeed and design dive airspeed[66] is particularly troubling given the definition of maximum operating limit speed in FAA's regulations:
The maximum operating limit speed . . . is a speed that may not be deliberately exceeded in any regime of flight (climb, cruise, or descent), unless a higher speed is authorized for flight test or pilot training operations. [The maximum operating limit speed] must be established so that it is not greater than the design cruising speed . . . and so that it is sufficiently below [dive speed and velocity] to make it highly improbable that the latter speeds will be inadvertently exceeded in operations.
14 C.F.R. sect. 25.1505.
In sum, despite having identified, as an issue for the hearing, the capability of Northrop Grumman's proposed aircraft to satisfy the airspeed requirements of this KPP threshold, we have been presented with no testimony or documented analysis that explains why simply [Deleted] on the KC-30 would ensure that the proposed aircraft would achieve required overrun airspeeds that are in excess of its FAA certified maximum airspeed.[67] Furthermore, neither the Air Force nor Northrop Grumman has directed us to any documentation establishing that the agency analyzed what would be entailed in designing the KC-30 to exceed the certified maximum operational airspeed limit.[68] Given Northrop Grumman's recognition in its EN response that selection of the maximum operational airspeed limit drives overall design characteristics of the aircraft, specifically the aerodynamic and structural design limits, handling quality definition, and thrust, see AR, Tab 184, Northrop Grumman Response to EN NPG-MC1-003a, at 2, it would seem apparent that some design and FAA re-certification efforts could be necessary.
Boeing also complains that the Air Force did not reasonably evaluate the capability of Northrop Grumman's aircraft to initiate emergency breakaway procedures when refueling the [Deleted].[69] Current Air Force procedures, as reflected by the KC-135 flight manual, specifies that the tanker will refuel the [Deleted] at an airspeed of [Deleted] KIAS, see AR, Tab 289, Flight Manual KC'135 (Tanker) Flight Crew Air Refueling Procedures, Supp. III, T.O. 1-1C-1-3, Jan. 1, 1987, as revised Sept. 1, 2004, at [Deleted], and Northrop Grumman's EN response indicates that the KC'30's airspeed is limited to [Deleted] KIAS with the aircraft's [Deleted]. See AR, Tab 184, Northrop Grumman Response to EN NPG-MC1-003a, at 9. Boeing contends, citing the statement of its former tanker/receiver pilot consultant, that there is insufficient margin between airspeed at which [Deleted] are refueled and the KC-30's operational airspeed limit during refueling (a [Deleted]-KIAS margin) to allow for emergency breakaway maneuvers. See Boeing's Comments, attach. 14, Declaration of Retired Air Force Pilot, at 3-4.
As was true with respect to whether the KC-30 can satisfy the current Air Force procedures with respect to overrun airspeed, there is no documentation in the record setting forth an analysis of whether Northrop Grumman's proposed aircraft has sufficient operational airspeed when refueling the [Deleted] to initiate an emergency breakaway procedure. The agency's counsel provided a proffer at the hearing that the SSET's analysis of whether the KC-30 was capable of performing a breakaway maneuver with the [Deleted] was contained in the SSET's Final Evaluation Summary Report for Northrop Grumman. See HT at 784; see AR, Tab 215, Evaluation Summary Report for Northrop Grumman, at 3. Neither the page referenced by agency counsel or any other part of that document contains any analysis of whether Northrop Grumman's proposed aircraft can perform a breakaway procedure while refueling the [Deleted]; rather, the page referenced by agency counsel merely states that [t]he Offeror has substantiated the ability to deliver a KC-X aircraft that meets (minimum requirement) all KPP thresholds associated with aerial refueling, and provides no reasons or analysis supporting this conclusion. AR, Tab 215, Evaluation Summary Report for Northrop Grumman, at 3.
Although the SSET mission capability factor team chief was examined extensively about the SSET's consideration of the KC-30's ability to perform breakaway procedures, he recalled little about the SSET's discussions in this regard. His testimony does indicate, however, that the SSET accepted that the KC-30's maximum operational airspeed when refueling ([Deleted]) was [Deleted] KIAS, and that the SSET apparently believed that, to initiate the emergency breakaway procedure, with Northrop Grumman's proposed aircraft, the tanker would have to start accelerating and [Deleted] simultaneously. See HT at 706. During cross examination, the SSET mission capability factor team chief admitted that he did not know how long it would take [Deleted] Northrop Grumman's proposed [Deleted] or what the procedure was for [Deleted], nor was he aware of whether this was ever analyzed by the agency in its evaluation.[70] HT at 685-87, 707.
In sum, we conclude that the record does not demonstrate that the agency reasonably determined that Northrop Grumman's proposed aircraft would be able to refuel all current Air Force fixed-wing tanker-compatible receiver aircraft in accordance with current Air Force procedures as was required by this KPP No. 1 threshold.
Operational Utility Area
Boeing also complains that the Air Force unreasonably evaluated the firms' proposals in the operational utility area under the key system requirements subfactor. The RFP provided that evaluation of this area would consist of an assessment of the offeror's approach to meeting (or exceeding, where appropriate) SRD requirements, including the following: aircraft maneuverability, worldwide airspace operations, communication/information systems (including Net-Ready capability), treaty compliance support, formation flight, intercontinental range, 7,000-foot runway operations, bare base airfield operations, and growth provisions for upgrades. RFP sect. M.2.2.1.2.c. Boeing contends that its proposal should have been found technically superior to Northrop Grumman's in this area, and not essentially equal, as the SSA and SSAC concluded. See AR, Tab 54, Source Selection Decision, at 8.
As noted above, the SSET found that both offerors satisfied the three KPP thresholds identified in this area, and partially met the one KPP objective identified; the SSET also found that both offerors met all associated KSA thresholds and objectives. See AR, Tab 55, PAR, at 20-21. The SSAC also identified two major discriminators in each of the firms' respective proposals; the discriminators for Boeing were the firm's (1) [Deleted] and (2) [Deleted], and for Northrop Grumman were (1) the ability of the KC'30 to operate from a 7,000-foot runway carrying more fuel than the KC-767, and (2) the KC-30's longer ferry range compared to the KC-767's.
Here, too, as we described above with respect to the aerial refueling area, the record does not evidence that the SSAC and SSA, in determining that the firms' proposals were essentially equal in the operational utility area, gave any consideration to the fact that Boeing's proposal was evaluated as satisfying more SRD requirements than Northrop Grumman's in this area, as was sought by the RFP. Given this failure by the SSAC and SSA to address Boeing's apparent advantage in meeting more SRD requirements than Northrop Grumman, we conclude that the agency's evaluation and selection decision was unreasonable in this regard.
Boeing also complains that the agency conducted misleading discussions with Boeing with respect to whether Boeing had fully satisfied the KPP No. 7 objective, Net-Ready Capability. RFP, SRD sect. 3.2.4.1.1; app. A, Net-Ready Capability KPP, at 3. The KPP No. 7 objective provides that the offeror's system should be capable of accomplishing all operational activities identified in Table 5. RFP, SRD, app. A, at 4. Table 5 of the appendix identified a number of information exchange requirements.
Specifically, Boeing complains that at its mid-term briefing it was informed of an uncertainty regarding the firm's net ready capability, see AR, Tab 129, Mid-Term Briefing to Boeing, at 77, and that ultimately the firm responded to an EN concerning the firm's System Requirements Matrix and System Specification with respect to complying with the SRD requirements for KPP No. 7. See AR, Tab 210, Boeing Response to EN BOE-MC1-041. Boeing believed that its EN response charted how its proposal met the KPP No. 7 thresholds and objective in total, see Boeing's Comments at 29, and during the firm's Pre-Final Proposal Revision Briefing the Air Force informed Boeing that the firm met both the KPP thresholds and the objective requirements for KPP No. 7. See AR, Tab 135, Boeing's Pre'Final Proposal Revision Briefing, at 57. Accordingly, Boeing made no further revisions to its proposal in this area. Boeing's Second Supplemental Protest at 53. The Air Force, however, changed its evaluation rating of this aspect of Boeing's proposal to partially met the KPP objective (the same rating that Northrop Grumman received) without further notice to Boeing.[71] Boeing contends that the Air Force's misleading discussions prevented the firm from addressing the agency's concerns with respect to this objective.
The Air Force does not dispute that it informed Boeing during discussions that the firm had satisfied all of the thresholds and the objective under KPP No. 7, but contends that at the time it later determined that Boeing had not fully satisfied this objective, discussions had already been closed. See Second Supplemental
We do not agree with the Air Force that the agency was permitted, after informing Boeing that its proposal fully met this objective, to change this evaluation conclusion without affording Boeing the opportunity to satisfy this requirement. It is a fundamental precept of negotiated procurements that discussions, when conducted, must be meaningful, equitable, and not misleading. See 10 U.S.C. sect. 2305(b)(4)(A)(i); AT&T Corp., B-299542.3, B'299542.4,
In contrast, the Air Force informed Northrop Grumman prior to the submission of that firm's final proposal revision that it had only partially met this KPP objective, which permitted that firm the opportunity to further address the KPP objective requirements. See AR, Tab 205, Northrop Grumman's Pre-Final Proposal Revision Briefing, at 61. Moreover, Boeing submitted its final submission addressing this KPP objective several months prior to the pre-FPR briefing, and, as indicated above, the agency actually reopened discussions on other subjects after submission of the FPRs and obtained revised FPRs. Boeing's Protest at 66; Boeing's Second Supplemental Protest at 53. In short, the Air Force misled Boeing when the agency advised the firm that it met this objective, but later determined that Boeing did not fully meet this objective, and did not reopen discussions with Boeing on this issue. The Air Force also treated the firms unequally when it provided Northrop Grumman, but not Boeing, with continued discussions on this same objective. It is axiomatic that procuring agencies may not conduct discussions in a manner that favors one offeror over another. See FAR sect. 15.306(e)((1); Chemonics Int'l, Inc., B-282555,
We also find a reasonable possibility that Boeing was prejudiced by the Air Force's misleading and unequal discussions, given the greater weight that KPPs were supposed to receive in the agency's evaluation. In this regard, if Boeing had been evaluated as fully satisfying this KPP objective, which was the only KPP objective in the operations utility area, it could well have been considered to be superior in this area to Northrop Grumman, which was evaluated as only partially satisfying this KPP objective.
Other Key System Requirements Subfactor Issues
Boeing also protests the Air Force's conclusion in the aerial refueling area that Northrop Grumman's proposed larger boom envelope (relative to that offered by Boeing) offered a meaningful benefit to the Air Force. See AR, Tab 55, PAR, at 14. From our review of the record, including hearing testimony on this issue, we do not find a basis to object to the Air Force's judgment that Northrop Grumman had offered a larger boom envelope and that this offer provided a measurable benefit.[72]
Boeing also challenges the Air Force's evaluation judgment in the airlift area that Northrop Grumman's proposed aircraft offered superior cargo, passenger, and aeromedical evacuation capability than did Boeing's aircraft. From our review of the record, including the hearing testimony, we see no basis to conclude that the Air Force's evaluation that Northrop Grumman's aircraft was more advantageous in the airlift area is unreasonable.
Product Support Subfactor Evaluation
Boeing also complains that the Air Force misevaluated Northrop Grumman's proposal under the product support subfactor. This subfactor required the agency to evaluate the offeror's proposed product support approach for an efficient, effective and comprehensive support program for the service life of the KC-X fleet. RFP sect. M.2.2.3. Specifically, Boeing contends that the Air Force improperly ignored Northrop Grumman's refusal to commit to providing the required support necessary to allow the agency to achieve initial organic depot-level maintenance capability within the time required by the RFP, namely, within 2 years after delivery of the first full-rate production aircraft.[73] Boeing's Post-Hearing Comments at 84. The Air Force evaluated Boeing's and Northrop Grumman's proposals to be essentially equal under the product support subfactor.[74] See AR, Tab 54, Source Selection Decision Document, at 10; Tab 55, PAR, at 34.
Offerors were informed that the long-term support concept for the KC-X program was for two levels of organic maintenance: organization level and depot level, and that a program objective was a product support approach that effectively addressed all the integrated support elements, including [t]imely, cost effective transition to organic support. RFP, SOO for KC-X SDD, at 1-2. One of the specific minimum program tasks required by the SOO with regard to logistics was for the contractor to
[p]lan for and support the Government to achieve an initial organic [depot]-level maintenance capability in accordance with the [Source of Supply Assignment Process] for core-designated workloads, at a minimum, within two years after delivery of the first full-rate production aircraft.
The Air Force recognized in its evaluation that, although Northrop Grumman promised to provide the necessary planning and support for the agency to achieve an initial depot-level maintenance capability, the firm did not commit to providing this required support within 2 years after delivery of the first full-rate production aircraft, as required by the RFP. Thus, at the mid-term briefing, Northrop Grumman was informed that the timing of the firm's proposed depot level maintenance support was unclear, see AR, Tab 199, Northrop Grumman's Mid-Term Briefing, at 134, and then again at the pre-final proposal briefing, Northrop Grumman was informed that the agency had assigned it a weakness for its failure to include the time frame for initial organic depot standup in Offeror's Production SOW (SOO states within two years after delivery of the first full-rate production aircraft).[76] See AR, Tab 205, Northrop Grumman's Pre-Final Proposal Revision Briefing, at 141. Northrop Grumman did not resolve its failure to commit to the 2-year timeframe for this product support requirement during the procurement. In the firm's final proposal revision, Northrop Grumman stated in one place that resolution of this timing issue will be determined in coordination with the Government at contract award and, in another place, that action to resolve government identified weaknesses would occur after contract award. See AR, Tab 187, Northrop Grumman's Final Proposal Revision, KC-X Program Summary Document, at 2-3.
In its final evaluation, the SSET evaluated Northrop Grumman's refusal to commit to providing these product support services within the 2-year timeframe as a weakness. AR, Tab 46, SSET Final Briefing to SSAC and SSA, at 360, 362. The SSAC concluded that this was an administrative documentation oversight because Northrop Grumman had promised to provide the required services and its cost/schedule documentation is consistent with standing up depot capability within two years of delivery of the first full-rate production aircraft. AR, Tab 55, PAR, at 34. The SSA concurred with the SSAC that this was merely an administrative oversight. AR, Tab 54, Source Selection Decision, at 10.
We agree with Boeing that Northrop Grumman's refusal to commit to the required 2'year timeframe within which to provide these depot-level maintenance planning and support services cannot be reasonably viewed as an administrative or documentation oversight. As noted above, Northrop Grumman was clearly informed several times by the Air Force of the agency's concern that the firm had not committed to the required timeframe, and Northrop Grumman responded that it was not resolving this failure before award. Although throughout the protest and during the hearing, the agency steadfastly asserted that Northrop Grumman's failure to so commit was an oversight,[77] see, e.g., Air Force's Memorandum of Law at 151-53, in its post-hearing rebuttal comments, the agency admitted for the first time that Northrop Grumman's omission appeared to be a conscious decision. See Air Force's Post'Hearing Rebuttal Comments at 9. Northrop Grumman also finally admits in its rebuttal comments that its decision to not commit to the 2-year timeframe was intentional.[78] Northrop Grumman's Post'Hearing Rebuttal Comments at 29 n.13.
The Air Force and Northrop Grumman argue, however, that, apart from Northrop Grumman's refusal to commit to the 2-year timeframe, Northrop Grumman committed generally and specifically to performing the planning and support services solicited by the RFP in its proposal and proposal revisions, and that the firm would otherwise be obligated to perform the required services under whatever schedule the agency chooses. See, e.g., Air Force's Post-Hearing Rebuttal Comments at 11; Northrop Grumman's Post-Hearing Rebuttal Comments at 29. The parties disagree as to whether Northrop Grumman's proposal demonstrates the ability to provide the required services within 2 years of delivery of the first full-rate production aircraft, and based on our review of Northrop Grumman's proposal and revisions, we find that it is far from clear whether or not Northrop Grumman's proposed schedule establishes that it would perform these services within the 2-year time frame.
Whether or not Northrop Grumman's proposed schedule accommodates providing these product-support services within the 2-year timeframe misses the point, however. By explicitly refusing to contractually commit to the 2-year timeframe for providing these services in the SOW as it was repeatedly requested to do, we think that Northrop Grumman has taken exception to this solicitation requirement. See C'Cubed Corp., B-272525,
The Air Force and Northrop Grumman also argue that the 2-year requirement is not a material solicitation provision. However, their arguments in this regard are belied by the agency's contemporaneous actions during the procurement and the testimony of the SSET product support subfactor team chief. As noted above, the agency repeatedly raised this matter with Northrop Grumman during discussions in an unsuccessful effort to have the firm commit to this solicitation requirement, and Northrop Grumman just as steadfastly refused to commit. Moreover, the SSET product support subfactor team chief identified the purpose or intent of this particular SOO requirement as follows: It was a binding function to bind it to a specific time line, see HT at 1216, and that this 2-year requirement was an important requirement. HT at 1245. We find, from our review of the record, that the requirement to plan for and support the agency's achieving an initial organic depot'level maintenance capability within 2 years after delivery of the first full-rate production aircraft was a material requirement.
In sum, the Air Force improperly accepted Northrop Grumman's proposal, where that proposal clearly took exception to a material solicitation requirement.[79]
System Integration and Software Subfactor Evaluation
Boeing also complains that, although both firms were evaluated as acceptable but with a moderate risk under the system integration and software subfactor, the Air Force should have viewed Northrop Grumman's proposal as riskier than Boeing's. See Boeing's Comments at 100-01. The Air Force states that it viewed both firms' offers of substantial software reuse to be risky, and this, with other weaknesses the agency noted in each firm's proposal under this subfactor, resulted in an assignment of a moderate risk. We see no basis in this record to object to the agency's evaluation under this subfactor.
Program Management Subfactor Evaluation
Boeing also complains that the Air Force did not reasonably assess schedule or cost risks under the program management subfactor with respect to Northrop Grumman's proposed changes during contract performance in its production approach and production lines. See Boeing's Comments at 75-96. The Air Force contends that Northrop Grumman agreed to appropriate mitigation measures that supported the agency's conclusion that the firm presented low cost or schedule risk under the program management subfactor. From our review of the record, including hearing testimony on this issue, we do not find a basis to object to the Air Force's evaluation of Northrop Grumman's proposal under this subfactor.
Past Performance Factor Evaluation
Boeing also challenges the Air Force's evaluation of Boeing's and Northrop Grumman's past performance, arguing that the agency's assessment of the relevance of contracts to be considered was unreasonable, that the agency treated the offerors disparately, and that the past performance evaluation judgments were not adequately documented. See Boeing's Comments at 148. We find from our review of the record no basis to object to the Air Force's past performance evaluation, under which both firms' past performance received a satisfactory confidence rating. We also find no basis to question the SSA's judgment that, despite the equal confidence ratings that the firms received under this factor overall, Northrop Grumman's higher satisfactory confidence rating, as compared to Boeing's little confidence rating, under the program management area was a reasonable discriminator. The Air Force evaluated Boeing's past performance as marginal in this area based on the agency's judgments as to Boeing's program management performance under the [Deleted] contract, the [Deleted] contract, and the [Deleted] contract. We have no basis, on this record, to find the Air Force's judgment unreasonable.
IFARA Factor Evaluation
Boeing also challenges the Air Force's evaluation of the firms' proposals under the IFARA evaluation factor. Boeing complains that the Air Force unreasonably concluded that Northrop Grumman's proposed aircraft was superior to Boeing's under this factor based only upon the fleet effectiveness value and without considering evaluated major insights and observations, which Boeing asserts favored its proposal. See Boeing's Comments at 146. Our review of the record discloses that the SSAC and SSA did consider the agency's evaluated insights and observations in their evaluation of the firms' proposals under this factor, and therefore find no basis to object to the agency's evaluation.
Evaluation of MILCON Costs
Boeing also complains that the Air Force did not reasonably evaluate the firms' cost/price proposals in accordance with the RFP. As noted above, the solicitation provided that the Air Force would calculate an MPLCC estimate for each offeror, which reflected the agency's independent estimate of all contract, budgetary, and other government costs associated with all phases of the aircraft's life cycle from SDD through production and deployment and O&S; MILCON costs were specifically identified as a cost that the agency would evaluate in calculating the firms' MPLCCs. See RFP sect. M.2.5.2. Boeing contends that the Air Force's evaluation of MILCON costs greatly understated the difference between the firms' MILCON costs and that Northrop Grumman's much larger and heavier aircraft would have correspondently higher MILCON costs. See Boeing's Comments at 110-18; Boeing's Post-Hearing Comments at 117-18.
The Air Force disputes Boeing's complaint, contending that it reasonably assessed the likely life cycle costs associated with each firm's proposed aircraft. In this regard, the agency states that, because it did not know at which bases (beddown sites) the new KC-X aircraft would be assigned, it conducted site surveys at four airbases ([Deleted] Air Force Base (AFB), [Deleted] AFB, [Deleted] AFB, and [Deleted] AFB) to determine what military construction would be required at those bases for the offerors' proposed aircraft. The agency then extrapolated those results to six other airbases to calculate the agency's MILCON costs for the offerors. Air Force's Memorandum of Law at 221-22; Air Force's Post-Hearing Comments at 120'22. As indicated above, the agency added $[Deleted] billion in MILCON costs to Boeing's MPLCC and $[Deleted] billion in MILCON costs to Northrop Grumman's MPLCC. AR, Tab 55, PAR, at 40-43.
An agency's life cycle cost evaluation, like other cost analyses, requires the exercise of informed judgment concerning the extent to which proposed costs or prices represent a reasonable estimation of future costs. Our review of the agency's cost analysis is limited to the determination of whether the evaluation was reasonable and consistent with the terms of the RFP. See Cessna Aircraft Co., B-261953.5,
As a threshold matter, the Air Force admits that in defending this protest it discovered five errors in its assessment of MILCON costs, which, when corrected, would result in Boeing displacing Northrop Grumman as the offeror with the lowest evaluated MPLCC. Specifically, the Air Force states that it underestimated Northrop Grumman's MILCON costs by $122.5 million, and overestimated Boeing's costs by $3.3 million. After correction of these $125.8 million in errors, Boeing's MPLCC would be $108.041 billion and Northrop Grumman's would be $108.133 billion.[80] Air Force's Memorandum of Law at 201-02.
Here, the record shows that the agency's MILCON cost evaluation was otherwise flawed. In this regard, the RFP contemplated that the agency's MILCON cost evaluation would be based upon the offeror's proposed KC-X aircraft solution, see RFP sect. M.2.5.2.4, which is consistent with the rule that an agency must consider an offeror's proposed approach in estimating the likely costs associated with that offeror's proposal. See Hughes STX Corp., B-278466,
For example, although the Air Force recognizes that there will be a need for seat storage associated with the KC-X aircraft, the survey teams were unable to assess the likely MILCON costs associated with this need because, at the time of the surveys, the agency did not know the number of seats associated with the firms' respective aircraft.[81] See Air Force's Post'Hearing Comments at 127. Accordingly, at [Deleted] AFB, the team assumed that the offerors' aircraft had seating capacities similar to that of the KC-10 and, on this basis, concluded that the facilities at [Deleted] AFB were adequate. HT at 497. The KC-10, however, has only 75 seats, which is far less than the [Deleted] seats carried by the KC-30 and less than the [Deleted] seats carried by the KC'767. Similarly, at [Deleted] AFB, the survey team assigned no MILCON costs associated with seat storage because it determined, without any actual knowledge of the number of seats the proposed aircraft would carry, that there would be adequate storage available. Air Force's Post-Hearing Comments at 128. At [Deleted] AFB, the survey team concluded that there would be insufficient storage space to accommodate the seats and that an additional storage facility would need to be constructed; the cost of this facility ($[Deleted] million) was estimated to be the same for both offerors because the team did not know how many seats the aircraft carried and therefore assigned a seat requirement the same for both aircraft. HT at 499'500.
As another example, the survey team at [Deleted] AFB noted that the battery shop at the base may not have enough capability to service the batteries for the KC-X aircraft, if the new aircraft used different batteries from the other aircraft (the KC'135 and C-17 aircraft) at the base. AR, Tab 297, Site Survey Report for [Deleted] AFB, at 13. The team assigned no cost for this concern:
Battery concern was noted because we did not know what the requirements were for the two different batteries, since we did not know the battery type on the A330, does that mean you only need to be separated by certain amount of spaces in the facility? Could you put up a wall? Would you actually need a whole new facility? So we didn't have enough detail to know if we needed to build anything or if there was going to be no cost.
HT at 506. Other hearing testimony indicated that Boeing's proposed aircraft uses the same batteries as the [Deleted] aircraft, but that Northrop Grumman's aircraft may not. HT at 546-47; see also Boeing's Comments at 116.
Also unexplained in the contemporaneous record is the agency's failure to consider in its evaluation of MILCON costs the offerors' own estimates of likely MILCON at Fairchild AFB that were included in their proposals. Specifically, the RFP instructed offerors, as part of its response to the product support subfactor, to
describe the offeror's approach to meet the government's 2-level maintenance requirements. This proposal shall lay out:
* * * *
KC-X facilities, infrastructure requirements and design criteria.
Facilities required to support the first operational bed down location at (assume Fairchild AFB, WA), including requirements for space, utilities or special requirements (such as clean rooms, special storage, etc.) with sufficient detail to assess installation capabilities to support the KC-X. The offeror shall describe facilities recommended to support the KC-X aircraft. The offeror shall, at a minimum, address the square footage for parking, maintenance facilities, infrastructure (e.g., power requirements, compressed air, office requirements, storage), personnel, and support equipment required to operate two squadrons of 16 aircraft for Main Operating Base (MOB) 1 and MOB 2. MOB 3, MOB 4 and MOB 5 will be determined at a later date.
RFP sections L.4.2.4.4, L.4.2.4.4.5, L.4.2.4.4.6.
Northrop Grumman informed the Air Force in its proposal that based upon a
[Deleted].
AR, Tab 167, Northrop Grumman's Pre-Final Proposal Revision, vol. II, Mission Capability/Proposal Risk, Book 2, at II-SF3-48. Northrop Grumman also informed the agency that, among other changes that would be needed, [Deleted] in an identified building on Fairchild AFB would require [Deleted].
The Air Force argues that it was reasonable to ignore the offerors' views as to the sufficiency of the facilities at Fairchild AFB with respect to their proposed aircraft because this information was requested in the solicitation instructions for the product support subfactor, and the offerors were not informed that this information would be used in the agency's evaluation of MPLCCs.[82] See Air Force's Post-Hearing Rebuttal Comments at 22-23. The agency does not explain, however, for what purpose this information was requested if not to aid in its evaluation of the facilities that would be needed to support the KC-X aircraft at Fairchild AFB. Given that both offerors responded to this solicitation instruction, it is apparent that neither offeror was confused as to the purpose of this instruction, which plainly sought the offerors' views as to whether the facilities at Fairchild AFB were adequate for their respective aircraft. In short, we find no reasonable basis to ignore the information that both offerors provided with respect to the adequacy of, or need for changes to, facilities with respect to their proposed aircraft.
We also find, as described below, that the record does not otherwise demonstrate the reasonableness of the Air Force's notional (hypothetical) methodology for assessing likely MILCON costs. A notional beddown plan was developed because the agency did not know where the KC-X aircraft would be assigned. Under this scheme, the KC-X aircraft would be assigned in specified numbers to a test base (Air Force Materiel Command), a training base (Air Education & Training Command), three major operating bases (Air Mobility Command) within the continental United States (CONUS), four air reserve command (ARC) bases, and two major operating bases outside the continental United States (OCONUS).[83] See AR, Tab 309, Notional KC-X Beddown Plan Memorandum,
Where, as here, anticipated requirements cannot be reasonably ascertained, an agency may establish a reasonable hypothetical, or notional, plan to provide for a common basis for evaluating costs. See, e.g., PWC Logistics Servs., Inc., B-299820, B'299820.3, Aug. 14, 2007, 2007 CPD para. 162 at 11-15 Aalco Forwarding, Inc., et al., B'277241.15, Mar. 11, 1998, 98-1 CPD para. 87 at 11. But that said, we are unable to conclude on this record that the agency's extrapolation of the [Deleted] AFB MILCON costs to the ARC airbases provided a reasonable basis to evaluate these costs. In this regard, Boeing argues that [Deleted] AFB, as a former Strategic Air Command, bomber base, has a great deal more infrastructure than do ARC airbases and thus cannot be used as a reasonable forecast of potential MILCON costs, such as for pavement improvements for runways, ramps, and parking aprons, at other bases. See Boeing's Post-Hearing Comments at 136-41.
Although the Air Force dismisses Boeing's argument as being speculative and argues that many ARC airbases have substantial infrastructure, see Air Force's Post'Hearing Rebuttal Comments at 28, the agency has not produced any explanation for selecting [Deleted] AFB other than its similar squadron size, nor presented any evidence, either through testimony or by reference to documentation in the record, showing why the infrastructure at [Deleted] AFB would be similar enough to ARC airbases to find that the costs evaluated for [Deleted] AFB are a reasonable representation of the MILCON costs to be expected for ARC airbases.
Similarly, no evidence has been presented by the Air Force to explain why the application of a 10-percent premium to the costs assessed for [Deleted] AFB provides a reasonable estimate of MILCON costs for the OCONUS major operating bases. In this regard, Boeing contends, with no rebuttal, that the OCONUS airbases would have different issues associated with MILCON costs, such as compliance with foreign labor laws and foreign exchange rates, and that overseas bases would have to accommodate parking for all of the assigned KC-X aircraft, as opposed to the 75 percent of assigned aircraft that was done for CONUS airbases, such as [Deleted] AFB. See Boeing's Post-Hearing Comments at 137. The only evidence supporting the 10-percent factor is the testimony of the agency's SSET cost/price factor team chief that it was based on estimator judgment of one of the cost/price factor evaluators. HT at 209. No contemporaneous written description or explanation of that judgment has been provided for the record, however.
In sum, we do not find reasonable support in the record for the agency's evaluation of the MILCON costs.
Evaluation of Boeing's Non'recurring Engineering Costs
Boeing also protests the Air Force's MPLCC adjustment for Boeing's estimated non'recurring engineering costs in the SDD phase of the contract. The Air Force added $[Deleted] million to the MPLCC beyond the $[Deleted] billion for non'recurring engineering that Boeing estimated for the SDD phase. Boeing states that its proposal approach is to acquire the baseline 767'200 LRF aircraft (which Boeing asserts and the Air Force concedes is a commercial item) from its commercial division, BCA, under a fixed-price subcontract, and that its estimated non'recurring engineering costs are included in the subcontract's fixed price.[84] Boeing argues that the agency unreasonably did not accept Boeing's commercial data in support of its estimated non'recurring engineering costs, and that it was improper to add costs to its MPLCC, given that the non'recurring engineering costs are part of a fixed-price subcontract for a commercial item. See Boeing's Comments at 118-22.
The Air Force responds that, despite repeated discussions with Boeing regarding the firm's need to substantiate its estimated non'recurring engineering costs, see, e.g., AR, Tab 116, EN BOE'CP-001, EN BOE-CP-023, Boeing did not adequately support its estimated non'recurring engineering costs, and that the agency therefore concluded that there was a moderate risk associated with Boeing's non'recurring engineering cost estimate, although the agency did not determine that Boeing's estimated non'recurring engineering costs were unrealistic.
The agency also decided that it was necessary to upwardly adjust the MPLCC by $[Deleted] million to reflect this risk. To calculate this amount, the agency used a
We find reasonable the agency's assignment of a moderate risk to Boeing's proposal because of its failure to adequately substantiate its SDD non'recurring engineering costs. As noted above, the RFP placed upon the offerors the responsibility for substantiating their cost estimates. See, e.g., RFP sections L.6.1.2, 6.4.7. Here, the Air Force found, reasonably we conclude, that despite repeated requests during discussions Boeing failed to substantiate its SDD non'recurring engineering cost estimate. In this regard, we disagree with Boeing that, even if its purchase of the baseline aircraft from its commercial division, is considered to be the purchase of a commercial item, this prohibited the Air Force from requesting substantiating cost information from Boeing. Although FAR sect. 15.403'1(b)(3) provides that a contracting officer should not request the submission of certified cost or pricing data when a commercial item is being procured, this does not limit the right of the agency to request other cost information to determine price reasonableness or realism. See FAR sect. 15.403-3(c). We also note that it is not clear that the subcontract between Boeing and its commercial division is a fixed-price subcontract, as Boeing asserts, given Boeing's response in discussions that indicated that the price would not be fixed until the aircraft's configuration specifications were established, which had not yet happened. See AR, Tab 119, Boeing Response to EN BOE-K-015, at 2-3; Tab 259, Subcontract between Boeing and BCA.
Nevertheless, as discussed below, we conclude for a somewhat different reason that the Air Force's MPLCC adjustment of Boeing estimated non'recurring engineering costs for SDD was unreasonable. When an agency evaluates proposals for the award of a cost'reimbursement contract, an offeror's proposed estimated cost of contract performance is not considered controlling since, regardless of the costs proposed by an offeror, the government is bound to pay the contractor its actual and allowable costs. Earl Indus., LLC, B-309996, B-309996.4,
Here, the record shows that the Air Force made no determination that Boeing's estimated $[Deleted] billion for SDD non'recurring engineering costs was unrealistic. See Air Force's Post'Hearing Comments at 90-91. In this regard, the SSET cost/price factor team chief testified under cross examination as follows:
Q: Yes. You're supposed to look at whether what -- what Boeing proposed for the [non'recurring engineering], for the fixed price [non'recurring engineering] was realistic for the work to be performed. . . .
A: No.
* * * *
Q: But when you made your adjustment, for example, one of the things that I would expect you would do is you would try to make an adjustment to make it, looking at the third item there, consistent with unique methods of performance and materials described in the offeror's technical proposal. Did you make any adjustments consistent with the unique methods of performance and materials described in Boeing's proposal when you adjusted upward using thisRand study?
A: We added cost risk.
HT at 111-12.
The Air Force and Northrop Grumman argue that section M.2.5.2.5 of the RFP provided for the quantification of pure cost risk, and for including that quantified dollar amount in the agency's evaluated MPLCC. Air Force's Post-Hearing Comments at 87; Northrop Grumman's Post-Hearing Comments at 27-28. We disagree. This solicitation section states in its entirety:
Risk Adjustments. The Government will assess the technical, cost, and schedule risk for the entire most probable life cycle cost estimate based upon the offeror's proposed approach. The Government will perform a Schedule Risk Assessment (SRA) and quantify the schedule risk accordingly. The Government will also assess risks associated with technical content as identified in the evaluation of the Mission Capability factor/subfactors 1 through 4, and other pure cost risks as identified during the cost evaluation. The impact of technical, schedule, and/or cost risk will be quantified (dollarized), where applicable, and included in the MPLCC. Additionally, the Government reserves the right to adjust budgetary estimates for technical, cost, and schedule risk.
RFP sect. M.2.5.2.5.
We do not agree that this section allows the agency to upwardly adjust the cost element of an offeror's probable costs of performance where the agency does not conclude that the proposed cost element is unrealistic or not probable. Rather, we find that this section allows the agency to assess the risk associated with an offeror's probable costs and, where applicable, to quantify that risk and add the quantified amount in the agency's evaluated MPLCC for an offeror.[86] The increase to the MPLCC is applicable where the agency concludes that the higher number is more probable or more realistic than the lower one. The Air Force's and Northrop Grumman's reading is also inconsistent with other sections of the RFP that provided that the Air Force would assess the realism of offerors' proposed costs in accordance with FAR sect. 15.404-1 and that the agency's evaluated MPLCCs would be the agency's estimates of the probable or likely life cycle costs associated with the offerors' aircraft. See RFP sections M.2.5.1.1, M.2.5.2. Such a reading is also inconsistent with FAR sect. 15.404-1, which, as noted above, provides for adjusting an offeror's proposed costs to realistic levels based on the results of the cost realism analysis. See FAR sect. 15.404'1(d)(2)(ii).
Moreover, even assuming a cost realism adjustment would have been proper in this case, we do not find reasonable the agency's use here of its
Cost Evaluation Errors Conclusion
In sum, we find that the Air Force unreasonably evaluated the MILCON costs associated with the firms' proposed aircraft and unreasonably adjusted Boeing's estimated non'recurring engineering costs, without finding those costs to be unrealistic. The correction of these errors in the Air Force's cost evaluation result in Boeing's MPLCC being lower than that of Northrop Grumman's.[88]
Other Cost Issues
Boeing also challenges a number of other aspects of the Air Force's evaluation of its cost proposal, including the Air Force's addition of $[Deleted] billion to Boeing's proposed costs for budgetary aircraft (lots 6 through 13) and the addition of $[Deleted] billion to reflect additional O&S repair costs. In addition, Boeing challenges a number of aspects of the Air Force's evaluation of Northrop Grumman's proposed costs, including that the agency did not evaluate the fuel costs associated with that firm's larger and heavier aircraft and the costs of upgrades (such as the [Deleted]) that may be added to Northrop Grumman's aircraft in the future. We find no basis from our review of the record to object to the agency's evaluation of these other aspects of the Air Force's evaluation of costs.[89]
CONCLUSION AND RECOMMENDATION
This decision should not be read to reflect a view as to the merits of the firms' respective aircraft. Judgments about which offeror will most successfully meet governmental needs are largely reserved for the procuring agencies, subject only to such statutory and regulatory requirements as full and open competition and fairness to potential offerors. Foundation Health Fed. Servs., Inc.; QualMed, Inc., B-254397.4 et al.,
The protest is sustained.
We recommend that the Air Force reopen discussions with the offerors, obtain revised proposals, re-evaluate the revised proposals, and make a new source selection decision, consistent with this decision. If the Air Force believes that the RFP, as reasonably interpreted, does not adequately state its needs, the agency should amend the solicitation prior to conducting further discussions with the offerors. If Boeing's proposal is selected for award, the Air Force should terminate the contract awarded to Northrop Grumman. We also recommend that Boeing be reimbursed the reasonable costs of filing and pursuing the protest, including reasonable attorneys' fees. 4 C.F.R. sect. 21.8(d)(1). Boeing should submit its claim for costs, detailing and certifying the time expended and costs incurred, with the contracting agency within 60 days after receipt of this decision. 4 C.F.R. sect. 21.8(f)(1).
Gary L. Kepplinger
General Counsel
[1] This acquisition has been identified as a Major Defense Acquisition Program. See Agency Report (AR), Tab 8, Acquisition Strategy Report, at 1.
[2] The record in this case, which the agency largely provided electronically to GAO and the private parties, is voluminous and complex, and some of the record is classified. Although we considered the classified information, it is not discussed in this decision.
[3] The Air Force acquired 732 KC-135A aircraft between 1957 and 1965. In the 1980s, a number of KC-135A aircraft were upgraded to the KC-135E aircraft, and later other KC-135A aircraft were upgraded to the KC-135R aircraft. Currently, the Air Force has 85 KC-135E aircraft and 418 KC-135R aircraft in its fleet. The agency also has 75 newer KC-10A aircraft in its fleet. See Air Force Refueling: The KC-X Aircraft Acquisition Program, CRS Report for Congress, at 4-5.
[4] LRIP is defined as:
The first effort of the Production and Deployment (P&D) phase. The purpose of this effort is to establish an initial production base for the system, permit an orderly ramp-up sufficient to lead to a smooth transition to Full Rate Production (FRP), and to provide production representative articles for Initial Operational Test and Evaluation (IOT&E) and full-up live fire testing. This effort concludes with a Full Rate Production Decision Review (FRPDR) to authorize the Full Rate Production and Deployment (FRP&D) effort.
Glossary of Defense Acquisition Acronyms & Terms,
[5] A drogue is a small windsock placed at the end of a flexible hose that trails from a tanker aircraft in flight in order to stabilize the hose and to provide a funnel for the receiver aircraft, which inserts a probe into the hose to receive fuel. See Aerial Refueling Methods: Flying Boom versus Hose-and-Drogue, CRS Report for Congress, No. RL32910,
[6] For example, a blue rating reflected an exceptional proposal that
[e]xceeds specified minimum performance or capability requirements in a way beneficial to the Government; proposal must have one or more strengths and no deficiencies to receive a blue.
A green rating reflected an acceptable proposal that
[m]eets specified minimum performance or capability requirements delineated in the [RFP]; proposal rated green must have no deficiencies but may have one or more strengths.
RFP sect. M.2.2.
[7] For example, a low risk rating reflected a proposal that
[h]as little potential to cause disruption of schedule, increased cost or degradation of performance. Normal contractor effort and normal Government monitoring will likely be able to overcome any difficulties.
A moderate risk rating reflected a proposal that
[c]an potentially cause disruption of schedule, increased cost, or degradation of performance. Special contractor emphasis and close Government monitoring will likely be able to overcome difficulties.
RFP sect. M.2.3.
[8] For example, the graph indicated an aircraft must be capable of offloading 117,000 pounds of fuel at a radius of 500 nautical miles and 94,000 pounds at a radius of 1,000 nautical miles. RFP, SRD sect. 3.2.1.1.1.1, Figure 3-1, Fuel Offload vs.
[9] Although identified as requirements by the RFP, these non-KPP requirements were not mandatory, but reflect features and performance of the aircraft that the agency desired. There were thresholds and objectives identified for some of the KSAs and the other SRD requirements.
[10] The Air Force described trade space as follows:
[the RFP] also provided the offerors considerable trade space, meaning some performance parameters of the tanker were required, while others were not. The optional capabilities or attributes could be traded away for better or different performance in other areas depending on the offeror's unique approach. . . . Essentially, this asked the offerors to tender their best proposals, and encouraged them to be creative in doing so. With such a structure, the RFP harnessed the power of the commercial marketplace competition to drive innovation as well as efficiency.
Air Force's Memorandum of Law at 5.
[11] The RFP provided that the PCAG would assign a confidence rating of high confidence, significant confidence, satisfactory confidence, unknown confidence, little confidence, or no confidence.
[12] The 2005 Mobility Capabilities Study assessed the mobility capabilities of DoD against the backdrop of a revised National Security Strategy; the study was intended to support decisions on future strategic airlift, aerial refueling, aircraft, and sealift procurements needed to meet varying military requirements. See Defense Transportation: Study Limitations Raise Questions about the Adequacy and Completeness of the Mobility Capabilities Study and Report, GAO-06-938, at 6.
[13] CMARPS is a system that is comprised of the Contingency Mating and Ranging Program, Tanker Mating and Ranging Program, and Graphically Supported Interactive Control System user interface. Using inputs, such as aircraft performance characteristics, and assumptions and ground rules (such as the maximum number of a particular aircraft that could be located at a particular base given ramp geometrics and aircraft dimensions and pavement strengths of ramps and runways), the agency would conduct simulations, or runs, where a proposed tanker fleet attempts to satisfy tanker demand; the results of these simulations are intended to reflect the effectiveness of those runs. See Fourth Supplemental Contracting Officer's Statement (COS) at 9-12.
[14] With regard to fuel costs, offerors were requested to provide a fuel-consumption sample calculation for an average mission-ready KC-X, including fuel, crew, and mission equipment on board, in gallons per hour per primary aircraft assigned multiplied by the number of flying hours in a given fiscal year. Offerors were required to document the source of the input data and rationale. RFP sect. L, attach. 15, KC-X O&S Data Form, at 7.
[15] Boeing stated in its proposal that the 767-200LRF is a new minor model (a family of variants as defined by the FAA such as 767-200, 767-300F, or 767-400ER) that includes design features that satisfy KC-X requirements. AR, Tab 61, Boeing Executive Summary, at V1-ES-1.
[16] Airbus is a division of European Aeronautic Defence & Space Company (EADS), Northrop Grumman's principal subcontractor for this procurement. After award, the Air Force changed the designation of Northrop Grumman's aircraft to the KC-45; throughout this decision, however, we refer to Northrop Grumman's aircraft by the firm's KC'30 designation.
[17] Although not requested by the parties, we conducted a hearing to receive testimony from a number of Air Force witnesses to complete and explain the record. In this regard, we provided a detailed description of the hearing issues to the parties in a pre-hearing conference and in a written Confirmation of Hearing notice. We also expressly informed the parties that identification of some of the protest issues as hearing issues did not indicate GAO's views as to the merits of any issue in the case. The Air Force was informed that it was responsible for identifying and producing those witnesses who could knowledgeably testify with respect to the identified issues. Although invited to do so, neither Boeing nor Northrop Grumman offered any witnesses. At the conclusion of the hearing, the parties were informed that they could address any aspect of the protest in their post-hearing comments and rebuttal comments. HT at 1524.
[18] The Air Force conducted numerous rounds of written and oral discussions with the firms; in total, Boeing received 271 ENs, and Northrop Grumman received 295 ENs. AR, Tab 46, SSET Final Briefing to SSAC and SSA, at 9-10.
[19] Limited information was provided to the offerors in the mid-term briefing with respect to the agency's schedule risk assessment and its impact on the offeror's MPLCC.
[20] Discussions were conducted with the offerors to address any possible impact on their proposals from section 804 of the National Defense Authorization Act for Fiscal Year 2008 (related to Buy American Act requirements with respect to specialty metals) and section 815 of the Act (related to treatment of major defense acquisition program systems, components, and spare parts as commercial items). Air Force's Memorandum of Law at 20 n.6.
[21] A major discriminator was defined to be
an offered feature evaluated as a strength that provided extensive capability and a substantial difference in magnitude of benefit to the Air Force, when compared to the other Offeror.
A discriminator offering less benefit was defined to be
an offered feature evaluated as a strength that provided some capability and some difference in benefit to the Air Force when compared to the other Offeror.
AR, Tab 55, PAR, at 12.
[22] The SSAC reported that the KC-30 met the objective by offering a fuel offload versus unrefueled range capability of [Deleted] pounds at 1,000 nautical miles and [Deleted] pounds at 2,000 nautical miles, which exceeded the threshold by [Deleted] percent at 1,000 nautical miles and by [Deleted] percent at 2,000 nautical miles. The KC-767 was also found to meet the objective by offering a capability of [Deleted] pounds at 1,000 nautical miles and [Deleted] pounds at 2,000 nautical miles, which exceeded the threshold by [Deleted] percent at 1,000 nautical miles and [Deleted] percent at 2,000 nautical miles. AR, Tab 55, PAR, at 13-14.
[23] The ATP is an aerial refueling publication issued by the North Atlantic Treaty Organization.
[24] Airlift efficiency was calculated using the following formula: (pounds of payload) x (nautical miles)/(pounds of fuel). The SSET performed this calculation at various distances for the offerors to derive a payload-range curve to provide for a comparative analysis. AR, Tab 55, PAR, at 17.
[25] The 463L pallet is the standard air cargo pallet used by the Air Force and within the defense transportation system.
[26] The SSAC noted, however, that the KC-30's total weight carriage capability on the main cargo deck was not substantially greater than that of the KC-767. AR, Tab 55, PAR, at 17.
[27] The KPP No. 7 objective, the only objective under this KPP, provides that the offeror's system should be capable of accomplishing all operational activities identified in Table 5. RFP app. A, Net'Ready Key Performance Parameter for the KC-X SRD,
[28] The capability to operate from a 7,000-foot runaway at sea level at the aircraft's maximum gross weight was a non-KPP/KSA trade space requirement, see RFP, SRD sect. 3.2.1.1.4.2, which both Boeing and Northrop Grumman satisfied. AR, Tab 55, PAR, at 21.
[29] An unrefueled ferry range of 9,500 nautical miles starting at maximum takeoff gross weight and using a maximum range flight profile was identified as a non'KPP/KSA trade space requirement. RFP, SRD sect. 3.2.1.1.1.4.
[30] The SRD provided that the KC-X shall be capable of supporting aerial refueling operations from bare base airfields with confined ramp space. RFP, SRD sect. 3.2.10.1.1.3.
[31] The SSAC did recognize that Boeing's proposal was more advantageous than Northrop Grumman's in the survivability area. AR, Tab 55, PAR, at 29.
[32] A satisfactory confidence rating was assigned where
[b]ased on the offeror's performance record, the government has confidence the offeror will successfully perform the required effort. Normal contractor emphasis should preclude any problems.
A little confidence rating was assigned where
[b]ased on the offeror's performance record, substantial doubt exists that the offeror will successfully perform the required effort.
RFP sect. M.2.4.1.
[33] Much of the information detailing the agency's evaluation under the IFARA factor is classified.
[34] The SSA concluded that the offerors' proposals were essentially equal under the remaining four mission capability subfactors. Regarding the evaluation of Northrop Grumman's proposal under the product support subfactor, although the Air Force found that Northrop Grumman had failed to specifically commit to providing planning and support for the initial organic D-level [depot-level] maintenance capability within 2 years following delivery of the first full-rate production aircraft as required by the RFP, see RFP, SOO for KSC-X SDD, at 14, the SSA agreed with the SSAC that this was merely an administrative oversight. AR, Tab 54, Source Selection Decision, at 10. (The SSAC termed Northrop Grumman's failure in this regard to be an administrative documentation oversight. AR, Tab 55, PAR, at 34.)
[35] Where, as here, a procurement is conducted on the basis of competitive proposals, an unsuccessful offeror, upon written request received by the agency within 3 days after the date on which the unsuccessful offeror receives the notification of the contract award, shall be debriefed and furnished the basis for the selection decision and award. 10 U.S.C. sect. 2305(b)(5)(A) (2000); Federal Acquisition Regulation (FAR) sect. 15.506(a)(1).
[36] Although our regulations allow a procuring agency 30 days to provide relevant documents to the protester, see 4 C.F.R. sect. 21.3(d) (2008), the Air Force provided many relevant, core documents to Boeing and Northrop Grumman within days of the filing of the initial protest. The Air Force, however, continued to produce relevant documents even after the filing of its agency report and up to the date of the hearing conducted in this protest, which resulted in Boeing filing a series of supplemental protests.
[37] Although we have not sustained all of Boeing's protest allegations, nor do we address them all in this decision, we considered them all, which required substantial development of the issues during the protest.
[38] Our document production rules are much narrower than other federal discovery rules, such as the Federal Rules of Civil Procedure (FRCP), which permits litigants to seek the existence of documents that are reasonably calculated to lead to the discovery of admissible evidence. See, e.g., FRCP Rule 26(b)(1). In contrast, our regulations provide for the production of relevant documents. See 4 C.F.R. sect. 21.3(d).
[39] Although we denied the Air Force's request for documents from Boeing, we also informed the agency that if, during the further development of the case, the agency became aware of a specific relevant document, or documents, that only the protester possesses, the agency was permitted to request that document or documents. No such request was made.
[40] We will more fully address below certain of the agency's and intervenor's dismissal requests, such as the arguments concerning the evaluation of Northrop Grumman's proposal with respect to the fuel offload versus unrefueled radius range.
[41] To require an offeror to file a protest each time during a procurement that it is advised of an evaluation judgment with which it disagrees or believes is inconsistent with the RFP would not be consistent with the regulatory requirement that such protests can only be filed after a required debriefing. The objective of this regulation is to avoid the filing of defensive protests out of fear that our Office may dismiss the protests as untimely and the associated potential to unnecessarily disrupt procurements.
[42] As noted above, the RFP provided that KPP requirements were more important than KSA requirements, which were in turn more important than non'KPP/KSA requirements.
[43] As discussed above, the RFP indicated that KPP thresholds were minimum, mandatory requirements that must be satisfied and that the remaining requirements, including KPP objectives and KSA thresholds and objectives, were desired functions or characteristics that the firms could choose to offer.
[44] The PAR combined the (1) and (2) major discriminators into one major discriminator, although they actually concern separate KPP No. 1 objectives.
[45] Although the Air Force credited Northrop Grumman with exceeding to a greater degree than Boeing the KPP No. 2 objective related to fuel offload versus unrefueled range, we find, as discussed below, that this was inappropriate, given that the RFP provided that no additional credit would be provided for exceeding KPP objectives. The evaluation record thus shows that, instead of this being a discriminator, Northrop Grumman and Boeing should have received equal credit for satisfying this KPP objective.
[46] While we here illustrate this issue in our discussion of the aerial refueling area, as further illustrated below, the agency's failure to account for the relative weights given the various SRD requirements or consider the RFP's request that offerors propose to satisfy as many of the trade space requirements as possible permeates the evaluation of the key system requirements subfactor.
[47] The Air Force recognizes that Boeing could have proposed an aerial refueling tanker based upon the larger 777 platform. See, e.g., Air Force's Memorandum of Law at 84 n.30.
[48] Thus, the Air Force's and Northrop Grumman's interpretation of this objective as unbounded would render the last sentence of section M.2.2.1.1.a meaningless, given that that sentence addresses the situation where unbounded credit will be given for exceeding a KPP threshold.
[49] The Air Force and Northrop Grumman argue that Boeing's interpretation is unreasonable because this would mean that meeting and exceeding the KPP threshold relating to fuel offload versus unrefueled radius range would necessarily approximate one another because both would be bounded by a single line on a chart, which would render the establishment of an objective meaningless. We do not agree. As Boeing notes, the establishment of KPP objectives was expressly for the purpose of limiting the KPP trade space available to offerors, and we find from our review of the entire record that Boeing had a reasonable basis for believing that this limitation--that no credit would be given for exceeding the KPP threshold amount--was consistent with real-world tanker operations. See Boeing's Comments at 16.
[50] Both the Air Force and Northrop Grumman cite our decision in PM Servs.
[51] The Air Force's mid-term briefings to Boeing and Northrop Grumman stated that [o]nly SRD KPP Threshold requirements must be met -- strengths may be awarded for greater capability, but not beyond Objective levels (if an Objective is stated). See AR, Tab 129, Mid-term Briefing to Boeing, at 17; Tab 199, Mid-Term Briefing for Northrop Grumman, at 17. The parties' pre-final proposal revision briefings did not include this language.
[52] Although the pre-award briefings provided to Boeing identified benefits associated with the firm's offer to exceed the fuel offload versus unrefueled radius range threshold, Boeing could not know until it received the source selection decision that the agency was actually providing additional credit for the degree to which the offerors were exceeding the fuel offload versus unrefueled radius range.
[53] Even were we to consider the limitation on consideration above the KPP objective for the fuel offload versus unrefueled range requirement to be a latent ambiguity, Boeing's protest would still be timely. See Vitro Servs. Corp., B-233040,
[54] The capability of Northrop Grumman's proposed aircraft to satisfy this KPP threshold concerns a matter of technical acceptability; stated differently, if Northrop Grumman could not establish the capability of its aircraft to refuel all current fixed'wing tanker-compatible fixed wing aircraft using current Air Force procedures, its proposal could not be accepted. HT at 625, 649.
[55] In aerial refueling operations, tankers maneuver to a rendezvous point and establish an orbit pattern at a constant airspeed to await receiver aircraft. See, e.g., AR, Tab 289, Flight Manual, KC-10A Aircraft, Flight Crew Tanker Air Refueling Procedures, USAF Series, Technical Order (T.O.) 1-1C-1-33, Sept. 1, 2002, as revised Jan. 31, 2005, at 2-2, 2-15. If a receiver aircraft overruns the tanker during the final phase of rendezvous, the tanker and receiver pilots are directed to adjust to specified overrun speeds, and after overtaking the receiver aircraft, the tanker will decelerate to a refueling airspeed.
[56] In the first EN to Northrop Grumman addressing that firm's aircraft overrun capability, the Air Force identified [Deleted] KIAS, as the required overrun speed for the [Deleted]. See AR, Tab 184, EN NPG'MC1-003, at 2. In the second EN to Northrop Grumman, the agency corrected this to [Deleted] KIAS, see id., EN NPG'MC1-003a, at 1, which reflects the overrun speed identified for the [Deleted] in the KC-135 flight manual. See Tab 289, Flight Manual KC-135 (Tanker) Flight Crew Air Refueling Procedures, Supp. III, T.O. 1-1C-1-3,
[57] Initially, Northrop Grumman informed the Air Force that the agency should change its current overrun procedures. See AR, Tab 184, Northrop Grumman Response to EN NPG'MC1-003, at 1-3. Thereafter, Northrop Grumman asserted that there was nothing in the RFP requirements that established airspeed limitations for specific aircraft in overrun situations.
[58] The Air Force and Northrop Grumman contend that Boeing's contention--that Northrop Grumman's [Deleted] causes [Deleted]''was untimely raised in Boeing's comments and must be dismissed. Air Force's Post'Hearing Comments at 18; Northrop Grumman's Post-Hearing Comments at 141. We disagree. In its second supplemental protest (filed within 10 days of receipt of the first production of documents), Boeing specifically challenged the Air Force's evaluation of Northrop Grumman's proposal to [Deleted], arguing that the Air Force never considered the feasibility of this extreme measure or its implications on the KC-30's ability to carry out the refueling mission. See Boeing's Second Supplemental Protest at 29-31. The arguments concerning the [Deleted] are thus within the scope of Boeing's timely protest.
[59] As noted above, our Office requested that the Air Force provide knowledgeable witnesses who could testify with respect to the previously identified hearing issues.
[60] FAA's regulations provide that the design dive speed of an aircraft be established so that the design cruise speed is no greater than 0.8 times the design dive speed. See 14 C.F.R. sect. 25.335(b) (2008).
[61] The record indicates that the evaluators did not consider during the procurement whether the [Deleted] during aerial refueling operations was a current Air Force procedure and how this may affect Northrop Grumman's proposed solution to satisfying the overrun speed requirements. Rather, these issues apparently were only considered in response to Boeing's protest allegations. See HT at 711.
[62] The flight manuals for the KC-135 and KC-10 both state that the [p]rocedures in this manual are mandatory and must be performed in the prescribed manner except where deviations are required in the interest of safety of flight. AR, Tab 289, Flight Manual KC-135 (Tanker) Flight Crew Air Refueling Procedures, Supp. III, T.O. 1'1C'1'3, Jan. 1, 1987, as revised Sept. 1, 2004, at i; Tab 289, Flight Manual, KC'10A Aircraft, Flight Crew Tanker Air Refueling Procedures, USAF Series, T.O. 1'1C-1-33, Sept. 1, 2002, as revised Jan. 31, 2005, at ii.
[63] In its post-hearing comments, Northrop Grumman argues that [Deleted]. See Northrop Grumman's Post'Hearing Comments at 141-43. We provide little weight to this post-procurement description of Northrop Grumman's proposed design, given that this argument seems inconsistent with Northrop Grumman's EN response and is not supported by statements of consultants or other knowledgeable sources, and it represents information that was not presented to the agency for its consideration during the procurement.
[64] In this regard, in response to cross examination, the SSET mission capability team chief testified that, although Northrop Grumman in its EN response was not very good at articulating what they were doing at the end there, okay, the evaluation team did not ask Northrop Grumman to clarify what it was proposing in its EN response. See HT at 664.
[65] Northrop Grumman provided to the Air Force with its EN response a FAA Type Certificate for the Airbus A330-200 and A330'300 series aircraft, which identified the maximum operating limit airspeed as 330 KIAS and the design diving speed as 365 KIAS. AR, Tab 184, Northrop Grumman Response to EN NPG-MC1-003a, attach., FAA Type Certificate Data Sheet No. A46NM, Rev. 10,
[66] We have been presented with no testimony, statements or documentation from any member of the SSET professing to understand the relationship of maximum operational airspeed and dive velocity or airspeed, or to otherwise support the agency's conclusion that the A330's certified design dive velocity indicated that the aircraft was capable of achieving that speed as a maximum operational airspeed.
[67] The SSET mission capability factor team chief also testified that Northrop Grumman's response indicated to the SSET that the KC-30 could achieve [Deleted] KIAS with both the [Deleted]. HT at 636. In this regard, Northrop Grumman's EN response contained a Note under case two ([Deleted]), which stated that the [Deleted]. AR, Tab 184, Northrop Grumman Response to EN NPG'MC1-003a, at 6. It is unexplained in what situation this occurs, given that the [Deleted] is supposed to [Deleted]. Moreover, neither the Air Force nor Northrop Grumman has identified any evidence in the contemporaneous record of the agency's consideration of this note.
[68] Although the Air Force argues that it considered whether there was any schedule or cost risk associated with the proposed changes to Northrop Grumman's aircraft to satisfy the airspeed requirements, see Air Force's Post-Hearing Comments, at 18, we have not been directed to documentation in the record establishing that such an analysis was performed. Instead, the Air Force relies upon the testimony of the SSET mission capability factor team chief that the SSET concluded that Northrop Grumman had provided associated costs and schedule impact for the firm's proposed approach to satisfying the airspeed requirements. See HT at 629. However, he was unable to point to anything in the record to support his testimony, except his statement that a structural engineer on the SSET reviewed Northrop Grumman's EN response and determined that any required changes to the proposed aircraft could be accomplished within Northrop Grumman's proposed schedule. HT at 721. The totality of that review by the structural engineer, however, was apparently captured in an e-mail sent during the evaluation. See HT at 757-59, 783 (proffer by Air Force counsel). This e-mail does not establish that the structural engineer validated the capability of Northrop Grumman's aircraft to satisfy the overrun airspeed requirements or that changes in the aircraft's maximum operational airspeed could be achieved within the offeror's proposed schedule or cost. Rather, the e-mail states that the effect of higher airspeed on the integrity of KC-30 aircraft structure has not been analyzed and that [b]ottom line, these [Deleted] are major concerns that must be addressed by Analysis for sure and Flight Test if warranted. AR, Tab 332, E-mail 32002 re: EN NPG-MC1-003a, Jan. 25, 2008.
[69] The Air Force did not contemporaneously express any concern to Northrop Grumman with respect to its aircraft's ability to achieve breakaway speeds. HT at 619.
[70] Boeing argues, citing the procedures identified in Northrop Grumman's final proposal, that [Deleted] Northrop Grumman's [Deleted] could take as long as [Deleted], which would require the tanker pilot to either accelerate beyond [Deleted] KIAS before [Deleted] or delay initiating the breakaway until after [Deleted]; Boeing contends that the Air Force did not assess these concerns. See Boeing's Post'Hearing Comments at 68, citing, AR, Tab 187, Northrop Grumman's Final Proposal Revision, vol. II, Mission Capability/Proposal Risk, Jan. 4, 2008, at II'SF116'16a.
[71] It is unclear from the record when the Air Force changed its evaluation of this KPP objective.
[72] As set forth above, the agency also identified a weakness for Northrop Grumman in the aerial refueling area related to the firm's boom approach. Because the record did not contain any documentation explaining why the Air Force's evaluated concern with Northrop Grumman's proposed boom design represents a low risk as to schedule or cost, we also identified this as an area in which hearing testimony would be required to explain why evaluated weaknesses in Northrop Grumman's boom have low schedule or cost risk. See GAO Confirmation of Hearing,
[73] Organic maintenance refers to maintenance that the agency does for itself as opposed to maintenance provided by the contractor. See HT at 1215.
[74] Unlike Northrop Grumman, Boeing committed to providing the required planning and support services within the specified 2-year timeframe. HT at 1221.
[75] The agency's product support subfactor team chief testified regarding this requirement:
the idea behind that is to support the government in standing up this capability, so their approach would have to include the planning and support, the planning part being those type of actionable steps that support the type of things they would support us within that time constraint.
HT at 1215.
[76] The SSET product support subfactor team chief stated that the pre-final proposal revision briefing slide erroneously did not also refer to the SDD SOW, in addition to the production SOW, and that both SOWs would be implicated by Northrop Grumman's failure to commit to providing these services within the required 2-year timeframe. See HT at 1266-67.
[77] Similarly, the SSET's product support subfactor team chief doggedly insisted that Northrop Grumman's failure to agree to perform the required services within the specified time frame was merely an oversight, even where he admitted under cross examination that Northrop [Grumman] didn't forget about this issue, that Northrop Grumman's [final proposal revision] was not silent on the issue, and that in fact Northrop Grumman did consider the issue; they just decided not to address it in their [final proposal revision]. See HT at 1274-76.
[78] Northrop Grumman does not explain why it made the intentional choice not to specifically include the 2-year requirement in the contractual SOW, even though it was repeatedly requested to do so by the Air Force.
[79] In any event, the SSAC's and SSA's judgment that the firms' proposals were essentially equal under the product support subfactor is undermined by their erroneous conclusion that Northrop Grumman's failure to commit to the 2-year timeframe was an oversight.
[80] Thus, the Air Force essentially concedes that the conclusion in the source selection decision that Boeing's evaluated MPLCC was more than Northrop Grumman's was in error. [In preparing the public version of the protected decision, as the Air Force correctly points out, one of the five acknowledged errors was actually with respect to repair costs. The magnitude of these five errors remains unchanged.]
[81] The KC-30 is capable of carrying [Deleted] passengers, while the KC-767 can carry [Deleted] passengers. AR, Tab 55, PAR, at 18-19.
[82] The Air Force also suggests that the RFP directed that the MILCON portion of the MPLCC would be estimated entirely by the Air Force, with inputs from both Air Mobility Command (AMC) and Air Force Materiel Command (AFMC). Air Force's Post-Hearing Comments at 119; see Air Force's Memorandum of Law at 221. This argument is based upon section M.2.5.2.4 of the RFP, which informed offerors that the agency's assessment of MPLCC would include evaluating MILCON costs and further informed offerors: Note: Air Mobility Command and Air Force Materiel Command are estimating MILCON. This section does not, however, inform offerors that the Air Force would not consider their proposals in preparing this estimate.
[83] The test base was not included in the agency's MILCON cost evaluation.
[84] The SDD aircraft are provided to the Air Force under a cost reimbursement line item.
[85] A
[86] We have in a number of decisions explained the relationship between probable cost adjustments and proposal risk, but in no case have we found that an agency's adjustment of an offeror's proposed costs of performing a contract should be based only upon risk, and not upon a reasoned assessment of the realism of the proposed costs being adjusted. Thus, for example, we have recognized that an agency is not required to upwardly adjust an offeror's proposed costs which the agency found realistic, even where the agency also assessed some risk with regard to those costs. See, e.g., ITT Indus., Inc., B-294389 et al., Oct. 20, 2004, 2004 CPD para. 222 at 15-16; Vinnell Corp., B-270793, B-270793.2, Apr. 24, 1996, 96-1 CPD para. 271 at 6. Conversely, an agency may both make cost realism adjustments and assign proposal risk, where the cost adjustments are necessary to reflect the offeror's probable costs of performance based on its proposal, and that there continued to be proposal risk despite the cost adjustment. See Raytheon Co., B'291449,
[87] Similarly, we do not see any validity to using the overall cost growth associated with the Boeing's Multi-Mission Maritime Aircraft contract with the Navy to forecast cost growth associated with Boeing's SDD non'recurring engineering costs. In addition, Boeing asserts, without rebuttal, that the cost growth under that contract was due to reasons unrelated to non'recurring engineering costs. See Boeing's Protest at 85.
[88] The Air Force argues that Boeing is not prejudiced by these errors because the SSA in [the SSA's] selection decision stated that [the SSA] would have selected Northrop Grumman's proposal for award even if Boeing's proposed cost/price had not been adjusted upward by the Government and Boeing's cost/price risk rating for SDD had been rated as LOW. AR, Tab 54, Source Selection Decision, at 19-20. We disagree. As concluded above, the Air Force erred in the evaluation of technical proposals and the conduct of discussions and this statement by the SSA does not address any of those errors. In any event, this statement by the SSA, which is unsupported by specific analysis, would not seem to reflect the reasoned consideration of cost or price to the government that a selection official is required to provide in performing a trade-off analysis. See, e.g., Shumaker Trucking and Excavating Contractors, Inc., B-290732,
[89] The Air Force's evaluation of the fuel costs associated with the firms' proposed aircraft has been the subject of much argument and hearing testimony, and the record indicates that the agency did not do much more than an assessment that the offerors' own proposed fuel burn rates (gallons of fuel burned per hour) was reasonable. The record also shows, however, that even a small increase in the amount of fuel that is burned per hour by a particular aircraft would have a dramatic impact on the overall fuel costs (for example, Boeing notes that even a [Deleted]'percent increase in the amount of fuel per hour that is burned by the KC-30 would result in a $[Deleted] million increase in Northrop Grumman's life cycle costs for fuel, see Boeing's Post-Hearing Comments, at 139). Given our recommendation below that the Air Force reevaluate proposals and obtain revised proposals, this is another matter that the agency may wish to review to ascertain whether a more detailed analysis of the fuel costs is appropriate.
[90] Our Office will not sustain a protest unless the protester demonstrates a reasonable possibility of prejudice, that is, unless the protester demonstrates that, but for the agency's actions, it would have had a substantial chance of receiving the award. See McDonald Bradley, B-270126,