BAE Systems Information and Electronic Systems Integration Inc

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: BAE Systems Information and Electronic Systems Integration Inc

File: B-408565; B-408565.2; B-408565.3

Date: November 13, 2013

DIGEST

1. Protest is sustained where the agency failed to reasonably evaluate technical risk in accordance with the terms of the solicitation.

2. Protest challenging awardee’s technical evaluation is sustained where the documentary record is insufficient to permit meaningful review of the agency’s basis for eliminating multiple risks and weaknesses from the evaluation.

3. Protest challenging awardee’s corporate experience evaluation is sustained where the agency acknowledges that it improperly credited the awardee with outdated experience.

4. Although we conclude that the agency improperly distributed post-final proposal submissions from the awardee to its technical evaluators, where the submissions were not made for the purpose of revising the awardee’s proposal, and were not material to the agency’s evaluation of the awardee’s proposal, we conclude that the protester was not prejudiced by the agency’s error.

DECISION

BAE Systems Information and Electronic Systems Integration, Inc., of Hudson, New Hampshire, protests the award of a contract to Raytheon Company, of Waltham, Massachusetts, by the Department of the Navy, Naval Air Systems Command, under request for proposals (RFP) No. N00019-12-R-0035, for the technology development phase of the Navy’s next generation jammer (NGJ) program.[1]

We sustain the protest.

BACKGROUND

The NGJ program is for the development of an airborne electronic attack platform to replace the Navy’s existing AN/ALQ-99 tactical jamming systems pods on the EA-18 aircraft. Electronic attack is the use of electromagnetic, directed energy, or antiradiation weapons to attack with the intent of degrading, neutralizing, or destroying enemy combat capability.

The Navy is using an “evolutionary acquisition approach” for the NGJ program. According to this approach, the Navy will pursue its requirements in three increments. Increment 1 relates to mid-band frequency jamming capabilities, while increments 2 and 3 relate to low-band and high-band frequency jamming capabilities, respectively. The increment 1 acquisition approach involves four phases: technology maturation, technology development, engineering and manufacturing development, and production and development. The Navy began the program by awarding technology maturation contracts to four firms, including BAE and Raytheon, in July 2010. The technology maturation contracts were later extended and ran for a total of 33 months. Following the technology maturation contracts, the Navy anticipated competitively awarding a single cost-plus incentive-fee contract for the technology development effort, and later making a sole source award to the same contractor for the engineering and manufacturing development effort. A series of contracts for the production and development phase would then follow.

As stated above, this protest concerns the Navy’s competitive acquisition for the technology development effort. The Navy issued an RFP for the technology development contract on July 10, 2012. As expected, the RFP anticipated a single award. The RFP provided that the award would be made on a best-value basis considering four evaluation factors: technical, past performance, corporate experience, and cost. The RFP advised that the technical factor was the most important factor, and that past performance and corporate experience were of equal importance with each being more important than cost. Additionally, the three non-cost evaluation factors, when combined, were significantly more important than cost.

The RFP provided for two subfactors within the technical factor: design approach, and program and schedules. The RFP stated that between the subfactors, design approach was more important than program and schedules. Concerning the technical factor evaluation, the RFP explained that both a “technical rating” and a “technical risk rating” would be assigned to each proposal. According to the RFP the technical rating would represent an evaluation of the offeror’s “technical solution for meeting the Government’s requirement,” and would encompass “an assessment of compliance with the solicitation requirements and merit which considers the benefits and detriments related to program performance and operations.” Agency Report (AR), Tab A.3, RFP, at 145. In contrast, the technical risk rating would consider “the risk associated with the technical approach in meeting the requirement,” including assessments of “potential for disruption of schedule, increase in costs, degradation of performance, the need to increase Government oversight, or the likelihood of unsuccessful contract performance.” Id.

For the past performance factor and the corporate experience factor, the RFP stated that separate “performance confidence assessment ratings” would be assigned. Concerning past performance, the RFP stated that the evaluation would address the government’s level of confidence in the offeror’s ability to “successfully perform the required effort based on the Offeror’s (including principal and critical subcontractors’ and/or JV/SLE team members’) relevant past performance and systemic improvement.” Id. Concerning corporate experience, the RFP stated that the evaluation would “focus on the Offeror’s (including principal and critical subcontractors’ and/or JV/SLE team members’) recent and relevant corporate experience on programs with similar complexity as they relate to the tasks required to be performed in support of the solicitation requirements.” Id. at 146.

Finally, with regard to the cost factor, the RFP required offerors to propose costs for both the technology development phase and the engineering and manufacturing development phase, with the proposed cost for the technology development phase being more important than the cost for the engineering and manufacturing development phase. The RFP stated that the evaluated cost for each contract phase would be “either the Offeror’s proposed cost or the Government’s Most Probable Cost . . . whichever is higher.” Id. at 149.

The Navy received three proposals, including the proposals of BAE and Raytheon, on September 19, 2012. All three offerors were considered to be within the initial competitive range. Discussions began immediately after the competitive range determination and concluded on April 26, 2013. Final proposal revisions were due on May 3.

After the Navy completed its evaluation, BAE’s and Raytheon’s proposals were rated as follows:

AR, Tab O.2, BAE Debriefing, at 91. On July 8, the Navy awarded the technology development phase contract to Raytheon, and notified BAE that its proposal was not selected as the best value to the government. BAE received written debriefing slides on July 12, and a face-to-face debriefing on July 15. This protest followed.

DISCUSSION

BAE presents wide-ranging issues in its protest and supplemental protests of this award, including allegations that the Navy conducted an unreasonable, unequal, and inadequately documented technical evaluation; applied more lenient standards to Raytheon’s proposal regarding a variety of documentation and substantiation requirements; failed to evaluate the proposals in accordance with the RFP’s evaluation criteria; made multiple factual and mathematical errors in its interpretation of the proposals; unequally evaluated the proposals under the corporate experience factor; failed to conduct adequate discussions with BAE; conducted improper post-FPR discussions with Raytheon; and made unreasonable adjustments to BAE’s proposed cost. As detailed fully below, we agree with BAE’s protests in several areas. Specifically, we conclude that the Navy failed to evaluate risk in accordance with the RFP’s technical evaluation criteria, and inadequately documented its analysis of weaknesses and risks in several technical areas. Additionally, we conclude that the agency committed errors in its evaluation of the proposals under the corporate experience factor.

Technical Evaluation Errors

While BAE’s protests present a variety of allegations concerning the Navy’s technical and engineering judgments with regard to multiple aspects of its own, and Raytheon’s technical evaluations, our Office’s review of the record in this case reveals straightforward errors in the Navy’s application of the RFP’s stated evaluation scheme, and in its documentation of the evaluation record. First, our review of the record demonstrates that the Navy failed to conduct an assessment of proposal risks concerning the “Aeromechanical/Air Vehicle” sub-element of the design approach technical subfactor, which was required pursuant to the terms of the RFP. Second, our review has identified multiple technical areas in which a weakness or risk concerning Raytheon’s proposal was identified, was not resolved through discussions, but nonetheless disappeared from the evaluation without explanation.[2]

The evaluation of an offeror’s proposal is a matter within the agency’s discretion. VT Griffin Servs., Inc., B-299869.2, Nov. 10, 2008, 2008 CPD ¶ 219 at 4; IPlus, Inc., B-298020, B-298020.2, June 5, 2006, 2006 CPD ¶ 90 at 7, 13. In reviewing a protest of an agency’s evaluation of proposals, our Office will examine the record to determine whether the agency’s judgment was reasonable and consistent with the stated evaluation criteria and applicable procurement statutes and regulations. Shumaker Trucking & Excavating Contractors, Inc., B-290732, Sept. 25, 2002, 2002 CPD ¶ 169 at 3. While we will not substitute our judgment for that of the agency, we will sustain a protest where the agency’s conclusions are inconsistent with the solicitation’s evaluation criteria, undocumented, or not reasonably based. DRS ICAS, LLC, B-401852.4, B-401852.5, Sept. 8, 2010, 2010 CPD ¶ 261 at 4-5.

With regard to the Navy’s risk evaluation, as stated above, the RFP set forth a technical evaluation scheme under which the Navy was required to evaluate proposals for both technical merit and technical risk. As set forth in the RFP, the technical risk evaluation was to consider “the risk associated with the technical approach in meeting the requirement,” including assessments of “potential for disruption of schedule, increase in costs, degradation of performance, the need to increase Government oversight, or the likelihood of unsuccessful contract performance.” AR, Tab A.3, RFP, at 145.

Turning to the specific “Aeromechanical/Air Vehicle” sub-element evaluation, the RFP stated that, under the design approach evaluation criteria:

The Government will evaluate and assess the Offeror’s overall system approach and its ability to meet the requirements of this solicitation (i.e., System Performance Specification), as well as its approach to System Overview, Built-in-Test (BIT), Platform Integration, Aeromechanical/Air Vehicle, Open Systems Architecture, Supportability and IPS Management, Key System Performance Criteria, EIRP, Frequency and Spatial Coverage, Jamming Techniques, System and Assignment Management, and Size and Weight.

Id. at 146 (emphasis added). The RFP’s system performance specification (SPS), referenced above, also set forth multiple specific requirements related to the aeromechanical/air vehicle sub-element, including requirements related to aircraft stability and control, airworthiness, as well as a prohibition on modification of the EA-18 aircraft or aircraft systems.

In the evaluation of BAE’s and Raytheon’s proposals under the aeromechanical/air vehicle sub-element, the Navy evaluator raised numerous concerns applicable to both offerors’ designs. For example, the evaluator found that each offeror’s NGJ pod was [DELETED], and that:

[DELETED] incurs a risk that the flying qualities requirements will not be met (SPS 885), preventing certification throughout the carriage envelope (SPS 873), with the required store loadouts (SPS 800), unless modifications are made to flight control system control laws (which is disallowed by SPS 889).

AR Exhibit Book, Tab T.4, BAE Evaluator Worksheet, at 1; AR, Tab U.2.i.18, Raytheon Evaluator Worksheet, at 1.

However, the evaluator also found additional challenges concerning Raytheon’s proposed NGJ pod. For example, while the evaluator found that BAE’s pod was [DELETED], Raytheon’s proposed pod was [DELETED]. Id. In fact, the evaluator noted that for Raytheon’s proposed pod:

[DELETED]. The aerodynamic impacts of [DELETED] may make airworthiness certification [DELETED] more difficult [DELETED].

AR, Tab U.2.i.18, Raytheon Evaluator Worksheet, at 1. Further, while the evaluator noted that both proposed pods posed the risk of “flight envelope restrictions” in [DELETED], the evaluator concluded that the Raytheon pod design also posed “some risk [DELETED] because we have never [DELETED].” AR Exhibit Book, Tab T.4, BAE Evaluator Worksheet, at 2; AR, Tab U.2.i.18, Raytheon Evaluator Worksheet, at 2.

Notwithstanding the evaluator’s assessment of the above concerns with the offerors’ NGJ pod designs, in the final version of the evaluator’s worksheet the entire analysis reviewed above was struck from the evaluation. In explanation of the decision to remove the analysis, the evaluator wrote:

Weakness closed. [. . .] This is a risk that should be tracked by the program through [technology development] and [engineering and manufacturing development], but it would be nearly impossible to fully assess compliance with the requirements without a full [flying qualities] analysis program. As the program is currently planned, we will be running [computational fluid dynamics] in the [technology development] phase to get an early look at this risk. In the [engineering and manufacturing development] phase, we will be running wind tunnel tests followed by flight tests. This weakness is closed, but I recommend that the issue is tracked as a risk in the [technology development] phase.

Id.

Despite closing the weakness and removing the above analysis from the evaluation, the evaluator went on to acknowledge that Raytheon’s discussions response on the aeromechanical/air vehicle issues “does not provide additional data that eliminates this weakness.” AR, Tab U.2.i.18, Raytheon Evaluator Worksheet, at 2. In contrast, the evaluator noted that BAE’s proposal “has done the most they can do to minimize the aerodynamic impact of the pod, aside from [DELETED].” AR Exhibit Book, Tab T.4, BAE Evaluator Worksheet, at 2. Ultimately, however, the record shows that no comparative assessment of these risks or notice of any aeromechanical/air vehicle risks whatsoever was included in any report or briefing presented to the source selection authority (SSA) in this procurement.

In its protest, BAE asserts that the Navy’s actions in this regard demonstrate that the Navy failed to conduct a reasonable risk evaluation of the aeromechanical/air vehicle sub-element in accordance with the RFP. The Navy responds that its evaluation did assess risk under the aeromechanical/air vehicle sub-element, as demonstrated by its evaluator’s worksheet, but that the evaluator reasonably determined in the final review that the concerns no longer merit a weakness given the “early stage of the development program, thus the weakness was closed (removed), as is evidence from the evaluator worksheet language itself as well as its strike-through font.” Supplemental AR, Legal Memo, at 233.

According to the Navy, the RFP required only limited aeromechanical information concerning pod drag, pod loads, corrosion prevention, size, weight, center of gravity, and mass moment of inertia--and no requirement to show detailed analysis of aerodynamic interactions between the proposed pods and the aircraft. On the basis of this limited information, the Navy asserts that it reasonably determined that in the cases of both offerors’ flight risks, “the full extent of the risk in either design could not be fully characterized and assessed until during the execution of the [technology development] and [engineering and manufacturing development] phases of the program.” Id. at 234.

We disagree that the agency’s decision to defer consideration of aeromechanical/ air vehicle technical risk until contract performance was reasonable. The RFP required the Navy to conduct an analysis of the technical risks posed by the offers design approach for the purposes of awarding the technology development contract. It did not allow the Navy to defer analysis of the risks posed by the competing designs until after the award had been made. The Navy’s failure to consider these risks was inconsistent with the terms of the RFP.

Further, while it is understandable that definitive assessments of an offeror’s compliance with specific SPS requirements may be impossible due to limited information in the early stages of a development effort, assessing the impact of that ambiguity is the very purpose of a technical risk analysis. Conducting an assessment of the positive and negative possibilities raised by limited developmental data is at the very core of a reasonable risk analysis. Here, the Navy evaluators’ original analysis of the risks inherent in the aeromechanical/air vehicle sub-element plainly demonstrates that the Navy possessed sufficient information about the two proposed designs to identify areas of the offerors’ proposed pod designs that increased the “potential for disruption of schedule, increase in costs, degradation of performance, the need to increase Government oversight, or the likelihood of unsuccessful contract performance.” AR, Tab A.3, RFP, at 145. We therefore conclude that the Navy’s decision to essentially ignore the RFP’s required technical risk analysis for the aeromechanical/air vehicle sub-element was unreasonable, and represents a failure to follow the RFP’s stated evaluation criteria.

Next, our review of the record reveals that the agency failed to adequately document its resolution of weaknesses and risks in several areas of the technical evaluation. Specifically, under the platform integration sub-element and size and weight sub-element of the design approach subfactor, the evaluation documents contain risks and weaknesses that were attributed to Raytheon’s proposal, were not resolved, but nonetheless disappeared from the evaluation record without contemporaneous documentation. Although an agency is not required to retain every document generated during its evaluation of proposals, the agency’s evaluation must be sufficiently documented to allow our Office to review the merits of a protest. Apptis, Inc., B-299457 et al., May 23, 2007, 2008 CPD ¶ 49 at 10. Where an agency fails to document or retain evaluation materials, it bears the risk that there may not be adequate supporting rationale in the record for us to conclude that the agency had a reasonable basis for its source selection decision. Navistar Def., LLC; BAE Sys., Tactical Vehicle Sys. LP, B-401865 et al., Dec. 14, 2009, 2009 CPD ¶ 258 at 13.

As reviewed above, platform integration was among the elements for evaluation by the Navy under the RFP’s design approach evaluation criteria. As relevant, under the platform integration sub-element, the Navy was to assess the offerors’ pods’ compliance with safe stores separation and stores compatibility requirements of the SPS.[3]

In the evaluation of Raytheon’s proposed pod’s compliance with these requirements, the Navy evaluator recorded a weakness due to risks associated with [DELETED]. Specifically, the evaluator found that:

[DELETED] represents a risk to safe separation, particularly [DELETED]. The Offeror’s current analysis predicts [DELETED] which is a violation of [DELETED] safe separation acceptance criteria [DELETED].

AR, Tab U.2.i.13, Evaluator Worksheet, at 3. In discussions, Raytheon proposed to mitigate the violation by [DELETED]. However, the evaluator found that risks remained. Specifically, the evaluator concluded that:

[DELETED] will have to be evaluated and may increase the number of [DELETED] safe separation test points required during EMD. [DELETED]. Ground and flight test evaluation of [DELETED] may be beyond the scope of the government's notional EMD schedule contained in the SOO. The impact (e.g. on repeatability, wind tunnel correlation, mission representativeness, etc.) of [DELETED], is unknown and may complicate the safe separation evaluation.

Id. The evaluator also wrote that “[b]oth the original approach and the option of [DELETED] to ensure safe separation criteria of [DELETED] warrant weaknesses either individually or combined.” Id. The evaluator worksheet plainly shows that these concerns remained weaknesses after the conclusion of discussions. However, this assessment did not appear in the evaluation summary sheets, nor did the assessment appear in later evaluation documents presented to the SSA.

Similarly, under the size and weight sub-element of the design approach subfactor, the Navy evaluator recorded a weakness in Raytheon’s proposal where [DELETED]. Specifically, the evaluator stated that:

The Government assesses risk against the proposed design meeting the weight requirements of SPS-894 (2,377 lbs. threshold). The Government’s per pod weight estimate [DELETED] . . . is [DELETED] than the Offeror’s proposed NGJ per pod weight [DELETED].

AR, Tab U.2.i.47, Evaluator Worksheet, at 4. In this instance, the above weakness appeared in the “unrated summary sheet” for the evaluation area, in the form of a significant weakness stating that “[t]he Offeror’s proposed design is assessed to have a significant risk of being [DELETED] heavier per pod than the Offeror’s allocation, resulting in a total system weight that is [DELETED] above the required maximum weight of 2,377 lbs. (SPS-894).” AR, Tab U.2.iii.2, Unrated Worksheet, at 7.

However, in the Navy’s “rated summary worksheet” for this area--containing the signatures of two of the same evaluators as the unrated summary worksheet and signed on the same date as the unrated summary worksheet--the significant weakness was removed without any contemporaneous explanation. See AR, Tab U.2.ii.2, Rated Worksheet, at 13. As was the case with the platform integration sub-element, the concerns set forth in the size and weight sub-element evaluation did not appear in later evaluation documents presented to the SSA.

BAE asserts that there is no reasonable basis in the record for the disappearance of these documented risks and weaknesses from the evaluation record. The Navy now contends that these weaknesses did not appear in later evaluation documents because they were reasonably deemed insignificant. For example, the Navy argues that because Raytheon had proposed a risk mitigation for the stores separation risk--[DELETED]--that did not “[DELETED], the weakness was reasonably deemed not significant and therefore properly did not roll up to the [later evaluation documents].” Supplemental AR, Legal Memo, at 249.

Nothing in the contemporaneous record documents or supports the Navy’s contention that the above weaknesses were reviewed and found minor.[4] Specifically, concerning the safe stores separation risk, the Navy evaluator’s worksheet plainly demonstrates that Raytheon’s pod design posed risks even considering Raytheon’s proposed mitigation, and the evaluator specifically stated that “[b]oth the original approach and the option of [DELETED] to ensure safe separation criteria of [DELETED] warrant weaknesses either individually or combined.” AR, Tab U.2.i.13, Evaluator Worksheet, at 3. Concerning the size and weight sub-element weakness, that weakness was specifically documented as a significant weakness in the Navy’s unrated summary worksheet for the evaluation area, only to disappear in the rated summary worksheet--apparently signed by two of the same individuals on the same day. On the basis of the evaluation record here, we cannot conclude that the Navy reasonably decided that the above weaknesses and risks were minor. There is simply no contemporaneous explanation of the resolution of these risks and weaknesses in the record here. Accordingly, on the basis of this record, we are unable to conclude that these matters were reasonably considered, and set aside by the agency evaluators.

Additionally, with regard to the size and weight sub-element evaluation, the contemporaneous record supports BAE’s contention that the Navy failed to consider the weight impact of certain Raytheon design changes made in response to evaluation notice (EN) T-RTN-245A, shortly before the date for the submission of final proposal revisions. While the Navy directs our Office to multiple documents in the record that it asserts included consideration of these final changes to Raytheon’s design, all of the documents cited by the agency appear to consider information through EN T-RTN-246, dated March 22, and none include a citation to EN T-RTN-245A, which prompted the final design changes, and is dated April 23. Accordingly, on the basis of the record in this case, we cannot conclude that the Navy’s weight evaluation considered all elements of the actual final NGJ pod design proposed in this procurement.

The Navy argues that BAE was not prejudiced where both Raytheon and BAE had risks that were struck from the aeromechanical/air vehicle evaluation, and where BAE had significant risks under the size and weight sub-element, as well as other technical sub-elements, resulting in its technical high risk rating. Our decision here does not discuss the Navy’s evaluation of multiple technical areas that resulted in BAE’s high risk rating because we find no error in the agency’s evaluation of those areas; however, we note that the NGJ technical development proposals were, in essence, tradeoffs between power generation, cooling systems, jamming systems, and size and weight.

Concerning the aeromechanical/air vehicle evaluation, BAE, in its proposal, achieved [DELETED]. Thus, despite BAE’s overall higher risk rating, we nonetheless consider it prejudicial for the Navy to neutralize a technical evaluation area in which [DELETED].

We also find the Navy’s errors prejudicial where weaknesses and risks disappeared from Raytheon’s evaluation, and where, in the source selection decision, the SSA emphasized that there were no evaluated weaknesses in Raytheon’s proposal. Specifically, the SSA stated that:

BAE [has] significant weaknesses [DELETED], which further challenge [its] ability [DELETED]. Raytheon on the other hand, has no significant weaknesses [DELETED], and in fact has no significant weaknesses at all.

AR, Tab N.2, Source Selection Decision Document, at 2 (emphasis original). Finally, [DELETED]. Id., at 3.

Where [DELETED], and where a reasonable and adequately documented evaluation may have decreased the risk advantage of Raytheon’s proposal, we cannot say whether the SSA’s reconsideration of the proposals’ relative technical risks and merits would [DELETED]. In such circumstances, we resolve any doubts regarding prejudice in favor of a protester since a reasonable possibility of prejudice is a sufficient basis for sustaining a protest. See Kellogg, Brown & Root Servs., Inc.--Recon., B-309752.8, Dec. 20, 2007, 2008 CPD ¶ 84 at 5.

Corporate Experience

BAE challenges the Navy’s evaluation of the corporate experience factor, specifically, the Navy’s evaluation of three of the five corporate experience subfactors: hardware development, aircraft integration, and logistics support experience. [5] AR, Tab A.3, RFP, at 145, and Protest, at 87. As discussed below, the record reflects that the Navy erred in its evaluation under the aircraft integration sub-factor.

The aircraft integration experience sub-factor included four elements: (1) obtaining U.S. government flight clearances under the NAVAIR 4.0P process; (2) aircraft stores integration; (3) operations in a ship’s electro-magnetic environment; and (4) integration with onboard AEA or EW systems. With regard to the first element, obtaining U.S. government flight clearances under the NAVAIR 4.0P process, Raytheon received a rating of “extensive corporate experience” based on the Navy’s conclusion that Raytheon had [DELETED] years of substantiated experience with obtaining government flight clearances, whereas BAE received a rating of only “corporate experience” based on having substantiated only [DELETED] years of experience in obtaining flight clearances.

The Navy, however, concedes that its evaluation under this element was erroneous in two respects. First, the Navy acknowledges that it failed to credit BAE with [DELETED] experience for its work on the [DELETED]. The Navy evaluators found that BAE gained [DELETED] experience in obtaining flight clearances in the course of its work on the [DELETED]. AR, at 248. The Navy, however, inadvertently failed to credit this experience in the corporate experience rating summary sheet, and as a result, the SSEB Report did not include any reference to this experience. Id. Accordingly, BAE should have been credited with [DELETED], rather than [DELETED] years of experience under this element.

Second, the Navy admits that its evaluation of BAE and Raytheon under the element was unequal. In this regard, the record reflects that both BAE and Raytheon claimed experience associated with obtaining flight clearances under the NAVAIR 4.0P process for work performed before 2004, as well as work performed after 2004. As explained by the Navy, the evaluation team decided to use 2004 as a cut-off for recent experience in obtaining clearances because changes were made to the NAVAIR flight clearance process and policy in 2004 which, in the Navy’s view, rendered pre-2004 experience not relevant.[6] Supplemental AR, Tab Z, Evaluator Declaration, Aug. 17, 2013, at 6-7. Based on the 2004 cutoff, the Navy did not credit BAE with [DELETED] years of substantiated experience from [DELETED]-2004. AR, Tab L, SSEB Report, at 151. The record reflects, however, that the Navy credited Raytheon with experience which it gained prior to 2004. AR, at 246. According to the Navy, Raytheon should not have received credit for its pre-2004 experience.

The precise impact of this error on Raytheon’s evaluation under this element is unclear. The SSEB report indicates that Raytheon and its subcontractors substantiated “more than [DELETED] years of combined experience in all areas of aircraft integration.” AR, Tab L, SSEB Report, at 175. With respect to obtaining clearances, however, the SSEB report identifies only [DELETED] contracts accounting for a total of [DELETED] years of experience under this element--[DELETED]. Based on the cut-off established by the Navy, none of the work under the [DELETED] performed before 2004 should have been credited to Raytheon.

Thus, based on the contemporaneous record provided to our Office, Raytheon would be left with only [DELETED] years of experience, [DELETED], when accounting for the agency’s error. In an affidavit, however, the corporate experience team lead asserts that the Navy credited Raytheon with [DELETED] years of experience in obtaining government clearances, and that “[DELETED] years of that experience was obtained prior to 2004.” Supplemental AR, Tab Z, Evaluator Declaration, Aug. 17, 2013, at 8. Accordingly, to the extent Raytheon was initially credited with [DELETED] years of experience, and [DELETED] of those years were found to be in error, as the agency’s evaluator asserts, Raytheon would be left with only [DELETED] years of experience, [DELETED] less than BAE.[7] In any event, as explained above, the record reflects, and the agency acknowledges, that both BAE and Raytheon were misevaluated under the aircraft integration subfactor element “obtaining U.S. government flight clearances under the NAVAIR 4.0P process.”[8]

Raytheon “Head Start” Program

In their proposals in response to the RFP, both BAE and Raytheon proposed to undertake certain pre-award efforts in order to mitigate elements of risk under the technical factor, program and schedules sub-factor. BAE, in its proposal, set forth a “ready on day 1” program, under which it would [DELETED] prior to the time of contract award. Raytheon, in its proposal, set forth a “head start” program, under which it would begin [DELETED] prior to the award decision, and would [DELETED].

In response to Raytheon’s proposal, the Navy advised Raytheon that it would consider the head start program as a potential risk reducer, but requested that Raytheon provide [DELETED], and provide a bi-weekly status report to the contracting officer concerning [DELETED]. Raytheon submitted its first head start submission to the Navy on February 28, and its final bi-weekly status report on July 2--more than 2 months after the May 3 deadline for submission of FPRs. The record demonstrates that all of Raytheon’s head start reports were reviewed by multiple Navy procurement officials, including five reports received after the closing date of discussions/FPR deadline.

BAE argues that the review of Raytheon’s post-FPR head start submissions constituted improper post-FPR discussions or, at minimum, unequal exchanges that favored Raytheon. BAE also asserts that prejudice should be presumed under these circumstances, citing our Office’s decision in Radiation Oncology Group of WNY, PC, B-310354.2, B-310354.3, Sept. 18, 2008, 2009 CPD ¶ 136. The Navy responds that while procurement officials did review Raytheon’s post-FPR head start reports, the contents of the head start reports were not considered in Raytheon’s design approach subfactor evaluation and, with regard to the program and schedules sub-factors, were only assessed to determine whether Raytheon’s proposal [DELETED].

We agree with BAE that the Navy procurement officials’ review and consideration of the contents of Raytheon’s post-FPR head start status reports was improper. In fact, at least one of the Navy’s evaluation personnel agreed that the post-FPR head start reports should not have been reviewed. In an email chain discussing the head start reports, the evaluator wrote “I disagree [that we can consider the reports]. Unless it was information received on an existing contract, I believe it is considered proposal information received after FPR.” AR, Tab R.1.iii, Evaluator Email, May 8, 2013.

However, in this case, Raytheon’s continued submission of bi-weekly status reports did not constitute an effort to revise its proposal in any way. The contents of the status report did not expand on or provide additional detail relating to Raytheon’s technical approach, and our review of the record shows that no evaluation reports were altered as a result of the Navy’s review of post-FPR head start reports. Accordingly, we conclude that the Navy’s distribution of the reports to the evaluators, although in error, did not impact the competition.

We have considered BAE’s emphasis on Radiation Oncology Group of WNY, PC, supra, in support of its contention that prejudice should be presumed where post-FPR communications provide, and the agency considers, “additional supporting material regarding [the offeror’s] technical proposal.” Id. at 3. BAE argues that under these circumstances, “the agency bears the burden of affirmatively proving that it did not consider [post-FPR materials] in its evaluation,” which the Navy has failed to do. BAE Supplemental Comments at 13.

We disagree that Radiation Oncology is applicable here. In that protest, the awardee submitted late information explicitly revising its technical proposal by providing “additional quality/performance evaluations.” The agency in that case did not argue that the additional materials were not material to the evaluation, and the source selection decision was not sufficiently documented to establish the extent to which the late materials were considered in the technical evaluation.

In contrast to Radiation Oncology, the reports at issue here were not an attempt to make late revisions to Raytheon’s proposal, but were instead submissions required by the agency as a condition of its consideration of Raytheon’s timely-proposed head start program. Further, while it is clear from the record that multiple Navy evaluators reviewed the head start reports, BAE has not itself identified material technical data in any head start report, nor identified any technical evaluation document that may have been modified as a result of the head start reports. Rather, BAE alleges only that the status updates in the head start reports could have “provided assurance” to Navy evaluators concerning Raytheon’s design approach and program and schedules. Where the information provided in Raytheon’s head start reports was not intended to modify or revise Raytheon’s proposal, and has not been shown to be material to the technical evaluation, our Office’s decision Radiation Oncology is not applicable.

RECOMMENDATION

We recommend that the Navy document a reevaluation of the proposals that is reasonable and consistent with the RFP’s evaluation criteria. Upon completion of the reevaluation, the Navy should perform and document a new cost/technical tradeoff analysis with the rationale for any tradeoff made. If the Raytheon proposal is not found to represent the best value to the government in accordance with the RFP’s evaluation factors, the agency should terminate the award to Raytheon and award a new contract to the offeror representing the best value to the government.

We also recommend that BAE be reimbursed its costs of filing and pursuing the protest. Bid Protest Regulations, 4 C.F.R. § 21.8(d)(1) (2013). The protester’s certified claims for such costs, detailing the time expended and costs incurred, must be submitted to the agency within 60 days after receipt of this decision.

The protest is sustained.

Susan A. Poling
General Counsel