Raytheon Company, B-291449, January 7, 2003 * REDACTED DECISION
B-291449, Jan 7, 2003
A FAB-T is a satellite communications vehicle consisting of an antenna that transmits and receives radio signals to and from communications satellites. The FAB-T will have: a common design approach with common interfaces that work across multiple platforms. Which is to provide the FAB-T architecture and satisfy the TRD. The RFP established a split proposal submission schedule under which cost proposals were submitted last. The first two factors were of equal importance and each was more important than cost. Proposals were to be rated under the mission capability/proposal risk factor with a color rating and a proposal risk rating. Was to be assigned under each subfactor to reflect the assessed strengths and inadequacies of each proposal.
Raytheon Company, B-291449, January 7, 2003 * REDACTED DECISION
Raytheon Company protests an award to The Boeing Company under request for proposals (RFP) No. F19628-01-R-0033, issued by the Department of the Air Force, Air Force Materiel Command, Hanscom Air Force Base, Massachusetts, for the development of the Family of Advanced Beyond-line-of-sight Terminals (FAB-T) and related tasks. Raytheon protests the agency's technical evaluation and source selection decision.
We deny the protest.
Currently, the Air Force has over 100 different types of satellite communication terminals, each of which requires specially trained personnel to operate and maintain it; the Air Force states that it cannot afford to continue that communications approach and seeks to replace it with the FAB-T program. Agency Report, Tab 5E, Technical Requirements Document (TRD), at 1. A FAB-T is a satellite communications vehicle consisting of an antenna that transmits and receives radio signals to and from communications satellites, a modem/processor that provides primary computer processing resources, operator interface devices that provide the operator with the capabilities to command and monitor the terminal operation, and supporting equipment. Hearing Transcript (Tr.) at 7-10; Agency Report, Tab 5E, TRD, at 4. Terminals can be located on the ground or on aircraft. The FAB-T will have: a common design approach with common interfaces that work across multiple platforms, multiple applications, and for all branches of the U.S. military; multiple configurations available to meet various user needs; and an architecture that can be modified (or extended) to incorporate technology/capability upgrades. Agency Report, Tab 5E, TRD, at 1.
The FAB-T program has an evolutionary acquisition approach. The present RFP addresses the first increment, which is to provide the FAB-T architecture and satisfy the TRD. The RFP requires prototype platforms to be developed for five airborne platforms, i.e., B-2, B-52, E-4B, E-6B, and RC-135, as well as for two ground platforms, i.e., ground-fixed and ground-transportable command posts. This acquisition does not include the production and delivery of FAB-T equipment.
The RFP, issued March 27, 2002, contemplated the award of a cost-plus-award-fee contract (with cost-plus-fixed-fee and fixed-price components and options) for a period of approximately 6 years. The RFP established a split proposal submission schedule under which cost proposals were submitted last, after the agency had evaluated technical proposals. Award would be on a "best value" basis considering three evaluation factors: (1) mission capability/proposal risk, (2) past performance and (3) cost. The first two factors were of equal importance and each was more important than cost. Mission capability/proposal risk had two subfactors with subfactor 1--architecture and system performance--more important than subfactor 2--integrated processes. /1/
The RFP evaluation plan under the mission capability/proposal risk factor provides for evaluating proposals based on the extent that they address, meet or exceed the statement of objectives (SOO) for the FAB-T program, /2/ the threshold requirements stated in the TRD, and additional non-mandatory "objective requirements" (enhancements) in support of future communications capability identified in the TRD. The RFP included a case study to evaluate proposals under the architecture and system performance subfactor to assess such matters as the proposal's flexibility to accommodate future changes and upgrades, and the ability to provide engineering development models for the B-2 and the ground command post terminals by February and October 2006, respectively.
Proposals were to be rated under the mission capability/proposal risk factor with a color rating and a proposal risk rating. A color rating--blue (exceptional), green (acceptable), yellow (marginal) or red (unacceptable)--was to be assigned under each subfactor to reflect the assessed strengths and inadequacies of each proposal. A proposal risk rating--low, moderate or high--was to be assigned under each subfactor to reflect the risks and weaknesses associated with a proposed approach. The evaluation of proposal risk assessed the potential for disruption of schedule, increased cost, degradation of performance, the need for increased government oversight, the likelihood of unsuccessful contract performance, and any proposed approach to mitigate risk and whether the mitigation approach is or is not manageable.
The agency received proposals from Raytheon and Boeing, and conducted multiple rounds of evaluations and discussions. The source selection authority (SSA) ultimately determined that the two proposals were essentially equivalent under the past performance and cost factors. In this regard, both Raytheon and Boeing received a "confidence" past performance rating, and the difference in evaluated costs was much less than 1 percent on a $300 million contract. The source selection decision determined that consideration of the Raytheon and Boeing proposals under these two factors did not provide a basis for selecting one proposal over another. Accordingly, the selection decision rested entirely on the evaluation of proposals under the mission capability/proposal risk factor. Agency Report, Tab 5B, Source Selection Decision, at 3-5. The initial and final ratings for the subfactors of the mission capability/proposal risk factor are stated in the following table:
. Raytheon Boeing . Initial Final Initial Final
Architecture & Blue Blue Blue Blue Performance Low Risk Moderate High Low Risk /3/ . Risk Risk
Integrated Green Green Green Green Processes Moderate Moderate High Moderate . Risk Risk Risk Risk
Because its reasonableness is the key issue in this protest, we reproduce here a long excerpt from the source selection decision:
Architecture & System Performance Subfactor
Both offerors not only met the threshold requirements, but also proposed meeting all 11 of the high priority objectives. Additionally, both offerors proposed meeting a significant number of the 64 remaining objectives and both offered additional capability beyond that. Significant positive consideration was given to each offeror for this added capability, and both proposals were rated as blue.
Both Offerors proposed architectures based on the Software Communication Architecture (SCA) version 2.2. While applying the SCA outside the low-frequency domain of the Joint Tactical Radio System (JTRS)[/4/] for which the SCA was developed has inherent risk for both Offerors, this risk is consistent with the overall program risk . . . . Boeing's software approach is based on their SCA infrastructure developed during the JTRS risk reduction phase, and includes [extremely high frequency (EHF)] and terminal control software components developed during their internal FAB-T risk reduction effort. These software components were developed specifically for operation with the SCA environment and promise standardized implementation and simplified upgrades. While Raytheon also proposed compliance with SCA and a standard implementation[,] their proposal does not assure the simplified upgrades that the Boeing approach offers. I believe Boeing's architecture is more consistent with the FAB-T [SOO], as set out in the solicitation, which call for the development of a Family of Terminals that can be easily upgraded or expanded to incorporate additional communications capabilities . . . and the provision of a layered, open system architecture that will provide the foundation to accommodate future increments in a cost effective and timely manner. Although both offerors have experience with SCA through the JTRS risk reduction effort, I believe that the FAB-T program would likely benefit from Boeing's SCA approach which simplifies future upgrades and from the lessons learned gained from Boeing's role as the prime contractor for the Army's Cluster I JTRS program, which is the first [Department of Defense (DOD)] operational application of SCA.
Boeing's proposal contained one weakness in the Architecture and System Performance subfactor characterized by the SSET as "significant." It was their antenna group design for two of the seven EHF FAB-T platforms/installations, the B-52 and RC-135. The SSET was unable to substantiate the performance (gain and receiver sensitivity) of the baseline antenna design based on the information in the proposal, and felt there was risk in achieving the higher range of data rates needed for these two platforms. Boeing has recognized this risk, and as a risk mitigator has included [DELETED] in their proposal, with [DELETED] planned for [DELETED]. Based on consultation with the SSAC, I am convinced that there exists sufficient antenna expertise within the Boeing team (Boeing [and other team members]) that this issue could be resolved within the current schedule and with a minimal cost impact. The potential program risk is further mitigated in that neither the B[-]2 nor the ground Command Post Terminal Replacement, the two schedule drivers for this program, make use of this antenna. Based on this analysis, I believe this issue does not constitute a Significant Weakness, and that a Low Risk assessment is more appropriate from my perspective than the Moderate Risk as assigned by the SSET. . . .
Raytheon's proposal also contained a significant weakness in this subfactor. Raytheon's reuse of Ada code, designed for the different software architectures of legacy programs, will adversely influence the SCA-based FAB-T architecture and seems at odds with the expandability and upgradeability objectives of the FAB-T program. I believe this is indeed a significant weakness and will limit the ability to achieve the Family of Terminals objectives outlined in both the SOO and evaluation criteria. Further, I am convinced that the long-term viability of Ada and its industrial base is uncertain, and will likely adversely impact the upgradeability and supportability of Raytheon's FAB-T software.
Raytheon's proposal includes more objective requirements and enhancements to EHF functionality. While these additional features and objective requirements offered in Raytheon's proposal are attractive when compared to Boeing's approach, Raytheon's proposed adaptation of reuse code and continued reliance on Ada create long-term risks which outweigh the near term advantage of their EHF expertise. I concluded that, in the aggregate, the Boeing proposal strengths were superior and the weaknesses less disconcerting in this subfactor than those found in the Raytheon proposal.
Integrated Processes Subfactor
Both Boeing's and Raytheon's proposals were rated green in this subfactor. The Boeing proposal had two strengths: providing [DELETED] which enhances future development and maintenance; and use of [DELETED]. The first strength promises additional flexibility in future upgrades and the second more efficient program management and thus the potential for greater cost control. The Boeing proposal had one weakness relating to test simulation. The weakness is readily correctable and a minor adjustment was made to the government[']s most probable cost estimate.
The Raytheon proposal had no strengths in this subfactor and one weakness which was an assessed overestimation of software productivity.
I have concluded that the strengths of the Boeing proposal in this subfactor offer greater benefit to the government. In addition, because FAB-T is a software-defined radio based on a new Software Communications Architecture, success of the program will depend on disciplined and repeatable processes for the software development and integration effort. Boeing's CMM level 5 [/5/] processes and software build plan increase my confidence in Boeing's ability to complete software development and integration.
Agency Report, Tab 5B, Source Selection Decision, at 3-5.
The SSA concluded that Boeing's proposal represented the best value and, on September 20, the agency awarded the contract to Boeing. Following a debriefing, Raytheon filed this protest.
Raytheon challenges the agency's evaluation of Boeing's proposed antenna design and of Raytheon's proposed reuse of Ada code, and challenges the source selection decision concerning the weight accorded Raytheon's proposal to provide more enhancements than Boeing's.
The evaluation of technical proposals is primarily the responsibility of the contracting agency; the agency is responsible for defining its needs and the best method of accommodating them and must bear the burden of any difficulties arising out of a defective evaluation. Microcosm, Inc., B-277326 et al., Sept. 30, 1997, 97-2 CPD Para. 133 at 4. In reviewing an agency's evaluation and source selection decision, we will not reevaluate the proposals; we will only review the evaluation to determine whether the evaluation was reasonable and consistent with the stated evaluation criteria, and with applicable procurement laws and regulations. Id.; Gemmo Impianti SpA, B-290427, Aug. 9, 2002, 2002 CPD Para. 146 at 3. A protester's disagreement with the agency's judgment is not sufficient to establish that the agency acted unreasonably. Microcosm, Inc., supra.
EVALUATION OF BOEING'S ANTENNA DESIGN
With regard to the evaluation of Boeing's antenna design, the protester alleges that Boeing's proposal did not meet the stated minimum requirements for transmit power and receive performance, and thus must be rated either red (unacceptable) or yellow (marginal) under the architecture and system performance subfactor. This allegation arises primarily from the SSET's assessment of risk associated with Boeing's proposed antenna design for the B-52 and RC-135 aircraft, where the SSET identified the following significant weakness:
Antenna group design approach may not meet TRD radiated power ([effective isotropic radiated power (EIRP)]) and sensitivity ([gain to noise temperature (G/T)]) requirements and will likely result in an inability to support all required data rates . . .
Agency Report, Tab 6A(5)c, SSET Briefing to SSAC, June 12, 2002, at 33.
The precise EIRP and G/T requirements, which are stated in a classified attachment to the RFP, are mandatory. Contracting Officer's Statement at 6; Tr. at 33, 359. Boeing's proposal specifically states that it will comply with these requirements. Boeing Proposal, Vol. IIA, Sec. 2.6, at 33. Indeed, the SSET did not find Boeing's proposal noncompliant with the TRD requirements. Tr. at 28, 35, 328, 364-65, 369. In this regard, preceding the above-quoted statement of the significant weakness, the SSET stated that no proposal inadequacies or deficiencies existed and that Boeing's proposal warranted a "blue" rating, which indicates that the proposal met and exceeded minimum performance or capability requirements, for this subfactor. Agency Report, Tab 6A(5)c, SSET Briefing to SSAC, June 12, 2002, at 30, 32. Our Office also performed an in camera review of the classified requirements, and of the corresponding classified portions of Boeing's proposal and the agency's evaluation. Based on our review, we find nothing in any part of Boeing's proposal that indicated an intent to take exception to the EIRP and G/T requirements or stated an inability to support all required data rates. Thus, considering the developmental nature of this procurement, the record does not support the protester's allegation of noncompliance with TRD requirements.
The protester alternatively alleges that the reduction of the risk rating under the architecture and system performance subfactor because of Boeing's proposed antenna design from high, as initially assessed by the SSET, to low, as assessed by the SSA, was unreasonable.
The SSET initially assessed the risk as high under this subfactor because of the identified significant weakness that Boeing's analysis of the design risks for the antennas for the B-52 and RC-135 platforms did not support the effects of the particular design features of this antenna on EIRP and G/T requirements, such that the SSET did not have confidence that Boeing's antenna design would achieve the higher range of data rates needed by the agency (in other words, there was a risk it would not). Tr. at 31-49. The significant weakness and high risk rating under this subfactor remained even after Boeing provided additional information addressing the weakness in response to discussions. Agency Report, SSET Briefings to SSAC, Tab 6A(5)b, Aug. 6, 2002, at 30, 33; Tab 6A(5)A, Aug. 26, 2002, at 30, 33.
In response to the agency's concerns expressed during discussions, Boeing also proposed a mitigation plan on July 18 and again in its final proposal revision on August 19. Protester's Hearing Exhibits, Tab 5, Boeing's Risk Mitigation Plan, at 1. The plan was to identify and develop [DELETED] with the development of the [DELETED], during the [DELETED] of the contract and [DELETED]. If, at the conclusion of this period, the [DELETED] does not satisfy the agency's concerns, the agency can proceed with [DELETED]. Essentially, Boeing would assign a [DELETED] to identify and develop [DELETED]; Boeing [DELETED] for this effort [DELETED]. Id. at 8-9; Tr. at 66-67.
Notwithstanding this mitigation plan, Boeing's antenna design for these two platforms was still considered to be a significant weakness, which was reported by the SSAC to the SSA, although the risk rating for this subfactor was lowered to moderate at the end of the evaluation and negotiation process in part because of the mitigation plan. Agency Report, Briefings to SSA, Tab 6A(6)d, June 25, 2002, at 30, 33; Tab 6A(6)c, July 10, 2002, at 33; Tab 6A(6)b, Aug. 7, 2002, at 30, 33; Tab 6A(6)a, Sept. 3, 2002, at 32, 37; Tab 5C, Proposal Analysis Report, Sept. 13, 2002, at 13; Tr. at 251-52.
The SSA lowered the risk rating under the architecture and system performance subfactor from moderate to low, and found that Boeing's antenna design problem did not represent a significant weakness but only a weakness. Agency Report, Tab 5B, Source Selection Decision, at 4. The protester notes that this rating change came late in the evaluation process, and not in close proximity to proposal revisions by Boeing, which indicates (in the protester's view) that the revised ratings are not reasonable. However, this type of late change in evaluation ratings was consistent with the SSA's responsibility and authority. Source selection officials have broad discretion in determining the manner and extent to which they will make use of technical and cost evaluation results, subject only to the tests of rationality and consistency with the evaluation criteria. KPMG Consulting LLP, B-290716, B-290716.2, Sep. 23, 2002, 2002 CPD Para. 196 at 13; A & W Maint. Servs., Inc.--Recon., B-255711.2, Jan. 17, 1995, 95-1 CPD Para. 24 at 4. Here, as explained below, the SSA, in changing the risk rating for Boeing's proposal, reasonably exercised and explained his judgment consistent with the evaluation criteria.
Upon receiving the final evaluation results, the SSA considered the proposals to be very close, the closest competition he has ever presided over. Tr. at 196. Thus, the SSA provided feedback to the SSAC and the SSET chairperson after they presented him briefings on the evaluation of proposals. Prior to the SSA's feedback, the risk evaluation focused primarily on Boeing's technical approach with regard to this antenna design issue; however, the SSA considered other aspects, including Boeing's proposed mitigation plan that offered and [DELETED] antenna design, the potential impact on schedule and cost if there was a problem in Boeing's baseline antenna design, and the relative importance of the antenna to the overall procurement. Tr. at 172-76, 202-03, 217-18, 259-60; Agency Report, Tab 5B, Source Selection Decision, at 4.
The SSA considered that the antenna design from which the risk arose applied only to two of seven platforms. The due date scheduled for completion of these two platforms was late in 2006, and their development was not on the critical path of the procurement schedule. Rather, the RFP identified other platforms as critical and with shorter schedules, i.e., the B-2 and ground command post terminals. See RFP Sec. M.2.2.1. The SSA thus found that any performance problems associated with the design of the B-52 and RC-135 antennas would not adversely affect performance for the critical path. Tr. at 173, 202-03, 217-19, 259-60; Agency Report, Tab 5B, Source Selection Decision, at 4.
Moreover, the costs associated with those platforms was found to be a small fraction of the overall program cost, and there also were adequate costs allocated to support the [DELETED] contemplated by the mitigation plan. Tr. at 173, 202-03. Furthermore, the SSA found that Boeing had the capability to overcome the design risk and successfully complete the design of the proposed baseline antenna, notwithstanding its proposal weakness in this area, and the SSA expected that outcome. /6/ Tr. at 217-19, 241-46.
Thus, when viewed in the overall scope of the procurement, the SSA found the technical risk associated with the baseline antenna approach to be narrow, isolated from the critical path, and not of great concern to the overall cost and success of the contract. Tr. at 172-76, 202-03, 217-19, 241-46, 259-60; Agency Report, Tab 5B, Source Selection Decision, at 4. Considering that antenna design was but one aspect of the architecture and system performance subfactor, the developmental nature of this procurement, and the fact that the SSA's rationale for changing the risk evaluation is apparent in the source selection decision, we find this rating change to be within the discretion of the SSA and reasonable. See KPMG Consulting LLP, supra, at 13-14. As indicated above, this type of analysis by the SSA, giving due consideration to the evaluation conclusions of the lower-level evaluators, was entirely appropriate and reasonable. See GTE Hawaiian Tel. Co., Inc., B-276487.2, June 30, 1997, 97-2 CPD Para. 21 at 18-19.
The protester alleges, however, that Boeing's mitigation plan did not propose anything of substance that could reasonably reduce the risk associated with the antenna design. The record shows that, although the SSET technical evaluators did not believe that the plan addressed the risk, the higher-level evaluators did. Protester's Hearing Exhibits, Tab 6, Draft Evaluation Comments, at 2-3; Tr. at 156-58, 162-63, 251-53, 258-65. That is, as indicated above, the SSET evaluators focused on technical concerns that the baseline approach might not succeed, and since the mitigation plan did nothing to make the baseline plan more likely to succeed, the evaluators did not change the high risk rating. Tr. at 68, 156-58. However, as discussed above, the SSA and other high-level selection officials looked at other considerations, such as the potential of the mitigation plan to address cost, schedule and performance risks in the event the baseline approach proved unworkable. Agency Report, Tab 5C, Proposal Analysis Report, at 14. While Boeing's plan to [DELETED] did not identify any technical aspects [DELETED], the basis of this risk mitigation plan was not to [DELETED] for proposal evaluation purposes, but to have [DELETED] from which the agency could [DELETED]. As such, the risk mitigation plan addressed reducing risks associated with contract cost, schedule and performance in the event of failure of the baseline antenna approach. These considerations are consistent with the RFP evaluation plan, which stated that the evaluation of proposal risk would assess the potential for disruption of schedule, increased cost, degradation of performance, increased oversight, and unsuccessful contract performance. /7/ See RFP Sec. M.2.2.
Raytheon also alleges that an assessment by a non-governmental technical advisor from Massachusetts Institute of Technology (MIT)/Lincoln Labs of Boeing's antenna design was not reasonably considered by the SSA in making his source selection. /8/ In August, after the SSA had asked questions about the characterization of the antenna design as a significant weakness, the SSAC and SSET chairpersons retained the services of this advisor to assess Boeing's antenna design. Tr. at 64; Protester's Hearing Exhibits, Tab 3, MIT/Lincoln Labs' Notes. The advisor submitted handwritten notes to the SSET chairperson that identified concerns not at issue here, and also confirmed the design risk identified by the SSET. His notes also stated the following:
Boeing corporate experience certainly includes antenna designers capable of developing workable design. Unclear as to why that expertise has not been brought to bear in FAB-T.
Protester's Hearing Exhibits, Tab 3, MIT/Lincoln Labs' Notes at 1. The advisor went on to identify two likely methods for addressing the risk, one of which was "extensive rework of feed (almost invention class work)." Id. at 2. He concluded that either method he had identified "could and should be able to use same [basic] design" proposed by Boeing. Id. This analysis does not appear in the written record of the subsequent briefings to the SSA, nor did the SSA otherwise see the advisor's notes prior to this protest. Tr. at 174. However, the SSA had been told, prior to his selection decision, that the advisor had reviewed Boeing's antenna design, had confirmed the nature of the risk identified by the agency's evaluation, and had concluded that Boeing had the expertise to address the risk and make the design workable. Tr. at 216-19.
Raytheon essentially alleges that the advisor's notes express doubt about Boeing's ability to make its proposed design work, and that the SSA was not given an accurate summary of the advisor's analysis. We disagree. Although the SSA was not informed of the advisor's comment about Boeing's experience not having been brought to bear in its proposal, nor that the advisor characterized one of the methods of resolution as "invention class" work, he was told that the advisor corroborated the agency's evaluation of the risk and that the risk could be resolved. Tr. at 217-18, 331-32. The advisor's opinion was sought to determine whether the SSET's evaluation of the risk was reasonable, which it did; it was not sought to replace the agency's detailed evaluation of the risk, which was presented to the SSA. Tr. at 64-66, 160-61. Thus, contrary to the protester's arguments, we think that SSA was presented with an accurate assessment of the evaluated significant weakness and associated high risk rating with regard to Boeing's antenna design.
In sum, we find the SSA had a reasonable basis to change Boeing's risk rating under the architecture and system performance subfactor to low, and to regard Boeing's antenna design as something less than a significant weakness.
EVALUATION OF RAYTHEON'S PROPOSED REUSE OF ADA CODE
The protester alleges that the agency unreasonably evaluated Raytheon's proposed reuse of Ada computer software code as a significant weakness, which unreasonably caused its proposal to be rated a moderate risk under the architecture and system performance subfactor. The protester also challenges the agency's reclassification of this risk/weakness from the evaluation under the integrated processes subfactor to the architecture and system performance subfactor.
Raytheon proposed to reuse a large amount of code previously written for other applications (i.e., legacy code), instead of writing new code. Much of this reused legacy code was written using Ada computer language.
Initially, the SSET evaluated a significant risk in Raytheon's proposal under the integrated processes subfactor. This significant weakness was attributed to: (1) an unrealistic productivity rate for writing software; (2) reuse of software from another procurement that the agency believed was being developed concurrently with the FAB-T procurement, so that Raytheon's proposal was dependent on the success of the concurrent procurement; and (3) reuse of multiple programs with multiple languages/programming environments that presented supportability and upgrade risks, which could affect life cycle costs. Agency Report, Tab 6A(5)c, SSET Briefing to SSAC (June 12, 2002), at 68. Taken together, the agency associated these concerns with Raytheon's management processes and scheduling, and thus evaluated the risk under the integrated processes subfactor. Tr. at 79-83. However, as discussions were conducted, the first concern decreased, and the second was eliminated based on schedule clarifications. Tr. at 84-85. The third concern remained and, as a result of discussions, the SSET began to better understand the risk associated with reusing multiple programs with multiple languages for this particular application. This risk was closely tied to Raytheon's proposed use of the Software Communication Architecture (SCA) version 2.2 for the FAB-T system architecture.
DOD has developed the SCA over recent years, although the SCA was not a requirement of this RFP. The SCA is designed to permit a complex software system to evolve over time. It is a layered and evolvable approach in which components of the software framework can be developed independently and be incorporated easily into the framework. This approach requires clear definitions of interfaces between components to facilitate replacing or adding components that are developed over time to take advantage of evolving technologies. The initial defining of the interfaces is difficult and requires significant work up-front to develop. Tr. at 93-97.
Although both offerors stated that they planned to use the SCA, neither offeror's initial proposal addressed it in detail. Tr. at 91-93. Thus, as discussions progressed, the agency's subsequent rounds of evaluations reflected the agency's better understanding of the proposed SCA approaches. The evaluated risk of Raytheon's proposal to reuse multiple programs with multiple languages became associated with system performance, and thus with the architecture and system performance subfactor, rather than the integrated processes subfactor. Tr. at 87-93, 96-97. The agency's later evaluation of Raytheon's proposal reflects a concern regarding the performance risk associated with using large amounts of legacy software designed for applications other than the SCA. The concern was not that integration of legacy software could not be done, but rather that risks associated with cost and schedule increases arose, as well as a risk that the primary long-term advantage of using the SCA--the ease of replacing or adding software components in the future--would be compromised. Tr. at 90-91.
More specifically, the legacy software proposed for reuse by Raytheon was not developed for the SCA and could not be incorporated into the SCA without writing new portions of code to make the interface possible. /9/ This process of modifying software code to facilitate the interfacing of legacy software with the SCA is known as "wrapping." Tr. at 123-24, 191, 478-80. The risks associated with "wrapping" legacy software to function within the SCA include "legacy drag," which means that evolution of the system is slowed by the adaptations and compromises that have been made to make the legacy software work within the system. Every new development to the system may require modifications to the legacy applications or adaptations to the SCA environment. The agency believed that such modifications would likely be a step away from the SCA framework, and thus a loss of some of the long-term benefits of using the open architecture that is the SCA. Tr. at 109-24, 190-92, 479-80. This is a risk to system performance.
In addition, a significant portion of Raytheon's reused code was in the Ada computer language. The agency, and particularly the SSA, consider Ada to be a language in decline. Tr. at 187, 225-27. In light of this, the agency found risk that obtaining qualified personnel to maintain and revise this legacy code may become increasingly difficult during the 25-year life cycle of the FAB-T program. Tr. at 82-83, 91, 101, 225-27.
The risk of Raytheon's use of the reused Ada code was first evaluated as a weakness and moderate risk under the architecture and system subfactor by the SSAC in July. Agency Report, Tab 6A(6)c, SSAC Briefing to SSA (July 10, 2002), at 37, 41. The SSET thereafter affirmed that rating. Agency Report, Tab 6A(5)b, SSET Briefing to the SSAC (Aug. 6, 2002), at 37, 41. This was the evaluation presented to the SSA.
As indicated above, the SSA considered this to be a very close competition and sought additional information after reviewing the evaluation results. In this case, he further considered the software code reuse issue in Raytheon's proposal and sought a top-level review by the Carnegie Mellon Software Engineering Institute (SEI), a non-governmental advisor identified in the RFP. SEI confirmed that Raytheon's proposal presented a risk arising from substantial reuse of legacy code with the SCA and from use of non-mainstream programming languages, such as Ada, which result in significant risk to program cost, schedule and performance. Agency Report, Tab 6A(4)a, SEI Outbrief to SSA (Aug. 9, 2002).
Based on his review, with the concurrence of representatives of the SSET and SSAC, the SSA determined that Raytheon's reuse of Ada code represented a significant weakness under the architecture and system performance subfactor. /10/ Agency Report, Tab 5B, Source Selection Document, at 4; Tr. at 425-26.
We think that this evaluation was reasonable. The SSA's personal experience indicated to him that for the reasons detailed above the reuse of legacy code for this application was risky, even though not impossible. The SSA's experience also led him to the conclusion that the future supportability of the Ada computer language was a serious concern. Tr. at 177-83, 187-92. Although there is evidence that individual evaluators may have disagreed that Raytheon's proposal presented a "significant weakness" on either of these points, Tr. at 222-24, 469; Protester's Hearing Exhibits, Tabs 21-27, Point Paper and E-mails from MITRE Representatives Regarding the Raytheon's Ada Reuse Approach, /11/ these are matters that are predominantly professional judgments on which different authorities may reasonably have different opinions. Here, the SSA's judgment was based on his own experience and supported by the testimony of knowledgeable experts. Tr. at 235-36, 484-85. The protester has not shown that substantial reuse of legacy code prepared for applications other than the SCA does not carry with it the risk identified by the agency when integrated into an SCA system. Also, although the protester has presented evidence to show that some experts consider the future of Ada sound, the record before us shows that, at best, the future of Ada is hotly debated within the computer science industry; the existence of such a debate supports, rather than weakens, the SSA's determination, which we find reasonable.
Raytheon alleges that this evaluation is contrary to one of the stated objectives of the FAB-T program: to leverage commercial-off-the-shelf, government-off-the-shelf, non-developmental items, and best commercial practices, where possible. RFP, attach. 3, SOO, at 2. While we agree that this objective encourages appropriate use of legacy software, this objective must not be considered in isolation from the other SOO objectives. Other objectives in the SOO include developing a family of terminals that can easily be upgraded or expanded to incorporate additional communication capabilities, and providing a layered, open system architecture that will provide the foundation to accommodate future increments in a cost effective and timely manner. Id. at 1. The agency reasonably evaluated that Raytheon's selection of legacy software created a risk to open architecture system performance, ease of upgrading and expanding, and accommodating future increments in a timely manner. Tr. at 126-27. This evaluation is not inconsistent with the SOO.
The protester also alleges that the evaluation concerning the supportability of Ada legacy software did not reasonably consider Raytheon's risk mitigation proposal, which was a proposed contract clause under which Raytheon [DELETED] for Ada software for [DELETED] years at [DELETED] in order to ensure that the government will have [DELETED] "at a rate not to exceed what the Government would expect to pay to a third party supplier for the same or similar service." Raytheon Proposal, vol. IV (Aug. 19, 2002), ESC-H-RAY3, at 18; Protester's Hearing Exhibits, Tab 8, Raytheon Letter & Risk Mitigation Plan (Aug. 19, 2002) with attach., at 1, 6. While this clause guarantees that the agency will have at least one source available for [DELETED] regardless of the state of Ada within the industry, it does not specifically control the cost risk associated with those services; specifically, "what the Government would expect to pay to a third party supplier" could be high to reflect the scarcity of such services in the marketplace. This plan also does nothing to address the risk of legacy drag on the system or other aspects of the risk evaluated by the agency. Tr. at 106-08, 129-32.
The protester further alleges that, since (in contrast to Raytheon's specific plan) the risk mitigation offered by Boeing had little substance and did not address the evaluated risk in the proposed antenna approach, the agency treated the offerors unequally by reducing Boeing's evaluated risk and not similarly reducing Raytheon's evaluated risk. While we believe that both offerors' proposed mitigation plans were limited in scope and did not fully address the respective risks, we think that the agency reasonably found that the nature of the two risks was markedly different. As explained above, Boeing's risk was found to be limited to a portion of the requirements that were not part of the critical path and comprised only a fraction of the overall contract cost; Raytheon's risk was systemic in that the performance risk could result in adverse effects on the entire architecture. Also, the agency reasonably found that Boeing's risk would either be resolved in the first 6 months of the contract or the agency could choose from among [DELETED] other options in comparable stages of development so as not to adversely affect the schedule of performance; Raytheon's risk would be recurring for as long as wrapped code and corresponding architectural compromises remained in the system. Furthermore, the potential costs associated with Boeing's risk were found to be smaller and more quantifiable than were Raytheon's. Also, Boeing [DELETED] associated with its mitigation plan [DELETED]; Raytheon did not. /12/ was based on more than the proposed mitigation plan; it was based on the limited effect of the risk on overall contract performance, cost and schedule concerns. Raytheon's proposal risk was not so limited. We think the agency's actions were reasonable and did not constitute unequal treatment.
Finally, Raytheon alleges that the agency unreasonably shifted the evaluation of this risk from the less important subfactor, integrated processes, to the more important subfactor, architecture and system performance, and did so to prejudice Raytheon. We disagree. Although the agency initially viewed this risk as part of a management process risk and evaluated it under the applicable integrated processes subfactor, as discussed above, the nature of the risk shifted to one associated with the offeror's technical approach. This shift in the nature of the evaluated risk occurred as the agency better understood the proposal's impact on system performance and ability to upgrade the system after Raytheon presented details about its SCA framework and approach. The architecture and system performance subfactor is the appropriate subfactor for assessing this type of risk. RFP Sec. M.2.2.1; Tr. at 96-99, 269-70, 291. As for the allegation that the agency actions in this respect were with the intent to prejudice Raytheon, there is no evidence to support the allegation. /13/
THE SOURCE SELECTION DECISION
The protester alleges that the agency did not reasonably account for the "objective" or "additional requirements" (i.e., features attractive to the agency as identified in the RFP, but not mandatory) that Raytheon's proposal offered to provide over and above those offered in Boeing's proposal under the architecture and system performance subfactor.
Boeing proposed [DELETED] "objective requirements," of which the agency deemed [DELETED] significant, and [DELETED] "additional requirements," of which the agency deemed [DELETED] significant. Agency Report, Tab 5C, Proposal Analysis Report (Sept. 13, 2002), at 12-13. Raytheon proposed [DELETED] objective requirements, of which the agency deemed [DELETED] significant, and [DELETED] additional requirements, of which the agency deemed [DELETED] significant. Id. at 14-15.
The protester essentially contends that the sheer numerical advantage of the objective and additional requirements in Raytheon's proposal should require that it be rated higher than Boeing's proposal under the architecture and system performance subfactor, and thus Raytheon's proposal should be selected for award over Boeing's proposal.
However, the SSA clearly knew of Raytheon's numerical advantage in offered additional features and expressly addressed it in the source selection document, stating that although Raytheon's additional features were attractive when compared to Boeing's, Raytheon's proposed adaptation of reuse code and reliance on Ada create long-term risks that outweighed any advantage in Raytheon's more numerous additional features. The SSA favored Boeing's proposal under the subfactor, concluding that Boeing's strengths were superior and its weaknesses less disconcerting than Raytheon's. Tr. at 195-96, 206-13. Raytheon does not identify any specific additional features or group of features that would refute the SSA's judgment under this subfactor.
Moreover, under the integrated processes subfactor, the agency evaluated Boeing's proposal as having two strengths and one readily correctible weakness; Raytheon's had no strengths and one weakness. Boeing's strengths provided additional flexibility in future upgrades, and more efficient program management that had the potential for greater cost control. Raytheon's weakness was that the proposal overestimated software productivity. The SSA concluded that Boeing's strengths offered greater benefit to the government under this subfactor. Furthermore, since the success of this software-based satellite terminal would depend on software development and integration, Boeing's higher software certification (CMM level 5, the highest level possible, versus Raytheon's CMM level 3) provided the SSA with higher confidence in Boeing's ability to complete software development and integration. Tr. at 207-11. Raytheon has not challenged this determination. In sum, the SSA determined that Boeing's proposal was superior under both subfactors under the mission capability/proposal risk factor, the only evaluation criterion under which the proposals differed. The record supports the SSA's selection decision, and the protester has not shown it to be unreasonable or inconsistent with the RFP or applicable procurement law and regulation.
The protest is denied.
Anthony H. Gamboa General Counsel
1. The architecture and system performance subfactor generally concerns the technical aspects of the proposal and the integrated processes subfactor generally concerns management and quality control. Tr. at 77-79.
2. The SOO defines the top-level objectives for the FAB-T program. These objectives include developing a family of terminals that can easily be upgraded or expanded to incorporate additional communications capabilities; providing a layered, open system architecture that will provide the foundation to accommodate future increments in a cost effective and timely manner; minimizing the number of unique configurations; and leveraging commercial-off-the-shelf, government-off-the-shelf, non-developmental items and best commercial practices, where possible. RFP attach. 3, SOO, at 1-2.
3. Boeing's risk rating under this subfactor was first reduced from high to moderate by the source selection evaluation team (SSET). Agency Report, Tab 6A(5)a, SSET Briefing to Source Selection Advisory Council (SSAC) (Aug. 26, 2002), at 30. As discussed below, the SSA reduced this rating to low in the source selection decision. Agency Report, Tab 5B, Source Selection Decision, at 4.
4. JTRS is another acquisition program under which both Boeing and Raytheon participated. Tr. at 184-85, 288, 452-53.
5. The Capability Maturity Model (CMM) rating assesses a firm's ability to develop quality software. Tr. at 209. The RFP required a minimum rating of CMM level 3. RFP Sec. L-III-4.2.2.
6. Thus, contrary to the protester's assertion, the source selection decision considered Boeing's baseline antenna design.
7. The protester also alleges that the evaluation revisions are suspect because they followed a letter from Boeing questioning whether the agency had considered its proposed mitigation plan. See Protester's Hearing Exhibits, Tab 7, Letter from Boeing to the Air Force (Aug. 23, 2002). However, since this letter was not submitted by Boeing as part of the normal procurement correspondence, the agency did not consider it; it was not part of the materials evaluated and upon which the source selection decision was made. Tr. at 423.
8. The RFP stated that the agency might use this and other non-governmental advisors to review and analyze proposals. RFP at K-15.
9. Boeing also proposed to reuse code; however, Boeing's proposal did not contemplate reuse to the same extent as Raytheon's, and the code to be reused was developed for the SCA. Thus, Boeing's proposal did not incur the same risk as Raytheon's. Tr. at 127-28, 186-87, 199-200, 207, 213.
10. The SSA did not change the moderate risk rating under the architecture and system performance subfactor. Tr. at 430.
11. MITRE was also used by the agency to assist in the evaluation.
12. The agency adjusted Raytheon's proposed costs upwards to account for the agency's estimate of increased costs associated with the evaluated risk associated with the reuse of Ada code. The protester alleges that it was unreasonable to both increase the offeror's evaluated cost and assess a moderate risk to the proposal. We disagree. An agency is not prohibited from making cost realism adjustments and also downgrading a technical proposal, where, as here, the cost adjustments are necessary to reflect the offeror's probable costs of performance based on its proposal, and the weaknesses assessed in the offeror's technical evaluation reflect the performance risk stemming from the inadequacy of the technical proposal. See Basic Contracting Servs., Inc., B-284649, May 18, 2000, 2000 CPD Para. 120 at 12. Here, regardless of the cost adjustment to Raytheon's proposed costs, the risk evaluated to exist in Raytheon's proposed reuse of legacy code written in the Ada language includes performance and schedule risks, which were not accounted for in the upward cost adjustment.
13. The protester quotes vague statements of evaluators/technical advisors expressing disagreement with the evaluation of risk associated with reuse of legacy software and the Ada language. As discussed above, the evaluation of the risk was reasonable, even if individual evaluators did not agree with it. None of these statements indicated that the agency was making the evaluation decision it made with the express intent of either harming or favoring either offeror. While the wording of some of the comments from the lower-level evaluators leaves room for creative inference, it is the protester's burden to show more than inference or suspicion in alleging that source selection officials acted with intent to harm the protester, a burden the protester did not satisfy here. See E.F. Felt Co., Inc., B-289295, Feb. 6, 2002, 2002 CPD Para. 37 at 3-4.
* DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.