EFW Inc.

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of:  EFW Inc.

File:  B-412608; B-412608.2

Date:  April 7, 2016

DIGEST

1.  Protest alleging that the agency engaged in misleading discussions is sustained where the record shows that the agency framed its discussion questions in a manner that did not reflect the breadth of the agency’s true concerns.

2.  Protest challenging the agency’s past performance evaluation is sustained where the record shows that the evaluation mechanically relied on a crosswalk table that obscured differences between the proposals, and failed to look beyond overall adjectival ratings in concluding that the offeror’s past performance was equivalent.

3.  Protest challenging the agency’s price realism analysis is sustained where the agency unreasonably assigned the protester’s proposal a significant weakness for “realism” based on considerations not properly within the scope of price realism analysis.

EFW Inc., of Fort Worth, Texas, protests the award of an indefinite-delivery, indefinite-quantity (IDIQ), contract to DRS ICAS, LLC, of Beavercreek, Ohio, by the Department of the Navy, Space and Naval Warfare Systems Command, under request for proposals (RFP) No. N00039-15-R-0030, for development, integration, test, support, and installation services for a Link 11 Data Terminal and Link 22 Signal Processing Control solution for the Navy’s Command and Control Processor family of systems.[1]  EFW alleges that Navy conducted misleading discussions by failing to alert it to the actual area of the agency’s concerns.  EFW also alleges that the Navy’s evaluation was unreasonable with regard to past performance and cost/price, and that the best-value decision was flawed.

We sustain the protest.

BACKGROUND

The Navy issued the RFP on March 18, 2015, seeking a contractor to develop and support a Link 11 Data Terminal and Link 22 Signal Processing Control solution.  The agency anticipated the award of a single hybrid-type IDIQ contract, to include fixed-price incentive, fixed-price, cost-plus-fixed-fee, and cost-reimbursement contract line item numbers (CLINs), for a five year base period and five one-year option periods.  CLIN 0100 was a fixed-price incentive CLIN for development, integration, and testing of the proposed solution, and for the delivery of two test article units.  CLINs 0200, 0600, and 0700, were fixed-price CLINs for the post-development delivery of Link 11/Link 22 production units and associated installation and service equipment.  CLIN 0300 was a cost-plus-fixed-fee CLIN for ongoing technical studies, software engineering, and logistics support services.  CLIN 0400 was a cost-reimbursement CLIN for incidental other direct costs, and CLIN 0500 was a not-separately-priced CLIN for documentation and data rights. 

The contract was to be awarded on a best-value basis considering four non-cost/price evaluation factors and cost/price.  The RFP provided that, when combined, the non-cost/price factors were more important than cost/price.  The  non-cost/price evaluation factors, listed in descending order of importance, were as follows:  (1) prospective task order 0001 (PTO 0001), (2) technical approach to production and sustainment, (3) past performance, and (4) small business approach. 

As relevant, factor (1) concerned evaluation of the offeror’s approach to a prospective task order (PTO 0001) consisting of all development, integration, and testing, and the delivery of 2 engineering design models (test article units) (CLIN 0100); delivery of 16 production units and associated support (under CLIN 0200); and incidental contractor travel and materials (under CLIN 0400).  The RFP provided that factor (1) would be evaluated for the extent to which the offeror’s solution demonstrated technical maturity, with greater credit for solutions with less development work remaining, and greater credit for solutions that can be delivered more quickly.  Additionally, the evaluation was to consider “the extent to which the proposed cost is realistic for the work to be performed, reflects a clear understanding of the requirements, and is consistent with the unique methods of performance and materials described in the Offeror’s proposed solution.”  Agency Report (AR), Tab 2, Conformed RFP, at 69. 

As also relevant, the RFP provided that for factor (3), past performance, the offerors were to provide up to five relevant experience references for contracts performed in the past five years.  The RFP further advised that:

There are two aspects to the past performance evaluation.  The first is to evaluate the Offeror’s past performance to determine how relevant a recent effort accomplished by the Offeror is to the effort to be acquired through this source selection.  The second aspect of the past performance evaluation is to determine how well the contractor performed on prior experiences.  This quality assessment will be made on the basis of [Contractor Performance Assessment Reporting System] or Past Performance Questionnaires (PPQs) submitted with the proposal, though in accordance with [Federal Acquisition Regulation (FAR)] 15.305(a)(2), the Government may consider past performance information from any other appropriate source.  At the factor level, one performance confidence assessment rating will be assigned for each Offeror based on an evaluation of the merit of all individual experiences’ relevancy and quality combined.

Id.

Concerning cost/price, the RFP provided that the evaluation would consider the evaluated cost/price of PTO 0001 as well as the evaluated cost/price for the IDIQ contract generally, to include the base period and option periods.  As described above, PTO 0001 encompassed the price associated with all development, integration, testing, and 2 test article units (CLIN 0100), plus the price for 16 production units and associated support (under CLIN 0200), and incidental travel and materials costs (under CLIN 0300). 

The evaluated cost/price for the IDIQ contract generally again included the price for all development work and delivery of two test article units (CLIN 0100).  Additionally, the cost/price for the IDIQ contract considered extended unit prices for delivery of 82 production units during the base period and 100 production units during the options periods (CLIN 0200 and options); as well as labor costs for ongoing technical studies engineering services and logistics support in accordance with the agency’s estimate of annual level of effort during the base period and option periods (CLIN 0300 and options).  The evaluated cost/price for the IDIQ contract also included lump sum amounts to be entered by the offerors for incidental other direct costs (CLIN 0400 and options), and installation and service equipment (CLINs 0600, 0700, and options).

The RFP explained that, for both the PTO 0001 cost/price evaluation and the IDIQ cost/price evaluation, the agency would conduct a cost realism analysis to determine whether proposed cost elements are “realistic for the work to be performed; reflect a clear understanding of the requirements; and are consistent with the unique methods of performance and materials described in the offeror’s technical proposal.”  AR, Tab 2, Conformed RFP, at 70.  The RFP also provided that, for both PTO 0001 and the IDIQ contract, “Technical Price Realism Analysis will be performed by the Government for the Fixed Price Incentive CLIN [CLIN 0100] as described under the Technical evaluation Factor 1 above.”  Id.

Finally, the RFP provided “General Evaluation Principles,” apparently applicable to both the technical and cost/price evaluations.  Id. at 71.  Among these principles, the RFP provided that:

Proposals which are unrealistic in terms of technical or schedule commitments, or unrealistically high or low in terms of cost, may be deemed to be reflective of an inherent lack of technical competence, or indicative of a failure to comprehend the complexity and risks of the proposed work and may be grounds for rejection of the proposal.[2]

Id.

Both EFW and DRS submitted proposals in response to the RFP.  After review of the initial proposals, the agency established a competitive range of both offerors and conducted discussions.  Both offerors then submitted final proposal revisions (FPRs).  Following receipt of the FPRs, a source selection evaluation board (SSEB) evaluated the proposals under the non-cost/price factors, while a cost evaluation board (CEB) evaluated the cost/price.  At the conclusion of this evaluation, the offerors’ proposals were rated as follows:

	EFW	DRS
Factor (1) PTO 0001	Acceptable	Good
Factor (2) Technical Approach to Production and Sustainment	Outstanding	Outstanding
Factor (3) Past Performance	Satisfactory Confidence	Satisfactory Confidence
Factor (4) Small Business	Marginal	Acceptable
Technical Overall	Acceptable	Good
PTO 0001 Evaluated Cost/Price	$4,168,572	$3,322,420
IDIQ Evaluated Cost/Price	$24,902,101	$25,638,019

AR, Tab 9, SSEB Final Report, at 8, 18; Tab 10, CEB Final Report, at 7, 19. 

In the source selection decision document (SSDD), the source selection authority (SSA) concluded that the proposal submitted by DRS represented the best value to the government.  In reaching this conclusion, the SSA first noted that:

DRS received an overall technical rating of Good, whereas EFW received an Acceptable.  DRS’s ratings for factors 1 and 4 were higher than EFW, and for factors 2 and 3 were the same.  Accordingly, factors 2 and 3 will not be discussed further.

AR, Tab 4, SSDD, at 2.  The SSA then reviewed discriminators between the proposals under factors 1 and 4.  As relevant here, the primary discriminator under factor 1 was a significant weakness assessed against EFW’s proposal.  The SSA explained that:

EFW had a significant weakness related to the realism of its proposal verses its solution, which was viewed as High risk, and the strengths did not compensate for this risk.  The SSEB considered [EFW’s] total proposed [level of effort] excessive, unrealistic and inconsistent with the amount of development remaining for EFW’s solution.

Id.  Additionally, the SSA identified significant concerns with EFW’s proposed cost/price.  With respect to the overall IDIQ, the SSA noted that EFW’s cost/price was 2.9 percent lower than that proposed by DRS.  Id.  However, the SSA also noted that, although both offerors’ IDIQ costs were considered realistic, EFW had proposed [DELETED], which was found to create a performance risk.  Id.  The SSA found both IDIQ cost/price proposals reasonable on the basis of the small difference in overall cost/price and adequate price competition.  However, with respect to the PTO 0001 cost/price, the SSA noted that EFW’s proposed cost/price was 25.5 percent higher than that proposed by DRS, and concluded, with input from the CEB, that “EFW’s overall [PTO 0001] cost is not considered fair and reasonable.”  Id. at 3.  Where DRS’s overall IDIQ cost/price was higher than that of EFW, the SSA recorded a tradeoff, concluding that DRS was technically superior, and that:

DRS’s lower, and reasonable cost for [PTO 0001], combined with their realistic IDIQ cost makes its cost proposal lower risk and more attractive than EFW’s slightly lower IDIQ cost, but unreasonable [PTO 0001] cost.  On the basis of my analysis, I have concluded that DRS proposal offers the best value to the Government.

Id.

The agency made the award to DRS on December 21, 2015.  EFW timely requested a debriefing, which the agency provided on December 23.  This protest followed.

DISCUSSION

Misleading Discussions

EFW first alleges that the agency’s discussion questions misled it into responding in a manner that did not address the agency’s true concern with respect to the significant weakness assigned to its proposal under factor (1) PTO 0001.  In this regard, EFW asserts that the agency’s discussion questions in this area never identified a concern with EFW’s total level of effort, or the consistency of that level of effort with all remaining developmental work on EFW’s Link 11/Link 22 solution, the DTS 2020.  EFW argues that instead, the agency’s discussion questions focused on only one discrete aspect of EFW’s remaining development work [DELETED] and the consistency between that development work and the associated level of effort. 

We agree with EFW that the agency’s discussion questions in this area were insufficient to alert EFW to the actual breadth of the agency’s concerns, and were therefore misleading.  It is a fundamental principle of negotiated procurements that discussions, when conducted, must be meaningful; that is, discussions must identify deficiencies and significant weaknesses in an offeror’s proposal that could reasonably be addressed so as to materially enhance the offeror’s potential for receiving award.  PAI Corp., B-298349, Aug. 18, 2006, 2006 CPD ¶ 124 at 8; Spherix, Inc., B-294572, B-294572.2, Dec. 1, 2004, 2005 CPD ¶ 3 at 13; FAR  § 15.306(d).  An agency may not mislead an offeror through the framing of a discussion question into responding in a manner that does not address the agency’s actual concerns, or otherwise misinform the offeror concerning a problem with its proposal.  Metro Mach. Corp., B-281872 et al., Apr. 22, 1999, 99-1 CPD ¶ 101 at 6.

The record here reflects that the agency’s evaluation of EFW’s initial proposal resulted in two significant weaknesses.  The first significant weakness related to EFW’s need to undertake [DELETED], indicating a moderate amount of development remaining.  The agency concluded that this level of remaining development work could possibly result in disruption of schedule, increase in cost, or degradation of performance.  The second significant weakness was categorized as relating to “Cost/Realism.”  AR, Tab 3, SSEB Initial Report, at 21.  This significant weakness also referenced the development work remaining “as it pertains to [DELETED],” but recorded the agency’s additional concern that EFW’s proposed level of effort was “excessive to complete this remaining development.”  Id.  The significant weakness reflected the agency’s conclusions that “EFW has demonstrated an understanding of the requirements, but the excessive level of effort raises serious concerns about the realism and consistency of their proposal,” and “appreciably increases the cost risk of this proposal.”[3]  Id.

As relevant to these significant weaknesses, the agency’s exchanges with EFW contained the following items for discussion:

[DELETED].  This indicates that a moderate amount of development remains to meet threshold/objective requirements. 

Section 1.1.3.1 of the proposal states that there is “modest” development remaining as it pertains to [DELETED].  This is consistent with the proposed [schedule]; however, the cost proposal indicates 11,625 hours for the remaining development.  The SSEB agrees that only ‘modest’ development remains, but considers the proposed amount of effort excessive to complete this remaining development.  EFW has demonstrated an understanding of the requirements, but what is perceived by the Government as an excessive level of effort raises serious concerns about the realism and consistency of the proposal.  EFW’s revised proposal should contain a basis of estimate clearly demonstrating the application of level of effort to remaining development activities and an explanation of your assumptions/justification for the amount of labor proposed. 

AR, Tab 7, Discussion Questions, at 3.  Additionally, the discussions included the following item regarding EFW’s cost/price proposal:

EFW did not provide Basis of Estimate information, making the evaluation of price realism of CLIN 0100 challenging.  With the information available, there appears to be a disconnect between the modest amount of development remaining and the robust amount of [level of effort] proposed.  The costs associated with the proposed [level of effort] cause the Government to consider EFW’s proposed price for [PTO 0001] as uncompetitively high.[4]

Id. at 5.

In its revised proposal, EFW provided a general basis of estimate explaining that the agency’s evaluation of the proposed level of effort was mistaken and that the required [DELETED] accounted for only 3,000 hours, not 11,625 hours, of the 18,244 CLIN 0100 hours set forth in EFW’s initial proposal (reduced to 15,900 hours in EFW’s revised proposal).  EFW explained that the remaining CLIN 0100 hours were not related to [DELETED] development, but related to contract data requirements list (CDRL) preparation, CDRL support, and program management.

On the basis of these revisions, the final SSEB report upgraded EFW’s first significant weakness to a weakness, finding that for the [DELETED], “[t]he 3,000 hours of work . . . is realistic and represents a moderate amount of remaining development,” and that the weakness was “very low risk.”  AR, Tab 9, SSEB Final Report, at 19.  However, the SSEB concluded that the second significant weakness for “realism” remained, concluding, “[t]he SSEB agrees that only ‘modest’ actual development remains ([DELETED]), but considered the total proposed amount of non-recurring effort disproportionate to complete this remaining effort.”  Id. at 20.  The SSEB reached this conclusion because EFW’s basis of estimate failed to explain its “total non-recurring development [level of effort] when applied to the actual development remaining.”  Id.  Accordingly, the significant weakness for “realism” persisted where “EFW has demonstrated an understanding of the requirements, but the excessive level of effort proposed and EFW’s failure to provide detailed BOE information, even after being requested during discussions, continues to raise serious concerns about the realism and consistency of their proposal.”  Id.  As noted by the SSA, this “realism” significant weakness was considered “high risk,” and was the primary discriminator between the proposals under factor (1)--the most important evaluation factor.  AR, Tab 4, SSDD, at 2.

On this record, we conclude that the agency’s discussion questions mislead EFW to provide a basis of estimate explaining the application of its level of effort for [DELETED] only, and did not alert EFW to the agency’s concern with the total level of effort proposed for all work under CLIN 0100.  While the agency contends that the request for basis of estimate information in both the technical and costs discussion items was general in nature, the discussion questions place the request in the context of [DELETED] through repeated specific reference to the moderate or modest development remaining for [DELETED] and to “Section 1.1.3.1” of EFW’s proposal--which exclusively concerned “[DELETED].”  AR, Tab 7, Discussion Questions, at 3; Tab 8, EFW Initial Proposal, at 35. 

We also conclude that the discussions were misleading in part due to the agency’s error, which it acknowledges, concerning its calculation of EFW’s total CLIN 0100 labor hours in its initial proposal.  See AR, Legal Memo, at 18.  In this regard, the agency’s discussion question apparently intended to reference EFW’s total CLIN 0100 hours--which may have alerted EFW to the agency’s concern about EFW’s total level of effort for all remaining development.  However, due to an error in the agency’s evaluation, the discussion question referenced 11,625 hours--rather than the 18,244 CLIN 0100 hours that EFW had initially proposed--and therefore did not reasonably alert EFW to a concern with its total CLIN 0100 level of effort, but instead reflected only a potential misunderstanding of the level of effort associated with EFW’s [DELETED].[5]  Accordingly, we sustain the protest on this basis.[6]

Past Performance

EFW next alleges that the agency’s past performance evaluation relied on a mechanical methodology that obscured qualitative differences between the offerors’ records of past performance.  EFW also argues that the consideration of past performance in the SSDD was plainly inadequate, where the SSDD recorded only that “factors 2 and 3 were the same” and, “[a]ccordingly, factors 2 and 3 will not be discussed further.”  AR, Tab 4, SSDD, at 2.  EFW asserts that the SSDD reflects a failure to conduct a reasonable comparative evaluation of the proposals where it did not look beyond the overall adjectival ratings to consider clear discriminators between the actual contents of the offerors’ proposals.

We agree with EFW that the record here does not demonstrate a reasonable comparative assessment of the offerors’ past performance.  As a general matter, the evaluation of an offeror’s past performance is within the discretion of the contracting agency, and we will not substitute our judgment for reasonably based past performance ratings.  Clean Harbors Envtl. Servs., Inc., B-296176.2, Dec. 9, 2005, 2005 CPD ¶ 222 at 3.  However, we will question an agency’s evaluation conclusions where they are unreasonable or undocumented.  OSI Collection Servs., Inc., B-286597, B-286597.2, Jan. 17, 2001, 2001 CPD ¶ 18 at 6.  Additionally, we have long stated that evaluation ratings should be merely guides for intelligent decision making, see Citywide Managing Servs. of Port Washington, Inc.,  B-281287.12, B-281287.13, Nov. 15, 2000, 2001 CPD ¶  6 at 11, and that therefore evaluators and SSAs should reasonably consider the underlying bases for ratings, including the advantages and disadvantages associated with the specific content of competing proposals, in a manner that is fair, equitable, and consistent with the terms of the solicitation.  Systems Research & Applications Corp.; Booz Allen Hamilton, Inc., B-299818 et al., Sept. 6, 2007, 2008 CPD ¶ 28 at 24.  In this regard, FAR § 15.308 requires a documented source selection decision based on a comparative assessment of proposals against all source selection criteria in the solicitation. 

In this case, the record shows that the SSEB reviewed each offeror’s five past performance references and documented a detailed determination of each reference’s relevance and quality.  This process resulted in the following relevance and quality ratings:

	EFW		DRS
Reference No.	Relevance	Quality	Relevance	Quality
Contract 1	Relevant	Exceptional	Relevant	Satisfactory
Contract 2	Very Relevant	Exceptional	Relevant	Very Good
Contract 3	Somewhat Rel.	Satisfactory	Relevant	Exceptional
Contract 4	Relevant	Very Good	Relevant	Unsatisfactory
Contract 5	Somewhat Rel.	Exceptional	Relevant	Exceptional

AR, Tab 9, SSEB Final Report, at 14, 23.  The SSEB then applied the relevance and quality ratings to a “Past Performance Confidence Assessment Crosswalk,” reproduced below:

	Exceptional	Very Good	Satisfactory	Marginal	Unsatisfactory
Very Relevant	Substantial Confidence	Substantial Confidence	Satisfactory Confidence	Limited Confidence	No Confidence
Relevant	Substantial Confidence	Satisfactory Confidence	Limited Confidence	Limited Confidence	No Confidence
Somewhat Relevant	Satisfactory Confidence	Limited Confidence	Limited Confidence	No Confidence	No Confidence
Not Relevant	Unknown Confidence	Unknown Confidence	Unknown Confidence	Unknown Confidence	Unknown Confidence

Id. at 6.  Using this table, the SSEB converted each contract reference’s individual relevance and quality ratings into confidence ratings.  The SSEB then averaged the confidence ratings to reach an overall adjectival confidence rating for factor (3), as follows:

	EFW	DRS
Contract 1	Substantial Confidence	Limited Confidence
Contract 2	Substantial Confidence	Satisfactory Confidence
Contract 3	Limited Confidence	Substantial Confidence
Contract 4	Satisfactory Confidence	No Confidence
Contract 5	Satisfactory Confidence	Substantial Confidence
Overall	Satisfactory Confidence	Satisfactory Confidence

Id. at 14, 23.

While this record demonstrates that the SSEB did initially conduct a qualitative assessment of each offerors’ individual past performance references, we see no evidence in the record that the agency ever comparatively assessed the offerors’ actual past performance information to assess whether any discriminators existed under the past performance factor.  Instead, the record reflects that--despite the detailed differences identified in the past performance feedback recorded by the SSEB--the agency’s comparative assessment of the proposals relied on the equivalent “satisfactory confidence” ratings generated by the mechanical application of the agency’s “Past Performance Confidence Assessment Crosswalk.”

Specifically, in a briefing to the SSA, the SSEB discussed its ratings of each contractor individually under the past performance factor, and noted that EFW had “relevant experience” and “very good overall quality,” while DRS had “relevant experience” and “satisfactory overall quality.”  SSEB and CEB Revised Proposals Briefing, at 7-8.  However, in the best-value recommendation, the briefing did not demonstrate any comparative assessment under the past performance factor, indicating only that “DRS had higher ratings for factors 1 and 4.”  Id. at 12.  The SSA, then, in the SSDD, recorded only that “factors 2 and 3 were the same” and, “[a]ccordingly, factors 2 and 3 will not be discussed further.”  AR, Tab 4, SSDD,  at 2. 

We cannot conclude that this evaluation record fulfills the requirement for a documented decision based on a comparative assessment of proposals against all source selection criteria in the solicitation.  The agency argues that the record is sufficient on the basis that the SSDD incorporated the SSEB’s final report, which did contain the detailed qualitative assessment of each offeror’s past performance references.  However, as outlined above, the SSEB’s past performance assessment does not evidence any comparative analysis for the purpose of identifying discriminators, and the record otherwise contains no explanation--other than the identical adjectival ratings produced by the “Past Performance Confidence Assessment Crosswalk”--for the SSA’s conclusion that no discriminators existed under the past performance factor.[7]

We also agree with EFW that the agency’s mechanical use of the “Past Performance Confidence Assessment Crosswalk” and reliance on adjectival ratings obscured actual qualitative differences between the offerors.  For example, the SSEB’s initial qualitative assessment appears to document substantial quality differences between the offerors’ past performance references.  Specifically, the SSEB evaluation recorded that past performance feedback for DRS’ contract 4 reference indicated unsuccessful performance in three areas  concerning issues with “responsiveness and reliability of reports” and “schedule delays.”  AR, Tab 9, SSEB Final Report, at 15.  Additionally, feedback on DRS’ contract 1 reference indicated that, although considered satisfactory overall, “[i]t was noted that DRS experience[d] a problem with delays in equipment deliveries.”  Id. at 14.  In contrast, none of EFW’s past performance references indicated any quality issues whatsoever, and although EFW’s past performance included two references that were considered less relevant, it also included the only “very relevant” reference, for which EFW was assigned an exceptional quality rating.  While it would not necessarily be unreasonable for an SSA to conclude that these past performance records were nonetheless equivalent, the record in this case provides no rationale--other than the equivalent adjectival ratings--to support that conclusion in the face of apparent quality differences between the two offeror’s references.  Accordingly, we sustain the protest on this basis. 

Technical Price Realism Evaluation

Finally, EFW alleges that the agency conducted unreasonable price realism and price reasonableness analyses, and inappropriately conflated “realism” and “reasonableness” analysis throughout the record.  We agree with EFW that the agency’s “technical price realism analysis” was unreasonable.

As stated above, the RFP provided that “Technical Price Realism Analysis will be performed by the Government for the Fixed Price Incentive CLIN [CLIN 0100] as described under the Technical evaluation Factor 1.”  AR, Tab 2, Conformed RFP,  at 70.  In turn, factor 1 provided that the evaluation would consider “the extent to which the proposed cost is realistic for the work to be performed, reflects a clear understanding of the requirements, and is consistent with the unique methods of performance and materials described in the Offeror’s proposed solution.”  Id. at 69. 

The FAR does not use the term “price realism,” but provides that cost realism analysis may be used to evaluate fixed-price proposals, as follows:

Cost realism analyses may also be used on competitive fixed-price incentive contracts or, in exceptional cases, on other competitive fixed-price-type contracts when new requirements may not be fully understood by competing offerors, there are quality concerns, or past experience indicates that contractors’ proposed costs have resulted in quality or service shortfalls.  Results of the analysis may be used in performance risk assessments and responsibility determinations.  However, proposals shall be evaluated using the criteria in the solicitation, and the offered prices shall not be adjusted as a result of the analysis.

FAR § 15.404-1(d)(3).  As our Office has held, price reasonableness and price realism are distinct concepts.  Logistics 2020, Inc., B-408543, B-408543.3, Nov. 6, 2013, 2013 CPD ¶ 258 at 7.  The purpose of a price reasonableness review is to determine whether the prices offered are too high, as opposed to too low.  See FAR § 15.404-1(b); Sterling Servs., Inc., B-291625, B-291626, Jan. 14, 2003, 2003 CPD ¶ 26 at 3.  Conversely, a price realism review is to determine whether prices are too low, such that there may be a risk of poor performance. 

The SSEB’s evaluation in this case, adopted by the SSA in the SSDD, assigned EFW’s proposal a “very significant weakness presenting high risk” relating to “realism” under the SSEB’s evaluation of factor 1, despite acknowledging within that weakness that “EFW has demonstrated an understanding of the requirements,” and that EFW--despite the significant weaknesses for “realism”--had an overall “acceptable” approach under the factor.  AR, Tab 9, SSEB Final Report, at 18, 20.  Nor did the SSEB record any apparent concern that EFW’s proposed costs may result in a quality or service shortfall.  Rather, the SSEB’s concern was that EFW’s proposed labor hours were too high in proportion to the amount of remaining development work described in EFW’s technical proposal.  Specifically, the SSEB recorded a conclusion that EFW’s “[level of effort] is still considered excessive, unrealistic, and inconsistent with the amount of development remaining,” which “indicates that there is still significant development remaining (engineering and otherwise) in order to deliver the final proposed solution.”  AR, Tab 9, SSEB Final Report, at 18.

On our review of the record we fail to see how the agency’s concern implicates the “realism” of EFW’s proposal under factor 1.  It is not apparent that this “technical price realism analysis” concerned “the extent to which the proposed cost is realistic for the work to be performed, reflects a clear understanding of the requirements, and is consistent with the unique methods of performance and materials described in the Offeror’s proposed solution,” as required by the RFP.  AR, Tab 2, Conformed RFP, at 69 (emphasis added).  We also fail to see how EFW’s potential overstaffing of its CLIN 0100 solution presents a performance risk, where the SSEB has acknowledged that EFW demonstrated an understanding of the requirements.  Accordingly, we see no logical basis for the agency to assign a significant weakness for “realism” in this context, and conclude that the agency’s technical price realism evaluation was unreasonable. 

We also question the agency’s conclusion, in the CEB report, that “the SSEB’s assessment of high risk for EFW’s technical price realism does not lend itself to a favorable reasonableness determination,” and that EFW’s CLIN 0100 cost/price was unreasonably high.  AR, Tab 10, CEB Report, at 19-20.  As discussed immediately above, we conclude that the “high risk” assigned for EFW’s technical price realism of technical factor (1) was unreasonable, where no apparent “realism” or performance risk existed.  Further, we cannot conclude that any cost/price risk existed where, although CLIN 0100 was a fixed-price incentive CLIN, EFW proposed a share ratio of 100 percent contractor responsibility for amounts above the target price.  AR, Tab 10, CEB Final Report, at 9.  While a price reasonableness determination is a matter of administrative discretion involving the exercise of business judgment by the contracting officer that we will question only where it is unreasonable, The Right One Co., B-290751.8, Dec. 9, 2002, 2002 CPD ¶ 214 at 5, here, we cannot conclude that the record demonstrates a reasonable basis for the CEB’s and SSA’s conclusion that EFW’s CLIN 0100 (and PTO 0001) price was unreasonably high, where the determination relied in large part on the SSEB’s flawed “high risk” assessment.[8]

CONCLUSION

On the basis of our review of the record, we sustain the protest because the agency conducted misleading discussions with EFW by framing its discussion questions as specifically relating to the level of effort and development remaining for [DELETED], which failed to alert EFW to the agency’s concern that EFW’s total level of effort for all CLIN 0100 work was overstated and required explanation.  We also conclude that the agency’s past performance evaluation was unreasonable where the SSEB relied on a mechanical application of evaluation ratings to convert relevancy and quality ratings to a single confidence rating, and the record does not reflect any comparative examination of the underlying strengths and weakness of the offeror’s past performance references. 

Finally, we conclude that the agency erred in concluding that excessive labor hours for remaining development work represented a significant weakness for price realism.  In this regard, price realism analysis concerns whether proposed prices are too low, not whether an offeror’s proposed price, or proposed level or effort, is too high.  Further, we fail to see how overstated development labor hours presents any performance risk to the agency in this context, where the agency acknowledged that EFW had an understanding of the requirements and acceptable approach, or any cost/price risk where EFW proposed a share ratio of 100 percent contractor responsibility for amounts above the target price.  We also conclude that where the agency relied on this “technical price realism” analysis to determine that EFW’s CLIN 0100 and PTO 0001 prices were unreasonably high, the agency’s price reasonableness determination was also unreasonable.

RECOMMENDATION

We recommend that the agency reopen discussions with both offerors and obtain a new round of final revised proposals.  We also recommend that the agency perform a new evaluation under the past performance factor, documenting a comparative analysis of the proposals considering the underlying bases for the ratings, including the advantages and disadvantages associated with the specific content of competing proposals.  Finally, we recommend that the agency conduct a new price realism analysis focused on whether prices are too low, such that there may be a risk of poor performance, and reconsider its price reasonableness determination and document an assessment of unbalanced pricing, if necessary.

We also recommend that the protester be reimbursed the reasonable costs of filing and pursuing the protests, including attorneys’ fees.  Bid Protest Regulations,  4 C.F.R. § 21.8(d)(1).  The protester should submit its certified claim for costs, detailing the time expended and the costs incurred, directly to the contracting agency within 60 days after receipt of this decision.

The protest is sustained.

Susan A. Poling
General Counsel

---