DKW Communications, Inc.

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: DKW Communications, Inc.

File: B-411182; B-411182.2

Date: June 9, 2015

DIGEST

1. Protest that agency failed to conduct a reasonable cost/technical tradeoff is sustained where the record shows that the agency did not consider cost to the government as part of its award decision and, to the extent the tradeoff considered non-cost features of proposals, it was inadequately documented.

2. Protest that agency unreasonably failed to consider protester’s relevant, positive past performance information that was close at hand is sustained where, under the circumstances, it was unreasonable for the agency not to consider such information.

DKW Communications, Inc., of Washington, D.C., protests the issuance of a task order to Agile Defense, Inc., of Fairfax, Virginia, by the Department of Defense, Defense Advanced Research Projects Agency (DARPA), under request for proposals (RFP) No. HR0011-14-R-0001 for unclassified information technology (IT) services and support. DKW challenges the agency’s evaluation of its proposal and the agency’s best value tradeoff decision.

We sustain the protest in part and deny it in part.

BACKGROUND

DARPA issued the RFP on August 14, 2014, for various services identified on the General Services Administration (GSA) Alliant Small Business Government-wide Acquisition Contract (GWAC), Special Item Number ASB. RFP at 1. The RFP contemplated the award of a cost-plus-award-fee task order to be performed over one base year and eight option years. Id. at 9. The awardee was to provide and manage a range of IT services, support and infrastructure necessary to implement the agency’s IT operational objectives. Agency Report (AR), exh. 5, Performance Work Statement (PWS), at 1.

Proposals were to be evaluated under five criteria listed in descending order of importance: technical excellence, management capability, personnel qualifications, past performance, and proposed cost and fee. RFP at 34-37. Award of a task order was to be made on a best-value basis utilizing tradeoff procedures between the various factors. Id. at 38; AR, exh. 6, Amendments to the RFP, at 2. Although the non-cost factors were each more important than proposed cost and fee, the importance of proposed cost and fee would increase as non-cost factors became closer in merit. AR, exh. 6, Amendments to the RFP, at 2.

Proposals were to be evaluated under the technical excellence factor to determine the extent to which the offeror demonstrated: (1) understanding of the PWS and the current DARPA IT environment; (2) ability and expertise to successfully accomplish the objectives and requirements of the PWS; (3) methods, techniques, and tools required to perform the objectives of the PWS, anticipate problems that may affect performance and ability to mitigate risks; and (4) experience in identifying areas of deficiency or inefficiency in technical processes and recommending process improvements/innovative technical solutions. RFP at 34. Technical risk was to be assessed to determine the degree to which the offeror’s proposed technical plan might result in unsuccessful performance. Id.

Under the management capability factor, the agency was to evaluate seven specific areas, including the extent to which an offeror demonstrated its overall management approach to successful execution, its plan for effective cost control and experience implementing cost efficiencies, and its organizational structure/staffing plan. Id. Management risk was to be assessed to determine the degree to which the offeror’s proposed management plan might result in unsuccessful performance. Id. at 35.

Under the past performance factor, the agency was to evaluate the quality and relevance of the recent past performance of the offeror and any of its proposed major subcontractors. For quality, the agency was to assess an offeror’s performance record, including timeliness and cost control. Id. For relevance, the agency was to assess the similarity of service/support, complexity, contract dollar value, contract type, contract length, staffing level, key personnel involvement, and dollar amount of subcontracting. Id. at 35-36. Performance confidence was to be assessed to determine the degree to which the government expected the offeror would successfully perform the required effort. Id. at 36.

Finally, cost proposals were to be evaluated for reasonableness and realism. Id. The cost realism evaluation was to assess and verify that the offeror’s (and any subcontractor’s) proposed costs were realistic; that the offeror understood the requirements; and the degree to which the costs in the cost proposal represented the work effort described in the technical proposal. DARPA was also to evaluate the level of effort and the mix of labor proposed. Id. Offerors were informed that:

The Government will identify any potential cost risks to an Offeror’s proposal that could affect the performance of the required effort. The Government will determine what the Government predicts the Offeror’s approach would most probably cost the Government when the work is performed as proposed. To the degree that the Government’s most probable cost estimate does not equal the Offeror’s proposed cost, the cost will either be adjusted upward or downward for the purposes of evaluation only.

Id. at 36-37.

The agency received seven proposals, including those from DKW and Agile. AR, exh. 16, Evaluation Team Consensus Report, at 2. After an initial evaluation, the evaluation team narrowed its consideration of proposals to three offerors. RFP at 34; AR, exh. 16, Evaluation Team Consensus Report, at 7:

	Agile	Offeror A	DKW
Overall Technical	Outstanding	Marginal	Acceptable
Technical Excellence	Outstanding	Marginal	Good
Management Capability	Outstanding	Marginal	Marginal
Personnel Qualifications	Good	Outstanding	Good
Past Performance	Substantial Confidence	Substantial Confidence	Satisfactory Confidence
Proposed Cost	$144.5 million	$97.2 million	$83.1 million
Most Probable Cost	$154.4 million	$140.2 million	$129.9 million
Cost Risk	$9.9 million	$43.0 million	$46.8 million
Cost Risk (%)[1]	6.80%	44.2%	56.29%

AR, exh. 16, Evaluation Team Consensus Report, at 5-7.

As part of its award recommendation, the evaluation team noted that Agile’s proposal received higher overall consensus ratings for technical excellence, management capability and personnel qualifications than either other proposal. Id. at 7. The team also observed that Agile’s proposal was rated higher for past performance than was DKW’s proposal. Id. Finally, the team considered that, while Agile’s evaluated/most probable cost of $154.4 million was higher than DKW’s evaluated/most probable cost of $129.9 million, Agile’s proposal represented “significantly less cost risk to the Government” than the other two proposals. Id. at 7-8. The team recommended that Agile’s proposal be awarded the task order as the best value offeror based on its significantly higher technical rating, substantial confidence past performance rating, and because of “Agile’s proposal being assessed as presenting the lowest cost risk to the Government.” Id. at 8. The team did not discuss the relative merits of each proposal other than in terms of the assigned adjectival ratings, and did not discuss their relative costs.

The source selection authority (SSA) adopted the evaluation team’s adjectival ratings, but did not discuss the proposal features underlying those ratings. AR, exh. 17, Selection Decision Document (SDD), at 3-4. He also concurred with the team’s finding that the proposed costs of Offeror A and DKW “are unreasonable and not realistic and represent a very high cost risk to the Government.” Id. at 4. The SSA found these two proposals close enough in merit to each other under the non-cost factors to increase the importance of cost in his tradeoff analysis. Id.; see also AR, exh. 6, Amendments to the RFP, at 2. Accordingly, the SSA ranked Offeror A above DKW, despite DKW’s superior overall technical rating and lower most probable cost, because Offeror A’s proposal had a lower cost risk to the government. AR, exh. 17, SDD, at 4. The SSA’s selection of Agile for award was based on his conclusion that the firm has “the technical and business expertise to most successfully perform the . . . requirements,” and the firm’s “proposed costs are reasonable and realistic and have the lowest overall cost risk to the Government.” Id. at 5. After a debriefing, this protest followed.[2]

DISCUSSION

DKW challenges various aspects of DARPA’s evaluation of proposals, including the agency’s best value tradeoff determination, its past performance evaluation, and its evaluation of DKW’s proposal under the technical excellence and management capability factors. For the reasons discussed below, we sustain DKW’s protest challenging the agency’s best value tradeoff decision and past performance evaluation, and deny the remainder of the protest.

Best Value Determination

DKW argues that DARPA failed to conduct a reasonable best value tradeoff in that the agency entirely failed to evaluate the comparative cost to the government of the proposals under consideration. Protester’s Comments at 6. DKW also alleges that the agency failed to identify any non-cost benefit in the awardee’s proposal and failed to evaluate whether any such benefit warranted payment of a $25 million cost premium. Id. at 14. In a related argument, DKW takes issue with the selection decision’s lack of any analysis of proposals below the adjectival ratings level for either the technical factors or past performance. Id.

In a best value procurement, it is the function of the source selection authority to perform a tradeoff between cost and non-cost factors, that is, to determine whether one proposal’s superiority under the non-cost factor is worth a higher cost. Even where, as here, cost is stated to be of less importance than the non-cost factors, an agency must meaningfully consider cost to the government in making its selection decision. See e-LYNXX Corp., B-292761, Dec. 3, 2003, 2003 CPD ¶ 219 at 7; see also 10 U.S.C. § 2304c(d)(3) (requiring consideration of cost in the award of orders under multiple-award contracts). Specifically, before an agency can select a higher-cost proposal that has been rated technically superior to a lower-cost, but acceptable one, the award decision must be supported by a rational explanation of why the higher-rated proposal is, in fact, superior, and explain why its technical superiority warrants paying a cost premium. ACCESS Sys., Inc., B-400623.3, Mar. 4, 2009, 2009 CPD ¶ 56 at 7.

DARPA argues that it conducted a rational tradeoff because Agile’s proposal was rated higher than DKW’s proposal under the non-cost factors, and because DKW’s proposal represented a high cost risk to the government. Supp. Agency Report at 1. DARPA states that the benefit identified in Agile’s proposal that was worth the cost premium was that its proposal was deemed reasonable, realistic and represented the lowest cost risk to the government in terms of dollars. Id. at 20.

Our review of the record shows that the agency did not conduct a rational best value tradeoff decision. As discussed above, the SSA accepted the adjectival ratings assigned by the evaluation team to each of the three proposals under consideration, but did not discuss the proposal features underlying those ratings. AR, exh. 17, SDD, at 3-4. The SSA also adopted the evaluation team’s assessment of cost proposals, which consisted entirely of proposals being evaluated for cost risk, as described above, and contained no meaningful consideration of the relative costs evaluated for each proposal. Id. at 4. The sum total of the SSA’s best value tradeoff decision consisted of the statement, “I determine that Agile has the technical and business expertise to most successfully perform the . . . requirements. I also determine that Agile’s proposed costs are reasonable and realistic and have the lowest overall cost risk to the Government.” Id. at 5.

There is no evidence in the record that the agency considered cost to the government in making its selection decision. As discussed, in deciding to award to a higher-cost proposal that has been rated technically superior to a lower-cost but acceptable one, an award decision must be supported by a rational explanation of why the higher-rated proposal is, in fact, superior, and why its technical superiority warrants paying a cost premium. Stated simply, such an analysis weighs the benefits of lower cost versus technical superiority. In this context, the cost premium represents the higher price the agency will pay in order to receive some evaluated benefit from the higher-rated proposal.

Here, a rational tradeoff by the agency would have required it to consider whether the benefits of Agile’s technical proposal warranted paying more for that proposal (i.e., a cost premium), as opposed to accepting DKW’s lower-rated, lower-cost proposal. While the agency asserts that it conducted a rational tradeoff, and gave meaningful consideration to cost, our review of the record shows that the analysis was not rational. While DKW’s evaluated most probable cost of $129.9 million[3] is clearly lower than Agile’s most probable cost of $154.4 million, the contemporaneous record does not reflect any consideration of this potential cost savings. Instead, the agency concluded that Agile’s cost proposal was more favorable simply because its higher proposed costs were closer to the agency’s estimate of the proposal’s most probable cost. Such an analysis does not consider the relative benefit of paying a lower cost, as required in a tradeoff analysis. Since the record shows that the agency did not meaningfully consider the overall cost to the government in selecting Agile’s higher-cost proposal in its tradeoff decision, we sustain the protest.

We also agree with the protester that, to the extent that the SSA considered the non-cost features of the proposals, the best-value tradeoff analysis was not sufficiently documented. An award decision must be supported by a rational explanation of why the higher-rated proposal is, in fact, superior, and explain why its technical superiority warrants paying a cost premium. ACCESS Sys., Inc., supra, at 7. Here, both the evaluation team and the SSA’s consideration of the technical merits of each proposal consisted only of the adjectival ratings assigned and the agency’s assessment of cost risk. As our Office has held, adjectival ratings serve only as guides to intelligent decision making, and source selection officials should reasonably consider the underlying bases for ratings, including the advantages and disadvantages associated with the specific content of competing proposals. See CPS Prof’l Servs., LLC, B-409811, B‑409811.2, Aug. 13, 2014, 2014 CPD ¶ 260 at 5. On this record, we cannot determine the reasonableness of the agency’s award decision.[4]

Past Performance Evaluation

DKW also contends that DARPA prejudicially failed to consider positive past performance information for the firm that was close at hand. Protester’s Supp. Comments at 60-61. DKW also argues that, in contravention of the RFP’s terms, the past performance evaluation improperly failed to assess the dollar value of past performance contracts identified by offerors, and that DARPA lacked any rational basis to award Agile a higher performance confidence rating. Id. at 47-59.

Our Office examines an agency’s evaluation of past performance to ensure that it was reasonable and consistent with the stated evaluation criteria and applicable statutes and regulations; however, the necessary determinations regarding the relative merits of offerors’ proposals are primarily matters within the contracting agency’s discretion. Advanced Envtl. Solutions, Inc., B-401654, Oct. 27, 2009, 2010 CPD ¶ 7 at 5. In this regard, our Office will not question an agency’s determinations absent evidence that those determinations are unreasonable or contrary to the stated evaluation criteria. Id.

DKW alleges that the agency failed to consider positive information found in a Contractor Performance Assessment Reporting System (CPARS) report related to one of the past performance references submitted for its subcontractor. DKW asserts that the final CPARS report, related to its subcontractors’ performance on a bridge contract for the predecessor services contemplated by this RFP, was completed around the time that the past performance evaluations for this solicitation were being conducted, and the report was completed by the same individual who served as the technical chair for the evaluations under the current RFP. Protester’s Comments at 60-61. Thus, according to the protester, the information was too close at hand for the agency to ignore. Id. at 61.

We have recognized that in certain limited circumstances, an agency has an obligation (as opposed to the discretion) to consider outside information bearing on the offeror’s past performance when it is too close at hand to require offerors to shoulder the inequities that spring from an agency’s failure to obtain and consider the information. See e.g., Int’l Bus. Sys., Inc., B-275554, Mar. 3, 1997, 97-1 CPD ¶ 114 at 5. The agency does not dispute the protester’s allegation that the CPARS report was completed and known to the evaluators. See Supp. Agency Report at 53-57. Instead, DARPA argues that the past performance information in the CPARS report is consistent with a past performance questionnaire that the agency did consider as part of its evaluation. Id. at 55-56. Moreover, the agency argues that it was within its reasonably exercised discretion, under the terms of the RFP, not to consider the CPARS report. Id. at 56.

Under the circumstances here, we conclude that DARPA unreasonably failed to consider the CPARS report as part of its evaluation. First, as DARPA does not refute the protester’s allegations, we can only conclude that it knew of and had access to the CPARS report at the time it conducted DKW’s past performance evaluation. Second, it is clear that the agency considered a past performance questionnaire relevant to the contract in question as part of its evaluation. See AR, exh. 14, DKW’s Past Performance Consensus Evaluation Report, at 3. Third, it appears that DKW could not have included the CPARS report as part of its proposal, as the information was not available at time of proposal submission. Finally, contrary to the agency’s argument, the information considered by the evaluators is not consistent with the evaluation found in the CPARS report.

The past performance questionnaire evaluated by the agency shows that DKW’s subcontractor received three good ratings, seven acceptable ratings and one non-applicable rating for its performance under the relevant contract. AR, exh. 20, Past Performance Questionnaire, at 2-3. Additionally, the narrative for the questionnaire can best be described as critical of the contractor’s performance on the contract. Id. at 4. This information is summarized in the agency’s past performance evaluation. AR, exh. 14, DKW’s Past Performance Evaluation Report, at 3. The CPARS report, which involves the bridge contract for the work preceding this solicitation, shows that the contractor received four very good ratings and two exceptional ratings. Protester’s Comments, Attachment 2. The narrative assessments of the contractor’s performance are also uniformly more positive than the assessment found on the past performance questionnaire evaluated by the agency. See generally id. In other words, the CPARS report is decidedly more positive than the questionnaire considered by the agency as part of its evaluation. Therefore, we do not find the agency’s position, that it considered all relevant information when it considered the past performance questionnaire, to be reasonable. We therefore sustain the protest on this basis.

DKW also alleges that the agency failed to assess the dollar value of past performance references. DARPA responds by arguing, specifically with respect to Agile’s evaluation, that while the record does not document such consideration, it is inconsequential since Agile’s proposal contained this information, and its proposal was fully considered by the past performance evaluation team. Supp. Agency Report at 50. After reviewing the record, we conclude that the record does not adequately document the agency’s consideration of the dollar value of past performance references in its past performance evaluation.[5] Even accepting the agency’s position that it considered all information within Agile’s proposal in its evaluation, agencies are required to adequately document their evaluations, and, where an agency fails to do so, it runs the risk that our Office will be unable to determine whether the agency’s evaluation conclusions are reasonable. IAP World Servs., Inc.; EMCOR Gov’t Servs., B‑407917.2 et al., Jul. 10, 2013, 2013 CPD ¶ 171 at 12. Therefore, we also sustain the protest on this basis. [6]

Technical Evaluation

DKW finally challenges the agency’s evaluation of the firm’s proposal under the technical excellence and management capability factors. DKW argues that the agency improperly evaluated the technical feasibility of its offered innovations in contravention of the RFP’s terms. Protester’s Comments at 20-26. DKW also argues that the agency improperly exaggerated the significant weakness assigned to the firm’s proposed staffing plan under the management capability factor. Id. at 26-46.

The evaluation of technical proposals is a matter within the discretion of the contracting agency, since the agency is responsible for defining its needs and the best method for accommodating them. SRA Int’l, Inc., B-408624, B-408624.2, Nov. 25, 2013, 2013 CPD ¶ 275 at 4. In reviewing an agency’s evaluation, we will not reevaluate technical proposals, but instead will examine the agency’s evaluation to ensure that it was reasonable and consistent with the solicitation’s stated evaluation criteria and with procurement statutes and regulations. Id. In order to facilitate our examination, contracting agencies are required to adequately document their evaluation results, and sufficiently support the findings on which award determinations are made. Savvee Consulting, Inc., B-408416, B-408416.2, Sep. 18, 2013, 2013 CPD ¶ 231 at 7.

DKW argues that the agency used an undisclosed evaluation criterion when it evaluated the feasibility of the innovation examples suggested by the firm. The protester contends that its proposed innovations were to be evaluated under the fourth element of the technical excellence factor--the experience element--because it offered them as examples of experience in recommending innovative technical solutions.[7] Protest at 16; Protester’s Comments at 20-26. However, the record shows that the risks and weaknesses assigned to DKW’s proposal for its proposed innovations were assigned under the third element of the factor--as methods, techniques and tools required to perform the objectives of the PWS--and not as experience. AR, exh. 13, DKW Technical Consensus Evaluation Report, at 3. Our review of DKW’s technical proposal shows that the innovations it describes are, in fact, proposed in the context of what the firm will do as part of its technical approach, and not what it has done as a matter of experience. As a result, we conclude that the agency’s evaluation was reasonable.[8]

DKW also challenges the assignment of three significant weaknesses to its proposal under the management capability factor related to the firm’s staffing plan, which proposed to decrease full time equivalent (FTE) personnel from [DELETED] to [DELETED] in the option years. DKW contends that DARPA improperly exaggerated the importance of its concern by triple-counting the significant weakness. Protester’s Comments at 45. DKW argues that if it was proper for the agency to assign a significant weakness for its staffing plan, it was within DARPA’s discretion to consider the proposed FTE reductions in its cost evaluation, but improper to exaggerate the concern by triple-counting the significant weakness. Id.

While DKW states that it does not concede that the significant weaknesses for its staffing plan was properly assigned, our review of the protester’s comments shows no meaningful challenge to the significant weaknesses assigned. While DKW initially protested the substance of the significant weaknesses, the firm presents no meaningful response to the agency’s position in its comments. Consequently, we view this protest ground as being abandoned. Alliance Tech. Servs., Inc., B‑410307, B‑410307.3, Dec. 1, 2014, 2014 CPD ¶ 345 at 3 n.2.

Moreover, our review of the record gives us no basis to question the assignment of three significant weaknesses under the management capability factor to DKW’s proposal for its proposed FTE reductions. Our Office has held that an agency may properly consider an element of a proposal under more than one evaluation criterion where the element is relevant and reasonably related to each criterion under which it is considered. See UNICCO Gov’t Servs, Inc., B-409111 et al., Jan. 23, 2014, 2014 CPD ¶ 55 at 11 n.6 (citing Teledyne Brown Eng’g, B-258078, B‑258078.2, Dec. 6, 1994, 94-2 CPD ¶ 223 at 4‑5). Here, the record shows that the agency was to consider seven specific elements under the management capability factor. RFP at 34. The record also shows that each assigned significant weakness reasonably relates to the element under which it was assigned. Thus, there is nothing inherently wrong with the agency’s assignment of significant weaknesses under the relevant elements of the management capability factor.

However, as noted above, the SSD does not discuss the non-cost features of the proposals other than in terms of the adjectival ratings assigned. The cost/technical tradeoff decision does not discuss, for instance, how the agency ultimately weighed the significant weaknesses assigned to DKW’s staffing plan and its proposal to reduce FTE personnel in the option years. In implementing our recommendation, the agency should document its conclusions with respect to these and any other pertinent non-cost features of the competing proposals.

RECOMMENDATION

We recommend that DARPA re-evaluate the past performance of both DKW and Agile consistent with this decision, and adequately document its evaluation. We also recommend that the agency conduct a new best value determination consistent with this decision. Should the agency conclude that a vendor other than Agile properly is in line for award, we recommend that the agency terminate the order issued to Agile, and make award to the new vendor, if otherwise proper. We also recommend that the agency reimburse DKW its reasonable costs of filing and pursuing its protest, including reasonable attorneys’ fees. 4 C.F.R. §21.8(d)(1). The protester’s certified claim for costs, detailing the time spent and the cost incurred, must be submitted to the agency within 60 days after receipt of this decision. 4 C.F.R. §21.8(f).

The protest is sustained in part and denied in part.

Susan A. Poling
General Counsel