Apprio, Inc.

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Apprio, Inc.

File: B-420627

Date: June 30, 2022

DIGEST

1. Protest challenging the agency’s cost realism evaluation is sustained where the record does not demonstrate that the agency assessed the awardee’s proposed direct labor rates or the awardee’s approach to rate escalation, and where the record shows that the agency employed a standard deviation methodology in an unreasonable manner.

2. Protest challenging the evaluation of the protester’s technical proposal under the corporate experience factor is sustained where the record shows that the agency assessed an unreasonable weakness, and where the agency’s response to the protest was not supported by the contemporaneous record.

3. Challenge to the evaluation of the awardee’s proposal under the staffing plan and key personnel experience factor is dismissed for failing to state a valid basis of protest.

Apprio, Inc., of Washington, District of Columbia, protests the issuance of a task order to Leidos, Inc., of Reston, Virginia, under request for quotations (RFQ) No. 70FA2022Q00000003, issued by the Department of Homeland Security, Federal Emergency Management Agency (FEMA), for training services to be performed at the Center for Domestic Preparedness (CDP) in Anniston, Alabama. The protester challenges the agency’s cost realism evaluation of the awardee’s proposal and the technical evaluation of the competing proposals.[1]

We sustain the protest.

BACKGROUND

FEMA issued the RFQ on November 22, 2021, under the fair opportunity procedures of Federal Acquisition Regulation (FAR) section 16.505. Agency Report (AR), Tab AI, e‑Buy Notice at 1; AR, Tab J, RFQ at 1. The agency sought proposals from holders of the General Services Administration’s (GSA’s) Human Capital and Training Solutions (HCaTS) Indefinite-Delivery, Indefinite-Quantity (IDIQ) contract. RFQ at 1. The awardee will be required to deliver training classes at CDP. AR, Tab K, Performance Work Statement (PWS) at 1. CDP is a national training facility that prepares state and local emergency response personnel to respond to various hazards such as terrorist attacks by conducting hands‑on and classroom‑based training. Id.

The RFQ contemplated the issuance of a hybrid task order that includes both fixed‑price and cost‑plus‑award‑fee contract line items, with a 1‑year base period and four 1‑year option periods. RFQ at 2‑6. Issuance of the task order was to be made on a best‑value tradeoff basis considering cost and price, and the following non‑price factors listed here in descending order of importance: (1) corporate experience; (2) technical and management plan; (3) staffing plan and key personnel; and (4) quality control plan. Id. at 59‑63. For purposes of award, the non‑price factors, when combined, were more important than cost and price. Id. at 59.

The solicitation provided for a two‑phase evaluation. Id. at 49, 59. In phase one, interested offerors were invited to submit technical proposals addressing only the corporate experience factor. Id. at 49‑52. Phase one proposals would be rated as high confidence, some confidence, or low confidence. Id. at 49; AR, Tab M, Source Selection Plan (SSP) at 14. The agency would then issue “down‑select” notices to the participating offerors and invite phase two proposals from “viable competitor[s].”[2] RFQ at 49. Phase two proposals were to be submitted in two parts. Id. at 49‑50. The first part addressed the remaining non‑price evaluation factors. Id. at 52‑53. Under each of the remaining non‑price factors, proposals would be evaluated as outstanding, good, satisfactory, or unsatisfactory. AR, Tab M, SSP at 14‑15. The second part of phase two required the submission of a “business proposal” that was to include information necessary to evaluate the cost and price factor. RFQ at 54.

The RFQ explained that all fixed‑price contract line item numbers (CLINs) would be evaluated for price reasonableness based on adequate competition. Id. at 62. The cost‑reimbursement CLINs would be evaluated for cost realism. Id. at 63. The agency’s cost realism evaluation would assess whether the offeror’s proposed costs were: (1) realistic for the work to be performed; (2) reflective of the offeror’s understanding of the requirements; and (3) consistent with the unique methods of performance and materials described in the offeror’s technical proposal. Id. The cost realism evaluation would include a determination of each offeror’s most probable cost of performance. Id. In making a best‑value tradeoff decision, the evaluated cost and price would be based on the total cost and price of all CLINs, inclusive of the option periods and fees, but exclusive of the offeror’s proposed fixed‑price CLIN for its phase‑in period. Id. at 62.

FEMA received seven phase one proposals by the due date of November 7. Id. at 1; AR, Tab P, Phase One Technical Evaluation Team (TET) Report at 1. Apprio’s phase one proposal earned an overall rating of some confidence.[3] AR, Tab P, Phase One TET Report at 1. In evaluating Apprio’s phase one proposal, the agency found that it warranted three strengths and one weakness. Id. at 2‑3. Leidos’s phase one proposal earned an overall rating of high confidence. Id. at 1. In evaluating Leidos’s phase one proposal, the agency found that it warranted five strengths with no weaknesses noted. Id. at 9.

Based on the results of the phase one evaluation, the agency found that six offerors‑‑including Apprio and Leidos‑‑were viable competitors. Id. at 16; AR, Tab H, Apprio Notice at 1; AR, Tab I, Leidos Notice at 1. Phase two proposals were due by January 25. RFQ at 1.

FEMA received five timely phase two proposals. Contracting Officer’s Statement (COS) at 5; AR, Tab Y, Phase‑Two TET Report at 1. FEMA rated the relevant proposals as follows:[4]

	Apprio	Leidos[5]
Corporate Experience	Some Confidence	High Confidence
Technical & Mgmt. Plan	Outstanding	Good
Staffing Plan & Key Personnel	Outstanding	Satisfactory
Quality Control Plan	Good	Good
Total Proposed Cost/Price	$96,412,301	$79,735,464
Most Probable Cost/Price	$98,889,539	$79,735,488
Total Evaluated Cost/Price[6]	$109,187,223	$87,965,748

 

AR, Tab AB, Source Selection Decision Document (SSDD) at 4; AR, Tab AF, Business Proposal Evaluation at 3‑6.

The source selection official concluded that Leidos’s proposal represented the best value to the government based on the superiority of Leidos’s technical proposal under the corporate experience factor and the firm’s “substantially lower evaluated cost/price.” AR, Tab AB, SSDD at 1, 5‑6. The source selection official noted that “the additional benefits offered by Apprio for factors 2 and 3 [technical and management plan; and staffing plan and key personnel, respectively] do not warrant a price premium of $21,221,475.” Id. On March 14, the agency issued the task order to Leidos and sent award notifications to the other offerors. AR, Tab AC, Award Document at 1; COS at 6. On March 17, FEMA provided Apprio with a written debriefing. COS at 6; AR, Tab AE, Apprio Debriefing. On March 24, Apprio timely protested the agency’s evaluation and selection decision.[7]

DISCUSSION

Apprio raises three primary challenges to the agency’s evaluation of proposals. First, Apprio challenges FEMA’s evaluation of the realism of Leidos’s proposed costs. Protest at 14‑18; Comments at 11‑23. Next, Apprio challenges the evaluation of its own proposal, asserting that the agency unreasonably assessed a weakness under the corporate experience factor. Protest at 10‑14; Comments at 3‑11. Finally, Apprio challenges FEMA’s evaluation of Leidos’s proposal under the staffing plan and key personnel factor. Protest at 19‑20; Comments at 23‑27. The agency argues that its evaluation of both proposals was conducted reasonably and in accordance with the RFQ. Based on our review of the record, we sustain the protest.[8]

In reviewing protests of the issuance of a task order, our Office will not reevaluate quotations or proposals, but will examine the record to determine whether the evaluation and source selection decision were reasonable and consistent with the solicitation and applicable procurement law and regulation. Ohio KePRO, Inc., B‑417836, B‑417836.2, Nov. 19, 2019, 2019 CPD ¶ 47 at 4. While we will not substitute our judgment for that of the agency, we will question the agency’s conclusions where they are inconsistent with the solicitation criteria, undocumented, or not reasonably based. DRS ICAS, LLC, B‑401852.4, B‑401852.5, Sept. 8, 2010, 2010 CPD ¶ 261 at 5.

In reviewing an agency’s evaluation of proposals, we do not limit our consideration to the contemporaneous record, but instead consider all the information provided, including the parties’ arguments and explanations. MicroTechnologies, LLC, B‑413091, B‑413091.2, Aug. 11, 2016, 2016 CPD ¶ 219 at 5. Although we generally give little weight to reevaluations and judgments prepared in the heat of litigation, see Boeing Sikorsky Aircraft Support, B‑277263.2, B‑277263.3, Sept. 29, 1997, 97‑2 CPD ¶ 91 at 5, post‑protest explanations that provide a detailed rationale for contemporaneous conclusions and simply fill in previously unrecorded details will generally be considered, so long as those explanations are credible and consistent with the contemporaneous record. Remington Arms Co., Inc., B‑297374, B‑297374.2, Jan.12, 2006, 2006 CPD ¶ 32 at 12.

Cost Realism Evaluation

Apprio argues that FEMA unreasonably evaluated the realism of Leidos’s proposed costs. The protester argues that the agency should have found Leidos’s proposed costs to be unrealistically low, and thus should have upwardly adjusted them, based on the firm’s proposed direct labor rates for certain positions and the lack of escalation for those rates during the option years. Protest at 15; Comments at 11‑23. For the reasons explained below, we conclude that FEMA’s cost realism evaluation was not reasonable.

Where an agency intends to issue a task order containing cost‑reimbursable contract CLINs, an offeror’s estimated costs of performing the cost‑reimbursable CLINs are not dispositive because, regardless of the costs proposed, the government is bound to pay the contractor its actual and allowable costs. Ohio KePRO, Inc., supra at 5. Consequently, the procuring agency must perform a cost realism analysis to determine the extent to which an offeror’s proposed costs are realistic for the work to be performed. FAR 15.404‑1(d);Target Media Mid Atlantic, Inc., B‑412468.6, Dec. 6, 2016, 2016 CPD ¶ 358 at 4 (considering FAR part 15 cost realism standards to a FAR part 16 task order procurement). An agency’s realism analysis need not achieve scientific certainty; rather, the analysis must provide a reasonable measure of confidence that the costs proposed are realistic based on information reasonably available to the agency at the time of its evaluation. Smartronix, Inc.; ManTech Advanced Sys. Int’l., Inc., B‑411970 et al., Nov. 25, 2015, 2015 CPD ¶ 373 at 9. Our review of an agency’s cost realism evaluation is limited to determining whether the cost analysis was reasonably based. Id.

Additionally, in order for us to review an agency’s evaluation judgments, the agency must have adequate documentation to support those judgments. Ohio KePRO, Inc., supra at 6‑7. Where an agency fails to document or retain evaluation materials, it bears the risk that there may not be adequate supporting rationale in the record for us to conclude that its judgments were reasonable. Id.; see also Solers Inc., B‑409079, B‑409079.2, Jan. 27, 2014, 2014 CPD ¶ 74 at 9-10.

FEMA’s Contemporaneous Evaluation

Apprio’s primary challenges to the evaluation of the realism of Leidos’s proposed costs concern certain of the awardee’s direct labor rates and the lack of any escalation for those rates for the option years. Although FEMA’s responses to the protest address these arguments, the contemporaneous evaluation record does not demonstrate any evaluation of the essential costs elements challenged by Apprio. For the reasons discussed below, we conclude that the contemporaneous record is inadequate for our Office to conduct a meaningful review of the agency’s evaluation judgments. Based on this record, we are unable to conclude that the agency’s evaluation of Leidos’s proposed costs was reasonable.

The solicitation called for the prospective contractor to provide “training delivery services,” which will require qualified personnel to carry out FEMA’s training mission at CDP. AR, Tab K, PWS at 4. The RFQ required offerors to thoroughly detail their proposed costs‑‑to include direct labor‑‑and required the agency to evaluate offerors’ proposed costs for realism. RFQ at 54‑58, 62‑63.

The record reflects that FEMA found Leidos’s proposed costs realistic. In this regard, the record includes a high-level statement of the agency’s conclusions as follows:

Leidos’s cost proposal is realistic for the work to be performed, reflects the offeror’s understanding of the requirements, and is consistent with the unique methods of performance and materials described in the offeror’s technical proposal. While Leidos’s [firm-fixed-price (FFP)] proposal is [DELETED]% lower than the standard deviation of all proposals (FFP only), the Leidos’s cost proposal is within the standard deviation of all proposals ([cost-plus-award-fee (CPAF)] only). The FFP proposal is not taken into consideration of the offeror’s cost realism in their cost proposal. Additionally, the total number of hours is [DELETED]% low for the base period and [DELETED]% low for the option periods based on the Government’s [independent government estimate (IGCE)].”

Leidos’s probable cost was adjusted upward in the amount of $24.00 to account for an apparent rounding error made to the total cost/price of the Base and all Option Periods in the offeror’s cost/price schedule.

Leidos’s cost proposal was evaluated for Cost Realism and found to be realistic for the work to be performed, reflected the offeror’s understanding of the requirements, and was consistent with the unique methods of performance and materials described in the offeror’s technical proposal. The offeror’s probable cost was adjusted from their proposed cost, as noted above in Table 9.

AR, Tab AF, Business Proposal Evaluation Report at 5‑6, 8; Tab AA, Award Basis (same quotes).

In response to the protest, the contracting officer described the evaluation of Leidos’s costs as follows:

The cost realism analysis included [] Leidos’s proposed labor categories, wage rates, labor hours, indirect rates (fringe, overhead, [general and administrative]), inclusion of Government provided expendables, non-expendables, and travel amounts, and profit (base and award fees).

As detailed in the Business Proposal Evaluation Report (TABs AF and AG), the business proposal from Leidos was evaluated for completeness. Among the elements of completeness was Element # 2, Cost/budget narrative/ breakdown for [cost-plus-fixed-fee] CLINs. This evaluation included five sub-elements consisting of a cost breakdown for Direct Labor, Overhead and General & Administrative, Other Direct Costs (including Government provided “plug-in” numbers), Subcontractors, and Base and Award Fee (separate CLIN, does not include Travel cost). The business proposal from Leidos included all the required cost breakdown elements.

COS at 30.

Aside from the general discussion of the cost elements discussed above, to include the representation of an analysis of Leidos’s wage rates (labor rates) and indirect rates, the contemporaneous record does not reflect any actual consideration of Leidos’s labor rates. The record also does not reflect any evaluation or consideration of the fact that, as discussed below, Leidos’s cost proposal did not include any escalation for its labor rates for the option years of performance. The contracting officer does not specifically explain how the agency evaluated Leidos’s labor rates or how the specific conclusions of those evaluations were made. None of the documents cited by the contracting officer, e.g., the Proposed Cost Spreadsheet or Business Proposal Evaluation report, address labor rates or the escalation of those rates for the option years. See COS at 31. For example, with regard to the evaluation of the completeness of the awardee’s proposed costs, the contracting officer states that the agency found that Leidos’s cost proposal included all cost breakdown elements, but does not state that the agency found individual elements, such as direct labor or escalation to be realistic. Id. (citing AR, Tab AG, Proposed Cost Spreadsheet; Tab AF, Business Proposal Evaluation Report).

As discussed in more detail below, Apprio’s comments on the agency report identified specific examples of the agency’s alleged errors in the evaluation of the awardee’s direct labor rates. FEMA requested an opportunity to respond to the protester’s comments, which our Office granted. While the agency’s response to the protester’s comments provides a detailed rebuttal to the protester’s arguments concerning the awardee’s direct labor rates and escalation, see FEMA Supp. Briefing at 3‑12, the agency’s response does not cite any contemporaneous evaluation of the issues raised, and is not consistent with the contemporaneous record. Instead, the agency primarily cites the awardee’s proposal for the proposition that the costs were adequately justified. Id. at 3‑21. Further, the agency’s response to the protester’s comments does not directly state that the agency performed the analyses that supported the agency’s response. Rather, the agency cites the contracting officer’s statement, quoted above, for the proposition that the agency evaluated the awardee’s direct labor rates and found them realistic. Id. at 3-4 (citing COS at 31). However, like the agency’s supplemental briefing, the contracting officer’s statement does not cite evidence from the contemporaneous record to support the agency’s responses.

In sum, we conclude that FEMA’s response to the protest does not show that the agency evaluated the realism of Leidos’s direct labor rates or the escalation of those rates for the option years. We conclude that the agency’s response to the protester’s arguments relies on judgements that are not found in the contemporaneous record and that the agency does not contend were made contemporaneously. In the following three subsections, we address Apprio’s specific challenges to the evaluation of the awardee’s proposed direct labor rates and escalation. Our discussion below addresses potential cost risks with the awardee’s proposal that the agency did not address in the contemporaneous record.

Wage Determination (WD) Rates Proposed for Incumbents

Apprio argues that FEMA’s cost realism evaluation of Leidos’s proposed costs unreasonably failed to assess whether Leidos’s proposed direct labor rates would be sufficient to retain the proposed incumbent personnel. Comments at 11‑23. The protester argues that Leidos proposed to staff the task order with its incumbent workforce but failed to propose costs that reasonably reflect that approach. Id. Based on this record, we agree with the protester.

The staffing plan and key personnel factor required offerors to describe how they would staff the task order. RFQ at 53. In this regard, the RFQ required business proposals to detail the costs associated with the offeror’s technical approach. Id. at 54-58. As discussed above, the RFQ stated that cost realism would be evaluated by determining if the proposed costs were realistic for the work to be performed, reflected the offeror’s understanding of the requirement, and were consistent with the offeror’s technical approach. Id. at 63.

Leidos’s technical proposal stated: “As the incumbent contractor, the Leidos Team supports the staffing requirements by having all key personnel, required staff, and a cadre of high‑quality instructors, covering every PWS element, available upon award.” AR, Tab V, Leidos Technical Proposal at 34. Leidos’s technical proposal stated that the firm has “letters of commitment from all current staff and instructors[.]” Id. at 37 (emphasis omitted). As Leidos confirms, its proposal “included the names of all incumbent personnel performing requirements under Contracting Line Item Number (“CLIN”) 0007, and provided direct labor rates for each.” Leidos Reply to FEMA Supp. Briefing at 2 (citing AR, Tab W, Leidos Business Proposal at 4-6; Tab X, Leidos Budget Worksheet at Rate Build By CLIN Tab, Direct Rates Tab).

In the narrative explaining its direct labor rates, Leidos stated that it used actual salaries for key personnel and non‑key named personnel where there was [DELETED]. AR, Tab W, Leidos Business Proposal at 15. However, Leidos also proposed several labor categories to be paid at WD rates. Id. at 18‑20. Regarding the labor categories to be paid at WD rates, Leidos’s proposal stated: “Leidos has reviewed the wage determination rates associated with these labor categories and based on an assessment by [Leidos’s human resources] department found that the Department of Labor (DOL) WD rates accurately reflect the prevailing wage rates for these skills in the location specified.” Id. at 18. Leidos priced the labor categories not exempt from the SCA (Service Contract Act) at hourly WD rates for Calhoun, Alabama. Id.

Leidos proposed [DELETED] positions to satisfy CLIN 0007‑‑a cost‑reimbursement CLIN covering instructors, logistical support, overtime, and other direct costs‑‑and proposed to pay [DELETED] of [DELETED] of those positions at WD rates.[9] See AR, Tab X, Leidos Budget Worksheet, Rate Build by CLIN Tab. The labor categories detailed in the rate build tab of Leidos’s business proposal, however, correlate to named individuals listed in the direct rates tab of the business proposal, thus indicating that Leidos proposed to use its incumbent workforce to perform the work. See FEMA Supp. Briefing at 4.

FEMA argues that Leidos’s proposal did not provide a basis to conclude that the proposed labor rates were too low to retain the incumbent staff. FEMA Supp. Briefing at 3, 12. The agency states that Leidos’s proposal indicated that the firm possessed letters of commitment from all current staff and instructors to remain on the contract, and contends that the agency could reasonably rely on that representation. Id. at 11‑12; see also Leidos Reply to FEMA Supp. Briefing at 2 (“the Agency was reasonable in relying on the information Leidos provided in its [proposal] to determine that its [WD] rates were sufficient to retain incumbent personnel who are covered by [Service Contract Labor Standards (SCLS)].”).

Apprio contends that while [DELETED] of the [DELETED] named individuals in Leidos’s business proposal were proposed at direct labor rates based on their incumbent salaries, the vast majority of the named individuals in Leidos’s business proposal were listed to be paid at WD rates governed by the SCLS.[10] See AR, Tab W, Leidos Business Proposal at 18‑20; AR, Tab X, Leidos Budget Worksheet at Rate Build by CLIN Tab, Direct Rates Tab; FEMA Supp. Briefing at 4 (“Leidos listed corresponding individuals from the incumbent contract in the ‘Direct Rates’ Worksheet of Tab X.”). In sum, Apprio argues that the agency’s evaluation failed to assess whether the direct labor rates that were based on the WD rates were realistic given that Leidos had proposed incumbent employees to fill the positions. According to Apprio, a reasonable estimate of Leidos’s costs should have been based on the actual labor costs for these incumbent employees rather than hypothetical WD rates.

Apprio also challenges the agency’s reliance on the awardee’s letters of commitment as a basis to determine that the WD rates were realistic and thus sufficient to retain incumbent staff. Here, Apprio argues that while Leidos may have letters of commitment from its entire incumbent workforce, nothing in the record demonstrates that the incumbents agreed to perform on the contract at WD rates instead of their actual salaries. Apprio Reply to FEMA Supp. Briefing at 8.

Based on our review of the record, we conclude that FEMA’s cost realism evaluation was unreasonable. Leidos’s technical proposal detailed a technical approach offering the incumbent workforce. Although Leidos’s business proposal identified direct labor rates for certain of the named incumbent personnel based on actual salaries, it identified WD rates for the vast majority of named incumbent personnel. The agency’s cost realism evaluation did not address whether the WD rates were the same as the direct labor rates under the incumbent contract, or whether any potential differences between the WD rates and the direct labor rates for these incumbent personnel showed that the WD rates were unrealistic. The agency’s cost realism evaluation therefore did not assess whether the WD rates proposed to be paid to the majority of the incumbent workforce would be sufficient to retain those employees.

In the absence of any documentation showing that the agency evaluated Leidos’s proposed direct labor rates for realism, we cannot conclude that the cost realism evaluation was reasonable. See Ohio KePRO, Inc., supra at 8 (sustaining protest where the contemporaneous record did not reflect an evaluation of the realism of direct labor rates). Accordingly, we sustain the protest on this basis. FAR 15.404‑1(d)(1); Smartronix, Inc.; ManTech Advanced Sys. Int’l., Inc., supra at 9‑10 (failure to evaluate the realism of any of the individual rates for any of the individual labor categories was not reasonable).

Escalation of WD Rates

Apprio argues that FEMA’s cost realism evaluation unreasonably failed to consider the fact that Leidos’s proposal did not apply any escalation to any of the proposed WD rates for the option years, and instead applied the same rate from the base period to all options. Comments at 21‑23; AR, Tab W, Leidos Business Proposal at 22; AR, Tab X, Leidos Budget Worksheet, Direct Rates Tab. Apprio notes that Leidos’s proposal provided for escalation of direct labor rates that are not subject to the SCA. Comments at 21 (citing AR, Tab W, Leidos Business Proposal at 22). Apprio also notes that the IGCE applied an escalation rate to all labor categories. Id. (citing AR, Tab AH, IGCE at Labor Base and Options Tab). Apprio argues that the agency and the awardee therefore both recognized that the cost of labor will increase over the task order’s period of performance and that the agency should have accounted for this probable increase in labor costs when it evaluated Leidos’s WD rates. Id. at 21‑23.

FEMA’s defense is that Leidos’s decision not to escalate its WD rates was consistent with FAR clause 52.222‑43, which was incorporated into the RFQ. FEMA Supp. Briefing at 19‑20; see RFQ at 22, 57. The clause provides that “[t]he contract price, contract unit price labor rates, or fixed hourly labor rates will be adjusted to reflect the Contractor’s actual increase or decrease in applicable wages and fringe benefits” resulting from, for example, changes to Department of Labor wage determinations. FAR clause 52.222‑43(d). In support of its position, FEMA points to language in the clause which says: “The Contractor warrants that the prices in this contract do not include any allowance for any contingency to cover increased costs for which adjustment is provided under this clause.” FEMA Supp. Briefing at 18‑19 (citing FAR clause 52.222‑43(b)).

First, the agency’s reliance on FAR clause 52.222-43 is misplaced. Section 22.1006(c) of the FAR requires solicitations to include FAR clause 52.222-43 for contracts expected to be awarded on a “fixed-price, time-and-materials, or labor-hour service contract” basis. By its terms, the purpose of the clause is to enable a contractor to obtain upward adjustments to its “price” for increased labor costs in what is otherwise a fixed-price contract. The clause would have no applicability to a cost-reimbursable contract because the government is already bound to pay the contractor’s actual and allowable costs, which can include increased labor costs. Ohio KePRO, Inc., supra at 5. Here, the solicitation includes both fixed-price CLINs and cost-reimbursable CLINs. Given the FARs instructions regarding the applicability of the clause and its purpose, the clause can only reasonably be understood to apply to the solicitation’s fixed-price CLINs, not the cost CLINs, which are the ones at issue, such as CLIN 0007. The agency offers no explanation of why FAR clause 52.222-43 applies to the cost‑plus-award-fee CLINs.

Second, as explained above, the purpose of an agency’s cost realism evaluation is to consider whether the costs proposed are realistic for the work to be performed as proposed by the contractor. FAR 15.404‑1(d)(1). Again, regardless of the costs proposed, the government is bound to pay the contractor its actual and allowable costs. Ohio KePRO, Inc., supra at 5. As a result, an agency must consider the realistic costs to perform in the option years. A wage determination is not a substitute for this analysis because a wage determination reflects the minimum wages or fringe benefits to which an employee may be entitled whereas a realism assessment determines whether a cost is realistic for the work to be performed by the individual contractor. In the absence of any contemporaneous documentation in the record supporting a conclusion that the agency considered the realism of potential labor cost increases in the option years, we are unable to find the evaluation reasonable. Accordingly, we sustain the protest on this basis.

Standard Deviation Methodology to Evaluate Cost Realism

The agency’s cost evaluation relied in part on a standard deviation analysis to assess the realism of offerors’ proposed costs. In this regard, the agency found that Leidos’s proposed costs were realistic, in part, because they were “within the standard deviation” of all proposals. AR, Tab AA, Award Basis at 40‑46. Apprio argues that FEMA’s application of a standard deviation analysis to assess the realism of offerors’ costs was unreasonable because it did not account for the competing offerors’ differing technical approaches. Comments at 15‑16; id. at 15 n.5. We agree.

The agency describes its standard deviation analysis as follows. For each proposal, FEMA added together the proposed costs for only the cost‑reimbursement CLINs to arrive at the respective offeror’s total proposed cost. See COS at 32; AR, Tab AA, Award Basis at 40‑46. FEMA then calculated the mean of the total proposed costs for all competing offerors. COS at 32. This value was found to be $53,329,577. Id. FEMA then calculated the value of one standard deviation from the mean. Id. This value was found to be $7,814,733. Id. FEMA then calculated a range which represented one standard deviation under the mean, and one standard deviation over the mean. Id. This range was $45,514,844 to $61,144,311. Id. The agency’s response to the protest suggests that if an offeror’s total proposed cost fell somewhere within that range, the offeror’s proposed costs were found to be realistic. Id.; Memorandum of Law (MOL) at 12; see also AR, Tab AA, Award Basis at 40‑46. Here, we conclude that FEMA’s standard deviation analysis was not sufficient to evaluate cost realism.

Our Office has explained that the use of standard deviation methodology as a tool in determining the realism of proposed labor rates is not per se objectionable where the solicitation does not require offerors to propose unique staffing approaches. See e.g., Booz Allen Hamilton, Inc., B‑414283, B‑414283.2, Apr. 27, 2017, 2017 CPD ¶ 159 at 10. However, where, as here, a solicitation invites unique approaches, using a standard deviation analysis that does not include an assessment of offerors’ unique technical approaches and the costs associated with them, produces an unreasonable result.

Although FEMA discussed its standard deviation analysis to support its position that the cost realism evaluation was reasonable, the agency does not specifically explain why use of a standard deviation analysis was reasonable in this procurement. MOL at 12‑17; COS at 32. The agency’s approach to evaluating cost realism disregards the obligation to consider “the unique methods of performance and materials” described in offerors’ technical approaches.[11] FAR 15.404‑1(d)(1); RFQ at 63. Therefore, FEMA’s reliance on the standard deviation analysis for its cost realism analysis was unreasonable.

In sum, we conclude that the record does not show that the agency reasonably evaluated the realism of Leidos’s proposed direct labor rates that were based on WD rates. We also conclude that the agency’s realism assessment failed to account for the lack of escalation of those rates for the option years. Apprio contends that the adjustment to the WD rates and escalation should have resulted in an upward adjustment to Leidos’s evaluated costs of “at least $11 million.” Comments at 2, 22. In light of the lack of a contemporaneous record concerning the evaluation of Leidos’s direct labor rates, we have no basis to know what potential cost realism adjustments may be required regarding the awardee’s proposed costs. For these reasons, as well as others discussed below, we conclude that the protester was prejudiced by the agency’s flawed evaluation and therefore sustain the protest.

Technical Evaluation Challenges

Apprio raises two challenges to FEMA’s evaluation of proposals under the technical factors. First, the protester challenges the agency’s evaluation of its own proposal under the corporate experience factor. Second, the protester challenges the agency’s evaluation of Leidos’s proposal under the staffing plan and key personnel factor. As discussed below, we find that the agency’s evaluation of Apprio’s proposal under the corporate experience factor was unreasonable. However, we dismiss the remaining challenge for failing to state a valid basis of protest.

Corporate Experience Evaluation

As discussed above, the procurement used a two‑phase evaluation. RFQ at 49, 59. The most important evaluation factor, corporate experience, was evaluated in phase one. Id. at 50‑51, 60. Under the corporate experience factor, offerors were required to supply three to five examples of their corporate experience. Id. at 50. Consideration was to be given to the experience of the prime offeror, a predecessor company, major subcontractors, contract team arrangements or joint ventures, or an offeror’s parent or affiliate. Id. at 60. The RFQ stated: “To be viable, Offerors shall have the necessary specialized experience to provide: (1) specialized training delivery in the deterrence, prevention, and response to [chemical, biological, radiological, nuclear and explosive (CBRNE)], mass casualty incidents and (2) training delivery involving hazardous materials by providing advanced, hands on, and classroom‑based state‑of‑the‑art training.” Id.

Apprio submitted five project examples to satisfy the corporate experience factor. AR, Tab N, Apprio Phase One Proposal at 1‑15. The agency assigned Apprio’s proposal an overall rating of some confidence under this factor. AR, Tab P, Phase‑One TET Report at 1. In contrast, the agency evaluated Leidos’s proposal as high confidence. Id. at 8.

In evaluating Apprio’s proposal under the corporate experience factor, the agency found three strengths and one weakness. Id. at 1. The weakness reads as follows: “The Offeror’s description of work and responsibilities, while necessary and essential to the functioning of training provided at the CDP, are not the same as the delivery of instruction. (reference projects 1 – 5).” Id. at 2‑3. In conducting the best‑value tradeoff, the source selection official wrote of Apprio’s proposal: “The TET identified one weakness which applied to all five experience projects provided by the Offeror. Apprio’s description of work and responsibilities, while necessary and essential to the functioning of training provided at the CDP, are not the same as the delivery of instruction.” AR, Tab AB, SSDD at 5.

Apprio argues that the agency unreasonably found that its proposal merited a weakness because it did not demonstrate experience with the delivery of instruction under any of the five experience projects. Protest at 10‑14; Comments at 3‑11. Apprio explains that it submitted five projects in response to the corporate experience factor, and that three of the five projects “explicitly involved the delivery of instruction.” Comments at 4.

For example, Apprio contends that its second project demonstrates that the firm performed on a federal contract as a subcontractor and includes a description of work and responsibilities that states in relevant part: “During the contract period of performance, Apprio delivered more than [DELETED] courses.” AR, Tab N, Apprio Phase One Proposal at 4. As a second example, Apprio contends that its third project demonstrates that its proposed subcontractor for the instant procurement performed on a federal contract for the [DELETED] which included “[d]elivery of a single Concepts of Real‑World CBRNE Response training course[.]” Id. at 7. As a final example, Apprio contends that its fourth project demonstrates that its proposed subcontractor performed on a federal contract for the [DELETED] which included “providing a battalion‑sized field training exercise [] designated as the battalion’s validation and certification exercise.” Id. at 10. This project example explains that the exercise “exposes the [DELETED] and [DELETED] to best practices in evolving tactics, techniques, and procedures [] during [DELETED] lanes training.” Id.

The agency argues that the weakness assessed to Apprio’s proposal under the corporate experience factor was reasonable. COS at 19‑23; MOL at 4‑9. In responding to the protest, the agency argues that the weakness was appropriately assigned because the projects identified by the protester had additional issues:

• Project No. 2 “was not for the delivery of instruction in the deterrence, prevention, and response to Chemical, Biological, Radiological, Nuclear and Explosive (CBRNE), mass casualty incidents or training delivery involving hazardous materials[.]”

• Project No. 3 “did provide some training delivery in CBRNE[.]” However, the project “[did] not meet the RFQ requirement of size, scope, and complexity to the immediate requirement.”

• Project No. 4 “did provide some training delivery in CBRNE[.]” However, the project “[did] not meet the RFQ requirement of size, scope, and complexity to the immediate requirement.”

COS at 20‑21.

Based on our review of the record, we find that the agency’s evaluation of Apprio’s proposal under the corporate experience factor was unreasonable. The contemporaneous record shows that the agency assessed a weakness for Apprio’s supposed failure to demonstrate its experience with the delivery of instruction in all five of its experience examples, and that this weakness factored into the best‑value decision. AR, Tab P, Phase‑One TET Report at 2‑3; AR, Tab AB, SSDD at 5. As cited above, the weakness is directly contradicted by the contents of Apprio’s proposal.

We conclude that the agency’s post hoc justifications for the weakness are not consistent with the contemporaneous record. We agree with the agency that the RFQ required experience examples to address CBRNE experience, and to be of similar size, scope, and complexity to the work described in the solicitation. See RFQ at 60. However, the issues cited by the agency in response to the protest offer three new and different rationales for the assessment of the weakness as compared to the technical evaluation report. The agency does not establish that the post hoc responses to the protest concerning size, scope, and complexity reflect the agency’s contemporaneous evaluation.

Because FEMA assessed a weakness to Apprio’s proposal which is directly contradicted by the contents of the proposal, we conclude that the weakness was unreasonable. The record reflects that the source selection official relied upon this unreasonable assessment of the weakness in making the best‑value decision. Accordingly, we sustain this protest ground.

Staffing Plan and Key Personnel Evaluation

Finally, Apprio challenges FEMA’s evaluation of Leidos’s proposal under the staffing plan and key personnel factor. Protest at 19‑20; Comments at 23‑27. The protester argues that it was unreasonable for the agency to rate Leidos’s proposal as satisfactory under this factor because “in order to achieve its low price, Leidos must have proposed unqualified personnel, an insufficient number of labor hours, and/or significantly below market salaries” in its business proposal. Protest at 20. Apprio further argues that the agency “failed to recognize that although Leidos’[s] Technical proposal states that Leidos will staff the entire task order with incumbent personnel from Leidos’[s] current contract, Leidos’[s] Business proposal indicates a radically different approach.” Comments at 23. On this basis, the protester argues that FEMA should have rated Leidos’s proposal as unsatisfactory under the staffing and key personnel factor. Id. at 27.

We dismiss this ground for failing to state a valid basis of protest. 4 C.F.R. §§ 21.1(c)(4), (f); § 21.5(f). The RFQ did not require the type of evaluation proffered by the protester here. Nothing in the plain terms of the solicitation required the agency to consider an offeror’s business proposal during the technical evaluation of the staffing and key personnel factor. See RFQ at 61‑62. Since the agency was not required to consider pricing when it conducted the technical evaluation of the staffing plan and key personnel factor, Apprio’s argument does not provide our Office with any legal basis on which to sustain the protest ground.[12]

Prejudice

Competitive prejudice is an essential element of a viable protest. MicroTechnologies, LLC, supra at 15. We find that the agency’s evaluation of the awardee’s proposal for cost realism was unreasonable, as was the agency’s evaluation of the protester’s proposal under the corporate experience factor. These errors affected the agency’s best‑value decision.

In this regard, the source selection decision noted that while Leidos’s proposal was more highly rated under the corporate experience factor, Apprio’s proposal was more highly rated under the technical and management plan and the staffing plan and key personnel factors. See AR, Tab AB, SSDD at 5. The agency did not specifically state that Leidos’s technical proposal was more highly rated than Apprio’s proposal, considering all of the non-price factors. Instead, the source selection decision explained that the benefits offered by Apprio did not justify the price premium of $21,221,475. Id. Based on the identified errors, we are unable to conclude that the total evaluated cost/price relied on by the source selection official accurately reflects Leidos’s total probable cost of performance. We are also unable to conclude that the source selection official would not have rated Apprio’s proposal differently in the absence of the unreasonable assessment of the weakness. But for the agency’s actions, we conclude that the protester would have a substantial chance for award. Accordingly, we find that Apprio was competitively prejudiced.

RECOMMENDATION

We recommend that FEMA conduct a new cost realism evaluation consistent with the terms of the RFQ, the FAR, and the issues identified in this decision, and adequately document the results of that evaluation. We also recommend that FEMA conduct a new evaluation of Apprio’s proposal under the corporate experience factor consistent with the terms of the RFQ and the issues identified in this decision, and adequately document the basis for any evaluation findings. FEMA should make a new source selection decision based on the results of the new evaluations. If Leidos is not found to offer the best value to the government, we recommend terminating Leidos’s task order for the convenience of the government and issuing the task order to the firm offering the proposal found to be the best value.

We also recommend that Apprio be reimbursed the reasonable costs of filing and pursuing its protest, including attorneys’ fees. 4 C.F.R. § 21.8(d)(1). Apprio should submit its certified claims for costs directly to the agency within 60 days after receipt of this decision. Id. § 21.8(f)(1).

The protest is sustained.

Edda Emmanuelli Perez
General Counsel

 

---