Criterion Systems, Inc.

Decision

Matter of:  Criterion Systems, Inc.

File:  B-416553; B-416553.2

Date:  October 2, 2018

DIGEST

Challenge to agency’s evaluation and award decision is denied where the evaluation was reasonable and consistent with the solicitation’s evaluation criteria.

Criterion Systems, Inc., a small business located in Vienna, Virginia, protests the issuance of a Federal Supply Schedule (FSS) task order to Edgewater Federal Solutions, of Ijamsville, Maryland, by the Department of Energy (DOE) pursuant to request for quotations (RFQ) No. DE-SOL-0010923 for application, infrastructure, and cyber security support for the DOE’s Office of Environment, Health, Safety, and Security (EHSS).[1]  The protester challenges the agency’s evaluation and award decision.

We deny the protest.

The DOE issued the RFQ on December 8, 2017, pursuant to the procedures of Federal Acquisition Regulation (FAR) § 8.405-2, to small business vendors holding contracts under GSA schedule 70 (the general purpose commercial information technology equipment, software, and services schedule).  RFQ at 203-4[2]; AR, Tab B.2, SSD, at 1.  The solicitation sought a vendor to provide technical and administrative support resources to develop and maintain applications and infrastructure supporting the mission and programmatic requirements of EHSS.  AR, Tab B.2, SSD, at 2.  The solicitation contemplated the issuance of a hybrid time-and-materials and fixed-price task order consisting of a 1-year base period and four 1-year option periods.  RFQ at 158.  The RFQ provided for selection of the best‑value vendor based on the following factors, which were listed in descending order of importance:  technical understanding, corporate experience, past performance, and price.  Id. at 204.

The solicitation provided that the agency’s evaluation of a vendor’s technical understanding would consider five elements including:  technical approach, key personnel, staffing plan, quality assurance plan, and transition plan.  Id. 205-6.  The agency would assign one adjectival rating for technical understanding overall based on its evaluation of the five elements.[3]  Id. at 205.  The agency’s evaluation of corporate experience would also assign an adjectival rating based on the agency’s assessment of a vendor’s recent and relevant experience for the references provided under the past performance factor.  Id. at 204, 206.  Under corporate experience, the agency would evaluate these references and the vendor’s narrative for experience working with classified networks and performance of services similar to those sought in the performance work statement’s (PWS) six work areas:  (1) information technology (IT) project planning and management; (2) application design, engineering, development, testing, implementation, and operations and maintenance; (3) web site; (4) EHSS information services--help desk and end user support; (5) cyber security support; and (6) classified local area network operations.  Id. at 207.  The solicitation provided that relevant contracts must have been performed within three years of the release date of the RFQ, and be similar in size, scope, and complexity.  Id. at 206-7.  The RFQ defined size as “dollar value ($10,000,000) and contract duration,” and scope and complexity as “whether the work bears a material similarity to the work listed in the PWS for this RFQ.”  Id. at 206.

The past performance evaluation would consider a vendor’s performance under existing and prior contracts that have been completed within three years of the release date of the RFQ for similar products or services.  Id. at 207.  The evaluation would focus on information demonstrating the quality of performance relative to the size, scope, and complexity of the current solicitation.  Id.  The solicitation defined size, scope, and complexity for past performance as it had for corporate experience.  Id.  The solicitation advised that past performance would be rated as follows:  favorable, unfavorable, or neutral.[4]  Id.

The agency received seven quotations by the January 22, 2018 closing date.  AR, Tab B.2, SSD, at 1.  The agency’s technical evaluation committee (TEC) evaluated five of the quotations, including Criterion’s and Edgewater’s.[5]  AR, Tab B.1, TEC Report.  The TEC’s evaluation resulted in the following ratings for Criterion and Edgewater:

	Technical Understanding	Corporate Experience	Past Performance	Total Price[6]
Criterion	Good	Outstanding	Favorable	$43,715,452
Edgewater	Good	Good	Favorable	$42,611,257

Id. at 37; Tab B.2, SSD, at 3.

The SSA reviewed the results of the TEC’s findings and concurred with the results.  AR, Tab B.2, SSD, at 3.  The SSA conducted a comparative assessment of the quotations and concluded that Edgewater provided the best value to the agency.  Id. at 4-7.  As between Criterion and Edgewater, the SSA found that the two vendors provided technical approaches that were “essentially equal in merit.”  Id. at 5.  The SSA also acknowledged Criterion’s outstanding rating versus Edgewater’s good rating in corporate experience.  Id.  The SSA concluded however that Criterion’s corporate experience was not much of a discriminator over that of Edgewater’s corporate experience due to their similar experience and strengths.  Id.  The SSA found overall that the minor technical advantages in Criterion’s quotation did not warrant the price premium ($1,104,194 or 2.59 percent).  Id.  Thus, the SSA concluded that Edgewater provided the best-value quotation.  Id. at 5, 7.

On June 29, 2018, Criterion received notice of the award to Edgewater and a brief explanation of the award decision.  Protest, attach. No 2, Award Notice and Brief Explanation, at 1-2.  Criterion filed this protest on July 6.

DISCUSSION

Criterion challenges the agency’s evaluation of quotations and the best-value award decision.  The protester argues that the agency misevaluated quotations under the technical understanding factor because it improperly elevated the importance of the technical approach element.  Criterion also asserts that the agency’s evaluation of Edgewater’s corporate experience and past performance is flawed because the awardee could not provide any relevant contract references.[7]  The protester also argues that the best-value decision was unreasonably based on errors in the evaluation and failed to consider the qualitative differences in the quotations.  Criterion contends that had the agency conducted a proper evaluation and best-value tradeoff, it would have received the award. 

Where, as here, an agency issues an RFQ under FAR subpart 8.4 and conducts a competition, see FAR § 8.405-2, we will review the record to ensure that the agency’s evaluation is reasonable and consistent with the terms of the solicitation.  DEI Consulting, B-401258, July 13, 2009, 2009 CPD ¶ 151 at 2.  Based on our review of the record, we find no basis to sustain the protest.  While we do not address each of the protester’s allegations, we have reviewed them all and find the agency’s evaluation and award decision reasonable and consistent with the solicitation’s evaluation criteria.

Technical Understanding

Criterion argues that the agency improperly elevated the importance of the technical approach element under the technical understanding factor.  The protester also argues that it was unreasonable for the agency to assign the same rating to its quotation and Edgewater’s given Criterion’s quotation contained multiple strengths across four of the five technical understanding elements, while Edgewater’s strengths were all assigned under one element.  We find the agency’s evaluation unobjectionable.

In reviewing an agency’s technical evaluation of vendor submissions under an RFQ, we will not reevaluate the quotations; we will only consider whether the agency’s evaluation was reasonable and in accord with the evaluation criteria listed in the solicitation and applicable procurement statutes and regulations.  HP Enterprise Servs., LLC, B‑411205, B-411205.2, June 16, 2015, 2015 CPD ¶ 202 at 5.  A protester’s disagreement with the agency’s judgment, without more, does not establish that an evaluation was unreasonable.  DEI Consulting, supra, at 2.

Under the technical understanding factor, the agency would evaluate a vendor’s quotation based on five elements including:  technical approach, key personnel, staffing plan, quality assurance plan, and transition plan.  RFQ at 205-6.  The record reflects that a good rating was assigned to Criterion’s and Edgewater’s technical understanding based on the TEC’s consideration of multiple strengths in each vendor’s proposal.[8]  AR, Tab B.1, TEC Report, at 3, 24.  With respect to Criterion’s quotation, the TEC identified seven strengths (under the technical approach, key personnel, and transition plan elements) and no weaknesses.  Id. at 3.  With respect to Edgewater’s quotation, the TEC identified two significant strengths and one strength (under the technical approach element), and no weaknesses.  Id. at 24.

We find no basis to conclude that the agency improperly elevated the importance of the technical approach element or unreasonably assigned an equivalent good rating to both quotations under the technical understanding factor.  The TEC properly evaluated the quotations based upon their unique approaches and rated the quotations based on the subjective quality and nature of each significant strength/strength.  That Criterion’s strengths were spread across multiple elements under the technical understanding factor and Edgewater’s significant strengths and strengths focused on one element does not demonstrate an improper emphasis on the technical approach element or an unequal evaluation.  Instead, it demonstrates that Edgewater’s quotation was particularly strong in one area, a notion reasonably reflected in the agency’s evaluation.  Amyx Inc., B-410623, B-410623.2, Jan. 16, 2015, 2015 CPD ¶ 45 at 13.  Moreover, adjectival ratings are guides for intelligent decision making, SENTEL Corp., B-407060, B-407060.2, Oct. 26, 2012, 2012 CPD ¶ 309 at 5, and the record here reflects that, ultimately, the SSA looked beyond the ratings in making his award determination.  AR, Tab B.2, SSD, at 5-7.  Thus, we have no basis to question the agency’s evaluation of the vendors’ technical understanding, which was consistent with the RFQ evaluation criteria.[9]

Past Performance and Corporate Experience

Criterion challenges the agency’s evaluation of Edgewater’s past performance and corporate experience.  The protester alleges that the agency’s evaluation was unreasonable because Edgewater has no relevant experience.  Thus, Criterion contends that the agency’s assignment of a favorable past performance rating and a good corporate experience rating was in error.  We find the agency’s evaluation reasonable.

An agency has broad discretion, when evaluating offerors’ experience and past performance, to determine whether a particular contract is relevant to an evaluation of experience.  See L&J Bldg. Maint., LLC, B-411827, Oct. 27, 2015, 2015 CPD ¶ 344 at 3.  Our Office will examine an agency’s evaluation of a vendor’s experience only to ensure that it was reasonable and consistent with the stated evaluation criteria and applicable statutes and regulations since determining the relative merit or relative relevance of a vendor’s performance history is primarily a matter within the agency’s discretion.  United Facility Servs. Corp. d/b/a EASTCO Bldg. Servs., B‑408749.2, Jan. 17, 2014, 2014 CPD ¶ 35 at 4. 

Under the past performance factor, the RFQ provided that a vendor would be evaluated on its performance under existing and prior contracts that have been completed within the past three years for similar products or services.  RFQ at 207.  Under the corporate experience factor, the RFQ provided that a vendor would be evaluated on its recent and relevant experience for the same contracts or projects referenced in its past performance information submissions (attachment D to the RFQ).  Id. at 206.  Using these three references the agency would evaluate corporate experience to determine the vendor’s classified network experience and experience in the six work areas identified in the PWS.  Id. at 207.  The RFQ defined relevant contracts for both the past performance and corporate experience factors as contracts that are similar in size, scope, and complexity.  Id. at 206-7.  Size was defined as contracts with a dollar value of $10 million and similar contract duration; scope and complexity were defined as work that bears a material similarity to the work listed in the PWS.  Id. 

Edgewater provided three recent references for the past performance and corporate experience factors.  AR, Tab B.1, TEC Report, at 30-31; Tab D.1, Criterion Quotation Vol. 1, at 53-58; Tab D.2, Criterion Quotation Vol. 2, at 1-19.  The first reference was for work performed under a contracting teaming agreement (of which Edgewater was a prime member) with a value of $81 million performed for the National Nuclear Security Administration (NNSA) Office of Chief Information Officer.  AR, Tab D.1, Criterion Quotation Vol. 1, at 53-54; Tab D.2, Criterion Quotation Vol. 2, at 2.  The second reference was for work performed for another contractor which manages and operates the DOE’s Waste Isolation Pilot Plant site for DOE.  AR, Tab D.1, Criterion Quotation Vol. 1, at 55-56; Tab D.2, Criterion Quotation Vol. 2, at 3.  This reference had a value of $24 million and involved information technology support.  Id.  The third reference was for work under a contracting teaming agreement (of which Edgewater was a prime member) for the DOE Chief Financial Officer with a value of $17 million.  AR, Tab D.1, Criterion Quotation Vol. 1, at 57-58; Tab D.2, Criterion Quotation Vol. 2, at 4.  The agency evaluated these references and concluded that the references were similar in size, scope, and complexity.  AR, Tab B.1, TEC Report, at 30-31.  The TEC assigned a favorable past performance rating.  Id. at 31.  Under the corporate experience factor, the TEC assigned a good rating with three significant strengths related to Edgewater’s experience for the DOE Office of Chief Financial Officer under PWS work areas one (IT project planning and management), two (application design, engineering, development, testing, implementation, and operation and maintenance), and four (EHSS information services-help desk and end use support).  Id. at 30.  In this regard, the TEC noted Edgewater performed directly relevant tasks in the same DOE EHSS IT environment as the current work.  Id.

On this record, we have no basis to question the agency’s evaluation of Edgewater’s past performance or corporate experience.  Contrary to the protester’s allegation that Edgewater has no relevant experience, the record demonstrates that Edgewater provided three recent and relevant references, which were evaluated by the agency.  Accordingly, we find the agency’s evaluation reasonable.

In its comments, Criterion alleges for the first time that the agency failed to reasonably consider the references provided by Edgewater and failed to document its evaluation.  In this regard, the protester alleges that the agency improperly credited Edgewater for work performed on specific contracts as a team member, or for another contractor, without analyzing what portion of the work Edgewater performed.  Criterion also argues that Edgewater does not have required core software competencies because the awardee’s quotation is void of any references to experience with many of the core competencies listed in the PWS.  Absent any reference to these core competencies, Criterion argues that the agency could not have found Edgewater’s quotation demonstrated corporate experience of similar scope and complexity to the functions of the PWS.  The protester also alleges that the agency failed to adequately document its evaluation.  Here, we find the protester’s allegations with respect to the agency’s evaluation of Edgewater’s specific references and documentation to be untimely. 

In accordance with our bid protest procedures, the agency engaged in early document production during the protest process.  In this regard, after the protest was filed on July 6, the agency provided all documents relevant to the protest allegations on July 15.  These documents included a copy of Edgewater’s quotation, the TEC report, and the SSD.  On July 23, Criterion filed a supplemental protest, based on these documents, arguing that the agency’s evaluation under the technical understanding factor was flawed because it improperly elevated the importance of the technical approach factor, which led to an unreasonable rating of good for Edgewater’s quotation.  The protester argued that the quotations of Criterion and Edgewater could not receive equivalent ratings because Criterion was assessed seven strengths across four of the five technical understanding elements, while all of Edgewater’s significant strengths and strengths were assessed under only one element.  Criterion also argued that the source selection authority failed to consider the substantive differences between Criterion and Edgewater under the technical understanding factor when making the source selection decision.  This supplemental protest did not however raise any allegations with respect to the past performance or corporate experience factors.  Rather, Criterion’s supplemental protest stated, “Criterion will address these bases of protest in its comments on the Agency Report when filed.”  Supp. Protest at 2.  Criterion then filed its comments on August 6, which challenged the agency’s evaluation of Edgewater’s specific references and lack of documentation.

In response to Criterion’s comments, the agency sought dismissal of the protester’s allegations that the agency failed to reasonably consider the references provided by Edgewater and failed to document its evaluation.  In this regard, the agency argues that these allegations present new arguments with respect to Edgewater’s specific references, the agency’s evaluation of the specific references, and the agency’s documentation of the evaluation.  The DOE contends that since the protester knew of these new allegations after receiving the agency’s early document production on July 15, it was required to file any supplemental protest within 10 days of receipt of these documents--by July 25. 

Our Bid Protest Regulations, 4 C.F.R. § 21.2(a)(2), require that protests other than those challenging the terms of a solicitation be filed within 10 days of when a protester knew or should have known of its basis for protest.  Further, where a protester initially files a timely protest, and later supplements it with new grounds of protest, the later‑raised allegations must independently satisfy our timeliness requirements, since our Regulations do not contemplate piecemeal presentation or development of protest issues.  Epsilon Sys. Sols., Inc., B-409720, B-409720.2, July 21, 2014, 2014 CPD ¶ 230 at 11 n.7.

Here, the agency produced Edgewater’s quotation, the TEC consensus evaluation, and the SSD on July 15.  Under our Bid Protest Regulations, any arguments based on these documents must then have been filed with our Office by July 25.  Since the protester waited until August 6 to raise these new allegations in its comments, we agree with the agency that these allegations provide new and independent supplemental protest bases, which should have been filed within 10 days after receipt of the agency’s early document production.  Thus, we dismiss as untimely the supplemental protest allegations filed for the first time in Criterion’s comments.

While Criterion argues that the allegations raised in its comments are not new grounds of protest, we disagree.  Allegations raised during the course of a protest constitute new protest grounds when the later-raised allegations are independent from, and provide no support for, the initial protest grounds. Medical Staffing Solutions USA, B-415571, B‑415571.2, Dec. 13, 2017, 2017 CPD ¶ 384 at 3-4.  Moreover, where a protester files a broad initial allegation and later supplements that broad allegation with allegations that amount to specific examples of the initial, general, challenge, these specific examples must independently satisfy our timeliness requirements where such examples involve different factual circumstances that require a separate explanation or defense from the agency; this is because our regulations do not contemplate the piecemeal presentation of protest arguments.  Savannah River Tech. & Remediation, LLC; Fluor Westinghouse Liquid Waste Servs., LLC, B-415637 et al., Feb. 8, 2018, 2018 CPD ¶ 70 at 6; Vigor Shipyards, Inc., B‑409635, June 5, 2014, 2014 CPD ¶ 170 at 5.

The arguments raised in Criterion’s comments constitute new protest grounds because the later-raised allegations are independent of the original protest grounds.  Whereas the original protest allegations were predicated on the assertion that Edgewater lacked any past performance or corporate experience of the relevant size, the allegations raised in the comments are predicated on the agency allegedly incorrectly evaluating Edgewater’s specific past performance and corporate experience references by failing to consider the portion of the work performed by Edgewater, failing to recognize Edgewater did not have certain core software competencies, and failing to adequately document its evaluation conclusions.  These later-raised allegations provide no support for the original protest grounds because these allegations do not support the protester’s initial allegation that Edgewater lacked any relevant experience.  Medical Staffing Sols. USA, supra.

While Criterion is correct in observing that its new protest grounds are related to its original protest grounds, a strong factual nexus between protest grounds does not change the independent legal nature of the allegations.  See Ti Hu, Inc., B-284360, Mar. 31, 2000, 2000 CPD ¶ 62 at 4 (protester’s subsequent allegation constituted a supplemental protest ground, even though both allegations were predicated on the agency’s past performance evaluation).  Accordingly, Criterion’s allegations raised in its comments constitute new protest grounds, and are dismissed as untimely since the protester raised those protest grounds more than 10 days after it knew or should have known of them.[10]  CH2M Hill Antarctic Support, Inc., B-406325 et al., Apr. 18, 2012, 2012 CPD ¶ 142 at 13 (fact that protester receives documents as part of early document production does not suspend application of GAO timeliness rules).

Source Selection Decision

Criterion contends that the agency’s best-value tradeoff decision was improperly based on an erroneous evaluation.  As described above, since the record does not support the protester’s challenges to the agency’s evaluation, we find no merit to Criterion’s objection to the agency’s selection decision based upon alleged errors in the underlying evaluation of the proposals.

The protester also argues that the SSA’s decision failed to consider the substantive differences in the proposals and unreasonably concluded that the benefits of Criterion’s proposal were not worth the price premium.

Where, as here, a procurement conducted pursuant to FAR subpart 8.4 provides for issuance of an FSS task order on a best-value tradeoff basis, it is the function of the SSA to perform a price/technical tradeoff; that is, to determine whether one quotation’s technical superiority is worth its higher price.  VariQ Corp., B-409114 et al., Jan. 27, 2014, 2014 CPD ¶ 58 at 14.  Even where a solicitation issued under FAR subpart 8.4 emphasizes technical merit over price, an agency properly may select a lower-priced, lower-rated quotation if the agency reasonably concludes that the price premium involved in selecting a higher-rated, higher-priced quotation is not justified in light of the acceptable level of technical competence available at a lower price.  Id. at 15.  The extent to which technical superiority is traded for a lower price is governed only by the test of rationality and consistency with the stated evaluation criteria.  Id.

Based on the record before us, we conclude that the SSA’s decision to select Edgewater’s quotation was reasonable and consistent with the evaluation criteria.  The contemporaneous record shows that the SSA compared Criterion’s and Edgewater’s quotations under each factor.  AR, Tab B.2, SSD, at 5-6.  The SSA considered the specific strengths assigned to both of the quotations under the technical understanding factor and found that the quotations were essentially equal in merit.[11]  Id. at 5.  In this regard, the SSA noted that both vendors offered advantages to the agency.  Id.  For example, the SSA valued Criterion’s [DELETED] and [DELETED].  Id.  The SSA also valued Edgewater’s understanding of the criticality of prioritizing security during the system development life cycle process, and the vendor’s plan for [DELETED] of key policies, procedures, and processes, as well as developing a [DELETED].  Id.  The SSA also recognized the evaluated superiority of Criterion’s quotation under the corporate experience factor and assessed the strengths assigned to both quotations.  Id.  The SSA concluded that both Criterion and Edgewater identified corporate experience with DOE or NNSA contracts, and noted that DOE experience is very relevant and will reduce the time spent learning DOE structure and processes.  Id.  Thus, the SSA found that “[d]ue to their similar experience and strengths, I do not find Criterion’s Corporate Experience to be much of a discriminator over that of Edgewater.”  Id. 

Finally, the SSA concluded that Criterion’s quotations did not warrant a price premium as a result of its minor technical advantages.  Id.  In this regard, the SSA stated that the additional significant strength and strengths that Criterion received for its corporate experience did not warrant an additional $1,104,194 in price.  Id.  The SSA provided that while Criterion’s corporate experience is noteworthy in the areas of [DELETED], Edgewater also demonstrated recent and relevant corporate experience on DOE and NNSA contracts in areas that directly correlate to the present requirements.  Id.  Accordingly, the SSA concluded that Edgewater’s quotation provided the best value, and selected it for award.  Id. at 7.

The contemporaneous record thus reflects both an understanding of the relevant importance of the factors, judgment about the degree to which each vendor’s quotation provided value, a reasonable comparison of the quotations under the evaluation criteria, and a documented tradeoff decision to select Edgewater’s quotation over Criterion’s. The source selection decision was thus reasonable and consistent with the RFQ evaluation criteria.

The protest is denied.

Thomas H. Armstrong
General Counsel

---