Skip to main content

edCount, LLC--Protest and Costs

B-407556.3,B-407556.4,B-407556.5,B-407556.6 Aug 15, 2013
Jump To:
Skip to Highlights

Highlights

edCount, LLC, of Washington, D.C., protests its exclusion from the competitive range under request for proposals (RFP) No. ED-ELA-12-R-0034, issued by the Department of Education for operations management of the National Clearinghouse for English Language Acquisition and Language Instruction Educational Programs. edCount argues that the agency misevaluated its proposal and unreasonably eliminated it from the competitive range.

We deny the protest.
View Decision

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: edCount, LLC--Protest and Costs

File: B-407556.3; B-407556.4; B-407556.5; B-407556.6

Date: August 15, 2013

Andrew J. Mohr, Esq., John J. O’Brien, Esq., Gabriel E. Kennon, Esq., and C. Kelly Kroll, Esq., Cohen Mohr LLP, for the protester.
Todd E. Davis, Esq., and Brian M. Stanford, Esq., Department of Education, for the agency.
Scott H. Riback, Esq., and Tania Calhoun, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.

DIGEST

Protest that agency misevaluated proposals in connection with its elimination of the protester’s proposal from the competitive range is denied where, even though the record shows that the agency assigned irrational point scores to the proposals, the agency’s error was not prejudicial to the protester.

DECISION

edCount, LLC, of Washington, D.C., protests its exclusion from the competitive range under request for proposals (RFP) No. ED-ELA-12-R-0034, issued by the Department of Education for operations management of the National Clearinghouse for English Language Acquisition and Language Instruction Educational Programs. edCount argues that the agency misevaluated its proposal and unreasonably eliminated it from the competitive range.

We deny the protest.

BACKGROUND

The RFP contemplates award of a fixed-price contract on a “best value” basis, considering price and several non-price considerations. RFP at 16. The RFP advised offerors that technical merit was considered significantly more important than price, but further advised that, as technical merit among the competing proposals was determined to be relatively equal, price would become a more significant consideration. Id.

The RFP included four technical evaluation factors and specified a point value for each factor as follows:

Evaluation Factor

Available Points

Organizational Capability and Experience

30

Technological Experience

25

Soundness of the Technical Work Plan

25

Management Plan

20

Total

100

RFP at 16. In addition, the solicitation divided each of the technical evaluation factors into detailed subfactors with corresponding point values for each subfactor. For example, as quoted below, the solicitation’s technological experience evaluation factor included the following subfactors, along with their respective point values:

Demonstrate relevant, successful experience in operating a project of this nature and scope, including previous contract performance (4 points); and

The combined resources, facilities, and equipment of the Offeror and any proposed subcontractors, partners, or consultants provide the feasibility of meeting the requirements of the contract on time and to high standards (5 points); and

Describes sound plans to enhance and improve upon the NCELA website, and utilize innovative technology efficiently, effectively, and in multiple formats (12 points); and

Demonstrate the ability to support a virtual Community of Practice (4 points).

RFP at 17.[1]

For price evaluation purposes, the RFP provided that proposals would be evaluated for reasonableness and completeness, and also would be reviewed to determine whether proposed prices were valid, realistic and consistent with the offeror’s technical proposal. RFP at 19.

The agency received proposals from six firms in response to the RFP, including one from edCount. After evaluating initial proposals, the agency awarded a contract on September 26, 2012, without conducting discussions. Contracting Officer’s Statement of Facts, at 8. After learning of the agency’s award decision, edCount filed a protest in our Office on October 5. In response to that protest, the agency advised our Office that it would take corrective action by reevaluating proposals and engaging in discussions, if necessary. Based on this proposed corrective action, we dismissed edCount’s protest as academic on October 31.

On April 2, 2013, after reevaluating proposals, the agency established a competitive range that did not include edCount. Contracting Officer’s Statement of Facts, at 10. edCount protested this action to our Office on April 22. The agency subsequently advised that it intended to take corrective action in response to this protest by making a new competitive range determination; the agency reserved the right to reevaluate proposals. We dismissed edCount’s second protest as academic on May 7.[2] On May 31, the agency made a new competitive range determination that again did not include edCount. This protest followed.

PROTEST

edCount has challenged virtually every aspect of the agency’s evaluation of its proposal. We have carefully reviewed each of edCount’s allegations and conclude that, although the protester is correct with respect to one of its contentions (relating to the agency’s failure to adhere to the evaluation scoring scheme outlined in the RFP), the agency’s error was not prejudicial to the protester. We discuss edCount’s principal allegations below.

We note at the outset that, in considering protests relating to an agency’s evaluation, we do not independently evaluate proposals; rather, we review the agency’s evaluation to ensure that it is consistent with the terms of the solicitation and applicable statutes and regulations. SOS Int’l, Ltd., B-402558.3, B-402558.9, June 3, 2010, 2010 CPD ¶ 131 at 2. A protester’s disagreement with the agency’s evaluation conclusions does not provide a basis for our Office to object to the evaluation. OPTIMUS Corp., B-400777, Jan. 26, 2009, 2009 CPD ¶ 33 at 6.

Adherence to the RFP’s Evaluation Scoring Scheme

edCount alleges that the agency failed to follow the evaluation scoring scheme outlined in the RFP when it assigned point scores to the proposals. According to the protester, the agency did not use the point score system detailed in the solicitation. edCount contends that there is no way to determine the basis for, or underlying reasonableness of, the point score assigned to its proposal.

We agree with this aspect of edCount’s protest. Agencies may not properly announce one basis for evaluating proposals and making a source selection decision and then evaluate proposals and make award on a different basis. Johnson Controls Security Systems, LLC, B-296490.3 et al, Mar. 23, 2007, 2007 CPD ¶ 100 at 7. For example, an agency may not properly specify the relative weight of evaluation considerations in a solicitation and then apply a different weighting scheme when evaluating proposals and making a source selection decision. Id. While an agency’s use of point scores can serve to guide intelligent decisionmaking, the assignment of the underlying point scores must be on an intelligible, reasonable, equal and consistent basis for all proposals. Nexant, Inc., B-407708, B-407708.2, Jan. 30, 2013, 2013 CPD ¶ 59 at 7.

Here, the record shows that the evaluators did not assign point scores to the proposals in a manner that was consistent with the evaluation criteria outlined in the RFP. Specifically, the record shows that the evaluators assigned only overall point scores for each technical evaluation factor without scoring the proposals using the more detailed scoring scheme outlined under the subfactors included under each factor. The agency concedes as much, stating:

As an initial matter, and for purposes of clarification, ED’s [the Department of Education’s] Technical Evaluation Panel (“TEP”) . . . specifically chose not to follow the instructions in evaluating offerors’ proposals set forth in the Technical Evaluation Plan and Instructions. [Citations to the record omitted]. Rather, the record shows that ED’s technical evaluators analyzed each offeror’s technical proposal in accordance with the narrative evaluation [subfactors] set forth under each technical evaluation subfactor [factor] (i.e. Organizational Capability and Experience; Technological Experience; Soundness of Technical Work Plan; and Management Plan). Instead, the TEP employed a more holistic approach to evaluating proposals under each technical [factor].[3]

Agency Legal Memorandum, July 15, 2013, at 2 (emphasis in original).

As an initial matter, the agency appears to take the position that it was not required to assign point scores in a manner consistent with the point values assigned to the evaluation subfactors because the express instruction to so score proposals was only articulated in the agency’s source selection plan, and nothing in the RFP specifically stated that the agency would assign point scores on a subfactor-by-subfactor basis. We disagree. The agency’s position ignores that these subfactors and their corresponding point scores also were identified in the RFP. RFP at 16-18. This solicitation language, which identified 23 separate subfactors (with specific point score values for each) obligated the agency to assign point scores in a manner consistent with the scheme articulated in the RFP.

There is nothing in the record to show that the agency scored proposals in a manner consistent with the terms of the RFP, and, as noted, the agency concedes this issue. On the contrary, the record includes numerous examples where the agency’s scoring of the protester’s proposal clearly was not compliant with the subfactor weighting scheme announced in the solicitation. These defects permeate both the individual evaluators’ scoring, as well as the consensus scoring.

For example, with respect to the individual scores assigned by the evaluators, the record shows that evaluator no. 3 assigned edCount’s proposal the lowest score of 12 points (out of a possible 25 points) under the technological experience factor. That individual’s score sheet does not show either the subfactors under which the evaluator assigned points, or the number of points deducted for each perceived weakness. Agency Report (AR), exh. 6, evaluator 3 worksheet, at 13-14. It also cannot be ascertained from the record whether this individual’s assignment of points was attributable to more important, or less important, subfactors.

In addition, the record shows that, in arriving at their consensus point score, the individual evaluators simply averaged their total point scores, even though there were wide variations in those scores, and even though a number of the weaknesses forming the bases for the individual evaluator point scores were not carried forward to the consensus evaluation report. [4] AR, exh. 20, Technical Evaluation Consensus Report, at 8. The record is devoid of any explanation of how the evaluators resolved the wide variation in the scores assigned by the individual evaluators, or why they concluded that a simple average of the individual scores assigned to the higher-level factors accurately reflected the collective judgment of the evaluators. While the use of a mathematic average to develop a consensus score is not per se improper, see, Smart Innovative Solutions, B-400323.3, 2008 CPD ¶ 220, at 3-4, we cannot conclude that it was reasonable here.

In sum, the record shows that, because the evaluators elected to ignore the solicitation’s detailed evaluation scheme--and more specifically, the relative weight of the evaluation subfactors established by the terms of the RFP--the evaluation results included pervasive mathematical anomalies in the scores assigned to the edCount proposal. We conclude that the cumulative effect of these errors renders the agency’s point scores essentially meaningless.

Prejudicial Effect of the Agency’s Point Scoring Error

Despite the agency’s failure to adhere to the RFP’s subfactor evaluation scheme in assigning a point score to the edCount proposal, we nonetheless conclude that the agency’s error was not prejudicial to edCount. Prejudice is an essential element of every viable protest, and where none is shown or otherwise evident, we will not sustain a protest, even where a protester may have shown that an agency’s actions arguably were improper. ITT Corp.-Electronic Sys. , B-402808, Aug. 6, 2010, 2010 CPD ¶ 178 at 7.

The record shows that, although the overall consensus point score assigned to edCount’s proposal essentially was meaningless for the reasons discussed above, the agency did identify 14 specific weaknesses in the proposal across every technical evaluation factor. In making her competitive range determination, the contracting officer described these weaknesses in detail. She found that several proposals, including edCount’s, were “so significantly weaker” than other proposals that there was no reasonable expectation that they could be improved without significant revisions; as a consequence, she eliminated edCount’s and the other proposals from the competitive range. AR, exh.21, Competitive Range Determination, at 22.

Thus, although the point score assigned to the edCount proposal could not have formed a rational basis for eliminating the proposal from the competitive range, the agency’s underlying detailed analysis of the edCount proposal does provide a reasonable basis for its elimination from the competitive range. Smart Innovative Solutions supra at 4. As discussed below, edCount has not shown that the agency’s substantive findings regarding the merits of its proposal were unreasonable.

Weaknesses Identified in the edCount Proposal

In its protest to our Office, edCount challenged each of the 14 weaknesses identified by the agency in its evaluation. edCount’s First Amended Protest, June 6, 2013, at 23-55. The agency’s report included a detailed rebuttal to each allegation. AR, June 24, 2013, at 4-23. In its comments responding to the agency’s report, edCount did not respond to the agency’s detailed position with respect to 13 of its original 14 allegations. Instead, edCount merely incorporated its original protest by reference. Where, as here, an agency provides a detailed response to a protester’s assertions and the protester fails to rebut or otherwise address the agency’s arguments in its comments, the protester provides us with no basis to conclude that the agency’s evaluation assessments were unreasonable. Atmospheric Research Sys., Inc., B-240187, Oct. 26, 1990, 90-12 CPD ¶ 338 at 3; see also, Israeli Aircraft Indus., Ltd.--TAMAM Div., B-297691, Mar. 13, 2006, 2006 CPD ¶ 62 at 6-7 (where protester either does not respond to the agency’s position or provides a response that merely references or restates the original allegation without substantively rebutting the agency’s position, we deem the originally-raised allegations essentially abandoned).

With respect to the one weakness identified by the agency that edCount continues to challenge, the record shows that the agency acted reasonably in identifying this aspect of the edCount proposal as a weakness. The evaluators criticized the edCount proposal for offering to use a [deleted] in connection with the preparation of a particular biennial report to Congress, along with other reports required under the terms of the contract. The agency’s evaluators were concerned because the [deleted] level of expertise was unknown and the level of supervision to which the [deleted] would be subject also was unknown. AR, exh. 20, Technical Evaluation Consensus Report, at 8-9; exh. 21, Competitive Range Determination, at 9, 21.

edCount maintains that the agency’s criticism is unreasonable because the proposed use of a [deleted] was only in connection with the performance of certain analyses to be used in the reports, and not for the preparation of the reports in their entirety. edCount also asserts that, because the [deleted] would be supervised by one of edCount’s proposed senior advisors, any concerns over the use of a [deleted] were unwarranted. edCount also contends that, by virtue of the fact that only a [deleted] would be used, the level of expertise possessed by the individual performing the work would be reasonably high, based on the fact that he or she was a [deleted].

The protester’s objection to the agency’s finding amounts to no more than disagreement with the evaluators’ conclusion and, as such, fails to demonstrate that the agency’s finding was unreasonable. OPTIMUS Corp., supra., at 6. The protester proposed to have a [deleted] perform analyses in connection with preparation of various reports to be provided under the contract. AR, exh. 4, edCount Proposal, at 26. The [deleted] was not identified in the edCount proposal, and there is no resume showing whether the qualifications of the individual (or individuals) in question were appropriate to perform the work at issue. Additionally, while the edCount proposal references the fact that the [deleted] would be supervised by one of edCount’s proposed senior advisors, there is simply no discussion in the proposal regarding the level of supervision to be afforded.

We conclude that the agency evaluators reasonably were concerned that the preparation of one of the contract’s most important deliverables--the biennial report to Congress--along with other deliverables would be left in some measure to an unidentified doctoral student whose level of expertise and skills were unknown and unidentified in the edCount proposal. The agency’s evaluators also were reasonably concerned that the level of supervision for the doctoral student was unknown and not discussed in the protester’s proposal. In view of these considerations, we deny this aspect of edCount’s protest.[5]

Strengths Associated With the edCount Proposal

edCount argues that the agency improperly eliminated a number of strengths that were identified in its proposal during the agency’s first evaluation. According to the protester, it was irrational for the agency to have eliminated these strengths. However, the fact that a reevaluation varies from an original evaluation does not constitute evidence that the reevaluation was unreasonable. It is implicit that a reevaluation could result in different findings and conclusions. QinetiQ North America, Inc., B-405163.2 et al., Jan. 25, 2012, 2012 CPD ¶ 53 at 13; Sabre Sys., Inc., B-402040.2, B-402040.3, June 1, 2010, 2010 CPD ¶ 128 at 5 n.3.

Moreover, the record shows that, although the agency recognized that the edCount proposal included strengths, it was the number and pervasiveness of the weaknesses identified that led to the elimination of edCount’s proposal from the competitive range. In this connection, the agency’s contracting officer, in establishing the competitive range, found as follows:

Notwithstanding the strengths demonstrated by edCount, LLC . . . (as found by the TEP [technical evaluation panel]), as noted above, each [edCount and the other offerors that were eliminated from the competitive range] had weaknesses that were of serious concern to the success of the contract. While edCount . . . proposed overall prices in line with the Government Cost Estimate, those prices were still higher than either [offeror A or offeror B]. More importantly, their proposals were so significantly weaker than [offeror A, offeror B or offeror C], that there was no reasonable expectation that their proposal could be improved without significant revisions.

AR, exh. 21, Competitive Range Determination, at 22. We therefore deny this aspect of edCount’s protest.

Other Allegations

edCount contends that the agency misevaluated the proposal of one of the two offerors included in the competitive range. According to the protester, this firm’s proposal also should have been eliminated from the competitive range.

We dismiss this aspect of edCount’s protest. Our Bid Protest Regulations, 4 C.F.R. § 21.1 (a), contemplate that only an interested party may maintain a protest. Even if edCount’s challenge to the agency’s evaluation of the other firm’s proposal is correct--and that firm also was eliminated from the competitive range--the other competitive range offeror, not edCount, would be in line for award of the contract. edCount therefore is not an interested party to maintain this aspect of its protest. OMV Medical, Inc., B-281490, Feb. 16, 1999, 99-1 CPD ¶ 38 at 9.

edCount also asserts that the agency’s competitive range determination was flawed because the agency included a proposal in the competitive range that had a point score somewhat--but not significantly--higher than edCount’s point score, and the agency concluded that it did not require major revisions, whereas the agency concluded that edCount’s proposal did require major revisions to have a reasonable chance of receiving award.

Inasmuch as this aspect of edCount’s protest relies entirely on the point scores assigned by the agency, which we have found meaningless, it fails to state a basis for protest. Moreover, as discussed, the record shows that the agency ultimately relied on the underlying strengths and weaknesses assigned to the proposals--rather than the point scores assigned--in establishing the competitive range. We therefore dismiss this aspect of edCount’s protest.

Finally, edCount argues that the agency considered adverse past performance information during its evaluation that edCount was never afforded an opportunity to address. The record shows that the past performance information to which edCount alludes is information referenced by the evaluators in their individual evaluation worksheets. The information was never used as a basis for an identified weakness in the agency’s consensus evaluation report or competitive range determination. Thus, the agency’s error in this respect was not prejudicial because the information never formed a basis for finding a weakness in the edCount proposal. We therefore deny this aspect of edCount’s protest.

The protest is denied.

Susan A. Poling
General Counsel



[1] In addition to the evaluation factors identified above, the RFP originally also provided for a separate evaluation of past performance. However, that stand-alone evaluation factor subsequently was removed from the RFP. RFP, Amendment No. 2, at 6. The RFP continued to provide for the evaluation of past performance under subfactors included in the the organizational capability and experience and the technological experience factors. RFP at 17, as revised by Amendment No. 2, at 7. The RFP, as amended, provided that the agency would obtain past performance information from other sources (such as the past performance information retrieval system), and advised offerors that they would be afforded an opportunity to comment on adverse past performance information. RFP, Amendment No. 2, at 4.

[2] edCount requests that we recommend it be reimbursed the costs associated with filing its April 22 protest. However, the agency advised our Office of its proposed corrective action prior to the deadline for submitting its report. Our Office may recommend that a protester be reimbursed the costs of filing and pursuing a protest where the contracting agency decides to take corrective action in response to the protest. Bid Protest Regulations, 4 C.F.R. § 21.8(e) (2013). Such recommendations generally are based upon a concern that an agency has taken longer than necessary to initiate corrective action in the face of a clearly meritorious protest, thereby causing protesters to expend unnecessary time and resources to make further use of the protest process to obtain relief. QuanTech, Inc.--Costs, B-278380.3, June 17, 1998, 98-1 CPD ¶ 165 at 2. Such a basis for the award of costs does not exist here, since the agency initiated corrective action promptly (i.e., prior to the agency report due date) in response to the April 22 protest. See Veda Inc.--Entitlement to Costs, B-265809.2, July 19, 1996, 96-2 CPD ¶ 27 at 2. As a result, edCount’s request is denied.

[3] For the record, the agency report uses the phrase “subfactors” to refer to the RFP’s four evaluation factors and the phrase “sub-criteria” to refer to the evaluation subfactors. Agency Legal Memorandum, July 15, 2013, at 2. For the sake of clarity, we use the phrases “factor” and “subfactor” in the quote above, and throughout the decision.

[4] For example, an examination of the agency’s consensus evaluation report shows that none of the weaknesses identified by evaluator no. 3 under the technological experience factor were carried over to the consensus evaluation conclusions, but that evaluator’s point score based on these weaknesses was used to calculate the final consensus point score. AR, exh. 20, Technical Evaluation Consensus Report, at 9-10.

[5] edCount also alleges in its comments for the first time that the agency’s evaluation improperly identified a weakness in its proposal for failing to recognize that the central requirement of the contract is for technological expertise rather than expertise in English language education. However, the record shows that this weakness was identified by one of the evaluators in that evaluator’s individual evaluation worksheet. AR, exh. 6, evaluator no. 3’s individual worksheet. This weakness does not appear in the consensus technical evaluation report or in the competitive range determination as a basis for eliminating the edCount proposal from further consideration.

Downloads

GAO Contacts

Office of Public Affairs