edCount, LLC--Protest and Costs
Highlights
edCount, LLC, of Washington, D.C., protests its exclusion from the competitive range under request for proposals (RFP) No. ED-ELA-12-R-0034, issued by the Department of Education for operations management of the National Clearinghouse for English Language Acquisition and Language Instruction Educational Programs. edCount argues that the agency misevaluated its proposal and unreasonably eliminated it from the competitive range.
DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.
Decision
Matter of: edCount, LLC--Protest and Costs
File: B-407556.3; B-407556.4; B-407556.5; B-407556.6
Date: August 15, 2013
Andrew J. Mohr, Esq., John J. OBrien, Esq., Gabriel E. Kennon, Esq., and C. Kelly Kroll, Esq., Cohen Mohr LLP, for the protester.
Todd E. Davis, Esq., and Brian M. Stanford, Esq., Department of Education, for the agency.
Scott H. Riback, Esq., and Tania Calhoun, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.
DIGEST
Protest that agency misevaluated proposals in connection with its elimination of the protesters proposal from the competitive range is denied where, even though the record shows that the agency assigned irrational point scores to the proposals, the agencys error was not prejudicial to the protester.
DECISION
edCount, LLC, of Washington, D.C., protests its exclusion from the competitive range under request for proposals (RFP) No. ED-ELA-12-R-0034, issued by the Department of Education for operations management of the National Clearinghouse for English Language Acquisition and Language Instruction Educational Programs. edCount argues that the agency misevaluated its proposal and unreasonably eliminated it from the competitive range.
We deny the protest.
BACKGROUND
The RFP contemplates award of a fixed-price contract on a best value basis, considering price and several non-price considerations. RFP at 16. The RFP advised offerors that technical merit was considered significantly more important than price, but further advised that, as technical merit among the competing proposals was determined to be relatively equal, price would become a more significant consideration. Id.
The RFP included four technical evaluation factors and specified a point value for each factor as follows:
Evaluation Factor | Available Points |
Organizational Capability and Experience | 30 |
Technological Experience | 25 |
Soundness of the Technical Work Plan | 25 |
Management Plan | 20 |
Total | 100 |
RFP at 16. In addition, the solicitation divided each of the technical evaluation factors into detailed subfactors with corresponding point values for each subfactor. For example, as quoted below, the solicitations technological experience evaluation factor included the following subfactors, along with their respective point values:
Demonstrate relevant, successful experience in operating a project of this nature and scope, including previous contract performance (4 points); and
The combined resources, facilities, and equipment of the Offeror and any proposed subcontractors, partners, or consultants provide the feasibility of meeting the requirements of the contract on time and to high standards (5 points); and
Describes sound plans to enhance and improve upon the NCELA website, and utilize innovative technology efficiently, effectively, and in multiple formats (12 points); and
Demonstrate the ability to support a virtual Community of Practice (4 points).
RFP at 17.[1]
For price evaluation purposes, the RFP provided that proposals would be evaluated for reasonableness and completeness, and also would be reviewed to determine whether proposed prices were valid, realistic and consistent with the offerors technical proposal. RFP at 19.
The agency received proposals from six firms in response to the RFP, including one from edCount. After evaluating initial proposals, the agency awarded a contract on September 26, 2012, without conducting discussions. Contracting Officers Statement of Facts, at 8. After learning of the agencys award decision, edCount filed a protest in our Office on October 5. In response to that protest, the agency advised our Office that it would take corrective action by reevaluating proposals and engaging in discussions, if necessary. Based on this proposed corrective action, we dismissed edCounts protest as academic on October 31.
On April 2, 2013, after reevaluating proposals, the agency established a competitive range that did not include edCount. Contracting Officers Statement of Facts, at 10. edCount protested this action to our Office on April 22. The agency subsequently advised that it intended to take corrective action in response to this protest by making a new competitive range determination; the agency reserved the right to reevaluate proposals. We dismissed edCounts second protest as academic on May 7.[2] On May 31, the agency made a new competitive range determination that again did not include edCount. This protest followed.
PROTEST
edCount has challenged virtually every aspect of the agencys evaluation of its proposal. We have carefully reviewed each of edCounts allegations and conclude that, although the protester is correct with respect to one of its contentions (relating to the agencys failure to adhere to the evaluation scoring scheme outlined in the RFP), the agencys error was not prejudicial to the protester. We discuss edCounts principal allegations below.
We note at the outset that, in considering protests relating to an agencys evaluation, we do not independently evaluate proposals; rather, we review the agencys evaluation to ensure that it is consistent with the terms of the solicitation and applicable statutes and regulations. SOS Intl, Ltd., B-402558.3, B-402558.9, June 3, 2010, 2010 CPD ¶ 131 at 2. A protesters disagreement with the agencys evaluation conclusions does not provide a basis for our Office to object to the evaluation. OPTIMUS Corp., B-400777, Jan. 26, 2009, 2009 CPD ¶ 33 at 6.
Adherence to the RFPs Evaluation Scoring Scheme
edCount alleges that the agency failed to follow the evaluation scoring scheme outlined in the RFP when it assigned point scores to the proposals. According to the protester, the agency did not use the point score system detailed in the solicitation. edCount contends that there is no way to determine the basis for, or underlying reasonableness of, the point score assigned to its proposal.
We agree with this aspect of edCounts protest. Agencies may not properly announce one basis for evaluating proposals and making a source selection decision and then evaluate proposals and make award on a different basis. Johnson Controls Security Systems, LLC, B-296490.3 et al, Mar. 23, 2007, 2007 CPD ¶ 100 at 7. For example, an agency may not properly specify the relative weight of evaluation considerations in a solicitation and then apply a different weighting scheme when evaluating proposals and making a source selection decision. Id. While an agencys use of point scores can serve to guide intelligent decisionmaking, the assignment of the underlying point scores must be on an intelligible, reasonable, equal and consistent basis for all proposals. Nexant, Inc., B-407708, B-407708.2, Jan. 30, 2013, 2013 CPD ¶ 59 at 7.
Here, the record shows that the evaluators did not assign point scores to the proposals in a manner that was consistent with the evaluation criteria outlined in the RFP. Specifically, the record shows that the evaluators assigned only overall point scores for each technical evaluation factor without scoring the proposals using the more detailed scoring scheme outlined under the subfactors included under each factor. The agency concedes as much, stating:
As an initial matter, and for purposes of clarification, EDs [the Department of Educations] Technical Evaluation Panel (TEP) . . . specifically chose not to follow the instructions in evaluating offerors proposals set forth in the Technical Evaluation Plan and Instructions. [Citations to the record omitted]. Rather, the record shows that EDs technical evaluators analyzed each offerors technical proposal in accordance with the narrative evaluation [subfactors] set forth under each technical evaluation subfactor [factor] (i.e. Organizational Capability and Experience; Technological Experience; Soundness of Technical Work Plan; and Management Plan). Instead, the TEP employed a more holistic approach to evaluating proposals under each technical [factor].[3]
Agency Legal Memorandum, July 15, 2013, at 2 (emphasis in original).
As an initial matter, the agency appears to take the position that it was not required to assign point scores in a manner consistent with the point values assigned to the evaluation subfactors because the express instruction to so score proposals was only articulated in the agencys source selection plan, and nothing in the RFP specifically stated that the agency would assign point scores on a subfactor-by-subfactor basis. We disagree. The agencys position ignores that these subfactors and their corresponding point scores also were identified in the RFP. RFP at 16-18. This solicitation language, which identified 23 separate subfactors (with specific point score values for each) obligated the agency to assign point scores in a manner consistent with the scheme articulated in the RFP.
There is nothing in the record to show that the agency scored proposals in a manner consistent with the terms of the RFP, and, as noted, the agency concedes this issue. On the contrary, the record includes numerous examples where the agencys scoring of the protesters proposal clearly was not compliant with the subfactor weighting scheme announced in the solicitation. These defects permeate both the individual evaluators scoring, as well as the consensus scoring.
For example, with respect to the individual scores assigned by the evaluators, the record shows that evaluator no. 3 assigned edCounts proposal the lowest score of 12 points (out of a possible 25 points) under the technological experience factor. That individuals score sheet does not show either the subfactors under which the evaluator assigned points, or the number of points deducted for each perceived weakness. Agency Report (AR), exh. 6, evaluator 3 worksheet, at 13-14. It also cannot be ascertained from the record whether this individuals assignment of points was attributable to more important, or less important, subfactors.
In addition, the record shows that, in arriving at their consensus point score, the individual evaluators simply averaged their total point scores, even though there were wide variations in those scores, and even though a number of the weaknesses forming the bases for the individual evaluator point scores were not carried forward to the consensus evaluation report. [4] AR, exh. 20, Technical Evaluation Consensus Report, at 8. The record is devoid of any explanation of how the evaluators resolved the wide variation in the scores assigned by the individual evaluators, or why they concluded that a simple average of the individual scores assigned to the higher-level factors accurately reflected the collective judgment of the evaluators. While the use of a mathematic average to develop a consensus score is not per se improper, see, Smart Innovative Solutions, B-400323.3, 2008 CPD ¶ 220, at 3-4, we cannot conclude that it was reasonable here.
In sum, the record shows that, because the evaluators elected to ignore the solicitations detailed evaluation scheme--and more specifically, the relative weight of the evaluation subfactors established by the terms of the RFP--the evaluation results included pervasive mathematical anomalies in the scores assigned to the edCount proposal. We conclude that the cumulative effect of these errors renders the agencys point scores essentially meaningless.
Prejudicial Effect of the Agencys Point Scoring Error
Despite the agencys failure to adhere to the RFPs subfactor evaluation scheme in assigning a point score to the edCount proposal, we nonetheless conclude that the agencys error was not prejudicial to edCount. Prejudice is an essential element of every viable protest, and where none is shown or otherwise evident, we will not sustain a protest, even where a protester may have shown that an agencys actions arguably were improper. ITT Corp.-Electronic Sys. , B-402808, Aug. 6, 2010, 2010 CPD ¶ 178 at 7.
The record shows that, although the overall consensus point score assigned to edCounts proposal essentially was meaningless for the reasons discussed above, the agency did identify 14 specific weaknesses in the proposal across every technical evaluation factor. In making her competitive range determination, the contracting officer described these weaknesses in detail. She found that several proposals, including edCounts, were so significantly weaker than other proposals that there was no reasonable expectation that they could be improved without significant revisions; as a consequence, she eliminated edCounts and the other proposals from the competitive range. AR, exh.21, Competitive Range Determination, at 22.
Thus, although the point score assigned to the edCount proposal could not have formed a rational basis for eliminating the proposal from the competitive range, the agencys underlying detailed analysis of the edCount proposal does provide a reasonable basis for its elimination from the competitive range. Smart Innovative Solutions supra at 4. As discussed below, edCount has not shown that the agencys substantive findings regarding the merits of its proposal were unreasonable.
Weaknesses Identified in the edCount Proposal
In its protest to our Office, edCount challenged each of the 14 weaknesses identified by the agency in its evaluation. edCounts First Amended Protest, June 6, 2013, at 23-55. The agencys report included a detailed rebuttal to each allegation. AR, June 24, 2013, at 4-23. In its comments responding to the agencys report, edCount did not respond to the agencys detailed position with respect to 13 of its original 14 allegations. Instead, edCount merely incorporated its original protest by reference. Where, as here, an agency provides a detailed response to a protesters assertions and the protester fails to rebut or otherwise address the agencys arguments in its comments, the protester provides us with no basis to conclude that the agencys evaluation assessments were unreasonable. Atmospheric Research Sys., Inc., B-240187, Oct. 26, 1990, 90-12 CPD ¶ 338 at 3; see also, Israeli Aircraft Indus., Ltd.--TAMAM Div., B-297691, Mar. 13, 2006, 2006 CPD ¶ 62 at 6-7 (where protester either does not respond to the agencys position or provides a response that merely references or restates the original allegation without substantively rebutting the agencys position, we deem the originally-raised allegations essentially abandoned).
With respect to the one weakness identified by the agency that edCount continues to challenge, the record shows that the agency acted reasonably in identifying this aspect of the edCount proposal as a weakness. The evaluators criticized the edCount proposal for offering to use a [deleted] in connection with the preparation of a particular biennial report to Congress, along with other reports required under the terms of the contract. The agencys evaluators were concerned because the [deleted] level of expertise was unknown and the level of supervision to which the [deleted] would be subject also was unknown. AR, exh. 20, Technical Evaluation Consensus Report, at 8-9; exh. 21, Competitive Range Determination, at 9, 21.
edCount maintains that the agencys criticism is unreasonable because the proposed use of a [deleted] was only in connection with the performance of certain analyses to be used in the reports, and not for the preparation of the reports in their entirety. edCount also asserts that, because the [deleted] would be supervised by one of edCounts proposed senior advisors, any concerns over the use of a [deleted] were unwarranted. edCount also contends that, by virtue of the fact that only a [deleted] would be used, the level of expertise possessed by the individual performing the work would be reasonably high, based on the fact that he or she was a [deleted].
The protesters objection to the agencys finding amounts to no more than disagreement with the evaluators conclusion and, as such, fails to demonstrate that the agencys finding was unreasonable. OPTIMUS Corp., supra., at 6. The protester proposed to have a [deleted] perform analyses in connection with preparation of various reports to be provided under the contract. AR, exh. 4, edCount Proposal, at 26. The [deleted] was not identified in the edCount proposal, and there is no resume showing whether the qualifications of the individual (or individuals) in question were appropriate to perform the work at issue. Additionally, while the edCount proposal references the fact that the [deleted] would be supervised by one of edCounts proposed senior advisors, there is simply no discussion in the proposal regarding the level of supervision to be afforded.
We conclude that the agency evaluators reasonably were concerned that the preparation of one of the contracts most important deliverables--the biennial report to Congress--along with other deliverables would be left in some measure to an unidentified doctoral student whose level of expertise and skills were unknown and unidentified in the edCount proposal. The agencys evaluators also were reasonably concerned that the level of supervision for the doctoral student was unknown and not discussed in the protesters proposal. In view of these considerations, we deny this aspect of edCounts protest.[5]
Strengths Associated With the edCount Proposal
edCount argues that the agency improperly eliminated a number of strengths that were identified in its proposal during the agencys first evaluation. According to the protester, it was irrational for the agency to have eliminated these strengths. However, the fact that a reevaluation varies from an original evaluation does not constitute evidence that the reevaluation was unreasonable. It is implicit that a reevaluation could result in different findings and conclusions. QinetiQ North America, Inc., B-405163.2 et al., Jan. 25, 2012, 2012 CPD ¶ 53 at 13; Sabre Sys., Inc., B-402040.2, B-402040.3, June 1, 2010, 2010 CPD ¶ 128 at 5 n.3.
Moreover, the record shows that, although the agency recognized that the edCount proposal included strengths, it was the number and pervasiveness of the weaknesses identified that led to the elimination of edCounts proposal from the competitive range. In this connection, the agencys contracting officer, in establishing the competitive range, found as follows:
Notwithstanding the strengths demonstrated by edCount, LLC . . . (as found by the TEP [technical evaluation panel]), as noted above, each [edCount and the other offerors that were eliminated from the competitive range] had weaknesses that were of serious concern to the success of the contract. While edCount . . . proposed overall prices in line with the Government Cost Estimate, those prices were still higher than either [offeror A or offeror B]. More importantly, their proposals were so significantly weaker than [offeror A, offeror B or offeror C], that there was no reasonable expectation that their proposal could be improved without significant revisions.
AR, exh. 21, Competitive Range Determination, at 22. We therefore deny this aspect of edCounts protest.
Other Allegations
edCount contends that the agency misevaluated the proposal of one of the two offerors included in the competitive range. According to the protester, this firms proposal also should have been eliminated from the competitive range.
We dismiss this aspect of edCounts protest. Our Bid Protest Regulations, 4 C.F.R. § 21.1 (a), contemplate that only an interested party may maintain a protest. Even if edCounts challenge to the agencys evaluation of the other firms proposal is correct--and that firm also was eliminated from the competitive range--the other competitive range offeror, not edCount, would be in line for award of the contract. edCount therefore is not an interested party to maintain this aspect of its protest. OMV Medical, Inc., B-281490, Feb. 16, 1999, 99-1 CPD ¶ 38 at 9.
edCount also asserts that the agencys competitive range determination was flawed because the agency included a proposal in the competitive range that had a point score somewhat--but not significantly--higher than edCounts point score, and the agency concluded that it did not require major revisions, whereas the agency concluded that edCounts proposal did require major revisions to have a reasonable chance of receiving award.
Inasmuch as this aspect of edCounts protest relies entirely on the point scores assigned by the agency, which we have found meaningless, it fails to state a basis for protest. Moreover, as discussed, the record shows that the agency ultimately relied on the underlying strengths and weaknesses assigned to the proposals--rather than the point scores assigned--in establishing the competitive range. We therefore dismiss this aspect of edCounts protest.
Finally, edCount argues that the agency considered adverse past performance information during its evaluation that edCount was never afforded an opportunity to address. The record shows that the past performance information to which edCount alludes is information referenced by the evaluators in their individual evaluation worksheets. The information was never used as a basis for an identified weakness in the agencys consensus evaluation report or competitive range determination. Thus, the agencys error in this respect was not prejudicial because the information never formed a basis for finding a weakness in the edCount proposal. We therefore deny this aspect of edCounts protest.
The protest is denied.
Susan A. Poling
General Counsel
[1] In addition to the evaluation factors identified above, the RFP originally also provided for a separate evaluation of past performance. However, that stand-alone evaluation factor subsequently was removed from the RFP. RFP, Amendment No. 2, at 6. The RFP continued to provide for the evaluation of past performance under subfactors included in the the organizational capability and experience and the technological experience factors. RFP at 17, as revised by Amendment No. 2, at 7. The RFP, as amended, provided that the agency would obtain past performance information from other sources (such as the past performance information retrieval system), and advised offerors that they would be afforded an opportunity to comment on adverse past performance information. RFP, Amendment No. 2, at 4.
[2] edCount requests that we recommend it be reimbursed the costs associated with filing its April 22 protest. However, the agency advised our Office of its proposed corrective action prior to the deadline for submitting its report. Our Office may recommend that a protester be reimbursed the costs of filing and pursuing a protest where the contracting agency decides to take corrective action in response to the protest. Bid Protest Regulations, 4 C.F.R. § 21.8(e) (2013). Such recommendations generally are based upon a concern that an agency has taken longer than necessary to initiate corrective action in the face of a clearly meritorious protest, thereby causing protesters to expend unnecessary time and resources to make further use of the protest process to obtain relief. QuanTech, Inc.--Costs, B-278380.3, June 17, 1998, 98-1 CPD ¶ 165 at 2. Such a basis for the award of costs does not exist here, since the agency initiated corrective action promptly (i.e., prior to the agency report due date) in response to the April 22 protest. See Veda Inc.--Entitlement to Costs, B-265809.2, July 19, 1996, 96-2 CPD ¶ 27 at 2. As a result, edCounts request is denied.
[3] For the record, the agency report uses the phrase subfactors to refer to the RFPs four evaluation factors and the phrase sub-criteria to refer to the evaluation subfactors. Agency Legal Memorandum, July 15, 2013, at 2. For the sake of clarity, we use the phrases factor and subfactor in the quote above, and throughout the decision.
[4] For example, an examination of the agencys consensus evaluation report shows that none of the weaknesses identified by evaluator no. 3 under the technological experience factor were carried over to the consensus evaluation conclusions, but that evaluators point score based on these weaknesses was used to calculate the final consensus point score. AR, exh. 20, Technical Evaluation Consensus Report, at 9-10.
[5] edCount also alleges in its comments for the first time that the agencys evaluation improperly identified a weakness in its proposal for failing to recognize that the central requirement of the contract is for technological expertise rather than expertise in English language education. However, the record shows that this weakness was identified by one of the evaluators in that evaluators individual evaluation worksheet. AR, exh. 6, evaluator no. 3s individual worksheet. This weakness does not appear in the consensus technical evaluation report or in the competitive range determination as a basis for eliminating the edCount proposal from further consideration.