Advanced Systems Development, Inc.

B-298411,B-298411.2: Sep 19, 2006

Contact:

Ralph O. White
(202) 512-8278
WhiteRO@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Advanced Systems Development, Inc. (ASD) protests the award of a contract to KENROB Information Technology Solutions, Inc. by Washington Headquarters Services, Department of Defense (DOD), pursuant to request for proposals (RFP) No. HQ0034-06-R-1012, seeking information technology (IT) services, including hardware and software. ASD argues that the award to KENROB was improper because the agency conducted flawed discussions regarding price and performed an unreasonable price and technical evaluation.

We sustain the protest.

B-298411; B-298411.2, Advanced Systems Development, Inc., September 19, 2006

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Advanced Systems Development, Inc.

File: B-298411; B-298411.2

Date: September 19, 2006

Paul E. Pompeo, Esq., Kara L. Daniels, Esq., and David J. Craig, Esq., Holland & Knight LLP, for the protester.

William L. Walsh, Jr., Esq., J. Scott Hommer, III, Esq., Peter A. Riesen, Esq., and Keir X. Bancroft, Esq., Venable LLP, for KENROB Information Technology Solutions, Inc., an intervenor.

Andrew Blumenfeld, Esq., and Andy Bramnick, Esq., Washington Headquarters Services, Department of Defense, for the agency.

Ralph O. White, Esq., and Christine S. Melody, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.

DIGEST

1. Protester's contention that the agency conducted flawed discussions regarding price is sustained where (1) the agency corrected an error in the awardee's pricing; (2) the agency concluded that the awardee's price, as corrected, violated the solicitation's price target; (3) the agency advised the awardee in discussions that its price violated the solicitation's price target, though it did not, but never disclosed the upward adjustment it had made to correct the pricing error; (4) the awardee lowered its price in its final proposal, but repeated the pricing error it had made before; and (5) the agency selected that offeror for award after concluding that its significant price advantage offered the best value to the government. Thus, the record, as a whole, shows that the flawed discussions led the awardee to significantly lower its price, and the selection decision turned on the price differential between awardee and the protester.

2. Protester's contention that the agency failed to evaluate price proposals for completeness is sustained where the record shows that: (1) the solicitation expressly advised that price proposals would be assessed for completeness, including an assessment of the traceability of price estimates, and required that offerors submit detailed pricing data showing the traceability of those estimates in a work breakdown structure; (2) the agency never performed the completeness review; and (3) it is reasonable to conclude that, had it not been compelled to structure its proposal to comply with this solicitation requirement, the protester could have employed a different approach to structuring its proposal which could have resulted in a lower price.

3. Protester's contention that the evaluation of technical proposals was unreasonable is sustained where the record shows that the evaluation deviated from the stated evaluation criteria under one of the technical subfactors.

DECISION

Advanced Systems Development, Inc. (ASD) protests the award of a contract to KENROB Information Technology Solutions, Inc. by Washington Headquarters Services, Department of Defense (DOD), pursuant to request for proposals (RFP) No. HQ0034-06-R-1012, seeking information technology (IT) services, including hardware and software. ASD argues that the award to KENROB was improper because the agency conducted flawed discussions regarding price and performed an unreasonable price and technical evaluation.

We sustain the protest.

BACKGROUND

Washington Headquarters Services is a field activity of the Department of Defense comprised of 11 directorates which provide support to the Secretary of Defense and other DOD activities. These activities are spread across the 280-acre Pentagon Reservation, as well as scattered throughout 15 leased office buildings in Arlington, Alexandria, and Vienna, Virginia. Desktop IT support for these organizations and locations is currently provided under six separate support contracts held by five different IT support contractors. In addition, WHS currently operates five different IT help desks and four separate e-mail systems. Contracting Officer's (CO) Statement at 1-3.

On March 9, 2006, WHS issued the solicitation here to consolidate these IT support services into a single, more efficient contract, and set aside the procurement for small businesses. The RFP anticipated the award of a performance-based services contract for a base period (of approximately 8 months) followed by three 1-year options. RFP at 2-24, 42.[1] The RFP also contained three additional options, each of which contained a 2-year extension of performance--totaling almost 10 years of performance--to be awarded –if contractor performance is good/excellent.— CO's Statement at 3.

The RFP identified six base period contract line items (CLIN) covering the different types of services sought here. The first five of the base period CLINs were repeated through the option periods; CLIN 0006, seeking a fixed price for IT Infrastructure Tools, was not repeated in the option periods.[2] With the exception of the CLINs for Surge and Special Support and Travel (CLINs 0002 and 0003), the RFP anticipated award of a fixed-price contract.

Of relevance to this protest, the RFP provided specific instructions to potential offerors regarding their prices for CLINs 0001 and 0005 (and the option periods associated with those CLINs). First, section L of the RFP authorized offerors to propose an award fee pool associated with CLIN 0001, and provided lower and upper limits on the amount of that pool. Specifically, the RFP advised that –[t]he contractor may propose an award fee pool, no less than 8% and no more than 10% of [the] proposed fixed price of CLIN 0001AA, in CLIN 0001AB (and corresponding option CLINs).— RFP at 73. Second, the same provision in section L provided –price targets— to offerors, based on agency budget estimates, and advised that the targets would increase by 1.7 percent per year. Id.This guidance is set forth below as it appeared in the RFP:

CLIN Base Period Annual Amount
0001AA plus 0001AB
(and corresponding option CLINs)
$8,1156,335[3] $12,174,503
0005
(and corresponding option CLINs)
$390,873 $586,310

Id. at 74. In its report to our Office, the agency explained that basic requirements of this solicitation were identified in CLINs 0001, 0005, and 0006, and that the agency had calculated a total estimate for the base period and all options of $125 million. CO's Statement at 3-4.

To evaluate proposals to perform this work, the RFP identified three non-price evaluation factors--technical approach, management approach, and past performance--and advised that the non-price factors, when combined, would be significantly more important than price. RFP at 77. The RFP also advised that: (1) the technical approach factor would be more important than the management approach factor; (2) the management approach factor would be more important than the past performance factor; and (3) price would be the least important factor. Id.

Under the technical approach evaluation factor, the RFP identified six subfactors: (1) enterprise services; (2) domain services; (3) initial transition; (4) information assurance program implementation and maintenance; (5) enterprise-wide configuration management; and (6) contractor's performance work statement and work breakdown structure. The solicitation advised that subfactors 1, 2, and 3 of the technical approach factor would be equal in importance and the most important of the technical subfactors; similarly, the solicitation advised that subfactors 4, 5, and 6 would be equal in importance, but less important than the first three subfactors. Finally, the solicitation identified specific elements under the first two technical subfactors which need not be set forth here.

Under the management approach factor, the RFP identified three subfactors of equal weight. These are: (1) program leadership; (2) human resource management; and (3) metrics and service level agreement reporting and management. Id. at 78-79.

For both the technical and management approach evaluation factors, the solicitation advised that the agency would assign a merit rating (outstanding, excellent, acceptable, marginal, or unacceptable), and a confidence rating (high confidence, significant confidence, confidence, little confidence, or no confidence). Id. at 82-83. The solicitation further explained that merit ratings would be used to assess compliance with the requirements of the RFP, while confidence ratings would be used to assess the extent to which the agency concludes the offeror can do what it proposes. In addition, the RFP separately advised under each technical and management evaluation subfactor that the evaluators would –assess the soundness, credibility, quality and affordability of the Offeror's approach.— RFP at 78-79.

Under the past performance factor, the RFP advised that information received about an offeror's past performance would be assessed for its relevance and the quality of past performance. RFP amend. 1, at 2-5. To rate offerors under this factor, the RFP explained that each offeror would be assigned a confidence rating to assess –demonstrated ability to successfully perform the requirements of the [instant] contract based on how well they have performed on recent, relevant contracts.— Id. at 4. The same confidence ratings identified above were to be assigned, with one difference: there was also a category of unknown confidence (UC), which was intended to be a neutral rating for offerors lacking relevant past performance. Id. at 5.

As with the detailed instructions to offerors regarding the calculation of prices (set forth above), the RFP also contained detailed provisions related to the evaluation of prices. First, the RFP advised that price proposals would be evaluated for completeness and reasonableness. RFP at 81. Completeness was defined in the RFP as an assessment of –the level of detail the offeror provided in cost information for all RFP requirements in the [Statement of Objectives] and technical documents, and assessing the traceability of estimates.— Id.Reasonableness was to be assessed –by comparing the prices received in response to the RFP or comparing prices with other similar work.— Id.Second, the RFP explained:

For evaluation purposes, the proposed price will be calculated as follows:

CLIN 0001AA Total proposed amount for base period
CLIN 0001AB Proposed award fee pool, assuming 100% award
CLIN 0002 The sum of each hourly rates [sic] for the five highest priced proposed labor categories times 200 hours (e.g., labor category 1 x 200 hours + labor category 2 x 200 hours + labor category 3 x 200 hours + labor category 4 x 200 hours + labor category 5 x 200 hours)
CLIN 0003 Not included in calculation
CLIN 0004 Not included in calculation
CLIN 0005 Total proposed amount for base period
CLIN 0006 Total proposed amount
Option CLINs Calculated similarly to corresponding base period CLINs

RFP amend. 2, at 6.

By the April 12 initial closing date, the agency received five proposals, including those submitted by the awardee and protester here. The technical/management proposals were evaluated by a proposal evaluation board (PEB), which ultimately prepared a consensus report (the PEB Report) that was used to brief the source selection authority (SSA). At the same time, price proposals were evaluated by the Cost Team. The results of the review by the PEB and Cost Team were presented to the SSA, who decided that only the proposals submitted by ASD and KENROB should be retained in the competitive range. CO's Statement at 48. The results of the initial evaluation of these two proposals are set forth below:

INITIAL EVALUATION RESULTS

ASD

KENROB

Technical Approach

Factor -- Overall

[deleted]

[deleted]

Enterprise Services

[deleted]

[deleted]

Domain Services

[deleted]

[deleted]

Initial Transition Plan

[deleted]

[deleted]

Information Assurance

[deleted]

[deleted]

Configuration Mgmt.

[deleted]

[deleted]

Contractor's Perf. Work

Statement and Work

Breakdown Statement

[deleted]

[deleted]

Management Approach

Factor -- Overall

[deleted]

[deleted]

Program Leadership

[deleted]

[deleted]

Human Resources

[deleted]

[deleted]

Perf. Measurement

[deleted]

[deleted]

Past Performance

[deleted]

[deleted]

Price

[deleted] million

[deleted] million

CO's Statement at 26, 56.

In order to evaluate KENROB's initial price proposal, it was necessary for the Cost Team to make a judgment about an error in the proposal. As quoted above, the RFP advised offerors that the price for CLIN 0004 would not be included in the calculation to determine total evaluated price. This provision in the RFP's evaluation scheme followed guidance in the agency's Industry Day presentation (held before the submission of offers) that CLIN 0004 was not to be separately priced, although this instruction was not repeated in the RFP's Schedule B. Compare AR at 183 (the Industry Day slides) with RFP at 3 (CLIN 0004), 73-75 (instructions on preparing price proposals). Apparently forgetting the Industry Day instructions, KENROB priced CLIN 0004, and was the only offeror to do so. In response, the agency's Cost Team decided to correct the error by adding KENROB's price for CLIN 0004 ([deleted]) to its price for CLIN 0001 ([deleted]) to calculate a new total price for CLIN 0001 ([deleted]). CO's Statement at 48. Having done so, however, the Cost Team noticed that its remedial action caused KENROB's price for CLIN 0001 to exceed the budget estimate for that CLIN established by the RFP ($8,156,335) by [deleted], or approximately [deleted] percent. Id.

By letters dated May 10, the CO notified both ASD and KENROB that their proposals had been included in the competitive range. In addition the letters included an attachment advising each offeror of all of the weaknesses, significant weaknesses, and deficiencies that had been identified in that company's proposal. With respect to the pricing matter described above, the letter to KENROB advised that –[t]he Offeror's price for CLIN 0001 and corresponding first option item exceeds the target budget amount by [deleted]% and [deleted]%, respectively.— AR at 1865.[4] The letter did not remind KENROB that the agency had asked offerors during the Industry Day Briefing not to enter a separate price for CLIN 0004, nor did the letter advise that the agency had corrected KENROB's error by adding its CLIN 0004 price to its CLIN 0001 price to calculate a new CLIN 0001 price.

In contrast, ASD's pricing for CLIN 001 did not exceed the target amount; however, ASD's proposal did exceed the RFP's target for CLIN 0005. In the May 10 letter to ASD, the agency attempted to advise the company of this matter, but included a typographical error in its reference to the CLIN. Thus, the letter to ASD stated that –[t]he price for CLIN 0006 (and corresponding option items) consistently exceeded the target budget.— AR at 1563. Upon receipt of the letter, ASD noted that there was no target budget for CLIN 0006 in the solicitation and contacted the CO for clarification; the CO confirmed that the letter should have referenced CLIN 0005. CO's Statement at 49. The CO also explains that during this same conversation, ASD expressed concern about whether its proposed price for CLIN 0001 exceeded the budget, and the CO advised ASD that any item not specifically identified in the discussions letter was considered acceptable by the agency.[5] Id.Finally, the May 10 discussions letter to ASD and KENROB both contained the following guidance regarding revisions to their respective proposals:

You may revise any portion of your original proposal, or nothing at all. If you decide to make no revisions to any portion of your proposal, you must submit a written statement by the deadline above stating that your original proposal dated 12 April 2006 is your final proposal. Since no further discussions will be allowed, [it is] critical that you conform to these directions and facilitate understanding by ensuring traceability from your original proposal to the [final proposal revision]. It is your responsibility to mark all changes in the manner described in this memo to indicate where you have made revisions from your basic proposal.

AR at 1559 (Letter to ASD); AR at 1705 (Letter to KENROB).

By the final closing date of May 17, both ASD and KENROB submitted revised proposals. The results of the final evaluation were summarized and presented to the SSA in the same format as the initial evaluation results, and are set forth below.

FINAL EVALUATION RESULTS

ASD

KENROB

Technical Approach

Factor -- Overall

[deleted]

[deleted]

Enterprise Services

[deleted]

[deleted]

Domain Services

[deleted]

[deleted]

Initial Transition Plan

[deleted]

[deleted]

Information Assurance

[deleted]

[deleted]

Configuration Mgmt.

[deleted]

[deleted]

Contractor's Perf. Work

Statement and Work

Breakdown Statement

[deleted]

[deleted]

Management Approach

Factor -- Overall

[deleted]

[deleted]

Program Leadership

[deleted]

[deleted]

Human Resources

[deleted]

[deleted]

Perf. Measurement

[deleted]

[deleted]

Past Performance

[deleted]

[deleted]

Price

$135.6 million

$124.3 million

CO's Statement at 50.

KENROB's ratings from the initial to the final evaluation were increased under two of the subfactors under the technical approach factor--enterprise services (from [deleted] to [deleted]), and contractor's performance work statement and work breakdown structure (from [deleted] to [deleted]). There was no change, however, to KENROB's rating under the technical approach factor overall, which remained at [deleted]. Under the performance measurement subfactor of the management approach factor, KENROB's rating was increased from [deleted] to [deleted], but again, there was no change to the overall rating under the management approach factor. There also was no change to KENROB's rating under the past performance evaluation factor, but the company lowered its total evaluated price from [deleted] million to $124.3 million. CO's Statement at 26, 51.

ASD's ratings from the initial to the final evaluation were increased under four of the subfactors under the technical approach factor--domain services (from [deleted] to [deleted]), initial transition plan (from [deleted] to [deleted]), information assurance services (from [deleted] to [deleted]), and contractor's performance work statement and work breakdown structure (from [deleted] to [deleted]). This led to an increased rating under the technical approach factor overall (from [deleted] to [deleted]). Under the performance measurement subfactor of the management approach factor, ASD's rating was increased from [deleted] to [deleted], but no change was made to ASD's overall rating for the management approach factor. There was also no change to ASD's rating under the past performance evaluation factor, but the company slightly lowered its total evaluated price from [deleted] million to [deleted] million. CO's Statement at 26, 57-59.

In looking at the final evaluation results, the SSA concluded that KENROB's and ASD's proposals were essentially equal under the most important evaluation factor, technical approach, and that ASD's proposal under the management approach factor was only slightly better than KENROB's. Source Selection Decision Memorandum, May 24, 2006, at 17. The remainder of the SSA's analysis focused on differences in the proposals as assessed under the human resources management subfactor of the management approach factor, past performance, and price. Specifically, the SSA stated:

The distinguishing subfactor was Human Resources Management where ASD received [deleted] rating while KENROB received [deleted] rating. ASD's [deleted] rating is attributed to the proposal for [deleted] reducing the risk to both the Government and the Contractor during transition. KENROB's lower merit rating results because their approach simply met the requirements with no identified strengths. Their lower confidence rating is due to what the PEB perceives is a low staffing level, which reduced confidence in KENROB's chances for successful performance. But, in my assessment, I considered that the resultant contract would be performance-based. KENROB will be held to performance standards that are results driven and not dependent on a particular level of effort. KENROB will have to provide whatever level of staffing is required to achieve the AQLs [acceptable quality levels]. ASD's past performance record resulted in a higher confidence rating, but KENROB's past performance leaves me with little doubt that they can perform successfully on this contract.

Id.Based on the analysis above, the SSA ultimately concluded that KENROB's significantly lower price ($124.3 million versus ASD's price of $135.6 million) made it the proposal that offered the best value to the government. Id.

On May 25, a contract was awarded to KENROB, and on June 9, the agency provided ASD with a debriefing. Five days later, on June 14, ASD filed the protest here, which was supplemented on July 10.

DISCUSSION

In its initial and supplemental protests, ASD argues that the agency held flawed discussions regarding price, and performed an unreasonable evaluation of the price and technical proposals. We agree.

Meaningful Discussions

In arguing that the agency failed to hold proper discussions, ASD raises several contentions. Specifically, ASD argues that: (1) it was unfair for the agency to point out the percentage by which KENROB's prices exceeded the CLIN 0001 target price, but not similarly point out the percentage by which ASD's price exceeded the RFP's target price for CLIN 0005; (2) the CO unfairly told ASD during a telephone conversation that it could not make changes to its final proposal except on matters that were raised during discussions; and (3) it was unfair to tell KENROB that its CLIN 0001 prices exceeded the RFP target prices for CLIN 0001, when, in fact, they did not.

WHS responds that: (1) there was nothing unreasonable about advising KENROB about the percentage by which its prices exceeded the RFP's CLIN 0001 target, while only telling ASD that its CLIN 0005 price exceeded the CLIN 0005 target; (2) the CO did not tell ASD that it could not lower its price in its final proposal, and even if she had, it was unreasonable for ASD to rely on such a statement when the discussions letter clearly advised both offerors they could revise any portion of their initial proposal they wished; and (3) its actions to correct KENROB's pricing error were reasonable, and it was reasonable to advise KENROB that its price, as corrected, exceeded the RFP's target price.

Before turning to the areas where we agree with ASD, we note that, as a general matter, agencies have broad discretion to determine the content and extent of discussions, and we limit our review of the agency's judgments in this area to a determination of whether they are reasonable. Creative Info. Techs., Inc., B'293073.10, Mar. 16, 2005, 2005 CPD para. 110 at 7. In this regard, we see nothing unfair about telling one offeror that its price exceeded the RFP's target amount, while telling another the percentage by which its price exceeded the target, when the target information is readily available in the RFP and either offeror could easily ascertain for itself the actual amount by which its price exceeded the target. We also disagree with ASD's contention that it was misled during a telephone conversation with the CO who allegedly advised ASD that its changes to its final proposal should be limited to areas raised in the discussions letter. The unambiguous written direction in the discussions letter advising offerors they could revise any portion of their proposals takes precedence over any perceived contradictory direction in an oral conversation. See Peckham Vocational Indus., Inc., B-257100, Aug. 26, 1994, 94'2 CPD para. 81 at 8.[6]

We turn next, however, to the area where we are troubled by the agency's discussions--the statement to KENROB that its CLIN 0001 price exceeded the RFP's target, when, in actuality, it did not. Federal Acquisition Regulation (FAR) sect. 15.306(e) specifies that discussions may not be conducted in a manner which –[f]avors one offeror over another.— Here, we think the series of actions by the agency--(1) correcting an error in KENROB's pricing; (2) concluding--based on a correction about which KENROB was never informed''that KENROB's price, as adjusted, violated the price target in the RFP; (3) advising KENROB during discussions that its price for CLIN 0001 exceeded the RFP's target amount (when on its face it did not); (4) receiving from KENROB a final proposal that made significant reductions in its already under-target CLIN 0001 price, but continued to contain a price for CLIN 0004 (which again had to be added to CLIN 0001 to determine its price for that CLIN); and (5) making award to KENROB, in essence, because of its significantly lower price--was unfair and prejudicial to ASD. Our basis for these conclusions is set forth below.

As explained above, the RFP here identified –price targets— for offerors to use in calculating the prices for CLINs 0001 and 0005 (and in calculating the prices for the corresponding option CLINs); these target amounts were stated for both CLINs for both the base period (approximately 8 months) and in an –annual amount.— RFP at 73. As also explained above, offerors were instructed during the agency's Industry Day presentation that they were not to separately price CLIN 0004, AR at 183, and were advised in the RFP that CLIN 0004 would not be included in the calculation to determine an offeror's total evaluated price. RFP amend. 2, at 6.

Despite these instructions, KENROB's initial proposal included a price for CLIN 0004; no other offeror included a price for this CLIN. When the agency Cost Team encountered KENROB's price for CLIN 0004 (which, according to schedule B, was to cover data deliverables, RFP at 193), the team added KENROB's CLIN 0004 price to its CLIN 0001 price to arrive at a new CLIN 0001 price of [deleted]. It was this agency-calculated CLIN 0001 price that exceeded the target identified in the RFP, not the price for CLIN 0001 found in KENROB's initial proposal.

The agency's discussions letter to KENROB makes no mention of the remedial action taken by the Cost Team to calculate a new price for KENROB for CLIN 0001. Instead, the letter's advice on this topic, in its entirety, was: –The offeror's price for CLIN 0001 and corresponding first option item exceeds the target budget amount by [deleted]% and [deleted]% respectively.— AR at 1865.

In the cover letter submitted with its final proposal revisions, KENROB responded to the agency's discussions question as follows:

The evaluation team's comment on our pricing caused the KENROB Team to reevaluate this portion of our submission, and we think you will be pleased with the results. In addition to ensuring that we do not exceed the identified budget thresholds, we have evaluated and amended pricing for the remaining option years.

AR at 1707. The cross-reference matrix appended to the final proposal cover letter further highlights the expansive scope of the actions taken by the company in response to the discussion question quoted above. Among other things, the letter explained:

--Team KENROB is discounting previously proposed KENROB Direct Labor Costs by [deleted]% for all Labor CLINs (both Base and Option Years).
--Team KENROB is discounting previously proposed NG-TASC Direct Labor Costs by [deleted]% for all Labor CLINs (both Base and Option Years).
--Team KENROB is discounting previously proposed PESystems Direct Labor Costs by [deleted]% for all Labor CLINs (both Base and Option Years).
--Team KENROB is reducing Award Fee from [deleted]% to [deleted]% for both Base and Option Years.

AR at 1711. Tellingly, KENROB's final proposal did not correct the error in its initial proposal of providing a price for CLIN 0004. Thus, the Cost Team was required again to add KENROB's CLIN 0004 price to its CLIN 0001 price to determine the total price for CLIN 0001, and to determine whether that price complied with the budget target identified in the RFP.

As an initial matter, we question whether there was any reasonable basis on which to determine what KENROB's intended price for CLIN 0001 was, either before or after discussions. In this regard, while KENROB included its price for CLIN 0004 (and the corresponding option CLINs) in its price total, there is no indication that KENROB meant to include its CLIN 0004 price in CLIN 0001. On the contrary, KENROB's approach to calculating its award fee strongly suggests that KENROB did not mean to do so. As noted above, the RFP here advised offerors that they could propose an award fee, under sub-CLIN 0001AB, of –no less than 8% and no more than 10%— (RFP at 73), of the proposed price of sub-CLIN 0001AA. The prices for sub-CLINs 0001AA and 0001AB added together generated an offeror's total price for CLIN 0001. RFP at 2. In its initial offer, KENROB proposed an award fee that was [deleted] percent of sub'CLIN 0001AA (KENROB's price for sub-CLIN 0001AA was [deleted], its award fee in sub-CLIN 0001AB was [deleted]). AR at 674. If KENROB understood that its initial CLIN 0004 price of [deleted], AR at 675, was to be added to its price for CLIN 0001AA, KENROB's intended [deleted] percent award fee would have been higher. Further, the agency's assumption regarding KENROB's intended price was not confirmed by KENROB's response to the discussions letter--KENROB continued to propose a price for CLIN 0004, and continued to calculate its award fee in CLIN 0001AB without regard to the costs that should have been included in that calculation.

In short, the record strongly suggests that KENROB never understood that the agency had added its CLIN 0004 price to its CLIN 0001 price to implement the instructions provided to offerors during the Industry Day Briefing.[7] As a result, based on the record here, it is in fact unclear what KENROB's intended price for CLIN 0001 was. This ambiguity raises a serious question as to whether KENROB's proposal properly could be selected for award. See Marine Pollution Control Corp., B-270172, Feb. 13, 1996, 96-1 para. 73 at 2-3 (requirement for fixed prices is a material term of an RFP requiring such pricing, and a proposal that does not offer fixed prices cannot be accepted for award).

With regard to this issue, WHS and KENROB argue that ASD cannot claim to have been prejudiced by these discussions. While WHS correctly points out that our Office will not sustain a protest unless the protester demonstrates that, but for the agency's actions, it would have had a substantial chance of receiving the award, McDonald-Bradley, B-270126, Feb. 8, 1996, 96-1 CPD para. 54 at 3; see Statistica, Inc. v. Christopher, 103 F.3d 1577, 1581 (Fed. Cir. 1996), we do not agree that the protester was not prejudiced here. First, to the extent that the agency and intervenor argue that the impact of the discussions was not material because the issues raised concerned only the prices proposed for the base period and first full year option period, we find the argument unpersuasive. The RFP here expressly states that the target amounts apply not only to CLINs 0001 and 0005, but to the corresponding option CLINs. RFP at 74. In addition, KENROB's final proposal clearly indicates that it has reflected the reductions taken in response to the discussions question throughout all the option periods. AR at 1711.

We also think there is a broader answer to the agency's and intervenor's claim that there was no prejudice as a result of these discussions. Simply put, any fair reading of the KENROB final proposal leads to the conclusion that the significant reductions in KENROB's total price followed directly from the agency's statement during discussions that its price for CLIN 0001 and the corresponding option item exceeded the budget estimates set out in the RFP. See AR at 1707, 1711. We cannot know, and will not speculate about, what KENROB would have done if it properly was told the basis for the agency's conclusion that its CLIN 0001 price exceeded the RFP's target amount. Given that the only likely outcome of telling KENROB (inaccurately) that its price for CLIN 0001 exceeded the target amounts for that CLIN was the lowering of KENROB's prices, and given that the selection decision here ultimately turned on the price differential between these two offerors,[8] we think that the flawed discussions with KENROB resulted in prejudice to ASD because they caused KENROB to significantly lower its price and in effect secure award for KENROB.

The Price Evaluation

ASD next argues that the agency abandoned the price evaluation scheme identified in the RFP by not performing one of the two prongs of the price evaluation--i.e., by not performing a review of price completeness. As indicated above, the RFP here advised offerors that price proposals would be evaluated to determine if they were complete and reasonable. The evaluation section of the RFP also explained the nature of the intended completeness review: –Completeness is evaluated to assess the level of detail the offeror provided in cost information for all RFP requirements in the [Statement of Objectives] and technical documents, and assessing the traceability of estimates.— RFP at 81.

The indication that the agency would conduct an evaluation of price completeness did not occur in a vacuum. Section L of the RFP provided detailed instructions to offerors about the level of information they should include in their price proposals. Among other things, the RFP required submission of a –person loading schedule,— broken down by work breakdown structure; back-up estimate sheets for the person loading schedule, also organized by work breakdown structure format; a schedule of hours by labor skill mix; a schedule of fully-burdened hourly labor rates; and a list of proposed probable subcontractors showing price and hours for each. RFP at 74-75; RFP amend. 2, at 5.

During the debriefing, ASD asked if the agency had reviewed the requested level of detail in the price proposals for completeness, or to assess the traceability of the price estimates. According to ASD, the agency's representative admitted that no such review was done, and stated that the agency –asked for too much price information in the solicitation.— Protester's Comments, Aug. 9, 2006, exh. 1 at 3.

In its report to our Office, the agency complains that ASD does not argue that KENROB failed to provide the requested information, or that KENROB's information was incomplete, but ASD argues only that the review was not performed. Agency Memorandum of Law, July 28, 2006, at 3. In addition, the agency asserts that it did look at completeness, pointing to a comment on a slide in the PEB's report to the SSA indicating that ASD's proposal –appears to be complete.— AR at 1976. Similarly, there is a comment--on the same slide--indicating that KENROB's price proposal –appears to be complete— with the exception of AQL measuring tools. Id.There is no other document in the record to support the agency's claim that it conducted an analysis to assess, among other things, –the traceability of estimates.— RFP at 81.

In our view, the record here lacks any persuasive evidence that the agency examined the completeness of the offerors' price proposals as called for by the RFP. We note that the solicitation here did not merely use the term –completeness— in setting out the parameters of the price evaluation, but explained, in detail, that the agency planned to evaluate completeness, and identified the kind of back-up pricing data that offerors needed to produce for the agency's evaluation; the agency's failure to conduct this review was clearly contrary to the solicitation's requirements. See OMNIPLEX World Servs. Corp., B'291105, Nov. 6, 2002, 2002 CPD para. 199 at 10.

In response to the agency's and intervenor's assertion that ASD was not prejudiced by any failure to analyze the completeness of price proposals, ASD argues that it spent significant resources to ensure that its prices were properly supported and that its estimates were traceable to the work breakdown structure. ASD also argues that had it not been compelled by the solicitation's requirement to build traceable cost estimates by work breakdown structure, it –would have employed a different approach to structuring its proposal, which may have resulted in a lower proposed price.— Protester's Comments, Aug. 9, 2006, at 18.

We find that ASD has made a sufficient showing of prejudice as a result of the agency's failure to evaluate the completeness of price proposals. By requesting in the solicitation that offerors provide detailed information showing how their price estimates were devised, and by advising them in the evaluation scheme that these estimates would be reviewed, WHS required offerors to be sure that their price proposals contained detailed and traceable estimates for all of the work involved. An RFP's stated evaluation scheme is the starting point for the development of proposals, and we think it is reasonable to assume that offerors, such as ASD, may have structured their proposals differently if they had known that the agency would not be assessing costs for traceability. See Rockwell Elec. Commerce Corp., B'286201 et al., Dec. 14, 2000, 2001 CPD para. 65 at 9. Given the closeness of the evaluation results, and given that the award decision ultimately was largely driven by price, we think the agency and intervenor cannot convincingly claim that the failure to conduct the promised review here had no impact on the award decision.

Evaluation of Non-price Factors

ASD argues that the agency's evaluation of KENROB's technical proposal was unreasonable under five separate evaluation subfactors--four subfactors under the technical approach evaluation factor, and one under the management approach factor. The technical approach subfactors under which ASD contends that KENROB's proposal was unreasonably evaluated are: (1) enterprise services, (2) domain services, (3) initial transition plan, and (4) contractor performance work statement and work breakdown structure. Under the management approach evaluation factor, ASD contends it was unreasonably evaluated under the metrics and service level agreement reporting and management subfactor. In addition, ASD argues that the agency technical evaluators unreasonably failed to evaluate the affordability of technical proposals, despite an indication in the solicitation's evaluation scheme that they would do so.

Our standard in reviewing a protester's challenges to an agency's evaluation of proposals is to examine the record to determine whether the agency's judgment was reasonable and consistent with stated evaluation criteria, and with applicable statutes and regulations. ESCO, Inc., B-225565, Apr. 29, 1987, 87-1 CPD para. 450 at 7. After reviewing all of ASD's challenges to the evaluation of the technical and management proposals, we find that under one of the technical subfactors, initial transition plan, the agency evaluators deviated from the stated evaluation scheme; we disagree with ASD on the remainder of its challenges.

As set forth above, the technical evaluation factor here identified six subfactors, the first three of which were equally weighted and more important than the second three, which were also equally weighted. One of the first group of three equally weighted and more important subfactors was an assessment of a proposal's initial transition plan. In this regard, the RFP advised potential offerors that the agency would –assess the soundness, credibility, quality and affordability of the Offeror's approach for achieving [Statement of Objectives] 6.0.— RFP at 78.

The Statement of Objectives guidance to potential offerors regarding the initial transition period was directed towards ensuring continuity of performance during the first 30 days after the beginning of performance. Specifically, this guidance, in its entirety, stated:

Ensure all existing and applicable services and support are successfully transitioned from existing WHS incumbent contractors to the WITS [WHS Information Technology Support] Contractor within 30-days after notice-to-proceed on the contract in accordance with the Contractor's Initial Transition Plan. Ensure no significant disruptions to user services and support occur during the transition period. During the initial 30-day transition period, the WITS Contractor is fully accountable and responsible for successful performance of all objectives and requirements on the contract.

AR at 340-41. In addition, section L of the RFP asked that offerors use this portion of their proposal to –clearly describe the specific actions you will accomplish in the first 30 days to achieve a non-disruptive transition of responsibilities from incumbent WHS contractors to the WITS contract.— RFP at 72.

KENROB's initial transition plan both addressed the steps it would take during the initial 30 days, and contained a significant discussion of things it would do after that time. See AR at 643-49. The agency evaluators assessed KENROB's transition plan as [deleted], and the following comments regarding the plan were included in the Source Selection Decision Memorandum:

KENROB's Initial Transition Plan was thorough and addressed all normal transition issues as well as clearly defining when the transition process is complete and implementation begins. In addition, they described an excellent approach to transfer knowledge from incumbent employees and a methodical approach to transition from the current environment of [deleted]. KENROB's Initial Transition Plan contained significant detail and presented sound approaches for the various aspects of transition. Their proposal demonstrated an excellent understanding of the project and a sound approach leaving me certain that KENROB could successfully perform the proposed effort with little Government intervention.

Source Selection Decision Memorandum at 4-5 (emphasis added).

ASD argues that the agency's glowing evaluation of KENROB's initial transition plan focuses, in part, on matters that were not to begin until approximately 120 days after receipt of the notice to proceed—i.e., the transition from [deleted]. See AR at 647-48. Not only do we agree, but the agency concedes that it should not have assessed the transition from [deleted] under this subfactor, as this transition was part of KENROB's method of performance, not part of its plan for the first 30 days. Agency Memorandum of Law at 36. Nonetheless, the agency argues that ASD cannot claim to be prejudiced as a result of this error in the agency's evaluation, because KENROB's initial transition plan contained other strengths that still would have merited the rating of [deleted].

We think ASD can reasonably claim prejudice as a result of the agency's admitted error in the evaluation of the initial transition subfactor. First, there is no way to ascertain from the record, at this juncture, if the agency would have rated KENROB's proposal [deleted] without reliance on the information about the later-scheduled transition from [deleted]. We give little weight to the agency's argument that this rating would not have changed when raised, as here, in the heat of litigation. See USA Fabrics, Inc., B-295737, B'295737.2 Apr. 19, 2005, 2005 CPD 82 at 6; Boeing Sikorsky Aircraft Support, B'277263.2, B'277263.3, Sept. 29, 1997, 97-2 CPD para. 91 at 15. Moreover, the record here shows that ASD also received a rating of [deleted] confidence under this subfactor. If the agency were to conclude that KENROB's rating should have only been [deleted], the SSA might have used that difference in ratings to distinguish between these proposals, instead of finding them essentially equal under the technical factor. As a result, we conclude that ASD has made an adequate showing of prejudice, and we sustain this basis of ASD's challenge to the agency's evaluation.

Our review of the protester's remaining challenges to the technical evaluation leads us to conclude that several of them arise from the same operative facts and none of them states a basis for concluding that the agency's evaluation was unreasonable. For example, under the contractor's performance work statement (CPWS) and work breakdown statement (WBS) subfactor of the technical evaluation factor (hereinafter, the –CPWS/WBS subfactor—), ASD argues that the agency unreasonably raised KENROB's rating in the final evaluation. Under this subfactor, KENROB's proposal initially was assessed [deleted]; in the final evaluation, the proposal's rating rose to [deleted]. The basis for the agency's decision to raise KENROB's rating in this area is set forth in detail below.

The initial evaluation of the KENROB proposal identified one significant strength and one deficiency under the CPWS/WBS subfactor. Specifically, the evaluators noted that KENROB was offering to staff its [deleted], and considered this a significant strength of the proposal. On the other hand, the evaluators also concluded that the PWS portion of KENROB's proposal simply repeated the language of the Statement of Objectives and did not contain a technical description of the work to be performed. AR at 1505. Because of the deficiency noted, the initial proposal was rated [deleted] under this subfactor.

The noted deficiency was pointed out to KENROB during discussions, and its final proposal offered sufficient detail to address the evaluator's concerns. ASD does not challenge this conclusion. On the other hand, the evaluators noted that KENROB had significantly reduced the number of hours it was offering to staff its [deleted]--from [deleted], to [deleted]. AR at 2353. In addition, the record shows that several of the individual evaluators were concerned about this reduction and at least one wrote notes suggesting that the reduction in hours –qualifies at least as a significant weakness.— AR at 2354.

In the materials prepared by the evaluators for the SSA, the evaluators left intact the comment identifying KENROB's initial proposed approach of staffing [deleted], but struck a line through the comment. The materials for the SSA did not include any explanation of the amount by which the hours had been reduced; they also indicated that the previous deficiency had been addressed, and that there were no weaknesses in the final proposal. AR at 1951-52. As a result, KENROB's rating in this area was raised from [deleted] to [deleted]--an increase of two rating levels.

ASD argues that comments reflected in a report prepared by the evaluators, labeled –KENROB Caucus,— AR 2351-58 (but referred to by the parties as the –Caucus Report—), do not support the ultimate conclusions reflected in the briefing for the SSA, and do not support raising KENROB's score by two rating levels given the significant reduction in the number of hours proposed for KENROB's [deleted]. ASD also contends that the concerns about the final KENROB proposal were not adequately presented to the selection official.

The agency responds that there was nothing unreasonable about the changes it made to KENROB's final rating under this subfactor, and that it did not fail to advise the SSA of the significant changes made by KENROB to its proposed proposal. It notes first that the Caucus Report relied upon by ASD in its arguments was not a consensus document, but was simply a compilation of all of the evaluators' individual views. In addition, it argues that its approach of striking a line through the significant strength identified in the initial proposal served the purpose of flagging the matter, and that the matter was discussed with the selection official during the briefing. CO's Statement at 52. Finally, the agency contends that since KENROB addressed the previously identified deficiency, and since there was no minimum requirement for the number of hours an offeror proposed [deleted], there was nothing unreasonable about the PEB's consensus conclusion, reflected in the briefing for the SSA, that the final proposal should be rated [deleted].

We have reviewed all of the materials cited by the protester in the record here, and we agree with the agency. First, our review of the Caucus Report provided in the record leads us to conclude that the views expressed there are the views of individual evaluators about the changes they have seen from the initial to the final proposal; the views quoted by ASD do not appear to be consensus views. In addition, there is no requirement that each of the individual evaluator views must be reflected in the briefing for the selection official. The overriding concern in the evaluation process is that the final results accurately reflect the actual merits of the proposals, not that they be mechanically traceable back to the isolated comments or ratings of individual evaluators. Manufacturing Eng'g Sys., Inc., B-293299.3, B'293299.4, Aug. 3, 2004, 2004 CPD para. 194 at 6.

Finally, we are not convinced by ASD's arguments that the approach used by the evaluators to brief the SSA resulted in obscuring the ways in which the assessments of the initial and final proposals changed. While the protester, and even our Office, might have found a narrative consensus report easier to review, we see nothing unreasonable about the approach of reflecting the changes from the initial to the final proposal on the briefing slides prepared for the selection official. In addition, we find credible the CO's representation that the approach generated discussion during the briefing for the SSA. Moreover, the agency points out that there seems to be very little difference in the hours ultimately proposed by KENROB for [deleted] services and those proposed by ASD, which also received a rating of [deleted] under this subfactor. Agency Memorandum of Law, July 28, 2006, at 28-30.

In terms of the other challenges to the technical evaluation raised by ASD, most follow the same pattern as above. Specifically, ASD points to unfavorable comments in the Caucus Report, suggests that the comments are consensus views, and argues that the ultimate rating reflected in the briefing for the SSA is inconsistent with those views. We find nothing about these arguments that leads us to conclude that the evaluation here was unreasonable, other than our conclusion above regarding the evaluation of KENROB under the initial transition plan subfactor.

Finally, ASD argues that the agency abandoned the evaluation scheme by not assessing affordability under each of the technical and management subfactors. In this regard, we note that the RFP indicates under each technical and management subfactor that the agency –will assess the soundness, credibility, quality and affordability of the Offeror's approach.— RFP at 78-79. In response, the agency makes no credible argument that its technical evaluators (as opposed to the Cost Team) undertook a review of affordability, as the RFP stated it would. The agency argues, however, that ASD cannot claim to be prejudiced as a result of any failure in this area.

We agree with the agency on the question of prejudice. While agencies should not include in their evaluation schemes reviews they do not intend to undertake, we can see no prejudice to ASD resulting from a failure of the non-price evaluators to conduct a review of affordability. At best, ASD asserts only that if the stated scheme had not referenced affordability it –likely would have structured its proposal differently and taken greater pricing risks.— Protester's Comments, Aug. 9, 2006, at 53. Unlike the agency's abandoned assessment of –completeness— in the price evaluation--where offerors were required to provide pricing information sufficient to establish traceability to the work breakdown statement--we think there is no reasonable chance that ASD was harmed by the failure of the technical evaluators to assess affordability. As a result, we deny this contention as well.

RECOMMENDATION

We find that the agency conducted flawed discussions with KENROB regarding price, improperly abandoned the review of completeness in its price evaluation, and performed an unreasonable evaluation of the KENROB proposal under the initial transition plan subfactor of the technical evaluation factor. As a result, we recommend that the agency reopen discussions, request revised proposals, evaluate those submissions consistent with the provisions of the solicitation, and make a new selection decision.

If, at the conclusion of the reevaluation, ASD's proposal is found to represent the best value to the government, we recommend that the agency terminate KENROB's award, and make award to ASD. We also recommend that the agency reimburse the protester the costs of filing and pursuing the protest, including reasonable attorneys' fees. Bid Protest Regulations, 4 C.F.R. sect. 21.8(d)(1) (2006). As required by section 21.8(f) of our Regulations, ASD's claim for such costs, detailing the time expended and the cost incurred, must be submitted directly to the agency within 60 days of receiving this decision.

The protest is sustained.

Gary L. Kepplinger

General Counsel



[1] The agency report in this protest was provided on CD-ROM with –Bates Stamp— numbers at the bottom of each page. As with the citation to the RFP above, this decision will generally cite to the specific document involved. For certain materials less commonly encountered, or less easily cited, however, we will cite to the agency report's Bates Stamp number for the specific page of the document involved.

[2] The CLINs were: Enterprise, Domain and Asset Management (CLIN 0001), which contained two sub-CLINs, one for all labor and materials necessary to accomplish the Statement of Objectives appended to the solicitation (CLIN 0001AA), and one for the award fee (CLIN 0001AB); Surge and Special Support (CLIN 0002); Travel (CLIN 0003); Data Deliverables (CLIN 0004); Life Cycle Replacement Support (CLIN 0005); and IT Infrastructure Tools (CLIN 0006).

[3] The RFP's estimate of $8,1156,335 for the base period of CLIN 0001 clearly contained a typographical error, which was later clarified by the agency in written responses to offeror questions; the correct amount was $8,156,335. AR at 597. These written questions and answers were later incorporated into the solicitation by reference. RFP amend. 2, at 6.

[4] The agency's discussions letter to KENROB appears in the record at page 1704, but the version there omitted the second page of the 2-page attachment identifying weaknesses, significant weaknesses, and deficiencies. This matter was raised with the parties in this case, who directed our Office to another copy of the attachment found in KENBOB's final proposal revisions. This citation is to the attachment found there; when possible the citations will instead be to the letter itself as provided in the agency report.

[5] The above description of the conversation during which ASD sought clarification from the CO is not in dispute. We will set forth later the areas where there is divergence between ASD's and the CO's description of this conversation.

[6] With respect to the protester's contention that the rule in Peckham is inapplicable here because that case involved oral direction that was allegedly in conflict with the terms of a solicitation, rather than the terms of a discussions letter, see Protester's Final Comments, Sept. 1, 2006, at 9-10, we would direct the protester to the full discussion in that decision at pages 6-9. The alleged oral direction there also contradicted the written discussions letter, as is the case here. We think application of a –bright-line— rule in this area is both fair and in the interest of the procurement system as a whole.

[7] During the course of this protest, KENROB attempted to show that it anticipated the addition of CLIN 0004 to CLIN 0001 to determine the total cost for CLIN 0001 by pointing to a table in its final proposal which plots the cost of CLIN 0001, together with what it calls –Corresponding Optional CLINs 0002 thru 0004,— against the cost of what it terms –Total Gov't Target Budget for CLINs 0001 – 0005.— AR at 1820. In our view, this chart raises more questions than it answers, not the least of which is whether KENROB considered the data deliverables required here to be an optional requirement--a reading not supported by the RFP. This chart also does nothing to explain why KENROB failed to include its price for CLIN 0004 in the calculation of its award fee.

[8] The final judgment of the SSA was that these two proposals were essentially equal under the most important evaluation factor, technical approach, and that any perceived edge for the ASD proposal as a result of its [deleted] rating under the human resources subfactor under the management approach factor (as opposed to KENROB's rating of [deleted] under that subfactor), and its higher rating under the past performance factor, was not sufficient to offset KENROB's [deleted] lower price. Source Selection Decision Memorandum, supra. As a result, the SSA concluded that KENROB's lower price made its proposal the best value to the government. Id.