Honeywell Technology Solutions, Inc.; Wyle Laboratories, Inc.

B-292354; B-292388, Honeywell Technology Solutions, Inc.; Wyle Laboratories, Inc., September 2, 2003

DOCUMENT FOR PUBLIC RELEASE

The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Honeywell Technology Solutions, Inc.; Wyle Laboratories, Inc.

File: B-292354; B-292388

Date: September 2, 2003

Richard Vacura, Esq., Holly Emrick Svetz, Esq., John R. Zitko, Esq., and Peter C. Sales, Esq., Morrison & Foerster, for Honeywell Technology Solutions, Inc., and Richard L. Moorhouse, Esq., James P. Gallatin, Jr., Esq., David T. Hickey, Esq., and Kurt D. Ferstl, Esq., for Wyle Laboratories, Inc., the protesters.

Robert J. Symon, Esq., and Christyne K. Brennan, Esq., Spriggs & Hollingsworth, for Sverdrup Technology, Inc., the intervenor.

Sumara M. Thompson-King, Esq., and Sandra M. Wozniak, Esq., National Aeronautics & Space Administration, for the agency.

John L. Formica, Esq., and James A. Spangenberg, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.

DIGEST

Agency's evaluation of proposals was not reasonable where the agency, during the cost realism evaluation, made significant adjustments to the protester's proposed staffing levels in calculating the most probable cost associated with the proposal that were inconsistent with the agency's view during the technical evaluation of the proposal that the proposed staffing level was appropriate, and where the record casts serious doubts on the reasonableness of the government estimate and the mechanical way it was used in the cost realism evaluation.

DECISION

Honeywell Technology Solutions, Inc. and Wyle Laboratories, Inc. protest the award of a contract to Sverdrup Technology, Inc. under request for proposals (RFP) No. 13SSC-O-02-38, issued by the National Aeronautics & Space Administration (NASA), for test operations services at the John C. Stennis Space Center and George C. Marshall Space Flight Center. The protesters primarily argue that the agency's cost realism evaluation of proposals was unreasonable.

We sustain the protests.

The solicitation is for the consolidation of services for the Stennis and Marshall centers that had previously been acquired under separate contracts. The record reflects that NASA chose to consolidate the contracts in order to obtain, among other things, improvements in efficiency and safety, more competition and lower costs to the government, and flexibility in staffing and management across the two centers. Agency Report (AR), Tab 3, Acquisition Strategy Meeting Presentation, at 15; RFP at 69.

The RFP provided for the award of a cost-plus-award-fee contract for a base period of 2 years with two 2-year options. The successful contractor will furnish the management, labor, facilities, materials and equipment (except for that to be furnished by the government) and "do all things required to provide Test Operations Services" for the Stennis and Marshall centers. RFP at 24. The bulk of this contract is for labor hours. Hearing Transcript (Tr.) at 28.

The RFP contained a detailed performance work statement (PWS) divided into three sections. Section one of the PWS provided for the "management and administration functions . . . necessary to successfully execute testing" at Stennis and Marshall, with the "focus on accomplishing safe test operations using sound business practices." RFP at 70. Section two of the PWS provided for the "test and engineering core capabilities" at Stennis and Marshall, which required the maintenance of a test capability at the centers "independent of whether there is an active test program or not." RFP at 78. The third section of the PWS was for "test capabilities" work "necessary to develop, construct, and operate test systems required for successful completion of a test program." RFP at 86.

The RFP stated that award would be made to the offeror submitting the proposal representing the best value to the government, considering the essentially equally weighted evaluation factors of mission suitability, past performance, and cost. The mission suitability factor was comprised of the following four subfactors (with the relative weight of each of the subfactors being disclosed in the RFP): technical performance (450 points); management (300 points); safety, health, and mission assurance (SH&M) (150 points); and small disadvantaged business (SDB) participation (100 points). RFP at 301.

The RFP specified that the agency would evaluate proposals for cost realism, and noted that because "the proposed cost of the work (and rates proposed) may be a significant indicator of an offeror's understanding and ability to perform the PWS," the proposals' "Mission Suitability scores may be adjusted for lack of cost realism." RFP at 301. The RFP included a table detailing the point deductions that would be made to mission suitability scores based on the percentage difference between the proposed costs and the most probable costs as calculated by the agency ( e.g. , an adjustment of 6 to 10 percent to an offeror's proposed costs to arrive at a most probable cost as calculated by the agency would result in a deduction of 50 points from the proposal's mission suitability score). RFP at 301-02.

The RFP required offerors to submit detailed staffing plans, and provided for the evaluation of the plans under the technical performance subfactor of the mission suitability factor. In this regard, offerors were to "[p]rovide a Staffing Plan for obtaining and maintaining a qualified workforce for the contract, including all subcontractors." This staffing plan, among other things, was to "[d]etail the number of personnel per skill category and by organizational element for the total work force" and address the "selection of skill levels and job qualifications and their relationship to the PWS for the total workforce." Offerors were also to "[i]nclude qualification standards for all positions proposed," and "[a]ddress fluctuating work force requirements, cross-utilization of multi-skilled personnel and retention of qualified personnel." RFP at 276-77.

The RFP (at 303) provided detailed information regarding the agency's intended evaluation of each offeror's staffing plan under the technical performance subfactor, stating in part that:

[t]he proposed Staffing Plan will be evaluated for soundness and adequacy for obtaining and maintaining a qualified workforce, including subcontractors. The numbers, skill categories, (particularly critical skills), and qualifications of personnel for each organizational element will be evaluated for the ability to perform assigned tasks, flexibility of approach and overall understanding of the requirements. The offeror's staffing plans and procedures will be evaluated for realism, particularly in terms of fluctuating work force requirements, cross-utilization of multi-skilled personnel and retention of qualified personnel. The offeror's Staffing Plan should identify retention of key personnel, and should identify by name and position or classification of all additional personnel. The offeror's sources and indicated availability of personnel will be evaluated for realism in providing the staffing compatible with the approach.

After issuing the RFP, the agency determined that because of questions raised by potential offerors regarding the consistency of the workload as it relates to actual testing, it would be "unreasonable to have offerors predict the staffing requirements" for section three of the PWS. Contracting Officer's Statement at 13; see AR at 21. Accordingly, the RFP was amended to provide a staffing estimate that included job titles as well as level and actual numbers of personnel for section three of the PWS, which estimated a total of 103 and 50 positions for Stennis and Marshall, respectively. RFP amend. No. 1, at 12-13.

Finally, the RFP "cautioned" each offeror "to submit its most realistic and competitive offer initially, since the Government intends to award a contract based on initial offers received." The RFP added here that "[d]iscussions will be held only if award on the basis of initial offers is determined not to be in the Government's best interest." RFP at 270.

The agency received proposals from five offerors, including Honeywell, Wyle, and Sverdrup. The mission suitability, past performance, and cost proposals were evaluated by the source evaluation board (SEB).

Honeywell's proposal was evaluated as "excellent" with a total score of 920 out of 1,000 points under the mission suitability factor, with evaluations of "excellent" under the technical performance, management, and SH&M subfactors, and "very good" under the SDB participation subfactor. [1] Honeywell's proposal was also evaluated as "very good" under the past performance factor, with a proposed total cost of114,416,312. AR, Tab 14, Source Selection Authority (SSA) Initial Briefing, at 45-46, 48. Although the SEB found in its technical evaluation of Honeywell's proposal that "staffing levels for all areas appear to be appropriate," AR Tab 14, Honeywell Proposal's Strengths and Weaknesses, at 7, [2] the SEB found, in evaluating Honeywell's cost proposal, that Honeywell's proposed staffing of a total of 248 full time equivalent (FTE) positions was not adequately justified, and adjusted Honeywell's staffing up to 291 FTEs (within 2 FTEs of the agency's independent government staffing estimate (IGSE) of 293 FTEs) in calculating the most probable cost of Honeywell's proposal. AR, Tab 44, Honeywell's Cost Evaluation Process & Results (June 12, 2003); Agency's Post-Hearing Comments, exh. B, Cost Proposal Evaluation Results (July 22, 2003). This resulted in an upward cost adjustment of15,426,351 to Honeywell's proposed cost for a most probable cost of $129,842,663. AR, Tab 14, SSA Initial Briefing, at 48. Additionally, because Honeywell's proposed cost differed from NASA's estimated most probable cost by 13.5 percent, Honeywell's evaluated score under the mission suitability factor of 920 points was downwardly adjusted by 100 points (to 820 points) in accordance with the solicitation. Id. at 45, 48. Honeywell's proposal was ultimately considered by the SEB as "very good" under the mission suitability and past performance factors, at a most probable cost of129,842,663. Id. at 87.

Wyle's proposal was evaluated as "good" with a total score of 600 out of 1,000 points under the mission suitability factor, with evaluations of "good" under the technical performance, management, and SH&M subfactors, and "very good" under the SDB participation subfactor. AR, Tab 14, SSA Initial Briefing, at 80. Wyle's proposal was also evaluated as "good" under the past performance factor, with a proposed total cost of95,321,282. Id. at 81-82. The SEB noted as a "weakness" in Wyle's proposal under the technical performance subfactor that "Wyle['s] approach of 241 full time employees appears to be insufficient to handle the projected NASA workload." AR, Tab 14, Wyle Proposal's Strengths and Weaknesses, at 6. In the cost evaluation, the SEB adjusted Wyle's staffing up to 291 FTEs in calculating Wyle's proposal's most probable cost. [3] AR, Tab 48, Wyle's Cost Evaluation Process & Results (June 12, 2003); Agency's Post-Hearing Comments, exh. B, Cost Proposal Evaluation Results (July 22, 2003). This resulted in an upward cost adjustment of20,511,749 to Wyle's proposed cost for a most probable cost of $115,833,031. AR, Tab 14, SSA Initial Briefing, at 82. Additionally, because Wyle's proposed cost differed from its evaluated most probable cost by 21.5 percent, Wyle's evaluation under the mission suitability factor was adjusted downward by 200 points (to 400 points). Id. at 80, 82. Wyle's proposal was ultimately considered by the SEB as "fair" under the mission suitability factor and "good" under the past performance factor, at a most probable cost of115,833,031. Id. at 87.

Sverdrup's proposal was evaluated as "excellent" with a total score of 983 points under the mission suitability factor, with evaluations of "excellent" under each of the subfactors to the mission suitability factor, and "very good" under the past performance factor, with a proposed total cost of126,060,425. AR, Tab 14, SSA Initial Briefing, at 69, 87. The SEB found in its cost evaluation that Sverdrup's proposed staffing of a total of 287 FTEs was not adequately justified and adjusted Sverdrup's staffing up to 297 FTEs. AR, Tab 50, Sverdrup's Cost Evaluation Process & Results (June 12, 2003); Agency's Post-Hearing Comments, exh. B, Cost Proposal Evaluation Results (July 22, 2003). This resulted in an upward cost adjustment of $3,387,718 to Sverdrup's proposed cost for a total most probable cost of $129,448,143. AR, Tab 14, SSA Initial Briefing, at 72. Given that there was only a 2.7 percent difference between Sverdrup's proposed cost and most probable cost, no adjustment was made to Sverdrup's evaluation under the mission suitability factor. Id. at 69, 72. Sverdrup's proposal was ultimately considered by the SEB as "excellent" under the mission suitability factor and "very good" under the past performance factor, at a most probable cost of129,448,143. Id. at 87.

The SSA was briefed as to the results of the SEB's evaluation of proposals. The SSA noted in his source selection decision "that Sverdrup was the only company not to receive a cost realism adjustment to its Mission Suitability score." AR, Tab 15, Source Selection Decision, at 9. The SSA ultimately determined that Sverdrup's proposal represented the best value to the government, and "that award, without discussions, was in the Government's best interest." Id. The offerors were informed that a contract under the solicitation had been awarded to Sverdrup, and after requesting and receiving debriefings, Honeywell and Wyle filed these protests.

The protesters argue that the agency's cost realism evaluation was unreasonable. They argue that the IGSE is unreasonable and that the agency used the IGSE in an overly mechanical manner, with the result being that the offerors' proposed staffing levels were adjusted upwards to approximate the IGSE, with concomitant increases in the most probable costs associated with their proposals, even though the protesters' proposed staffing levels were reasonable and consistent with the RFP.

Where agencies evaluate proposals for the award of cost-reimbursement contract, an offeror's proposed costs are generally not dispositive because, regardless of the costs proposed, the government is bound to pay the contractor its actual and allowable costs. Pueblo Envtl. Solution, LLC , B-291487, B-291487.2, Dec. 16, 2002, 2003 CPD paragraph 14 at 13. Consequently, a cost realism analysis must be performed by the agency to determine the extent to which an offeror's proposed costs represent what the contract should cost, assuming reasonable economy and efficiency. Federal Acquisition Regulation (FAR) Sections 15.305(a)(1), 15.404-1(d); CACI, Inc.-Fed. , B-216516, Nov. 19, 1984, 84-2 CPD paragraph 542 at 5. In reviewing an agency's evaluation of proposals and source selection decision, our review is confined to a determination of whether the agency acted reasonably and consistent with the stated evaluation factors. Pueblo Envtl. Solution, LLC , supra .

Although the FAR does not specify what is required to be "documented in the contract file" in support of an agency's evaluation of proposals, FAR Section 15.305(a), the fundamental principle of government accountability dictates that an agency maintain a record adequate to allow for the meaningful review of the merits of a protest. Checchi and Co. Consulting, Inc. , B-285777, Oct. 10, 2000, 2001 CPD paragraph 132 at 6; Matrix Int'l Logistics, Inc. , B-272388, B-272388.2, Dec. 9, 1996, 97-2 CPD paragraph 89 at 5. An agency which fails to adequately document its evaluation of proposals or source selection decision bears the risk that its determinations will be considered unsupported, and absent such support, our Office may be unable to determine whether the agency had a reasonable basis for its determinations. Matrix Int'l Logistics, Inc. , supra. That is not to say that our Office, in determining the reasonableness of an agency's evaluation and award decision, limits its review to the contemporaneous evaluation and source selection documentation. Rather, we will consider, in addition to the contemporaneous documentation, all information provided to our Office for consideration during the protest, including the parties' arguments and explanations, and testimony elicited at a hearing, so long as the information is credible and consistent with the contemporaneous record. Id. ; NWT, Inc.; PharmChem Labs., Inc. , B-280988, B-280988.2, Dec. 17, 1998, 98-2 CPD paragraph 158 at 16. In considering the entire record, we accord greater weight to contemporaneous evaluation and source selection material than to the parties' later explanations, arguments, and testimony. Matrix Int'l Logistics, Inc. , supra.

Here, as discussed more fully below, the record before us, which includes both the contemporaneous written record as well as testimony of three members of the SEB, is inconsistent or incomplete to such an extent that we cannot find the agency's evaluation of proposals to be reasonable. Specifically, the SEB found in the technical evaluation of Honeywell's proposal that its proposed staffing level was appropriate and warranted a "strength," and then determined in the cost realism evaluation that Honeywell's proposed staffing level was significantly inadequate. The agency did not reasonably reconcile this anomaly in the contemporaneous record or at the hearing. Moreover, the record raises serious questions concerning the reasonableness of the agency's development and use of the IGSE in the cost evaluation to determine the reasonableness of the offerors' proposed staffing levels.

Regarding Honeywell's proposed staffing, the SEB identified nine "strengths" in Honeywell's proposal and one "weakness" in evaluating Honeywell's proposal under the technical performance subfactor paragraph relating to the staffing plans." [4] Among the strengths noted by the SEB were "Honeywell has provided [DELETED]." AR, Tab 14, Honeywell's Proposal's Strengths and Weaknesses, at 7. Additionally, the record reflects that, based upon its review of two exhibits in Honeywell's proposal pertaining to its proposed staffing plan (detailing Honeywell's staffing by, among other things, PWS section, skill category, number of personnel, and minimum qualification standards proposed), the SEB found that one exhibit "demonstrates that Honeywell has thought through the critical skill mix required to operate the [centers]," and that the other exhibit "further demonstrates a strong knowledge of the qualification standards required by each permanent member of the [contract work team]." Id. As mentioned previously, Honeywell's proposal received an overall rating of "excellent" under the technical performance subfactor. AR, Tab 14, SSA Initial Briefing, at 40.

However, during its cost realism evaluation of Honeywell's proposal, the same SEB found that the proposal had "limited supporting rationale for being 45 FTE's below the Government estimate from day one." AR, Tab 43, Honeywell Cost Realism Rationale, at 1. The contemporaneous documentation of the agency's cost realism evaluation of Honeywell's proposal with regard to proposed staffing is two pages long, with only one page actually addressing the agency's "rationale" for the adjustments made. [5] On this page, the agency identified each of the specific skill categories where adjustments were found to be necessary in the proposed staffing, typically concluding in each case, with little further elaboration, that Honeywell had not provided "sufficient rationale" for its proposed staffing. [6] This evaluation resulted in a significant increase in the evaluated most probable cost of Honeywell's proposal, as well as a significant deduction from the proposal's mission suitability score, such that Honeywell's proposal score fell from the "excellent" range to the "very good" range.

The contemporaneous record, however, does not discuss or attempt to reconcile the SEB's conclusions in the cost evaluation, where Honeywell's staffing plan was deemed "inadequate" by 43 FTEs, with the evaluation of the staffing plan under the technical performance subfactor, where Honeywell's plan was found to contain "strengths" and an "appropriate" number of proposed personnel. Thus, the contemporaneous record provides no basis to conclude whether the SEB's evaluation of Honeywell's proposed staffing under the technical performance subfactor or with regard to cost realism, or either, was reasonable.

The agency was afforded an opportunity to explain this inconsistency at a hearing conducted at our Office, yet failed to do so. For example, one member of the SEB testified that the SEB performed a comprehensive evaluation of the offerors' (including Honeywell's) proposals under the technical performance subfactor with regard to staffing. Tr. at 132, 148. This individual also testified that the SEB in the technical evaluation evaluated the positions and labor mix proposed. Tr. at 110, 112, 128. This same individual testified that although he was very familiar with the IGSE, given that he had prepared it with regard to the requirements for the Marshall center, Tr. at 135, and had "looked at . . . the numbers" of staff proposed by the offerors at this time, Tr. at 115, 128, neither he nor the SEB specifically evaluated the staffing levels proposed under the technical performance subfactor. Tr. at 129, 151-52. Rather, according to this individual's testimony here, the SEB actually evaluated the offerors' proposed staffing levels only when performing its cost evaluation, Tr. at 115, 137-39, even though the solicitation expressly provided that the "numbers, skill categories, (particularly critical skills), and qualifications of personnel for each organizational element" would be evaluated under the technical performance subfactor. See RFP at 303.

Another member of the SEB also testified that the SEB had performed a comprehensive evaluation of the offerors' staffing plans under the technical performance subfactor, Tr. at 300, and that he was aware of the IGSE while evaluating proposals under the technical performance subfactor, given that he had prepared the IGSE with regard to the Stennis center requirements. Tr. at 296. This individual testified that Honeywell's staffing plan was detailed, that he knew during the evaluation of Honeywell's proposal under the technical performance subfactor that the staffing level Honeywell proposed differed from the IGSE, and that during the evaluation under the technical performance subfactor the SEB looked at the general staffing levels in the proposals. Tr. at 296-98. This individual also testified, however, that it was not until the SEB performed its cost realism evaluation that the SEB actually considered Honeywell's staffing levels in detail. Tr. at 295-96.

The Chairperson of the SEB conceded that the evaluation record was inconsistent with regard to Honeywell's staffing plan. Tr. at 361. He also testified that "the Honeywell staffing was not unreasonable." [7] Tr. at 359. The Chairperson added later in his testimony, however, that Honeywell's proposed staffing level was inadequate, and that the SEB ultimately concluded that Honeywell "could not do the job without risk with the number that they proposed." Tr. at 365.

As indicated, the SEB members' post-protest explanations as to why Honeywell's detailed staffing plan was found by the SEB under the technical performance subfactor to provide a number of "strengths" and an "appropriate" staffing level, yet was found during the cost realism evaluation to have a significantly understated staffing level, center on their assertion that the SEB did not actually evaluate the staffing levels proposed by offerors until the SEB performed its cost realism evaluation. These post-protest explanations are inconsistent with the contemporaneously documented technical evaluation conclusions regarding the strength of Honeywell's staffing plan and the adequacy of Honeywell's proposed staffing levels. As mentioned previously, the contemporaneous record demonstrates that SEB specifically found Honeywell's "[s]taffing levels for all areas appear to be appropriate." AR, Tab 14, Honeywell Proposal Strengths and Weaknesses, at 7. Moreover, the record shows that the SEB evaluated and reached conclusions regarding staffing levels proposed by other offerors during the SEB's evaluation of proposals under the technical performance subfactor. In this regard, as noted above, the SEB concluded in its evaluation under the technical performance subfactor that Wyle's "approach of 241 full time employees appears to insufficient to handle the projected NASA workload," Wyle Proposal's Strengths and Weaknesses, at 6, and the contemporaneous record of the SEB's evaluation of another offeror's proposed staffing level under the technical performance subfactor includes a detailed discussion as to why the SEB determined that the particular staffing level proposed constituted a "significant weakness." AR, Tab 14, Second Subtab, at 7. Further, notwithstanding the post-protest denials by the SEB members that staffing levels were considered in the technical performance subfactor evaluation, these individuals testified that the SEB performed a "comprehensive" evaluation of the offerors' proposals, including staffing plans, under the technical performance subfactor. Given the inability of the agency to credibly explain the apparent inconsistency between its evaluation of Honeywell's staffing plan under the technical performance subfactor and its cost realism evaluation, we cannot find the agency's determinations here to be reasonable with regard to either its technical performance subfactor evaluation or its cost realism evaluation.

Moreover, as discussed below, the record also raises serious questions as to both the development of the IGSE and its use during the evaluation process. Turning to the development of the IGSE, we note first by way of background it is the sum of the separately prepared estimates for Marshall and Stennis.

With regard to the Marshall portion of the IGSE, which totaled 70 FTEs, the SEB member who prepared this portion of the IGSE explained that the estimate was developed in part through a "bottoms up" examination of the agency's needs based upon the requirements set forth in the RFP. Tr. at 36-40, 75-76. In this regard, the contemporaneous record includes a matrix providing a staffing estimate for Marshall setting forth the number of FTEs required by labor category and work to be performed. AR, Tab 36, Government Staffing Estimate Worksheets, at 5. The SEB member who prepared the Marshall estimate testified that Marshall actually needed 70 FTEs in the labor categories set forth in the estimate, that is, that the 70 FTE estimate represented Marshall's actual needs and that any lesser number of FTEs would have been considered "[c]ompletely unacceptable." Tr. at 77-78, 80-81, 100, 107. This SEB member also testified that the Marshall estimate was prepared without any consideration of any efficiencies that may be gained through the consolidation of the Marshall and Stennis requirements into one contract, or through the cross-utilization or cross-training of personnel, as encouraged by the solicitation. Tr. at 96, 229-30.

The SEB member who prepared the Stennis estimate testified that the estimate was prepared based upon his extensive knowledge of the Stennis requirements, information he received regarding staffing from various Stennis personnel, as well as input from the various knowledgeable members of the SEB, including the chairperson. Tr. at 250, 253-54, 264. This SEB member added that he also used the organizational chart for the "entire Stennis contract," which lists positions in addition to those needed to perform the services requested by this solicitation. Tr. at 252; see AR, Tab 60, Stennis Contractor Operations Programs Staffing.

The record reflects that this SEB member also considered a list of all positions covered by the RFP here that was received from the incumbent contractor at Stennis in preparing this portion of the estimate. Tr. at 274-77; AR, Tab 62, Incumbent Contractor's Extension Estimate and Test Operations Monthly Staffing. Although the contractor-provided data listed only 182 positions, the SEB member testified that he made no adjustments to the Stennis estimate of 223 FTEs. Tr. at 277. The SEB member explained that the Stennis contract included much more than the test services included under this RFP, and that the staffing list for test services provided by the incumbent therefore did not include positions--such as general manager--that would be in common for the combination of this RFP's services and the remainder of the services which will be provided under a separate contract. [8] Tr. at 273-77. Although the SEB member characterized his review of the incumbent contractor-provided data as a "detailed analysis," he also conceded that he only "glanced" at the actual positions set forth in the data. Tr. at 281. He added that he looked at the total number of positions listed by the incumbent contractor (182) to see if it was "close" to the Stennis center estimate of 223 positions, and that the contractor-provided data had acted as a "sanity check," because the total of 182 FTEs as provided by the incumbent contractor was, in the SEB member's view, "pretty close" to the IGSE for the Stennis of 223 FTEs. Tr. at 276-77.

This individual added that, in his view, the 223 FTE estimate for the services needed at Stennis was not a requirement, but rather, only an estimate of the number of personnel needed to perform the work required. Tr. at 262-63. Finally, this individual testified that the estimate for the Stennis center was prepared "based upon the statement of work and how we know it would be done today" without consideration of any efficiencies that may be gained, for example, through the consolidation of the Marshall and Stennis requirements into one contract. Tr. at 286.

The record reflects that NASA chose to consolidate the Stennis and Marshall requirements into one contract in order to obtain, among other things, improvements in efficiency and safety, more competition and lower costs to the government, and flexibility in staffing and management across the two centers, AR, Tab 3, Acquisition Strategy Meeting Presentation, at 15; RFP at 69. Consistent with this, the RFP specifically "encouraged" offerors "to propose innovative techniques and methods that would benefit the Government," and advised offerors that they "should not limit proposed methods to processes currently used." RFP at 270. However, the IGSE of 293 FTEs is the sum of two distinct, separate staffing estimates ( i.e. , Marshall's and Stennis's) that were prepared with no consideration of the efficiencies that NASA intended to achieve through the consolidation, Tr. at 96, 229-30, 286, or proposed cross-utilization of personnel (which was also encouraged by the RFP). Tr. at 230; see RFP at 70.

This problem with the way the IGSE was developed is significant because the IGSE was used in what appears to be a mechanical way in the cost realism evaluation. A reasonably derived estimate of labor hours and material costs can provide an objective standard against which the realism of proposed costs can be measured. IT Facility Servs.-Joint Venture , B-285841, Oct. 17, 2000, 2000 CPD paragraph 177 at 6-7; Theta Eng'g, Inc. , B-271065, B-271065.2, June 12, 1996, 96-2 CPD paragraph 76 at 6. However, an agency may not mechanically apply that estimate to determine the most probable costs associated with proposals, without regard to the individual proposal's technical approach. The Jonathan Corp.; Metro Mach. Corp. , B-251698.3, B-251698.4, May 17, 1993, 93-2 CPD paragraph 174 at 11; Kinton, Inc. , B-228260.2, Feb. 5, 1988, 88-1 CPD paragraph 112 at 4. This is so because in some instances an estimate has limited applicability to a particular proposal due to, for example, the skill of the labor force or innovative work methods proposed. In those cases, any rigid reliance on the government estimate could have the effect of arbitrarily and unfairly penalizing (or rewarding) one firm and depriving the government of the benefit available from the different approaches of the various offerors. Accordingly, in order to undertake a proper cost realism evaluation, the agency must independently analyze the realism of an offeror's proposed costs based upon its particular approach, personnel, and other circumstances. The Jonathan Corp.; Metro Mach. Corp. , supra .

Here, the record does not indicate that the agency engaged in an absolutely rigid application of the IGSE to the offerors' proposals. There do appear to be instances where the agency accepted proposed staffing for a particular function or labor category that was less than that reflected in the IGSE, and in other instances, the agency made adjustments to an offeror's proposed staffing that put the probable staffing associated with the proposal at a level between that set forth in the proposal and that provided by the agency in its IGSE.

Nevertheless, the record also reflects that in the vast majority of instances where an offeror proposed a staffing level that differed from the IGSE, the staffing level was adjusted during the cost evaluation to the IGSE staffing level, with the primary documented reason by the agency being that the proposal did not provide "sufficient rationale" for the proposed staffing, with little further elaboration. See AR, Tab 43, Honeywell Cost Realism Rationale; at 2; Tab 47, Wyle Cost Realism Rationale, at 2; Tab 49, Sverdrup Cost Realism Rationale, at 2. As the protesters point out, the consistency of the agency's approach in this regard is readily apparent when the offerors' varying proposed staffing levels are compared to their staffing levels as adjusted by the agency and the IGSE. That is, although the agency found "that there was a wide range of disparate approaches from the offerors," with the five offerors proposing staffing levels that varied considerably ( i.e. , 198, 241, 248, 260, and 287 FTEs), the SEB adjusted the staffing levels proposed by all five offerors to within 4 FTEs of the IGSE of 293. Agency's Post-Hearing Comments at 10; exh. B, Cost Proposal Evaluation Results (July 22, 2003). This, along with the sparse evaluation documentation, suggests that the IGSE was used in a mechanical way in the cost realism evaluation, notwithstanding the encouragement in the RFP for proposing innovative approaches.

That the agency used the IGSE in the cost realism evaluation in a mechanical way is supported by other evidence in the record. For example, before this issue became the gravamen of the protests, the agency stated in its initial report on the protests that "[u]niformly, NASA base lined [the offerors'] FTEs against the government staffing estimate." AR at 22, 25. Moreover, given the testimony of the SEB member that the Marshall estimate was essentially inflexible and any proposed staffing plan that did not comport precisely with the Marshall estimate would be "[c]ompletely unacceptable," the fact that variances from the estimate were not tolerated was understandable. [9] Tr. at 78, 80-81, 100, 107. Of most significance in showing that the IGSE was used in a mechanical way is the fact that Honeywell's proposed staffing was characterized as "appropriate" during the evaluation of its proposal under the technical performance subfactor, but was then adjusted upwards by 43 FTEs to within 2 FTEs of the IGSE during the cost realism evaluation.

In sum, we find the agency's cost realism evaluation unreasonable.

As a final matter, the agency and the intervenor argue that, in any event, neither Honeywell nor Wyle was prejudiced by the agency's flawed evaluation. Our Office will not sustain a protest unless the protester demonstrates a reasonable possibility that it was prejudiced by the agency's actions, that is, unless the protester demonstrates that but for the agency's action, it would have had a substantial chance of receiving the award. McDonald-Bradley , B-270126, Feb. 8, 1996, 96-1 CPD paragraph 54 at 3; Statistica, Inc. v. Christopher , 102 F.3d 1577, 1581 (Fed. Cir. 1996).

Although we agree that Honeywell would not have been prejudiced by the agency's actions, if the only flaw in the record had been that Honeywell's proposal was overrated under the technical performance subfactor because it proposed an inadequate staffing level (so it should not have been evaluated as "excellent"), we simply cannot determine from this record which aspect of the agency's evaluation was reasonable and which was unreasonable. That is, if the SEB's evaluation of Honeywell's proposal under the technical performance subfactor was reasonable, where the SEB found that "[s]taffing levels for all areas appear to be appropriate," there may have been no need for any adjustment to Honeywell's proposed costs, such that Honeywell's proposal should have been considered during the source selection as "excellent" (rather than "very good") under the mission suitability factor, at its proposed cost of114,416,312 (rather than $129,842,663). In that case, Honeywell would clearly have a reasonable possibility of being selected for award.

With regard to Wyle (as well as Honeywell), we believe that the numerous inconsistencies in the record are indicative of systemic problems that undermine the reasonableness of the agency's evaluation. Again, we simply cannot determine from this record which aspect of the agency's evaluation was reasonable and which was unreasonable, or whether the agency's evaluation of the technical merit or cost realism of proposals was reasonable at all. Moreover, we believe that questions raised by this record regarding the development of the IGSE and its use in what appears to be a mechanical way during the costs realism evaluation also demonstrate a reasonable possibility that Wyle was prejudiced by the agency's actions, given the significant upward adjustment made by the agency to Wyle's proposed cost, and the concomitant downward adjustment made by the agency to Wyle's evaluation score under the mission suitability factor.

We recommend that the agency review the IGSE to determine whether or not it is reasonably accurate and review the solicitation to determine whether it accurately reflects the agency's actual staffing requirements. If the solicitation is amended, revised proposals should be obtained and a new source selection made. If the solicitation is not revised, the agency should reevaluate proposals, conduct discussions if appropriate, and make a new source selection. [10] If a proposal other than Sverdrup's is selected for award, the agency should terminate Sverdrup's contract and make award to the successful offeror. We also recommend that the protesters be reimbursed the reasonable costs of filing and pursuing their protests, including reasonable attorneys' fees. 4 C.F.R. Section 21.8(d) (2003). The protesters should submit their claims for costs, detailing and certifying the time expended and costs incurred, to the contracting agency within 60 days after receipt of this decision. 4 C.F.R. Section 21.8(f)(1).

The protests are sustained.

Anthony H. Gamboa

General Counsel

[1] The proposals were evaluated under the mission suitability and past performance factors, and each of the mission suitability subfactors, as "excellent" (percentile range of 91-100), "very good" (percentile range of 71-90), "good" (percentile range of 51-70), "fair" (percentile range of 31-50), and "poor" (percentile range of 0-30). AR, Tab 14, SSA Initial Briefing, at 20.

[2] The agency's evaluation record of the technical performance subfactor consists primarily of the SEB's consensus as to the strengths and/or weakness in the offerors proposals.

[3] The SEB noted during its cost realism evaluation of Wyle's proposal that the "rationale [in the proposal] was insufficient to support being 52 FTE's below the Government estimate on the first day of the contract." AR, Tab 47, Wyle Proposal Cost Realism Rationale, at 1.

[4] The identified weakness referred to one of Honeywell's proposed subcontractors [DELETED], which in the SEB's view, among other things, [DELETED]. AR, Tab 14, Honeywell Proposal's Strengths and Weaknesses, at 8. Thus, this weakness has nothing to do with Honeywell's proposed staffing levels.

[5] This sparse documentation is consistent with the cost realism evaluation documentation for the other proposals.

[6] As a typical example, [DELETED] has not provided sufficient rationale to support needing [DELETED], therefore; proposal will be increased by [DELETED]. AR, Tab 43, Honeywell Cost Realism Rationale, at 2.

[7] Because the chairperson used the phrases "I still feel like" and "we feel like" in the same sentence, it is difficult to discern from his testimony regarding the continuing belief that "the Honeywell staffing was not unreasonable" whether he was testifying regarding only his belief or that of the SEB as a whole. Tr. at 359.

[8] The general manager position was the only example provided by the SEB member of these common positions, who did not estimate how many such common positions were not in the incumbent-provided staffing list for testing services.

[9] On the other hand, the use of the IGSE as a "baseline" to which the offerors' staffing levels were adjusted for cost realism analysis purposes appears inconsistent with the view of the SEB member who prepared the Stennis estimate that the total number of positions listed by the incumbent contractor (182) was "pretty close" to his estimate of 223 positions. Tr. at 276-77. It seems incongruous that the SEB member who prepared this estimate was not troubled by its significant variance from the incumbent's staffing estimate, yet when the estimate was used in the evaluation, proposals with different staffing levels were adjusted to approximate the IGSE.

[10] The decision that discussions are not necessary must be reasonably based on the particular circumstances of the procurement, including consideration of the proposals received and the bases for the evaluation judgments and selection decision. For example, where the proposals deviate from the government estimate, the agency should consider the possibility that the proposals that deviate from the government estimate may nevertheless be advantageous to the government, such that discussions should be conducted to address this discrepancy. See The Jonathan Corp.; Metro Mach. Corp. , supra .