Kolaka No eau, Inc., B-291818, April 2, 2003
B-291818: Apr 2, 2003
DIGEST Protest that an agency did not select protester's proposal for a phase I Small Business Innovation Research contract is denied. Although the proposal was recommended for award. The proposal was reasonably not as highly ranked as other proposals for which the agency had sufficient funding to make awards. The SBIR program is designed to increase the participation of small business concerns in federally funded research or research and development (R&D). Federal agencies (such as NASA) with R&D "extramural" budgets in excess of $100 million are required to provide a program under which a portion of the agency's research or R&D effort is reserved for award to small business concerns through a three-phased process.
Kolaka No eau, Inc., B-291818, April 2, 2003
Kolaka No eau, Inc. protests the rejection of its proposal by the National Aeronautics & Space Administration (NASA) under the agency's 2002 Small Business Innovation Research (SBIR)/Small Business Technology Transfer (STTR) program solicitation No. 2002-1.
We deny the protest.
The SBIR program is designed to increase the participation of small business concerns in federally funded research or research and development (R&D). See Small Business Innovation Research Program Act of 1982, 15 U.S.C. Sec. 638 (2000). Pursuant to this authority, federal agencies (such as NASA) with R&D "extramural" budgets in excess of $100 million are required to provide a program under which a portion of the agency's research or R&D effort is reserved for award to small business concerns through a three-phased process. See 15 U.S.C. Sec. 638 (e)(4), (f). Under phase I, small businesses are invited to submit proposals to conduct research on one or more topics specified in the annual SBIR program solicitation. Under phase II, firms that received phase I awards may, on their own initiative, submit proposals for further development work on the topic. Phase III contemplates that, unlike phases I and II, non-SBIR funds will be used to pursue comm ercial applications of the R&D.
NASA issued a program solicitation inviting small business concerns to submit proposals for fixed-price SBIR contracts under the following research areas: aerospace technology, biological and physical research, earth science, human exploration and development of space, and space science. Topics and subtopics were identified for each area. There were a total of 28 topics and 120 subtopics.
Under the aerospace technology area, the solicitation identified the "engineering for complex systems" topic, and, under that, the "modeling and simulation of aerospace vehicles in a flight test environment" subtopic. Program Solicitation Sec. 9.1.1, Aerospace Technology, at 50, 70. Detailed information was provided for each area, topic, and subtopic. With respect to the modeling and simulation of aerospace vehicles in a flight test environment subtopic, on which the protester submitted its proposal, offerors were informed that the agency sought more efficient software tools for predicting and understanding airframe response "under the simultaneous influence of aerodynamics and the control system, in addition to pilot commands." Id. at 70.
Each subtopic was assigned to one of NASA's 10 field centers for evaluation. NASA's Dryden Flight Research Field Center was identified as the lead for four subtopics and a participating center for another subtopic. Dryden was the lead center for the "modeling and simulation of aerospace vehicles in a flight test environment" subtopic.
Offerors were informed that the phase I contracts could be for no more than $70,000 and 6 months. Detailed instructions for the preparation of proposals were provided. The solicitation provided that proposals for phase I contracts must be based on a "unique innovation, . . . be limited in scope to just one subtopic, and . . . be submitted only under that subtopic." Id. Sec. 3, at 5. Format and page limitations were stated for proposals, which were required to be submitted via the Internet. In their technical proposals, offerors were required to, among other things, identify the significance of the innovation proposed, state technical objectives, detail the proposed work plan, and identify key personnel.
Offerors were also informed that NASA would select for award those proposals offering "the best value to Government and the Nation" and that "primary consideration [would be given to] scientific and technical merit, and feasibility of the proposal, and its benefit to NASA." Program Solicitation Sec. 4.1.2, Phase I Evaluation Criteria, at 14. In this regard, offerors were informed that evaluations would be performed "by NASA scientists and engineers and by qualified experts outside of NASA." Id. Sec. 4.1.1, at 13.
The following technical evaluation factors were identified: (1) scientific/technical merit and feasibility; (2) experience, qualifications and facilities; (3) effectiveness of the proposed work plan; and (4) commercial merit and feasibility. The RFP stated that factors (1) through (3) would be point scored and that factor (4) would be adjectivally rated. Factor (1) was stated to be worth 50 percent of the total evaluation weight, and factors (2) and (3) were worth 25 percent each; the sum of a proposal's point scores for these factors comprised the proposal's technical merit score. The technical merit score was stated to be more important than commercial merit. Offerors were also informed that non-cost factors were substantially more important than cost. Id. Sec. 4.12, at 14.
The RFP provided that after evaluation of the proposals under the evaluation factors, proposals would be ranked relative to all other proposals and that "[s]election decisions will consider the recommendations from all Centers, Strategic Enterprises, overall NASA priorities, and program balance." Program Solicitation Sec. 4.1.3, at 14. Offerors were also cautioned that awards would be subject to availability of funds and that there was no obligation to make a specific number of awards under the solicitation.
In response to the RFP, 2,238 proposals were received. Dryden received 108 phase I proposals to evaluate, spread over the five subtopics for which Dryden had responsibility. Under the "modeling and simulation of aerospace vehicles in a flight test environment" subtopic, Dryden received three proposals, including the protester's. Agency Report, Tab 6, Dryden Phase I SBIR/STTR Proposal Ranking, Oct. 22, 2002, at 2.
Each proposal was reviewed by at least two evaluators with knowledge of the subtopic area. Kolaka's proposal received the following evaluation scores:
. Evaluator A Evaluator B
(1) Scientific/technical merit 42 44 and feasibility
(2) Experience, qualifications 22 23 and facilities
(3) Effectiveness of the 18 23 proposed work plan
TECHNICAL MERIT POINTS 82 90
(4) Commercial merit
. NASA Excellent Very Good . Non-NASA Very Good Very Good
Agency Report, Tabs 5a, 5b, SBIR Phase I Technical Evaluation Forms. Kolaka's proposal (priced at $69,998) received an average evaluation score of 86 technical merit points and "very good" commercial merit scores, and was the second highest rated of the three proposals received under this subtopic. The highest-rated proposal (priced at $69,995) under this subtopic received an average technical merit score of 97.5 points and "very good" commercial merit scores, and the lowest-rated proposal (priced at $69,972) received a technical merit score of 76.5 points and commercial merit scores of "below average" (NASA) and "average" (non-NASA). Kolaka's and the highest-rated offeror under this subtopic were recommended for award by the evaluators who rated these proposals; the lowest-rated proposal was not recommended for award.
The evaluation results were presented to subtopic managers to rank the 25 proposals that were recommended for award. /1/ The subtopic managers (or, in some cases, another technical expert) presented the proposals recommended for award (including Kolaka's) to a center ranking committee at Dryden. The presentations were comprised of brief summaries of each proposal and the evaluation results. In ranking proposals, the Dryden center ranking committee established two sets of proposals. It first ranked (from first through fifth) the highest-rated proposal from each of the five subtopics for which Dryden was responsible; the highest-rated proposal in the subtopic for which Kolaka submitted its proposal was ranked third by the committee. The remaining twenty proposals, including Kolaka's, were then ranked between sixth and twenty-fifth. Kolaka's proposal was ranked twentieth by the committee. Contracting Officer's Statement at 5; Agency Report, Tab 10, Final Dryden Center Proposal Ranking.
Dryden and the other NASA centers' rankings and recommendations were then presented to NASA's program management office, which reviewed the selection recommendations and presented the evaluation results to the source selection authority (SSA). Contracting Officer's Statement at 6. The NASA centers recommended a total of 919 proposals for award. Based upon the estimated available budget, the program office ultimately recommended that 270 proposals be selected for award. /2/ Contracting Officer's Statement at 6; Agency Report, Tab 11, Presentation to SSA, at 6. In addition to providing the evaluation results, proposed prices, and disadvantaged or women-owned business status for recommended proposals, the program management office presented to the SSA the number and percentage of recommendations made by each center, the number of subtopics led by each center, and the number and percentage of "success stories" at each center. See Agency Report, Tab 11, Presentation to SSA, 8-11.
The SSA selected 270 proposals (not including Kolaka's) for award. See Agency Report, Tab 12, Source Selection Decision (Nov. 18, 2002). Of the 25 phase I proposals recommended for award at Dryden, only 12 were selected by the SSA to receive awards. These 12 proposals had higher technical merit scores and equal or higher commercial merit scores than Kolaka's proposal. /3/ Following notification of the proposals selected for award and receipt of a debriefing letter, this protest followed.
Kolaka complains that the evaluation of its proposal was unreasonable. /4/ Specifically, the protester argues that evaluator A's assessment of Kolaka's proposal under the effectiveness of the proposed work plan factor was unfair. Evaluator A assigned Kolaka's proposal 18 of the 25 available points (that is, 72 percent) for this factor, because the evaluator found that although the "work plan described is adequate for the proposed effort," the "actual models planned to be used were not detailed, and no work breakdown structure was provided for the effort." Agency Report, Tab 5a, Evaluation Form for Evaluator A, at 1. Kolaka complains that the solicitation did not require a work breakdown schedule and that it provided sufficient information to warrant a higher technical score for this factor. In this regard, Kolaka argues that evaluator B assigned Kolaka's proposal 23 points and found that the protester's "proposed work plan [was] detailed out in a comprehensive fashion, . . . is adequate and will prove to be effective." Agency Report, Tab 5b, Evaluation Form for Evaluator B, at 1.
In reviewing protests against allegedly improper evaluations, it is not our role to reevaluate proposals. Rather, our Office examines the record to determine whether the agency's judgment was reasonable and in accord with the evaluation criteria. Abt Assocs., Inc., B-237060.2, Feb. 26, 1990, 90-1 CPD Para. 223 at 4. Such judgments are by their nature often subjective; nevertheless, the exercise of these judgments in the evaluation of proposals must be reasonable and bear a rational relationship to their announced criteria upon which competing offers are to be selected. Southwest Marine, Inc.; Am. Sys. Eng'g Corp., B-265865.3, B-265865.4, Jan. 23, 1996, 96-1 CPD Para. 56 at 10. The protester's mere disagreement with the agency's judgment does not establish that an evaluation was unreasonable. UNICCO Gov't Servs., Inc., B-277658, Nov. 7, 1997, 97-2 CPD Para. 134 at 7. Moreover, the agency is accorded considerable discretion to determine which proposals will be funded under the SBIR program. Virginia Accelerators Corp., B-271066, May 20, 1996, 97-2 CPD Para. 13 at 2.
As explained below, we find no basis to conclude that the agency's evaluation of Kolaka's proposal was unreasonable. As noted above, NASA considered Kolaka's proposal to be very good, assigning the proposal an average technical merit score of 86 of 100 available points. Although Kolaka complains that one evaluator assigned its proposal fewer points under the effectiveness of the proposed work plan factor than another evaluator, and found problems with its plan that the other evaluator did not, this in and of itself does not demonstrate that the evaluation was unreasonable, inasmuch it is not unusual for evaluators to reach different conclusions and assign different scores when evaluating proposals, since both subjective and objective judgments are involved. /5/ See Medical Info. Servs., B-287824, July 10, 2001, 2001 CPD Para. 122 at 5; Microeconomic Applications, Inc., B-258633.2, Feb. 14, 1995, 95-1 CPD Para. 82 at 9 n.6.
With respect to the effectiveness of the proposed work plan factor, the solicitation explained that an offeror's proposed:
work plan will be reviewed for its comprehensiveness, effective use of available resources, cost management and proposed schedule for meeting the Phase I objectives. The methods planned to achieve each objective or task should be discussed in detail.
Program Solicitation Sec. 4.1.2, at 14. The solicitation instructed offerors that their:
work plan should indicate how, what and where it will be done. . . . The methods planned to achieve each objective or task should be discussed in detail. Schedules, task descriptions and assignments, resource allocations, estimated task hours for each key personnel, and planned accomplishments including project milestones shall be included.
Id. Sec. 3.2.4, at 7.
Here, evaluator A found that although Kolaka's proposal was "adequate" under this factor, the proposal "did not detail how much work was to be done at which points in the schedule and by which participants." Statement of Evaluator A, Feb. 26, 2003, at 2-3. From our review of Kolaka's proposal, we find no basis to object to the evaluator's judgment. /6/ See Agency Report, Tab 4d, Kolaka Technical Proposal, at 16-20. We also note that evaluator B, who assigned Kolaka's proposal 23 of 25 points for this factor, noted that although he judged Kolaka's proposal to be "very good," he recognized that the higher-rated proposal in this subtopic provided a "more comprehensive and complete" work plan that "was more effectively detailed than" Kolaka's. Statement of Evaluator B, Feb. 26, 2003, at 2. In sum, we do not find that Kolaka's technical merit score reflected an unreasonable evaluation. /7/
Kolaka also complains that NASA's ranking and selection of proposals were unreasonable. First, Kolaka objects that the phase I proposals were evaluated by various pairs of evaluators and that no single person evaluated all of the proposals to ensure that there was a "consistent" evaluation to allow for a fair ranking of proposals. However, the use of evaluation teams across the various NASA field centers (such as the Dryden Center) to evaluate the more than 2,000 phase I proposals received by NASA was consistent with the peer review nature of the evaluation of phase I proposals contemplated by the solicitation and the SBIR program. In any event, Kolaka has not shown that NASA's use of teams of evaluators to assess proposals resulted in an inconsistent evaluation that affected the ranking of proposals.
Kolaka also complains that its proposal was not fairly presented by the applicable subtopic manager to Dryden's ranking committee. /8/ Specifically, Kolaka argues that the presentation slides prepared by the subtopic manager to explain Kolaka's proposal were not as numerous as slides prepared for other proposals and that Kolaka's presentation slides did not accurately represent Kolaka's proposal. Kolaka, however, does not demonstrate how the presentation slides presented to the ranking committee were inaccurate or failed to accurately portray Kolaka's proposal. Furthermore, NASA presented evidence showing that the number of presentation slides provided for an offeror did not affect the selection of proposals for funding. See Statement of Dryden SBIR Field Center Program Manager, Feb. 26, 2003, at 7. Moreover, the record provides no support for Kolaka's speculation of bad faith on the part of NASA officials in presenting its proposal to the ranking committee. Government officials are presumed to act in good faith and, where a protester contends that contracting officials are motivated by bias or bad faith, it must provide convincing proof, since this Office will not attribute unfair or prejudicial motives to procurement officials on the basis of inference or suppositions. ACC Constr. Co., Inc., B-289167, Jan. 15, 2002, 2002 CPD Para. 21 at 4.
Kolaka also complains that the methodology for ranking phase I proposals was not consistent across NASA's field centers. That is, the Dryden Center reserved the highest ranking for the highest-rated proposal in each of the five subtopics for which the Dryden Center was responsible, while the remainder of the proposals (that were recommended for award) were then ranked from sixth through twenty-fifth. Kolaka complains that not only was this ranking method not identified in the solicitation, but that other NASA field centers did not reserve the highest rankings for top-rated proposals from each subtopic.
While it is true that not all of NASA's field centers used a ranking scheme that ensured priority in ranking for the highest-rated proposal in each subtopic, we see nothing unfair about this methodology as it was used in this case. Indeed, this methodology provides one means of ensuring program balance to the agency, which the solicitation informed offerors would be considered in the selection decisions. See Program Solicitation Sec. 4.1.3, at 14. In any event, we fail to see how the use of such a ranking methodology at the Dryden Center prejudiced the protester. The top-rated proposals in the five subtopics each received significantly higher technical merit scores than Kolaka's proposal, and none received lower commercial merit ratings. Competitive prejudice is an essential element of every viable protest. Lithos Restoration, Ltd., B-247003.2, Apr. 22, 1992, 92-1 CPD Para. 379 at 5. Where the record does not demonstrate that, but for the agency's actions, the protester would have had a reasonable chance of receiving the award, our Office will not sustain a protest, even if a deficiency in the procurement is found. McDonald-Bradley, B-270126, Feb. 8, 1996, 96-1 CPD Para. 54 at 3; see Statistica, Inc. v. Christopher, 102 F.3d 1577, 1581 (Fed. Cir. 1996).
Kolaka raises numerous other objections to NASA's failure to select the protester's proposal for funding. From our review of the record, we find that that none of Kolaka's arguments provide us with a basis to object to NASA's evaluation, ranking, and selection of proposals. /9/ The fact is that Kolaka's proposal received very good ratings from NASA and was recommended for award. The record reflects that Kolaka's proposal was not selected for award, not because of a misevaluation or mis-ranking of proposals, but because NASA did not have sufficient funding to select all the recommended proposals. Kolaka's proposal was simply not as highly regarded as those that NASA was able to fund.
The protest is denied.
Anthony H. Gamboa General Counsel
1. Proposals that were not recommended for award were not ranked.
2. Initially, the program office recommended 310 proposals be funded, but reduced that number to 270 based upon the amount of funds NASA would have available.
3. The proposed prices of Kolaka's proposal and the 12 proposals selected for award were essentially equal (within several dollars of each other), other than a few proposals that were as much as $10,000 lower than Kolaka's proposed price.
4. Kolaka also complains that NASA used both NASA employees and contractor employees to evaluate proposals. The solicitation specifically informed offerors that NASA may use qualified individuals from outside the government in the proposal review process. Program Solicitation Secs. 4.1.1, 5.5, at 13, 18. NASA states that all non-NASA evaluators who performed such reviews had executed nondisclosure agreements. In any event, both of the evaluators that reviewed Kolaka's proposal were NASA employees.
5. Kolaka complains that NASA did not comply with its Evaluation Guidelines, which require that, where the evaluators' scores are "widely differing," a third evaluator will review the proposal. NASA disagrees that the evaluation ratings Kolaka's proposal received from the two evaluators were widely differing, and we cannot say that this position is unreasonable. In any event, the Evaluation Guidelines, which Kolaka admits were not readily available to offerors (see Comments Para. 2.3.1), are merely internal guidance to the agency's evaluators and do not themselves establish legal rights and responsibilities so as to render inconsistent agency action illegal. Pike Creek Computer Co., Inc., B-290329, June 21, 2002, 2002 CPD Para. 106 at 3 n.2. In a similar vein, Kolaka complains that the protest record reflects a number of different evaluation "processes," such as, for example, the process described in the "Dryden ISO 9000 flowchart" and the Evaluation Guidelines, which, as noted, do not provide a basis to object to the agency's evaluation. Moreover, it is the evaluation scheme in the solicitation, not internal agency documents, to which an agency is required to adhere in evaluating proposals and in making the source selection decision.
6. We disagree with Kolaka's suggestion that evaluator A's comment regarding Kolaka's failure to provide a work breakdown structure indicated that the agency had used an unstated evaluation factor. A "work breakdown structure" is merely a means of dividing work to be performed into logical segments to track performance. See "The Government Contracts Reference Book" (2nd ed. 1998) at 546. The solicitation's request for detailed work plan information encompassed this information.
7. While Kolaka's now suggests that these evaluators may have acted arbitrarily or in a biased manner in their evaluation, the fact is they both recommended Kolaka's proposal for award.
8. The manager for the subtopic under which Kolaka proposed was one of the evaluators that assessed Kolaka's proposal. See Statement of Evaluator B, Feb. 26, 2003, at 1.
9. For example, Kolaka also complains that the record does not establish that NASA fairly used the commercial merit ratings in ranking proposals for award. The record shows, however, that the proposals that were selected for award all had commercial merit ratings equal to or higher than Kolaka's and all of these proposals had significantly higher technical merit scores.