Synovate, Inc.

Decision

Matter of: Synovate, Inc.

File: B-404689

Date: March 29, 2011

DIGEST

Protest that agency should have awarded higher adjectival ratings to protester's quote provides no basis for sustaining the protest, where the evaluation was reasonable and consistent with the evaluation criteria.

Synovate, Inc., of Falls Church, Virginia, protests the award of a contract to Altarum Institute, of Ann Arbor, Michigan, under request for quotations (RFQ) No. 507703, issued by the Department of the Army, Army Medical Command, Center for Health Care Contracting, for Army patient level satisfaction surveys. Synovate contends that the agency unreasonably evaluated its quote and applied unstated evaluation criteria in rating its quote as unsatisfactory under the technical capability evaluation factor. Synovate also argues that the agency's improper evaluation resulted in a flawed best value decision. <a name=Paragraph1End></a>

We deny the protest.

BACKGROUND

The Army issued the RFQ on September 2, 2010 via the GSA e-Buy website,[1] seeking a contractor to provide personnel, services, materials, equipment, supplies, and facilities necessary to perform Army patient level satisfaction surveys. Army patient surveys use statistical sampling to ensure statistical representation and validation of all beneficiaries' responses throughout the year, provide baseline and comparison group measures, and provide comparable benchmark measures for civilian medical services. Contracting Officer's Statement at 1. The survey respondents whose healthcare experiences are captured under this contract include soldiers, dependents, and retirees who receive outpatient care at military treatment facilities or clinics, soldiers assigned to Warrior in Transition units, healthcare maintenance organization (HMO) beneficiaries, healthcare providers, and soldiers in the medical evaluation board process. Id. The survey results provide data on beneficiaries' perceptions of outpatient and inpatient experiences. Id.

The RFQ provided that award would be made to the vendor whose quote was found to represent the best value to the government, based on the evaluation of the following three factors: technical, past and present performance, and price. RFQ at 6. The technical factor was significantly more important than past and present performance, and those two factors, when combined, were more important than price. Id. The technical factor had four equally weighted subfactors: quality control plan; technical performance plan; management capabilities; and information assurance approval, implementation, and sustainment. Id.

As relevant here, under the quality control plan subfactor, the RFQ stated that

The Quoter's quality control program shall be evaluated for its description of methodology for identifying, resolving, and preventing quality assurance problems. The Quoter's quality control program will be evaluated for its description of methodologies that demonstrate monitoring of overall contract performance with deficiencies identified before the level of services becomes unacceptable.

Id. Under the technical performance plan subfactor, the RFQ provided that

The technical performance plan will be evaluated to measure the Quoter's understanding of the [performance work statement] requirements/tasks/deliverables and to ensure that the approach meets the capability standards outlined in the [performance work statement].

Id.[2] Under the management capabilities subfactor, the RFQ stated that

The management capabilities will be evaluated to determine the Quoter's understanding of the organization and logistics required to meet the tasks and deliverables required in the [performance work statement]. The qualifications of key personnel and experience, as well as the Quoter's description of their companies overall management capabilities will be evaluated.

Id.

The RFQ provided that the agency would evaluate the technical factor and subfactors using an adjectival rating system of excellent, good, satisfactory, marginal, and unsatisfactory.[3] Id. at 6-7. As is relevant here, satisfactory was defined as:

The quote demonstrates an adequate understanding of the requirements and the approach meets all performance or capability standards. The quote has no strengths that exceed the requirement. The quote has no material weaknesses and no deficiencies. The quote has some weakness that can potentially cause disruption of schedules, increase in cost, or degradation of performance. Special contractor emphasis and close government monitoring will minimize any risk. The quote meets the minimum requirements; therefore, has a Satisfactory probability of successfully meeting the requirements.

Id. at 6-7. Unsatisfactory was defined as:

The quote fails to demonstrate a basic understanding of the requirements and the approach fails to meet performance or capability standards. The quote has major omissions, weaknesses or deficiencies, and inadequate details to assure the evaluators that the Quoter has an understanding of the requirements. The quote presents an Unsatisfactory risk and cannot meet the requirements without major revisions.

Id. at 7. The RFQ provided that in order to receive consideration for award, a rating of no less than satisfactory had to be achieved for each technical factor and subfactor. Id. at 6. The RFQ also stated that failure to meet a requirement could result in a quote being ineligible for task order award. Id. at 1.

The agency received three quotes by the October 6 closing date and convened a technical evaluation panel (TEP) to evaluate and rate the technical quotes. Contracting Officer's Statement at 3. As is relevant here, the final evaluation ratings for Synovate's and Altarum's quotes were as follows:

AR, Tab 18, Source Selection Decision, at 2-3.

As noted above, Synovate's quote received a rating of unsatisfactory for the quality control plan subfactor, with the TEP assigning four major omissions and two weaknesses or deficiencies, and the TEP noting that the quote contained inadequate details to assure the evaluators that Synovate understood the requirements. AR, Tab 14, TEP Report, at 3. Specifically, the TEP found the following flaws in Synovate's quote: (1) the quote failed to indicate how and how often Synovate would specifically handle the quality control of the mail survey--for example, it did not explain how Synovate would ensure that the correct questionnaire was placed in the correct envelope (major omission); (2) the quote failed to indicate how Synovate would revalidate incorrect mailing addresses (deficiency); (3) the quote did not indicate how Synovate would manage quality control with the Warriors in Transition unit surveys (major omission); (4) the quote failed to explain how telephone interviewers would be trained (omission); (5) the quote failed to indicate how printing processes would be controlled by intelligent print and collation equipment (deficiency); and (6) the quote failed to indicate the quality control process for the survey website and reports (major omission). Id.

Synovate's quote also received a rating of unsatisfactory for the technical performance plan subfactor, with the TEP assigning one weakness, six deficiencies, and noting that the quote contained inadequate details to assure the evaluators that Synovate understood the requirements. Id. at 3-4. Specifically, the TEP found the following flaws in Synovate's quote: (1) the quote indicated that the requirement for a minimum number of patient surveys cannot be performed and that the "goal is impossible to achieve" (major deficiency); (2) the quote indicated that a factor contributing to Synovate's inability to meet the requirement to obtain the minimum number of patient surveys was that "under the current process, the [medical treatment facilities] choose which of their providers are to be included in the [Army patient level satisfaction survey] program" (invalid assumption and major deficiency); (3) the quote indicated that Synovate cannot meet the requirement for a minimum number of patient surveys due to a lag time between when a provider begins in a military treatment facility and when the contractor is notified or receives approval to include the provider in the quota (invalid assumption and major deficiency); (4) the quote indicated that Synovate cannot meet the requirement for a minimum number of patient surveys due to the differences in the number of patients seen between specialties and locations (major deficiency); (5) the quote indicated that Synovate cannot meet the requirement for a minimum number of patient surveys because patients can not be surveyed for the same provider within a 12'month period (invalid assumption and major deficiency); (6) the quote used incorrect terminology for the Warrior in Transition unit population (weakness); and (7) the quote misunderstood the medical evaluation board survey (major deficiency). Id. at 3-4.

Synovate's quote also received a rating of unsatisfactory for the management capabilities subfactor, with the TEP finding three major omissions and noting that the quote contained inadequate details to assure the evaluators that Synovate understood the requirements. Id. at 4. Specifically, the TEP found the following flaws in Synovate's quote: (1) the quote failed to articulate a management plan (major omission); (2) the quote failed to list anyone with information assurance capabilities (major omission); and (3) the quote failed to describe overall management capabilities and how they would relate to the various tasks (major omission). Id. at 4.

In making the award decision, the contracting officer determined that because the quotes of Synovate and another vendor had received technical ratings of unsatisfactory, the firms were ineligible for award. The contracting officer also noted that Altarum's price of $19,915,232.84 was the lowest of the three quotes received. AR, Tab 18, Source Selection Decision, at 4. Synovate's price of $22,369,578.54 was the highest of the three quotes, and was approximately $2.4 million higher than Altarum's quote. Id. Based on the fact that Altarum's quote received the highest ratings, had the lowest price, and was the only quote eligible for award, the agency determined that Altarum's quote represented the best value to the government. Id. After receiving notice of the award, Synovate protested.

DISCUSSION

Synovate challenges the deficiencies and omissions that the agency assigned to its quote under the quality control plan, technical performance plan, and management capabilities subfactors. The protester contends that it should have received a rating of at least satisfactory for each subfactor and for the technical factor overall. Synovate contends that the agency applied unstated evaluation criteria and arrived at unreasonable conclusions regarding Synovate's quote. Further, the protester argues that the agency's improper evaluation of Synovate's quote resulted in a flawed best value decision.

In reviewing protests challenging an agency's evaluation of quotes, we will not conduct a new evaluation or substitute our judgment for that of the agency; rather we will examine the record to determine whether the agency's judgment was reasonable and consistent with the solicitation's evaluation criteria, and with procurement statutes and regulations. George T. Brown Assoc., B-404398, Jan. 26, 2011, 2011 CPD para. 28 at 3; All Points Int'l Distrib., Inc., B-402993, B'402993.2, Sept. 3, 2010, 2010 CPD para. 209 at 2-3. A vendor has the burden of submitting an adequately written quote, and it runs the risk that its quote will be evaluated unfavorably when it fails to do so. George T. Brown Assoc., supra. A protester's mere disagreement with the agency's evaluation does not establish that the evaluation was unreasonable. Id.

As a part of our consideration of each of Synovate's challenges, we have reviewed the pleadings, the evaluation materials, and the protester's quote. While we do not discuss in detail each of the protester's numerous challenges, we have considered them all and find them to be without merit. We address a few of these arguments below.

For example, under the technical performance plan subfactor, Synovate challenges the agency's assignment of a major deficiency based on its conclusion that Synovate's quote "indicates that the requirement cannot be performed and that the 'goal is impossible to achieve.'" See AR, Tab 14, TEP Report, at 3. The RFQ stated that contractors "shall ensure that there are 150 survey responses for approximately 25% [of the service providers] and 100 survey responses for approximately 75% of 5600 [service] providers within a 12-month period." RFQ at 55. In addressing this requirement, Synovate's quote stated that "the target of 150 or even 100 [completed surveys] will not be achieved for all 5,600 providers. There are factors that make this goal impossible to achieve." AR, Tab 11, Synovate's Quote, at 4. On the following page of Synovate's quote, it reiterated that it was "impossible to 'ensure' the number of completes for 5,600 providers," and "it will not be possible to achieve [the goal] for all providers." Id. at 5.

In evaluating this aspect of Synovate's quote, the agency concluded that the protester's statements were a major deficiency. AR, Tab 14, TEP Report, at 3. Synovate maintains that this aspect of its quote should have been evaluated as meeting the RFQ's minimum requirements, and that it was only presenting a frank discussion of performance risk. Synovate's Comments at 9. The protester also argues that its quote did not state that it was impossible to meet the requirements of the RFQ with regard to every provider, but rather that it would not meet the requirements of the RFQ with regard to some providers.[5] Id. Further, Synovate contends that this particular RFQ requirement would be impossible to achieve for all vendors, and that any vendor who represented that there was no risk in meeting the Agency's goals was "committing a fraud upon the Agency." [6] Id.

We agree with the agency that Synovate's statements in its quote took exception to the requirement of the RFQ to obtain a minimum number of survey responses. Although the protester attempts to explain its statements as a discussion of risk, the fact remains that Synovate's quote stated clearly that it would not meet the solicitation's requirements with regard to a portion of the service providers. AR, Tab 11, Synovate's Quote, at 4-5. Based on this record, we find the agency's assessment of a major deficiency, and a rating of unsatisfactory to Synovate's quote for the technical performance plan subfactor, to be reasonable.[7]

The protester also challenges the agency's assessment of an unsatisfactory rating to Synovate's quote under the management capabilities subfactor for, among other reasons, the firm's failure to articulate a management plan. See AR, Tab 14, TEP Report, at 4. For the management capabilities subfactor, the RFQ instructed vendors to "[d]escribe a detailed plan that demonstrates your understanding of the organization, logistics and management required by this solicitation." RFQ at 3. Synovate contends that, although it did not submit a separate formal management plan, its quote satisfied the solicitation's requirements. The protester contends that its quote satisfied the RFQ's requirements because the quote included information about key personnel and their experience, described the firm's accreditation plan for the army patient level satisfaction survey and the Warrior in Transition survey programs, and explained Synovate's plan for managing the outpatient satisfaction survey. Protest at 30; Synovate's Comments at 16. The quote also included a graphic flow chart with pictures of buildings and a computer, which Synovate claims demonstrated the Army medical department satisfaction measurement system program, and a description of daily, weekly, and monthly processes undertaken to meet the RFQ's requirements. Synovate's Comments at 16; see AR, Tab 11, Synovate's Quote, at 16. The protester's quote also stated that its survey operations management plan would be provided at a later date as a contract deliverable. See AR, Tab 11, Synovate's Quote, at 18 ("Synovate will prepare a . . . Management Plan").

Based on our review of the record, we find the agency's determination that the protester failed to comply with the solicitation's requirement to describe a detailed management plan to be reasonable. The pieces of information set forth above do not meet the solicitation's requirement for vendors to provide a detailed management plan. The protester's promise to fulfill the requirement later as a contract deliverable is similarly insufficient. Therefore, we find the agency's determination that this represented a major quote omission, which contributed to an unsatisfactory rating under the management plan subfactor, to be reasonable.

Finally, the protester challenges the agency's evaluation of its quote as unacceptable under the quality control plan subfactor. The protester contends that the agency imposed unannounced evaluation criteria with regard to Synovate's failure to sufficiently describe its quality control procedures for the mail survey, the Warrior in Transition survey, and the training of phone interviewers. Protest at 13-16; Synovate's Comments at 1-8. The protester further argues that its quote adequately addressed these issues and met the requirements of the RFQ. Synovate's Comments at 6-8.

Where an RFQ sets forth factors and subfactors for evaluation, the evaluation of quotes must be based on these criteria. In this regard, an agency may not consider unstated evaluation criteria that are not reasonably related to the stated evaluation factors or subfactors. USGC Inc., B 400184.2 et al., Dec. 24, 2008, 2009 CPD para. 9 at 6. As set forth above with regard to quality control, the solicitation here instructed vendors to, among other things, "[d]escribe your company's quality control program and procedures for monitoring performance" and "[e]xplain your methodology for identifying, resolving, and preventing quality assurance problems." RFQ at 2-3. The issues evaluated by the agency were reasonably encompassed within these evaluation criteria. That is, Synovate's failure to adequately explain its approach for handling the quality control of the mail surveys, quality control with the Warrior in Transition survey, and training telephone interviewers all relate to requirements to ensure the quality control of the Synovate's performance and are well within the requirements to explain the firm's quality control plan, procedures, and methodology.

We further find that the agency's conclusion that the protester's quote did not adequately address these and other aspects of the quality control plan to be reasonable. For example, the protester's quote states that interviewing stations will be staffed with interviewers "who receive in-depth training," but it provides no explanation of what that training entails. See AR, Tab 11, Synovate's Quote, at 30. Similarly, the protester's description of its website and the quote's statement that "[c]omputed results are doubled processed . . . verifying inclusion of all survey records" provides little insight as to how Synovate will handle quality control of the various surveys. Id. at 20, 27. Accordingly, we find the agency's assessment of an unacceptable rating for the quality control plan subfactor to be reasonable.

In sum, based on our review of the entire record and considering all of the arguments raised by the protester challenging its rating under the technical factor, including the examples discussed above, we find no basis to question the agency's evaluation of Synovate's quote. Synovate's complaints constitute mere disagreement with the agency's evaluation and, as such, do not provide a basis for sustaining the protest. Because we find that the agency properly evaluated Synovate's quote as unsatisfactory under the technical factors and subfactors, we need not address the protester's challenge to the best value determination. The protest is denied.

Lynn H. Gibson
General Counsel