Focus Diagnostics, Inc.

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of:      Focus Diagnostics, Inc.

File:                B-409614.3; B-409614.4

Date:              November 24, 2014

DIGEST

1.  Protest that discussions regarding protester’s biomedical diagnostic system were not meaningful is denied where record reflects that agency provided technical descriptions of all significant weaknesses and deficiencies evaluated against protester’s system.

2.  Protest that agency improperly failed to consider effect of proposal revisions on protester’s initial performance-based evaluation ratings is denied where record reflects that evaluation was consistent with terms of solicitation and that agency considered proposal revisions under stand-alone evaluation criterion established for that purpose.

3.  Protest regarding training of agency personnel responsible for testing protester’s biomedical diagnostic system is denied where record shows agency actions to be reasonable and consistent with solicitation.

4.  Protest that agency unreasonably considered user errors experienced by agency personnel testing protester’s biomedical diagnostic system in simulated military healthcare deployment is denied where record reflects that testing methodology and findings reasonably relate to solicitation’s military utility assessment subfactor.

Focus Diagnostics, Inc., of Cypress, California protests the downselection of BioFire Defense, LLC, of Salt Lake City, Utah, by the Department of the Army, Army Contracting Command, under request for proposals (RFP) No. W911QY-12-R-0021 for development and production of a portable system for analyzing human clinical samples to detect chemical, biological, radiological, and nuclear threat conditions.  Focus alleges that the agency’s evaluation and discussions were flawed.

We deny the protest.

BACKGROUND

The solicitation, issued on July 20, 2012, contemplated the award of three indefinite-delivery, indefinite-quantity (ID/IQ) contracts with fixed-price and cost-type contract line item numbers for development of the Next Generation Diagnostics System Increment 1 Deployable Component system.  RFP §§ A.1, A.2.  The system was described as a “lightweight, low cost, man-portable capability to diagnose [chemical, biological, radiological, and nuclear] threats in deployed military health care environments.”  Id. § C.1.2.1.  The requirements included delivery of Food and Drug Administration (FDA)-approved, commercially available off-the-shelf (COTS) diagnostic systems, as well as “COTS assays, newly developed FDA-cleared [in vitro diagnostic] and environmental assays, non-assay consumables, support equipment, training and training materials, system documentation, on-site technical support and logistic support.”  Id. § C.1.1.

The solicitation provided that upon award of the ID/IQ contracts, the agency would issue delivery orders for a nine-month “competitive prototyping” phase.  See RFP § A.2.  The solicitation further provided that after completion of the competitive prototyping phase, the agency would select one of the three contractors to perform future delivery orders for a two-year “technology development” phase and a seven-year “production and deployment” phase; this second selection decision is referred to as a “downselection.”  Id.; Contracting Officer’s Statement at 1.

The solicitation included evaluation criteria for award of the initial ID/IQ contracts, as well as separate evaluation criteria for the downselection.  RFP §§ M.1-M.4.  The solicitation expressly provided that the downselection would be based on the contractors’ performance during the competitive prototyping phase.  Id. § M.4.1.

After receiving and evaluating proposals and conducting discussions, the agency awarded ID/IQ contracts to three offerors, including Focus and BioFire, on January 9, 2013.  Contracting Officer’s Statement at 1.  The ID/IQ contracts incorporated the solicitation’s downselection criteria.  Id. at 6.  The downselection criteria, and their relative importance, are shown in the table below.

FACTOR/SUBFACTOR		FACTOR/SUBFACTOR RELATIVE IMPORTANCE
Factor 1	Operational Assessment
Subfactor 1.1	U.S. Army Medical Department Board Military Utility Assessment		SF1.1 > SF 1.2	F1 = F2
Subfactor 1.2	Government Adapted Training Course
Factor 2	Developmental Testing Performance
Subfactor 2.1	Assay Development		SF 2.1 = SF 2.2 = SF 2.3	F2 > F3
Subfactor 2.2	Operational Environment Suitability
Subfactor 2.3	Intended Use Suitability
Factor 3	Development, Manufacturing, and Regulatory Risk Assessment
Subfactor 3.1	Development and Manufacturing Risk Assessment		SF 3.1 = SF 3.2	F3 = F4
Subfactor 3.2	Regulatory Risk Assessment
Factor 4	Cost and Pricing and Program Affordability
Subfactor 4.1	First Technology Development Phase Delivery Order Cost and Pricing		SF 4.1 = SF 4.2 SF 4.2 > SF 4.3	F4 = F5
Subfactor 4.2	Program Lifecycle Cost Estimate
Subfactor 4.3	Contract Cost and Pricing
Factor 5	Contract Performance/Contract Management
Subfactor 5.1	Contract Performance		SF 5.1 = 5.2	F4 > F6
Subfactor 5.2	Contract Management
Factor 6	First Technology Development Phase Delivery Order Technical Approach			F6 = F7
Factor 7	Performance Specification
Subfactor 7.1	Next Generation Diagnostics System Performance Specification		SF 7.1 >> SF 7.2 (>> means much greater than)
Subfactor 7.2	Tactical Variant Performance Specification

RFP §§ M.4.2.1 - M.4.2.7.  In addition to the criteria shown above, the solicitation also stated that the agency “may” consider “finalized program requirements, yet to be established by the Joint Requirements Office for Chemical Biological Radiological and Nuclear Defense.”  Id. § M.4.1.  The solicitation did not further address the “finalized program requirements” or the evaluation thereof.

On February 28, the agency issued delivery orders to the three ID/IQ contract holders for performance of the competitive prototyping phase.  Contracting Officer’s Statement at 1.  In the ensuing months, the agency evaluated the contractors’ deliverables under the solicitation’s downselection evaluation criteria.  See id. at 2.  On August 29, the agency requested revised proposals from the contractors.  Agency Report (AR), Tab 18, Agency Ltr. to Focus (Aug. 29, 2013) at 1.  The revised proposals were to address changes to the statement of work, provide any cost/price revisions, and include proposals for an initial technology development phase delivery order (i.e., the delivery order for the first phase that the downselected contractor would perform).  Id.

During October and November, a downselection evaluation board (DEB) convened and evaluated the revised proposals.  Contracting Officer’s Statement at 3.  Thereafter, the agency opened discussions with the contractors.  Id. at 4-5.  During this time, the third contractor withdrew from consideration for the downselection.  Id. at 5.  On February 10, 2014, the agency notified Focus that it had selected BioFire.  AR, Tab 24, Focus Downselection Notification Ltr., at 1.  On March 18, following a debriefing, Focus filed a protest with our Office.

Focus’s protest alleged that the agency’s downselection reflected a number of flaws, including inadequate discussions and unreasonable evaluation findings.  On April 14, the agency notified our Office of its intent to take corrective action.  Our Office later dismissed Focus’s protest as academic.  Focus Diagnostics, Inc., B‑409614, Apr. 23, 2014, at 1-2.

On May 9, the agency reopened discussions by providing Focus and BioFire with lists of evaluated significant weaknesses and deficiencies, and affording the firms an opportunity to submit further proposal revisions.[1]  See, e.g., AR, Tab 50, Focus Contract Mod. No. P00014.  On May 23, Focus submitted its revised proposal.  Contracting Officer’s Statement at 10.  On June 3, the agency informed Focus that its response to the evaluated significant weaknesses and deficiencies was unsatisfactory because it did “not detail any definitive changed technical approaches associated with [the] system design,” but instead promised future “investigations, reviews, tests, [and] assessments” to address the significant weaknesses and deficiencies.  AR, Tab 53, Agency Ltr. to Focus (June 3, 2014), at 2.  The agency afforded Focus another opportunity to submit a response to the issues.  Id. at 3.  On June 9, Focus did so.  AR, Tab 57, Focus Final Proposed Technical Revisions.

Also on June 9, Focus filed another protest with our Office.  In this protest, Focus alleged that the second round of discussions was not meaningful because the agency had not provided sufficient information for Focus to meaningfully address the evaluated significant weaknesses and deficiencies.  Because the protest concerned the sufficiency of the agency’s discussions with Focus, rather than a change to the ground rules of the competition, and because the agency had not yet made a new source selection decision, we dismissed Focus’s protest as premature.  Focus Diagnostics, Inc., B‑409614.2, June 20, 2014, at 2.

On July 31, after evaluating the final round of revised proposals, the DEB completed its downselection evaluation report.  AR, Tab 74, DEB Report.  The report included final ratings and evaluated cost/prices for Focus and BioFire under the solicitation’s downselection evaluation criteria, as shown in the table below.

	FOCUS		BIOFIRE
	Technical Rating	Risk Rating	Technical Rating	Risk Rating
Factor 1	Marginal	Moderate	Acceptable	Low
Subfactor 1.1	Marginal	Moderate	Marginal	Low
Subfactor 1.2	Acceptable	Low	Good	Low
Factor 2	Poor	High	Marginal	Moderate
Subfactor 2.1	Poor	High	Marginal	High
Subfactor 2.2	Marginal	Moderate	Marginal	Moderate
Subfactor 2.3	Poor	High	Marginal	Moderate
Factor 3	Good	Low	Acceptable	Low
Subfactor 3.1	Acceptable	Low	Marginal	Low
Subfactor 3.2	Excellent	Low	Excellent	Low
Factor 4
Subfactor 4.1	$[DELETED]	n/a	$[DELETED]	n/a
Subfactor 4.2	n/a	n/a	n/a	n/a
Subfactor 4.3	$217,446,723	n/a	$227,268,482	n/a
Factor 5	Good	Low	Acceptable	Low
Subfactor 5.1	Acceptable	Low	Marginal	Low
Subfactor 5.2	Excellent	Low	Good	Low
Factor 6	Acceptable	Moderate	Good	Moderate
Factor 7	Marginal	Moderate	Marginal	Moderate
Subfactor 7.1	Marginal	Moderate	Marginal	Moderate
Subfactor 7.2	Marginal	Moderate	Marginal	Moderate

AR, Tab 74, DEB Report, at 18, 60.  In addition, the report included the following ratings and evaluated cost/pricing related to evaluation of the revised proposals.

	FOCUS		BIOFIRE
	Technical Rating	Risk Rating	Technical Rating	Risk Rating
Final Proposed System Design	Acceptable	High Risk	Acceptable	Moderate Risk
Supplemental Agreement Implementation Schedule	16 Weeks		6-8 Weeks
Supplemental Agreement Cost & Pricing	$[DELETED]		$[DELETED]

Id. at 16.

On August 8, the downselect authority executed a downselect decision document that included a lengthy tradeoff analysis.  AR, Tab 47, Best Value Tradeoff Analysis, at 14-32.  In this document, the downselect authority concluded that BioFire’s superior technical ratings and lower risk assessments under factors 1 and 2 (the highest-weighted factors), as well as factor 6, outweighed the benefits associated with Focus’s higher ratings under factors 3 and 5 and Focus’s lower evaluated cost/price.  Id. at 28-32.  From this, the downselect authority decided that BioFire represented the best value to the government.  Id.

After receiving notice of BioFire’s selection, Focus again filed a protest with our Office.

DISCUSSION

Focus alleges that the agency’s selection of BioFire was flawed in numerous respects.  We have considered all of Focus’s arguments, and we conclude, based on the record, that none furnishes a basis on which to sustain the protest.  Below we discuss Focus’s principal contentions.

Allegations Regarding Discussions

Focus asserts that the agency’s discussions with the firm were not meaningful, arguing--as it did in its second protest to our Office--that the agency did not provide sufficient information for Focus to meaningfully address the evaluated significant weaknesses and deficiencies in its revised proposal.  Protest at 18-20; Comments at 9-12.

As discussed above, as part of the corrective action following Focus’s initial protest, the agency provided Focus a list of evaluated significant weaknesses and deficiencies.   AR, Tab 50, Focus Contract Mod. No. P00014, at 2-3.  Each item in the list related to test failures that occurred during the agency’s evaluation of Focus’s system.  Id.  Focus asserts that due to the scientific nature of the system and the testing, the “root cause” of the errors had to be identified for Focus to be able to meaningfully address the issues.  Protest at 19‑20; Comments at 10-12.  Focus further asserts that to identify the root cause, it would need to interview the personnel who performed the testing or examine information such as “run data” generated during the agency’s testing.  Protest at 19-20; Comments at 10.

The agency responds that the discussions were meaningful because Focus was provided with details of each evaluated significant weakness and deficiency.  Memorandum of Law at 18-19, 22.  The agency rejects Focus’s call for interviews of the evaluators.  See AR, Tab 53, Agency Ltr. to Focus (June 3, 2014), at 2.  With regard to Focus’s call for run data, the agency states that such data is raw, electronic data associated with the inner, proprietary workings of an offeror’s system and that obtaining such data may be impossible due to the “high likelihood that [it was] consumed during testing.”  AR, Tab 190, DEB Chairman Decl., at 3.  To the extent it was not consumed, the agency states, the collection of such data would be highly impracticable due to the difficulty of compiling and loading data from systems that may have been compromised during destructive tests or biological agent decontamination procedures.  Id.  Finally, the agency explains that run data was not considered in the agency’s evaluation process; rather, the agency considered the testing results themselves.  See id.

When discussions are conducted, they must identify deficiencies and significant weaknesses in an offeror’s proposal.  Federal Acquisition Regulation § 15.306(d)(3); Metro Mach. Corp., B-295744, B‑295744.2, Apr. 21, 2005, 2005 CPD ¶ 112 at 19.  To be “meaningful,” discussions must be sufficiently detailed to lead an offeror to the areas of its proposal requiring amplification or revision.  Metro Mach. Corp., supra; Am. States Utilities Servs., Inc., B-291307.3, June 30, 2004, CPD ¶ 150 at 6.  However, the content and extent of discussions is a matter of the contracting officer’s judgment based on the particular facts of the procurement. Alpha Genesis, Inc., B-299859, Sept. 12, 2007, 2007 CPD ¶ 167 at 8; Heritage Garden Ctr., Inc.; S.C. Jones Servs., Inc., B-248399.4, Oct. 28, 1992, 92-2 CPD ¶ 290 at 4.  In reviewing whether there has been sufficient disclosure of deficiencies, the focus is not on whether the agency described the deficiencies in such detail that there could be no doubt as to their identity and nature, but whether the information was sufficient in the context of the procurement to afford the offeror a fair and reasonable opportunity to identify and correct deficiencies in its proposal.   Alpha Genesis, Inc., supra; Metro Mach. Corp., supra, at 20.

The list of significant weaknesses and deficiencies that the agency provided to Focus is shown below.

During Government testing, the Contractor’s system completed two out of seven susceptibility tests per Mil-Std 461 before becoming nonfunctional on the third test (CS114). . . .

The system had . . . numerous quality control failures and errors reading bar codes on discs.  [These] errors/problems were due to problems with the assay test consumables . . . .  [T]hese problems required re-running the specimens using new assay test consumables and loss of time in obtaining acceptable test results therefore reducing the reliability of the system.

A significant number of test runs resulted in multiple assay failures, most due to the lack of internal control amplification.  This extremely high failure rate indicates the assay is not performing optimally. ­ The Marburg assays exhibited some level of cross-reactivity with Ebola Reston at high concentrations. . . .

[Biological warfare agent] assays require appropriate agent identification for inclusivity/exclusivity, however based upon Government test results, the Contractor delivered [viral hemorrhagic fever] and Anthrax assays did not meet cross-reactivity and inclusivity/exclusivity requirements and exhibited inappropriate detections of non-target strains or organisms.  Test results demonstrated cross-reactivity with non-target strains or organisms, significantly decreasing assay performance and the intended clinical and environmental use.

The Government testing revealed numerous occurrences of contaminated instrumentation. . . .  The Contractor’s integrated cycler had a few high level contamination situations.  If the exterior of the sample disk becomes contaminated during sample loading this contamination can be easily disseminated due to the fact that the assay disc spins in the instrument.  Since the current Direct Amplification unit design accepts raw patient samples by add[ing] it with a pipette into a tiny well, with potentially still active pathogens, a droplet transfer or formation from ‘tip flicking’ is possible.  There is nothing to contain such a droplet of live pathogen on the surface of the disc from being spread as an aerosol into the lab by the strong airflow used during the cooling step of each cycle. . . .

The Contractor’s system was unable to meet inclusivity/exclusivity requirements for B. anthracis.  The assay will result in false negatives and/or false positives . . . .

Ebola Zaire was only detected using the pan-Ebola portion of the assay and was not properly speciated according to Government requirement.  The material failure to differentiate Ebola Zaire from other Ebola species reduces the utility to the Government due to the inability to align with the Ebola Zaire therapeutic . . . .

AR, Tab 50, Focus Contract Mod. No. P00014, 2-3.  These summaries show that the agency provided Focus with technical descriptions of the testing errors underlying each of the significant weaknesses and deficiencies.

As stated above, as part of its revised proposal, Focus submitted a response to each of the significant weaknesses and deficiencies.  AR, Tab 57, Focus Final Proposed Technical Revisions.  Focus’s response stated that it was based on an “incomplete root cause determination” and that many of the issues were “not operative and not consistent with [Focus’s] system design.”  Id. at 1, 4, 8, 10, 14-15, 18.  Nevertheless, for each issue, the response set forth a detailed, technical approach aimed at resolving the testing errors.  Id. at 3-19.

The agency evaluated one of these technical approaches as low risk, two as moderate risk, and five as high risk.  AR, Tab 77, Focus Finalized Program Requirements Evaluation Report, at 6-20.  The agency documented--in detail--the technical basis for each of these risk ratings.  Id.  The downselect authority later specifically considered this evaluation in his best value determination.  AR, Tab 47, Best Value Tradeoff Analysis, at 18, 23-25, 29-30.

Despite the mostly adverse ratings that the agency assigned to Focus’s responses to the discussions items, we view the discussions here to have been meaningful.  In this regard, there is no indication that the agency failed to disclose any evaluated significant weaknesses or deficiencies to Focus.  Further, the information that the agency provided included a technical description of the basis for each of the agency’s concerns (albeit without as much detail as Focus would have preferred).  Finally, Focus was able to use these descriptions to formulate technical approaches aimed at resolving the issues.  Focus has not shown, and it is not apparent to us, that the agency’s subsequent evaluation of these technical approaches was unreasonable. 

Our decision in Apptis, Inc., B-299457 et al., May 23, 2007, 2008 CPD ¶ 49, addressed discussions in the context of a procurement where, as here, an offeror’s revised proposal itself could not change the results of previously-evaluated testing.  In Apptis, the agency was required to point out the weaknesses it observed in the testing in order to provide the firm with an opportunity to address them by, for example, refuting the agency’s purported observations, providing explanations as to why the results occurred, or proposing methods to address the agency’s concerns.  Id. at 19.  That is precisely what occurred here.  In sum, we find that the agency’s discussions with Focus were meaningful.

Focus also contends that the discussions were flawed because the agency did not evaluate the firm’s revised proposal relative to its initial evaluation ratings; i.e., the agency did not consider the effect of Focus’s responses to the discussions items on Focus’s initial evaluation ratings or make any changes to those ratings based on the Focus’s responses.  Protest at 21-23; Comments at 12.  Instead, according to Focus, the agency only considered the responses under a new, unstated evaluation criterion known as “finalized program requirements.”  Protest at 23; Comments at 15‑17.

The agency responds by acknowledging that it did not evaluate Focus’s revised proposal relative to Focus’s initial ratings.  See Memorandum of Law at 36-38.  The agency asserts that its methodology of essentially freezing the contractors’ initial evaluation ratings, even after discussions took place, is reasonable in the context of this type of procurement; namely, a downselection whereby the agency first evaluates the performance of a pool of contractors as they develop prototypes of a complex biomedical device pursuant to government contracts, and then chooses one contractor, based on its performance, for continued development and production of that device.  See id.  The agency points out that it did evaluate and assign ratings to Focus’s and BioFire’s revised proposals under the heading of finalized program requirements, and that it considered those ratings in its downselection decision.  Id. at 40.

The agency asserts that this methodology was consistent with the solicitation, which expressly provided that the contractors’ performance during the competitive prototyping phase would be the basis of agency’s downselection evaluation, and that the agency “may” consider “finalized program requirements” in its evaluation.  Memorandum of Law at 38-39 (referencing RFP § M.4.1).  The agency argues that the methodology advanced by Focus, of adjusting contractors’ performance-based ratings based on proposal revisions, would amount to granting the contractors a “do‑over,” thereby skewing the results of the tests that the agency conducted in its initial evaluation.  Id. at 37.

Based on the terms of the solicitation and context of this procurement, we conclude that the agency’s decision not to evaluate the contractors’ responses to the discussions items relative to the initial evaluation ratings was reasonable.  In this regard, the solicitation was clear that the contractors’ performance in the competitive prototyping phase would be the basis of the agency’s downselection evaluation.  RFP § M.4.1.  Further, the record reflects that the agency did in fact evaluate Focus’s responses to the discussions items.  AR, Tab 77, Focus Finalized Program Requirements Evaluation Report, at 6-20.  Finally, the record reflects that for purposes of making the source selection decision, the downselection authority (and the DEB that assisted with the decision) specifically considered the results of that evaluation, albeit solely under the heading of finalized program requirements.  AR, Tab 47, Best Value Tradeoff Analysis, at 18, 23-25, 29-30; AR, Tab 74, DEB Report, at 3-4, 9, 19, 44-58.

It is true, as Focus points out, that the solicitation was not clear regarding the nature of and the relative weight to be accorded to the finalized program requirements in the agency’s evaluation.  However, to the extent that Focus believed this lack of clarity constituted a solicitation impropriety, it could have filed a protest prior to the solicitation’s closing date, but did not.  At this juncture, Focus’s allegations amount to an untimely challenge against an ambiguity apparent on the face of the solicitation.  4 C.F.R. § 21.2(a)(1) (2014); Am. Cybernetic Corp., B-310551.2, Feb. 1, 2008, 2008 CPD ¶ 40 at 2 n.1.

In any event, we fail to see how the agency’s actions prejudiced Focus.  As discussed above, the agency evaluated Focus’s responses to the discussions items adversely, and there is nothing to show that these evaluation findings were unreasonable.  Thus, we see no basis to conclude that Focus would have gained an advantage if the agency had imposed these evaluation results on the firm’s initial evaluation ratings.  Focus’s claims regarding the agency’s conduct of discussions are denied.

Allegations Regarding Training

Focus raises two allegations regarding training that was conducted during the contractors’ performance of the competitive prototyping phase.  To develop the record regarding these and other allegations, our Office conducted a hearing.  The following background information relevant to Focus’s claims was established through hearing testimony and hearing-related filings.

On April 16‑29, 2013, and in connection with the evaluation under subfactor 1.1, military utility assessment, the agency conducted an “early operational assessment” test event for the contractors’ systems.  Hearing Transcript (Tr.) at 183-85.  The test took place in a simulated operational environment (tents in a desert) and was performed by military personnel of the same skill level as those who would use the systems during actual military operations.  Id. at 184-85.  The event involved a simulation of a deployed military healthcare response to outbreaks of three types of illnesses.  Id. at 186-87.

On April 2-12 (i.e., before the test event), the agency conducted what was known as a “government adapted training course” to teach the personnel performing the testing how to use the contractors’ systems.  Agency Pre-Hearing Statement at 14.  The solicitation--and the contractors’ ID/IQ contracts--specified two deliverables that, according to the agency, relate to the agency’s development of this training course.[2]  Contracting Officer’s Statement at 31.  The first deliverable was a “train-the-trainer” session, during which contractor representatives were to provide training on their systems to agency personnel charged with developing and conducting the government adapted training course.  See Contracting Officer’s Statement at 31 (referencing RFP § C.3.1.5.1).  The second deliverable was instructor and student training documents.  See id. (referencing RFP § C.3.1.5.3).  Focus conducted its train-the-trainer session on March 14-15.  Agency Post-Hearing Comments at 36.  Focus submitted its training materials by the contractually-required due date of April 11.[3]

Separate from the operational assessment test event under subfactor 1.1, the agency also conducted laboratory testing.  This testing was conducted under subfactor 2.1, assay development, and was designed to assess the analytical performance of the contractors’ assay deliverables.[4]  Tr. 172; RFP § M.4.2.2.1.  The testing was performed in biosafety level 3 and 4 laboratories by agency technicians.  See Agency Post-Hearing Comments at 13-14, 18‑19.  The ID/IQ contracts included a training deliverable connected with this testing.  More specifically, the contractors were required to provide an “expert level training” session for agency personnel who were responsible for the testing.  See Agency Pre-Hearing Statement at 15-16; RFP § C.3.1.5.2.  Focus conducted its expert-level training session on June 20‑21.  Tr. at 288.  The Focus representative who conducted this training session was the same individual who had conducted Focus’s train-the-trainer session.  Id. at 367-70.  Focus’s allegations concern the expert level training session.

Focus first asserts that for the expert level training session, the agency improperly failed to provide training materials that Focus previously had prepared and submitted to the agency.  Protest at 24-25; Comments at 18-19.  The training materials in question are those that Focus submitted to the agency on April 11, i.e., the training documents that the agency asserts were for the agency’s use in developing the government adapted training course.  Focus argues that having these “comprehensive training materials” on hand during the expert level training would have “materially enhanced Focus’ training and would have enabled the trainees to better understand the Focus system and prepare to train and assist the laboratory operators before and during the [testing].”  Focus Post-Hearing Comments at 18.

In essence, Focus is alleging that the agency’s failure to provide Focus’s previously submitted training materials at the expert level training session compromised the agency’s testing of Focus’s system under subfactor 2.1.  As reflected in the table of evaluation ratings above, Focus’s system was rated poor/high risk under this subfactor.  AR, Tab 74, DEB Report, at 14.

In reviewing an agency’s evaluation in a downselection, it is not our role to perform a reevaluation; rather, we examine the record to determine whether the agency’s evaluation conclusions were reasonable and consistent with the terms of the solicitation (or the underlying contracts), as well as applicable procurement laws and regulations.  HDT Tactical Systems, Inc., B-403875, Dec. 14, 2010, 2011 CPD ¶ 8 at 3; Engineered Elec. Co. d/b/a DRS Fermont, B-295126.5, B-295126.6, Dec. 7, 2007, 2008 CPD ¶ 4 at 3-4.

It is undisputed that during Focus’s expert level training session, the agency did not provide the materials that Focus submitted on April 11.  The reason that they were not provided, the agency maintains, was that they never were intended to be used during the contractors’ expert level training sessions; rather, they were intended to be used internally in connection with the above-discussed government adapted training course.  See Contracting Officer’s Statement at 31.  The agency asserts that nothing in the solicitation--or the ID/IQ contracts--required it to provide the materials during the expert level training session, and that it did not provide the materials during any of the contractors’ expert level training sessions.  Agency Post‑Hearing Comments at 43.  The agency further asserts that prior to Focus’s expert level training session, the firm did not communicate any requests or expectations about the use of the materials during the session.  Id.  Finally, the agency asserts that the Focus representative who conducted the training did not expect the materials to be at the session, and that this issue apparently arose only because she learned--at the beginning of the expert level training session--that unlike her, the other contractors had brought PowerPoint slides and training manuals to their training sessions.  Id. at 43-44.

Testimony elicited at the hearing largely confirms the agency’s characterizations.  See e.g., tr. at 384-85.  For example, Focus’s training representative testified that it was not her practice to use PowerPoint slides or comprehensive training manuals in trainings, but, rather, to rely on a “hands-on” method of teaching and “quick guide” and “package insert” materials.  Id. at 375-79, 386-87.  Additionally, she testified that she did not feel she needed a PowerPoint presentation or comprehensive training materials in order to teach the attendees of the expert level training how to operate Focus’s system.  Id. at 386.  Her testimony also showed that the types of training materials that she typically used--quick guides and package inserts--were available at the expert level training session.[5]  Tr. at 382, 391.

In sum, the record reflects that the agency did not treat Focus any differently than the other contractors with regard to the expert level training, and we see nothing the solicitation that required the agency to provide specific materials at the expert level training session.  We conclude that in this procurement, it was up to the contractors
‑‑not the agency--to decide how to structure the training and what materials would be used.  Accordingly, in our view, the agency’s decision not to provide the training materials during Focus’s expert level training session furnishes no basis to question the ratings assigned to Focus’s system under subfactor 2.1.

Focus also alleges that its expert level training session was compromised because one of the agency trainees was disruptive and biased against Focus.  Protest at 25; Comments at 18-20.  In support of this allegation, Focus’s training representative testified that this individual’s questions, tone, and body language during the training “did not convey an interest in learning the Focus system, but instead, were confrontational and reflected a lack of interest in learning how to run assays on the Focus system.”  Focus Post-Hearing Comments at 20 (citing tr. at 416-17).  Focus asserts that this conduct resulted in an unreasonable evaluation of Focus’s system under subfactor 2.1 for two main reasons.  Id. at 19-23.  First, his conduct allegedly prevented the other trainees from “receiving the full benefit of Focus’ intended training.”  Id. at 19.  Second, this individual was responsible for training some of the laboratory operators who tested Focus’s system under subfactor 2.1.  Id.

It is clear from testimony at the hearing that the conduct of the individual in question did not halt the training or even prevent Focus’s trainer from conducting the session in a manner that she herself believed was successful.  See, e.g., tr. at 418-19.  For example, she testified that notwithstanding the individual’s questions, “everybody else in the room was still engaged, asking questions, learning and actually running the assays that we had there for them to run.”  Id.  Based on this statement, and the hearing testimony as a whole,[6] we are unpersuaded that the alleged disruptions reasonably can be tied to Focus’s evaluation ratings under subfactor 2.1.

In support of its allegation of bias, Focus points to testimony of the individual in question regarding his expressed concern that Focus did not bring a presentation or a training manual to the session.  Focus Post-Hearing Comments at 20 (referencing tr. at 456).  Focus also points to testimony regarding how the individual had expressed concern over a technical feature (foil peeling) of Focus’s system.  Focus Post-Hearing Comments at 20 (referencing tr. at 465).  As a final example, Focus points to the individual’s testimony that he had “never heard of” or used Focus’s instrument.  Id. (quoting tr. at 462).

Government officials are presumed to act in good faith, and a protester’s contention that procurement officials are motivated by bias or bad faith must be supported by convincing proof; our Office will not consider allegations based on mere inference, supposition, or unsupported speculation.  Career Innovations, LLC, B-404377.4, May 24, 2011, 2011 CPD ¶ 111 at 7-8; Shinwha Elecs., B-290603 et al., Sept. 3, 2002, 2002 CPD ¶ 154 at 5 n.6.  Based on the record here, we do not consider the conduct cited by Focus to meet the threshold showing for an allegation of bias.  Focus’s claims regarding the expert level training session are denied.

Allegations Regarding Agency Testing

Focus alleges that the ratings assigned to its system under subfactor 1.1, military utility assessment, and subfactor 2.1, assay development, reflect mishandling by the agency in the testing process.  Below we discuss Focus’s allegations in turn.

Under subfactor 1.1, Focus’s system was assigned a rating of marginal based in part on three evaluated weaknesses and one significant weakness.  AR, Tab 74, DEB Report, at 19-21.  The DEB summarized the significant weakness as follows:

The system had numerous errors/problems including numerous quality control failures . . . .  Since these errors/problems were due to problems with the assay test consumables, the analyzers did not experience downtime; however these problems required re-running the specimens using new assay test consumables and loss of time in obtaining acceptable test results therefore reducing the reliability of the system.

Id. at 21.  As part of its debriefing, Focus asked for additional information regarding the errors experienced by the agency.  AR, Tab 43, Focus E-Mail to Agency (Feb. 25, 2014), at 1.  The agency responded that the errors included “[l]oading of controls in the wrong position,” “[p]ositive control came out negative,” and “[n]egative control came out positive.”  AR, Tab 44, Agency Ltr. to Focus (Mar. 13, 2014), at 3.

Focus alleges that by their nature, these errors indicate mishandling by the agency personnel testing the system rather than issues with the system itself.  Protest at 26-27; Comments at 21.  On this basis, Focus asserts that the significant weakness assigned to its system under subfactor 1.1 was unreasonable.  Protest at 26-27; Comments at 21; Post-Hearing Comments at 5-10.

As stated above, subfactor 1.1 concerned the “military utility” of the system.  RFP § M.4.2.1.1.  As also stated above, the agency’s test involved military personnel with skill levels commensurate to the intended military users operating the systems in a simulation of a deployed military healthcare response to outbreaks of three types of illnesses.  At the hearing, the DEB chairman testified that the conditions and standards of the test were intended to simulate use of the systems under the stress of realistic, operational field conditions.  Tr. at 184-88. 

The agency points out that in addition to the significant weakness at issue here, Focus’s system also was assigned a weakness for its complexity and the number of steps required for its use.[7]  Agency Post-Hearing Comments at 11.  The agency essentially acknowledges that the errors at issue may have resulted from the manner that its personnel used the system, but argues that one purpose of the test was to determine how well military personnel could set up and use the system in an operational environment.  See id. at 7-8.  In this regard, one of the DEB board advisors testified that “the attributes of the system that . . . a user could inherently confuse or make mistakes in [when in] its intended use environment . . . is something that the United States military would want to understand and . . . evaluate.”  Tr. at 168-69.

The system being developed under this procurement is a portable device for diagnosing chemical, biological, radiological, and nuclear threats in the field, i.e., in a deployed military healthcare environment.  RFP § C.1.2.1.  Thus, the agency included the military utility assessment subfactor in the solicitation, and it structured the evaluation to include a test that simulated a realistic military deployment of the system.  An agency’s evaluation considerations properly may take into account specific, albeit not expressly identified, matters that are logically encompassed by, or related to, the stated evaluation criteria.  MINACT, Inc., B‑400951, Mar. 27, 2009, 2009 CPD ¶ 76 at 3; Indep. Constr., Inc., B-292052 , May 19, 2003, 2003 CPD ¶ 105 at 4.  We see a nexus between the military utility assessment subfactor and the degree of reasonable user error that may occur when the system is deployed in an operational environment, and we therefore find nothing improper in this aspect of the agency’s evaluation.

With regard to the alleged user errors that may have adversely affected Focus’s rating under this subfactor, we observe that it is evident from the record that the agency undertook reasonable efforts to train the personnel who performed the testing.  See, e.g., AR, Tab 12, Early Operational Assessment Training Readiness Statement; AR, Tab 14, Next Generation Diagnostic System Training Course Schedule; AR, Tab 15, Next Generation Diagnostic System Training Course Quiz; AR, Tab 16, Early Operational Assessment Test Plan, at 2-5 - 2-7, 2-11, 3-6 - 3-12.  We observe also that Focus offered a system that was found to be complex to operate.  See AR, Tab 6, Early Operational Test Report, at 2-21, 2-27; AR, Tab 74, DEB Report, at 21-22.  For all these reasons, we see no merit to Focus’s claim regarding the agency’s testing of its system under subfactor 1.1.

Finally, Focus alleges that agency mishandling of its system occurred under subfactor 2.1, assay development.  Under this subfactor, Focus received a rating of poor based in part on three evaluated weaknesses, two evaluated significant weaknesses, and two deficiencies.  AR, Tab 74, DEB Report, at 22-23.  Focus alleges that the nature of the significant weaknesses and deficiencies indicates that contamination occurred during the testing of Focus’s assays, and that this contamination was the result of the agency’s laboratory technicians mishandling Focus’s assays and/or system.  Protest at 26; Comments at 22-23; Post-Hearing Comments at 10‑15.

At the hearing and in its post-hearing comments, the agency presented testimony and information from the contemporaneous record that adequately refutes Focus’s claim.  See Agency Post-Hearing Comments at 13-31.  For example, the DEB’s senior scientist testified that in his view, reports that Focus submitted to the agency regarding pilot assay lots reflected that Focus’s quality control testing may have failed to detect contamination at low concentrations.  Tr. at 147-49.

As another example, the DEB’s senior scientist testified that contamination issues may have occurred because two of the three Focus assays that the agency tested [DELETED].  Tr. at 140-42.  The agency asserts that the contamination-related errors occurred only in the testing of these two assays, and not in the testing of Focus’s third assay, which, according to the agency, [DELETED].  Post-Hearing Comments at 27.  In other words, the agency posits that a technical feature of Focus’s assays, and not agency mishandling, led to the contamination issues.  See id. at 20‑21, 27-29.

As a final example, the agency asserts that the contamination-related errors occurred consistently across tests conducted by highly-trained agency technicians at four laboratories at geographically dispersed locations.  Post-Hearing Comments at 26-27.

Focus has advanced a number of arguments expressing its disagreement with the agency’s position.  See Focus Post-Hearing Comments at 10-14.  However, based on the record as discussed above, we are not prepared to conclude that the contamination-related errors reflect agency mishandling.  Accordingly, we will not disturb the agency’s evaluation findings for Focus’s system under subfactor 2.1. 

The protest is denied.[8]

Susan A. Poling
General Counsel