PavCon, LLC

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: PavCon, LLC

File: B-420640

Date: July 5, 2022

DIGEST

1. Protest that agency failed to conduct meaningful discussions as part of a commercial solutions opening procurement is denied where negotiated procurement procedures were not required and where no discussions in fact occurred.

2. Protest challenging the agency’s decision not to fund proposal under a commercial solutions opening procurement for research and development is denied where the evaluation was reasonable and consistent with the solicitation.

PavCon, LLC, a small business of Latrobe, Pennsylvania, protests the decision to not fund its proposal under commercial solutions opening (CSO) No. FA8684-20-S-C001, Call 4, issued by the Department of the Air Force, Air Force Life Cycle Management Center, Rapid Sustainment Office. The protester challenges the agency’s evaluation of its proposal, and raises other challenges to the agency’s handling of the procurement.

We deny the protest.

BACKGROUND

The use of CSOs as a form of general solicitation was first authorized as a pilot program under section 879 of the fiscal year (FY) 2017 National Defense Authorization Act (NDAA), Pub. L. No. 114-328, § 879, 130 Stat. 2000, 2312-2313 (2016), and permanently authorized under section 803 of FY 2022 NDAA. Pub. L. No. 117-81, § 803, 135 Stat. 1541, 1814-1816 (2021). Department of Defense (DOD) guidance on the use of CSOs is found in Class Deviation-Defense Commercial Solutions Opening Pilot Program, 2018-O0016, June 26, 2018, now superseded by Class Deviation-Defense Commercial Solutions Opening, 2022-O0007, Feb. 4, 2022.[1] Agency Report (AR), Tab 33, 2018 CSO Class Deviation at 1-3; Tab 34, 2022 CSO Class Deviation at 1-3.

CSOs may be used by DOD agencies as an alternative to other procurement methods set forth in the Federal Acquisition Regulation (FAR) to acquire “innovative commercial items, technologies, or services.” AR, Tab 33, 2018 CSO Class Deviation at 1. Under a CSO, the agency competitively selects proposals received in response to the general solicitation based on a review of proposals by scientific, technological, or other subject-matter expert peers. Id. at 2. Proposals are not evaluated against each other since they are not submitted in response to a common performance work statement or statement of work. Id. Rather, each proposal is evaluated against the criteria specified in the solicitation, with primary focus being given to the following factors: technical; importance to agency programs; and fund availability. Id. Finally, “[w]hen using a CSO in acquisitions for research and development, contracting officers shall use the procedures in [the] class deviation in conjunction with FAR part 35 [Research and Development Contracting].”[2] Id. at 1.

The Air Force seeks to increase mission readiness and capabilities “by scaling innovative solutions to advance and modernize [its] sustainment operations.” Contracting Officer’s Statement (COS) at 2. In support thereof, on July 17, 2020, the agency issued the CSO here (No. FA8684-20-S-C001) announcing its intent to seek solutions that deliver innovative sustainment and operational advances to the agency. AR, Tab 4, CSO at 1; COS at 2. The CSO identified a total of eight technology focus areas (TFA), including artificial intelligence and machine learning (AI/ML). The CSO also explained the agency’s intent to subsequently identify areas of interest (AoI) related to the various TFAs, and to publish “calls” soliciting solutions that addressed specified objectives. AR, Tab 4, CSO at 3; COS at 2.

On July 2, 2021, the Air Force issued CSO Call 4, seeking innovative solutions for condition-based maintenance plus (CBM+). AR, Tab 5, CSO Call 4 at 1-7; COS at 3. In general terms, the call sought innovative approaches that would successfully implement AI/ML to enhance predictive maintenance, and reduce unscheduled maintenance, so as “to enable maintainers, logisticians, equipment specialists, item managers, operators, and engineers to make proactive, knowledge-based decisions” regarding aircraft maintenance.[3] AR, Tab 5, CSO Call 4 at 2; see COS at 3. Call 4 also included a detailed list of AoI objectives that the proposed solutions were to meet. AR, Tab 5, CSO Call 4 at 1-3.

Call 4 contemplated a two-step procurement process. In step one, firms were to submit white papers addressing the AoIs set forth in the Call. Id. at 3. Firms with favorably evaluated white papers would then be issued a request for proposals (RFP). Id. Call 4 also stated that the agency anticipated making one contract award, but “reserve[d] the right to award zero, one, or more Contracts for all, some, or none of the solicited effort.” Id. PavCon was among the firms which submitted Call 4 white papers by the August 2 closing date, and to which the Air Force thereafter issued a Call 4 RFP on September 24. COS at 4.

The Call 4 RFP included a statement of objectives (consisting of “lines of effort” and tasks) and proposal evaluation criteria as follows: technical; importance to agency programs/commercial innovation; funds availability; and price.[4] AR, Tab 14, Call 4 RFP amend. 1 at 1-3; Tab 15, Call 4 RFP amend. 1, attach. 1, Proposal Instructions at 1-9. Also, “[p]roposed solutions shall be evaluated on the basis of the merit of the Offeror’s proposed innovative solution, not against other proposed solutions of other Offerors.” AR, Tab 15, Call 4 RFP amend. 1, attach. 1, Proposal Instructions at 2.

PavCon was among the firms which submitted Call 4 proposals by the December 22 closing date. On February 23, 2022, the agency completed its evaluation of PavCon’s proposal, identified a number of deficiencies in the proposal, and found it to be technically unacceptable. AR, Tab 28, PavCon Proposal Evaluation at 3. Specifically, the technical evaluators identified three areas in which PavCon’s approach was deemed unsatisfactory, or deficient, as follows: (1) PavCon proposed working with the Air Force to obtain priority lists of top aircraft “degraders” rather than generating such priority lists itself as required by the RFP’s objectives; (2) the proposal lacked sufficient detail regarding new work unit codes (WUC)[5] for aircraft platforms, how the offeror would sustain “previously fielded” (i.e., existing) WUCs, and the offeror’s approach to the randomized sampling of WUC data; and (3) the proposal provided insufficient detail on the necessary information that would be contained in the required machine learning model configuration report. Id. at 3-5. Ultimately, as the overall technical approach was deemed unacceptable, the evaluators concluded that PavCon’s proposal did not represent an acceptable basis of merit to warrant funding. Id. at 1.

After providing PavCon with notice of the agency’s nonaward decision on March 17, and proposal feedback on March 24, this protest followed.

DISCUSSION

PavCon raises two challenges regarding the agency’s evaluation of its CSO proposal. The protester first alleges the Air Force failed to engage in adequate and meaningful discussions. Second, PavCon contends that the agency unreasonably found the firm’s proposal to be unacceptable.[6] Had the agency held meaningful discussions and conducted a proper evaluation, PavCon argues, it would have been selected for award. We have considered all of the protester’s assertions and arguments and, while we do not address them all, find no basis on which to sustain the protest.

Alleged Lack of Meaningful Discussions

PavCon protests the agency failed to conduct adequate and meaningful discussions. Specifically, the protester alleges that, although the agency conducted discussions with the firm, those discussions did not include the three deficiencies which made PavCon’s proposal technically unacceptable. Had the agency properly raised such matters, PavCon argues, it would have had the chance to modify its proposal or explain why the agency’s perceived concerns did not render the proposal unacceptable. Protest at 18-22. As detailed below, because the agency was not required to conduct discussions, and did not in fact do so, we find no merit to the protester’s argument that the “discussions” were inadequate.

It is well-established that contracting agencies acquiring research and development have substantial discretion to determine which proposals they will fund, consistent with their needs and available funding. EH Group, Inc., supra at 6 (denying protester’s challenges to the evaluation and nonaward determination in a CSO procurement); Front End Analytics, LLC, B-420024.2, B-420024.3, Feb. 2, 2022, 2022 CDP ¶ 53 at 6. Our Office has specifically recognized that, with respect to CSO procurements, contracting agencies have broad discretion to determine their needs and the best way to meet them. EH Group, Inc., supra; see ASRC Fed. Data Network Techs., LLC, B-418765, Aug. 28, 2020, 2020 CPD ¶ 339 at 6.

In light of this discretion, our review of a CSO procurement is limited to determining whether the agency acted in bad faith or violated any applicable regulations or solicitation provisions. EH Group, Inc., supra. A protester’s disagreement with the agency’s judgment, by itself, does not establish that an evaluation was unreasonable. Glatz Aeronautical Corp., B-405851, B-405851.2, Jan. 6, 2012, 2012 CPD ¶ 19 at 4. This is particularly true under a CSO procurement, which is not based on design or performance specifications for existing equipment, but rather emphasizes scientific and technological innovation and has as its objective the development of new technology. It is precisely because of the scientific and innovative nature of this type of procurement that the agency is given substantial discretion in determining which proposals it will fund. EH Group, Inc., supra; see Wang Electro-Opto Corp., B-418523, June 4, 2020, 2020 CPD ¶ 187 at 5.

An agency is not required to conduct discussions under CSO procedures. See AR, Tab 33, 2018 CSO Class Deviation at 1-3; see also Blue Origin Federation, LLC; Dynetics, Inc.-A Leidos Co., B-419783 et al., July 30, 2021, 2021 CPD ¶ 265 at 28 (finding that FAR part 35 does not include any comparable provisions to FAR part 15’s discussion rules). However, where an agency as part of a CSO procurement avails itself of negotiated procurement procedures, the agency should treat firms fairly and reasonably in the conduct of those procedures. See ERIE Strayer Co., B-406131, Feb. 21, 2012, 2012 CPD ¶ 101 at 4.

In this regard, FAR section 15.306 describes a range of exchanges that may take place when an agency decides to conduct exchanges with offerors during negotiated procurements. Clarifications are “limited exchanges” between an agency and an offeror for the purpose of eliminating minor uncertainties or irregularities in a proposal, and do not give an offeror the opportunity to revise or modify its proposal. FAR 15.306(a)(2); American Material Handling, Inc., B-410899, March 12, 2015, 2015 CPD ¶ 106 at 5. Clarifications are not to be used to cure proposal deficiencies or material omissions, or materially alter the technical or cost elements of the proposal, or otherwise revise the proposal. Id.

Discussions, on the other hand, occur when an agency communicates with an offeror for the purpose of obtaining information essential to determine the acceptability of a proposal, or provides the offeror with an opportunity to revise or modify its proposal in some material respect. Alliant Enter. JV, LLC, B-410352.4, Feb. 25, 2015, 2015 CPD ¶ 82 at 5; see FAR 15.306(d). We have also stated that discussions, when conducted, must be meaningful; that is, discussions must identify deficiencies and significant weaknesses in an offeror’s proposal that could reasonably be addressed so as to materially enhance the offeror’s potential for receiving award. See, e.g., Shearwater Mission Support, LLC, B-416717, Nov. 20, 2018, 2018 CPD ¶ 402 at 7.

Relevant to the protest here, on January 19, 2022, after the receipt of Call 4 proposals, the Air Force conducted a “Pitch Day” with each offeror, including PavCon. COS at 8. Pitch Day was essentially an oral presentation intended to augment the offeror’s written submission and to allow for interaction between agency and contractor representatives. AR, Tab 23, Pitch Day Instructional Guide at 1. Further, the information presented at Pitch Day was to be considered together with the offeror’s written submission as part of the agency’s evaluation. Id. The Pitch Day instructions provided for up to a 30-minute presentation by the offeror (using a pre-submitted Power Point slide deck), followed by up to a 20-minute “question and answer” session. Id. at 2. PavCon conducted its Pitch Day presentation using a slide deck which reviewed the firm’s background and capabilities, and its technical approach to AI/ML in support of CBM+. AR, Tab 24, PavCon Pitch Day Briefing Charts at 1-22. The agency posed two questions to PavCon regarding its technical approach, and took notes of the presentation. AR, Tab 25, Pitch Day Notes Regarding PavCon at 1-3; COS at 8.

After Pitch Day, on January 26, the agency sent eight written questions to PavCon seeking clarification regarding its proposed technical approach. AR, Tab 26, Request for Clarification for PavCon at 1-2. For example, the Air Force asked, “[f]or clarification purposes, . . . [PavCon] assumption 22 and 30 states ‘the government will acquire necessary licenses will be provided.’ No licenses are required for access to [the Air Force’s predictive analytics and decision assistance] PANDA [software platform]. Please disclose any licenses required to execute the proposed technical solution,” and “[i]f COVID-19 limits or prohibits site . . . visits, will PavCon be able to meet the requirements of the contract?” Id. On February 3, PavCon provided “answers/ clarification” to the agency’s questions. AR, Tab 27, PavCon Response to Request for Clarification at 1-3. The record reflects that on February 23, the Air Force subsequently considered PavCon’s written proposal, Pitch Day presentation, and clarification answers when evaluating the firm’s technical approach. AR, Tab 28, Agency Evaluation; COS at 24-26.

The Air Force argues that it was not required to engage in discussions with PavCon by the terms of the CSO Call 4 solicitation or the DOD CSO Class Deviation, nor did it actually hold discussions with any firm, including PavCon. MOL at 12-13. The agency also contends that the questions asked of the protester orally during the Pitch Day presentation as well as subsequently in writing constituted clarifications and not discussions. Id. at 13.

PavCon’s argument that the agency failed to engage in meaningful discussions is essentially as follows: the agency’s exchanges with PavCon (both the Pitch Day and subsequent written questions) were discussions because the information exchanged was relied on by the evaluators to determine that PavCon’s proposal was technically unacceptable; and once the agency engaged in discussions, it was required to conduct meaningful discussions by raising all aspects of PavCon’s proposal that were identified as deficiencies. Comments at 2-7, citing International Waste Indus. (IWI),
B-411338, July 7, 2015, 2015 CPD ¶ 196 at 5. The agency was thus required, the protester argues, to “raise any concerns it may have had regarding PavCon’s proposal during the Pitch Day exchanges and in the written [request for information] RFIs, and to inform PavCon of any Significant Weaknesses or Deficiencies that the Agency believe existed in PavCon’s proposal which made PavCon’s proposal technical Unacceptable.” Id. at 6. We find the protester’s assertions to be meritless.

As a preliminary matter, as noted above, the agency identified deficiencies in the protester’s proposal in its completed technical evaluation on February 23. The protester fails to explain how its proposal deficiencies could have been raised during the Pitch Day presentation, or the written clarification request, January 19 and 26 respectively, when the Air Force’s evaluation was not completed--and the deficiencies not identified--until after such exchanges occurred--February 23. The FAR establishes that when conducting discussions, an agency must convey, at a minimum, all significant weaknesses and deficiencies in the offeror’s proposal. FAR 15.306(d)(3). Implicit in this requirement is the fact that the agency identifies significant weaknesses and deficiencies from an offeror’s proposal first, and provides the offeror with an opportunity to address such concerns second.[7] Thus, the protester’s argument that the agency was required to inform PavCon of its proposal deficiencies as part of the Pitch Day presentation or subsequent written exchange puts the cart before the proverbial horse.

We also find PavCon’s reliance upon our decision in IWI to be misplaced. In IWI, a procurement conducted using simplified acquisition procedures pursuant to FAR part 13, we sustained the protest where the agency engaged in discussions with the awardee but not with the protester. IWI, supra at 5. We stated that “[a]lthough an agency is not required to conduct discussions under simplified acquisition procedures, where an agency avails itself of negotiated procurement procedures, the agency should fairly and reasonably treat offerors in the conduct of those procedures.” Id. at 5. Here, however, there is simply no evidence, or even an assertion, that the agency treated PavCon unfairly as compared to other Call 4 RFP offerors with regard to the conduct of exchanges.

Finally, the protester also argues that these exchanges must have been discussions because the agency utilized the information from the exchanges to find the firm technically unacceptable. Importantly, PavCon does not assert that the Pitch Day and written exchanges went beyond clarifying minor uncertainties in the offeror’s proposal. Protest at 20; Comments at 3-4. Here, the protester mischaracterizes both the FAR’s definition of discussions and our decisions. As a result, we conclude that the agency did not engage in discussions.

In this regard, we have consistently stated that discussions occur when a contracting agency communicates with an offeror for the purpose of obtaining information essential to determine the acceptability of a proposal, or provides the offeror with an opportunity to revise or modify its proposal in some material regard. See, e.g., Alliant Enter. JV, LLC, B-410352.4, Feb. 25, 2015, 2015 CPD ¶ 82 at 5; Highmark Medicare Servs., Inc. et al., B-401062.5 et al., Oct. 29, 2010, 2010 CPD ¶ 285 at 11; see FAR 15.306(d). Stated otherwise, when an agency engages in an exchange to obtain information that is required in order to cure an otherwise unacceptable proposal (i.e., to make an unacceptable proposal acceptable), that exchange constitutes discussions. See Serco Inc., B-406061.1, B-406061.2, Feb. 1, 2012, 2012 CPD ¶ 61 at 13 (discussions occur when an agency communicates with an offeror to obtain information essential to determine the acceptability of a proposal); Gulf Copper Ship Repair, Inc., B-293706.5, Sept. 10, 2004, 2005 CPD ¶ 108 at 6 (same). In sum, the protester’s legal predicate for its argument is not accurate; the FAR does not define discussions as exchanges used to find a proposal unacceptable.

In essence, PavCon contends that any exchange of information with an agency after an offeror submits its proposal which the agency then considers when finding the proposal to be unacceptable, also “legally” constitutes discussions. We find no logic to this argument. In such a circumstance, the exchange is clearly not of a type where the agency obtains information required to cure an otherwise unacceptable proposal, i.e., discussions. The protester’s contention would also transform even the most minor of clarifications into discussions if the information obtained was considered when finding a proposal to be unacceptable. In sum, as PavCon has failed to show that the exchanges exceeded eliminating minor uncertainties, or included information required to cure an otherwise unacceptable proposal, it has failed to demonstrate that the agency engaged in discussions. As the agency was neither required to conduct discussions as part of a CSO procurement nor did it actually do so, we find no merit to the protester’s argument that the “discussions” were inadequate.[8]

Technical Evaluation of PavCon

PavCon also challenges the agency’s evaluation of its technical approach. Specifically, the protester contends that the three identified deficiencies were improper insofar as the agency did not evaluate the full substance of the offeror’s proposal or alternatively, evaluated the submission against requirements not indicated in the solicitation. Had the agency conducted a proper evaluation, PavCon argues, its proposal would not have been found unacceptable. Protest at 11-18, 25; Comments at 8-24.

As stated above, contracting agencies have substantial discretion to determine which proposals they will fund under CSO procurements, as contracting agencies have broad discretion to determine their research and development needs and the best way to meet them. EH Group, Inc., supra; see ASRC Fed. Data Network Techs., LLC, supra. In light of this discretion, our review of a CSO procurement is limited to determining whether the agency acted in bad faith or violated any applicable regulations or solicitation provisions. EH Group, Inc., supra. Moreover, as stated above, a protester’s disagreement with the agency’s judgment, by itself, does not establish that an evaluation was unreasonable. Glatz Aeronautical Corp., supra. Against this background, we find no reason to question the agency’s evaluation of PavCon’s proposal or its conclusion not to select the proposal for award.

For example, with regard to the second assigned deficiency, the RFP’s statement of objectives stated that,

Analyze at a minimum 125 new WUCs per aircraft platform group . . . for currently fielded aircraft platforms or new aircraft platforms with the goal to field as many new WUCs as possible. The successful Offeror shall make recommendations as to which new WUCs should be fielded, however the Government shall be the final approval authority in selecting the new WUCs for fielding. The successful Offeror shall perform analysis by utilizing historical flight, maintenance, and supply records to create part profiles that include supply demand, failure distributions, and probability models, to establish remaining useful life forecasts. The Government shall approve WUC removal thresholds. Analyzed WUCs will require acceptance for fielding by the government. Historical data suggests that approximately 25% of the analyzed WUCs will transition to fielding. Execution and sustainment processes will be required for all fielded WUCs including any new fielded WUCs under this contract.

AR, Tab 15, Call 4 RFP amend. 1, attach. 1, Proposal Instructions at 7. The evaluators found PavCon’s proposal to be unacceptable in this area for the following reasons:

• The proposal presented a broad technical solution that lacked sufficient details.
• “PavCon stated that they will ‘Analyze up to 250 new WUCs across the Sustainment Platforms with the goal to field at least 25% of the newly analyzed WUCs during the base period. . . .[’] The proposal also stated, ‘historically, about 25% of components analyzed proceed to the fielded status; the remaining 75% are placed into a holding pattern due to limiting factors. . . .’ The [RFP] requirement[, however,] is that the contractor will analyze at a minimum 250 . . . WUCs.”
• The proposal provided insufficient detail regarding how the offeror will sustain fielded WUCs, specifically because the sustainment plan does not address analysis for previously-fielded WUCs. The Government includes post-WUC analysis as part of the sustainment process. [PavCon] Proposal Assumption 11 states “No post-WUC analyses or data studies will be completed for already fielded WUCs.” PavCon’s assumption that they will not be performing this work leaves an overarching gap in the sustainment requirements for fielded WUCs.
• PavCon states “’PavCon assumes that Subject Matter Experts will label a randomized sampling of 10% of data per WUC historical data set to train the WUC ML model for each new WUC analyzed.’” PavCon’s approach to limiting the randomized sampling to 10 percent of data per WUC historical data set is not a technically sound approach. The technical team conducting the scientific and peer review determined that labeling a 10 percent data set sample is insufficient to analyze and field WUCs specified in the RFP. Data quality often drives a need to label a percentage of data greater than 10 percent in order to accomplish WUC analysis and fielding at the level acceptable to meet the objective, therefore the manner proposed is technically inadequate as the technical team determines placing a 10 percent maximum threshold on the data sampling erodes the soundness and technical merit of the proposed solution for the desired objective.

AR, Tab 28, PavCon Evaluation Report at 3-4.

We find the agency’s evaluation of PavCon’s proposal provides no basis on which to sustain the protest. The record reflects that the agency reasonably determined that PavCon’s proposal lacked sufficient detail overall; stated that it would analyze up to 250 new WUCs when the RFP required analyzing a minimum of 250 new WUCs; failed to address analysis of previously-fielded WUCs; and found the proposed 10 percent data sampling to be insufficient to analyze and field WUCs such that it “erode[ed] the soundness and technical merit of the proposed solution.” Id. at 4.

We also find no merit in the protester’s assertion that sustainment analysis of previously-fielded WUCs was an unstated evaluation criterion. The RFP established that under the technical evaluation factor, the agency would assess the “soundness and technical merit of the proposed solution” in addressing the statement of objectives. AR, Tab 15, Call 4 RFP amend. 1, attach. 1, Proposal Instructions at 1. In turn, the statement of objectives stated that “[e]xecution and sustainment processes will be required for all fielded WUCs including any new fielded WUCs under this contract.” Id. at 7.

In light thereof, we find the agency’s consideration of PavCon’s failure to include an analysis of previously-fielded WUCs in its sustainment plan was, even if not expressly stated, reasonably related to or encompassed by the stated evaluation criteria. Front End Analytics, LLC, supra at 8; Quimba Software, B-299000, Jan. 18, 2007, 2007 CPD ¶ 14 at 3. In sum, while PavCon disagrees with the reasonableness of the agency’s evaluation, we find this disagreement does not provide a basis on which to sustain the protest, especially where, as here, contracting agencies have substantial discretion to determine which proposals they will fund under CSO procurements,

The protest is denied.

Edda Emmanuelli Perez
General Counsel

 

---