Spry Methods, Inc.; Castalia Systems, LLC
Highlights
Spry Methods, Inc., a small business of McLean, Virginia, and Castalia Systems, LLC, a small business of Tampa, Florida, protest the award of a contract to RiVidium, Inc., d/b/a TripleCyber, a small business of Vienna, Virginia, under request for proposals (RFP) No. HM0476-21-R-0025, issued by the National Geospatial-Intelligence Agency (NGA) for cybersecurity capabilities and services in support of NGA's chief information security officer. The protesters challenge various aspects of the agency's evaluation of proposals and the resulting source selection decision.
DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.
Decision
Matter of: Spry Methods, Inc.; Castalia Systems, LLC
File: B-421640.3; B‑421640.4; B-421640.5; B-421640.6
Date: April 17, 2024
Olivia L. Lynch, Esq., Louis A. Chiarella, Esq., Sarah F. Burgart, Esq., Zariah T. Altman, Esq., and Emily P. Golchini, Esq. Crowell & Moring LLP, for Spry Methods, Inc.; Lewis P. Rhodes, Esq., and Orest J. Jowyk, Esq., Reston Law Group LLP, for Castalia Systems, LLC, the protesters.
James Y. Boland, Esq., Lindsay M. Reed, Esq., Allison M. Siegel, Esq., and Kelly M. Boppe, Esq., Venable LLP, for RiVidium, Inc., d/b/a TripleCyber, the intervenor.
Anthony Lascola, Esq., Mary Elizabeth C. McKenney, Esq., and Richard L. Donaldson, Esq., National Geospatial-Intelligence Agency, for the agency.
Christopher Alwood, Esq., and Alexander O. Levine, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.
DIGEST
1. Protests challenging agency’s evaluation of proposals under the technical/management and price factors are denied where the evaluation was reasonable and consistent with the terms of the solicitation.
2. Protests challenging agency’s evaluation of past performance are denied where the evaluation was reasonable and consistent with the solicitation’s criteria or the protesters did not establish that they were competitively prejudiced by the agency’s actions.
3. Protests challenging agency’s best‑value tradeoff are denied where the agency’s tradeoff was reasonable, adequately documented, and consistent with the terms of the solicitation.
DECISION
Spry Methods, Inc., a small business of McLean, Virginia, and Castalia Systems, LLC, a small business of Tampa, Florida, protest the award of a contract to RiVidium, Inc., d/b/a TripleCyber, a small business of Vienna, Virginia, under request for proposals (RFP) No. HM0476-21-R-0025, issued by the National Geospatial-Intelligence Agency (NGA) for cybersecurity capabilities and services in support of NGA’s chief information security officer. The protesters challenge various aspects of the agency’s evaluation of proposals and the resulting source selection decision.
We deny the protests.
BACKGROUND
On November 29, 2021, the agency issued the RFP as a small business set-aside under the commercial item procedures of Federal Acquisition Regulation (FAR) part 12, using the negotiated procurement policies and procedures established under FAR part 15. Contracting Officers Statement and Memorandum of Law (COS/MOL), B‑421640.3. The solicitation sought proposals to provide cybersecurity capabilities and services under what the RFP referred to as the Defender contract.[1] AR, Tab A.8.a, RFP at 34, 54; AR, Tab A.6.b, SOW at 5.[2] The RFP contemplated the award of a contract for a 6‑month base period, seven 1-year option periods, and one 6-month option period. RFP at 31‑33.
The RFP provided for award on a best‑value tradeoff basis, considering price and the following four non-price factors: (1) technical/management; (2) past performance; (3) security; and (4) intellectual property. Id. at 56‑57. The security factor was to be evaluated on a pass/fail basis. Id. at 56. The RFP stated that, for the purposes of the best‑value tradeoff, the other non-price factors, when combined, were significantly more important than price. Id. at 54. Within the non-price factors, the RFP explained that the technical/management factor was significantly more important than past performance, which was in turn more important than the intellectual property factor. Id. at 56. The RFP specified that the technical/management factor consisted of the following four subfactors listed in descending order of importance: (1.1) computer network exploitation, (1.2) cybersecurity risk management, (1.3) key personnel, and (1.4) contract dashboard/web portal. Id.
The RFP provided for a multi-phase procurement. In phase 1, offerors were to submit written proposals addressing the computer network exploitation subfactor. RFP at 54. The agency was to evaluate the phase 1 proposals before conducting an advisory “down‑select.” Id. Based on the evaluation, the agency was to advise each offeror whether the offeror was recommended to proceed to phase 2. Id. The RFP provided that, regardless of the agency’s recommendation, any offeror whose proposal was evaluated as acceptable or better under the computer network exploitation subfactor was eligible to continue to phase 2. Id. For phase 2, offerors were to submit written proposals addressing the remainder of the evaluation factors and subfactors. Id. The agency was to evaluate phase 2 proposals without further evaluating phase 1 proposals. Id.
The RFP provided that the agency would evaluate proposals under the technical/management factor to “assess the extent to which the [o]fferor understands and provides the capabilities to satisfy” the SOW’s requirements. Id. at 57. The agency was to assign an adjectival rating of outstanding, good, acceptable, marginal, or unacceptable to the overall technical/management factor and to each of its four subfactors.[3] The RFP specified that, in arriving at the ratings for each subfactor, the evaluators would consider the quality of the offerors’ technical solutions along with the technical risk.[4] Id. The agency was to assess the offerors’ approaches in relation to the evaluation criteria by identifying strengths, weaknesses, and deficiencies in the proposals.[5] Id.
The agency was to evaluate proposals under the computer network exploitation subfactor by assessing the offerors’ written responses to a provided computer network exploitation scenario. Id. at 59. As relevant here, the agency was to assess the extent to which each offeror demonstrated the ability to develop an execution plan that would leverage multiple tactics, techniques, and procedures to breach and exfiltrate data. Id. Under the cybersecurity risk management subfactor, the agency was to assess the offerors’ technical approaches to execute the risk management framework process as identified in National Institute of Standards and Technology (NIST) special publication (SP) 800-37 R2, including any “innovative technical approaches [or] solutions to accelerate” the risk management framework process.”[6] Id.
Under the key personnel subfactor, offerors were required to submit resumes for the three proposed key personnel described in the SOW. Id at 47‑48. The agency was to evaluate the resumes to assess whether proposed key personnel met the SOW’s work role certification requirements and had the knowledge, skills, and abilities to perform the work for the proposed position. Id. at 59.
For the contractor dashboard/web portal subfactor, offerors were to propose an interactive dashboard “to act as the principal conduit for all contract, program, performance, and schedule” documents and data between the agency and the contractor. RFP at 48; AR, Tab A.6.b, SOW at 15. The RFP instructed offerors to describe how the proposed dashboard would provide interactive views into NGA’s cybersecurity status and how the dashboard would integrate with other NGA enterprise tools and dashboards. RFP at 48. The SOW required that the dashboard comply with RFP contract data requirements list (CDRL) 0010. AR, Tab A.6.b, SOW at 15. CDRL 0010 described the types of information that the dashboard was required to include and stated that the “initial submission” was due “60 calendar days after contract start,” with weekly updates, at a minimum, thereafter. AR, Tab A.4.c, RFP amend. 0003, attach. 3, CDRLs at 13‑14.
Under this subfactor, the agency would assess each offeror’s “innovative technical approaches/solutions” and the extent to which the offeror demonstrated a clear understanding of how best to provide access to and maintain the dashboard. RFP at 59. The agency was also required to consider the ability of the dashboard “to provide the Government with interactive view(s) into NGA’s cybersecurity status” and how the dashboard integrates with other NGA enterprise tools and dashboards. Id.
Under the past performance factor, the agency would assess the ability of the offeror and its subcontractors to perform the requirement based on the offeror’s “recent and relevant record of performance in supplying services that are similar to the contract requirements.” Id. The RFP provided that the agency would consider information provided by the offerors as part of their proposals, responses to past performance questionnaires (PPQs), interviews, and information obtained from government databases, such as the contract performance assessment reporting system or NGA performance assessment reports. Id. at 60. The RFP advised that the agency could consider past performance on contracts other than those identified by the offerors. Id.
The agency was to evaluate how relevant offerors’ recent past performance was to the current requirement based on the contractor’s role on the project, the similarity of services provided, the dollar value and complexity, the availability of similarly certified staff, the contract type, and the extent of subcontracting used. Id. The agency was to assign each contract a rating of either very relevant, relevant, somewhat relevant, or not relevant. Id. at 60‑61.
The agency was to consider the quality of recent and relevant past performance based on the contractor’s technical performance, management performance, and price performance. Id. Based on all the information collected, the agency would assign an overall past performance confidence assessment rating of either substantial, satisfactory, neutral, limited, or no confidence. Id. The RFP specified that “[m]ore relevant [past] performance efforts will have a greater impact on the confidence rating” than less relevant efforts. Id.
With regard to price, the RFP instructed offerors to complete an attached price model using only fully burdened labor rates. Id. at 53. The price model contained 71 labor categories mapped to the SOW’s requirements. AR, Tab A.8.b, RFP amend. 0007, attach. 7 Price Model. For each labor category, the price model included a specified number of full-time-equivalents (FTE) and estimated labor hours per year that would be multiplied by a proposed labor rate to calculate the proposed price. Id. The RFP specified that proposals were only to input price information where instructed by the price model and were not otherwise permitted to modify the data, equations, labor categories, or FTE requirements already contained in the price model. RFP at 53. Separately, the RFP required offerors to include in their price proposals “a list of all key ground rules and assumptions that have significant impacts on the proposed price.” Id. The RFP specified that “[t]he ground rules and assumptions included in this section shall track to the other sections of the offer as appropriate.” Id.
As relevant here, the RFP stated that the agency could, but was not required to, conduct a price realism analysis “on the proposed fully burdened labor rates and/or the overall price proposed.” Id. at 66. The RFP explained that such an analysis would be “applied in order to assess the risk of performance due to unrealistically low prices.” Id.
Following an advisory down‑select, on May 6, 2022, the agency received timely phase 2 proposals from six offerors. COS/MOL, B‑421640.3 at 5. Following the initial evaluation of phase 2 proposals, the agency established a competitive range of three offerors: Spry, Castalia, and RiVidium. Id. at 6. The agency subsequently conducted discussions with the three offerors and, on January 27, 2023, requested final proposals from all three offerors. Id. On April 7, the agency awarded the contract to RiVidium.
Castalia subsequently filed a protest with our Office, challenging various aspects of the agency’s evaluation of proposals, conduct of discussions, and source selection decision. Castalia Sys., LLC, B‑421640, May 23, 2023 (unpublished decision). Spry also filed a protest with our Office, arguing that the agency unreasonably evaluated proposals and made an unreasonable best-value tradeoff decision. Spry Methods, Inc., B‑421640.2, May 23, 2023 (unpublished decision). Following these protests, the agency notified our Office that it intended to take corrective action by, at a minimum, terminating the award to RiVidium, reevaluating final proposals, and making a new award decision. See id. Based on the agency’s intended corrective action, we dismissed the protests as academic. Id.; Castalia Sys., LLC, supra.
The agency reevaluated the final proposals as follows:
RiVidium |
Spry |
Castalia |
|
---|---|---|---|
Technical/Management |
Outstanding |
Good |
Outstanding |
Computer Network Exploitation |
Good |
Good |
Outstanding |
Cybersecurity Risk Management |
Outstanding |
Good |
Acceptable |
Key Personnel |
Outstanding |
Outstanding |
Outstanding |
Contract Dashboard/ Web Portal |
Acceptable |
Acceptable |
Acceptable |
Past Performance |
Satisfactory Confidence |
Substantial Confidence |
Substantial Confidence |
Security |
Pass |
Pass |
Pass |
Intellectual Property |
Low Risk |
Low Risk |
Low Risk |
Price |
$249,058,235 |
$290,017,987 |
$266,433,485 |
AR, Tab E.2, Castalia Source Selection Decision Document (SSDD) at 4; AR, Tab E.4, Spry SSDD at 4.
The source selection authority (SSA) independently assessed the proposals and reviewed the source selection advisory council’s (SSAC) and the source selection evaluation board’s (SSEB) reports. See AR, Tab E.4, Spry SSDD at 12‑13. The SSA agreed with the evaluation findings and the SSAC’s recommendation that RiVidium’s proposal represented the best value. Id. The SSA concluded that RiVidium’s proposal was a better value than Spry’s because it was evaluated to be “slightly more beneficial” under the non-price factors at a lower evaluated price. Id. at 10. The SSA also concluded that, while Castalia’s proposal was more advantageous than RiVidium’s under the past performance factor, the specific advantages arising from Castalia’s past performance did not justify a $17,375,250 price premium. NGA subsequently notified both Spry and Castalia that they had not been selected for award and provided them with debriefings. See COS/MOL, B-421640.3 at 8‑9. These protests followed.
DISCUSSION
The protesters generally challenge the agency’s evaluation of proposals and resulting source selection decision. We note that the protesters raise many collateral arguments. While our decision does not specifically address every argument, we have reviewed all the arguments and conclude that none provide a basis to sustain the protests.[7] We discuss several representative issues below.
Technical/Management
Adjectival Ratings
Castalia challenges the agency’s evaluation of RiVidium’s proposal under the technical/management factor. Castalia Protest at 19; Castalia Comments at 3‑4. In this regard, Castalia argues that the agency’s evaluation was inconsistent because it assigned RiVidium’s proposal a rating of outstanding under the computer network exploitation subfactor and a rating of outstanding under the cybersecurity risk management subfactor despite assessing the same number of moderate and slight strengths under each subfactor. Castalia Comments at 4. Castalia also argues that RiVidium’s overall technical/management rating of outstanding was unreasonable because the agency did not assess RiVidium’s proposal a significant strength under either of the two most important subfactors. Id.
The agency responds that it reasonably assessed these ratings in accordance with the RFP. COS/MOL, B-421640.4 at 51‑52. The agency contends that Castalia’s arguments unreasonably focus on the adjectival ratings rather than the agency’s detailed consideration of the underlying bases for the ratings, which the agency documented in its evaluation. Id. at 52. We agree with the agency.
In reviewing a protest challenging an agency’s evaluation, our Office will not reevaluate proposals, nor substitute our judgment for that of the agency, as the evaluation of proposals is a matter within the agency’s discretion. AECOM Mgmt. Servs., Inc., B‑417639.2, B‑417639.3, Sept. 16, 2019, 2019 CPD ¶ 322 at 9. Rather, we will review the record to determine whether the agency’s evaluation was reasonable and consistent with the stated evaluation criteria and with applicable procurement statutes and regulations. Id. A protester’s disagreement with the agency’s judgment, without more, is insufficient to establish that the agency acted unreasonably. Vertex Aerospace, LLC, B‑417065, B‑417065.2, Feb. 5, 2019, 2019 CPD ¶ 75 at 8.
Castalia’s disagreement with RiVidium’s ratings based on the number of strengths assessed and the classification of those strengths is misplaced. Evaluation ratings and the number of strengths and weaknesses assessed are merely a guide to, and not a substitute for, intelligent decision making in the procurement process. Affolter Contracting Co., Inc., B‑410878, B‑410878.2, Mar. 4, 2015, 2015 CPD ¶ 101 at 11 n.10.
As noted above, under the technical/management factor, a rating of outstanding required the agency to conclude that the proposal indicated an exceptional approach and understanding of the RFP’s requirements, contained multiple strengths, and had a low risk of unsuccessful performance. RFP at 57. On the other hand, a rating of good indicated that the proposal presented a thorough approach and understanding of the requirements, contained at least one strength, and had a low to moderate risk of unsuccessful performance. Id. Castalia has not explained why the different subfactor ratings objected to here fail to meet the respective rating definitions based on the number of strengths assessed. Additionally, nothing in the rating definitions, or the solicitation generally, requires the agency to assign the same rating under different subfactors simply because the offeror’s proposal received the same number of strengths under both subfactors. Castalia has also failed to explain why the agency could not assign an overall rating of outstanding to a proposal that had not been assessed a significant strength when the RFP’s outstanding rating definition simply refers to “strengths” generally and does not further differentiate between significant strengths or less significant strengths. RFP at 58. Accordingly, we find that Castalia’s challenge to these adjectival ratings simply constitutes disagreement with the agency’s judgements and we deny this ground of protest.
Technical/Management Evaluation of RiVidium
Next, both Spry and Castalia contend that the agency unreasonably failed to evaluate implementation risks in RiVidium’s proposed interactive dashboard. Spry Protest at 86‑87; Spry 1st Comments & Supp. Protest at 4‑17; Spry 2nd Comments & Supp. Protest at 13‑21; Castalia Protest at 13; Castalia Comments at 4. As noted above, offerors were required to propose an interactive dashboard “to act as the principal conduit for all contract, program, performance, and schedule” documents and data between the agency and the contractor. RFP at 48; AR, Tab A.6.b, SOW at 15. To meet this requirement, RiVidium proposed to use proprietary software it calls “Orion.”[8] See AR, Tab B.2.o, RiVidium Proposal, Vol. 2 at 1‑2. The protesters argue that, because this software is not currently deployed within NGA’s systems, its implementation is likely to face several issues that increase the risk of unsuccessful performance.[9] See Spry 1st Comments & Supp. Protest at 9‑14. Spry avers that under the RFP’s evaluation scheme, the agency was required to consider performance risk and therefore should have assessed several significant weaknesses or deficiencies in RiVidium’s proposal to reflect multiple risks arising from the implementation of Orion. Id at 17.
The agency responds that its evaluation of RiVidium’s proposed interactive dashboard was reasonable and consistent with the requirements of the RFP. Supp. COS/MOL, B‑421640.5 at 3‑7. The agency concedes that it did not consider the implementation risks described by the protesters but argues that it was not required to do so as part of its evaluation under the contractor dashboard/web portal subfactor. See Id. at 5. In this regard, the agency contends that the RFP only required the agency to evaluate the offerors’ approaches to providing an interactive dashboard and that considering offerors’ implementation plans, or the risks therein, was not required by the solicitation. Id.
Where a dispute exists as to a solicitation’s actual requirements, we will first examine the plain language of the solicitation. Bauer Techs., Inc., B‑415717.2, B‑415717.3, June 22, 2018, 2018 CPD ¶ 217 at 4. Where a protester and an agency disagree over the meaning of solicitation language, we will resolve the matter by assessing whether each posited interpretation is reasonable. Anders Constr., Inc., B‑414261, Apr. 11, 2017, 2017 CPD ¶ 121 at 3. To be reasonable, an interpretation must be consistent with the solicitation when read as a whole and in a reasonable manner. Planned Sys. Int’l, Inc., B‑413028.5, Feb. 21, 2018, 2018 CPD ¶ 126 at 6.
The RFP required the agency to consider technical risk as part of its evaluation of “the quality of the [o]fferor’s technical solution” under the technical/management factor. RFP at 57. As part of the evaluation of risk, the agency was to consider whether a proposal contained weaknesses or significant weaknesses that had the potential to cause “disruption of schedule, increased cost or degradation of performance” and whether government monitoring would be likely to overcome difficulties. Id. at 58. Under the contractor dashboard/web portal subfactor, the agency was to evaluate proposals by assessing each offeror’s “innovative technical approach [or] solutions and the extent to which the [o]fferor demonstrates a clear understanding of how best to provide access to and maintain” the interactive dashboard. RFP at 59. The agency was also required to evaluate the ability of the dashboard “to provide the Government with interactive view(s) into NGA’s cybersecurity status and how [the dashboard] integrates with other NGA [e]nterprise [t]ools and [d]ashboards.” Id.
Based on the plain language of the solicitation, when read as a whole, we find reasonable the agency’s interpretation that it was only required to assess the approach of the proposed dashboard, the offeror’s understanding of the RFP’s dashboard requirements, and the technical risks of these evaluated proposal aspects. While the RFP required the agency to evaluate how the dashboard “integrates with other NGA systems,” we note that this language is part of the same evaluation criterion as the RFP’s instruction that the agency would evaluate the ability of the dashboard “to provide the Government with interactive view(s).” Id. When read together, we view these as a requirement to consider what functionality the offeror’s dashboard provides. That is, the agency was required to consider what information the dashboard would display and which NGA systems the dashboard would automatically connect to, rather than an explanation of how the offeror was going to mechanically accomplish those connections. Further, our review of the RFP does not reveal, and the protester has not identified, any specific requirement that the agency evaluate offerors’ plans for implementing or deploying their proposed interactive dashboards.[10] Accordingly, without more, we find that the agency reasonably focused its evaluation on the merits of the functionality of the proposed dashboards rather than the logistics of deploying them.
In light of the above, we find nothing improper about NGA’s evaluation of RiVidium’s proposal under the contractor dashboard/web portal subfactor. The record demonstrates that the agency reasonably considered RiVidium’s proposed approach and the features of the interactive dashboard and identified positive aspects that NGA found “slightly advantageous to the [g]overnment.” AR, Tab G.2.b, RiVidium Technical Evaluation at 14. Further, the agency did not identify any weaknesses related to the proposed interactive dashboard and ultimately assessed RiVidium’s proposal a rating of acceptable under this subfactor. Id. at 13‑14. While the protesters clearly object to the agency’s non-assessment of weaknesses, the protesters have not demonstrated that the agency’s decision not to evaluate software implementation risks was unreasonable or failed to comply with the terms of the solicitation.[11] On this record, without more, we view the protesters objections as nothing more than disagreement with the agency’s judgement and deny this ground of protest.[12] Vertex Aerospace, LLC, supra.
Technical/Management Evaluation of Spry
Spry also argues that the agency unreasonably ignored or downplayed several benefits found in Spry’s technical/management proposal. In this regard, Spry identifies several aspects of its technical/management proposal that it believes the agency should have considered either a strength or a more significant strength than was assessed. Spry Protest at 56‑85; Spry Comments & Supp. Protest at 31‑39. Spry also generally contends that the evaluation record does not sufficiently document NGA’s consideration of the advantages of Spry’s proposal, and that a more fulsome assessment would have led to more assessed strengths and higher factor and subfactor ratings.
As an initial matter, we disagree with the protester that the agency failed to adequately document its evaluation under the technical/management factor. The agency explains that the technical/management evaluation team (TMET) considered the entirety of the proposals submitted by each offeror when conducting its evaluation under the technical/management factor. See COS/MOL, B-421640.3 at 46. The record shows that, as contemplated by the RFP, the TMET documented its evaluation conclusions through narratives that described aspects of the proposals the agency considered strengths. RFP at 57; AR, Tab G.1.b, Castalia Technical Evaluation at 8‑14; AR, Tab G.2.b, RiVidium Technical Evaluation at 8‑14; AR, Tab G.3.c, Spry Technical Evaluation at 8‑15. The TMET rated Spry’s technical/management proposal as having three significant strengths, three moderate strengths, and three slight strengths. AR, Tab G.3.c, Spry Technical Evaluation at 8.
That the TMET did not discuss in its report every proposal aspect identified by the protester does not mean that the agency did not consider them. Contrary to Spry’s assertions, an agency is not required to document all determinations of adequacy or explain why a proposal did not receive a strength, weakness, or deficiency for a particular item. Building Operations Support Servs. LLC, B‑407711, B‑407711.2, Jan. 28, 2013, 2013 CPD ¶ 56 at 5. That the TMET’s report primarily documented the strengths noted in Spry’s proposal was consistent with the solicitation.[13] See RFP at 57; see also FAR 15.305(a) (“The relative strengths, deficiencies, significant weaknesses, and risks supporting proposal evaluation shall be documented”). Accordingly, we see no basis to object to the manner and documentation of the agency’s evaluation of proposals.
Further, we have reviewed Spry’s numerous specific challenges regarding the agency’s evaluation of its proposal under the technical/management factor and find none of Spry’s arguments provide a basis to sustain the protest. For example, Spry contends that the agency unreasonably failed to credit Spry’s proposal with a significant strength under the computer network exploitation subfactor based on its cyber threat intelligence (CTI) methodology. Spry Protest at 56‑57; Spry Comments & Supp. Protest at 31‑33. In this regard, the protester explains that its proposed CTI methodology uses [DELETED], tailored to NGA’s actual environment, and “enhances it through research from multiple other reporting avenues.”[14] Spry Protest at 57. The protester argues that the agency’s evaluation failed to consider these and other elements of Spry’s CTI methodology that should have supported a significant strength.[15] Id. at 57‑58.
The agency responds that its evaluation explicitly considered the CTI methodology proposed by the protester, which formed part of the basis for a moderate strength assessed to Spry’s proposal under the computer network exploitation subfactor. COS/MOL, B‑421640.3 at 46‑47. The agency further notes that the RFP did not request that offerors tailor their scenario solution to NGA’s environment, and that the agency did not consider such tailoring to merit an additional strength. Id. at 47.
As noted above, under the computer network exploitation subfactor, the RFP provided that the agency would assess the extent to which the offeror demonstrates “the ability to develop an execution plan that leverages multiple Tactics, Techniques and Procedures [] to breach and/or exfiltrate data.” RFP at 59. The record demonstrates that the agency evaluated this aspect of Spry’s proposal, specifically crediting the offeror with a moderate strength for its approach to leverage [DELETED] to breach or exfiltrate data. AR, Tab G.3.a, Spry Phase 1 Evaluation at 2‑3. While the narrative of this moderate strength does not specifically address the other aspects of the CTI methodology identified by the protester, the agency clearly considered Spry’s CTI methodology in response to the scenario and documented the aspects it found supported the assessment of a strength. Id.
On this record, we see no basis to question the agency’s assessment of a moderate strength for Spry’s CTI methodology. We find it reasonable for the agency to have considered certain aspects offered by Spry’s CTI methodology to be moderately advantageous while not considering other aspects to support a more significant strength. As noted above, an agency is not required to explain why a proposal did not receive a strength, weakness, or deficiency for a particular aspect. Building Operations Support Servs. LLC, supra. While Spry may disagree with the agency’s judgements, it has failed to establish that those judgements were unreasonable here. Accordingly, we deny this ground of protest.
As another example, Spry argues that it should have been credited with a significant strength instead of a slight strength, under the cybersecurity risk management subfactor, for its ability to execute the risk management framework process.[16] Protest at 69. Spry contends that the agency should have credited this strength as even more advantageous due to the “thoroughness” of its proposal’s explanations of the actions it would take under each risk management framework step. Id. at 69; Spry Comments & Supp. Protest at 37.
We see no basis to sustain this protest ground. As an initial matter, the record shows that the agency specifically considered the thoroughness of this aspect of Spry’s proposal and concluded that Spry’s proposal “did not result in a higher magnitude of strength based upon the level of thoroughness.” AR, Tab G.3.c, Spry Technical Evaluation at 10. Moreover, Spry does not identify, and our review of the record does not reveal, anything in the solicitation that requires the agency to consider a strength more advantageous because the language describing that aspect of the proposal is very thorough. See RFP at 58. On this record, we find that Spry has not demonstrated that the agency’s failure to give additional credit to an existing strength based on the thoroughness of Spry’s proposal was unreasonable or contrary to the terms of the RFP. As such, we deny this ground of protest.
As a final example, Spry contends that the agency unreasonably failed to credit its proposal with “numerous significant benefits” under the contract dashboard/web portal subfactor.[17] Spry Protest at 83‑85; Spry Comments & Supp. Protest at 38‑39. The protester specifically argues that the agency should have credited its proposed dashboard’s [DELETED]. Spry Protest at 83‑84.
The agency responds that its evaluation of Spry’s proposal under the contractor dashboard/web portal subfactor was reasonable and consistent with the terms of the RFP. COS/MOL, B-421640.3 at 61‑63. The agency argues that its evaluation considered the aspects identified by Spry and reasonably determined that Spry’s proposed dashboard only merited a slight strength. Id.
As noted above, the RFP provided that the agency would evaluate proposals under the contractor dashboard/web portal subfactor by assessing “the [o]fferor’s innovative technical approach/solutions and the extent to which the [o]fferor demonstrates a clear understanding of how best to provide access to and maintain an interactive dashboard.” RFP at 59.
We have reviewed the evaluation record and find no basis to object to the agency’s evaluation here. In this regard, the record demonstrates that all three features that Spry claims the agency failed to consider were described in table 10 of Spry’s proposal. AR, Tab B.3.r, Spry Proposal, Vol. 2 at 33.[18] The agency’s evaluation of Spry’s proposal demonstrated that it considered this table, and explicitly considered seven of the table features, including [DELETED]. AR, Tab G.3.c., Spry Technical Evaluation at 14‑15 (citing AR, Tab B.3.r, Spry Proposal, Vol. 2 at 33). Notably, the [DELETED] cited in the agency’s evaluation report is the name of the [DELETED] feature that the protester claims was not considered by the agency. AR, Tab B.3.r, Spry Proposal, Vol. 2 at 33. The agency concluded that the dashboard features proposed by Spry merited a slight strength but explained that they “were not assessed to be so innovative in a manner that resulted in a moderate or significant advantage.” AR, Tab G.3.c., Spry Technical Evaluation at 15.
On this record, we find that the agency reasonably considered the interactive dashboard features proposed by Spry. As discussed above, the agency was not required to document or otherwise explain why a proposal did not receive a strength, weakness, or deficiency for each particular proposal aspect. Building Operations Support Servs. LLC., supra. Spry has not demonstrated that the agency’s evaluation was unreasonable or inconsistent with the RFP. Accordingly, we deny this ground of protest.
Past Performance
Both Spry and Castalia challenge the reasonableness of the agency’s evaluation of RiVidium’s past performance. Spry Protest at 87‑96; Spry Comments & Supp. Protest at 39‑43; Spry 2nd Comments & Supp. Protest at 34‑39; Castalia Protest at 19‑20; Castalia Comments at 4‑5; Castalia Supp. Briefing at 2‑4. The agency responds that its evaluation was reasonable and consistent with the terms of the solicitation. COS/MOL, B-421640.3 at 67‑68, 72‑73; COS/MOL, B‑421640.4 at 54‑57. We have reviewed the protesters’ arguments and the evaluation record and find that none provide a basis to sustain the protest.
Relevancy Ratings
As an initial matter, the agency requests that we dismiss Castalia’s argument that the agency’s assignment of relevant and somewhat relevant ratings to several of RiVidium’s past performance references was unreasonable because of the allegedly significant differences in their contract value and complexity from the current effort. Castalia Supp. Briefing at 2‑4. The agency requests that we dismiss these allegations as untimely supplemental protest grounds because they were not raised within 10 days of Castalia having been provided RiVidium’s past performance evaluation as part of the agency report. Req. for Partial Dismissal, B‑421640.4 at 2‑3. Castalia counters that the allegations raised in its supplemental briefing are not new grounds of protest and instead provide support for its initial protest allegation that RiVidium lacked any relevant past performance. Castalia Resp. to Req. for Dismissal at 2.
As noted above, our Bid Protest Regulations require protests to be filed no later than 10 days of when a protester knows, or should have known, of a basis for protest. 4 C.F.R. § 21.2(a)(2). Further, where a protester timely files a broad initial allegation and later supplements that broad allegation with allegations that amount to specific examples of the initial, general, challenge, and these examples involve different factual circumstances that require a separate explanation or defense from the agency, these specific examples must independently satisfy our timeliness requirements. This is because our regulations do not contemplate the piecemeal presentation of protest arguments. Savannah River Tech. & Remediation, LLC; Fluor Westinghouse Liquid Waste Servs., LLC, B-415637 et al., Feb. 8, 2018, 2018 CPD ¶ 70 at 6 (citing Vigor Shipyards, Inc., B-409635, June 5, 2014, 2014 CPD ¶ 170 at 5).
Here, Castalia broadly argues in its initial protest that RiVidium did not have any relevant past performance based on the RFP’s relevancy definition. Castalia Protest at 14. In its comments, however, Castalia did not specifically challenge the agency’s assignment of relevancy ratings to RiVidium’s past performance references. Castalia Comments at 4‑5. Instead, the protester waited until a March 4 supplemental briefing to challenge the agency’s ratings based on the dollar value and complexity of RiVidium’s past performance references. Castalia Supp. Briefing at 2‑4. The record demonstrates that the agency’s consideration of the dollar values and complexity of RiVidium’s past performance references was included in the past performance evaluation team (PPET) report, which was available to the protester as part of the February 9 agency report.[19] AR, Tab G.2.c, RiVidium PPET Report at 8‑11. We find that the protester’s failure to raise these specific allegations until it filed its supplemental briefing on March 4 constitutes the untimely piecemeal presentation of protest issues.
While Castalia argues that the allegations raised in its supplemental briefing are not new grounds of protest and instead provide support for its initial protest allegation that RiVidium lacked any relevant past performance, this argument does not change the piecemeal nature of Castalia’s presentation of its arguments. Castalia Resp. to Req. for Dismissal at 2. As explained above, where a protester timely files a broad initial allegation and later supplements that broad allegation with allegations that amount to specific examples of the initial, general, challenge, the specific examples must independently satisfy our timeliness requirements. Castalia’s specific arguments do not independently satisfy our timeliness requirements when they were filed more than 10 days after February 9 when it knew or should have known of the basis for these allegations.
Moreover, the original protest allegations were predicated on the assertion that RiVidium lacked any relevant past performance. Castalia Protest at 14. In its comments, Castalia does not challenge the agency’s assignment of relevancy ratings to RiVidium’s past performance references, but rather argues that the assigned ratings confirm Castalia’s initial argument that RiVidium had no relevant past performance. Castalia Comments at 4‑5. On the other hand, the allegations raised in the supplemental briefing are predicated on the underlying reasonableness of the relevancy ratings assigned to RiVidium’s past performance references based on the agency’s consideration of the contract value and complexity of the references. Castalia Supp. Briefing at 2‑4.
While Castalia maintains that its past performance protest grounds “cannot be deemed significantly different” from each other, a strong factual nexus between protest grounds does not change the independent legal nature of the allegations. See Medical Staffing Sols. USA, B‑415571, B‑415571.2, Dec. 13, 2017, 2017 CPD ¶ 384 at 3‑4 (protester’s subsequent allegation constituted a supplemental protest ground, even though both allegations were predicated on the agency’s past performance evaluation). Accordingly, Castalia’s allegations raised in its supplemental briefing are dismissed as untimely since the protester raised these protest grounds more than 10 days after it knew or should have known them.
RiVidium’s Past Performance Confidence Rating
Castalia also argues that the agency unreasonably assigned RiVidium a past performance confidence rating of satisfactory based on its submitted past performance record. Castalia Comments at 4‑5. Castalia notes that the agency found two of RiVidium’s subcontractor past performance references to be relevant, while only considering the remaining three past performance references to be somewhat relevant. Id. (citing AR, Tab E.1, SSAC Report at 51). Castalia contends that these relevancy results do not support RiVidium’s past performance confidence rating and that the agency’s considerations were insufficiently documented. Id.
An agency’s evaluation of past performance, including its consideration of the relevance, scope, and significance of an offeror’s performance history, is a matter of discretion which we will not disturb unless the agency’s assessments are unreasonable or inconsistent with the solicitation criteria. Metropolitan Interpreters & Translators, Inc., B‑415080.7, B‑415080.8, May 14, 2019, 2019 CPD ¶ 181 at 10; see also SIMMEC Training Sols., B‑406819, Aug. 20, 2012, 2012 CPD ¶ 238 at 4. A protester’s disagreement with the agency’s judgment does not establish that an evaluation was unreasonable. FN Mfg., LLC, B‑402059.4, B‑402059.5, Mar. 22, 2010, 2010 CPD ¶ 104 at 7.
The RFP instructed offerors to identify between three and five recent and relevant contracts performed by the offeror or a proposed major subcontractor. RFP at 48. The RFP required that at least two of the contracts “reflect performance of the prime [o]fferor (as either a prime or subcontractor).” Id. The RFP provided that the agency would evaluate the relevance of the offerors’ past performance considering the contractor’s role, the similarity of services provided, the dollar value, the complexity, the availability of similarly certified staff, the contract type, and the extent of subcontracting used. Id. at 60. The agency was to assign each past performance contract a relevancy rating of very relevant, relevant, somewhat relevant, or not relevant.[20] Id. at 60‑61. The agency was to consider the quality of recent and relevant past performance and assign an overall past performance confidence assessment rating of either substantial, satisfactory, neutral, limited, or no confidence. Id. The RFP specified that “[m]ore relevant [past] performance efforts will have a greater impact on the confidence rating” than less relevant efforts. Id.
RiVidium submitted two of its own contract references and three contract references from proposed major subcontractors. AR, Tab B.2.p, RiVidium Proposal, Vol.3 at 4‑18. The PPET then assessed the recency and relevancy of each submission. AR, Tab G.2.c, RiVidium PPET Report at 8‑11. The PPET concluded that all five references were recent, that RiVidium’s two own contract references were somewhat relevant, one of its major subcontractor’s references was somewhat relevant, and two of its major subcontractor references were relevant. Id. The record shows that the agency then considered the quality of RiVidium’s past performance and concluded that the agency had a reasonable expectation that RiVidium would successfully perform the required effort and therefore assigned RiVidium a rating of satisfactory confidence. Id. at 18‑20.
In light of the RFP’s evaluation criteria and the broad discretion afforded to the agency in a past performance evaluation, we see no basis to question the agency’s documentation of its past performance evaluation or its assignment of a satisfactory confidence rating for RiVidium’s past performance. The record shows that the agency considered the recency, relevance, and quality of RiVidium’s past performance as required by the RFP. While Castalia objects to the weight the agency gave to relevant subcontractor and somewhat relevant prime contractor past performance references, it has not demonstrated that the agency’s conclusions were unreasonable or inconsistent with the RFP. We deny this ground of protest.
Evaluation of the Source of Past Performance Information
Spry also contends that the agency failed to properly consider the source of past performance information in violation of FAR section 15.305(a)(2)(i).[21] Spry 1st Comments & Supp. Protest at 39‑43; Spry 2nd Comments & Supp. Protest at 34‑39. In this regard, Spry explains that one of RiVidium’s own past performance references was performed as a subcontractor to Calibre Systems, which is also one of RiVidium’s proposed subcontractors for the Defender contract. Spry 2nd Comments & Supp. Protest at 34‑35. Spry avers that the agency unreasonably relied on information from the PPQ provided by Calibre because Calibre stands to benefit if the award is made to RiVidium. Id. Spry argues that, had the agency properly rejected the PPQ from Calibre, then “RiVidium’s proposal would not fulfill the RFP’s requirements for two prime past performance references.” Id. at 39. Spry maintains that, at a minimum, the agency’s evaluation was unreasonable for considering the quality of RiVidium past performance from the allegedly biased Calibre PPQ. Spry Comments & Supp. Protest at 43.
As an initial matter, we disagree with Spry that any errors identified in one of RiVidium’s own references would require a neutral confidence rating “for failing to propose two references for itself” upon which the agency could evaluate the quality of RiVidium’s past performance. Spry 1st Comments & Supp. Protest at 42‑43. While the RFP’s submission instructions required offerors to submit two references that “reflect performance of the prime [o]fferor,” our review of the record does not reveal any RFP requirement that the agency automatically assign a neutral confidence rating, where the agency did not have a basis to evaluate the quality of one of those references. Here, the record demonstrates that RiVidium met the RFP’s requirement to submit two references reflecting its own performance. AR, Tab B.2.p, RiVidium Proposal, Vol.3 at 4. Without more, we see no basis to conclude that, but for the alleged error, the agency could not conduct a confidence assessment using the remaining RiVidium reference and the subcontractor past performance.
In light of the above, we need not address the remaining merits of Spry’s arguments here because Spry has not demonstrated that it was prejudiced by this alleged error in the agency’s analysis. Competitive prejudice is an essential element of a viable protest; where the protester fails to demonstrate that, but for the agency’s actions, it would have had a substantial chance of receiving the award, there is no basis for finding prejudice, and our Office will not sustain the protest. Engility Corp., B-413120.3 et al., Feb. 14, 2017, 2017 CPD ¶ 70 at 17. As noted above, the agency was to evaluate the quality of performance for each recent and relevant past performance reference, then assign each offeror an overall confidence rating. RFP at 61‑62.
Spry does not meaningfully explain how not considering the quality of, or at a minimum giving less weight to, one of RiVidium’s least relevant past performance references from the agency’s confidence assessment would alter the reasonableness of the agency’s assessment of a satisfactory confidence rating or RiVidium’s other evaluated advantages. In this regard, the agency noted that the two subcontractor references it evaluated as relevant had “the greatest impact on the confidence rating,” based on their demonstrated outstanding and very good performance quality. AR, Tab G.2.c, RiVidium PPET Report at 19.
While Spry disagrees with the agency’s evaluation conclusions, Spry does not demonstrate that RiVidium’s remaining past performance references were not recent, relevant, or indicative of high-quality past performance. Further, the SSA found RiVidium’s proposal presented the best value when compared to Spry’s based on its evaluated advantages under the technical/management factor and its significant price advantage, despite being lower rated under the past performance factor. AR, Tab E.2, Castalia SSDD at 8; AR, Tab E.4, Spry SSDD at 10. Accordingly, even if we were to credit Spry’s challenge to the agency’s evaluation of this somewhat relevant past performance effort, Spry has not shown that, but for this error, the protesters would have a substantial chance of receiving the award. Accordingly, we deny this ground of protest.
Price Realism
Spry and Castalia both challenge the agency’s evaluation of price, primarily objecting to the agency’s price realism analysis. Spry Protest at 39‑52; Spry 1st Comments & Supp. Protest at 22‑30; Spry 2nd Comments & Supp. Protest at 23‑29; Castalia Protest at 17‑18; Castalia Comments at 5‑7. The agency generally responds that its evaluation was reasonable and consistent with the solicitation’s evaluation criteria. We have reviewed the protesters’ arguments and the price evaluation record and find no basis to sustain the protest. We discuss the protesters’ principal allegations below.
Where, as here, the award of a fixed-price contract is contemplated, a proposal’s price realism is not ordinarily considered, since a fixed-price contract places the risk and responsibility for contract costs and resulting profit or loss on the contractor. Integrity Mgmt. Servs., Inc., B‑283094.2, May 3, 2000, 2000 CPD ¶ 67 at 4. Nonetheless, a solicitation may provide for a price realism analysis for purposes of measuring an offeror’s understanding of the solicitation requirements or assessing risk. Mortgage Contracting Servs., LLC, B‑418483.2, B‑418483.3, Sept. 10, 2020, 2020 CPD ¶ 340 at 10. The nature and extent of such an analysis are matters within the discretion of the agency, and our review of a realism analysis is limited to determining whether it was reasonable and consistent with the terms of the solicitation. Rust Consulting, Inc., B‑406410, May 18, 2012, 2012 CPD ¶ 173 at 3. Absent a specific provision in a solicitation, agencies are not required to conduct price realism analyses using any particular methodology. NTT DATA Servs. Fed. Gov’t, Inc., B‑417235.2, B‑417235.3, Jan. 2, 2020, 2020 CPD ¶ 19 at 4.
As relevant here, the RFP stated that the agency could “in its sole discretion, decide to conduct [a] price realism analysis on the proposed fully burdened labor rates and/or the overall price proposed.” Id. at 66. The RFP provided that such an analysis would, if conducted, be “applied in order to assess the risk of performance due to unrealistically low prices.” Id.
The record shows that the agency evaluated the proposed fully burdened labor rates for price realism through a two-step statistical analysis. First, the agency identified what it termed “outlier rates,” or proposed fully burdened labor rates that were more than one standard deviation lower than the mean of all the proposed rates for that labor category.[22] AR, Tab G.2.e, RiVidium Price Evaluation Report at 10‑14. Second, the agency compared these outlier rates to the mean of the proposed labor rates for that category to determine whether the proposed rate was more than 20 percent lower than the mean. Id. The agency explained that it considered a proposed rate “a concern” if it was both more than one standard deviation below and more than 20 percent lower than the mean of the proposed labor rates for that labor category. Id. The agency stated it would consider an offeror’s proposal to be unrealistic if more than 20 percent of its 71 proposed labor rates were evaluated as a concern. Id.
The agency determined 20 percent was a realistic variation from the mean based on an analysis using the General Services Administration’s (GSA) Contract-Awarded Labor Category (CALC) tool.[23] Id. at 10. The price evaluation team searched the GSA CALC database using the RFP’s labor categories and identified 10 labor categories that had enough data points for statistical analysis.[24] The agency then calculated the standard deviation for each set of actual rates for the 10 identified labor categories to find how much the rates varied within each labor category. Id. The agency found that the average standard deviation for these labor categories was a 30 percent difference from the mean.[25] Id. The price evaluators, however, considered this 30 percent variation to be too large to reasonably evaluate the realism of proposed labor rates and chose to utilize a more restrictive 20 percent allowable variation instead.
In accordance with the above, the agency reviewed RiVidium’s proposed base year labor rates and found that 19 of its labor rates were more than one standard deviation lower than the mean of the proposed rates for their labor category. Id. at 11‑14. However, the agency found that none of these 19 outliers were more than 20 percent lower than the mean of the proposed rates for their labor category. Id. The agency therefore did not consider any of RiVidium’s rates to be concerning and found its proposed labor rates to be realistic.[26] Id.
The agency also evaluated the realism of RiVidium’s total price by comparing it to the other proposed total prices and to the independent government cost estimate (IGCE). Id. at 20‑21. The agency found that RiVidium’s total proposed price was within one standard deviation of the mean of all of the proposed prices and 13.7 percent lower than the IGCE. Id. Based on RiVidium’s total price being within these thresholds, the agency concluded that the price did not significantly increase the risk of RiVidium being unable to properly staff the contract. Id. at 21. Accordingly, the agency found RiVidium’s price to be realistic. Id. at 21.
Challenge to the Manner of the Price Realism Analysis
Spry contends that the agency’s price realism analysis was unreasonably mechanical and failed to properly consider risks to contract performance. Spry 1st Comments & Supp. Protest at 23‑24. In this regard, Spry argues that the agency improperly failed to consider whether RiVidium could realistically perform is technical solution--specifically by delivering qualified cybersecurity professionals--at the proposed labor rates. Spry 2nd Comments & Supp. Protest at 25. We disagree.
As an initial matter, we see no basis to conclude that the agency was generally required to consider RiVidium’s technical approach as part of its price realism analysis based on the structure of this RFP. Where, as here, a solicitation includes the labor categories and hours that offerors are required to use to prepare their proposals, an agency may generally perform its price realism analysis by determining if the proposed rates are realistic without additional analysis of the offerors technical approach. Trilogy Federal, LLC, B‑418461.11, B‑418461.18, Feb. 23, 2021, 2021 CPD ¶ 144 at 9; Leidos Inc., B‑421476.4, B‑421476.5, July 19, 2023, 2023 CPD ¶ 176 at 6. As noted above, the RFP provided that any price realism analysis would be conducted “on the proposed fully burdened labor rates and/or the overall price proposed . . . in order to assess the risk of performance due to unrealistically low prices.” RFP at 66. Given that the solicitation provided the labor categories and hours for offerors to include in their proposals, and in the absence of a specific solicitation requirement to consider an offeror’s technical approach as part of the realism analysis, we conclude there was nothing improper about the agency generally limiting its realism analysis to consideration of the offerors’ rates and prices.
In addition, we conclude that the price realism methodology adopted by the agency included meaningful consideration of the relevant price risks. As discussed above, the agency developed the 20 percent range for its realism analysis based on a review of the variation in historical rates of the RFP’s labor categories using the GSA CALC tool. AR, Tab G.2.e, RiVidium Price Evaluation at 10. Contrary to the protester’s assertions, the agency rejected the mechanical result of the analysis of the GSA CALC data--that a 30 percent variation from the mean of a labor rate was realistic. Instead, the agency chose to employ a more restrictive 20 percent labor rate variation standard. Id. The price evaluation team then engaged in a further review in deciding to permit up to 20 percent of an offeror’s rates to be below the variation standard before the offeror’s price was found unrealistic. In this regard, the agency reasoned that permitting some low rates would allow for “some rate differences across the small business [o]fferors” while ensuring that the rates did not significantly increase the “risk of staffing ability.” Id. For the evaluation here, the agency found that none of RiVidium’s proposed labor rates were more than 20 percent below the respective mean for that labor category. Id. at 10‑15. Accordingly, the agency found it did not need to further consider the realism of RiVidium’s labor rates. Id. On this record, we see no basis to conclude that the agency failed to consider the risk to contract performance in developing the thresholds for its statistical analysis or in applying them to RiVidium’s proposal.
Challenge to the Relevance of the Historical Labor Rate Data
Castalia separately contends that the agency’s price realism analysis was unreasonable because it was based on a flawed analysis of “irrelevant” labor rate data. Castalia Protest at 17; Castalia Comments at 5‑6. In this regard, Castalia alleges that the agency’s use of the GSA CALC tool failed to properly consider that “the vast majority” of the workforce for this contract requires higher-level security clearances, for which the GSA CALC tool does not differentiate.[27] Castalia Comments at 5‑6. Castalia argues that the agency should have done more to account for security clearances in its creation of its allowable range of labor rate variation, or, at a minimum, used GSA CALC’s relatively limited security clearance filter. Id.
The record shows that the agency utilized the GSA CALC data only to derive a general range of rate variation to be used in the realism analysis. AR, Tab G.2.e, RiVidium Price Evaluation at 10. Nothing in the record indicates that the agency used the GSA CALC data as a basis for direct rate comparison with the offerors’ proposed rates. Rather, the agency used the actual labor rates proposed by offerors in response to the solicitation to calculate the mean against which the 20 percent variation would be applied. Id. While Castalia argues that the GSA CALC data used was insufficient to derive a range of variation for labor categories requiring high‑level security clearances, the protester does not demonstrate that the agency unreasonably relied on this data.[28]
As noted above, absent a specific provision in a solicitation, agencies are not required to conduct price realism analyses using any particular methodology. NTT DATA Servs. Fed. Gov’t, Inc., supra. Nor did the solicitation here commit the agency to use any particular method to evaluate price realism; in fact, it stated that whether to even conduct a price realism assessment was solely within the discretion of the agency. RFP at 66. To the extent the protester generally contends that the agency should have utilized a different methodology to calculate ranges of variation, we find the protester’s disagreement with the agency’s judgement does not provide a basis to sustain the protest.[29]
Failure to Consider Technical/Management Assumptions
Spry also argues that the agency’s price realism evaluation failed to consider the effects of RiVidium’s proposed dashboard solution on its proposed price. Spry 2nd Comments & Supp. Protest at 25‑29. Spry notes that the RFP required offerors to include any relevant assumptions or exceptions in their price proposals and that the price evaluators did not consider information found in other proposal volumes. Id. at 25‑26 (citing RFP at 52‑53). Spry contends that RiVidium’s price proposal failed to disclose “significant assumptions” relating to the implementation of its proposed dashboard solution that RiVidium included in its technical/ management proposal. Id. Spry avers this resulted in an unreasonable price realism analysis because the agency evaluators could not consider the price impacts of RiVidium’s technical/management assumptions. Id. The agency responds that it reasonably considered the assumptions and ground rules in RiVidium’s price proposal and argues that the agency was not otherwise required to consider any technical/management assumptions as part of its price evaluation. Supp. COS/MOL at 13‑15.
As discussed above, we find it reasonable for the agency to have limited its realism analysis to considering offerors’ rates and prices, particularly as the solicitation did not request a unique technical solution and instead provided offerors with the relevant labor hours and categories to be used.
Further, we are unpersuaded by Spry’s arguments that RiVidium was required to include its technical/management assumptions in its price proposal. The RFP required offerors to include in their price proposals “a list of all key ground rules and assumptions that have significant impacts on the proposed price.” RFP at 53. The RFP also instructed offerors that ground rules and assumptions in the price proposals “shall track to other sections of the offer as appropriate” but did not require similar tracking for assumptions originally made in the technical/management proposal. See RFP at 53. Accordingly, we see no basis to conclude that the RFP required RiVidium to include its technical/management assumptions in its price proposal unless it believed that the assumptions would have a significant impact on its proposed price.
Here, RiVidium’s price proposal stated that it “makes no assumptions regarding pricing information and takes no exceptions to the requirements,” but also included two ground rules unrelated to its proposed dashboard that impacted its proposed pricing. AR, Tab B.2.k, RiVidium Price Proposal at 1. Specifically, RiVidium’s ground rules identified the labor categories that various designated leadership positions would be drawn from during performance and explained the offeror’s planned escalation of labor costs.[30] Id. While Spry alleges that the risks of implementing RiVidium’s proposed dashboard could impact price, the protester has not meaningfully explained how, given the fixed-price nature of the contract, any realization of the alleged risks would significantly impact the price paid by the agency.[31] On this record, without more, we find nothing improper about RiVidium’s decision not to include further discussion of its technical/management assumptions in its price proposal. We deny this ground of protest.
In sum, although the protesters disagree with the agency’s price realism analysis and risk assessment, the agency has considerable discretion in determining the nature and extent of its price realism and proposal risk assessments in the context of a fixed‑price contract. See TekSynap Corp., B‑419464.3, B‑419464.4, Jan. 5, 2023, 2023 CPD ¶ 40 at 7‑8 (finding reasonable a price realism analysis of labor rates using a 20 percent variation that was developed based on historical data). We conclude that the protesters have not shown that the agency’s price realism evaluation was inconsistent with its obligations under the solicitation; accordingly, we find these protest grounds to be without merit.
Best‑Value Tradeoff
Finally, the protesters challenge the agency’s best‑value tradeoff and source selection decision. Castalia Comments at 7‑8; Spry 1st Comments & Supp. Protest at 45‑49. The protesters generally contend that the agency’s best-value tradeoff and source selection decision were unreasonable because they were based on the alleged underlying errors discussed above. Castalia Comments at 7‑8; Spry 1st Comments & Supp. Protest at 45‑49. Because we find that these alleged underlying errors do not provide a basis to sustain Spry’s and Castalia’s protests, we similarly deny these protest grounds. See Advanced Alliant Sols. Team, LLC, B‑417334, Apr. 10, 2019, 2019 CPD ¶ 144 at 6.
Castalia also argues that the SSA’s tradeoff decision failed to comply with the terms of the solicitation. Castalia Comments at 7‑8. In this regard, the protester contends that the agency should have considered the advantage offered by Castalia’s superior past performance to be worth its higher price relative to RiVidium. Id.
When making tradeoff decisions in a best-value source selection, selection officials have considerable discretion. Omega Apparel, Inc., B‑411266, June 26, 2015, 2015 CPD ¶ 205 at 6. The propriety of the price/technical tradeoff decision does not turn on the difference in the technical scores or ratings per se, but on whether the selection official’s judgment concerning the significance of the difference was rational and consistent in light of the RFP’s evaluation scheme. Id. The documentation supporting the decision must be sufficient to establish that the SSA was aware of the relative merits and costs of the competing proposals. General Dynamics--Ordnance & Tactical Sys., B‑401658, B‑401658.2, Oct. 26, 2009, 2009 CPD ¶ 217 at 8.
Here, the record reflects that the agency’s evaluation team reviewed and documented the relative merits of the proposals. AR, Tab E.5, SSEB Report; AR, Tab E.1, SSAC Report; see also, e.g., AR, Tab G.1.b, Castalia Technical Evaluation. The record also demonstrates that the SSA independently assessed the proposals and reviewed the SSAC and SSEB reports. See AR, Tab E.2, Castalia SSDD at 12‑13. The SSA’s comparative analysis of the proposals clearly documented the SSA’s consideration of the relative merits of the proposals and documented the aspects of Castalia’s proposal that it found superior to RiVidium’s. Id. at 5‑8. Specifically, the SSA found that Castalia’s proposal presented a “solid advantage” over RiVidium’s under the past performance factor because Castalia “provided more examples of demonstrated high quality performance in more similar efforts than RiVidium.”[32] Id. at 6. The SSA found that none of the identified advantages in Castalia’s or RiVidium’s proposals under the other non-price factors resulted in a comparative advantage for either offeror. Id. at 6‑7.
The SSA noted that the non-price factors, when combined, were significantly more important than the price factor, but stated that this alone did not provide a basis to “infer that [past performance], on its own, is significantly more important” than price. Id. In the best‑value tradeoff analysis, the SSA concluded that the specific advantages arising from Castalia’s past performance did not justify a $17,375,250, or 7 percent, price premium. Id. at 7‑8. On this record, we see nothing objectionable in the SSA’s conclusion that RiVidium’s proposal provided a better value than the slightly higher-rated but more-expensive Castalia proposal.
The protests are denied.
Edda Emmanuelli Perez
General Counsel
[1] The statement of work (SOW) provides as follows:
The scope of the NGA Defender contract is to provide state of the art cybersecurity capabilities and services that perform Cybersecurity Risk Management, Automation and Optimization, Computer Network Exploitation[], Cross Domain Governance, Cyber-Supply Chain Risk Management[], Cybersecurity Integration, and Cybersecurity Software Assurance to secure and deliver the nation’s most trusted geospatial-intelligence[] services and data.
Agency Report (AR), Tab A.6.b, SOW at 5.
[2] The agency amended the RFP eight times. Unless otherwise noted, citations to the RFP in this decision are to the conformed version issued as part of amendment 0007. Citations to the agency report in this protest are to the uniform record created by the agency in response to both protests.
[3] As relevant here, the RFP defined an outstanding rating as “[p]roposal indicates an exceptional approach and understanding of the requirements and contains multiple strengths, and risk of unsuccessful performance is low.” RFP at 57. The RFP defined a good rating as “[p]roposal indicates a thorough approach and understanding of the requirements and contains at least one strength, and risk of unsuccessful performance is low to moderate.” Id.
[4] The RFP defined low technical risk as:
Proposal may contain weakness(es) which have little potential to cause disruption of schedule, increased cost or degradation of performance. Normal contractor effort and normal Government monitoring will likely be able to overcome any difficulties.
Id. at 58.
Further, the RFP defined moderate technical risk as:
Proposal contains a significant weakness or combination of weaknesses which may potentially cause disruption of schedule, increased cost or degradation of performance. Special contractor emphasis and close Government monitoring will likely be able to overcome difficulties.
Id.
[5] The RFP specified that the agency could assess significant strengths, moderate strengths, slight strengths, slight weaknesses, moderate weaknesses, significant weaknesses, or deficiencies. Id. at 58.
[6] NIST SP 800-37 R2 provides guidelines for applying risk management framework to information systems and organizations. NIST SP 800-37 Rev. 2, Computer Security Resource Center, https://csrc.nist.gov/pubs/sp/800/37/r2/final (last visited April 5, 2024).
[7] Spry initially argued that the agency failed to properly consider an impaired objectivity organizational conflict of interest, unreasonably evaluated Spry’s proposed approach to minimize the risk of detection during red team engagements, unreasonably evaluated RiVidium’s response to the computer network exploitation scenario, and unreasonably evaluated RiVidium’s proposed key personnel. Spry Protest at 29‑39, 58‑59, 61‑62, 76‑82. The protester later withdrew these allegations. Spry Comments & Supp. Protest at 4, 31.
The protesters also abandoned several arguments. For example, Castalia’s initial protest included several grounds challenging the agency’s evaluation of Castalia’s proposal, the agency’s conduct of discussions, and the agency’s affirmative responsibility determination. Castalia Protest at 9‑12, 14‑17, 20. The agency provided a detailed response to these protest allegations. COS/MOL, B-421640.4 at 28‑43, 57‑60, 63‑65. In response, Castalia did not rebut or substantively address the agency’s arguments; rather, the protester briefly complains about the sufficiency of the record and, without further discussion states that it “relies [on] the arguments made in its [p]rotest.” See Castalia Comments at 8. Accordingly, we dismiss these protest grounds on which Castalia did not comment as abandoned. See Tec-Masters, Inc., B‑416235, July 12, 2018, 2018 CPD ¶ 241 at 6.
[8] RiVidium’s proposal described Orion as “a dashboard/web portal developed by TripleCyber that provides near real time information to decision makers.” AR, Tab B.2.o, RiVidium Proposal, Vol. 2 at 1‑2. The proposal stated that Orion is “a web-based application hosted in either a cloud or on‑premise . . . designed to aggregate disparate information from separate NGA applications/data sources.” Id. at 26.
[9] For example, Spry argues that the agency unreasonably failed to evaluate, among other risks, the impact of other software RiVidium intends to utilize in conjunction with its Orion software, the assessments or approvals that would allegedly be required to deploy new software on NGA’s systems, and the feasibility of creating connections between the sources of data proposed by RiVidium and its proposed dashboard. See Spry 1st Comments & Supp. Protest at 9‑13.
[10] The agency notes that neither Spry’s nor RiVidium’s proposals included “a specific implementation plan,” without which the agency would be unable to properly consider the effects of the implementation risks on any given proposal. Supp. COS/MOL, B‑421640.5 at 5‑6. The agency argues that any such implementation risks of timely providing a functional dashboard could be applicable to “any potential awardee.” Id. at 6. The agency explains that this includes Spry, which, as an incumbent, proposed to provide a dashboard that developed upon a currently implemented platform but did “not detail the dependencies and requirements to implement this development.” Id. at 5.
[11] Spry, for the first time in its second supplemental protest, argues that the agency broadly failed to evaluate risk throughout its evaluation of the technical/management factor instead of in specific instances such as in evaluating the above dashboard solution. Spry 2nd Comments & Supp. Protest at 3‑12. Our Bid Protest Regulations, 4 C.F.R. § 21.2(a)(2), require protests to be filed no later than 10 days of when a protester knows, or should have known, of a basis for protest. Further, our regulations do not contemplate the piecemeal presentation of protest arguments. See Vigor Shipyards, Inc., B‑409635, June 5, 2014, 2014 CPD ¶ 170 at 5. While Spry argues that it did not know the full extent of the agency’s limited consideration of risk until the agency filed its supplemental COS/MOL on February 27, our decisions have repeatedly concluded that a protester need not await perfect knowledge before filing a protest. See, e.g., Valkyrie Enters., LLC, B‑414516, June 30, 2017, 2017 CPD ¶ 212 at 3‑4 n.2. Here, Spry’s basis for this argument arises from the documents constituting the agency’s evaluation under the technical/management factor. These documents were produced to Spry on February 9 as part of the original agency report in the B-421640.3 protest, more than 10 days before Spry filed its second supplemental protest on March 1. We therefore dismiss this contention as untimely.
[12] Spry also argues that a slight strength assessed to RiVidium’s proposal under the cybersecurity risk subfactor is unreasonable because it is based, in part, on RiVidium’s proposed interactive dashboard. Spry Protest at 74‑75; Spry Comments & Supp. Protest at 18‑19. Spry maintains that the implementation risks discussed above are likely to cause delays in RiVidium’s ability to deploy the dashboard. Spry Comments & Supp. Protest at 18‑19. Like above, we see nothing in the evaluation criteria of the cybersecurity risk subfactor that required the agency to evaluate the implementation risks of proposed new software. Accordingly, without more, we see no basis to object to the agency’s conclusion that aspects of RiVidium’s proposed interactive dashboard are part of an advantageous approach “to accelerate the [risk management framework] process.” See AR, Tab G.2.b, RiVidium Technical Evaluation at 10.
[13] The RFP required the agency to “develop narratives on the strengths, weaknesses[,] and deficiencies of the proposal.” RFP at 57. The agency did not assess any weaknesses or deficiencies to Spry’s proposal.
[14] The protester explains that [DELETED] is “a [DELETED].” Protest at 57.
[15] In this regard, the protester complains that the agency fails to specifically address specific attributes of Spry’s CTI methodology including [DELETED]. Spry Comments & Supp. Protest at 32.
[16] As noted above, the RFP required that the agency assess the offerors’ technical approaches to execute the risk management framework process identified in NIST SP 800-37 R2. The SOW explained “NGA’s specific implementation of the [risk management framework] . . . will serve as the baseline upon which processes and procedures are improved or changed.” AR, Tab A.6.b, SOW at 16. The SOW specified that the risk management framework process had seven steps. Id.
[17] Spry also argues that the agency failed to recognize the value of its experience as the incumbent under the contract dashboard/web portal subfactor. Protest at 85. However, our Office has consistently stated that there is no requirement that an incumbent be given extra credit for its status as an incumbent, or that an agency assign or reserve the highest rating for the incumbent offeror. Integral Consulting Servs., Inc., B‑415292.2, B‑415292.3, May 7, 2018, 2018 CPD ¶ 170 at 7. Where, as here, the RFP did not require the agency to consider incumbency or experience under the technical management factor and its subfactors, we see no basis to object to the agency’s choice to focus instead on the protester’s approach.
[18] The sections of Volume 2 of Spry’s proposal were separately paginated. Accordingly, citations to this document are to the electronic page number of the Adobe PDF.
[19] We are unpersuaded by the protester’s argument that it did not know the bases for these specific protest grounds until it received RiVidium’s past performance proposal on February 26. Castalia Resp. to Req. for Partial Dismissal at 1. Our decisions have repeatedly concluded that a protester need not await perfect knowledge before filing a protest. See, e.g., Valkyrie Enters., LLC, supra. While the protester complains that it was only “able to actually understand the flaws in the [a]gency’s evaluation” with RiVidium’s proposal, Castalia Resp. to Req. for Partial Dismissal at 1, it does not identify any fact from the proposal relevant to its protest ground that was not also included in the earlier‑produced PPET Report. Compare AR, Tab B.2.p, RiVidium Proposal, Vol. 3, with AR, Tab G.2.c, RiVidium PPET Report.
[20] The RFP stated that the agency would not consider the quality of past performance contracts determined to be not relevant. Id. at 61.
[21] With regard to the evaluation of past performance information under FAR part 15, the FAR requires that agencies consider “[t]he currency and relevance of the information, source of the information, context of the data, and general trends in contractor’s performance.” FAR 15.305(a)(2)(i).
[22] Standard deviation is a statistical analysis that measures the dispersion of a dataset relative to its mean. COS/MOL, B-421640.3 at 34 (citing Spry Protest at 44). According to the Department of Defense’s (DOD) Contract Pricing Reference Guides, the standard deviation is a measure of dispersion of the samples or observations using the square root of the variance (with the variance of a sample being the average of the squared deviations between each observation and the mean). In a distribution that is approximately normal, plus or minus one deviation will include approximately 68 percent of the total observations in the sample; plus or minus two deviations will include approximately 95 percent of the total observations; and plus or minus three deviations will include approximately 99.7 percent of the total observations. However, of particular importance here, because all values are squared, a single observation that is far away from the mean can substantially affect both the variance and the standard deviation. See Multimax, Inc. et al., B‑298249.6 et al., Oct. 24, 2006, 2006 CPD ¶ 165 at 8 n.2 (citing DOD Contract Pricing Reference Guides, vol. 2, ch. 3.3).
[23] The GSA CALC tool is a database that “helps federal contracting officers and others find awarded prices to use in negotiations for labor contracts,” and provides data on “ceiling prices, fully burdened costs, services data, and worldwide rates.” Digital Tools, https://www.gsa.gov/buy-through-us/products-and-services/professional-s… (last visited April 5, 2024).
[24] The agency did not utilize the data from labor categories that had two or fewer data points in the GSA CALC database. Id.
[25] The agency calculated the percentage difference of each labor category’s standard deviations as follows:
Program Manager (11 [percent]), [Information Technology (IT)] Program Manager (26 [percent]), Data Analyst (37 [percent]), Software Developer (35 [percent]), Database Administrator (31 [percent]), Network Operations Specialist (18 [percent]), Sr. Enterprise Architect (14 [percent]), Systems Security Analyst (33 [percent]), Information Systems Security Manager (37 [percent]), IT Program Auditor (26 [percent]), and Knowledge Manager (39 [percent]).
Id.
[26] Based on a review of inflation and increases in the cost of labor over the last ten years, the agency determined that annual escalations in labor rates between two and three percent were acceptable to account for increased labor costs over the contracts 8‑year period of performance. Id. at 19. RiVidium did not propose to escalate its labor rates in the first option year and proposed to escalate its rates by three percent in option years two through eight. Id. Accordingly, the agency conducted the two-step statistical analysis it had performed on the base year labor rates to RiVidium’s labor rates for the first option year. Id. at 15‑18. The agency found that only one of RiVidium’s proposed option year one fully burdened labor rates was more than one standard deviation and 20 percent lower than the mean of the proposed option year one rates. Id. The agency therefore found RiVidium’s proposed labor rates for the first option year to be realistic.
[27] The GSA CALC tool allows users to filter labor rate data for a given labor category by whether or not the labor category required a security clearance on that contract but does not allow users to differentiate between the levels of security clearance. See Labor Category Ceiling Rates, https://buy.gsa.gov/pricing/qr/mas (last visited April 5, 2024).
[28] Although Castalia asserts that for two of the labor categories (enterprise architect and network analyst) limiting a search in GSA CALC to labor rates requiring security clearances would result in a very low variation in prices, the protester does not demonstrate that the agency would have reasonably used this data as part of its realism analysis. Id. As noted above, the price evaluation team declined to use GSA CALC data from labor categories that had two or fewer data points. AR, Tab G.2.e, RiVidium Price Evaluation at 10. Castalia’s allegations did not include evidence to substantiate that these low variation labor categories were not a result of limited data sets, and our review of the GSA CALC tool under the search settings proffered by Castalia returned only two rates for each of the identified labor categories. See Labor Category Ceiling Rates, https://buy.gsa.gov/pricing/qr/mas (last visited April 5, 2024).
[29] Castalia also argues that the agency’s price realism analysis was unreasonable because the standard by which the agency would find an offeror’s labor rates unrealistic--that 20 percent or more of its proposed labor rates were evaluated as a concern--is arbitrary and lacks a rational basis. Castalia Comments at 6‑7. We find that Castalia cannot demonstrate it was prejudiced by this alleged error. See Engility Corp., supra. Here, the agency did not evaluate any of RiVidium’s proposed labor rates to be a concern. AR, Tab G.2.e, RiVidium Price Evaluation at 10‑15. Accordingly, Castalia has not shown that using some other threshold for the number of permissible “concerning” rates would have changed either the results of the agency’s price realism analysis or the ultimate source selection decision.
[30] The agency considered RiVidium’s pricing ground rules as part of its price evaluation. AR, Tab G.2.e, RiVidium Price Evaluation at 9‑10. This included consideration, as part of the realism analysis, of the impact of RiVidium’s proposed labor escalation rates. Id. at 10, 19.
[31] Notably, Spry acknowledges that, even if the alleged implementation risks did result in delays to RiVidium’s proposed automated dashboard, RiVidium would be able to manually satisfy the dashboard requirement until those implementation issues were resolved. Spry 2nd Comments & Supp. Protest at 8. While Spry does argue that the manual creation of the weekly dashboard could result in “significantly fewer hours than planned working on other SOW requirements,” the protester does not meaningfully explain how this would impact price. Spry 1st Comments & Supp. Protest at 30.
[32] Castalia argues that the SSA’s analysis was flawed because it concluded there was minimal differentiation between Castalia’s and RiVidium’s past performance. Castalia Supp. Briefing at 4‑5. However, the record shows that the SSA only considered there to be minimal differentiation when “comparing the level of quality” between the two offerors’ past performance. AR, Tab E.2, Castalia SSDD at 7. The SSA explained that there was differentiation between the relevancy of Castalia and RiVidium’s past performance and specifically noted that there was a benefit to Castalia’s experience with other NGA contracts compared to RiVidium’s lack of experience with the agency. Id.