Accenture Federal Services, LLC

Decision

Matter of:  Accenture Federal Services, LLC

File:  B-417857

Date:  November 22, 2019

DIGEST

Protest challenging an agency’s evaluation of the protester’s quotation is denied where the protester has not shown that the evaluation conclusions were unreasonable, inconsistent with the solicitation, or applied unstated technical requirements.

Accenture Federal Services, LLC (Accenture), of Arlington, Virginia, protests its exclusion from consideration for award of a task order under request for quotations (RFQ) No. 70SBUR19Q00000066, which was issued by the Department of Homeland Security, United States Citizenship and Immigration Services (USCIS) for integrated data analytics, artificial intelligence/machine learning, and development, security, and operations services. The protester argues that USCIS unreasonably concluded Accenture’s quotation was technically unacceptable based on an evaluation that applied unstated technical requirements.

We deny the protest.

BACKGROUND

USCIS issued the RFQ on May 13, 2019, under the Federal Supply Schedule (FSS) procedures of Federal Acquisition Regulation (FAR) subpart 8.4, seeking quotations for integrated data analytics, artificial intelligence/machine learning, and development, security, and operations services.  Agency Report (AR), Exh. 1, RFQ at 1, 38.[1]  The RFQ anticipated the award of a single task order with a base period of 1 year, and two 1-year options.  Id. at 2.  The RFQ was limited to vendors holding a contract under the General Services Administration Schedule 70 special item number 132-51.  Id. at 1. 

Vendors were to submit quotations in two parts:  a coding submission, completed by current employees of the prime and/or subcontractor; as well as price and other information.  With respect to the first part, the RFQ presented a problem statement, to which quotations were required to respond with a coding submission to solve the problem.  Id. at 27-30.  The solution was required to include a single script that would provide a link to launch an interactive notebook and launch at least three environments (development, end-to-end testing, and production).  Id. at 28-29.  The interactive notebook and environments were to run within Amazon Web Services (AWS).  Id.  As relevant to this protest, the RFQ neither required nor prohibited vendors from using the AWS service “SageMaker” as part of their solution.  See Id.  The RFQ also required the coding submission to include a plain‑text file entitled “ReadMe.md” which was to include “[a]ll instructions for deploying the solution in an AWS account.”  Id. at 29. 

The RFQ provided for a two-step evaluation process.  RFQ at 32-33.  In step one, USCIS was to evaluate the coding submission, and review whether prices were fair and reasonable.  Id. at 33.  Evaluations of the coding submissions were to consider the extent to which the solution and its supporting documentation adhered to the required instructions.  Id.  The agency was to assign strengths, weaknesses, significant weaknesses, and deficiencies to elements of the coding submission that exceeded requirements of the RFQ or increased the risk of unsuccessful contract performance.  Id. at 37.  Each coding submission would be rated outstanding, good, acceptable, or unacceptable.  Id. at 36-37.  Quotations from vendors whose coding submissions were found to be the highest technically rated with a fair and reasonable price would be considered to proceed to step two of the evaluation.  Id. at 33.  The agency intended to select three vendors to proceed to step two.  Id. 

In step two, the agency was to evaluate how well the selected vendors performed in technical demonstrations conducted by the agency.  Id. at 33,35.  Award was to be made to the vendor whose quotation offered the best value to the agency when considering the technical demonstration and price.  Id. at 33. 

The agency received 13 timely quotations, including Accenture’s.  AR, Exh. 5, Down Select Document, at 6.  Based on the results of the step one evaluation, the agency concluded the three most highly rated quotations all had proposed fair and reasonable pricing, and therefore selected these three to advance to step two of the competition.  Id. at 9.  Accenture was not selected to move on to step two because its coding submission received an unacceptable rating.  Id.  On August 5, the agency provided Accenture an explanation of its evaluation.  AR, Exh. 6, Notice of Non-Selection at 1.  On August 15, Accenture filed this protest. 

DISCUSSION

Accenture challenges the agency’s evaluation of its quotation.  The protester argues that the evaluation relied on unstated technical requirements to find Accenture’s quotation unacceptable, namely, that there were limits on the availability of the AWS service “SageMaker” in the USCIS AWS account’s us-east-1 region.  Based on our review of the record, we find no basis to sustain the protest.[2] 

Where, as here, an agency issues an RFQ to FSS vendors under FAR subpart 8.4 and conducts a competition for the issuance of an order, we will review the record to ensure that the agency’s evaluation was reasonable and consistent with the terms of the solicitation and applicable procurement laws and regulations.  Carahsoft Tech. Corp., B‑401169; B‑401169.2, June 29, 2009, 2009 CPD ¶ 134 at 3.  It is a vendor’s responsibility to submit a well-written quotation, with adequately detailed information which clearly demonstrates compliance with the solicitation requirements; the vendor runs the risk that the agency will unfavorably evaluate its proposal where it fails to do so.  See The Concourse Group, LLC, B-411962.5, Jan. 6, 2017, 2017 CPD ¶ 36 at 7.  A protester’s disagreement with the agency’s judgment does not establish that an evaluation was unreasonable.  Id.   

To provide our Office some understanding of the functionality of AWS, the agency pleadings pointed to some of Amazon’s relevant public-facing documentation.  See Contracting Officer’s Statement/Memorandum of Law (COS/MOL) at 5, n.8.  The AWS cloud computing platform relevant here is hosted in various locations worldwide.  See Regions and Availability Zones, AWS Documentation, https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-avail… (last visited Nov. 5, 2019).  These AWS hosting locations are organized into regions represented by a region code; for example, “us-east-1”.  Id.  Each region is a separate geographic area that contains multiple locations known as availability zones.  Availability zones are named within an AWS account by a region code followed by a letter; for example, “us‑east-1a”.  Id.  Of particular relevance here, Amazon independently names availability zone locations within each AWS account, but for the purpose of resource distribution, may assign similarly named availability zones in different accounts to different locations on the cloud computing platform.  Id.  This means two separate AWS accounts could be operating in what each account calls “us‑east-1a”, but not be at the same location in the cloud computing platform.  Id.  However, separate from the names assigned to availability zones within an AWS account, each availability zone location can be uniquely identified by an availability zone ID (AZ ID); for example, “use1-az1”.  Id.  To coordinate specific locations on the AWS cloud computing platform between different AWS accounts, users must use AZ IDs instead of region or availability zone codes.  Id.  The RFQ did not require that vendors use a specific availability zone or AZ ID. 

Accenture’s coding submission solution chose to use the AWS SageMaker service, specifically, the Sagemaker ml.p3.2xlarge instance. [3]  See AR, Exh. 3, Accenture Quotation at 11.  Accenture’s coding submission instructions specified that the solution it was submitting required two such SageMaker ml.p3.2xlarge instance slots to be open within the AWS us‑east‑1 region.  Id.  The instructions did not specify the use of a specific availability zone or AZ ID within the AWS us-east-1 region.  See Id.  During the evaluation, the agency followed Accenture’s instructions and opened two SageMaker ml.p3.2xlarge instance slots within the AWS us-east-1 region, specifically, in availability zones us-east-1f and us-east-1b.  AR, Exh. 11, Technical Evaluator’s Declaration at ¶ 3.  The evaluators then attempted to launch Accenture’s solution.  COS/MOL at 6.  The Accenture solution launch script was coded to operate in AWS availability zone us‑east‑1d.  AR, Exh. 11, Technical Evaluator’s Declaration at ¶ 11.  The Accenture solution launch script did not identify an AZ ID.  Id.  While the evaluators were attempting to launch the required production environment in USCIS’s account, the script stopped and an error message indicated that the SageMaker ml.p3.2xlarge instance was not available in the availability zone us‑east‑1d.  COS/MOL at 6. 

The agency concluded that Accenture’s coding submission was unacceptable because it did not address the problem statement in accordance with the parameters set forth in the solicitation.  AR, Exh. 4, Technical Evaluation at 15.  In reaching this conclusion, the evaluators found that Accenture’s coding submission “failed to launch the three environments and provide a link to launch the notebook,” and “provided incorrect steps in the README.md file to deploying the solution in AWS as the proper availability zone and instance type for [S]age[M]aker did not match up.”  Id. at 15, 18.

After receiving notice that its quotation was found technically unacceptable, Accenture confirmed through testing that its solution could fail in the manner described by the agency outside of Accenture’s own AWS account.  Protest at 11.  Accenture also learned through correspondence with AWS that there are limitations on the availability of the SageMaker ml.p3.2xlarge instance within the AWS us-east-1 region at the availability zone level.  Id. 

Accenture argues that Amazon’s undisclosed limitations on the availability of the SageMaker ml.p3.2xlarge instance in the USCIS AWS account represent an unstated technical requirement.  Protest at 15-16; Comments at 3-7.  Accenture contends that because there was no public documentation of the SageMaker ml.p3.2xlarge instance limitation at the availability zone level, no vendor could have reasonably known to ensure it was utilizing the SageMaker ml.p3.2xlarge instance using a specific AZ ID.  Comments at 7.  Accenture contends that once the agency became aware of the AWS limitations on the SageMaker ml.p3.2xlarge instance, it was required to amend the RFQ and allow vendors to submit new coding submissions.  Comments at 7. 

The agency responds that this limitation is not an unstated technical requirement because the RFQ did not require the use of the SageMaker ml.p3.2xlarge instance.  COS/MOL at 8-9.  The agency further contends that given Accenture’s decision to use the SageMaker ml.p3.2xlarge instance, it was the protester’s coding choices, not the limitations of a particular availability zone, that caused its solution to fail and rendered its coding submission unacceptable.  Id. at 9.  Specifically, the agency argues Accenture’s solution failed because it specified its operating location by availability zone name, not AZ ID, and the USCIS AWS account mapped the availability zone name to a location where the SageMaker ml.p3.2xlarge instance was not available.  See COS/MOL at 8-9; see also AR, Exh. 11, Technical Evaluator’s Declaration at ¶ 22.

Based on the record before us, we find no basis to object to the agency’s evaluation.  In this regard, the record shows that the evaluation was not based on any unstated technical requirement and further, the flaws and oversights in Accenture’s coding submission support the agency’s conclusion that the quotation was unacceptable. 

While we agree with the protester that the limited availability of the SageMaker ml.p3.2xlarge instance was not specified in the solicitation’s requirements, we find the agency’s evaluation reasonable because ensuring a vendor’s utilized instances are available within the availability zone in which its solution attempts to launch is reasonably related to, and encompassed by, the RFQ’s stated requirements.[4]  The RFQ required that the solution provide a single script that launches the three environments and provide a link to launch the interactive notebook.  RFQ at 34.  The RFQ did not require the use of the AWS SageMaker service, much less the use of the SageMaker ml.p3.2xlarge instance.[5]  The RFQ specified that the coding submission would be tested in a USCIS AWS account.  Id. at 34, 57.  As noted above, public-facing AWS documentation states that in order to coordinate specific locations on the AWS cloud computing platform between different AWS accounts, users must use AZ IDs instead of simply region or availability zone codes.  Regions and Availability Zones, AWS Documentation, supra. 

The record shows the agency found Accenture’s coding submission technically unacceptable, in part, because Accenture’s solution failed to generate a link to the interactive notebook and launch the three required environments, not specifically because it attempted to open the SageMaker ml.p3.2xlarge instance in a location where it was unavailable.[6]  See AR, Exh. 4, Technical Evaluation at 15.  The agency’s unacceptable rating due to the coding submission’s failure to launch does not, as the protester argues, amount to a reliance on an unstated technical requirement.  Quite simply, the protester’s coding submission failed to provide an effective solution to the problem statement; in other words, the protester failed to submit a well‑written quotation that complied with the solicitation’s requirements.  Accenture’s attempt to blame its unacceptable rating on AWS’s documentation instead of its own coding choices to not use AZ IDs or otherwise plan for an unavailable instance is nothing more than disagreement with the agency’s judgment and does not establish that the evaluation was unreasonable.  The Concourse Group, LLC, supra.  Accenture’s protest regarding this matter is without merit. 

We deny the protest.

Thomas H. Armstrong
General Counsel