Armorworks Enterprises, LLC

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Armorworks Enterprises, LLC

File: B-400394.3

Date: March 31, 2009

DIGEST

Protest challenging agency's body armor testing, which is based on findings contained in a Department of Defense Inspector General Report, is dismissed where the findings contained in the report do not reasonably support a conclusion that the protester suffered any prejudice in the testing of its body armor, which ultimately failed testing.

Armorworks Enterprises, LLC protests the exclusion of its proposal from the competitive range under request for proposals (RFP) No. W91CRB-07-R-0041, issued by the Army Materiel Command (AMC) for body armor. The protester challenges the testing methodology and procedures used by the agency in its pre-award body armor testing, and also alleges that post-award changes to body armor testing requirements reflect unequal treatment of offerors.

We dismiss the protest.

This is Armorworks' third protest challenging the Army's body armor test results in connection with the subject solicitation. Armorworks' initial protests challenged various aspects of the Army's test procedures and the test results in which each body armor model proposed by Armorworks suffered a "catastrophic" testing failure. Our Office dismissed Armorworks' initial protests in part and denied them in part. See Armorworks Enters., LLC, B-400394, B-400394.2, Sept. 23, 2008, 2008 CPD para. 176.[1] While styling its current protest as a "supplemental protest rather than a new protest," Armorworks asserts that its current protest is based on new and independent grounds, which first came to light as a consequence of a January 29, 2009 report of the Department of Defense Inspector General, Report No. D-2009-047 (Project No. D208-D000JA-0263.000) (IG Report). Protest at 2.

In summarizing its grounds of protest, Armorworks states as follows:

• 1. 1. The IG report, and Army responses thereto, provide substantial and material new information, unavailable when ArmorWorks' initial protest was filed, supporting ArmorWorks' position that the [Preliminary Design Model (PDM)] testing pursuant to which ArmorWorks was excluded from the competition was fundamentally flawed such that the ballistic test results were arbitrary and inconsistent with the testing protocols required by the terms of the Solicitation.
  2. As a result of the acknowledged material flaws in its ballistic testing program, the Army has determined that the previously performed PDM tests will be repeated as first article tests for all products of all awardees -- but in accordance with a new and revised testing protocol. The new and revised testing protocol will be different from that which was used to disqualify ArmorWorks, i.e., the criteria for award of this contract have been changed retroactively because the previous criteria were deficient and were applied in an inconsistent manner.

Id. at 6.

The Army requested summary dismissal, arguing that the protest allegations do not support a valid basis of protest since the findings contained in the IG Report, which forms the basis of Armorworks' protest, do not establish a reasonable possibility that Armorworks was prejudiced by the agency actions it now alleges. Agency Legal Memorandum at 3. We agree.

Prejudice is an essential element of any protest and our Office will not sustain a protest unless the protester demonstrates a reasonable possibility that it was prejudiced by the agency's actions, that is, unless the protester demonstrates that, but for the agency's actions, it would have had a substantial chance of receiving the award. Armorworks Enters., LLC, supra, at 8 n.7; McDonald'Bradley, B'270126, Feb. 8, 1996, 96'1 CPD para. 54 at 3; see Statistica, Inc., v. Christopher, 102 F.3d 1577, 1581 (Fed. Cir. 1996).

Here, it is important to note from the outset that the IG Report, upon which Armorworks bases its protest, does not actually concern the PDM testing conducted by the Army under the subject solicitation--the particular testing which resulted in Armorworks' elimination from the competition because all of its PDMs suffered catastrophic testing failures. Rather, the IG Report addresses first article testing of body armor conducted in 2005 and 2007 in connection with a separate Army contract, No. W91CRB-04-D-0040 (Contract 0040). Moreover, the substantive findings regarding problems with the first article testing identified in the IG Report concern particular instances where, according to the IG Report, body armor which passed testing instead should have been recorded as failing testing. Given that the entire thrust of the report concerns body armor which passed testing, the report by its terms is of limited relevance to Armorworks' protest here, since all of the body armor models submitted by Armorworks in fact failed testing under the subject solicitation.

Notwithstanding these fundamental differences, Armorworks asserts that the findings in the IG Report regarding the Army's testing are relevant to the PDM testing under the subject solicitation because "the first article testing protocol is identical to the PDM testing protocol for ballistic testing." Protest at 2. From this premise, which, as discussed below, is not factually accurate, Armorworks contends that the three specific testing flaws identified in the IG Report with respect to the Army's testing under Contract 0040 can be attributed to the testing process employed by the Army with respect to PDM testing under the current solicitation.

The IG Report identifies several specific problems with the first article testing for Contract 0040. In this regard, the report concludes that "testing facility officials did not consistently follow the test plan or [Contract Purchase Description] COPD requirements for the fair shot determination, measurement of [Back Face Deformation] BFD, or plate size." IG Report at 7. Armorworks contends that based on these findings, the Army's PDM testing under the solicitation here likewise was flawed in these three areas: measurement of BFD; plate size; and fair shot determination. Protest at 9-10. As discussed below, we conclude that the findings in the IG Report do not support a conclusion that Armorworks was prejudiced with respect to the PDM testing at issue here.

BFD Measurement

As explained in our prior decision, measuring BFD involves measuring the indentation on the clay backing material on which body armor is placed during ballistic testing. Like the PDM body armor testing, the first article testing at issue in the IG Report was designed to determine the body armor's resistance to penetration from various ballistic threats. For each threat, the body armor must defeat the specified number of impacts within established parameters. The parameters identify acceptable numbers of complete and partial penetrations and the maximum depth of the BFD for partial penetrations. For the purpose of addressing this protest, it is sufficient to note that the testing parameters for the first article testing and the PDM testing were largely the same. For example, as it relates to BFD, both defined a catastrophic failure as a complete penetration of hard and soft armor on the first shot, or a BFD of greater than or equal to 48 millimeters on the first shot.

The testing parameters differed materially, however, with respect to how BFD would be measured. The IG Report explains that first article testing required testing officials to measure BFD at the deepest point in the clay depression after the bullet impacted the plate. IG Report at 11. During testing, however, contractors complained that the BFD measurement was not fair if the deepest point in the clay was not behind the point of impact. Based on this concern, the Army used an "offset correction technique (a mathematical formula used to adjust BFD)" to measure BFD if the deepest point (depth) in the clay depression was not behind the bullet's point of impact. Id. at 11.

The IG Report faulted the Army's use of this "offset correction technique" as a methodology for measuring the point of deepest impact. According to the IG Report, use of this methodology constituted a deviation from the testing requirements. Id. at 17. In addition, while the Army argued that the technique it used is common industry standard, the IG Report disputed this assertion. Id. According to the IG Report, had the Army not deviated from the testing standards by using the "offset correction technique," body armor identified as passing in fact would have failed testing. Id. at 11.

The findings in the IG Report regarding the Army's methodology for measuring BFD in the context of its first article testing, however, have no applicability to the PDM testing at issue since the methodology established in the solicitation for measuring BFD of PDMs was entirely different from the methodology used in the first article testing at issue in the IG Report. As explained in our prior decision, the solicitation here provided for measuring BFD at the "point of aim" as opposed to the deepest point (depth) within the post-impact clay depression. Armorworks Enters., LLC, supra, at 4-6.[2] Since the relevant BFD measurement for PDM testing did not concern measuring the point of deepest impact, the findings in the IG Report regarding the Army's alleged use of an improper methodology for measuring the point of deepest impact are not relevant to the PDM testing at issue here.

Plate Size

Regarding plate size, the IG report identified several instances where testing officials did not use the correct size body armor plates as required by the first article test plan. Id. at 11. Thus, the concerns in the IG Report simply reflect specific instances where testing officials failed to adhere to established protocols during first article testing, and thus are not relevant to the PDM testing at issue here. See Vistron, Inc., B-277497, Oct. 17, 1997, 97-2 CPD para. 107 at 3 n.1. Moreover, notwithstanding the fact that Armorworks received detailed information concerning the testing of its body armor, including the size of each plate which was scored for testing, Armorworks has not in this protest, or in its prior protests, identified any instance where the Army failed to use the correct size plates in testing its body armor. As a consequence, Armorworks has not established prejudice in connection with the findings contained in the IG Report regarding plate size.

Fair Shot Determination

Armorworks argues that the concerns identified in the IG Report regarding the Army's fair shot determination during the first article testing under contract 0040 cast doubt on the validity of the PDM testing. Specifically, the IG Report raises concerns regarding the Army's treatment of "over-velocity" shots--where a bullet strikes a plate at a velocity in excess of the velocity specified for testing purposes. According to the IG Report, for over-velocity shots, the first article testing COPD stated as follows:

if the shot does not result in a complete penetration, the shot should be considered fair and the test should proceed; but

if the shot results in a complete penetration, the shot should not be considered fair, and the plate should be discarded.

IG Report at 10. This treatment of over-velocity shots "is based on the premise that if the plate can withstand a higher velocity, it should withstand the required velocity."[3] Id.

The IG Report concludes that the Army failed to follow the fair shot acceptance criteria by retesting plates in certain instances involving over-velocity shots. The IG's conclusions relate to tests where one of the shots was over-velocity, the over-velocity shot nonetheless constituted a fair hit under the testing criteria, but the plate ultimately failed testing as a result of subsequent shots that were not over-velocity. At that point, according to the IG, the plate should have been regarded as having failed the testing; instead, the Army (improperly, in the IG's view) retested the plate. In response to the IG Report, the DOD Director for Operational Test and Evaluation (DOT&E), which reviewed the Army's testing, explained that where there is an over-velocity shot and the plate fails, retesting is appropriate since a "plate subjected to an over-velocity shot experiences stresses and strains beyond what it was designed to experience, whether or not the plate initially successfully defeats the threat. A subsequent shot on that plate that results in a failure may be the result of a generally weakened plate as a result of the first over-velocity shot, and not necessarily the result solely of the second shot." Id. at 52.

Armorworks argues that it may have been prejudiced as a result of the Army's failure to follow the over-velocity shot procedures as explained in the IG Report. According to Armorworks, if an over-velocity shot on one of its plates was a fair hit, but the Army ultimately discarded the plate and re-tested a different Armorworks plate, "it is possible that the second test plate would have a complete penetration, resulting in a 'failure' score that otherwise would have been a 'pass' if testing had continued on the first plate." Protester's Response to Request for Dismissal at 6.

There are two fundamental problems with this argument. First, the information in the IG Report reflects that the Army conducted re-testing only in those instances where there was an over-velocity shot and a plate ultimately failed testing. The IG Report does not identify any instance where the Army conducted a re-test on a plate which ultimately passed testing with an over-velocity shot.[4] Setting aside the question of whether it was proper for the Army to have re-tested the failed plates in the context of the over-velocity shot situations, there is nothing to suggest that such actions would have had any bearing on the validity of Armorworks' test failures. This is the case because, had the Army implemented its procedure of re-testing failing plates notwithstanding an otherwise "fair hit" over-velocity shot, Armorworks could only have benefited from such a re-test procedure since it would have had a second chance to pass testing.

Setting aside the fact that the basis of Armorworks' prejudice argument is inconsistent with the record as reflected in the IG Report, the second problem with the argument is that it is based on a hypothetical situation which is incompatible with the purpose of the testing process. As noted above, Armorworks posits a theoretical situation where there was an over-velocity shot on one of its plates, the shot was otherwise a fair hit and the plate would have ultimately passed testing, yet the Army discarded the plate and re-tested a different Armorworks plate, which failed. As a preliminary matter, there is no basis for the factual premise of Armorworks' argument since, as noted above, the Army only retested failed plates with a different plate. Nevertheless, based on the hypothetical facts it now posits, Armorworks contends that it may have suffered prejudice based on the theory that, but for the Army's improper re-test decision, the Army would never have taken the step of selecting another one of Armorworks' plates for testing--a plate which then failed. In essence, Armorworks seeks the potential benefit of a game of chance with respect to which of its plates is selected for testing, a particularly inappropriate approach given that soldiers' lives depend on the performance of the body armor being tested.

As we indicated in our prior decision, notwithstanding the fact that the agency provided Armorworks with the detailed ballistic test results of the plates which failed testing, Armorworks has not presented any evidence to indicate that shots were misfired, that plates were subject to improper shot velocity or shot angle, that measurements were improper, or that the agency did not follow appropriate timing procedures. Armorworks Enters., LLC, supra, at 8. Accordingly, we see no basis to conclude that the findings in the IG Report support Armorworks' assertions of prejudice.

As a final matter, Armorworks alleges that, after award, the Army has, or imminently will, change its first article testing protocol for the body armor contracts already awarded. According to Armorworks, in responding to the IG Report, the Army has admitted that the testing procedures it used were flawed and that it has taken various steps to change body armor testing protocols, to include using a "three-tier scoring methodology to 'ensure scoring accuracy'" and a laser scanner to measure BFD at the deepest point. Protest at 14. Armorworks notes that in responding to the IG Report, the Director of DOT&E indicated that the agency intends to implement expanded testing, review current practices to address "systemic" issues identified in the report, and develop standardized test protocols for use across the Department of Defense. Id. at 15. Because the first article testing procedures identified in the solicitation were identified as being identical to the PDM testing, which Armorworks failed, Armorworks contends that these alleged changes regarding the first article testing requirements "retroactively and prejudicially change the criteria for award without affording all offerors an opportunity to compete on that basis." Protester's Response to Request for Dismissal at 3-4. We disagree.

As a general matter, an agency may not make an award, then immediately modify or waive material requirements included in the solicitation which formed the basis of the competition; rather, awards must be based on the requirements and criteria disclosed in the solicitation. Low & Assocs., Inc., B-297444.2, Apr. 13, 2006, 2006 CPD para. 76 at 6. Offerors are prejudiced where they might have been able to meet the agency's needs if afforded an opportunity to compete based on the relaxed requirements. Haworth, Inc., B-297077, B-297077.2, Nov. 23, 2005, 2005 CPD para. 215 at 3. Here, as discussed below, there is nothing to reasonably suggest that the Army has relaxed the first article testing requirements.

First, the statements by the Army and the Director of DOT&E, on which Armorworks relies, must be read in their proper context. They were made in response to the particular concerns articulated in the IG Report to the effect that the body armor testing procedures and results were not sufficiently reliable because certain body armor that passed testing in fact should be regarded as having failed the testing. As discussed above, we see no basis to conclude that Armorworks, whose body armor failed the PDM testing, could have been prejudiced in connection with the findings in the IG Report since they address the improper passing of tested body armor.

Moreover, the agency's response regarding efforts to change testing were directed toward taking measures to ensure a higher level of confidence in the protective capability of the body armor that passes testing. By way of example, regarding the new standards, the Director for DOT&E explains that it is "DOT&E's goal to develop a [first article test]-like protocol that requires a 90 percent lower confidence limit on a reliability of 90 percent that the material under test passes the requirement (a '90/90')." IG Report at 55. This responds to concerns in the IG Report indicating that the first article testing methodology at issue reflected "a 20 percent chance that at least 36 percent of the plates will not be detected as failures." Id. at 31. Thus, the responses to the report clearly evidence the fact that the intended changes are designed to more effectively identify body armor which does not meet agency requirements--not to relax testing requirements.

In sum, given the context of the IG Report, the responses thereto, and the information otherwise presented by Armorworks, there is no reasonable basis to conclude that any changes to the first article testing requirements could have caused Armorworks any competitive prejudice since Armorworks's body armor failed testing that, by all indications, the agency now intends to make even more stringent.[5]

The protest is dismissed.

Gary L. Kepplinger
General Counsel