Applied Technology Systems, Inc.

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Applied Technology Systems, Inc.

File: B-404267, B-404267.2

Date: January 25, 2011

DIGEST

1. Protest that agency was required to assign the maximum possible ratings to protester's proposal under evaluation factors where protester's proposal contained evaluated strengths and no evaluated weaknesses fails to state a valid basis for protest.

2. Protester's assertions that various portions of its proposal that were rated as "satisfactory" or "very good" should have received higher ratings reflects protester's mere disagreement with the agency's various judgments, and provides no basis for sustaining the protest.

3. Agency reasonably evaluated protester's proposal as reflecting multiple weaknesses, including the conclusion that protester's proposal provided limited information regarding multiple instances of negative past performance and little insight into how protester intended to improve its prior performance.

4. Where agency led protester into the area of its proposal that required additional information, protester's failure to fully respond to agency's question does not create a requirement for the agency to re-open discussions.

Applied Technology Systems, Inc. (ATSI), of Cleveland, Ohio, protests the Department of Labor's award of a contract to Alutiiq Education and Training, LLC, pursuant to request for proposals (RFP) No. DOLJ09RFP00002 for operation of the Westover Job Corps Center in Chicopee, Massachusetts. ATSI challenges the agency's evaluation of ATSI's proposal with regard to the non-cost/price evaluation factors.

We deny the protest.

BACKGROUND

On September 4, 2009, the agency published the RFP at issue, seeking proposals to perform a cost-plus-incentive-fee contract to "provide material, services, and all necessary personnel to operate [the Westover Job Corps Center]."[1] RFP at 5. The RFP established various evaluation factors, advised offerors that "all evaluation factors other than cost or price, when combined, are significantly more important than cost or price," and stated that award would be made on a "best value" basis.[2] Id. at 150. The solicitation further provided that, following submission and evaluation of initial proposals, the agency would make a competitive range determination, and that offerors within the competitive range could be required to make oral presentations to the agency.[3] Id. at 137.

As initially issued, the solicitation contained three questions that offerors would be required to address during their oral presentations.[4] On September 16, the agency issued RFP amendment No. 1; among other things, that amendment changed the questions that offerors were required to address during their oral presentations. Specifically, the amendment revised question "b" and added two new questions related to corporate oversight and financial integrity, as follows:

b. How will you implement Job Corps' New Vision, specifically the Boston Business Model and what is the corporate commitment to collaboration as it relates to the Boston Business Model?

* * * * *

d. Please explain how the Corporation will provide oversight in the implementation of Job Corps' programmatic changes, e.g. CTT [career technical training] instruction changes, certification processes, and industry specific OA [outreach and admission] and CTS [career transition services] approaches.

e. What strategic methodologies will be utilized to ensure the integrity of the financial and programmatic outcome data reported to the government?

RFP Amend. 1 at 5.

On November 9, 2009, initial proposals were submitted by three offerors, including ATSI and Alutiiq;[5] these proposals were thereafter evaluated. In evaluating ATSI's initial proposal under the past performance/experience evaluation factor, the agency expressed multiple concerns regarding ATSI's prior performance of similar Job Corps contracts in the cities of Jacksonville, Cleveland, and Detroit. In summarizing these concerns, an agency evaluator stated:

There were several common issues noted in the CPER's [contractor past effectiveness reports] that will need to be addressed by the offeror. These include not meeting contract goals, poor financial management and oversight, poor and slow corporate response to identified issues, little or no improvement for contracts on Contract Administration Plans, poor student behaviors, decrease in student satisfaction scores, assessment of liquidated damages, high staff vacancies and inability to fill positions in a timely fashion, and poor property management.[[6]]

AR, Tab 4, Proposal Evaluation Form, Dec. 30, 2009, at 2.

By letter dated March 4, 2010, the agency advised ATSI that its proposal had been included in the competitive range, and scheduled ATSI's oral presentation for March 22. However, with the March 4 letter, the agency provided ATSI with a 5-page document, titled "Initial Concern Statements," that listed 86 questions or requests for information regarding various matters, including "[m]ajor weaknesses/deficiencies" in ATSI's proposal. AR, Tab 15, Letter from DOL to ATSI, Mar. 4, 2010, at 2; AR, Tab 3, Final Technical Evaluation Report, at 3. For example, with regard to the past performance/experience evaluation factor, the agency requested, among other things, that ATSI: "[d]iscuss efforts to improve student behaviors and culture at Jacksonville and Detroit"; "[d]iscuss poor financial management and efforts taken to improve this function on all . . . contracts"; "[d]iscuss efforts made to improve data integrity at your centers"; [d]iscuss efforts made to improve corporate responsiveness and intervention for your contracts, particularly as it relates to financial issues and audit reports"; "[d]iscuss efforts made to improve staff retention and ensure centers are fully staffed"; [d]iscuss efforts made to improve property management at your centers." AR, Tab 15, Letter from DOL to ATSI, Mar. 4, 2010, at 3-4. Finally, the agency's March 4 letter stated, "[y]ou will be provided with an opportunity to clarify and ask any questions you may have about these concerns in a telephonic discussion on Tuesday, March 9^th [nearly two weeks prior to the oral presentation date]." AR, Tab 15, Letter from DOL to ATSI, Mar. 4, 2010, at 1.

On March 22, 2010, ATSI made its oral presentation to the agency. During that presentation, ATSI did not address the revised and additional questions contained in RFP amendment No. 1.[7] Contracting Officer's Statement at 4. Since, in its oral presentation, ATSI failed to even attempt to address three of the solicitation's five required questions, the agency evaluated ATSI's oral presentation as "unsatisfactory," which corresponded to a score of 0 points. Id. at 5; RFP at 140. Despite ATSI's failure to comply with the solicitation requirements regarding its oral presentation, the contracting officer advised ATSI that it would be permitted to address the omitted questions in its final proposal revision (FPR).[8] Contracting Officer's Statement at 4.

Accordingly, by letter dated March 26, the agency requested that ATSI submit its FPR, provided another comprehensive list of the agency's ongoing concerns,[9] and advised ATSI to "please ensure your FPR is complete since the agency intends to make award without obtaining any further revisions." AR, Tab 16, Letter from DOL to ATSI, Mar. 26, 2010, at 1. Final proposals were submitted thereafter.

In its FPR, ATSI acknowledged multiple negative aspects regarding its prior performance of various Job Corps contracts. Specifically, with regard to questions concerning its performance of the Detroit contract, ATSI acknowledged that it had "experienced problems in several areas that contributed to declines in performance." ATSI further noted that it recently competed for, and lost, the reprocurement of this contract'-but blamed "Michigan's economy" for ATSI's performance problems. AR, Tab 13, ATSI Past Effectiveness FPR, Apr. 7, 2010, at 6. With regard to ATSI's Jacksonville contracts, ATSI's FPR acknowledged that the firm had experienced a "decline" in performance during 2009 and that one of these contracts was "ranked second to last on [a Job Corps performance] report." Id. at 3, 7. Finally, in responding to the agency's request that ATSI discuss "[ATSI's] poor financial management and efforts taken to improve this function," ATSI acknowledged that it had submitted "several inaccurate and untimely [financial] reports" under its prior Job Corp contracts, had engaged in "inaccurate vouchering," and had not "secure[ed] [approved] indirect rates." Id. at 9. In this regard, ATSI elaborated that:

The situation with ATSI's indirect rate is complicated. It began with the ignorance of the corporation. We did not realize our indirect rate submission had not been filed for several years.

Id. at 10.

In evaluating ATSI's FPR under the technical evaluation factor the agency identified various strengths and virtually no weaknesses. In contrast, in evaluating ATSI's FPR under the other non-cost/price evaluation factors, the agency identified some strengths, but multiple weaknesses. For example, with regard to past performance/experience, the agency summarized its evaluation, stating:

Very serious issues were noted regarding financial management, data integrity, and property at all the centers. There appears to have been poor corporate oversight and slow corporate response to identified issues. . . . It should also be noted that [ATSI] claims "ignorance" with many of their financial issues.

AR, Tab 3, Final Technical Evaluation Report, at 21.

More specifically, the agency determined that ATSI's FPR reflected ongoing past performance weaknesses in that the FPR "provided little insight on how they plan to improve performance at several centers," and "[d]id not address liquidated damages assessed at Cleveland." Id. The agency's evaluation documentation acknowledged that ATSI has taken steps to improve, but concluded that "the overall effect of [some of] the actions taken has not had sufficient time to truly manifest itself." Id. Accordingly, the agency assigned the proposal only 13 of 25 possible points under this factor, referring to the various CPER (contractor past effectiveness report) ratings, on which it primarily relied, as reflecting performance "in the Satisfactory to Marginal range." Id.

Similarly, in evaluating ATSI's FPR under the oral presentation evaluation factor, the agency identified multiple weaknesses, including criticisms of ATSI's proposal regarding corporate oversight. Specifically, the agency concluded that ATSI's FPR "does not clearly describe oversight for all areas," summarizing its concerns as follows: "The amount of corporate oversight is questionable. [ATSI] seems to rely almost entirely on a single corporate staff to provide data integrity and programmatic oversight to the center." Id. at 22-23. More specifically, the technical evaluation panel chair stated:

[ATSI] provides a very weak response to [question d regarding corporate oversight]. Instead of taking oversight as a whole, they literally addressed only the 3 example items. There's no indication how information moves from the corporate office to the actual line staff. There's no description of monitoring or oversight being provided, beyond the single corporate POC [point of contact] identified. Does not seem as if the offeror really understood the intent of this question. Entire oversight seems to be placed on one person.

AR, Tab 4, Proposal Evalaution Form, Apr. 19, 2010, at 13.

Based on various agency concerns, including those discussed above, the agency assigned the FPR only 9.5 of a possible 20 points with regard to ATSI's oral presentation. AR, Tab 3, Final Technical Evaluation Report, at 22.

Finally, in evaluating ATSI's FPR under the staff resources evaluation factor, the agency criticized the candidate ATSI proposed for the position of deputy director on the basis that the she "does not have any relevant experience for the position." Id. at 20. The agency noted that, while ATSI "claims that [the deputy director candidate] has been instrumental in improving outcomes and community relations," its proposal "failed to show support for those statements." Id. Additionally, the agency criticized the proposal for unrealistically assuming that a proposed secretary could perform an overly broad range of tasks, noting that ATSI had proposed for the secretary to "provide support to Academics and 2 CTT [career technical training] Cluster[s] while also [being] responsible [for] creating the student schedules and entering them in [the center information system]." Id. at 19.

Overall, the agency rated ATSI's and the awardee's proposals as follows:

	Score/Rating for Non-Cost/Price Evaluation Factors (120 points)	Evaluated Cost/Price
Alutiiq	78.447 / Very Good	$78,470,029
ATSI	72.625 / High Satisfactory	$78,711,582

AR, Tab 18, ATSI Debriefing at 2.

Thereafter, the agency selected Alutiiq's proposal for award on the basis of its higher-technically-rated, lower-priced proposal. This protest followed.

DISCUSSION

ATSI's initial protest was based on the assertion that the agency improperly evaluated ATSI's proposal under the non-cost/price evaluation factors.[10] As discussed below, we have reviewed all of ATSI's assertions and find no basis to sustain its protest.

Evaluation Under Technical Factor

ATSI first notes that, with regard to the technical evaluation subfactors,[11] ATSI's proposal was evaluated as containing various strengths and virtually no weaknesses, and that the proposal was rated "satisfactory" or "very good," but not "excellent," under the various subfactors. AR, Tab 3, Final Technical Evaluation Report, at 13-19. ATSI asserts that it was unreasonable for the agency to rate its proposal as anything other than "excellent," and to award less then the maximum possible point scores, under each subfactor. ATSI Comments, Dec. 3, 2010, at 5. Alternatively, ATSI asserts that, even if the agency reasonably assigned less than the highest rating and maximum point scores, its proposal should have received higher ratings and point scores under each subfactor. We disagree.

The evaluation of technical proposals is generally a matter within the agency's discretion, which our Office will not disturb unless it is shown to be unreasonable or inconsistent with the RFP's evaluation criteria. See, e.g., METAG Insaat Ticaret A.S., B-401844, Dec. 4, 2009, 2010 CPD para. 86 at 4; Manassas Travel, Inc., B'294867.3, May 3, 2005, 2005 CPD para. 113 at 2-3. In this regard, a protester's mere disagreement with the agency's judgments does not render an evaluation unreasonable. Id. Further, there is no legal requirement that an agency must award the highest possible rating, or the maximum point score, under an evaluation factor simply because the proposal contains strengths and/or is not evaluated as having any weaknesses. See, e.g., Archer Western Contractors Ltd., B'403227, B-403227.2, Oct. 1, 2010, 2010 CPD para. 262 at 5; Pannesma Co. Ltd., B'251688, Apr. 19, 1993, 93-1 CPD para. 333 at 4.

Here, the agency established internal guidelines for the assessment of adjectival ratings, defining the applicable criteria as follows:

Exceptional: Programs, procedures and systems proposed are well organized, clearly conveyed and address all aspects completely. The proposal consistently presents concrete details to illustrate and explain proposed systems. It demonstrates thorough knowledge and understanding of the Job Corps mission and policies and recognizes new initiatives.

The proposal is easily understood and complete. In most cases, it includes details to illustrate and explain proposed systems. With few exceptions, the proposal is internally consistent, and demonstrates knowledge of and consistency with Job Corps policies and mission.

Satisfactory: The proposal addresses generally all aspects of the RFP and meets all basic requirements. The proposal is understandable, and contains adequate detail to convey understanding of the systems and approaches proposed. No major or serious inconsistencies with policy or initiatives exist. Proposed systems do not contradict Job Corps mission or policies in any significant or serious way.

AR, Tab 18, ATSI Debriefing Letter, at 2.

The above definitions are consistent with the provisions of section M of the solicitation, which advised offerors that proposals would be evaluated regarding the extent to which proposed systems, procedures, and approaches are viewed by the agency as "clear, complete, concrete and consistent with Job Corps mission and policies," "tailored to local needs," and "effective to meet performance outcomes and quality indicators." RFP at 142. To the extent ATSI asserts that the agency was required to assign an "excellent" rating, and maximum point scores, for subfactors where ATSI's proposal was evaluated as having strengths and/or no weaknesses, the protest fails to state a valid basis.

To the extent ATSI alternatively argues that, in any event, its proposal should have received higher ratings and point scores than those the agency assigned, we have reviewed the record and find no basis to sustain the protest. In this regard, ATSI's alternative assertions are primarily based on ATSI's recitation of the number of evaluated strengths its proposal received under each subfactor, maintaining that identification of those strengths required various higher scores.[12] We disagree. Nothing in the solicitation tied any particular adjectival rating, or associated point score, to any quantity of evaluated strengths.[13] Further, we have reviewed the evaluation record and find nothing unreasonable in the agency's various judgments regarding ATSI's proposal under the technical evaluation subfactors. ATSI's assertions to the contrary constitute mere disagreement with the agency's judgments, and provide no bases for sustaining its protest.

Evaluation Under Other Evaluation Factors

Next, ATSI's challenges the agency's identification of various weaknesses in ATSI's proposal with regard to the past performance/experience, oral presentation, and staff resources evaluation factors. We have reviewed all of ATSI's arguments, and find no basis for sustaining its protest.

ATSI first challenges the agency's identification of weaknesses with regard to past performance/experience. As discussed above, the agency concluded that there have been "[v]ery serious issues" in ATSI's past performance with regard to, among other things, "financial management, data integrity, and property," and that there has been "poor corporate oversight and slow corporate response to identified issues." AR, Tab 3, Final Technical Evaluation Report, at 21. The agency further concluded that, despite multiple requests for ATSI to discuss and address these matters, ATSI's proposal "provided little insight" on how it intended to improve performance, did not in any way address the matter of liquidated damages, and claimed "ignorance" with regard to some of the basic financial issues. Id.

ATSI protests that, despite its acknowledged submission of inaccurate financial reports, inaccurate vouchers, and failure to obtain approved indirect rates, its proposal was "misevaluated" under the past performance/experience factor because ATSI had "submitted a corrective action plan which was accepted by the Department of Labor," because "the liquidated damages had been addressed,"[14] and because the agency "failed to take into account" certain positive aspects of its prior performance. Protest at 5; ATSI Comments, Dec. 3, 2010, at 14.

The evaluation of past performance, including the agency's assessments regarding relevance, scope, and significance of the offeror's performance history, is a matter of agency discretion, which we will not disturb unless those assessments are unreasonable, inconsistent with the solicitation criteria, or undocumented. See, e.g., Acepex Mgmt. Corp., B'283080 et al., Oct. 4, 1999, 99-2 CPD para. 77 at 3, 5. Further, an agency's past performance evaluation may be based on a reasonable perception of inadequate prior performance, regardless of whether the contractor disputes the agency's interpretation of the underlying facts or significance of those facts. See, e.g., Ready Transp., Inc., B-285283.3, B-285283.4, May 8, 2001, 2001 CPD para. 90 at 5. Finally, a protester's mere disagreement with the agency's assessments is not sufficient to establish that the agency acted unreasonably. Birdwell Bros. Painting & Refinishing, B-285035, July 5, 2000, 2000 CPD para.129 at 5.

Here, the record is replete with undisputed statements regarding ATSI's negative past performance history under prior Job Corps contracts.[15] Indeed, the contemporaneous record in this regard provides considerably more than a "reasonable perception of inadequate prior performance." See Ready Transp., Inc., supra. Further, in conducting this procurement the agency repeatedly sought information from ATSI regarding the causes of its negative past performance, and information relevant to ATSI's potential to improve such performance. While ATSI, in fact, provided some information regarding its pending and ongoing efforts to address its prior problems, failures, and inadequacies--which the agency properly acknowledged, considered, and documented--the agency concluded that, many of the issues remained inadequately addressed and, to the extent they were addressed, there had been insufficient time to assess the effectiveness of ATSI's corrective actions. Based on our review of the record, we find nothing materially flawed or unreasonable in the agency's assessments. ATSI's assertions to the contrary reflect mere disagreement with the agency's judgment and/or provide no bases for sustaining the protest.[16]

Next, ATSI protests the agency's evaluation of ATSI's oral presentation, maintaining that it was improper for the agency to assign a weakness regarding ATSI's proposed corporate oversight. In this regard, ATSI complains that its proposal "us[ed] the same oversight model as the Department of Labor [DOL]," and therefore, its proposed approach to corporate oversight "should have been evaluated as a strength." Protest at 5.

As noted above, the evaluation of technical proposals, is generally a matter within the procuring agency's discretion, which our Office will not disturb unless it is shown to be unreasonable or inconsistent with the RFP's evaluation criteria. See, e.g., METAG Insaat Ticaret A.S., B-401844, Dec. 4, 2009, 2010 CPD para. 86 at 4; Manassas Travel, Inc., B'294867.3, May 3, 2005, 2005 CPD para. 113 at 2-3.

Since it was unclear to this Office what DOL "oversight model" ATSI was referring to, we requested that ATSI provide additional information regarding this matter. Specifically, in order to fully understand ATSI's assertion, we requested that ATSI "identify the source/authority on which you rely to establish what you describe as the '[corporate oversight] model' utilized by the Department of Labor."[17] Email from GAO to ATSI Counsel, Dec. 15, 2010.

In response, ATSI again asserted that "ATSI uses the same corporate oversight as DOL," elaborating that the "organizational position" of ATSI's Chief Executive Officer "directly parallels DOL's Regional Director/Contracting Officer"; that ATSI's Senior Director of Programs and Development "parallels DOL's Deputy Chief for Programs"; that ATSI's Chief Financial Officer "parallels with DOL's Deputy Chief of Administration," and that ATSI's Corporate Office Technical Representative "parallels DOL's Contracting Officer Technical Representative." ATSI Comments, Dec. 21, 2010, at 2-3.

ATSI misses the point. The agency's evaluation did not criticize ATSI's organizational structure; rather, it criticized ATSI's proposal for failing to identify specific oversight activities that would be performed in all areas of contract performance (not limited to the specific examples identified in the agency's question) by anyone other than a single ATSI employee. ATSI's assertions regarding the similarity of its organizational structure to that of DOL do nothing to address the fact that its proposal failed to adequately address ATSI's corporate oversight activities.[18] Accordingly, we find no basis to question the agency's evaluation of a weakness with regard to ATSI's proposed corporate oversight.[19]

Meaningful Discussions

Finally, following receipt of the agency report responding to its initial protest, ATSI amended its protest to assert that the agency failed to conduct meaningful discussions regarding ATSI's proposed corporate oversight. Specifically, ATSI complains that the agency's question regarding corporate oversight was "ambiguous," that ATSI misunderstood the question, and that the agency "should have reopened discussions with ATSI." ATSI Amended Protest, Nov. 29, 2010, at 5-6; ATSI Comments on Supplemental AR, Dec. 21, 2010 at 5-9.

When an agency engages in discussions with an offeror, the discussions must be meaningful, that is, they must lead the offeror into the areas of its proposal that require correction or additional information. Hanford Envtl. Health Found., B'292858.2, B'292858.5, Apr. 7, 2004, 2004 CPD para. 164 at 8. However, an agency is not obligated to reopen negotiations to give an offeror the opportunity to remedy a defect that first appears in a revised proposal. American Sys. Corp., B-292755, B'292755.2, Dec. 3, 2003, 2003 CPD para. 225 at 8.

Here, as discussed above, the agency first advised ATSI of the requirement to address corporate oversight in RFP amendment No. 1, which stated:

Please explain how the Corporation will provide oversight in the implementation of Job Corps' programmatic changes, e.g. CTT [career technical training] instruction changes, certification processes and industry specific OA [outreach and admission] and CTS [career transition services] approaches.

RFP Amend. 1 at 5.

On March 22, 2010, ATSI provided its oral presentation to the agency, but ATSI states that it "was previously unaware" of the amended solicitation's requirements to explain its approach to corporate oversight. Declaration of ATSI Chief Executive Officer, Dec. 21, 2010, para. 5. Notwithstanding ATSI's failure to comply with the solicitation's explicit requirements regarding its oral presentation, the agency provided written discussion questions to ATSI--specifically repeating its request that ATSI address its proposed corporate oversight, and directing ATSI to "ensure your FPR is complete since the [agency] intends to make award without obtaining any further revisions." AR, Tab 16, Letter from DOL to ATSI, Mar. 26, 2010, at 1. As discussed above, the agency reasonably concluded from ATSI's subsequent FPR response that ATSI failed to fully and adequately address its approach to corporate oversight.

In short, the agency clearly provided ATSI with a meaningful opportunity to fully discuss its proposed corporate oversight during its oral presentation; ATSI did not. Thereafter, the agency again requested that ATSI address this matter; again, ATSI failed to fully and adequately respond. On this record, there is no merit to ATSI's assertions that the agency failed to conduct meaningful discussions regarding ATSI's corporate oversight, and ATSI's failure to fully address this matter did not create a requirement for the agency to re-open discussions.

The protest is denied.

Lynn H. Gibson

General Counsel