IBM Global Business Services

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: IBM Global Business Services

File: B-298833.4; B-298833.5

Date: March 1, 2007

DIGEST

In procurement for system integrator to deploy commercial off'the-shelf software product suite and change management techniques, quotation was reasonably evaluated as high risk where vendor intended to involve too few government personnel in change management process to accomplish important change management activities; protester's defense of its ratio of government personnel to contractor consultants is undercut by fact that ratio was significantly lower than that in typical commercial implementations.

IBM Global Business Services protests the Department of the Air Force's issuance of a task order to Computer Sciences Corporation (CSC), under request for quotations (RFQ) No. FA8770-05-Q-0022, to serve as the Expeditionary Combat Support System (ECSS) System Integrator (SI). IBM challenges the terms of the solicitation and the evaluation of quotations.

We deny the protest.

The RFQ was issued, in accordance with Federal Acquisition Regulation subpart 8.4, to vendors holding Federal Supply Schedule Department of Defense Enterprise Resource Planning (ERP) Systems Integration Services Blanket Purchase Agreements. The RFQ called for an SI to deploy a commercial off'the-shelf (COTS) software product suite and change management techniques, which would replace legacy logistics systems and antiquated business processes. The SI is required to configure, integrate and implement the COTS software, as well as provide leadership and resources to redesign and transform Air Force logistics operations so as to improve logistics results and reduce support costs. The RFQ generally contemplated a phased approach. The initial phase encompasses preparing the organization for change and defining how the selected COTS product suite will be configured to meet the agency's requirements, which will include such activities as: (1) change management, that is, the process by which executives, middle managers, and end users are prepared to accept changes in business processes; (2) blueprinting, that is, creating descriptions of the Air Force's business processes and determining how the COTS product suite will support the agency's business needs; and (3) continuing efforts of –Pathfinders— to prototype COTS implementations and to prove that they can work in an Air Force logistics environment. In the second phase, the SI will undertake implementation of ECSS, performing all work necessary to configure and test the ECSS solution in preparation for deployment; in the third phase, the SI will undertake deployment of ECSS into a live production operation; and in the fourth phase, the SI will transition ECSS into Air Force managed sustainment. ECSS Request for Quote sections 3, 4, 5; ECSS Task Descriptions; ECSS Statement of Objectives (SOO); Agency Comments, Feb. 10, 2007, at 11 n.10, 41 n.30, 57.

Award was to be made to the vendor whose quotation represented the –best value— to the government based on three evaluation factors: (1) mission capability/quote risk (including subfactors for program execution, change management, and program management); (2) past performance; and (3) price (comprised of the value of the task order plus the value of any government-furnished personnel and property). Under each mission capability subfactor, quotations were to be rated + for exceeding minimum performance or capability objectives in a beneficial way, for meeting minimum objectives, - for not clearly meeting minimum objectives, or F for failing to meet minimum objectives. In addition, quote risk ratings were assigned at the subfactor level. Mission capability/quote risk and past performance, combined, were significantly more important than price.

CSC and IBM submitted quotations. After conducting extensive discussions with the vendors, the agency requested final quotation revisions (FQR). On September 8, 2006, the Air Force issued a task order to CSC on the basis that its quotation represented the best value to the government. IBM thereupon filed several protests with our Office challenging the award (B-298833, B'29883.2, B-298833.3). The agency subsequently determined to take corrective action, including reevaluation of quotations, and we therefore dismissed the protests. After reevaluating quotations, the agency determined that CSC's quotation was more advantageous than IBM's under each of the evaluation factors, and remained the best value. The ratings were as follows:

	IBM	CSC
Mission Capability
Program Execution	/low risk	/moderate risk
Change Management	/high risk	+ /low risk
Program Management	/low risk	+ /low risk
Past Performance	Confidence	Significant Confidence
Evaluated Price
Task Order Value	$747.0 million	$627.8 million
Value Government Furnished Personnel	$51.1 million	$118.9 million
Value Government-Furnished Property	$0 million	$5.0 million
Total Price	$798.1 million	$751.7 million

CSC's quotation was rated + under the change management and program management subfactors for exceeding ECSS objectives, while IBM's quotation was rated only . Further, IBM's approach to change management was rated as high risk due to IBM's request for an insufficient number of government personnel to be furnished to support its approach; this was found likely to cause significant disruption of schedule, significant increased cost, and significant degradation of performance. Source Selection Decision (SSD) at 4-5.

Both quotations were rated under the program execution subfactor. In this regard, IBM planned to initially deploy ECSS to agency Air Logistic Centers, while CSC planned to initially deploy ECSS to Air Force bases. IBM's quotation was evaluated as exceeding ECSS objectives by offering full operational capability 4 months earlier than specified in the notional solicitation schedule (and 5 months earlier than CSC), which would potentially accelerate cost savings from retiring legacy systems and reducing inventory. CSC's –bases first— approach of deploying a complete set of process functionality to end-users at each base also was favorably evaluated on the basis that it would make ECSS available to 70 percent of the end users up front and (unlike under IBM's approach) in a single deployment per site, which would eliminate the need for dual transactions by users.[1]

As for past performance, the agency assigned CSC's quotation a rating of significant confidence and IBM's a rating of only confidence, on the basis that CSC had better demonstrated the capability to develop and deploy large-scale, highly complex COTS/ERP integrated business solutions comparable to the magnitude and complexity of the ECSS. On the other hand, the agency found that IBM had demonstrated the skills and capabilities necessary to transform small'to'medium-scale business operations using COTS/ERP applications and industry best practice, but determined that IBM (unlike CSC) had not demonstrated the capability to deploy large-scale, highly complex COTS/ERP solutions reasonably comparable to the magnitude and complexity of the ECSS.

Given CSC's advantages under the non-price factors and its lower evaluated price, the Air Force again made award to CSC. IBM thereupon filed this protest.

In reviewing protests against allegedly improper evaluations, it is not our role to reevaluate proposals. Rather, our Office examines the record to determine whether the agency's judgment was reasonable and in accord with the RFP criteria and applicable procurement statutes and regulations. See Rolf Jensen & Assocs., Inc., B'289475.2, B-289475.3, July 1, 2002, 2002 CPD para. 110 at 5. Here, based on our review of all of IBM's timely arguments, we find no basis for questioning the award decision. We discuss several of IBM's arguments below.

GOVERNMENT-FURNISHED PERSONNEL

IBM raises various arguments concerning the role government'furnished personnel were to play in the SI contractor's efforts under the ECSS program. In this regard, the RFQ made clear the agency's view that a successful implementation of ECSS would require a partnership between the agency and the SI. According to the RFQ, –[t]he ECSS COTS Product Suite represents industry best practices which, when combined with the experience and knowledge of the Air Force Logistics Community, ECSS project team, and Systems Integrator (SI), will provide the best opportunity to achieve the objective as described in the RFQ documents.— RFQ, Best Value Selection Factors, sect. 4.1. Likewise, the RFQ (as amended after the receipt of initial quotations) indicated that, consistent with the –governing umbrella philosophy of Enterprise Resource Planning System Integration Services[,] . . . the parties anticipate a unique sharing of concerns, knowledge and ideas via a partnership whereby Government personnel are made available to provide a source of information, and to work side-by-side with the contractor.— RFQ amend. 5.

In order to account for vendors' varying approaches to making use of government personnel in designing and implementing ECSS, the RFQ required vendors to complete a Government Personnel Matrix, to be incorporated into the awarded order as attachment 14, entitled –Government Personnel—; vendors were to specify by contract year the number of government personnel in each of three experience levels (senior, –Mid,— and junior) and the percentage of their time that would be required under the vendor's proposed approach for each ECSS activity (e.g., program management, change management, business blueprinting). According to the RFQ, attachment 14 –defines the maximum amount of Government Personnel available, by skill level, per contract year.— RFQ amend. 5. For each category of government personnel for each contract year, the matrix specified a cost per full-time equivalent (FTE) that would be added to the vendor's quoted price for purposes of the evaluation in order to –eliminate any competitive advantage resulting from the vendor's proposed use of Government Personnel.— RFQ amend. 4; RFQ, Best Value Selection Factors, attach. 14.

The RFQ further provided that –[i]t is incumbent upon the Government to make the Government Personnel available [in accordance with] . . . Attachment 14, Government Personnel—; according to the RFQ, the contractor –shall be entitled to an equitable adjustment should the Government fail to make the required personnel available [in accordance with] . . . Attachment 14.— RFQ amend. 5. However, the RFQ also provided that the contractor's request for government personnel shall be filed so as to allow a lead time of 60 days in order to ensure availability, and that

[b]ecause Government resources availability varies by location, the request shall allow flexibility in scheduling. Flexibility may require allowance for alternate dates of availability within the same contract year, a variance in labor mix or other variances.

Id.In addition, the government reserved the –right to unilaterally substitute qualified Advisory and Assistance Services (A&AS) contractors, with similar skill levels, for Government Personnel.— Id.

The RFQ further provided for government personnel to be available to support the contractor's ECSS effort, stating as follows:

The Government recognizes that Subject Matter Experts (SMEs) are essential to the success of the ECSS program. Key functional SMEs from across the Air Force Logistics Enterprise will be integrated into the PMO [Program Management Office]. The Government anticipates that the SMEs' team will be comprised of approximately three (3) GS'15 or O-6 personnel, twelve (12) GS-14 or O-5 personnel, and twenty (20) GS-13 or O-4 personnel. At appropriate times these SMEs will be augmented with additional functional experts from Air Staff and the MAJCOMs.

Statement of Objectives sect. 4.1.2. Vendors were to exclude from attachment 14, and thus from the price evaluation, government personnel to be furnished in support of its ECSS effort who were from the ECSS PMO or from the Logistics Transformation Office (LTO). Attach. 13.

CSC listed in its attachment 14 a total of 1,202.6 government FTEs, spread over 7 years, that would be required for its ECSS effort; the agency added an adjustment of $118.9 million (at the rates specified by the agency in attachment 14) to its evaluated price to reflect the cost of providing these government personnel. In addition, pursuant to the above RFQ provisions, CSC's quotation specified a total of approximately 163 unpriced government FTEs, spread over 7 years, which would be required from the PMO/LTO. In contrast, IBM listed 486.3 government FTEs, which resulted in a $51.1 million adjustment to its evaluated price. IBM did not specifically enumerate the government FTEs from the PMO/LTO that it would require.

Government Obligations

As an initial matter, IBM asserts that issuance of a task order to CSC was improper because the Air Force lacked authority to contractually commit government personnel to work with CSC on the ECSS effort. This argument is untimely. As discussed above, the RFQ made it clear that the agency anticipated –a unique sharing of concerns, knowledge and ideas via a partnership whereby Government personnel are made available to provide a source of information, and to work side-by-side with the contractor.— RFQ amend. 5. Further, the RFQ included attachment 14, wherein vendors were to list the government personnel time that would be required under the vendor's proposed approach, with the contractor generally –entitled to an equitable adjustment should the Government fail to make the required personnel available [in accordance with] . . . Attachment 14.— Id.Under our Bid Protest Regulations, protests based upon alleged improprieties in a solicitation apparent prior to the time set for receipt of initial quotations must be filed prior to that time, while alleged improprieties subsequently incorporated into the solicitation must be protested not later than the next closing time for receipt of quotations. 4 C.F.R. 21.2(a)(1) (2006). Since it was clear from the solicitation that government personnel would be made available to work with the SI under the contract, and since IBM did not protest this aspect of the solicitation prior to the next closing time, the protest in this regard is untimely.

In any case, IBM's quotation was based on the same approach about which IBM complains with regard to CSC's quotation--it provided in attachment 14 that the Air Force would be required to furnish 486.3 government FTEs. Having submitted a quotation incorporating government-furnished personnel, IBM cannot now assert that it was improper for the agency to accept CSC's quotation due to its reliance on such personnel. The integrity of the protest process does not permit a protester to espouse one interpretation or position during the procurement, and then argue during a protest that the interpretation or position is unreasonable or otherwise improper. Northrop Grumman Sys. Corp., B-298954 et al., Jan. 12, 2007, 2007 CPD para. __ at 8; BST Sys., Inc., B-298761, B-298761.2, Dec. 1, 2006, 2006 CPD para. __ at 6; AAI Eng'g Support, Inc., Nov. 16, 1994, 95-1 CPD para. 2 at 3-4.

IBM further asserts that CSC rendered its quotation unacceptable by improperly conditioning its performance on the Air Force's furnishing government personnel in a certain manner. IBM notes in this regard that CSC included in its proposed Performance Work Statement (PWS) an –Assumption[] and Constraint[]— that the –change agent roster will remain stable with limited turnover and replacements. No more than 10% turnover of change agents per year.— CSC PWS at 47.

Here too, however, IBM is challenging an aspect of CSC's quotation that is similar to its own. In this regard, while CSC's quotation set forth certain –Assumptions and Constraints,— IBM's defined numerous –Government Roles and Responsibilities.— For example, IBM's quotation provided that it was the Air Force's responsibility to: [REDACTED] See, e.g., IBM PWS at 3, 17, 26-27, 59-61, 67, 86, 93. While perhaps not as specific as some of the government obligations set forth in CSC's quotation, IBM's definition of the –Government['s] Roles and Responsibilities— likewise defined government obligations under the contemplated order, and to that extent could be viewed as similarly conditioning IBM's quotation. Again, we view this argument as inconsistent with INM's position during the procurement, and we therefore will not consider it. To the extent that CSC allegedly acted inconsistently with the solicitation, so did IBM. See Northrop Grumman Sys. Corp., supra; BST Sys., Inc., supra; AAI Eng'g Support, Inc., supra.

Availability of Government Personnel

IBM asserts that the Air Force did not adequately account for the risk that it would be unable to furnish the 1,202.6 government FTEs, and the approximately 163 additional FTEs from the PMO/LTO, specified in CSC's quotation.[2] In this regard, the protester notes that the Air Force has already decided to acquire the services of up to 28 contract personnel under the General Services Administration's Mission Oriented Business Integrated Services (MOBIS) contract in order to supplement the LTO staff. More generally, the protester notes that the Air Force has announced plans to reduce overall staffing by approximately 40,000 active duty personnel and 18,000 civilian/guard/reserve personnel. The Air Force responds that it reasonably determined that adequate government staffing would be available to support either vendor's proposed ECSS approach.

Based on our review of the record, including testimony at the hearing our Office conducted in this matter, we find nothing unreasonable in this aspect of the evaluation. As noted by the agency, ECSS is a major program that will completely transform the Air Force logistics process by replacing over 400 legacy systems and supporting over 250,000 concurrent users. The record indicates that, during the course of the procurement, the Air Force Director of Transformation for logistics, who determined that the agency could furnish the 400-1,200 government personnel (not including PMO or LTO staff) the vendors had proposed, repeatedly briefed the leadership of the nine Air Force Major Commands involved in the Air Force logistics process on the extent of the commitment of agency personnel to ECSS that would be necessary. The director testified that the ECSS program had received a commitment from the Major Commands to furnish the requisite support. Hearing Transcript (Tr.) at 65-67, 96-98, 139-42; ECSS Briefing Index; see, e.g., AF Logistics Transformation eLog21 Briefing Slides, Sept. 13, 2006. Given that the Air Force logistics workforce consists of at least 160,000 personnel, and the evident commitment of the Major Commands to furnish the resources necessary to transform the Air Force logistics process, the agency reasonably could conclude that the total of 1,365.6 government FTEs called for under CSC's approach would be made available, especially since only a part-time commitment would be required for most affected employees. CSC attach. 14.[3]

IBM's Change Management Approach

As noted above, IBM's approach to Change Management was rated as high risk due to its identification of and request for an insufficient number of government personnel to be furnished. IBM disputes the agency's determination that its staffing was inadequate in this regard.

The evaluation in this area was reasonable. The RFQ described change management'-that is, the process by which executives, middle managers, and end users are prepared to accept changes in business processes--as –the key to successful implementation and adoption of a logistics enterprise solution.— SOO sect. 3.2. The RFQ explained the importance of change management as follows: –To succeed, ECSS must gain user buy-in, demonstrate the value/benefits of common approaches across the Air Force Logistics Enterprise, and create confidence in the ECSS solution with its associated processes and supporting organizational changes.— Id.

IBM acknowledged in its quotation that –[t]he importance of change management to the ultimate success of the endeavor simply cannot be overstated or overemphasized,— and that –success is built on a true partnership with the AF and that active participation of the AF community in all aspects of the program is critical.— IBM Technical Proposal sect. 3.0, Change Management; IBM Response to Evaluation Notice (EN) A-PE-CR-255. However, the record supports the agency's determination that IBM's quotation provided insufficient participation by Air Force personnel in the change management process. In this regard, the agency explains that, in the proposed sequence of ECSS activities, commencing with planning and continuing through blueprinting, implementation, deployment, and transition to sustainment, change management staffing should increase at the end of planning and the beginning of blueprinting, and then begin to decline during the last part of deployment and the beginning of transition to sustainment. The agency found that IBM's approach did not follow this staffing pattern. Rather, IBM's attachment 14 requested 21.25 government FTEs for participation in change management activities in each of the first 3 contract years, and then only 17 in year 4, that is, prior to implementation, 12.75 in year 5, 8.5 in year 6, and none in year 7. In contrast, CSC's approach more closely followed the desired pattern, requesting the participation of 4 government FTEs in change management activities in year 1, 5.6 in year 2, 76.5 in year 3, 133 in year 4, 111.3 in year 5, 26.2 in year 6, and none in year 7. CSC FQR attach. 14.

The agency found as another indication of inadequate involvement of government personnel in the change management process, the fact that the 425 members of IBM's proposed Stakeholder Engagement Team were to participate in change management activities for only 5 percent of their time in years 1-3, 4 percent in year 4, 3 percent in year 5, 2 percent in year 6, and 0 percent in year 7. The agency found that this level of participation, after accounting for conferences, meetings and reviews, would leave little time for true change management activities. Proposal Analysis Report (PAR) at 30.

Further, the agency determined that another potential approach to facilitating change management--relying on super users in the user activities--was not part of IBM's approach as priced in attachment 14. In this regard, under the RFQ, the SI was to train super users (estimated to number 5,000), who would then provide support and advice on the new systems and processes to non'super users. The agency noted that, while IBM's quotation indicated that –[t]he ECSS program will benefit greatly from the involvement of the ECSS super users,— IBM did not include in its attachment 14 calculations any time for the super users to provide this assistance, explaining instead that –the super users will optionally choose to support their co'workers.— RFQ Technical Parameters sect. 3.2; IBM Response to Evaluation Notice (EN) A-PE-CR-299; PAR at 31. The agency asked IBM to explain why it characterized super users as –optional resources— (and accordingly did not list them in attachment 14 for pricing purposes). IBM responded that, while the ECSS program would –benefit greatly— from the participation of the super users, –ECSS implementation and deployment is not dependent on the estimated 5,000 ECSS super-users serving as local area experts and providing local support.— IBM Response to Evaluation Notice (EN) A-PE-CR-299. The agency ultimately evaluated IBM's approach as relying on the super users to facilitate change management, since IBM had listed only limited other government participation in the change management process. Since, however, the super users were not priced in IBM's attachment 14, the agency determined that IBM's approach to super users was further evidence of IBM's inadequate involvement of government personnel (as priced in attachment 14). Tr. at 1380; Agency Comments, Feb 13, 2007, at 74.[4]

Finally, the agency viewed as still further evidence of IBM's inadequate involvement of government personnel the fact that IBM's overall ratio of government personnel to SI contractor personnel was only approximately 0.75:1. According to the agency, a ratio of up to 3:1 was recommended in large ERP implementations, while the minimum government participation level was 1:1. PAR at 30-31. IBM disputes the agency's understanding of the appropriate government/contractor personnel ratio. We note, however, that IBM itself advised the agency during a February 3, 2005 Industry Day briefing that –[t]he typical commercial ERP implementation has 3 to 4 clients for every consultant.— Feb. 3, 2005 Industry Day Briefing Slides. According to the protester's briefing, –[m]any public sector clients do not commit the level of resources required to successfully implement and sustain their program.—

We think the Air Force could reasonably view the above aspects of IBM's quotation as indicative that its approach (in attachment 14) to government participation was inadequate. IBM has not shown otherwise. Id.In particular, given the agency's stated intent to adopt –commercial best practices,— RFQ sect. 3.1, Project Summary/Strategy, and IBM's own recommendation that the government adopt a more commercial approach to ERP implementation, IBM's defense of its proposed much more limited level of government participation is not persuasive. We thus find no basis to question the agency's determination that IBM's approach to government participation was inadequate.

IBM asserts that it was misled by the agency as to the expected level of government personnel participation in the ECSS process. Specifically, IBM notes that the RFQ stated that –[t]he Government intends for the SI to provide as many of the necessary project resources as possible.— SOO sect. 4.2. IBM further notes that, in response to an IBM question during discussions as to whether it was –the Government's intention to allow unlimited use of Air Force resources,— the agency responded as follows: –No, the Government has limited resources and must fully understand the quantity and timing of personnel required to support each vendor's approach.— Air Force Memo for File, Apr. 20, 2006.

We find nothing misleading here. The statements cited by IBM are general in nature and, we think, did not warrant an interpretation, essentially, that the agency intended vendors to minimize their use of government personnel. In particular, we view the agency's statement--in response to IBM's inquiry as to whether agency resources were –unlimited—--that the government has limited resources as no more than a statement of the obvious. Moreover, IBM's assertion that it was misled ignores the fact that, in response to its listing of only 146.3 government FTEs in its initial response to the April 26, 2006 amendment adding the attachment 14 government personnel matrix, the agency advised the firm in a deficiency notice that –[y]our response to the Government Personnel Template is unrealistically low.— EN A'AD'DR-294. As evidenced in IBM's subsequent more than three'fold increase in the number of requested government FTEs listed in its revised attachment 14, ultimately reaching 486.3 FTEs, IBM clearly understood that proposing too limited government participation in the ECSS process would be evaluated unfavorably.

PAST PERFORMANCE

As noted above, the Air Force rated CSC's past performance –significant confidence— and IBM's only –confidence.— IBM challenges CSC's rating primarily on the basis that it failed to reflect CSC's poor performance under contracts concerning the Internal Revenue Service's (IRS) Business Systems Modernization (BSM) and Integrated Financial System (IFS) programs.

The evaluation of past performance, including the determination of the relevance and scope of vendors' performance history to be considered, is a matter of agency discretion, which we will not question unless it is unreasonable or inconsistent with the solicitation criteria, or procurement statutes or regulations. National Beef Packing Co., B-296534, Sept. 1, 2005, 2005 CPD para. 168 at 4. The past performance evaluation here was unobjectionable.

The RFQ provided for the agency to evaluate the –recent and relevant performance— of the vendor and its three principal subcontractors, –focusing on and targeting performance which is relevant to the Mission Capability subfactors and the requirements of the solicitation.— RFQ, Best Value Selection Factors, sect. 5.1. The RFQ listed several factors that would be considered, including –[m]anagement actions employed and indicators/metrics used to achieve success and overcome problems.— Id.The RFQ added that –more relevant past performance efforts will be considered more heavily in the rating than less relevant past performance efforts.— Id.In addition, the RFQ defined –recent— past performance as follows: –In determining recency, any past performance within the last 60 months (from the date of RFQ release) shall be considered.— Id.

CSC's past performance rating of significant confidence was based on its having –demonstrated the requisite capabilities to develop and deploy large-scale, highly complex COTS/ERP integrated business solutions comparable to the magnitude and complexity of the ECSS.— SSD at 5; Performance Confidence Assessment Group (PCAG) Final Report at 111. In particular, the agency noted CSC's strong performance under three contract efforts that were evaluated as –Very Relevant large-scale, highly complex, integrated business solutions:— (1) a contract under the Army's Logistics Modernization Program (LMP), an ERP within a Department of Defense logistics environment, under which CSC reduced legacy system operations from more than 1,000 non'integrated applications to a single solution; the Air Force rated CSC's performance very good (the Army program manager evaluated CSC's performance as exceptional); (2) a contract under General Dynamics Land Systems' (GDLS) e-Business program, an ERP within a commercial logistics domain, under which CSC implemented an Oracle-based ERP solution; CSC's performance was rated very good by the Air Force; and (3) a contract under a Dupont Chemical Solutions Enterprise ERP program, under which CSC implemented a solution within a commercial logistics domain; CSC's performance was rated exceptional by the Air Force. In addition, the agency determined that five other CSC contracts and seven subcontractor contracts were relevant; the quality of performance was rated by the Air Force as exceptional for six of these, very good for four, and satisfactory for two''CSC's contracts with IRS for the BSM and IFS programs. Five other CSC contracts were considered somewhat relevant; CSC's performance was rated by the Air Force as exceptional for four of these and very good for the fifth. PCAG at 63'111.

IBM's past performance rating of confidence was based on its having –consistently delivered products or services, developed transformational processes, drove best business practices into the operational environment, and developed COTS and/or ERP applications which improved their client's business operations.— PCAG at 57. However, unlike its findings with regard to CSC, the agency found –less evidence— that IBM had performed projects comparable to ECSS, and rated none of IBM's contracts as very relevant. SSD at 5. The agency did identify three relevant contracts that it characterized as medium-scale, moderately complex projects: (1) a contract for a Delta Airlines ERP implementation within a commercial logistics domain; IBM's performance was rated as satisfactory by the Air Force; (2) a subcontractor's contract for a Defense Logistics Agency ERP implementation within a DOD logistics domain; the subcontractor's performance was rated as very good by the Air Force; and (3) a subcontractor's contract for an Oracle ERP implementation within a commercial logistics domain; the subcontractor's performance was rated as exceptional by the Air Force. Twelve additional contracts performed by IBM or its subcontractors were found to be relevant; the performance on eight of these was rated exceptional, on three as very good, and on one as satisfactory. One IBM contract was considered to be only somewhat relevant, with IBM's performance rated as satisfactory.

IBM primarily asserts that CSC encountered significant performance problems under the two IRS contracts, and that it was unreasonable for the agency to rate CSC's performance on those contracts as satisfactory--the same rating IBM received for its relevant Delta contract, during which it experienced no significant performance problems.

This argument is without merit. As noted by the Air Force, while CSC encountered performance problems on the IRS contracts--as reflected in questionnaires completed by IRS officials, interviews with IRS officials, and IRS Contractor Performance Assessment Reports (CPAR)--the record indicates that CSC's rating reflected the fact that IRS was viewed as sharing some responsibility for those problems, and the fact that CSC's performance significantly improved during the contract term. In this regard, IRS's BSM program was a complex, multi-billion dollar effort encompassing 17 major modernization projects, for which CSC served as the prime integrator, responsible for an alliance of subcontractors (including IBM). The record indicates that CSC initially appeared to be overwhelmed by the magnitude and complexity of the project, and was late in recognizing performance problems with several of the subcontractors (including, especially, IBM's performance with respect to IRS's Customer Account Data Engine (CADE)). CSC received several unsatisfactory ratings in IRS CPARs due to resulting delays. The Air Force noted, however, that there were reports of role confusion between IRS and CSC; that the IRS commissioner at the beginning of the program had stated that it was a mistake to believe that any vendor could manage such a huge undertaking without significant input from IRS; and that IRS's associate chief information officer had not only concurred with the commissioner's view, but also attributed some of the failure points to the fact that the contract model was a flawed approach. The Air Force also was advised that there was a significant improvement when CSC took over CADE development from IBM, that IRS was –very happy— with the quality of the CADE end product, and that CSC received an overall excellent rating for quality (as well as a fair rating for schedule and a good rating for cost control) under CSC's most recent BSM CPAR. CSC Performance Documentation, IRS BSM Contract.

Likewise, the record of CSC's performance on the IRS IFS contract, as reported to the Air Force, indicated that CSC had successfully overcome early performance problems. CSC was tasked under the IFS contract with selecting and deploying a new integrated financial system. The record indicated that CSC initially failed to assign staff with the requisite expertise, was late in discovering problems with its subcontractor's performance, and fell behind the required schedule. IRS IFS Performance Questionnaire. However, the record also indicated that some of the delay had resulted from problems with the joint IRS/CSC requirements development process, and that the schedule established by IRS had been very aggressive, so that the time CSC required for the project actually favorably compared to that required for similar projects. Project Notes, IRS IFS. In any case, the record indicated that CSC's performance subsequently improved, and that the project was successfully completed. Indeed, the record indicated that IRS's new integrated financial system resulted in the agency's receiving its first unqualified audit opinion from GAO. CSC Performance Documentation, IRS IFS Contract.

In view of CSC's ultimately successful performance on the IRS contracts, as well as the reported extenuating circumstances (i.e., the magnitude and complexity of the challenges, IRS having contributed to the performance problems, and/or IRS having established an overly aggressive schedule), there is no basis for us to object to the Air Force's rating of CSC's performance on these contracts as satisfactory.[5]

Given the Air Force's determination that CSC had exceptional or very good performance on three very relevant contracts, and that IBM's quotation cited no very relevant contracts, there is no basis for questioning the ratings in this area.

The protest is denied.

Gary L. Kepplinger
General Counsel