B-188272, AUG. 31, 1978, 57 COMP.GEN. 715

B-188272, AUG. 31, 1978, 57 COMP.GEN. 715

CONTRACTS - NEGOTIATION - REQUESTS FOR PROPOSALS - CONSTRUCTION - EQUIPMENT VERIFICATION PROVISIONS PROCUREMENT DOCUMENTS IN "FOUR-STEP" PROCUREMENT ESTABLISHED GOAL FOR MAXIMUM USE OF "TRIED AND TRUE" COMPUTER EQUIPMENT BUT DID NOT NECESSARILY RULE OUT MODIFIED EQUIPMENT BASED ON PREEXISTING TECHNOLOGY OR NEW EQUIPMENT IF BASED ON PREEXISTING EQUIPMENT OR TECHNOLOGY. DOCUMENTS WERE WRITTEN BROADLY ENOUGH TO PERMIT USE OF TRIED TECHNOLOGY OR EQUIPMENT. UNDER LITERAL READING OF PROVISIONS REQUIRING EQUIPMENT VERIFICATION, PREEXISTING TECHNOLOGY-- PROTOTYPE RELATED EQUIPMENT-- WOULD QUALIFY SO LONG AS TECHNOLOGY HAD VERIFIED PERFORMANCE CHARACTERISTICS. CONTRACTS - NEGOTIATION - EVALUATION FACTORS - "TRIED AND TRUE" STANDARD - NEW V. PREEXISTING EQUIPMENT/TECHNOLOGY GIVEN ACCEPTANCE OF AIR FORCE'S INTERPRETATION OF "TRIED AND TRUE" PROVISIONS, FACT THAT SUCCESSFUL OFFEROR PROPOSED RELATIVELY NEW MINICOMPUTER-- BASED ON PROVEN TECHNOLOGY AND USE WITHIN IBM CORPORATION-- SHOULD NOT HAVE DISQUALIFIED PROPOSAL. SIMILAR CONCLUSION APPLIES TO PROPOSED USE OF PREEXISTING COMPILER. "TRIED AND TRUE" EVALUATION STANDARD-- NEVER IDENTIFIED IN REQUEST FOR PROPOSALS (RFP) AS SEPARATE EVALUATION FACTOR-- IS OF AN ENTIRELY SUBJECTIVE CHARACTER. ALL OFFERORS SHOULD HAVE EXPECTED THAT AIR FORCE WOULD NECESSARILY HAVE HAD TO EXERCISE EXTREMELY BROAD DISCRETION IN EVALUATING OFFERORS' EFFORTS UNDER STANDARD. RECORD REVEALS, MOREOVER, THAT PROPOSALS WERE EVALUATED UNDER STANDARD. EQUIPMENT - AUTOMATIC DATA PROCESSING SYSTEMS - SELECTION AND PURCHASE - EVALUATION PROPRIETY GIVEN THAT RFP PROVISION ON "PROGRAMMING LANGUAGES" DID NOT EXPRESSLY REQUIRE-- OR PROHIBIT-- USE OF "HIGH ORDER" PROGRAMMING LANGUAGE, THAT PROVISIONS OF DOD DIRECTIVE 5000.29 DID NOT APPLY TO PROCUREMENT, AND THAT AIR FORCE HAS REFUTED BY FORCE OF ARGUMENT ALLEGED AUTOMATIC SUPERIORITY OF "HIGH ORDER" PROGRAMMING LANGUAGE, VIEW OF IMPLICIT PROCUREMENT REQUIREMENTS FOR "HIGH ORDER" LANGUAGE IS REJECTED. CONTRACTS - SPECIFICATIONS - RESTRICTIVE - UNWARRANTED TO EXTENT THAT PROTESTER OBJECTS TO AIR FORCE'S DETERMINATION THAT LESS RESTRICTIVE SPECIFICATION-- PERMITTING OFFERORS TO USE EITHER "HIGH ORDER" OR "LOW ORDER" PROGRAMMING LANGUAGE-- WILL MEET AIR FORCE'S NEEDS, GROUND OF PROTEST IS NOT FOR REVIEW. CONTRACTS - NEGOTIATION - REQUESTS FOR PROPOSALS - SPECIFICATION REQUIREMENTS - INFORMATION - SPECIFICITY AS PRACTICAL MATTER, IT WOULD HAVE BEEN IMPOSSIBLE TO HAVE OBTAINED FROM COMPETITIVE-RANGE OFFERORS DETAILED INFORMATION NEEDED TO EVALUATE LIFE- CYCLE COSTS DOWN TO MODULE LEVEL SINCE DESIGN OF SOFTWARE TO MODULE LEVEL WOULD NOT OCCUR UNTIL AFTER AWARD. CONTRACTS - NEGOTIATION - FOUR-STEP PROCUREMENT - PROCEDURES - NATIONAL AERONAUTICS AND SPACE ADMINISTRATION V. DOD IN BOTH NATIONAL AERONAUTICS AND SPACE ADMINISTRATION (NASA) AND DEPARTMENT OF DEFENSE (DOD) PROCEDURES THERE ARE STATEMENTS OF NEED TO ALLOW COMPETITIVE-RANGE OFFERORS OPPORTUNITY FOR DISCUSSIONS. BOTH PROCEDURES STRESS NEED, HOWEVER, TO RESTRICT DISCUSSION OF TECHNICAL PROPOSALS TO CLARIFYING OR SUBSTANTIATING PROPOSAL AND SPECIFICALLY PROHIBIT DISCUSSIONS OF TECHNICAL WEAKNESSES (NASA'S TERM) OR DEFICIENCIES (DOD'S TERM) RELATING TO OFFEROR'S LACK OF COMPETENCE, DILIGENCE, INVENTIVENESS, OR LACK OF MANAGEMENT ABILITIES, ENGINEERING OR SCIENTIFIC JUDGMENT. BOTH PROCEDURES ALSO PROVIDE FOR INDEPENDENT COST PROJECTION OF "MOST PROBABLE" COST OF DOING BUSINESS WITH OFFEROR. CONTRACTS - NEGOTIATION - COST, ETC., DATA - "REALISM" OF COST SINCE IT IS FUNDAMENTAL THAT PROPOSED COSTS OF COST-REIMBURSEMENT CONTRACT BE ANALYZED BY GOVERNMENT IN TERMS OF REALISM, APPROVAL HAS BEEN GRANTED TO PROCESS OF AWARD SELECTION BASED ON GOVERNMENT-ADJUSTED COSTS OF PROPOSALS AFTER CLOSE OF NEGOTIATIONS EVEN IN NON-FOUR STEP PROCUREMENTS. CONTRACTS - NEGOTIATION - COMPETITION - CHANGES SUBSEQUENT TO NEGOTIATION - "SOURCE SELECTION" CONCEPT NO SIGNIFICANT DIFFERENCE IS SEEN BETWEEN PROCESS (IN NON-FOUR-STEP PROCUREMENT) WHICH PERMITS COST ADJUSTMENT OF PROPOSED COSTS AFTER CLOSE OF DISCUSSIONS FOR PURPOSES OF AWARD SELECTION-- EVEN THOUGH NO FORMAL ADJUSTMENT OF PROPOSED CONTRACT PRICE IS MADE-- AND FOUR-STEP PROCESS WHICH, THROUGH COST ADJUSTMENT PROCESS, PERMITS CHANGED CONTRACT PRICE IN LINE WITH GOVERNMENT-EVALUATED PRICE. CONTRACTS - NEGOTIATION - COMPETITION - COMPETITIVE RANGE FORMULA - SELECTION BASIS REQUIREMENT IN DOD PROCEDURES THAT SELECTED PROPOSAL MUST MEET GOVERNMENT'S "MINIMUM REQUIREMENTS" IS NOTHING MORE THAN REQUIREMENT THAT- - ASIDE FROM BEING MOST ADVANTAGEOUS PROPOSAL-- PROPOSAL IS TO SATISFY GOVERNMENT'S CORE REQUIREMENTS TO EXTENT THAT PROPOSAL IS IN COMPETITIVE RANGE AND NOT ALL REQUIREMENTS AS PROTESTER INSISTS. CONTRACTS - NEGOTIATION - COMPETITION - DISCUSSION WITH ALL OFFERORS REQUIREMENT - TECHNICAL TRANSFUSION OR LEVELING SINCE (1) SELECTED PROPOSAL WAS RATIONALLY FOUND TO BE IN COMPETITIVE RANGE; (2) DISCUSSIONS COULD NOT HAVE BEEN HELD WITH SELECTED OFFEROR IN CONTESTED AREAS WITHOUT VIOLATING PROCEDURES; (3) APPROPRIATE DISCUSSIONS WITH SELECTED OFFEROR WERE OTHERWISE CONDUCTED; (4) PROTESTER ALLEGES LACK OF DISCUSSION WITH ITSELF LARGELY IN THE ABSTRACT; (5) POST-SELECTION DISCUSSIONS WITH HIGHEST-RATED OFFEROR DID NOT RESULT IN "LEVELING," IT CANNOT BE CONCLUDED AIR FORCE FAILED TO COMPLY WITH REQUIREMENTS OF 10 U.S.C. 2304(G). BASED ON REVIEW OF RECORD, IT IS CONCLUDED THAT AGENCY- EVALUATED COST AND TECHNICAL DIFFERENCES BETWEEN PROPOSALS OF PROTESTER AND SELECTED OFFEROR ARE RATIONALLY FOUNDED.

IN THE MATTER OF GTE SYLVANIA, INC., NOVEMBER 30, 1977 (PUBLISHED AUGUST 31, 1978):

GTE SYLVANIA, INC., HAS PROTESTED THE DEPARTMENT OF THE AIR FORCE AWARD OF A COST-PLUS-INCENTIVE-FEE CONTRACT UNDER REQUEST FOR PROPOSALS (RFP) F19628-76-4-0102 TO INTERNATIONAL TELEPHONE AND TELEGRAPH CORPORATION (ITT) FOR THE "SATIN IV SYSTEM." ("SATIN IV" IS THE AIR FORCE DESIGNATION FOR THE STRATEGIC AIR COMMAND (SAC) AUTOMATED TOTAL INFORMATION NETWORK, A COMMUNICATION SYSTEM DESIGNED TO CONNECT FIVE MAJOR CENTERS AND SUBCENTERS WITH SAC, INCLUDING INDIVIDUAL MISSILE LAUNCH CONTROL CENTERS. THE SATIN IV SYSTEM WILL BE A COMPLEX OF COMPUTERS, TERMINALS AND RELATED SWITCHING EQUIPMENT CAPABLE OF SIMULTANEOUSLY SENDING, RECEIVING AND SORTING MESSAGES.)

DURING THE PENDENCY OF THE PROTEST, SYLVANIA FILED SUIT IN THE U.S. DISTRICT COURT FOR THE DISTRICT OF COLUMBIA, GTE SYLVANIA, INC. V. REED, CIVIL ACTION NO. 77-0519, REQUESTING THE COURT, AMONG OTHER THINGS, TO "FIND THAT THE AWARD TO ITT * * * IS * * * ILLEGAL AND VOID." THE REQUESTED FINDING, ACCOMPANIED BY MOTIONS FOR APPROPRIATE INJUNCTIVE RELIEF, WAS PREFACED WITH EXTENSIVE DISCUSSION OF THE ISSUES RAISED IN THE PROTEST BEFORE OUR OFFICE. ON MARCH 28, 1977, THE COURT DENIED PLAINTIFF'S REQUEST FOR A TEMPORARY ORDER, BUT OTHERWISE EXPRESSED INTEREST IN HAVING THE VIEWS OF OUR OFFICE ON THE PROTEST. SINCE THE COURT WANTS OUR VIEWS, WE WILL CONSIDER THE ISSUES RAISED EVEN THOUGH ONE OR MORE ISSUES MIGHT OTHERWISE BE CONSIDERED UNTIMELY FILED (AS URGED BY THE AIR FORCE) UNDER OUR BID PROTEST PROCEDURES (4 C.F.R.PART 20 (1977)). CONTROL DATA CORPORATION, 55 COMP.GEN. 1019 (1976), 76-1 CPD 276.

THE AIR FORCE, THROUGH ITS ELECTRONIC SYSTEMS DIVISION (ESD), FORMALLY RELEASED THE SATIN IV PROCUREMENT PROGRAM BY ISSUANCE OF THE RFP ON JANUARY 9, 1976. THE RFP INFORMED OFFERORS THAT THE PROCUREMENT WAS DIVIDED INTO TWO MAIN PHASES:

PHASE I CALLS FOR THE CONTRACTOR TO PROVIDE EQUIPMENT, COMPUTER PROGRAMMING (SOFTWARE) AND TEST DATA SUFFICIENT TO SHOW THAT THE SATIN IV SYSTEM IS TECHNICALLY AND ECONOMICALLY FEASIBLE. PHASE II CALLS FOR THE CONTRACTOR TO DEVELOP ADDITIONAL ITEMS, WHILE INSTALLING AND TESTING PRODUCTION EQUIPMENT AND SOFTWARE FOR THE COMPLETED SYSTEM. (UPON SUCCESSFUL COMPLETION OF THE PHASE I EFFORT AND RECEIPT OF FINAL APPROVAL, THE PHASE I CONTRACTOR IS TO BE AWARDED THE PHASE II CONTRACT.)

THE RFP LISTED GENERAL CONSIDERATIONS FOR THE SELECTION OF THE SUCCESSFUL OFFEROR, AS FOLLOWS:

A. UNDERSTANDING OF THE REQUIREMENTS * * *. B. COMPLIANCE WITH REQUIREMENTS * * *. C. SOUNDNESS OF APPROACH * * *. D. SOUNDNESS OF PRODUCTION ENGINEERING AND MANAGEMENT * * *.

E. COMPUTER SECURITY APPROACH-- THE PROPOSAL MUST EMPHASIZE THE APPROACH TO SATISFYING THE MULTILEVEL SECURITY REQUIREMENTS OF THE SATIN IV SYSTEM. THE PROPOSAL MUST INDICATE THE USE OF PREVIOUSLY IMPLEMENTED TECHNOLOGY TO SATISFY THE * * * SECURITY REQUIREMENT.

F. PROGRAM MANAGEMENT * * *.

THE RFP ALSO LISTED THE ORDER OF IMPORTANCE OF THE EVALUATION CRITERIA FOR THE PROCUREMENT AS FOLLOWS:

4.1 TECHNICAL AREA

DESIGN AND PERFORMANCE. COMPUTER SECURITY. COMPUTER PROGRAM FUNCTIONAL DESIGN. RELIABILITY/MAINTAINABILITY/AVAILABILITY. SACCS REPLACEMENT KEYBOARD. COMSEC. INTERFACES. NUCLEAR HARDNESS. HUMAN ENGINEERING. SYSTEM SAFETY.

THE OFFEROR'S PROPOSAL WILL BE ASSESSED ON THE SOUNDNESS OF THE PROPOSED SYSTEM DESIGN AND THE RESPONSIVENESS TO THE SYSTEM SPECIFICATION. STANDARD EQUIPMENT UTILIZING DEMONSTRATED TECHNIQUES IS EXPECTED TO BE USED; THEREFORE, THE PROPOSED DESIGN WILL BE ASSESSED AS TO THE RISK IN TECHNICALLY IMPLEMENTING IT IN THE ALLOTTED TIME AND HOW IT REDUCES KNOWN RISK AREAS IN THE PROGRAM SUCH AS: COMPUTER SECURITY, COMSEC, INTERFACES, MISSILE FIELD REQUIREMENTS, MESSAGE ACCURACY, SYSTEM RESPONSE, AND RECONFIGURATION. PRODUCIBILITY OF THE PROPOSED SPM DESIGN WILL ALSO BE ASSESSED.

4.2 COMPUTER PROGRAM DESIGN AND MANAGEMENT

THE EVALUATION OF THIS AREA WILL BE BROKEN DOWN INTO THE FOLLOWING ITEMS WHICH ARE LISTED IN THEIR ORDER OF IMPORTANCE.

MANAGEMENT OF COMPUTER PROGRAM DEVELOPMENT COMPUTER PROGRAMMING TECHNIQUES LANGUAGE DESCRIPTION ORGANIZATION AND PERSONNEL BACKGROUND AND EXPERIENCE ON OTHER COMPUTER PROGRAM PROJECTS

THE OFFEROR'S PROPOSAL WILL BE ASSESSED ON THE FEASIBILITY OF ITS MANAGEMENT PROGRAM TO ASSURE TIMELY AND COMPLETE COMPUTER PROGRAMS. HIS MANAGEMENT PROGRAM WILL BE ASSESSED AS TO ITS ABILITY TO PROVIDE VISIBILITY OF PROGRESS AND RESPONSE TO CONTINGENCIES. THE OFFEROR'S PROPOSED USES OF DESIGN TECHNIQUES AND LANGUAGE WILL BE ASSESSED FOR RESPONSIVENESS TO THE RFP. THE OFFEROR WILL ALSO BE ASSESSED ON DEMONSTRATED EXPERIENCE ON LIKE PROJECTS.

4.3 SYSTEM OPERABILITY

4.4 COST * * *

PHASE I * * *

THE PROPOSALS WILL BE EVALUATED IN TERMS OF THE TOTAL PROPOSED TARGET COST OF PHASE I * * * TO DETERMINE WHETHER THE ESTIMATE IS REASONABLE * *

EVALUATION WILL BE MADE OF THE REALISM OF PROPOSAL COSTS AS THEY RELATE TO THE OFFEROR'S DESIGN. THIS PART OF THE EVALUATION WILL INCLUDE A COMPARISON OF THE OFFEROR'S PROPOSED COST WITH THE MOST PROBABLE COST DERIVED BY THE GOVERNMENT AFTER CONSIDERING THE OFFEROR'S TECHNICAL APPROACH.

PHASE II * * *

THE COST/PRICE ESTIMATES FOR (PHASE II) WILL BE FULLY EVALUATED TO ESTABLISH THE SATIN IV SYSTEM DESIGN TO COST GOAL * * *.

EVALUATION WILL ALSO BE MADE OF THE CREDIBILITY OF THE ESTIMATED COSTS FOR (PHASE II) * * * (BASED ON) COMPARISON * * * WITH THE MOST PROBABLE COST DERIVED BY THE GOVERNMENT * * *.

PHASE I * * * II * * *

THE CONTRACTING OFFICER WILL DETERMINE AND IDENTIFY DEFICIENCIES CONTAINED IN THE SELECTED OFFEROR'S PROPOSAL, AND DIRECT THE SELECTED OFFEROR TO CORRECT DEFICIENCIES AND ADVISE OF COST IMPACTS RESULTING THEREFROM.

* * * LIFE CYCLE COST (IS) A MAJOR AND IMPORTANT FACTOR IN THE ACQUISITION OF THE SATIN IV SYSTEM. * * * LCC (LIFE CYCLE COST) EVALUATION (WILL CONSIDER) * * *:

THE OFFEROR'S * * * DOCUMENTATION AS TO THE ACCURACY OF HIS DATA INPUTS.

THE OFFEROR'S ABILITY TO PROVE * * * COSTS * * * INVOLVED IN ARRIVING AT THE * * * LCC.

THE OFFEROR'S ABILITY TO CONDUCT AN EFFECTIVE LCC PROGRAM * * *.

4.5 MANAGEMENT

4.6 LOGISTICS

4.7 TEST AND DEPLOYMENT

THE RFP ALSO INCORPORATED DEPARTMENT OF DEFENSE DIRECTIVE 4105.62 WHICH DEFINES A "FOUR-STEP" SOURCE SELECTION PROCESS WHICH WAS TO BE FOLLOWED IN SELECTING THE SUCCESSFUL CONTRACTOR. A SUMMARY OF THE FOUR STEP SELECTION PROCESS IS CONTAINED WITHIN THE DIRECTIVE, AS FOLLOWS:

STEP 1. SEPARATE TECHNICAL PROPOSALS SHALL BE SOLICITED AND EVALUATED AND DISCUSSIONS HELD WITH ALL OFFERORS * * *.

STEP 2. A COST/PRICE PROPOSAL SHALL THEN BE OBTAINED FROM EACH OFFEROR TOGETHER WITH ANY NECESSARY REVISIONS TO CORRECT THE DEFICIENCIES IN THE TECHNICAL PROPOSALS DISCUSSED IN STEP 1. SUBSEQUENT TO THE RECEIPT OF THE COST/PRICE PROPOSALS AND ANY TECHNICAL REVISIONS, THE COMPETITIVE RANGE WILL BE ESTABLISHED. THOSE PROPOSALS OUTSIDE OF THE COMPETITIVE RANGE AT THIS POINT MAY BE ELIMINATED AND THE OFFERORS SO NOTIFIED. MEANINGFUL DISCUSSIONS WILL BE HELD WITH THE REMAINING OFFERORS * * *.

STEP 3. FOLLOWING SUCH DISCUSSIONS, A COMMON CUTOFF DATE FOR THE RECEIPT OF FINAL REVISIONS TO TECHNICAL AND COST/PRICE SUBMITTALS WILL BE ESTABLISHED AND THE REMAINING OFFERORS SO NOTIFIED. AFTER RECEIPT OF ANY REVISED SUBMITTALS, THE PROPOSALS SHALL BE EVALUATED BASED UPON THE OFFEROR'S TOTAL PROPOSAL AND A CONTRACTOR SELECTED FOR NEGOTIATION OF THE CONTRACT.

STEP 4. A DEFINITIVE CONTRACT WILL THEN BE NEGOTIATED WITH THE SELECTED OFFEROR.

TECHNICAL PROPOSALS, CALLED FOR UNDER STEP 1 OF THE SELECTION PROCESS, WERE SUBMITTED BY SYLVANIA AND THREE OTHER OFFERORS, INCLUDING ITT, ON MARCH 23, 1976. STEP 2 COST PROPOSALS WERE SUBMITTED BY THE FOUR OFFERORS ON JUNE 8, 1976, AFTER WHICH THE DEPARTMENT SPENT NEARLY 2 MONTHS IN EVALUATING PROPOSALS.

ON AUGUST 20, 1976, THE DEPARTMENT INFORMED SYLVANIA THAT ITS PROPOSAL WAS FOUND TO BE IN THE COMPETITIVE RANGE FOR THE PROCUREMENT AND THAT, FOLLOWING DISCUSSIONS WITH EACH OF THE OFFERORS WITHIN THE COMPETITIVE RANGE, STEP 3 PROPOSALS WERE TO BE SUBMITTED. FOLLOWING THESE DISCUSSIONS, SYLVANIA SAYS THAT IT SUBMITTED ITS STEP 3 PROPOSAL TO THE DEPARTMENT ON SEPTEMBER 20, 1976. THEREAFTER, THE DEPARTMENT INFORMED SYLVANIA THAT THE SUCCESSFUL OFFEROR WAS ITT.

HISTORY OF THE SOURCE SELECTION

THE AIR FORCE EVALUATION OF SUBMITTED PROPOSALS WAS INITIALLY TO BE MADE BY A SOURCE SELECTION EVALUATION BOARD. THE BOARD FOUND THAT THE ITT PROPOSAL MET OR EXCEEDED ALL STANDARDS AND REQUIREMENTS. ALTHOUGH THE COMPANY CLAIMED THAT ONLY "TRIED AND TRUE" HARDWARE, FIRMWARE AND SOFTWARE WOULD BE USED FOR THE WORK, THE BOARD NOTED THAT CAUTION HAD TO BE EXERCISED WITH RESPECT TO THE CLAIM SINCE ADDITIONAL DEVELOPMENT APPEARED TO BE NECESSARY IN ONE AREA; ALSO A PROPOSED COMPONENT WAS CONSIDERED NOT NEARLY AS ADVANCED AS FIRST SUGGESTED IN THE COMPANY'S INITIAL PROPOSAL. FURTHER, THE BOARD FOUND THAT ITT'S APPROACH TO THE MANAGEMENT OF SOFTWARE DEVELOPMENT WAS WELL DISCIPLINED. BESIDES SHOWING AN EXCELLENT UNDERSTANDING OF PROGRAMMING METHODOLOGIES, THE METHODOLOGIES WERE EXTENDED AND COMPLEMENTED BY OTHER TOOLS, ESPECIALLY THE USE OF AN AUTOMATED SOFTWARE DEVELOPMENT LIBRARY SYSTEM. THE BOARD CONSIDERED ACCEPTABLE ITT'S "DUAL LANGUAGE" APPROACH WHICH INVOLVED THE USE OF A "HIGH ORDER" COMPUTER PROGRAMMING LANGUAGE (COMPILER) AND "LOW ORDER" ASSEMBLY LANGUAGE FOR THE COMPUTER PROGRAM. ADDITIONALLY, THE BOARD NOTED ONE OF ITT'S PROPOSED SUBCONTRACTORS WOULD ESTABLISH A COMPUTER PROGRAM DEVELOPMENT FACILITY THEREBY STRENGTHENING ITT'S PROPOSAL.

NOTWITHSTANDING THE OVERALL JUDGMENT OF THE BOARD THAT ITT'S PROPOSAL MET OR EXCEEDED THE REQUIREMENTS AND STANDARDS OF THE RFP, ITT'S PROPOSAL WAS FOUND TO CONTAIN "SIGNIFICANT WEAKNESSES" IN SYSTEM CONTROL, RESPONSE TIME AND IN THREE OTHER AREAS-- MAINLY DEALING WITH SECURITY AND CERTAIN TESTS. ITT'S INITIAL COST PROPOSAL WAS ADJUSTED-- THROUGH USE OF THE SO- CALLED "PARAMETRIC" COST TECHNIQUE-- BY THE AIR FORCE COST EVALUATORS TO A FINALLY ESTIMATED COST. SIMILARLY, PHASE II COSTS-- INCLUDING SOME ELEMENTS OF LIFE-CYCLE COSTS-- WERE ADJUSTED. BECAUSE OF THE DISCREPANCY BETWEEN ITT PROPOSED COSTS AND AIR FORCE EVALUATED COSTS, ITT'S COSTS PROPOSAL WAS TERMED UNREALISTICALLY LOW. BASED, IN PART, ON THE ANALYSIS, ITT'S TECHNICAL PROPOSAL WAS RATED "ACCEPTABLE."

THE BOARD'S EVALUATION OF SYLVANIA'S PROPOSAL SHOWS THAT, ALTHOUGH THE COMPANY'S PROPOSAL IN AREAS SUCH AS HUMAN ENGINEERING AND SYSTEM SAFETY DEMONSTRATED SYLVANIA'S UNDERSTANDING OF THESE REQUIREMENTS, THE COMPANY'S PROPOSAL IN OTHER AREAS DEMONSTRATED LACK OF SUFFICIENT DETAIL, CONTRADICTIONS AND INCONSISTENCIES. FOR EXAMPLE, THE EVALUATORS FOUND SYLVANIA'S PROPOSAL TO CONTAIN (A) A FRAGMENTED DESIGN APPROACH RESULTING IN LACK OF TECHNICAL CONSISTENCY; (B) A LACK OF INFORMATION REGARDING SYLVANIA'S INNOVATIVE APPROACH TO COMPUTER SECURITY; AND (C) A POOR SHOWING OF HOW THE PROPOSED DESIGN MET "INTERFACE" REQUIREMENTS.

SYLVANIA'S PROPOSAL WAS ALSO CONSIDERED TO SHOW AN EXCELLENT UNDERSTANDING OF MANAGEMENT CONCEPTS AND STRUCTURED PROGRAMMING TECHNOLOGY TO BE USED FOR THE SOFTWARE DEVELOPMENT. THE USE OF "FLEXIBLE ARCHITECTURE," THE USE OF A SINGLE HIGH ORDER LANGUAGE FOR ALL SOFTWARE AND THE EXCELLENT DESIGN DOCUMENTATION APPROACH WERE CONSIDERED TO ENHANCE SYLVANIA'S APPROACH.

INHERENT IN SYLVANIA'S FLEXIBLE APPROACH IN SOFTWARE REQUIREMENTS, THE BOARD FOUND, WERE TWO MAJOR PROBLEMS: (1) FLEXIBILITY OF SOFTWARE REQUIRED STRINGENT CONTROLS; AND (2) THE GENERAL LACK OF GOVERNMENT EXPERIENCE WITH THE APPROACH LEADING TO AN "UNCERTAINTY RISK" AS OPPOSED TO A "THREAT-TYPE RISK."

SYLVANIA'S INITIAL COST PROPOSAL FOR PHASE I WORK WAS ADJUSTED BY THE AIR FORCE TO A FINALLY EVALUATED COST. SIMILARLY, SYLVANIA'S TOTAL SYSTEM COST-- INCLUDING ELEMENTS OF LIFE-CYCLE COST-- WAS ADJUSTED TO REFLECT THE PARAMETRIC ESTIMATE. SYLVANIA'S PROPOSED COSTS WERE CONSIDERED VERY OPTIMISTIC BUT ON THE LOWER RANGE OF COST REALISM. BASED ON THE FOREGOING ANALYSIS, IN PART, THE BOARD RATED SYLVANIA'S TECHNICAL PROPOSAL "MARGINAL."

THE BOARD'S FINDINGS WERE THEN REVIEWED BY A SOURCE SELECTION ADVISORY COUNCIL. THE COUNCIL TERMED THE RELATIVE RANKING OF SYLVANIA AND ITT TO BE RELATIVELY CLOSE. ITT WAS CONSIDERED TO HAVE A SOMEWHAT BETTER OVERALL UNDERSTANDING IN THE TECHNICAL AREA, ONLY A MARGINALLY WEAKER POSITION IN COMPUTER PROGRAM DESIGN AND MANAGEMENT THAN SYLVANIA, AN EXCELLENT VIEW OF SYSTEM OPERATING PROBLEMS, AND THE PROBABILITY OF GENERALLY LESS RISK OF UNKNOWN SCHEDULE PROBLEMS AFTER NEGOTIATIONS. BECAUSE OF THESE VIEWS, THE COUNCIL CONCLUDED THAT ITT'S PROPOSAL PROVIDED THE BETTER FOUNDATION FOR A SUCCESSFUL SATIN IV PROGRAM. THE GENERAL FINDINGS OF THE COUNCIL WERE THAT NONE OF THE PROPOSALS, AS SUBMITTED AND MODIFIED THROUGH STEP 2 PROCEDURES, OFFERED A CLEAR DEMONSTRATION ON THE PART OF THE OFFERORS THAT THEY TOTALLY UNDERSTOOD AND COULD SATISFY THE AIR FORCE'S REQUIREMENTS. BUT THROUGH NEGOTIATIONS WITH ANY OF THE OFFERORS REMAINING IN THE COMPETITIVE RANGE-- INCLUDING SYLVANIA AND ITT-- "DISCREPANCIES" COULD PROBABLY BE CLEARED UP, POINTS OF CONCERN COULD BE EASED, AND A CONTRACT AGREED TO THAT WOULD TECHNICALLY MEET AIR FORCE NEEDS. AS TO SPECIFICS, THE COUNCIL AGREED WITH THE BOARD THAT ITT'S PROPOSAL, WHILE SERIOUSLY DEFICIENT IN AREAS OF SYSTEM CONTROL AND MICROPROGRAMMING DOCUMENTATION, COULD BE CORRECTED THROUGH NEGOTIATIONS. MOREOVER, THE COUNCIL CONCURRED IN THE BOARD'S FINDING THAT ITT'S TECHNICAL APPROACH PRESENTED LOWER RISKS THAN ANY OTHER OFFEROR'S PROPOSAL. OTHER FINDINGS OF THE COUNCIL WHICH EVIDENCE CONCURRENCE IN THE BOARD'S CONCLUSIONS WERE:

(1) ITT'S PROBABILITY OF SUCCESSFUL PERFORMANCE WAS SLIGHTLY HIGHER THAN SYLVANIA'S PROBABILITY OF SUCCESS;

(2) LACK OF SUPPORTING DESIGN DETAIL IN SYLVANIA'S PROPOSAL RAISED UNCERTAINTIES AS TO THE COMPANY'S UNDERSTANDING OF THE REQUIREMENTS; AND

(3) ITT PRESENTED THE BEST OVERALL MANAGEMENT STRUCTURE.

THE SOURCE SELECTION OFFICIAL CONCURRED IN THE ANALYSES OF THE COUNCIL AND BOARD AND SELECTED ITT FOR STEP 4 DISCUSSIONS. AS A RESULT OF THESE DISCUSSIONS, ITT'S PROPOSED COSTS FOR THE PHASE I WORK WERE RAISED-- THROUGH CORRECTION OF DEFICIENCIES-- FROM APPROXIMATELY $23 MILLION TO APPROXIMATELY $32 MILLION. THIS $32 MILLION COST FIGURE WAS WITHIN THE COST PROJECTION FOR THE CORRECTION OF ITT DEFICIENCIES WHICH THE AIR FORCE MADE PRIOR TO THE SELECTION OF THE COMPANY'S PROPOSAL.

ALTHOUGH THE NEGOTIATED PHASE I PRICE FOR ITT WAS HIGHER THAN SYLVANIA'S PROPOSED PRICE OF $29 MILLION, THE AIR FORCE FELT THAT ITT'S PROPOSAL WAS STILL THE MOST DESIRABLE BECAUSE ALL DEFICIENCIES AND UNKNOWN CHARACTERISTICS HAD BEEN REMOVED BY STEP 4 DISCUSSIONS. ON THE OTHER HAND, SYLVANIA'S PROPOSAL (BASED ON ESTIMATED COSTS OF $29 MILLION) CONTAINED A SIGNIFICANT QUANTITY OF DEFICIENCIES. FURTHER, BASED ON AIR FORCE COST PROJECTIONS AND ANALYSIS, THE AIR FORCE FELT SYLVANIA'S PROPOSED COST WOULD INCREASE IN SIMILAR PROPORTION TO ITT'S PROPOSED COST SHOULD STEP 4 DISCUSSIONS BE HELD WITH SYLVANIA. THIS CONCLUSION WAS BASED ON AIR FORCE FINDINGS THAT SYLVANIA DEFICIENCIES AS AN AGGREGATE APPEARED TO BE OF A SIMILAR OVERALL MAGNITUDE TO ITT'S DEFICIENCIES. SINCE THIS EVALUATION CONFIRMED THE ORIGINAL AWARD SELECTION, THE AIR FORCE DECIDED TO PROCEED WITH THE AWARD TO ITT.

SYLVANIA'S PROTEST, AS AMENDED, RAISES THREE BASIC ISSUES: (1) THE COMPUTER AND RELATED SOFTWARE PROPOSED BY ITT ARE NOT "TRIED AND TRUE" AND FAILED TO MEET THE RFP REQUIREMENTS; (2) THE AIR FORCE'S SELECTION OF A SYSTEM CONTAINING A "LOW ORDER LEVEL" (LOL) PROGRAMMING LANGUAGE WAS ARBITRARY AND A PRODUCT OF THE AIR FORCE'S FAILURE TO EVALUATE PROPERLY THE SOFTWARE ASPECTS OF THE PROPOSALS; AND (3) THE AIR FORCE AND ITT REPRESENTATIVES NEGOTIATED MAJOR, MATERIAL CHANGES TO THE ITT PROPOSAL DURING STEP 4 OF THE SATIN IV PROCUREMENT PROCESS IN VIOLATION OF DOD DIRECTIVE 4105.62.

ISSUE 1-- "TRIED AND TRUE" REQUIREMENT

SYLVANIA ARGUES THAT ITT'S PROPOSED USE OF THE "IBM SERIES/1 COMPUTER AND ITS ASSOCIATED SOFTWARE * * * IS NEITHER 'TRIED AND TRUE' NOR 'VERIFIED IN A MILITARY OR COMMERCIAL ENVIRONMENT' AS REQUIRED BY THE RFP." SYLVANIA DRAWS ATTENTION TO THE FOLLOWING RFP REQUIREMENTS AND PROVISIONS IN "OTHER PROCUREMENT DOCUMENTS:"

PROGRAM MANAGEMENT PLAN, PARAGRAPH 1.1.5:

* * * AS A RESULT, THE PROCUREMENT WILL BE: A. OFF-THE-SHELF AS FAR AS POSSIBLE; B. MODIFICATION OF OFF-THE-SHELF EQUIPMENT AS NECESSARY TO MEET OPERATIONAL REQUIREMENTS (WITHIN STATE-OF-THE-ART) AND; C. NEW HARDWARE/SOFTWARE DESIGN (WITHIN STATE-OF-THE-ART) ONLY WHERE NECESSARY. REQUIREMENTS FOR DESIGN OF NEW HARDWARE SHOULD BE OF A VERY LOW MAGNITUDE. DEVELOPMENT OF NEW TECHNOLOGIES WILL NOT BE REQUIRED. * * *

INSTRUCTIONS FOR PROPOSAL PREPARATION, SECTION 1.1:

IT IS INTENDED THAT MAXIMUM USE OF "TRIED AND TRUE" EQUIPMENTS/COMPUTER PROGRAMS (THE DESIGN OF WHICH IS KNOWN AND THE PERFORMANCE CHARACTERISTICS OF WHICH HAVE BEEN VERIFIED IN A MILITARY OR COMMERCIAL ENVIRONMENT) BE UTILIZED THROUGHOUT THE ENTIRE ACQUISITION OF THE SATIN IV PROGRAM.

INSTRUCTIONS FOR PROPOSAL PREPARATION, SECTION 6.3.1.2:

* * * THE OFFEROR SHALL NOT PROPOSE ANY NEW COMPUTER PROGRAMMING LANGUAGE (ASSEMBLY LANGUAGE OR HIGH ORDER LANGUAGE) OR ANY NEW LANGUAGE TRANSLATOR. THIS DOES NOT PRECLUDE MODIFYING EXISTING TRANSLATORS OR USING A COMPILER GENERATOR. * * *

EVALUATION FACTORS FOR AWARD, SECTION 3.0E:

* * * THE PROPOSAL MUST INDICATE USE OF PREVIOUSLY IMPLEMENTED TECHNOLOGY TO SATISFY THE SATIN IV MULTILEVEL SECURITY REQUIREMENT. * * *

EVALUATION FACTORS FOR AWARD, SECTION 4.1:

* * * THE OFFEROR'S PROPOSAL WILL BE ASSESSED ON THE SOUNDNESS OF THE PROPOSED SYSTEM DESIGN AND THE RESPONSIVENESS TO THE SYSTEM SPECIFICATION STANDARD EQUIPMENT UTILIZING DEMONSTRATED TECHNIQUES IS EXPECTED TO BE USED; THEREFORE, THE PROPOSED DESIGN WILL BE ASSESSED AS TO THE RISK IN TECHNICALLY IMPLEMENTING IT IN THE ALLOTTED TIME AND HOW IT REDUCES KNOWN RISK AREAS IN THE PROGRAM SUCH AS: * * *

PREPROPOSAL BRIEFING, ATTACHMENT #1, DATED 5 FEB. 1976, SECOND FULL PARAGRAPH:

AND THIS LEADS TO THE THIRD POINT-- THERE IS TO BE NO NEW TECHNOLOGY DEVELOPED TO IMPLEMENT SECURITY FEATURES INTO THE SOFTWARE AND HARDWARE OF THE SYSTEM. ESPECIALLY IN REGARD TO THE HARDWARE, THE WAYS IN WHICH THE CONTRACTOR CHOOSES TO COMBINE EXISTING HARDWARE TECHNIQUES OR MECHANISMS WITH THE SOFTWARE MAY BE UNIQUE, BUT THE ACTUAL HARDWARE MUST BE HARDWARE THAT HAS BEEN PREVIOUSLY IMPLEMENTED.

STATEMENT OF WORK, SECTION 1010.03:

COMPLY WITH ATTACHMENT 3 TO THIS SOW, TASKING AND RELATIONSHIP WITH CCPC. DELIVER TO CCPC HARDWARE AND A COMMERCIALLY AVAILABLE GENERAL PURPOSE OPERATING SYSTEM THAT WILL ALLOW CCPC TO DEVELOP, PRODUCE, AND TEST APPLICATION SOFTWARE.

SYLVANIA ARGUES THAT THE "INTENT OF THE DEPARTMENT TO ENFORCE THESE REQUIREMENTS IS CONTAINED IN THE (LITIGATION-RELATED) TESTIMONY OF COLONEL WOODRUFF"-- ONE OF THE AIR FORCE'S EVALUATORS FOR THE PROCUREMENT-- AT PAGES 125-149 OF THE TESTIMONY.

RECOGNIZING THESE REQUIREMENTS, WHICH SYLVANIA CONSIDERS TO BE A CLEAR PREFERENCE FOR "MINIMIZATION OF RISKS IN THE SYSTEM" AND A DIRECTION TO OFFERORS "NOT (TO) SEEK THE DEVELOPMENT OF NEW HARDWARE AND SOFTWARE," SYLVANIA SAYS THAT IT PROPOSED THE "BURROUGHS MODEL A MACHINE." THIS MACHINE, SYLVANIA FEELS, IS BETTER-- IN STATE-OF-THE ART AND PRIOR RECORD- - THAN ANY OTHER COMPUTER IN A "MILITARY OR COMMERCIAL ENVIRONMENT." CONTRAST, THE IBM MACHINE PROPOSED BY ITT IS CONSIDERED TO BE "COMMERCIALLY COMPETITIVE FOR RELATIVELY LOW ORDER REQUIREMENTS AND NOT WITH THE MORE EXTENSIVE SATIN IV APPLICATIONS IN MIND." TO AMPLIFY ITS ARGUMENT THAT THE IBM MACHINE IS NOT "TRIED AND TRUE" SYLVANIA ARGUES:

AT THE TIME ITT SUBMITTED ITS "SATIN IV PROPOSAL, THE IBM SERIES/1 PROCESSOR INCORPORATED IN THE PROPOSAL WAS NOT IN COMMERCIAL USE. IBM HAD NOT EVEN ANNOUNCED ITS AVAILABILITY AT THAT TIME. SPECIFICALLY, THE SERIES/1 MACHINE HAD NEVER BEEN BUILT AND USED IN EITHER A MILITARIZED OR COMMERCIAL APPLICATION. THE CS-1 PROCESSOR (THE MILITARIZED VERSION OF THE SERIES/1) HAS NOT BEEN BUILT TO THIS DATE. NEITHER THE SERIES/1 MACHINE NOR ITS MILITARIZED COUSIN THE CS-1 CAN, THEREFORE, BE CONSIDERED EITHER "OFF-THE-SHELF" OR "TRIED AND TRUE," AND THE ITT PROPOSAL INCORPORATING THIS EQUIPMENT FAILS TO MEET THE REQUIREMENTS OF THE RFP. SYLVANIA WOULD EMPHASIZE THAT THE PROCESSOR (COMPUTER) IS THE DRIVING, CRITICAL COMPONENT OF THE SYSTEM WITHOUT WHICH THE SYSTEM COULD NOT OPERATE. ALL OTHER EQUIPMENT IN THE SYSTEM IS PERIPHERAL TO AND COMPLETELY DEPENDENT UPON THE COMPUTERS TO WHICH THE STANDARD OF "TRIED AND TRUE" SHOULD HAVE BEEN STRICTLY APPLIED.

THE SOFTWARE ASSOCIATED WITH THE SERIES/1 MACHINE IS EQUALLY "UNTRIED." AT THE TIME THE ITT PROPOSAL WAS SUBMITTED, THE SOFTWARE IT PROPOSED WAS NON-EXISTENT OR AS A MINIMUM HAD NEVER BEEN UTILIZED IN EITHER A MILITARY OR COMMERCIAL CONTEXT. THE MACHINE-ORIENTED, LOW ORDER LANGUAGE (LOL) REQUIRED TO PROGRAM THE NEW SERIES/1 MACHINE IS ITSELF A NEW LANGUAGE AND, WHEN PROPOSED BY ITT, CONSTITUTED A BLATANT VIOLATION OF THE RFP WHICH PROHIBITED THE PROPOSAL OF A NEW COMPUTER PROGRAMMING LANGUAGE.

THE AIR FORCE REPLIES (CONTAINED IN WRITTEN REPORTS DATED MARCH 21, AND JUNE 10, 1977) TO ISSUE ONE AND SYLVANIA'S COUNTER-RESPONSE (MAY 17, 1977) ARE SUMMARIZED AS FOLLOWS:

SYLVANIA:

(1) THE QUOTED RFP DOCUMENTS NOWHERE REFER TO "TRIED AND TRUE TECHNOLOGY" AS THE DESIRED STANDARD BUT RATHER "TRIED AND TRUE" EQUIPMENT. ALL NEW EQUIPMENT MUST BE BASED ON PRECEDENT-- EXISTING TECHNOLOGY-- AND THEREFORE ANY NEW ITEM OF HARDWARE WOULD MEET THE AIR FORCE'S TONGUE-IN-CHEEK CHARACTERIZATION OF THE INTENT OF THE RFP. THE AIR FORCE ATTEMPTED TO IGNORE THE "TRIED AND TRUE" REQUIREMENT BY DEFINING IT SO AS TO BE MEANINGLESS. IT ADMITS THAT THE IBM MACHINE IS A NEW EQUIPMENT. MOREOVER, THE DEFECT IN THE AIR FORCE EVALUATION APPROACH AFFECTS BOTH ITT'S PROPOSED HARDWARE (SERIES/EQUIPMENT AND COMPILER) AND SOFTWARE.

SYLVANIA DETERMINED PRIOR TO SUBMITTING A PROPOSAL, THAT THE IBM MACHINE DID NOT MEET THE "TRIED AND TRUE" REQUIREMENTS OF THE RFP. SYLVANIA ALSO CONSIDERED A BURROUGHS MACHINE COMPARABLE TO THE IBM SERIES/1 UNIT BUT REJECTED IT AS NOT BEING "TRIED AND TRUE."

AIR FORCE:

(1) AN ANALYSIS OF THE ITT PROPOSAL SHOWS NOT ONLY THAT IT IS RESPONSIVE TO THE REQUIREMENTS, BUT ALSO THAT IT SATISFACTORILY ACHIEVED THE OTHER GOALS CITED BY SYLVANIA. THE MINICOMPUTER OFFERED IN THE ITT PROPOSAL (REFERRED TO AS THE SERIES/1 BY SYLVANIA) IS DERIVED FROM THE IBM 4955 COMMERCIAL PROCESSOR AND MEMORY. THIS MODEL IS RELATIVELY NEW, BUT IS BASED ON PROVEN TECHNOLOGY WHICH HAS BEEN SUCCESSFULLY MILITARIZED IN OTHER DEFENSE PROGRAMS. THIS MINICOMPUTER IS NOW ON THE COMMERCIAL MARKET AND REQUIRES NO ADDITIONAL DEVELOPMENT FOR SATIN IV BUT FOR CONVERSION TO MIL PACKAGING. NEWLY DEVELOPED TECHNOLOGY IS NOT A PART OF THE PROPOSAL. FURTHERMORE, THE SUBSIDIARY EQUIPMENTS (TAPE DEVICES, DISCS, MODEMS, ETC.), MORE NUMEROUS IN NUMBER THAN THE MINICOMPUTERS TO BE USED, ARE ESSENTIALLY STANDARD, OFF-THE SHELF EQUIPMENTS. THEREFORE, THE ITT PROPOSAL FULLY UTILIZES "TRIED AND TRUE" TECHNOLOGY, AS REQUIRED, AND ALSO PROVIDES FOR THE EXTENSIVE USE OF EXISTING EQUIPMENTS OTHER THAN THE SERIES/1 MINICOMPUTER. FURTHERMORE, THERE IS NO NEW DEVELOPMENT FOR THE SATIN IV PROGRAM. THE DEVELOPMENT OF THE SERIES/1 MINICOMPUTER WAS A PRIVATE EXPENSE AND HAS PRECEDED ANY SATIN IV PROCUREMENT.

IN SHORT, ITT'S PROPOSAL IS BASED ON PROVEN TECHNOLOGY AND IS COMPOSED OF STANDARD EQUIPMENT UTILIZING DEMONSTRATED TECHNIQUES AS IS REQUIRED BY THE RFP, RATHER THAN BEING COMPLETELY DEPENDENT UPON USE OF AN UNTRIED COMPUTER AS SYLVANIA HAS ALLEGED. THE SOFTWARE IS SIMILARLY DERIVED FROM PROVEN TECHNOLOGY.

AS TO SYLVANIA'S ANALYSIS OF PROCUREMENT DOCUMENTS, THE PROTESTER RELIES ON A NUMBER OF EXCERPTS TO ESTABLISH THE SUPPOSED REQUIREMENT FOR "TRIED AND TRUE," OFF-THE-SHELF HARDWARE AND SOFTWARE. TAKEN TOGETHER, THESE CHARACTERIZATIONS ESTABLISH GOALS TO BE WORKED TOWARD RATHER THAN RIGID REQUIREMENTS THAT THE ENTIRE SYSTEM BE "TRIED AND TRUE." THESE EXCERPTS SHOW THAT SATIN IV WAS NOT TO BE A RESEARCH AND DEVELOPMENT EFFORT IN THAT NEW TECHNOLOGIES WERE NOT TO BE DEVELOPED. EACH OFFEROR WAS ENCOURAGED TO MAXIMIZE THE USE OF "TRIED AND TRUE" EQUIPMENTS/COMPUTER SOFTWARE. HOWEVER, IT IS CLEAR THAT MODIFIED AND/OR NEW EQUIPMENTS COULD BE USED WHERE NECESSARY. THERE IS NO EXISTING HARDWARE/SOFTWARE THAT CAN PERFORM THE SATIN IV FUNCTION AS IS.

SYLVANIA:

(2) MR. WILLIAM C. JANOFSKY, WHO WAS HEAD OF THE PANEL CHARGED WITH THE EVALUATION OF COMPUTER PROGRAM DESIGN AND MANAGEMENT, TESTIFIED THAT, AT LEAST WITH RESPECT TO SOFTWARE, THE "TRIED AND TRUE" NATURE OF THE PROPOSALS WAS NOT EVEN EVALUATED OR SCORED. FURTHER, THE SOFTWARE FOR THE SERIES/1 IS PRACTICALLY NON-EXISTENT SINCE THE FIRST IBM MACHINES WERE NOT DELIVERED UNTIL AFTER THE SUBMISSION OF SATIN IV PROPOSALS. ONE COMMERCIAL CUSTOMER HAS TESTIFIED THAT THE DELIVERED IBM MACHINE WAS ACCOMPANIED BY "SKIMPY" SOFTWARE. THERE IS NO WAY THAT THIS BAREBONES ASSORTMENT OF SOFTWARE COULD MEET THE "VERIFIED IN A MILITARY OR COMMERCIAL ENVIRONMENT" PROVISION OF THE RFP.

SYLVANIA'S PROPOSAL WAS PREPARED SO AS TO MAKE THE MAXIMUM USE OF EXISTING SOFTWARE AVAILABLE FROM OTHER APPLICATIONS. MOREOVER, THE SATIN IV SOFTWARE PACKAGE IS COMPOSED OF A NUMBER OF MAJOR COMPONENTS WHICH WERE NOT NECESSARILY UNIQUE TO SATIN IV. FOR EXAMPLE, EXISTING COMPONENTS SUCH AS A REAL-TIME OPERATING SYSTEM AND THE SOFTWARE NEEDED TO ACHIEVE A REAL- TIME MULTI-PROCESSOR CAPABILITY COULD BE ADAPTED FROM OTHER APPLICATIONS. SYLVANIA DID ADAPT THESE TRIED COMPONENTS. BUT SYLVANIA'S EFFORTS TO USE TRIED SOFTWARE WERE NOT RECOGNIZED-- NOTWITHSTANDING THE AIR FORCE'S EFFORTS TO ENCOURAGE OFFERORS TO MINIMIZE RISKS IN ALL PROPOSED AREAS.

IN ATTEMPTING TO COMPLY WITH THE IMPORTANT HARDWARE AND SOFTWARE "TRIED AND TRUE" REQUIREMENT, SYLVANIA MADE NUMEROUS TRADEOFFS IN THE COMPUTERS TO BE USED, THE FORM OF THE SOFTWARE AND EVEN AS TO SUBCONTRACTORS WHICH WOULD BE USED. BUT THE AIR FORCE IGNORED THE REQUIREMENTS AND SYLVANIA'S EFFORTS. AIR FORCE:

(2) SYLVANIA HAS MISUNDERSTOOD MR. JANOFSKY'S TESTIMONY. WHAT MR. JANOFSKY SAID WAS THAT HIS PANEL DID NOT EVALUATE UNDER THE "TRIED AND TRUE" TEST. SYLVANIA IGNORES MR. JANOFSKY'S PRIOR TESTIMONY IN WHICH HE POINTED OUT THAT HIS PANEL WAS CONCERNED WITH SOFTWARE MANAGEMENT AND, THEREFORE, WAS NOT CONCERNED WITH "TRIED AND TRUE." ANOTHER EVALUATOR, CAPTAIN FURST, HAS EXPLAINED THAT "TRIED AND TRUE" WAS NOT RELEVANT TO CERTAIN PARTS OF THE SATIN IV APPLICATION SOFTWARE SINCE NO EXISTING SOFTWARE COULD HAVE MET SATIN IV REQUIREMENTS IN THESE AREAS. INDEED, IN THE APPLICATION SOFTWARE NO OFFEROR PROPOSED PREEXISTING SOFTWARE, AND THAT OFFEROR COMING CLOSEST TO SUCH AN OFFERING WAS NOT SYLVANIA. MOREOVER, THE QUESTION OF "TRIED AND TRUE" WAS CONSIDERED BY THE AIR FORCE, NAMELY: SOFTWARE, EXCEPT APPLICATION SOFTWARE, BY CAPTAIN FURST'S PANEL AND HARDWARE BY THE HARDWARE SUBPANEL. NEITHER TOOK THE PRECISE APPROACH WHICH SYLVANIA IMPLIES SHOULD HAVE BEEN USED BECAUSE "TRIED AND TRUE" WAS A GOAL, NOT A REQUIREMENT. THE EXTENT TO WHICH THE GOAL WAS MET BY EACH OFFEROR PROVIDED ONE OF THE MANY EVALUATION INPUTS ANALYZED.

SYLVANIA:

(3) THE "UNTRIED NATURE OF THE IBM SOFTWARE IS SHOWN BY THE PROPOSED USE OF "PL-1" PROGRAMMING LANGUAGE IN THAT, AT THE TIME ITT SUBMITTED ITS PROPOSAL AND TODAY, THERE EXISTS NO PL-1 COMPILER FOR THE IBM MACHINE. MOREOVER, SINCE IBM RECENTLY ANNOUNCED THAT A PL-1 COMPILER WOULD NOT BE AVAILABLE UNTIL APRIL 1978, IT IS CLEAR THAT THE 1978 COMPILER WILL BE DEVELOPED AT AIR FORCE EXPENSE. FURTHER, IT HAS BEEN REVEALED THAT A SOFTWARE OPERATING SYSTEM WILL NOT BE AVAILABLE UNTIL LATE 1977 DESPITE THE REQUIREMENT THAT IT BE PROVIDED WITHIN 5 MONTHS OF CONTRACT AWARD.

AIR FORCE:

(3) SYLVANIA'S ASSUMPTION THAT ITT HAS NOT YET DEVELOPED A COMPILER FOR PL-1 IS INCORRECT. THE ITT PROPOSAL USES A MODIFIED PREEXISTING COMPILER AND COMPLIES WITH THE RFP REQUIREMENT THAT: "THE OFFEROR SHALL NOT PROPOSE ANY NEW COMPUTER PROGRAMMING LANGUAGE * * *. THIS DOES NOT PRECLUDE MODIFYING EXISTING TRANSLATORS OR USING A COMPILER GENERATOR."

SYLVANIA:

(4) THE LOW ORDER COMPUTER LANGUAGE PROPOSED BY ITT IS DESIGNED TO OPERATE ON A "NEW" MACHINE AND IS, THEREFORE, A "NEW" LANGUAGE PROHIBITED BY THE RFP.

AIR FORCE:

(4) FOR THE IBM COMPUTER, THE LANGUAGE PROPOSED IS ASSEMBLY LANGUAGE. WHILE THE LANGUAGE MIGHT BE CATEGORIZED AS "NEW" WHEN COMPARED TO ALGOL OR PL-1, IT IS OFF-THE-SHELF AS FAR AS SATIN IV IS CONCERNED, SINCE IT IS ONE THAT IS IN EXISTENCE AND USED WITH PROCESSORS RIGHT NOW.

ANALYSIS-- ISSUE ONE

THE PROCUREMENT DOCUMENTS CITED BY SYLVANIA FOR THE PROPOSITION THAT THE AIR FORCE INTENDED A FIXED REQUIREMENT FOR "TRIED AND TRUE" HARDWARE AND SOFTWARE-- THAT IS, COMPLETELY DEVELOPED, PREEXISTING, OFF THE-SHELF MACHINERY AND PROGRAMMING-- DO NOT, IN OUR VIEW, SUPPORT THE PROPOSITION ADVANCED. INSTEAD, WE AGREE WITH THE AIR FORCE VIEW THAT, IN THE MAIN, THE DOCUMENTS ESTABLISHED A GOAL FOR MAXIMUM USE OF "TRIED AND TRUE" EQUIPMENT, BUT DID NOT NECESSARILY RULE OUT MODIFIED EQUIPMENT BASED ON PREEXISTING TECHNOLOGY OR NEW EQUIPMENT IF BASED ON PREEXISTING EQUIPMENT OR TECHNOLOGY. NOR DO WE AGREE WITH SYLVANIA THAT THE "TRIED AND TRUE" STATEMENT REFERRED ONLY TO EXISTING EQUIPMENT RATHER THAN TO EXISTING TECHNIQUES OR EXISTING TECHNOLOGY.

FOR EXAMPLE, THE PROGRAM MANAGEMENT PLAN PERMITTED MODIFICATION OF EXISTING EQUIPMENT AS WELL AS A NEW HARDWARE/SOFTWARE DESIGN (WHERE NECESSARY). IF THERE WERE A FIXED REQUIREMENT FOR "TRIED AND TRUE" EQUIPMENT, IT IS OBVIOUS THAT THE CITED PERMISSION WOULD NOT HAVE BEEN ALLOWED. SIMILARLY, THE PHRASES INSTRUCTING OFFERORS TO PROPOSE "DEMONSTRATED TECHNIQUES" IN RESPONDING TO THE TECHNICAL CRITERIA AND TO USE "PREVIOUSLY IMPLEMENTED TECHNOLOGY TO SATISFY THE * * * SECURITY REQUIREMENT" ALSO SUPPORT THE VIEW THAT AN OFFEROR COULD PROPERLY RESPOND TO THE "TRIED AND TRUE" GOAL BY PROPOSING PREVIOUSLY TRIED TECHNOLOGY WHICH MIGHT NOT NECESSARILY BE LINKED TO PREVIOUSLY TRIED EQUIPMENT COMPLETELY IDENTICAL TO THE PROPOSED EQUIPMENT. NEITHER DO WE AGREE WITH SYLVANIA'S VIEW THAT PERMITTING USE OF "TRIED AND TRUE" TECHNOLOGY AS OPPOSED TO ACCEPTING "TRIED AND TRUE," PREVIOUSLY USED EQUIPMENT RENDERS THE "TRIED AND TRUE" PROVISIONS MEANINGLESS. IN OUR VIEW, THE PROVISIONS WERE WRITTEN BROADLY ENOUGH TO PERMIT USE OF TRIED TECHNOLOGY OR EQUIPMENT. WE FIND NOTHING NECESSARILY INCONSISTENT OR IMPROPER IN THIS APPROACH.

FURTHER, ALTHOUGH SYLVANIA READS THE REQUIREMENT THAT THE PERFORMANCE CHARACTERISTICS OF PROPOSED EQUIPMENTS/COMPUTER PROGRAMS OF KNOWN DESIGN WERE TO HAVE BEEN "VERIFIED IN A COMMERCIAL OR MILITARY ENVIRONMENT" TO MEAN THAT THE ACTUAL EQUIPMENT/PROGRAMS WERE TO BE SO VERIFIED, WE DO NOT AGREE THAT THE LITERAL READING OF THE PROVISIONS SUPPORTS THAT VIEW. ALL THAT IS REQUIRED UNDER THIS PROVISION IS THAT PERFORMANCE CHARACTERISTICS OF KNOWN DESIGN-- AS CONTRASTED WITH THE ACTUAL EQUIPMENT/PROGRAMS-- BE SO VERIFIED. UNDER THE LITERAL READING OF THE PROVISION, WE AGREE THAT PREEXISTING TECHNOLOGY-- PROTOTYPE RELATED EQUIPMENT-- WOULD QUALIFY SO LONG AS THE TECHNOLOGY HAD VERIFIED PERFORMANCE CHARACTERISTICS WHICH WOULD BE PRESENT IN THE DELIVERED EQUIPMENT/PROGRAMS.

FINALLY, WE SEE NOTHING IN THE RECORD OF THE LITIGATION-RELATED TESTIMONY OF COLONEL WOODRUFF WHICH IS NECESSARILY INCONSISTENT WITH THIS INTERPRETATION. AS WAS STATED BY COLONEL WOODRUFF ON PAGE 130 OF THE TESTIMONY:

BECAUSE OF THE PHILOSOPHIES, THAT WE WANTED TO OBVIOUSLY DERIVE THE MOST MODERN TECHNOLOGIES AND THE MOST MODERN CAPABILITIES IN TERMS OF HARDWARE TECHNOLOGIES FOR OUR SYSTEM, BUT WANTED TO BE CAREFUL THAT WE DID NOT BURDEN THE SYSTEM WITH DEEP RESEARCH AND THE DEVELOPMENT AND THAT KIND OF THING.

HAVING THIS KIND OF VERBIAGE, IT GAVE THE OFFEROR THE OPPORTUNITY TO OFFER TO US HIS BEST BALANCE BETWEEN STATE OF THE ART AND MODERN EQUIPMENT WITHOUT PUSHING IT INTO THE R&D REALM THAT WE DIDN'T WANT TO GET INTO.

GIVEN OUR ESSENTIAL ACCEPTANCE OF THE AIR FORCE'S INTERPRETATION OF THE CITED PROVISIONS, THE FACT THAT ITT PROPOSED A RELATIVELY NEW MINICOMPUTER -- BASED ON PROVEN TECHNOLOGY (AN ASSERTION NOT CONTRADICTED BY SYLVANIA)- - SHOULD NOT OF ITSELF DISQUALIFY THE ITT PROPOSAL UNDER THE STATED PROVISIONS. UNDER THIS VIEW, THE FACT THAT AT THE TIME PROPOSALS WERE SUBMITTED THE MINICOMPUTER WAS ONLY BEING USED WITHIN IBM IS NOT DECISIVE, SINCE THE MACHINE WAS BASED ON PREEXISTING, PROVEN TECHNOLOGY. MOREOVER, GIVEN THE VAGUENESS OF THE "VERIFIED IN A COMMERCIAL OR MILITARY ENVIRONMENT" TEST, WE CANNOT CONCLUDE THAT TESTING WITHIN IBM ITSELF, OR WITHIN ANY OTHER CONCERN, IS NOT REQUIRED VERIFICATION. ADDITIONALLY, TO THE EXTENT THAT PROVEN TECHNOLOGY SUPPORTED THE ITT-SPONSORED MINICOMPUTER, WE THINK MERIT WOULD JUSTIFIABLY BE ACCORDED THE PROPOSAL. SIMILARLY, ITT'S PROPOSED USE OF A MODIFIED, PREEXISTING /1/ COMPILER AND ASSOCIATED PROGRAMMING LANGUAGE IS NOT CONTRARY TO THE RFP PROVISIONS AND COULD IN FACT EARN MERIT FOR THE PROPOSAL TO THE DEGREE THE PREEXISTING COMPILER AND ASSOCIATED TECHNOLOGY WERE PROVEN.

FINALLY, WE DO NOT AGREE WITH SYLVANIA'S ASSERTION THAT THE "TRIED AND TRUE" STANDARD WAS NOT EVALUATED. FIRST, LET US BE CLEAR AS TO HOW THE RFP PORTRAYED "TRIED AND TRUE" AS AN EVALUATION STANDARD. THE STANDARD IS NEVER IDENTIFIED AS A SEPARATE EVALUATION FACTOR-- THE STANDARD IS ALWAYS FOUND DESCRIBED WITHIN SOME OTHER EVALUATION CRITERION. FOR EXAMPLE, THE STANDARD OF EMPLOYING "PREVIOUSLY IMPLEMENTED TECHNOLOGY" FOR THE SECURITY REQUIREMENT IS FOUND IN THE SECOND SENTENCE OF THE "COMPUTER SECURITY APPROACH" GENERAL EVALUATION STANDARD. SIMILARLY, THE REFERENCE TO "STANDARD EQUIPMENT UTILIZING DEMONSTRATED TECHNIQUES" IS IN THE SECOND SENTENCE OF THE "TECHNICAL" EVALUATION FACTOR AND IS NOT EVEN LISTED AS ONE OF THE 11 SPECIFIC SUBCRITERIA UNDER THE FACTOR; RATHER, THE REFERENCED STANDARD IS IDENTIFIED AS BEING LINKED TO A "RISK ASSESSMENT" JUDGMENT IN CERTAIN AREAS SOME OF WHICH-- FOR EXAMPLE, MESSAGE ACCURACY-- ARE NOT EVEN FOUND AS LISTED SUBCRITERIA WITHIN THE "TECHNICAL" EVALUATION FACTOR. FURTHER, THE "COMPUTER PROGRAM DESIGN AND MANAGEMENT" EVALUATION FACTOR DOES NOT EVEN MENTION THE "TRIED AND TRUE" STANDARD.

ALTHOUGH THERE ARE CERTAIN BROAD STATEMENTS-- ESPECIALLY SECTION 1.1 OF THE INSTRUCTIONS FOR PROPOSAL PREPARATION, SUPRA-- WHICH STATE A PREFERENCE FOR "MAXIMUM USE" OF "TRIED AND TRUE" EQUIPMENTS/PROGRAMS, OFFERORS ARE NOT TOLD THROUGH THESE STATEMENTS HOW THE BROADLY STATED PREFERENCE WAS TO BE SPECIFICALLY LINKED WITH PROPOSAL EVALUATION. THIS CONTEXT, OFFERORS-- ABSENT QUESTIONING THE AIR FORCE ABOUT THE SPECIFIC WAYS) THIS PREFERENCE WOULD BE EVALUATED BEFORE PROPOSALS WERE DUE-- SHOULD NOT HAVE AUTOMATICALLY EXPECTED-- AS SYLVANIA APPEARS TO HAVE ASSUMED-- THAT THIS BROAD PREFERENCE WOULD BE SEPARATELY IDENTIFIED AND SPECIFICALLY SCORED. INSTEAD, IT SEEMS CLEAR THAT THE RFP, REASONABLY READ, PROMISES NO MORE THAN THAT THE PREFERENCE WOULD, IN SOME WAY, BE EVALUATED AS PART OF THE TECHNICAL EVALUATION UNDER OTHER SEPARATELY IDENTIFIED FACTORS. IN ANY EVENT, THE BOARD DID IN FACT QUESTION-- AND THUS, IN OUR VIEW, EVALUATE-- A "TRIED AND TRUE" ASPECT OF ITT'S PROPOSAL. THUS, WE TAKE THE AIR FORCE STATEMENT THAT THE "EXTENT TO WHICH THE GOAL WAS MET BY EACH OFFEROR PROVIDED ONE OF THE MANY EVALUATION INPUTS" AS INDICATING THE "NOT SEPARATELY SCORED AND IDENTIFIED" NATURE OF THE "TRIED AND TRUE" PROVISIONS. FURTHER, GIVEN THE ENTIRELY SUBJECTIVE CHARACTER /2/ OF THE "TRIED AND TRUE" PROVISIONS, ALL OFFERORS SHOULD HAVE EXPECTED THAT THE AIR FORCE WOULD NECESSARILY HAVE TO EXERCISE EXTREMELY BROAD DISCRETION IN EVALUATING OFFERORS' EFFORTS UNDER THESE PROVISIONS WITHIN THE CONTEXT OF THE SPECIFICALLY IDENTIFIED FACTORS AND SUBFACTORS.

GIVEN THE RFP'S CLEAR INDICATION THAT THE "TRIED AND TRUE" STANDARD WOULD NOT BE SEPARATELY EVALUATED BUT ONLY CONSIDERED WITHIN THE CONTEXT OF OTHER ESTABLISHED CRITERIA AND SUBFACTORS, IT IS NOT SURPRISING THAT THE RECORD OF SOURCE SELECTION EVALUATION DOES NOT CONTAIN-- TO OUR READING-- SPECIFIC SCORES AND EVALUATION ON THE STANDARD. THIS DOES NOT MEAN, AS SYLVANIA SUGGESTS, THAT THE GOAL WAS NOT CONSIDERED. AS NOTED ABOVE, WE FIND AT LEAST ONE REFERENCE TO THE GOAL IN THE EVALUATION OF ITT'S PROPOSAL. PRESUMABLY, THE OFFERORS' EVALUATION SCORES IN OTHER AREAS REFLECT, IN PART, THE AIR FORCE'S CONSIDERED VIEWS OF OFFERORS' EFFORTS TOWARD THE GOAL. MOREOVER, THERE IS NOTHING IN THE RECORD WHICH CONTRADICTS THE AIR FORCE'S POSITION THAT (A) "TRIED AND TRUE" ASPECTS OF SOFTWARE, EXCEPT APPLICATION SOFTWARE, AND HARDWARE WERE EVALUATED BY THE APPROPRIATE PANELS; (B) NO EXISTING SOFTWARE COULD HAVE MET SATIN IV REQUIREMENTS IN CERTAIN AREAS; AND (C) NO EXISTING HARDWARE COULD HAVE MET SATIN IV REQUIREMENTS.

BECAUSE OF THE FOREGOING ANALYSIS, WE DO NOT AGREE WITH SYLVANIA'S ARGUMENT-- BASED ON CITATION OF SEVERAL PRIOR GAO DECISIONS-- THAT THE AIR FORCE OMITTED THE "TRIED AND TRUE" EVALUATION STANDARD OR THAT THE EVALUATION WAS BASED ON UNDISCLOSED EVALUATION STANDARDS. MOREOVER, THE FURTHER QUESTIONS POSED BY SYLVANIA-- SO-CALLED "AREAS TO INVESTIGATE"-- RELATING TO THE PRECISE WAYS IN WHICH THE BOARD, THE COUNCIL AND THE FINAL SELECTION OFFICIAL EVALUATED "SOFTWARE AND HARDWARE" DEFICIENCIES ARE ALSO PEGGED TO THE ERRONEOUS ASSUMPTION THAT THE "TRIED AND TRUE" STANDARD WAS A SEPARATELY IDENTIFIED EVALUATION CRITERION. RELATING SPECIFICALLY THE PRECISE WAYS IN WHICH THE BOARD, THE COUNCIL AND THE FINAL SELECTION OFFICIAL EVALUATED THESE DEFICIENCIES COULD BE SEEN AS A VIOLATION OF RESTRICTIONS PLACED ON THE DOCUMENTS EVIDENCING THE SELECTION RATIONALE.

("AREAS TO INVESTIGATE" ARE ALSO CITED BY SYLVANIA UNDER ITS OTHER ISSUES. PROVIDING ANSWERS TO THE QUESTIONS POSED BY SYLVANIA COULD ALSO BE SEEN AS A VIOLATION OF THE RESTRICTIONS PLACED ON THE RELEVANT AGENCY DOCUMENTS. CONSEQUENTLY, THESE OTHER "AREAS TO INVESTIGATE" WILL NOT BE DISCUSSED EITHER. MOREOVER, SOME OF THE QUESTIONS ARE NOT RELEVANT TO OUR ISSUE ANALYSIS.)

ISSUE 2-- ALLEGED IMPROPER SELECTION OF PROPOSAL CONTAINING LOL LANGUAGE

SYLVANIA HAS EXPLAINED THAT IT IS THE COMPANY'S UNDERSTANDING THAT THE "PRINCIPAL AND CONTROLLING DIFFERENCES BETWEEN THE PROPOSALS (OF SYLVANIA AND ITT) REST IN THEIR DATA PROCESSING ASPECTS." IN ORDER TO EXPLAIN THESE DIFFERENCES, SYLVANIA HAS PROVIDED AN EXPLANATION OF THE TECHNICAL ASPECTS OF THE SYSTEM:

THE SATIN IV NETWORK IS DEPENDENT UPON THE USE OF 300-400 COMPUTERS WORKING TO SORT AND CONTROL THE FLOW OF MESSAGES BETWEEN NODES AT VARYING SECURITY LEVELS. TO DO THIS EACH MACHINE MUST CONTAIN THE APPROPRIATE PROGRAM. TO SOME EXTENT THERE ARE PROGRAMS THAT WILL HAVE COMMON APPLICATION TO MANY MACHINES AND LOCATIONS AND OTHER PROGRAMS THAT ARE UNIQUE TO A PARTICULAR LOCATION AND APPLICATION. THE JOB OF PROGRAMMING ALL OF THE EQUIPMENT FOR THE SATIN IV SYSTEM IS A MONUMENTAL TASK. MOREOVER, IT IS ONE THAT WILL NEED TO BE CONTINUOUSLY UPDATED AS THE SYSTEM GROWS OR IS MODIFIED TO MEET AS YET UNANTICIPATED NEEDS.

EACH PROGRAM MUST BE WRITTEN IN A LANGUAGE THE COMPUTER CAN "READ"OR ACCEPT. PROGRAMMING LANGUAGES BREAK DOWN INTO TWO BROAD TYPES: HIGH ORDER LANGUAGES (HOL'S) AND ASSEMBLY OR LOW ORDER LANGUAGES (LOL'S). THE DIFFERENCES BETWEEN THE TWO ARE SIGNIFICANT TO THIS PROTEST.

HIGH ORDER LANGUAGES GREATLY FACILITATE THE WRITING AND READING OF COMPUTER PROGRAMS, THEIR MAINTENANCE AND THE TRAINING OF PROGRAMMERS-- ALL OF WHICH RESULTS IN LOWER PROGRAMMING DEVELOPMENT AND MAINTENANCE COST. HOL ATTEMPTS TO LIGHTEN THE LOAD OF THE PROGRAMMER AND CODER BY MAKING THE COMPUTER ITSELF HELP TO PREPARE THE PROGRAM (OR CODE). THIS IS ACCOMPLISHED BY THE USE OF ANOTHER COMPUTER PROGRAM, A COMPILER, WHICH TRANSLATES FROM A FUNCTIONAL (HIGH-LEVEL) LANGUAGE TO THE BASIC (LOW- LEVEL) INSTRUCTIONS CARRIED OUT BY THE COMPUTER'S INTERNAL LOGIC. ASSEMBLY LANGUAGE ON THE OTHER HAND IS A LOW-LEVEL LANGUAGE IN WHICH THE PROGRAMMER INSTRUCTS THE COMPUTER TO PERFORM ITS OPERATIONS AT THE LEVEL CORRESPONDING TO THE INTERNAL OPERATIONS OF THE COMPUTER HARDWARE ITSELF. WHILE ASSEMBLY LANGUAGE PROVIDES THE PROGRAMMER DIRECT CONTROL OF THE INNER WORKINGS OF THE COMPUTER HARDWARE, IT REQUIRES THE PROGRAMMER TO UNDERSTAND AND CONCERN HIMSELF MORE WITH THE LOGIC AND ARCHITECTURE OF THE COMPUTER. AS A RESULT, THERE IS A GREATER DANGER OF PROGRAMMING INCORRECTLY WITH LOW ORDER ASSEMBLY LANGUAGE THAN WITH HOL. A PROGRAM WRITTEN IN LOW ORDER ASSEMBLY LANGUAGE IS MACHINE DEPENDENT, I.E., EXECUTABLE ONLY ON THE SPECIFIC MACHINE FOR WHICH IT IS WRITTEN, WHILE A PROGRAM WRITTEN IN HOL IS MACHINE-INDEPENDENT, I.E., EXECUTABLE ON ANY COMPUTER WHICH HAS THE SAME LANGUAGE COMPILER.

IN ITS PROPOSAL, SYLVANIA CHOSE TO USE THE BURROUGHS "D" MACHINE AS ITS PRINCIPAL PIECE OF COMPUTER HARDWARE. THE BURROUGHS MACHINE IS A PROVEN PRODUCT WITH AN AVAILABLE SOFTWARE COMPILER PERMITTING IT TO BE PROGRAMMED IN A HIGH ORDER LANGUAGE SPECIFICALLY SUITED FOR COMMUNICATIONS WORK. THE USE OF HOL PERMITS THE MILITARY ASSOCIATE CONTRACTOR (CCPC) TO ACCOMPLISH ITS TASK WITH LESS SKILLED PROGRAMMERS AND AT A REDUCED COST. INDEED, SYLVANIA'S SELECTION OF THE BURROUGHS' MACHINE WAS DRIVEN BY THESE FACTORS AND THE CLEAR RFP REQUIREMENTS INCLUDING THOSE FOR OFF THE SHELF HARDWARE.

IT IS SYLVANIA'S UNDERSTANDING THAT THE ITT PROPOSAL, ON THE OTHER HAND, INCORPORATED IBM'S NEW SERIES/1, ITS FIRST ENTRY INTO THE MINICOMPUTER FIELD. THIS MACHINE, UNPROVEN AT THE PRESENT TIME, CAN ONLY BE PROGRAMMED IN ASSEMBLY LANGUAGE, REQUIRING ITT TO PERFORM ITS PROGRAMMING IN LOL.

THE CHOICE OF SPECIFIC COMPUTER HARDWARE AND SPECIFIC PROGRAMMING LANGUAGE CONSTITUTES A PIVOTAL DECISION IN THE SYSTEM APPROACH TO THE SATIN IV REQUIREMENT. THE CHOICE OF LANGUAGE, IN PARTICULAR, PERMEATES AND CONTROLS MANY OTHER ASPECTS OF THE SYSTEM DESIGN.

ONE EXAMPLE OF THIS EFFECT CAN BE TRACED TO THE DIFFERENT MEMORY CAPACITIES REQUIRED BY HOL AND LOL. SOFTWARE DEVELOPED WITH LOL, UTILIZES LESS MEMORY CAPACITY WITHIN A MACHINE THAN A SIMILAR PROGRAM DEVELOPED WITH HOL. TRANSLATED INTO COSTS, USE OF LOL ALLOWS THE PROPOSER TO PROVIDE LESS MEMORY CAPACITY, I.E., LESS HARDWARE TO PERFORM THE MINIMUM NUMBER OF FUNCTIONS REQUIRED, AND WILL THEREFORE HAVE LOWER "FRONT END" HARDWARE COST THAN A PROPOSAL BASED UPON HOL. IN CONTRAST, SYSTEMS BASED UPON HOL SOFTWARE HAVE GREATER FLEXIBILITY TO MEET FUTURE NEEDS, ARE MORE RELIABLE AND RESULT IN LOWER MAINTENANCE AND LIFE CYCLE COSTS. A TRADEOFF THEREFORE EXISTS WHEN A PROPOSER DETERMINES WHICH TYPE OF SOFTWARE IT WILL UTILIZE.

SYLVANIA INSISTS THAT USES OF "HIGH ORDER PROGRAMMING LANGUAGES" FOR THE PROCUREMENT WAS "IMPLICIT IN THE SATIN IV RFP, WHICH INCLUDED EMPHASIS UPON LIFE-CYCLE COSTS, SYSTEM FLEXIBILITY, MAINTENANCE OF SOFTWARE, AND THE REQUIREMENT TO USE STRUCTURAL PROGRAMMING CONCEPTS." PERTINENT RFP PROVISIONS CITED BY SYLVANIA IN SUPPORT OF THIS ARGUMENT ARE THE FOLLOWING:

EVALUATION FACTORS FOR AWARD, SECTION 4.0: SPECIFIC CRITERIA COMPUTER PROGRAM DESIGN AND MANAGEMENT INSTRUCTIONS FOR PROPOSAL PREPARATION, SECTION 6.3.1.6:

* * * DESCRIBE THE TECHNIQUES TO BE USED TO ENHANCE THE EFFECTIVENESS AND MAINTAINABILITY OF SOFTWARE DOCUMENTATION. * * *

INSTRUCTIONS FOR PROPOSAL PREPARATION, SECTION 6.3.2.5:

* * * DISCUSS HOW THE PROGRAMMING LANGUAGES AND HARDWARE CHARACTERISTICS MEET THE SOFTWARE REQUIREMENTS FOR UPWARD COMPATIBILITY AMONG PROCESSORS AND PROMOTE COMMONALITY AND EFFICIENT DEVELOPMENT. DISCUSS SOFTWARE TRANSFERABILITY BETWEEN THE SOFTWARE DEVELOPMENT/SOFTWARE MAINTENANCE FACILITIES AND THE OPERATIONAL PROCESSORS.

SATIN IV SYSTEM SPECIFICATION, PARAGRAPH 3.3.8.3:

PROGRAMMING LANGUAGES. * * * OTHER CONSIDERATIONS, SUCH AS PROGRAMMER TRAINING, PROGRAMMER PRODUCTIVITY, AND EASE OF MAINTENANCE, MAKE IT DESIRABLE THAT ALL SATIN IV SOFTWARE BE DEVELOPED IN A SUITABLE COMMON LANGUAGE. AS A MINIMUM REQUIREMENT, ALL COMMUNICATION PROCESSORS, I.E., THE SCPS, BCPS, AND MBCPS, SHALL USE THE SAME UPWARDLY COMPATIBLE PROGRAMMING LANGUAGE. IN ORDER FOR A LANGUAGE TO BE SUITABLE FOR ANY PROCESSOR, IT SHALL INCLUDE, BUT NOT BE LIMITED TO, THE FOLLOWING CHARACTERISTICS. * * *

SYLVANIA ALSO ARGUES THAT THE SELECTION OF THE ITT "LOW ORDER LANGUAGE" APPROACH RAN COUNTER TO THE PROVISIONS OF DEPARTMENT OF DEFENSE DIRECTIVE 5000.29 (ISSUED APRIL 26, 1976) WHICH PROVIDES, IN PERTINENT PART, AS FOLLOWS:

SOFTWARE LANGUAGE STANDARDIZATION AND CONTROL. DOD APPROVED HIGH ORDER PROGRAMMING LANGUAGES (HOLS) WILL BE USED TO DEVELOP DEFENSE SYSTEM SOFTWARE, UNLESS IT IS DEMONSTRATED THAT NONE OF THE APPROVED HOLS ARE COST EFFECTIVE OR TECHNICALLY PRACTICAL OVER THE SYSTEM LIFE CYCLE. * *

SYLVANIA SAYS THAT SINCE ITS "HIGH ORDER LANGUAGE" APPROACH WAS FOUND TECHNICALLY ACCEPTABLE AND COST EFFECTIVE, USE OF THE HIGH ORDER LANGUAGE IS CLEARLY IN ORDER AND ITT'S USE OF "LOW ORDER LANGUAGE" SHOULD NOT HAVE BEEN FOUND TO BE ACCEPTABLE. FURTHER, SYLVANIA IS OF THE OPINION THAT ANY COST SAVINGS-- ESTIMATED TO BE $2-$3 MILLION-- WHICH MIGHT HAVE FOLLOWED FROM AN OFFEROR'S USE OF THE "LOW ORDER LANGUAGE" WOULD BE MORE THAN OFFSET BY THE "TOTAL SYSTEMS LIFE" SAVINGS OF "HIGH ORDER LANGUAGE" USE.

THE AIR FORCE REPLY TO THE "CHOICE OF LANGUAGE" ISSUE AND SYLVANIA'S SUPPLEMENTAL COMMENTS OF THIS ISSUE ARE SUMMARIZED:

SYLVANIA:

(1) EVEN THOUGH DIRECTIVE 5000.29 WAS NOT PER SE APPLICABLE TO THE PROCUREMENT, THE UNDERLYING RATIONALE OF THE DIRECTIVE IS APPLICABLE. THE GOALS OF THE DIRECTIVE ARE BEST ACHIEVED BY USE OF "HIGH ORDER LANGUAGE" PROGRAMMING.

AIR FORCE:

(1) THE DICTATES OF DIRECTIVE 5000.29 WHICH MENTION APPROVED "HIGH ORDER LANGUAGES" WERE NOT EFFECTIVE UNTIL NOVEMBER 1976 WHEN THE FIRST LIST OF DEFENSE DEPARTMENT APPROVED "HIGH ORDER LANGUAGES" WAS PUBLISHED. MOREOVER, IN NOVEMBER 1976, THE DEFENSE DEPARTMENT SAID THE PROVISIONS RELATING TO "HIGH ORDER LANGUAGES" WERE NOT TO BE RETROACTIVELY APPLIED. NO OFFEROR, INCLUDING SYLVANIA, PROPOSED AN APPROVED "HIGH ORDER LANGUAGE." ALTHOUGH THE GOVERNMENT MAY HAVE BEEN DETERMINED THAT IN MOST INSTANCES CERTAIN "HIGH ORDER LANGUAGES" MAY BE PRESUMED TO MEET THE GOVERNMENT'S NEEDS BETTER, THE DETERMINATION OF WHICH LANGUAGE INCLUDED IN A TOTAL PROPOSED SYSTEM BEST MEETS THE GOVERNMENT'S NEEDS IS DETERMINED ACCORDING TO DIRECTIVE 5000.29 BY THE REQUIREMENTS OF THE SPECIFIC PROGRAM.

SYLVANIA:

(2) THE AIR FORCE'S CONCERN WITH THE IMPORTANCE OF SOFTWARE IS SHOWN IN THE PROCUREMENTS WHICH IDENTIFY "COMPUTER PROGRAM DESIGN AND MANAGEMENT" AS A SEPARATE CRITERION, SECOND ONLY TO THE "TECHNICAL"CRITERION. THERE CAN BE NO DOUBT THAT PROPER EVALUATION OF SOFTWARE WAS CRITICAL TO THE PROCUREMENT. CHOICE OF PROGRAMMING LANGUAGE IS CENTRAL TO THE ACCOMPLISHMENT OF THE SOFTWARE COST AND RISK MINIMIZATION OBJECTIVES.

THE AIR FORCE'S SELECTION OF "LOW ORDER PROGRAMMING LANGUAGE" IGNORED THE REQUIREMENTS OF THE RFP, WAS CONTRARY TO SOFTWARE ACQUISITION POLICY, AND WAS ARBITRARY (AND THEREFORE ILLEGAL.)

AIR FORCE

(2) SINCE NEITHER THE DIRECTIVE NOR ANY "POLICY" REGARDING "HIGH ORDER LANGUAGES" WAS TO APPLY RETROACTIVELY, THE SATIN IV RFP WAS DRAFTED SO AS TO PERMIT EITHER "HIGH ORDER" OR "LOW ORDER" LANGUAGE. FURTHER, THE GOVERNMENT WAS UNABLE TO VERIFY THE EXISTENCE OF THE CLAIMED "HIGH ORDER LANGUAGE" BENEFITS PRIOR TO THE ISSUANCE OF THE RFP AND THE PROPOSALS DID NOT PROVE OTHERWISE. MOREOVER, IT IS IMPORTANT TO REALIZE THAT THE AWARD DECISION WAS NOT SOLELY PROMPTED UPON AN ANALYSIS OF LANGUAGE CHOICE AS SYLVANIA SUGGESTS. THE CHOICE OF A COMPUTER LANGUAGE WAS ONLY A SMALL ASPECT OF THE PROGRAM. THE AIR FORCE CHOSE THE ITT PROPOSAL BECAUSE IT FELT THE PROPOSAL WAS THE "BEST BUY" UNDER THE RFP CRITERIA, AND THIS PROPOSAL USED "LOW ORDER LANGUAGE."

SYLVANIA

(3) MR. JANOFSKY SAID THAT THE ADVANTAGES THE AIR FORCE COULD EXPECT FROM A SYSTEM INCORPORATING "LOW ORDER LANGUAGE" WERE THAT THE SYSTEM WOULD BE MORE CONSERVATIVE, USE LESS MEMORY SPACE, RUN FASTER, AND BE MORE FAMILIAR TO THE MILITARY ASSOCIATE CONTRACTOR WHO IS GOING TO PERFORM PART OF THE SATIN IV SYSTEM. EACH OF THESE BASES FOR FAVORING "LOW ORDER LANGUAGE" IS EITHER IRRELEVANT TO THE SYSTEM OR A DISTORTION OF THE TRUTH. THE DIRECTIVE SHOWS THAT HIGH ORDER LANGUAGE OFFERS FEWER RISKS AND, IN THAT SENSE, SHOULD BE CONSIDERED MORE CONSERVATIVE.

AIR FORCE

(3) MR. JANOFSKY OF THE AIR FORCE DID NOT SAY THAT ITT WAS CHOSEN BECAUSE OF THE PROPOSED USE OF "LOW ORDER LANGUAGE," NOR DID HE SAY THAT "LOW ORDER LANGUAGE" WAS SELECTED AS THE MORE "TRADITIONAL" APPROACH. RATHER, HE WAS SPEAKING OF THE REASONS WHICH LED THE AIR FORCE TO WRITE AN RFP WHICH DID NOT DICTATE "LANGUAGE" CHOICE.

SYLVANIA

(4) WHILE "LOW ORDER LANGUAGES" REQUIRE LESS COMPUTER TIME AND MEMORY SPACE-- BOTH FACTORS RELATING TO SYSTEM RESPONSE TIME-- THE RFP REQUIRES ONLY THAT RESPONSE TIMES MEET MINIMUM LEVELS-- LEVELS MET BY SYLVANIA. ANY EVENT, GREATER OR LESSER RESPONSE TIME WAS NOT RELEVANT AND NOT AN EVALUATION FACTOR.

AIR FORCE:

(4) RESPONSE TIME OF "LANGUAGES" WAS EVALUATED; MOREOVER, RESPONSE TIME AND MEMORY SPACE REQUIREMENTS HAVING A DIRECT IMPACT ON HARDWARE COSTS AND IN INDIRECT IMPACT ON MAINTENANCE COSTS (MANPOWER AND EQUIPMENT) WERE PROPERLY EVALUATED.

SYLVANIA:

(5) MR. JANOFSKY CLEARLY ADMITTED THAT DURING EVALUATION THE AIR FORCE DETERMINED THAT THE SOFTWARE COMPONENT OF THE LIFE-CYCLE COST MODEL WAS INADEQUATE-- IT WAS THEREAFTER IGNORED. IN OTHER WORDS, THE LIFE-CYCLE COSTS OF THE VARIOUS SOFTWARE PROPOSALS WERE NEVER EVALUATED OR CONSIDERED. THE AIR FORCE IGNORED A MAJOR EVALUATION CRITERION RATHER THAN ASK OFFERORS FOR WHATEVER FURTHER DATA WAS NEEDED TO EVALUATE SOFTWARE LIFE-CYCLE COSTS PROPERLY. SINCE THE CRITERIA OF MAINTAINABILITY, RELIABILITY, RISK, ETC., ALL HAVE COST CONSEQUENCES THAT WOULD BE REFLECTED IN LIFE-CYCLE COSTS, THE FAILURE TO EVALUATE THESE COSTS AS THEY RELATE TO SOFTWARE MEANS THE AIR FORCE EFFECTIVELY IGNORED THESE CRITERIA AS WELL. THE ASSUMPTION THAT SOFTWARE LIFE CYCLE COSTS WOULD BE THE SAME WAS ARBITRARY IN VIEW OF THE DIRECTIVE'S STATEMENT THAT "HIGH ORDER LANGUAGE" WOULD HAVE PRODUCED LOWER LIFE CYCLE COSTS AND GREATER SOFTWARE RELIABILITY, MAINTAINABILITY AND RISK MINIMIZATION.

AIR FORCE:

(5) IN RESPONSE TO MR. JANOFSKY'S CONCERN THAT THE ITT PROPOSAL MIGHT BE MORE COST EFFECTIVE IF MORE "HIGH ORDER LANGUAGE" PROGRAMMING WERE USED, THE AIR FORCE ALLOWED DISCUSSION WITH ITT LIMITED TO THE COMPANY'S REASONS FOR CHOOSING THE LANGUAGE APPROACH. THIS DISCUSSION SATISFIED THE GOVERNMENT.

THE ANALYSIS OF LIFE-CYCLE COST CENTERED ON THOSE ELEMENTS OF FOLLOW ON SUPPORT WHICH WERE CONSIDERED SIGNIFICANT AND SUSCEPTIBLE TO VARIATIONS AMONG THE COMPETING CONTRACTOR'S DESIGNS, AND WHICH WOULD BE MEANINGFUL IN MAKING A CONTRACT AWARD DECISION. THE CONCLUSION REACHED WAS THAT, EXCEPT FOR TWO COST ELEMENTS, ALL OTHER ELEMENTS DID NOT DIFFER SIGNIFICANTLY ENOUGH TO AFFECT THE AWARD DECISION, OR CREDIBILITY IN THE PROPOSED FIGURES COULD NOT BE ACHIEVED, THUS RENDERING THEIR USE IN COMPARATIVE ANALYSIS MEANINGLESS AND POSSIBLY INEQUITABLE TO COMPETING OFFERORS.

IN ANY EVENT, THE AIR FORCE DETERMINED THAT ITS INTERPRETATION OF THE TOTAL MINIMUM NEEDS OF THE GOVERNMENT (LOWEST TOTAL SYSTEM LIFE CYCLE COST, ETC.) WERE MET BY ITT'S PROPOSAL WHICH INCORPORATED A LESSER DEGREE OF "HIGH ORDER LANGUAGE" THAN SYLVANIA'S PROPOSAL.

ANALYSIS-- ISSUE TWO

GIVEN THAT THE SATIN IV SYSTEM SPECIFICATION PROVISION ON "PROGRAMMING LANGUAGES," SUPRA, DID NOT EXPRESSLY REQUIRE-- OR PROHIBIT - THE USE OF "HIGH ORDER PROGRAMMING LANGUAGE," SYLVANIA'S ARGUMENT THAT USE OF "HIGH ORDER PROGRAMMING LANGUAGE" NECESSARILY REPRESENTED SUPERIOR VALUE FOR EVERY PHASE OF THE SATIN IV SYSTEM PRIMARILY RESTS ON THE PRESUMED APPLICABILITY OF THE PROVISIONS OF DIRECTIVE 5000.29 TO THE SUBJECT PROCUREMENT. ALTHOUGH SYLVANIA ADMITS THAT THE DIRECTIVE DID NOT EXPRESSLY APPLY TO THE PROCUREMENT, IT STILL ARGUES THAT THE "UNDERLYING RATIONALE" OF THE DIRECTIVE-- A STATED PREFERENCE FOR "HIGH ORDER LANGUAGE"-- IS APPLICABLE.

WE AGREE WITH THE AIR FORCE VIEW THAT SINCE THE DIRECTIVE WAS NOT EXPRESSLY APPLICABLE TO THE PROCUREMENT, THE "UNDERLYING RATIONALE" OR POLICY VIEWS FOUND IN THE DIRECTIVE ARE NOT EXPRESSLY APPLICABLE TO THE PROCUREMENT. TO THE EXTENT THAT ANY VIEWS OF THE DIRECTIVE MAY BE SAID TO BE APPLICABLE BECAUSE OF THE FORCE OF LOGIC, IT IS APPARENT THAT THESE VIEWS MIGHT BE REFUTED BY THE WEIGHT OF EQUALLY SUPERIOR ANALYSIS. THINK THE AIR FORCE HAS PROVIDED THIS ANALYSIS.

WE AGREE WITH THE AIR FORCE'S OBSERVATION THAT, ALTHOUGH IN MANY INSTANCES, USE OF CERTAIN "HIGH ORDER LANGUAGES" MAY BE PRESUMED TO MEET THE GOVERNMENT'S NEEDS BEST, THE DECISION AS TO WHICH PROGRAMMING LANGUAGE IS BEST FOR A GIVEN REQUIREMENT-- SAY, THE SATIN IV SYSTEM-- IS DETERMINED BY THE REQUIREMENTS OF THE SPECIFIC SYSTEM. THE REASONS GIVEN BY ONE OF THE AIR FORCE EVALUATORS AS TO WHY THE RFP WAS DRAFTED SO AS TO NOT RULE OUT THE USE OF "LOW ORDER PROGRAMMING LANGUAGE"-- THAT "LOW ORDER LANGUAGE" WAS CONSIDERED THE MORE "CONSERVATIVE" SYSTEM, WOULD USE LESS MEMORY SPACE, RUN FASTER AND BE MORE FAMILIAR TO THE MILITARY ASSOCIATE CONTRACTOR WHO WAS GOING TO PERFORM PART OF THE SATIN IV WORK-- PRESUMABLY WERE OF SOME INFLUENCE ON THOSE EVALUATORS WHO DID NOT EXCLUDE THE ITT PROPOSAL FROM CONSIDERATION FOR AWARD MERELY BECAUSE OF ITS PROPOSED LANGUAGE CHOICE. ALTHOUGH IT IS TRUE THAT THESE REASONS WERE NOT LISTED AS THE CRITERIA BY WHICH OFFERED PROGRAMMING LANGUAGES WOULD BE EVALUATED, THE FACT REMAINS THAT THE SATIN IV SYSTEM SPECIFICATION PROVISION ON PROGRAMMING LANGUAGES (SEE PARAGRAPH 3.3.8.3A-E) SPECIFIES ONLY THAT PROPOSED LANGUAGES ARE TO POSSESS CERTAIN BASIC CHARACTERISTICS-- RELATING TO DATA STRUCTURE, PROGRAM STRUCTURE, INPUT/OUTPUT, OPERATING SYSTEM CALLS, AND MACROCAPABILITY-- NONE OF WHICH ARE APPARENTLY INCAPABLE OF FULFILLMENT WITH "LOW ORDER LANGUAGE." IT IS THESE SPECIFICATIONS, THEREFORE, WHICH HAVE DEFINED THE GOVERNMENT'S NEEDS FOR PROGRAMMING LANGUAGE CHOICE IN THE SPECIFIC PROGRAM-- NEEDS WHICH WERE NOT QUESTIONED IN ANY WAY BEFORE SYLVANIA SUBMITTED ITS PROPOSAL. IN VIEW OF THESE DETAILED SPECIFICATIONS, THOSE OTHER PROCUREMENT DOCUMENTS IN WHICH SYLVANIA FINDS AN "IMPLICIT" REQUIREMENT FOR "HIGH ORDER LANGUAGE" MUST BE READ IN CONJUNCTION WITH THESE SPECIFICATIONS WHICH OTHERWISE PERMIT USE OF "LOW ORDER LANGUAGE." UNDER THIS READING, WE REJECT THE VIEW OF "IMPLICIT" REQUIREMENTS FOR "HIGH ORDER LANGUAGE" IN OTHER PROCUREMENT DOCUMENTS. TO THE EXTENT, MOREOVER, THAT SYLVANIA'S PROTEST OBJECTS TO THE AIR FORCE'S DETERMINATION THAT A LESS RESTRICTIVE SPECIFICATION-- PERMITTING OFFERORS TO USE EITHER "HIGH ORDER" OR "LOW ORDER" PROGRAMMING LANGUAGE-- WILL MEET AIR FORCE NEEDS FOR THIS PARTICULAR REQUIREMENT, THE GROUND OF PROTEST IS NOT FOR REVIEW. AS WE RECENTLY SAID IN MILTOPE CORPORATION-- RECONSIDERATION, B-188342, JUNE 9, 1977, 77-1 CPD 417:

* * * WHERE * * * IT IS ASSERTED THAT THE GOVERNMENT'S INTEREST AS USER * * * IS NOT ADEQUATELY PROTECTED (BY A LESS RESTRICTIVE SPECIFICATION) * * * THE PROTESTER'S * * * INTEREST CONFLICTS WITH THE OBJECTIVE OF OUR BID PROTEST FUNCTION, THAT IS, TO INSURE ATTAINMENT OF FULL AND FREE COMPETITION. ASSURANCE THAT SUFFICIENTLY RIGOROUS SPECIFICATIONS ARE USED IS ORDINARILY OF PRIMARY CONCERN TO PROCUREMENT PERSONNEL AND USER ACTIVITIES. IT IS THEY WHO MUST SUFFER ANY DIFFICULTIES RESULTING BY REASON OF INADEQUATE EQUIPMENT. WE, THEREFORE, BELIEVE IT WOULD BE INAPPROPRIATE TO RESOLVE SUCH ISSUES PURSUANT TO OUR BID PROTEST FUNCTION, ABSENT EVIDENCE OF FRAUD OR WILLFUL MISCONDUCT BY PROCUREMENT OR USER PERSONNEL ACTING OTHER THAN IN GOOD FAITH.

THERE IS NO EVIDENCE THAT THE AIR FORCE DETERMINED ITS NEEDS FOR COMPUTER PROGRAMMING-- THAT IS, PERMITTED EITHER "HIGH ORDER" OR "LOW ORDER" PROGRAMMING LANGUAGE FOR THIS SPECIFIC PROGRAM-- IN OTHER THAN IN GOOD FAITH.

NEITHER CAN WE DISAGREE WITH THE AIR FORCE'S ANALYSIS AS TO WHY IT DID NOT PURSUE EVALUATION OF SOFTWARE DESIGN LIFE-CYCLE COSTS TO THE EXTENT SYLVANIA BELIEVES THE COSTS SHOULD HAVE BEEN EXAMINED. IN OUR VIEW, THE AIR FORCE POSITION THAT THE SINGLE LARGEST ELEMENT AFFECTING LIFE-CYCLE COSTS-- THAT IS, THE COST OF MILITARY MAINTENANCE PERSONNEL FOR FULL-TIME MAINTENANCE COVERAGE-- WAS OUT OF THE CONTROL OF ANY PROSPECTIVE CONTRACTOR IS RATIONALLY FOUNDED. SIMILARLY, WE VIEW AS RATIONALLY FOUNDED THAT FURTHER AIR FORCE VIEW THAT THE "SMALL AMOUNT (OF COST) ADDED BY SOFTWARE WOULD NOT MATERIALLY AFFECT THE TOTAL MANPOWER COST" REGARDLESS OF THE CHOICE OF PROGRAMMING LANGUAGE USED. ALSO, WE DO NOT AGREE THAT THIS APPROACH ELIMINATED, AS SYLVANIA URGES, LIFE-CYCLE COSTS AS AN EVALUATION STANDARD, SINCE IT IS CLEAR THAT CERTAIN COST ELEMENTS PERTAINING TO THIS STANDARD WERE CONSIDERED. FINALLY, WE AGREE WITH THE AIR FORCE POSITION THAT, AS A PRACTICAL MATTER, IT WOULD HAVE BEEN IMPOSSIBLE TO OBTAIN FROM COMPETITIVE-RANGE OFFERORS DETAILED INFORMATION NEEDED TO EVALUATE LIFE-CYCLE COSTS DOWN TO THE MODULE LEVEL SINCE THE DESIGN OF THE SOFTWARE TO THE MODULE LEVEL WOULD NOT OCCUR UNTIL AFTER CONTRACT AWARD.

WE FURTHER NOTE THAT SYLVANIA'S PROPOSAL WAS GIVEN A SLIGHT EDGE OVER ITT'S PROPOSAL IN COMPUTER PROGRAM DESIGN, REFLECTING, IN PART, SYLVANIA'S LANGUAGE CHOICE. TO THIS EXTENT, SYLVANIA WAS ACCORDED-- AS IT NOW URGES SHOULD HAVE BEEN THE CASE IN ITS PROTEST-- AN EVALUATION EDGE OVER ITT. TO THE EXTENT, HOWEVER, ITS PROTEST UNDER THIS ISSUE CAN BE VIEWED AS AN ARGUMENT THAT IT SHOULD HAVE BEEN ACCORDED A GREATER ADVANTAGE OR THAT ITT'S PROPOSAL SHOULD HAVE BEEN RENDERED UNACCEPTABLE BECAUSE OF ITS LANGUAGE CHOICE, WE DO NOT AGREE, SINCE WE FIND RATIONAL SUPPORT FOR THE AIR FORCE'S CONTRARY EVALUATION RESULTS.

ISSUE 3-- ALLEGED NEGOTIATION OF MAJOR, MATERIAL CHANGES TO ITT'S PROPOSAL DURING STEP 4 OF THE SATIN IV PROCUREMENT PROCESS IN VIOLATION OF DEPARTMENT OF DEFENSE DIRECTIVE 4105.62

SYLVANIA'S INITIAL PROTEST ALLEGED THAT IT "HAD REASON TO BELIEVE THAT CONTRARY TO THE (DOD DIRECTIVE REGARDING THE PROCEDURES TO BE FOLLOWED ON STEP 4) THE AIR FORCE (WAS) CURRENTLY CONTEMPLATING A CHANGE IN ITT'S STEP 3 PROPOSAL TO PERMIT ITT TO CHANGE FROM LOWER ORDER PROGRAMMING LANGUAGE TO 'HIGHER ORDER' AS WAS PROPOSED BY SYLVANIA." SINCE THAT TIME, THE AIR FORCE HAS INFORMED SYLVANIA THAT IT DID NOT PERMIT THIS CHANGE. RESPONSE TO THIS INFORMATION SYLVANIA HAS REVISED ITS GROUND OF PROTEST TO ATTACK THE PROPRIETY OF ALL THE CHANGES WHICH THE AIR FORCE HAS ADMITTED WERE MADE IN THE ITT PROPOSAL DURING THE STEP 4 PROCUREMENT STAGE.

INITIALLY, SYLVANIA ARGUED THAT:

DOD DIRECTIVE 4105.62 IN ITS SECTION III.D.5.C, DELINEATES, IN CONSIDERABLE DETAIL PRECISELY HOW THE STEP 4 NEGOTIATIONS ARE TO BE HANDLED. THIS INCLUDES WHAT CAN AND CANNOT BE DISCUSSED IN THESE NEGOTIATIONS. SUBPARAGRAPH (4) OF THIS SECTION STATES:

"NEGOTIATIONS AFTER SELECTION SHALL NOT INVOLVE MATERIAL CHANGES IN THE GOVERNMENT'S REQUIREMENTS OR THE CONTRACTOR'S PROPOSAL WHICH AFFECT THE BASIS FOR SOURCE SELECTION. IN THE EVENT THAT SUCH CHANGES ARE DESIRED BY THE GOVERNMENT, THE COMPETITION WILL BE REOPENED IN ACCORDANCE WITH EXISTING ASPR REQUIREMENTS."

THUS, STEP 4 CANNOT BE USED TO IMPLEMENT ANY CHANGE THAT WOULD AFFECT THE SOURCE SELECTION DECISION.

SYLVANIA ALSO URGED THAT IF THE AIR FORCE FELT THERE WERE DEFICIENCIES IN THE PROPOSAL THE APPROPRIATE TIME TO HAVE BROUGHT THEM TO ITT'S ATTENTION WOULD HAVE BEEN SUBSEQUENT TO "STEP 1 AND 2 SUBMISSIONS (SO AS TO PERMIT MODIFICATIONS) IN THE * * * PROPOSALS SUBMITTED IN STEP 3."

THE RELEVANT PARTS OF DEFENSE PROCUREMENT CIRCULAR NO. 75-7, FEBRUARY 27, 1976, WHICH PROMULGATED DIRECTIVE 4105.62 AND "SPECIAL TEST ASPR 3-805.3 LANGUAGE" PROVIDE:

* * * THE SELECTED (FOR STEP 4 DISCUSSIONS) OFFEROR'S PROPOSAL MUST SATISFY THE GOVERNMENT'S MINIMUM REQUIREMENT.

NEGOTIATIONS AFTER SELECTION SHALL NOT INVOLVE MATERIAL CHANGES IN THE GOVERNMENT'S REQUIREMENTS OR THE CONTRACTOR'S PROPOSAL WHICH AFFECT THE BASIS FOR SOURCE SELECTION. IN THE EVENT THAT SUCH CHANGES ARE DESIRED BY THE GOVERNMENT, THE COMPETITION WILL BE REOPENED IN ACCORDANCE WITH EXISTING ASPR REQUIREMENTS.

THE FOLLOWING SPECIAL TEST ASPR 3-805.3 LANGUAGE (DUPLICATIVE OF CERTAIN KEY PROVISIONS OF THE DIRECTIVE) IS APPLICABLE ONLY TO THOSE PROCUREMENTS INVOLVED IN THE TEST.

3-805.3 DISCUSSIONS WITH OFFERORS.

(A) EXCEPT AS PROVIDED IN (B) BELOW, ALL OFFERORS SELECTED TO PARTICIPATE IN DISCUSSIONS SHALL BE ADVISED OF DEFICIENCIES IN THEIR PROPOSALS AND SHALL BE OFFERED A REASONABLE OPPORTUNITY TO CORRECT OR RESOLVE THE DEFICIENCIES AND TO SUBMIT SUCH PRICE OR COST, TECHNICAL OR OTHER REVISIONS TO THEIR PROPOSALS THAT MAY RESULT FROM THE DISCUSSIONS. DEFICIENCY IS DEFINED AS THAT PART OF AN OFFEROR'S PROPOSAL WHICH WOULD NOT SATISFY THE GOVERNMENT'S REQUIREMENTS.

(B) IN DISCUSSING TECHNICAL PROPOSALS FOR PROCUREMENTS INVOLVING ADVANCED, ENGINEERING OR OPERATIONAL SYSTEMS DEVELOPMENT (SEE 4-101), CONTRACTING OFFICERS SHALL APPRISE OFFERORS SELECTED TO PARTICIPATE IN DISCUSSIONS OF ONLY THOSE IDENTIFIED DEFICIENCIES IN THEIR PROPOSALS THAT LEAD TO A CONCLUSION THAT (I) THE MEANING OF THE PROPOSAL OR SOME ASPECT THEREOF IS NOT CLEAR, (II) THE OFFEROR HAS FAILED TO ADEQUATELY SUBSTANTIATE A PROPOSED TECHNICAL APPROACH OR SOLUTION, OR (III) FURTHER CLARIFICATION OF THE SOLICITATION IS REQUIRED FOR EFFECTIVE COMPETITION. TECHNICAL DEFICIENCIES CLEARLY RELATING TO AN OFFEROR'S MANAGEMENT ABILITIES, ENGINEERING OR SCIENTIFIC JUDGMENT, OR HIS LACK OF COMPETENCE OR INVENTIVENESS IN PREPARING HIS PROPOSAL SHALL NOT BE DISCLOSED. MEANINGFUL DISCUSSIONS SHALL BE CONDUCTED WITH THE RESPECTIVE OFFERORS REGARDING THEIR COST/PRICE PROPOSALS. SUCH DISCUSSION MAY INCLUDE:

(I) COST REALISM;

(II) MATHEMATICAL ERRORS OR INCONSISTENCIES;

(III) CORRELATION BETWEEN COSTS AND RELATED TECHNICAL ELEMENTS, AND

(IV) OTHER COST/PRICE FACTORS NECESSARY FOR COMPLETE UNDERSTANDING OF BOTH THE GOVERNMENT REQUIREMENT AND THE PROPOSAL FOR MEETING IT, INCLUDING DELIVERY SCHEDULE, OTHER CONTRACT TERMS, AND TRADE-OFF CONSIDERATIONS (WITH SUPPORTING RATIONALE) AMONG SUCH ELEMENTS AS PERFORMANCE, DESIGN TO COST, LIFE CYCLE COST, AND LOGISTIC SUPPORT. OFFERORS SHALL BE AFFORDED A REASONABLE OPPORTUNITY TO CORRECT OR RESOLVE DEFICIENCIES AND SUBMIT REVISIONS TO EITHER THEIR TECHNICAL OR COST/PRICE PROPOSALS.

SYLVANIA'S SUPPLEMENTAL COMMENTS TO ITS INITIAL PROTEST AND THE AIR FORCE REPLY ARE SUMMARIZED AS FOLLOWS:

SYLVANIA:

(1) THAT AIR FORCE HAS ADMITTED THAT IT SAVED MANY OF ITT'S PROPOSAL DEFICIENCIES FOR NEGOTIATION DURING STEP 4 AFTER SELECTING THE COMPANY IN STEP 3. THE AIR FORCE ADMISSION OF THESE DEFICIENCIES (AS DEFINED IN THE SPECIAL ASPR PROVISION) IS ALSO AN EXPRESS ADMISSION THAT ITT'S PROPOSAL AT STEP 3 DID NOT MEET THE GOVERNMENT'S REQUIREMENTS. SINCE IT DID NOT MEET THE GOVERNMENT'S REQUIREMENTS, THE ABOVE-QUOTED PROVISION OF THE DEFENSE PROCUREMENT CIRCULAR SHOULD HAVE PREVENTED SELECTION OF ITT'S PROPOSAL. THE ONLY PERMISSIBLE CHANGES THAT MAY TAKE PLACE DURING STEP 4 PROPOSAL DISCUSSIONS ARE IMMATERIAL ONES. THE AMOUNT OF THE CHANGES PERMITTED HERE, AS WELL AS THE GRANTING TO ITT OF A 2-MONTH DELIVERY EXTENSION, ARE MATERIAL CHARGES.

GRANTED THAT PRESELECTION DISCUSSIONS ARE TO BE LIMITED IN SCOPE, THE AIR FORCE SHOULD HAVE DISCUSSED WITH ITT DEFICIENCIES IN ITS SOFTWARE, SINCE THE SEVERAL MILLION DOLLAR CHANGE IN ITT'S STEP 4 CONTRACT PRICE INDICATES THAT ITT HAD NOT SUBSTANTIATED ITS PROPOSED TECHNICAL APPROACH. THIS LACK OF A SUBSTANTIATED APPROACH IS AN AREA SPECIFICALLY MANDATED FOR DISCUSSIONS UNDER THE DIRECTIVE AND TEST ASPR PROVISION. MR. JANOFSKY OF THE AIR FORCE CONFIRMED THAT ITT'S SOFTWARE APPROACH HAD NOT BEEN SUBSTANTIATED AS LATE AS STEP 4.

ADDITIONALLY, THE AIR FORCE WAS MANDATED TO INVESTIGATE-- THROUGH DISCUSSIONS-- THE COST REALISM OF ITT'S PROPOSAL ESPECIALLY AS RELATED TO COMPLETELY UNDERSTANDING AN OFFEROR'S DELIVERY SCHEDULES, TRADEOFFS AND LIFE-CYCLE COSTS. THESE COST AND TECHNICAL DISCUSSIONS ARE AIMED AT SELECTING THE PROPOSAL WITH THE HIGHEST DEGREE OF REALISM AND CREDIBILITY. SINCE ITT'S PROPOSAL WAS INCREASED BY 35 PERCENT IT SHOULD NOT HAVE BEEN CONSIDERED COST REALISTIC.

THESE TECHNICAL AND COST DISCUSSIONS SHOULD HAVE BEEN HELD BEFORE SOURCE SELECTION. FOR EXAMPLE, THE SUBSTANTIAL DOLLAR CHANGE PERMITTED IN ITT'S STEP 4 PROPOSAL COULD CLEARLY HAVE BEEN DISCUSSED UNDER THE SPECIAL TEST ASPR PROVISION.

AIR FORCE:

(1) TAKEN TOGETHER, THE PROVISIONS OF DIRECTIVE 4105.62 (SECTIONS III.D.5.(B)(2)(A)-(B) AND III.D.5.B.(3)(A)) AND SPECIAL TEST ARMED SERVICES PROCUREMENT REGULATION (ASPR) 3-805.3(B) CREATE A VERY RESTRICTED BOUNDARY UNDER WHICH TECHNICAL DISCUSSIONS MAY BE HELD. UNDER THESE PROVISIONS, THE AIR FORCE WAS PREVENTED FROM DISCLOSING TECHNICAL DEFICIENCIES RELATING TO AN OFFEROR'S MANAGEMENT ABILITIES, ENGINEERING OR SCIENTIFIC JUDGMENT, OR LACK OF COMPETENCE OR INVENTIVENESS UNTIL STEP 4 OF THE PROCUREMENT. NEVERTHELESS, THE AIR FORCE RECOGNIZED THAT STEP 4 DISCUSSIONS COULD NOT INVOLVE MATERIAL CHANGES IN THE GOVERNMENT'S REQUIREMENTS OR THE CONTRACTOR'S PROPOSAL WHICH AFFECT THE SOURCE SELECTION. CONSEQUENTLY, IF, DURING STEP 4 DISCUSSIONS, THE AIR FORCE DISCOVERED THAT ITT'S PROPOSAL COULD NOT ACCOMPLISH THE AIMS OF SATIN IV, OR OTHER SIGNIFICANT DETAILS WERE DISCOVERED WHICH IF THOROUGHLY UNDERSTOOD AT THE TIME OF SELECTION WOULD HAVE AFFECTED SOURCE SELECTION, THE AIR FORCE WOULD HAVE REOPENED THE COMPETITION.

SYLVANIA:

(2) EVEN THOUGH THE AIR FORCE HAS DENIED THAT ITT WAS PERMITTED TO SUBSTITUTE #HIGH ORDER" FOR "LOW ORDER" LANGUAGE DURING STEP 4 NEGOTIATIONS, THE AIR FORCE HAS ADMITTED THAT ITT'S CONTRACT PRICE WAS INCREASED NEARLY 35-PERCENT OR $9 MILLION ON STEP 4. THE SHEER MAGNITUDE OF THESE CHANGES MAKES THEM MATERIAL AND A VIOLATION OF THE DIRECTIVE. THIS APPROACH PREJUDICED SYLVANIA AND OTHER OFFERORS BY DENYING OFFERORS THE OPPORTUNITY TO CORRECT DEFICIENCIES IN A COMPETITIVE ENVIRONMENT.

THE AIR FORCE APPROACH OF DEFERRING DISCUSSIONS OF ALL DEFICIENCIES TO STEP 4 FOR FEAR OF "TECHNICAL LEVELING" STILL RESULTS IN TECHNICAL LEVELING-- ALTHOUGH LIMITED TO THE SELECTED OFFEROR. THE PROHIBITION AGAINST LEVELING MUST EXTEND TO STEP 4.

AIR FORCE:

(2) NOTWITHSTANDING THAT MATERIAL CHANGES-- AMOUNTING TO A 35 PERCENT INCREASE IN THE PRICE OF ITT'S COST PROPOSAL-- OCCURRED ON STEP 4, THE CHANGES DID NOT AFFECT SOURCE SELECTION AND HENCE WERE PERMISSIBLE. ORDER TO CONSTITUTE A "MATERIAL CHANGE * * * WHICH AFFECTS) THE BASIS FOR SOURCE SELECTION" THE CHANGE MUST BE AN OCCURRENCE WHICH BOTH (1) WAS UNEXPECTED BY THE SOURCE SELECTION AUTHORITY AT THE TIME OF HIS SOURCE SELECTION (STEP 3) DECISION AND (2) WOULD HAVE CHANGED A FACTOR WHICH CONSTITUTED A SIGNIFICANT PORTION OF THE INPUTS USED BY THE AUTHORITY AT THE TIME OF HIS DECISION. WITHOUT THE FIRST, THE CHANGE WOULD NOT BE THE ONE WHICH WOULD AFFECT THE SELECTION; WITHOUT THE SECOND, THE CHANGE WOULD NOT BE ONE WHICH WOULD BE MATERIAL TO THE BASIS FOR SOURCE SELECTION. SYLVANIA:

(3) IT IS SHEER SPECULATION FOR THE AIR FORCE TO ASSERT THAT, HAD STEP 4 DISCUSSIONS BEEN CONDUCTED WITH SYLVANIA, SYLVANIA'S PROPOSED CONTRACT PRICE WOULD HAVE INCREASED $4-$7 MILLION. THE AIR FORCE TECHNIQUE OF AVOIDING NEGOTIATION OF ALL TECHNICAL AND COST DEFICIENCIES AND, PRIOR TO SOURCE SELECTION, DOING ITS OWN ESTIMATING OF WHAT IT WOULD COST TO CORRECT DEFICIENCIES MEANS THE END OF COMPETITION AS OF THE TIME ORIGINAL PROPOSALS WERE SUBMITTED. THE BENIGN QUESTIONS ASKED DURING ACTUAL DISCUSSIONS WERE NOT AIMED AT, AND DID NOT RESULT IN, MEANINGFUL DISCUSSIONS. IN ANY EVENT, SYLVANIA'S INCREASE IN PRICE WOULD STILL HAVE BEEN BELOW THE INCREASE AFFORDED ITT ON STEP 4. FOR EXAMPLE, IN THE SOFTWARE AREA, THE AIR FORCE CORRECTED ITT DEFICIENCIES THAT AMOUNTED TO AT LEAST $3.4 MILLION. NO SYLVANIA DEFICIENCIES IN THE SOFTWARE AREA WERE IDENTIFIED BY THE AIR FORCE. THEREFORE, THIS MAJOR PART OF THE ITT PRICE INCREASE WOULD NOT HAVE BEEN INCLUDED IN ANY NEGOTIATED SYLVANIA INCREASE AND THE PRICE DIFFERENCE BETWEEN SYLVANIA AND ITT WHICH EXISTED AT STEP 3 WOULD HAVE DISAPPEARED. THE AIR FORCE'S FAILURE TO GIVE DUE WEIGHT TO THE SOFTWARE DEFICIENCIES IN ITT'S PROPOSAL DESPITE THE PRIMARY FOCUS GIVEN SOFTWARE AND ASSOCIATED RISKS IN THE RFP UNDERSCORES THE MAJOR DEFECTS IN THE EVALUATION.

FURTHERMORE, A SIDE-BY-SIDE COMPARISON OF SYLVANIA DEFICIENCIES AS COMPARED WITH ITT DEFICIENCIES-- AS SHOWN IN A MARCH 22, 1977, AIR FORCE LETTER TO SYLVANIA-- CLEARLY EVIDENCES THAT SYLVANIA DEFICIENCIES WERE NOT AS SERIOUS (AND HENCE NOT AS COSTLY) AS THOSE OF ITT.

OF THE DEFICIENCIES IN SYLVANIA'S PROPOSAL, ONLY ONE WAS A TRUE TECHNICAL WEAKNESS RATHER THAN A PRESENTATION WEAKNESS. BY CONTRAST, ITT'S DEFICIENCIES RELATED TO THE TWO MOST IMPORTANT EVALUATION CRITERIA.

IN SUMMARY, THE PROCEDURE FOLLOWED BY THE AIR FORCE IMPROPERLY SERVED TO DEFER TO STEP 4 MANY MATTERS THAT SHOULD HAVE BEEN CORRECTED BY OFFERORS PRIOR TO SELECTION WHILE THE PROCUREMENT WAS STILL IN A COMPETITIVE PHASE. THE PROCEDURE PREMATURELY CUT SHORT COMPETITION AND RESULTED IN A SOLE-SOURCE PROCUREMENT BY ALLOWING AN OFFEROR TO PROVIDE A DEFICIENT PROPOSAL ON THE ASSUMPTION THAT IT COULD BE CORRECTED ON STEP 4.

AIR FORCE:

(3) THOSE CHANGES IN ITT'S PROPOSAL DURING STEP 4 WERE CHANGES WHICH WERE EXPECTED AT THE TIME OF THE STEP 3 SELECTION. AT THE END OF STEP 3 ALL OFFERORS REMAINING WITHIN THE COMPETITIVE RANGE HAD TECHNICAL WEAKNESSES AND RISKS WHICH COULD RESULT IN COST INCREASES. IN ADDITION TO TRADITIONAL COST ANALYSIS, SPECIFICALLY TAILORED ESTIMATES PROJECTED EACH OFFEROR'S MOST PROBABLE COST FOR THE WORK. THIS COST PROJECTION TECHNIQUE ENABLED COMPARATIVE ANALYSIS OF PROPOSED COSTS AND PROVIDED A PROJECTION OF THE LIKELY RESULTS OF STEP 4 DISCUSSIONS. THE PROJECTIONS WERE USED IN MAKING THE SELECTION. AS LONG AS STEP 4 DISCUSSIONS DID NOT INVOLVE CHANGES WHICH SIGNIFICANTLY AFFECTED THE PROJECTIONS ON WHICH THE SELECTION WAS BASED, THERE COULD HAVE BEEN NO "MATERIAL CHANGES WHICH AFFECTED) THE BASIS FOR SOURCE SELECTION." THESE CHANGES DID NOT OCCUR.

THE AIR FORCE ALSO REVIEWED THE COST INCREASE WHICH TOOK PLACE DURING STEP 4 DISCUSSIONS AND THE TECHNICAL CHANGES WHICH WERE MADE. THOSE COST INCREASES WHICH DID NOT OCCUR WERE WITHIN THE ESTIMATES WHICH HAD BEEN PROVIDED DURING STEP 3. ADDITIONALLY, AT THE END OF STEP 4, REVISED COST MODELS WERE USED TO REVALIDATE THE COST ANALYSES USED DURING STEP 3. THE SELECTION AUTHORITY RATIFIED THE STEP 3 SELECTION OF ITT ONLY AFTER RECEIVING AND REVIEWING IT.

SYLVANIA'S APPROACH ASSUMES THAT THERE WOULD HAVE BEEN NO REASON TO PREVENT THE AIR FORCE FROM DISCUSSING THE VARIOUS DEFICIENCIES IN THE ITT PROPOSAL PRIOR TO THE STEP 3 SOLICITATION. DISCUSSION OF TECHNICAL DEFICIENCIES RELATING TO LACK OF COMPETENCY OR THE PROBLEMS ATTENDING UNREALISTICALLY LOW COST PROPOSALS WAS PROHIBITED. ALTHOUGH THE AIR FORCE MIGHT HAVE REJECTED A PROPOSAL IF DEFICIENCIES AND PROBLEMS WERE PRESENT, IN QUESTIONABLE CASES WHERE DISCUSSIONS ARE DESIRED PROPOSALS SHOULD NOT BE REJECTED. HAD EXPANDED DISCUSSIONS BEEN CONDUCTED, ITT'S PROPOSAL CERTAINLY WOULD NOT HAVE BEEN ALONE IN UNDERGOING CHANGES.

FROM A COMPARISON BETWEEN THOSE AREAS DISCUSSED WITH ITT DURING STEP 4 AND THOSE AREAS OF WEAKNESSES IDENTIFIED IN A MARCH 22 AIR FORCE LETTER TO SYLVANIA, SYLVANIA SEEKS TO CREATE A COST PROJECTION OF ITS WEAKNESSES AND THEN COMPARE THAT WITH THE COST CHANGES NEGOTIATED IN ITT'S PROPOSAL. THE LETTER DOES NOT PURPORT, HOWEVER, TO CONTAIN A DETAILED LIST OF WEAKNESSES FROM WHICH SYLVANIA CAN MAKE COST PROJECTIONS. THIS FUNCTION IS DEFERRED TO POST-AWARD DEBRIEFING.

ANALYSIS-- ISSUE THREE

THE GENESIS OF THE "FOUR-STEP" PROCEDURES INVOLVED IN THE SUBJECT PROTEST LIES IN PROCEDURES ADOPTED SEVERAL YEARS AGO BY THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION (NASA). SPECIFICALLY, NASA PROCUREMENT DIRECTIVE 70-15, DECEMBER 1, 1970, PROVIDED THAT DURING DISCUSSIONS LEADING TO THE AWARD OF COST-REIMBURSEMENT CONTRACTS (OF THE TYPE AWARDED TO ITT HERE) "AMBIGUITIES AND UNCERTAINTIES IN THE PROPOSALS * * * SHALL BE POINTED OUT * * * BUT NOT DEFICIENCIES." NASA EXPLAINED ITS REASONING FOR ADOPTING THIS APPROACH IN RESPONDING TO A PROTEST WHICH WAS THE SUBJECT OF OUR DECISION IN B-173677(2), MARCH 31, 1972 (SUMMARIZED IN 51 COMP.GEN. 621 (1972)). THE EXPLANATION WAS RECITED IN B-173677(2), AS FOLLOWS:

IN 1968 AFTER (NASA'S) ATTENTION WAS DIRECTED BY (THE GENERAL ACCOUNTING) OFFICE TO A NUMBER OF NEGOTIATED PROCUREMENTS WHERE DISCUSSIONS HAD BEEN RATHER SHALLOW, NASA PROMULGATED PRD 69-5 PRESCRIBING A BROADER SCOPE FOR ORAL AND WRITTEN DISCUSSIONS BY PROVIDING THAT "DEFICIENCIES AND OMISSIONS AS WELL AS AMBIGUITIES" SHOULD BE POINTED OUT AND A REASONABLE OPPORTUNITY AFFORDED FOR SUPPORTING, CLARIFYING, CORRECTING, IMPROVING OR REVISING PROPOSALS. NASA BELIEVES THAT THIS WENT CONSIDERABLY BEYOND THE ARMED SERVICES PROCUREMENT REGULATION AND THE FEDERAL PROCUREMENT REGULATIONS, BOTH OF WHICH EMPHASIZED "COMPLETE AGREEMENT" AS THE OBJECTIVE AND CALLED FOR DISCUSSIONS "TO THE EXTENT NECESSARY TO RESOLVE UNCERTAINTIES." IT IS SAID THAT OUR DECISIONS EMPHASIZING THE CORRECTION OF DEFICIENCIES REFER TO ALL OF THESE REGULATIONS WITHOUT DRAWING DISTINCTIONS AMONG THEM AND HAVE EMPHASIZED THE POINTING OUT OF DEFICIENCIES AND WEAKNESSES AS WELL AS CLARIFICATION AND SUPPORT, CITING 50 COMP.GEN. 117, 123 (1970). IT IS CONTENDED, HOWEVER, THAT THIS AND OTHER DECISIONS EMPHASIZING THE CORRECTION OF DEFICIENCIES ARE ALL BASED ON THESE REGULATIONS WHICH EITHER REQUIRE OR PERMIT THE CORRECTION OF DEFICIENCIES ON THE INITIATIVE OF THE GOVERNMENT.

PRD 69-5 WAS SUPERSEDED ON DECEMBER 1, 1970, BY PRD 70-15 * * * .

IT IS REPORTED THAT THIS CHANGE WAS PROMPTED BY EXPERIENCE UNDER PRD 69-5 WHICH INDICATED THAT DISCUSSIONS INVOLVING DEFICIENCY CORRECTIONS HAD RESULTED IN A LEVELING PROCESS WITH THE FOLLOWING UNDESIRABLE RESULTS: THE REVISED PROPOSALS AS FINALLY EVALUATED WERE COMBINATIONS OF THE EFFORTS OF THE OFFERORS AND THE GOVERNMENT; PROSPECTIVE CONTRACTORS WERE DISCOURAGED FROM INITIALLY SUBMITTING THEIR BEST TECHNICAL PROPOSALS FOR FEAR OF BEING OVERTAKEN BY TECHNICALLY INFERIOR BUT LOWER COST OFFERORS; INDEPENDENT EFFORTS AS THE DETERMINING FACTOR IN THE COMPETITION WERE DISCOURAGED BECAUSE OF THE RISK OF BEING OVERTAKEN BY COMPANIES WITH GENERAL COMPETENCE AND GREATER RESOURCES FOR USING THE NEGOTIATION PROCESS TO UPGRADE THEIR PROPOSALS; ACTUAL OR SUSPECTED TECHNICAL TRANSFUSION RESULTED; AND THERE WAS AN OBLITERATION OF TECHNICAL DISTINCTIONS AND A RESULTING UNREALISTIC EMPHASIS ON COST ESTIMATES AS THE DECISIVE FACTOR.

FURTHERMORE, IT IS ARGUED THAT THERE IS A VALID BASIS FOR DISTINGUISHING BETWEEN RESEARCH AND DEVELOPMENT CONTRACTS AND COST REIMBURSEMENT CONTRACTS AS COMPARED TO FIXED-PRICE CONTRACTS NOT INVOLVING RESEARCH AND DEVELOPMENT, WHERE THERE ARE WELL DEFINED SPECIFICATIONS WITHIN THE STATE OF THE ART, IN TERMS OF THE EXTENT AND NATURE OF PROPER NEGOTIATION. THIS CONNECTION, IT IS STATED THAT JUST AS THE SCOPE AND DEPTH OF DISCUSSIONS DEPEND ON THE FACTS OF A PARTICULAR CASE, SO ALSO SHOULD THE RULES APPLICABLE TO NEGOTIATION DEPEND ON THE CHARACTERISTICS OF THE TYPE OF PROCUREMENT. MOREOVER, IT IS ASSERTED THAT THE CURRENT REGULATION PROJECTS AND FOSTERS THE COMPETITIVE RELATIONSHIP BETWEEN THE OFFERORS AND ASSURES THE INTEGRITY OF COMPETITION EVEN THOUGH DEFICIENCIES ARE NOT TO BE POINTED OUT DURING NEGOTIATION OF RESEARCH AND DEVELOPMENT CONTRACTS AND COST REIMBURSEMENT CONTRACTS. AMBIGUITIES AND UNCERTAINTIES ARE TO BE POINTED OUT, AND AN OPPORTUNITY GIVEN TO SUPPORT AND CLARIFY PROPOSALS. THE AIM OF DISCUSSIONS AS STATED IN THE REGULATION IS TO ASSIST THE EVALUATORS IN FULLY UNDERSTANDING THE PROPOSALS AND THEIR STRENGTHS AND WEAKNESSES BASED UPON THE INDIVIDUAL EFFORTS OF EACH OFFEROR; IN EVALUATING THE PERSONNEL PROPOSED BY EACH FIRM; AND IN PRESENTING A REPORT TO THE SELECTION OFFICIAL THAT MAKES THE DISCRIMINATIONS AMONG PROPOSALS CLEAR AND VISIBLE. THE REPORT TO THE SOURCE SELECTION OFFICIAL IS TO INCLUDE AN ESTIMATE OF THE POTENTIAL FOR CORRECTION OF THE PRINCIPAL WEAKNESSES IDENTIFIED, AS WELL AS AN ESTIMATE OF THE APPROXIMATE IMPACT ON COST OR PRICE THAT WILL RESULT FROM THE ELIMINATION OF CORRECTABLE WEAKNESSES.

NASA CONTENDS THAT THE STATUTORY REQUIREMENT (10 U.S.C. 2304(G)) FOR WRITTEN OR ORAL DISCUSSIONS IS BROAD AND GENERAL; THAT PROCURING AGENCIES HAVE AUTHORITY TO PRESCRIBE IMPLEMENTING RULES SO LONG AS THEY ARE NOT INCONSISTENT WITH STATUTE; THAT NASA PRD 70-15 IS A REASONABLE IMPLEMENTATION OF THE STATUTE AND NOT INCONSISTENT WITH IT OR DECISIONS OF THE COMPTROLLER GENERAL INTERPRETING THE STATUTE; AND THAT IN THE INSTANT CASE DISCUSSIONS WERE EXTENSIVE AND CONTRIBUTED TO A FAIR AND KEEN COMPETITION.

IT IS NASA'S POSITION THAT NOTHING IN THE LANGUAGE OF THE STATUTE, ITS LEGISLATIVE HISTORY, OR THE DECISIONS OF THE COMPTROLLER GENERAL IMPLY THAT THE STATUTE REQUIRES DISCUSSIONS WHICH ENCOMPASS A COMPLETE NEGOTIATION OF THE CONTRACT DOCUMENTS OR IDENTIFICATIONS OF WEAKNESSES. NASA POINTS OUT THAT THE STATUTE DOES NOT DEFINE THE EXTENT OF DISCUSSIONS REQUIRED AND THAT IN DRAFTING THE STATUTORY LANGUAGE ON THE POINT CONGRESS RECOGNIZED THE NEED FOR FLEXIBILITY, CITING THE FOLLOWING FROM SENATE REPORT NO. 1884, AUGUST 17, 1962:

"IF DISCUSSIONS ARE UNNECESSARY IN THE ORDINARY CASE, IT IS DIFFICULT TO UNDERSTAND THAT THE PROCUREMENT COULD NOT HAVE BEEN ACCOMPLISHED BY FORMAL ADVERTISING. AT THE SAME TIME, AN INFLEXIBLE REQUIREMENT FOR DISCUSSIONS WITH ALL OFFERORS COULD ENCOURAGE THE OFFERORS TO PAD THEIR INITIAL PROPOSALS AND NOT QUOTE THEIR BEST PRICES FIRST."

MOREOVER, NASA POINTS OUT, OUR OFFICE HAS RECOGNIZED THAT THE CIRCUMSTANCES WHICH NECESSITATE A NEGOTIATED PROCUREMENT ALSO NECESSITATE THE EXERCISE OF DISCRETION ON THE PART OF THE CONTRACTING OFFICER IN DETERMINING THE EXTENT OF SUCH NEGOTIATIONS, CITING B-170855, DECEMBER 21, 1970; B-169043, JUNE 16, 1970. THEREFORE, NASA CONTENDS THAT IT HAS BROAD AUTHORITY TO PROMULGATE IMPLEMENTING REGULATIONS WHICH, INSOFAR AS THEY ARE NOT INCONSISTENT WITH THE STATUTE, HAVE THE FORCE AND EFFECT OF LAW, CITING G. L. CHRISTIAN V. UNITED STATES, 160 CT.CL. 1, 312 F.2D 418; 160 CT.CL. 58, 320 F.2D 345; CERT. DENIED 375 U.S. 954 (1963); STEINTHAL & COMPANY V. SEAMANS ETC., ET AL., CCA D.C. NO. 24,595 (OCTOBER 14, 1971).

COUNSEL FOR THE PROTESTER IN B-173677(2), SUPRA, CITED CERTAIN OF OUR DECISIONS (SEE, FOR EXAMPLE, 47 COMP.GEN. 336 (1967) AND 50 COMP.GEN. 117 (1970)), WHICH CONTAIN STATEMENTS TO THE EFFECT THAT FOR COMPETITIVE NEGOTIATION TO BE MEANINGFUL, OFFERORS SHOULD BE INFORMED OF "WEAKNESSES, EXCESSES OR DEFICIENCIES" IN ORDER TO ENABLE OFFERORS TO UPGRADE THEIR PROPOSALS AND PROVIDE SUFFICIENT INFORMATION NECESSARY TO PERMIT EVALUATION OF THE PROPOSALS. BECAUSE OF THE POSITIONS IN THESE DECISIONS, COUNSEL ARGUED THAT NASA PROCUREMENT DIRECTIVE 70-15 WAS CONTRARY TO THE PROVISIONS OF 10 U.S.C. 2304(H) AND THAT THE DISCUSSIONS HELD WITH THE PROTESTER WERE NOT MEANINGFUL.

ON THE OTHER HAND, NASA AND COUNSEL FOR AN INTERESTED PARTY NOTED THAT NEGOTIATION PROCEDURES ARE DESIGNED TO BE FLEXIBLE AND INFORMAL AND THAT PROCURING AGENCIES ARE PERMITTED BROAD DISCRETION IN THE CONDUCT OF DISCUSSIONS (SEE 47 COMP.GEN. 279 (1967); 49 ID. 625 (1970); B-169042, JUNE 16, 1970); THAT THE ISSUANCE OF AMENDMENTS AND AN OPPORTUNITY TO REVISE PROPOSALS CONSTITUTE DISCUSSIONS (50 COMP.GEN. 202 (1970)); THAT TO POINT OUT EVERY AREA IN WHICH ANOTHER OFFEROR HAS ACHIEVED A HIGHER POINT SCORE OR PROVIDED DETAIL IS NOT REQUIRED (B-164552, FEBRUARY 24, 1969); AND THAT THE CORRECTION OF PROPOSAL UNCERTAINTIES COULD CONSTITUTE MEANINGFUL DISCUSSIONS (51 COMP.GEN. 102 (1971)).

WE RECOGNIZED THAT, ALTHOUGH THE PROVISIONS OF 10 U.S.C. 2304(G) (1970) DO NOT DEFINE THE NATURE, SCOPE OR EXTENT OF THE REQUIRED DISCUSSIONS, THE LEGISLATIVE HISTORY OF THE LAW EVIDENCED A CONGRESSIONAL INTENT THAT NEGOTIATIONS BE CONDUCTED UNDER COMPETITIVE PROCEDURES TO THE EXTENT PRACTICABLE AND THAT THEY BE "MEANINGFUL BY MAKING THEM DISCUSSIONS IN FACT AND NOT JUST LIP-SERVICE." WE FURTHER OBSERVED: THE MANY DECISIONS CITED BY THE PARTIES TO THIS PROTEST, AS WELL AS OTHERS DEALING WITH THE MATTER OF "DISCUSSIONS," WERE NOT DECIDED IN A VACUUM OR INTENDED TO BE MERELY ABSTRACT STATEMENTS OF LAW. THEY INVOLVED ACTUAL DISPUTES CONCERNING THE CONDUCT OF NEGOTIATIONS FOR VARIOUS SERVICES AND SUPPLIES, RANGING FROM MAINTENANCE SERVICES TO SOPHISTICATED ELECTRONIC EQUIPMENT; THE JUSTIFICATIONS FOR NEGOTIATION INVOLVED MANY OF THE 17 EXCEPTIONS TO FORMAL ADVERTISING, INCLUDING PUBLIC EXIGENCY, RESEARCH AND DEVELOPMENT, AND PROPERTY OR SERVICES FOR WHICH IT WAS IMPRACTICABLE TO OBTAIN COMPETITION; AND THE METHODS OF CONTRACTING INCLUDED FIXED PRICE AND ONE OF SEVERAL COST REIMBURSEMENT TYPES. NECESSARILY, THESE VARIED PROCUREMENTS INVOLVED DIFFERENT CONSIDERATIONS, REQUIRING JUDGMENTS AS TO THE METHODS AND TECHNIQUES UTILIZED IN CONSUMMATING THE CONTRACTS. RECOGNITION OF THESE FACTS, WE HAVE NOT CONSTRUED THE REQUIREMENT FOR "WRITTEN OR ORAL DISCUSSIONS" AS AN INFLEXIBLE, STEREOTYPED MANDATE UNRELATED TO THE PARTICULAR PROCUREMENT INVOLVED. THUS, IN MANY CASES WE HAVE FOUND THAT DEFICIENCIES HAD TO BE POINTED OUT IN ORDER TO HAVE MEANINGFUL DISCUSSIONS. ON THE OTHER HAND, IN OTHER CASES THE FACTS AND CIRCUMSTANCES CALLED FOR A DIFFERENT CONCLUSION. FOR EXAMPLE, IN 50 COMP.GEN. 202 (1970), WHICH NASA HAS CITED AS AN INSTANCE WHERE WE HELD THAT THE MERE ACCEPTANCE, IN EFFECT, OF A LATE REVISION CONSTITUTED DISCUSSIONS UNDER 10 U.S.C. 2304(G), THE ISSUE WAS WHETHER THE OTHER OFFERORS SHOULD ALSO BE GIVEN AN OPPORTUNITY TO REVISE THEIR INITIAL PROPOSALS. WE STATED THAT SINCE DISCUSSION HAD BEEN CONDUCTED WITH ONE OFFEROR, DISCUSSIONS MUST BE CONDUCTED WITH ALL OFFERORS WITHIN THE COMPETITIVE RANGE. IN B-170297, MAY 26, 1971, ALSO CITED BY NASA, THE PROCUREMENT CALLED FOR A QUANTITY OF GENERATORS ON A FIRM FIXED-PRICE BASIS. ADDITIONAL TESTS WERE REQUIRED AFTER THE INITIAL PROPOSALS WERE RECEIVED, AND THE OFFERORS WERE REQUESTED TO SUBMIT REVISED PRICES TO REFLECT THESE ADDITIONAL TESTS. AWARD WAS MADE AFTER RECEIPT OF THE REVISED PRICES. IT WAS CONTENDED IN PART THAT THESE PROCEEDINGS DID NOT CONSTITUTE "ORAL OR WRITTEN DISCUSSIONS" BUT RATHER THE ACCEPTANCE OF AN INITIAL PROPOSAL WITHOUT DISCUSSIONS. WE DISAGREED WITH THIS CONTENTION BUT STATED THAT, "WE DO NOT MEAN TO DISCOURAGE MORE EXTENSIVE NEGOTIATIONS OF PRICE IN SIMILAR SITUATIONS NOR TO IMPLY THAT THEY WOULD BE INAPPROPRIATE." THUS, WE HAVE ATTEMPTED TO RESOLVE THESE DISPUTES NOT ONLY IN LIGHT OF THE PARTICULAR PROCUREMENT, BUT IN RECOGNITION OF THE CLEAR CONGRESSIONAL MANDATE AS EVIDENCED BY THE LEGISLATIVE HISTORY OF 2304(H), FOR COMPETITIVE NEGOTIATIONS DESIGNED TO OBTAIN FOR THE GOVERNMENT THE MOST ADVANTAGEOUS CONTRACT.

THEREFORE, IT IS OUR VIEW THAT WHETHER THE STATUTORY REQUIREMENT FOR DISCUSSIONS MUST INCLUDE THE POINTING OUT OF DEFICIENCIES, AND THE EXTENT THEREOF, IS A MATTER OF JUDGMENT PRIMARILY FOR DETERMINATION BY THE PROCURING AGENCY IN LIGHT OF ALL THE CIRCUMSTANCES OF THE PARTICULAR PROCUREMENT AND THE REQUIREMENT FOR COMPETITIVE NEGOTIATIONS, AND THAT SUCH DETERMINATION IS NOT SUBJECT TO QUESTION BY OUR OFFICE UNLESS CLEARLY ARBITRARY OR WITHOUT A REASONABLE BASIS. HOWEVER, THE STATUTE SHOULD NOT BE INTERPRETED IN A MANNER WHICH DISCRIMINATES AGAINST OR GIVES PREFERENTIAL TREATMENT TO ANY COMPETITOR. ANY DISCUSSION WITH COMPETING OFFERORS RAISES THE QUESTION AS TO HOW TO AVOID UNFAIRNESS AND UNEQUAL TREATMENT. OBVIOUSLY, DISCLOSURE TO OTHER PROPOSERS OF ONE PROPOSER'S INNOVATIVE OR INGENIOUS SOLUTION TO A PROBLEM IS UNFAIR. WE AGREE THAT SUCH "TRANSFUSION" SHOULD BE AVOIDED. IT IS ALSO UNFAIR, WE THINK, TO HELP ONE PROPOSER THROUGH SUCCESSIVE ROUNDS OF DISCUSSIONS TO BRING HIS ORIGINAL INADEQUATE PROPOSAL UP TO THE LEVEL OF OTHER ADEQUATE PROPOSALS BY POINTING OUT THOSE WEAKNESSES WHICH WERE THE RESULT OF HIS OWN LACK OF DILIGENCE, COMPETENCE, OR INVENTIVENESS IN PREPARING HIS PROPOSAL.

WE THINK THE PROPRIETY OF THE PROHIBITION IN NASA PROCUREMENT DIRECTIVE 70-15 AGAINST DISCUSSING "DEFICIENCIES" MUST BE CONSIDERED IN THE LIGHT OF THESE PROBLEMS. WE THINK CERTAIN WEAKNESSES, INADEQUACIES, OR DEFICIENCIES IN PROPOSALS CAN BE DISCUSSED WITHOUT BEING UNFAIR TO OTHER PROPOSERS. THERE WELL MAY BE INSTANCES WHERE IT BECOMES APPARENT DURING THE COURSE OF NEGOTIATIONS THAT ONE OR MORE PROPOSERS HAVE REASONABLY PLACED EMPHASIS ON SOME ASPECT OF THE PROCUREMENT DIFFERENT FROM THAT INTENDED BY THE SOLICITATION. UNLESS THIS DIFFERENCE IN THE MEANING GIVEN THE SOLICITATION IS REMOVED, THE PROPOSERS ARE NOT COMPETING ON THE SAME BASIS. LIKEWISE, IF A PROPOSAL IS DEEMED WEAK BECAUSE IT FAILS TO INCLUDE SUBSTANTIATION FOR A PROPOSED APPROACH OR SOLUTION, WE BELIEVE THE PROPOSER SHOULD BE GIVEN THE OPPORTUNITY, TIME PERMITTING, TO FURNISH SUCH SUBSTANTIATION. THUS, IT SEEMS TO US THAT THE PROHIBITION IN NASA PROCUREMENT DIRECTIVE 70-15 AGAINST DISCUSSING "DEFICIENCIES" NEEDS CLARIFICATION.

DESPITE OUR BELIEF THAT THE DIRECTIVE NEEDED TO BE CLARIFIED, WE WERE UNABLE TO CONCLUDE-- BASED ON ANALYSIS OF THE PARTICULAR FACTS INVOLVED-- THAT THE NEGOTIATIONS HAD WITH THE PROTESTER "DID NOT COMPORT WITH THE STATUTORY MANDATE FOR ORAL OR WRITTEN DISCUSSIONS." PARTICULAR FACTS ENTERING INTO THIS CONCLUSION WERE:

1. THE PROTESTER HAD CONSIDERABLE "INFORMAL AND FORMAL CONTACT" REGARDING TECHNICAL REQUIREMENTS OF THE PROCUREMENT FOR A 1-YEAR PERIOD PRIOR TO SUBMITTING A PROPOSAL;

2. THE PROCUREMENT WAS FOR RESEARCH AND DEVELOPMENT AND REQUESTED INDEPENDENT APPROACHES SUBSTANTIATED BY EXTENSIVE DATA;

3. MANY OF THE PROTESTER'S WEAKNESSES RESULTED FROM FAILURE TO SUBMIT BACKUP DATA;

4. WRITTEN AND ORAL DISCUSSIONS WERE IN FACT CONDUCTED ALTHOUGH THEY DID NOT INCLUDE POINTING OUT OF DEFICIENCIES AS SUCH;

5. MANY OF THE TECHNICAL QUESTIONS ASKED DID RELATE TO AREAS LATER JUDGED WEAK, ALTHOUGH THEY WERE FRAMED IN THE CONTEXT OF CLARIFICATIONS;

6. THE PROTESTER DID SUBMIT SUBSTANTIAL REVISIONS TO ITS PROPOSALS;

7. ALTHOUGH SOME INFORMATIONAL DEFICIENCIES IN ONE AREA OF THE PROTESTER'S PROPOSAL MIGHT HAVE BEEN THE SUBJECT OF "FRUITFUL DISCUSSIONS," ANY POSSIBLE UPGRADING OF THE PROTESTER'S PROPOSAL IN THIS ONE AREA WOULD HAVE BEEN INSIGNIFICANT BECAUSE THE SOURCE SELECTION OFFICIAL'S AWARD DECISION WAS PRIMARILY BASED ON A PROPER CONSIDERATION-- CONFIDENCE IN ENGINE DESIGN-- NOT INVOLVING THIS ONE AREA; AND

8. THE WEAKNESSES IN THE PROTESTER'S PROPOSAL WERE DEFICIENCIES ONLY IN COMPARISON WITH RELATIVE STRENGTHS OF THE SELECTED COMPANY; THEREFORE, DISCUSSIONS CONCERNING DEFICIENCIES IN COMPARATIVE WEAKNESSES WOULD INITIALLY HAVE INVOLVED TECHNICAL LEVELING.

IN RESPONSE TO OUR EXPRESSED CONCERN THAT THE PROHIBITION AGAINST DISCUSSING DEFICIENCIES IN NASA PROCUREMENT DIRECTIVE 70-15 NEEDED CLARIFYING, NASA ISSUED REVISED PROCUREMENT DIRECTIVE 70-15 WHICH PROVIDED:

* * * COST-REIMBURSEMENT TYPE CONTRACTS AND ALL CONTRACTS FOR RESEARCH AND DEVELOPMENT. THE CONTRACTING OFFICER, IN CONCERT WITH OR ON BEHALF OF THE SEB, WILL CONDUCT WRITTEN OR ORAL DISCUSSIONS OF THE WORK TO BE DONE AND THE COST OF THE WORK WITH THOSE CONCERNS WHOSE PROPOSALS ARE WITHIN THE COMPETITIVE RANGE. THE DISCUSSIONS ARE INTENDED TO ASSIST THE SEB OR OTHER EVALUATORS (I) IN UNDERSTANDING FULLY THE PROPOSALS AND THEIR STRENGTHS AND WEAKNESSES BASED UPON THE INDIVIDUAL EFFORTS OF EACH PROPOSER; (II) IN ASSURING THAT THE MEANINGS AND THE POINTS OF EMPHASIS OF RFP PROVISIONS HAVE BEEN ADEQUATELY CONVEYED TO THE OFFERORS SO THAT ALL ARE COMPETING EQUALLY ON THE BASIS INTENDED BY THE GOVERNMENT; (III) IN EVALUATING THE PERSONNEL PROPOSED BY EACH FIRM; AND (IV) IN PRESENTING A REPORT TO THE SELECTION OFFICIAL THAT MAKES THE DISCRIMINATIONS AMONG PROPOSALS CLEAR AND VISIBLE. IN THIS PROCESS, PRIOR TO CONTRACTOR SELECTION, THE GOVERNMENT'S INTERESTS ARE NOT SERVED BY ITS ASSUMING THE ROLE OF AN INFORMATION EXCHANGE OR CLEARING-HOUSE.

IN COST-REIMBURSEMENT TYPE CONTRACTS AND ALL RESEARCH AND DEVELOPMENT CONTRACTS, THE CONTRACTING OFFICER SHALL POINT OUT INSTANCES IN WHICH THE MEANING OF SOME ASPECT OF A PROPOSAL IS NOT CLEAR; AND INSTANCES IN WHICH SOME ASPECT OF THE PROPOSAL FAILS TO INCLUDE SUBSTANTIATION FOR A PROPOSED APPROACH, SOLUTION, OR COST ESTIMATE.

HOWEVER, WHERE THE MEANING OF A PROPOSAL IS CLEAR, AND WHERE THE BOARD HAS ENOUGH INFORMATION TO ASSESS ITS VALIDITY, AND THE PROPOSAL CONTAINS A WEAKNESS WHICH IS INHERENT IN A PROPOSER'S MANAGEMENT, ENGINEERING, OR SCIENTIFIC JUDGMENT, OR IS THE RESULT OF ITS OWN LACK OF COMPETENCE OR INVENTIVENESS IN PREPARING ITS PROPOSAL, THE CONTRACTING OFFICER SHALL NOT POINT OUT THE WEAKNESSES. DISCUSSIONS ARE USEFUL IN ASCERTAINING THE PRESENCE OR ABSENCE OF STRENGTHS AND WEAKNESSES. THE POSSIBILITY THAT SUCH DISCUSSIONS MAY LEAD AN OFFEROR TO DISCOVER THAT IT HAS A WEAKNESS IS NOT A REASON FOR FAILING TO INQUIRE INTO A MATTER WHERE THE MEANING IS NOT CLEAR OR WHERE INSUFFICIENT INFORMATION IS AVAILABLE, SINCE UNDERSTANDING OF THE MEANING AND VALIDITY OF THE PROPOSED APPROACHES, SOLUTIONS, AND COST ESTIMATES IS ESSENTIAL TO A SOUND SELECTION. PROPOSERS SHOULD NOT BE INFORMED OF THE RELATIVE STRENGTHS OR WEAKNESSES OF THEIR PROPOSALS IN RELATION TO THOSE OF OTHER PROPOSERS. TO DO SO WOULD BE CONTRARY TO OTHER REGULATIONS WHICH PROHIBIT THE USE OF "AUCTION TECHNIQUES." IN THE COURSE OF DISCUSSIONS, GOVERNMENT PARTICIPANTS SHOULD BE CAREFUL NOT TO TRANSMIT INFORMATION WHICH COULD GIVE LEADS TO ONE PROPOSER AS TO HOW ITS PROPOSAL MAY BE IMPROVED OR WHICH COULD REVEAL A COMPETITOR'S IDEAS.

THE FOREGOING GUIDELINES ARE NOT ALL-INCLUSIVE; CAREFUL JUDGMENT MUST BE EXERCISED IN THE LIGHT OF ALL THE CIRCUMSTANCES OF EACH PROCUREMENT TO PROMOTE THE MOST ADVANTAGEOUS SELECTION FROM THE STANDPOINT OF THE GOVERNMENT WHILE AT THE SAME TIME MAINTAINING THE FAIRNESS OF THE COMPETITIVE PROCESS.

THE EVALUATORS SHOULD ESTIMATE * * * THE APPROXIMATE EFFECT ON COST OR PRICE THAT WILL RESULT FROM THE ELIMINATION OF CORRECTABLE WEAKNESSES DURING NEGOTIATIONS AFTER SELECTION. (THE IDENTICAL PROVISIONS ARE FOUND IN NASA PROCUREMENT DIRECTIVE 70-15, DECEMBER 3, 1975, CURRENTLY IN EFFECT.)

INSTEAD OF THE BLANKET PROHIBITION AGAINST THE DISCUSSION OF DEFICIENCIES CONTAINED IN THE 1970 NASA PROCUREMENT DIRECTIVE, THE 1972 AND 1975 NASA PROCUREMENT DIRECTIVES OMITTED MENTION OF THE WORD "DEFICIENCY" AND EMPHASIZED THE FOLLOWING POINTS:

1. ALTHOUGH THE GOVERNMENT'S INTERESTS ARE NOT SERVED BY ITS ASSUMING THE ROLE OF AN INFORMATION EXCHANGE PRIOR TO CONTRACTOR SELECTION, THE GOVERNMENT (CONTRACTING OFFICER) SHOULD INSURE THE MEANINGS AND THE POINTS OF EMPHASIS OF THE RFP PROVISIONS HAVE BEEN ADEQUATELY CONVEYED TO THE OFFERORS SO THAT ALL ARE COMPETING EQUALLY;

2. THE CONTRACTING OFFICER SHOULD POINT OUT INSTANCES WHERE A PROPOSAL IS EITHER NOT CLEAR OR A PROPOSED APPROACH, SOLUTION OR COST ESTIMATE HAS NOT BEEN SUBSTANTIATED;

3. WEAKNESSES RELATED TO LACK OF COMPETENCE AND INVENTIVENESS SHALL NOT BE POINTED OUT;

4. OFFERORS SHOULD NOT BE INFORMED OF THE RELATIVE STRENGTHS OR WEAKNESSES OF THEIR PROPOSALS; AND

5. THE APPROXIMATE COST OF CORRECTING WEAKNESSES IN A PROPOSAL SHOULD BE PROJECTED FOR USE IN SOURCE SELECTION.

THE OBSERVATIONS MADE IN B-173677(2), SUPRA, HAVE BEEN USED AS GUIDING PRINCIPLES IN DECIDING SEVERAL OTHER NASA PROTESTS. SEE, FOR EXAMPLE, LOCKHEED PROPULSION COMPANY; THIOKOL CORPORATION, 53 COMP.GEN. 977 (1974), 74-1 CPD 339; SPERRY RAND CORPORATION ET AL., 54 COMP.GEN. 408 (1974), 74-2 CPD 276; DYNALECTRON CORPORATION; LOCKHEED ELECTRONICS COMPANY, INC., 54 COMP.GEN. 562 (1975), 75-1 CPD 17; MANAGEMENT SERVICES, INC., 55 COMP.GEN. 715 (1976), 76-1 CPD 74; UNION CARBIDE CORPORATION, 55 COMP.GEN. 802 (1976), 76-1 CPD 134.

IN LOCKHEED PROPULSION COMPANY, SUPRA, THE PROTESTER ALLEGED THAT NASA'S 1972 PROCUREMENT DIRECTIVE IMPROPERLY ELIMINATED THE NEED FOR AN OFFEROR TO RESPOND TO FINDINGS OF TECHNICAL WEAKNESSES BY PROSCRIBING DISCUSSIONS RELATED TO DESIGN WEAKNESSES. LOCKHEED ARGUED THAT NASA'S TECHNIQUE OF CORRECTING DESIGN WEAKNESSES ONLY AFTER SELECTION OF A COST-REIMBURSEMENT CONTRACTOR-- IN THIS CASE THIOKOL-- PUT "NASA EXPERTISE TO WORK IN BEHALF OF THIOKOL" AND RESULTED IN A CONTRACT MATERIALLY DIFFERENT FROM THE CONTRACT PROPOSED BY THIOKOL. ADDITIONALLY, THE PROTESTER CONTENDED THAT THE DEFICIENCIES SHOULD NOT HAVE BEEN MADE THE SUBJECT OF A "COST CORRECTION" UNDER THE PROVISIONS OF THE PROCUREMENT DIRECTIVE BUT RATHER SHOULD HAVE RESULTED IN REJECTION OF THE PROPOSAL.

IN REPLY, WE EMPHASIZED, CITING B-173677(2), SUPRA, THE AUTHORITY OF THE PROCURING AGENCY TO DECIDE-- SUBJECT TO A TEST OF REASONABLENESS-- THE MANNER OF COMPLYING WITH THE STATUTORY REQUIREMENTS FOR DISCUSSIONS IN NEGOTIATED PROCUREMENTS. MOREOVER, SINCE WE COULD NOT CONCLUDE THAT ANY SINGLE DEFICIENCY OR AGGREGATE OF WEAKNESSES IN THIOKOL'S PROPOSAL COULD BE CATEGORIZED AS MAJOR WEAKNESSES, WE COULD NOT CONCLUDE THAT NASA WAS (1) REQUIRED TO DISCUSS THESE DEFICIENCIES WITH THIOKOL PRIOR TO SELECTING THE COMPANY; (2) PROHIBITED FROM PROJECTING THE COSTS NEEDED TO CORRECT THESE DEFICIENCIES AS A TECHNIQUE TO BE USED IN SELECTING THE SUCCESSFUL OFFEROR (IN THIS CASE NASA COST ADJUSTMENTS TO OFFERORS' PROPOSALS AMOUNTED TO $27 MILLION); (3) PROHIBITED FROM REFUSING TO DISCUSS THESE ADJUSTMENTS WITH THE OFFERORS; OR (4) PROHIBITED FROM CORRECTING THE DEFICIENCIES PURSUANT TO DISCUSSIONS WITH THIOKOL AFTER SOURCE SELECTION. FURTHER, ALTHOUGH WE HAD SOME QUESTIONS ABOUT THE PROPRIETY OF SOME OF THE COST ADJUSTMENTS MADE BY NASA, WE DID NOT QUESTION THE PREMISE IMPLICIT IN THE COST ADJUSTMENT TECHNIQUE, NAMELY, THAT THE PROCUREMENT LAWS DO NOT PROHIBIT THE ADJUSTMENT OF OFFERORS' PROPOSED COSTS-- EVEN IF THE ADJUSTMENTS RUN INTO THE MILLIONS OF DOLLARS-- AND DO NOT REQUIRE DISCUSSION OF THE ADJUSTMENTS WITH THE OFFERORS PRIOR TO SELECTION SO LONG AS THE ADJUSTMENTS RELATE TO CORRECTION OF WEAKNESSES WHICH ARE NOT OTHERWISE FOR DISCUSSION. IN SPERRY RAND CORPORATION, SUPRA, WE OBSERVED:

THE NASA PROCEDURE REPRESENTS ONE APPROACH TO MEETING THE STATUTORY REQUIREMENT FOR WRITTEN AND ORAL DISCUSSIONS, 10 U.S.C. 2304(G). IN PART, AT LEAST, THE UNDERLYING RATIONALE IS THAT TO POINT OUT (CERTAIN WEAKNESSES) DURING THE DISCUSSIONS WOULD COMPROMISE THE COMPETITION, BECAUSE WEAKER PROPOSALS WOULD BE IMPROVED, AND A LEVELING EFFECT WOULD OCCUR. TO AVOID THIS, DISCUSSIONS ARE LIMITED TO CLARIFICATION OF PROPOSALS; AFTER SELECTION, THE AGENCY THEN NEGOTIATES THE BEST POSSIBLE CONTRACT ON TERMS MOST ADVANTAGEOUS TO THE GOVERNMENT. CONSIDERED IN THE ABSTRACT, POTENTIAL CONFLICTS BETWEEN THE PROCEDURE AND THE STATUTORY REQUIREMENT CAN BE ENVISIONED; FOR INSTANCE, AS APPEARS TO BE CONTEMPLATED BY UNIVAC, A SITUATION WHERE THE DISCUSSIONS ARE SO LIMITED IN SCOPE AND CONTENT THAT THEY AMOUNT TO LITTLE MORE THAN A CEREMONIAL EXERCISE, WITH THE MEANINGFUL DISCUSSIONS TRANSPORTED ALMOST ENTIRELY INTO THE FINAL NEGOTIATIONS STAGE.

NOTWITHSTANDING OUR RESERVATIONS ABOUT THE POSSIBILITY OF CEREMONIAL NEGOTIATIONS, WE FOUND THAT THE PROTESTER HAD ALLEGED THE LACK OF MEANINGFUL DISCUSSIONS LARGELY IN THE ABSTRACT. ON THIS FINDING, AND, AFTER REVIEWING THE RECORD OF DISCUSSIONS CONDUCTED, WE COULD NOT CONCLUDE THAT NASA HAD VIOLATED THE STATUTORY MANDATE FOR DISCUSSIONS. ADDITIONALLY, WE REJECTED RELATED COMPLAINTS THAT NASA HAD IMPROPERLY PROJECTED THE COST OF CORRECTING THE PROTESTER'S DEFICIENCIES. WE ALSO NOTED:

THE FACT THAT THE EVALUATORS JUDGED THAT A DEFICIENCY IN ONE PROPOSAL REQUIRED AN UPWARD ADJUSTMENT, WHILE A DEFICIENCY IN ANOTHER PROPOSAL DID NOT SIGNIFICANTLY IMPACT ITS COST, DOES NOT PROVE THAT THE EVALUATION OF EITHER WAS IMPROPER.

IN DYNALECTRON CORPORATION, SUPRA, WE DID NOT QUESTION NASA'S DECISION TO CONSIDER A PROPOSAL WEAKNESS INVOLVING RETENTION OF PROPOSED PERSONNEL STEMMING FROM PROPOSED SALARY REDUCTIONS AS FALLING WITHIN THE PROCUREMENT DIRECTIVE'S LIST OF WEAKNESSES THAT MAY NOT BE DISCUSSED WITH OFFERORS. SIMILARLY, IN MANAGEMENT SERVICES, INC., SUPRA, WE AGREED THAT NASA PROPERLY OMITTED DISCUSSION OF A WEAKNESS STEMMING FROM AN OFFEROR'S FAILURE TO USE APPROPRIATE WATER RATE INFORMATION IN ITS PROPOSAL AND PROPERLY ADJUSTED THE OFFEROR'S COST PROPOSAL BECAUSE OF THIS WEAKNESS ALTHOUGH WE EXPRESSED SOME RESERVATIONS ABOUT THE ADEQUACY OF THE COST ANALYSES INVOLVED. FINALLY, IN UNION CARBIDE CORPORATION, SUPRA, WE DISAGREED WITH NASA'S VIEW THAT AN OFFEROR'S REQUEST FOR DIRECT REIMBURSEMENT BY THE GOVERNMENT OF ITS INTEREST EXPENSE WAS AN INNOVATIVE IDEA NOT SUBJECT TO DISCUSSION WITH OFFERORS WHO HAD NOT PROPOSED REIMBURSEMENT. ON THE CONTRARY, WE THOUGHT THE REQUEST FOR REIMBURSEMENT WAS A DEPARTURE FROM PROCUREMENT "GROUND RULES" WHICH SHOULD HAVE BEEN COMMUNICATED TO ALL OFFERORS.

DOD'S ADOPTION OF THE SUBSTANCE OF THE NASA PROCEDURES

THE PERCEIVED ADVANTAGES OF NASA'S PROCEDURES PROMPTED THE DEPARTMENT OF DEFENSE TO ISSUE SIMILAR PROCEDURES. THUS, DEFENSE PROCUREMENT CIRCULAR #75-7 AND "SPECIAL TEST" ASPR SEC. 3-805.3 WERE PROMULGATED. A SIDE-BY- SIDE COMPARISON OF THE NASA AND DOD PROCEDURES IS AS FOLLOWS:

NASA:

(1) DISCUSSIONS SHALL BE CONDUCTED WITH THOSE CONCERNS WHOSE PROPOSALS ARE IN THE COMPETITIVE RANGE. THE GOVERNMENT, HOWEVER, IS NOT TO BE A "CLEARING HOUSE." EACH COMPETITIVE-RANGE OFFEROR SHALL BE GIVEN A REASONABLE OPPORTUNITY TO SUPPORT AND CLARIFY ITS PROPOSAL.

DOD

(1) OFFERORS ELECTED TO PARTICIPATE IN DISCUSSIONS SHALL BE INFORMED OF DEFICIENCIES AND GIVEN A REASONABLE OPPORTUNITY TO CORRECT THE DEFICIENCIES WITH CERTAIN EXCEPTIONS. A DEFICIENCY IS DEFINED AS THAT PART OF A PROPOSAL WHICH DOES NOT SATISFY THE GOVERNMENT'S REQUIREMENTS.

NASA:

(2) DISCUSSIONS ARE HELD TO ENSURE THAT OFFERORS UNDERSTAND THE MEANING AND POINTS OF EMPHASIS OF THE RFP PROVISIONS; TO POINT OUT UNCLEAR PARTS OF PROPOSALS; AND TO ALLOW AN OFFEROR TO INCLUDE SUBSTANTIATION FOR A PROPOSED APPROACH, SOLUTION AND COST ESTIMATE.

DOD:

(2) OFFERORS SHALL BE INFORMED ONLY OF THOSE TECHNICAL DEFICIENCIES THAT LEAD TO A CONCLUSION THAT THE MEANING OF THE PROPOSAL IS NOT CLEAR; THE OFFEROR HAS FAILED TO SUBSTANTIATE A PROPOSED TECHNICAL APPROACH; THE SOLICITATION NEEDS TO BE FURTHER CLARIFIED FOR EFFECTIVE COMPETITION.

NASA:

(3) WHERE THE MEANING OF THE PROPOSAL IS CLEAR AND THE PROPOSAL CONTAINS WEAKNESS INHERENT IN THE OFFEROR'S JUDGMENT, OR LACK OF COMPETITIVENESS AND INVENTIVENESS, THE WEAKNESS SHALL NOT BE POINTED OUT. OFFERORS SHOULD NOT BE INFORMED OF RELATIVE STRENGTHS AND WEAKNESSES OF THEIR PROPOSALS.

DOD:

(3) DISCUSSIONS OF TECHNICAL PROPOSALS SHALL NOT INVOLVE TECHNICAL DEFICIENCIES CLEARLY RELATING TO AN OFFEROR'S MANAGEMENT ABILITIES, ENGINEERING OR SCIENTIFIC JUDGMENT, OR LACK OF COMPETENCE OR INVENTIVENESS IN PREPARING THE PROPOSAL.

NASA:

(4) SEE PARAGRAPHS 2 & 3 ABOVE.

DOD:

(4) MEANINGFUL DISCUSSIONS CONDUCTED WITH OFFERORS REGARDING THEIR COST PROPOSALS SHALL INCLUDE COST REALISM; CORRELATION BETWEEN COSTS AND RELATED TECHNICAL ELEMENTS; DELIVERY SCHEDULES; TRADE-OFF CONSIDERATIONS RELATING TO PERFORMANCE, DESIGN TO COST, LIFE-CYCLE COST, AND LOGISTIC SUPPORT.

NASA:

(5) SEE PARAGRAPH 3 ABOVE.

DOD:

(5) DISCUSSIONS SHALL NOT DISCLOSE THE STRENGTHS AND WEAKNESSES OF COMPETING OFFERORS, OR DISCLOSE ANY INFORMATION FROM AN OFFEROR'S PROPOSAL WHICH WOULD ENABLE ANOTHER OFFEROR TO IMPROVE HIS PROPOSAL.

NASA:

(6) THE EVALUATION BOARD MAY DISCONTINUE EVALUATION OF A PROPOSAL CONTAINING MAJOR TECHNICAL OR BUSINESS DEFICIENCIES OR OMISSIONS OR OUT OF -LINE COSTS.

DOD:

(6) THE SELECTED OFFEROR'S PROPOSAL MUST SATISFY THE GOVERNMENT'S MINIMUM REQUIREMENTS.

NASA:

(7) THE EVALUATION BOARD IS TO PREPARE A BEST ESTIMATE OF PROBABLE COSTS OF PERFORMANCE FOR EACH PROPOSER, IF SELECTED, AND AN ESTIMATE OF SIGNIFICANT CHANGES IN EACH PROPOSAL THAT WOULD HAVE TO BE NEGOTIATED AFTER SELECTION WITH A DISCUSSION OF NEGOTIATION COST OBJECTIVES. THIS INFORMATION IS TO BE PRESENTED TO THE SOURCE SELECTION OFFICIAL. (FROM THE NASA SOURCE EVALUATION BOARD MANUAL.)

DOD:

(7) AN INDEPENDENT COST ESTIMATE SHALL BE DEVELOPED TO ASSIST IN DETERMINING THE MOST PROBABLE COSTS OF EACH COMPETITOR'S PROPOSAL. PARAMETRIC COST ESTIMATING TECHNIQUES OR SIMILAR APPROACHES SHOULD BE USED TO THE EXTENT PRACTICABLE TO DETERMINE THE REASONABLENESS OF THESE COSTS. THE SOURCE SELECTION AUTHORITY SHALL BASE HIS SELECTION ON WHAT IS THE MOST PROBABLE OUTCOME FOR EACH PROPOSAL.

NASA:

(8) FINAL CONTRACT NEGOTIATION WITH THE SELECTED OFFEROR SHOULD INCLUDE THE CORRECTION OF CORRECTIBLE WEAKNESSES AND THE NEGOTIATION OF ESTIMATED COSTS TO FAVORABLE LEVELS. (NASA SOURCE EVALUATION BOARD MANUAL.)

DOD:

(8) FINAL NEGOTIATIONS LEADING TO A DEFINITE CONTRACT WILL BE HELD ONLY WITH THE SELECTED OFFEROR.

NASA:

(9) NO COMPARABLE PROVISION.

DOD:

(9) NEGOTIATIONS AFTER SELECTION OF THE SUCCESSFUL OFFEROR SHALL NOT INVOLVE MATERIAL CHANGES IN THE GOVERNMENT'S REQUIREMENTS OR CONTRACTOR'S PROPOSAL WHICH AFFECT THE BASIS FOR SOURCE SELECTION.

THE COMPARISON REVEALS THE SIMILARITY OF THE PROCEDURES. IN BOTH PROCEDURES THERE ARE STATEMENTS OF THE NEED TO ALLOW COMPETITIVE-RANGE OFFERORS THE OPPORTUNITY FOR DISCUSSIONS. BOTH PROCEDURES STRESS THE NEED, HOWEVER, OF RESTRICTING DISCUSSION OF TECHNICAL PROPOSALS TO CLARIFYING OR SUBSTANTIATING THE PROPOSAL (OR CLARIFYING THE SOLICITATION MEANING WHEN NEEDED) AND SPECIFICALLY PROHIBIT DISCUSSION OF TECHNICAL WEAKNESSES (NASA'S TERM) OR DEFICIENCIES (DOD'S TERM) RELATING TO AN OFFEROR'S LACK OF COMPETENCE, DILIGENCE, INVENTIVENESS, OR LACK OF MANAGEMENT ABILITIES, ENGINEERING OR SCIENTIFIC JUDGMENT. BOTH PROCEDURES ALSO PROVIDE-- MORE CLEARLY IN NASA'S PROCEDURE, ALTHOUGH OBVIOUSLY IMPLIED ON DOD'S PROCEDURE-- FOR INDEPENDENT COST PROJECTIONS OF THE "MOST PROBABLE" COST OF EACH PROPOSAL INCLUDING THOSE COSTS MADE NECESSARY BY SIGNIFICANT CHANGES IN EACH PROPOSAL THAT WOULD HAVE TO BE NEGOTIATED WITH THE SUCCESSFUL OFFEROR AFTER SELECTION. THESE COST PROJECTIONS ARE ALSO STATED TO BE USED IN SELECTING THE SUCCESSFUL OFFEROR. BOTH PROCEDURES ALSO CALL FOR DISCUSSION OF "CORRECTABLE WEAKNESSES" (EXPLICIT IN THE NASA PROCEDURE; IMPLICIT IN THE DOD PROCEDURE) WITH THE SELECTED OFFEROR ONLY.

SEEMING DIFFERENCES BETWEEN THE PROCEDURES ARE: (1) DOD EXPRESSLY MANDATES "MEANINGFUL DISCUSSIONS" OF THE COST PROPOSAL; NASA DOES NOT; AND (2) DOD EXPRESSLY REQUIRES THAT THE NEGOTIATIONS WITH THE SUCCESSFUL OFFEROR AFTER SELECTION NOT INVOLVE MATERIAL CHANGES IN THE GOVERNMENT'S REQUIREMENTS OR CONTRACTOR'S PROPOSAL WHICH AFFECT THE BASIS FOR SOURCE SELECTION; NASA DOES NOT.

SINCE THE DOD PROCEDURES, IN THE MAIN, ARE COMPARABLE TO THE NASA PROCEDURES, OUR DECISIONS INVOLVING CONTESTED NASA PROCUREMENTS MAY BE OF AID IN RESOLVING THE ISSUE RAISED HERE. SEE, AIRESEARCH MANUFACTURING COMPANY OF AMERICA, 56 COMP.GEN. 989 (1977), 77-2 CPD 229.

ISSUE ANALYSIS

THE BULK OF THE SYLVANIA CRITICISM OF THE AIR FORCE'S USE OF THE DOD PROCEDURES GOES TO THE SUBSTANTIAL INCREASE IN THE COST OF THE WORK NEGOTIATED BY THE AIR FORCE WITH ITT AFTER SELECTION OF THE COMPANY. SYLVANIA BELIEVES THAT ONLY IMMATERIAL CHANGES MAY BE MADE IN THE SUCCESSFUL OFFEROR'S PROPOSAL IN FINAL POST-SELECTION NEGOTIATIONS WITH ANY OFFEROR AND THAT THE ADMISSION OF THE AIR FORCE THAT A SUBSTANTIAL INCREASE IN THE PRICE OF ITT'S CONTRACT WAS NEGOTIATED RENDERS INVALID THE AIR FORCE PROCEDURE.

IT IS FUNDAMENTAL IN THE AWARD OF COST-REIMBURSEMENT CONTRACTS OF THE TYPE AWARDED HERE THAT PROPOSED COSTS BE ANALYZED IN TERMS OF THEIR REALISM, SINCE, REGARDLESS OF THE ESTIMATE SUBMITTED, THE GOVERNMENT IS REQUIRED-- WITHIN CERTAIN LIMITS-- TO PAY THE CONTRACTOR'S ACTUAL, ALLOWABLE AND ALLOCABLE COSTS. SEE BELL AEROSPACE COMPANY; COMPUTER SCIENCES CORPORATION, 54 COMP.GEN. 352, 359 (1974), 74-2 CPD 248, AND CASES CITED THEREIN. THUS, GOVERNMENT-EVALUATED COSTS RATHER THAN CONTRACTOR-PROPOSED COSTS ARE IMPORTANT IN DETERMINING THE SUCCESSFUL CONTRACTOR FOR A COST-REIMBURSEMENT CONTRACT. THIS PRINCIPLE IS FOR APPLICATION WHETHER THE PROCUREMENT IS MADE UNDER NASA NEGOTIATION PROCEDURES OR OTHERWISE.

GENERALLY, THE TIME FOR EVALUATING COSTS IN A COST-REIMBURSEMENT CONTRACT IS DURING THE COURSE OF NEGOTIATIONS. AS WE SAID IN 50 COMP.GEN. 739 (1971), AT PAGE 745:

* * * THE TIME FOR EXPLORING THE COST ASPECTS OF A PROPOSAL-- THAT IS, ALL PROPOSALS WITHIN A COMPETITIVE RANGE-- IS DURING THE COURSE OF NEGOTIATIONS AND NOT AT SOME TIME AFTER THE RECEIPT OF BEST AND FINAL OFFERS. * * *

NEVERTHELESS, IN BELL AEROSPACE COMPANY, SUPRA, INVOLVING A NON NASA, NON -FOUR-STEP PROCUREMENT, WE APPROVED THE DEPARTMENT OF THE ARMY'S DECISION TO MAKE SIGNIFICANT COST ADJUSTMENTS TO SUBMITTED BEST AND FINAL PROPOSALS. WE REJECTED THE ARGUMENT THAT 10 U.S.C. 2304(G) REQUIRED THAT OFFERORS BE INFORMED OF THOSE ADJUSTMENTS AND BE PERMITTED-- THROUGH THE REOPENING OF NEGOTIATIONS-- TO SUBMIT ANOTHER ROUND OF PROPOSALS. AS WE STATED IN THE DECISION:

WHILE WE AGREE THAT NEGOTIATIONS ARE NECESSARY TO RESOLVE UNCERTAINTIES RELATING TO THE PURCHASE OR PRICE TO BE PAID, THERE IS A POINT AFTER WHICH COST NEGOTIATIONS MUST BE CONCLUDED AND COST ANALYSIS MUST BEGIN. U.S.C. 2304(G) (1970) HAS BEEN INTERPRETED SO AS TO REQUIRE CONDUCTING MEANINGFUL NEGOTIATIONS. HOWEVER, ONCE THIS REQUIREMENT HAS BEEN MET AND BEST AND FINAL OFFERS HAVE BEEN RECEIVED, IT IS, IN THE ABSENCE OF MORE, THEN INCUMBENT UPON THE AGENCY TO CONCLUSIVELY EVALUATE THESE BEST AND FINAL OFFERS. WE DO NOT FEEL THAT THE FAILURE TO DISCLOSE THE QUANTUM OF COST ADJUSTMENTS MADE IN COST ANALYSIS OF THE BEST AND FINAL OFFERS, WITH AN OPPORTUNITY FOR THE OFFERORS TO POINT OUT ERRORS, CONSTITUTES A FAILURE TO HAVE MEANINGFUL NEGOTIATIONS.

IN THIS CASE, THE COST REALISM STUDY WAS PERFORMED AFTER SUBMISSION OF BEST AND FINAL OFFERS. WE RECOGNIZE THAT SUCH A STUDY SHOULD BE MADE IN THIS KIND OF SITUATION. ON THE OTHER HAND, THE NEGOTIATION PROCESS CANNOT BE INDEFINITELY EXTENDED FOR THE PURPOSE OF PROVIDING THE OFFEROR AN OPPORTUNITY TO TAKE ISSUE WITH THE COST REALISM STUDY OR ANY OTHER EVALUATION DETERMINATION. IF THE OFFEROR FEELS THAT ANY ASPECT OF THE EVALUATION WAS IMPROPER, HE MAY PROTEST AND THE MATTER WILL BE CONSIDERED.

ALTHOUGH IN THE BELL AEROSPACE COMPANY CASE COST PROPOSALS WERE ADJUSTED FOR PURPOSE OF AWARD EVALUATION, THERE IS NO INDICATION-- CONTRARY TO THE CASE HERE-- THAT THE DEPARTMENT ACTUALLY AWARDED A CONTRACT AT THE ADJUSTED PRICE. WE DID NOTE THAT THE DEPARTMENT'S AWARD WAS "BASED ON * * * KNOWLEDGE" OF THE ADJUSTED COST, HOWEVER. NEVERTHELESS, WE DID APPROVE THE PROCESS OF GOVERNMENT ADJUSTMENT OF COST PROPOSALS AFTER THE CLOSE OF FORMAL NEGOTIATIONS EVEN WHEN THE NON NASA, NON-FOUR-STEP NEGOTIATION PROCEDURES WHICH GOVERNED THE PROCUREMENT DID NOT EXPRESSLY PROVIDE FOR THIS ADJUSTMENT PROCESS.

WE SEE NO SIGNIFICANT DIFFERENCE BETWEEN A PROCESS WHICH ALLOWS COST ADJUSTMENT OF PROPOSED COSTS AFTER THE CLOSE OF DISCUSSIONS FOR PURPOSES OF DETERMINING THE SUCCESSFUL CONTRACTOR-- EVEN THOUGH NO FORMAL ADJUSTMENT OF CONTRACT PRICE IS ULTIMATELY MADE-- AND AN UNDISCLOSED COST ADJUSTMENT PROCESS USED IN AWARD SELECTION WHICH ULTIMATELY RESULTS IN A CHANGED CONTRACT PRICE MORE IN LINE WITH THE GOVERNMENT-EVALUATED PRICE AS WAS DONE HERE.

IN BOTH CASES, THE UNDISCLOSED COST ADJUSTMENTS ARE USED TO DETERMINE-- ALONG WITH OTHER FACTORS-- THE SUCCESSFUL OFFEROR. FROM THE STANDPOINT OF EQUAL COMPETITION AMONG CONTENDING OFFERORS SEEKING AWARD, THE NET RESULT IS THE SAME, NAMELY, AWARD SELECTION ON THE BASIS OF UNDISCLOSED COST ADJUSTMENTS. MOREOVER, IT IS CLEAR THAT OUR OFFICE HAS IMPLICITLY SANCTIONED THE NASA PROCEDURE OF ALLOWING UNDISCLOSED COST ADJUSTMENTS TO BE USED NOT ONLY IN DETERMINING THE SUCCESSFUL OFFEROR BUT AS A MEANS OF ALTERING THE SELECTED OFFEROR'S PROPOSED COSTS AFTER SELECTION BUT PRIOR TO AWARD. SEE, FOR EXAMPLE, LOCKHEED PROPULSION COMPANY, SUPRA, AT PAGE 1032. TO THE EXTENT THAT DOD'S FOUR STEP PROCEDURE SIMILARLY TREATS COST ADJUSTMENTS, IT IS NOT SUBJECT TO QUESTION. IT IS IMPLICIT IN SYLVANIA'S ARGUMENT THAT THE DOD PROCEDURE IS DIFFERENT FROM THE NASA PROCEDURE BECAUSE THE DOD PROCEDURE SPECIFICALLY DIRECTS THE CONDUCT OF "MEANINGFUL DISCUSSIONS" REGARDING "COST REALISM" AND "CORRELATION BETWEEN COSTS AND RELATED TECHNICAL ELEMENTS" WHEREAS THE NASA PROCEDURE DOES NOT CONTAIN A SIMILAR, EXPRESS INJUNCTION.

ALTHOUGH THIS EXPRESS DIRECTION IS FOUND IN THE DOD PROCEDURE, THE DOD PROCEDURE ALSO EXPRESSLY REQUIRES NEGOTIATIONS AFTER SELECTION OF THE SUCCESSFUL OFFEROR WITHOUT IN ANY WAY PROHIBITING CHANGES IN THE OFFEROR'S PROPOSED COSTS TO BRING THEM MORE IN LINE WITH THE GOVERNMENT'S ESTIMATE. THUS, THE TWO PROCEDURES, ALTHOUGH NOT COMPLETELY IDENTICAL ON A WORD-BY- WORD COMPARISON, BOTH CONTEMPLATE COST AND TECHNICAL ADJUSTMENTS IN THE SELECTED PROPOSAL PRIOR TO AWARD.

FURTHER, WE DO NOT AGREE THAT SIGNIFICANT PERCENTAGE ADJUSTMENTS MAY NOT BE MADE IN THE SELECTED OFFEROR'S COST PROPOSAL. WE HAVE ALREADY APPROVED THE CONCEPT OF UNDISCLOSED COST ADJUSTMENTS BOTH IN THE BELL AEROSPACE AND LOCKHEED PROPULSION COMPANY CASES. THIS APPROVAL IS BASED, HOWEVER, ON ASSUMPTIONS THAT ADEQUATE COST AND TECHNICAL DISCUSSIONS HAVE BEEN PREVIOUSLY CONDUCTED AMONG COMPETITIVE-RANGE OFFERORS; THAT ALL OFFERORS HAVE BEEN PERMITTED TO SUBMIT BEST AND FINAL OFFERS AS A RESULT OF THOSE DISCUSSIONS; THAT THE GOVERNMENT PROJECTIONS OF ULTIMATE COST ARE SOUND; AND THAT THE ULTIMATE CHANGES IN THE SUCCESSFUL OFFEROR'S PROPOSAL DO NOT AFFECT THE UNDERLYING ASSUMPTIONS WHICH PROMPTED THE SELECTION.

SYLVANIA, IN EFFECT, QUESTIONS WHETHER ITT'S PROPOSAL SHOULD HAVE BEEN CONSIDERED IN THE COMPETITIVE RANGE BECAUSE OF THE ADMITTED WEAKNESSES IN THE TECHNICAL PROPOSAL, THE CORRECTION OF WHICH, AT LEAST IN PART, RESULTED IN THE SIGNIFICANT INCREASE (OVER ITT'S-PROPOSED COST) IN THE GOVERNMENT-EVALUATED COST USED FOR AWARD SELECTION AND THE ACTUAL INCREASE IN CONTRACT PRICE NEGOTIATED BY THE AIR FORCE AND ITT IN POST-SELECTION DISCUSSIONS.

SYLVANIA MAKES THIS ARGUMENT BY NOTING THE APPARENT INCONSISTENCY BETWEEN THE AIR FORCE POSITION THAT ITT'S PROPOSAL WAS PROPERLY FOR ACCEPTANCE AND POST-SELECTION DISCUSSIONS (EVEN THOUGH IT CONTAINED SIGNIFICANT DEFICIENCIES-- THE PHRASE USED BY THE BOARD AND THE COUNCIL) AND SOME OF THE "SPECIAL TEST" ASPR REQUIREMENTS. THOSE REQUIREMENTS PROVIDE THAT A SELECTED OFFEROR'S PROPOSAL MUST SATISFY THE GOVERNMENT'S MINIMUM REQUIREMENTS AND THAT A DEFICIENCY IS THAT PART OF THE PROPOSAL WHICH DOES NOT MEET THE GOVERNMENT'S REQUIREMENTS.

WE FIND NO REAL INCONSISTENCY IN THE AIR FORCE'S POSITION. IT SEEMS TO US THAT THE PROVISION THAT THE SELECTED PROPOSAL MUST MEET THE GOVERNMENT'S "MINIMUM REQUIREMENTS" IS NOTHING MORE THAN A REQUIREMENT THAT-- ASIDE FROM BEING THE MOST ADVANTAGEOUS PROPOSAL FOR ACCEPTANCE UNDER THE STATED EVALUATION CRITERIA-- THE PROPOSAL IS TO SATISFY THE GOVERNMENT'S CORE REQUIREMENTS FOR THE WORK TO BE DONE TO THE EXTENT THAT THE PROPOSAL IS GENUINELY CONSIDERED TO BE IN THE COMPETITIVE RANGE FOR THE PROCUREMENT. THEREFORE, WE DO NOT VIEW THE "MINIMUM REQUIREMENTS" PROVISION AS CALLING FOR A PROPOSAL MEETING ALL REQUIREMENTS BEFORE SELECTION, AS SYLVANIA URGES. THIS VIEW IS CONSISTENT WITH THE ORDINARY UNDERSTANDING OF WHAT CONSTITUTES A COMPETITIVE-RANGE PROPOSAL. AS WE STATED IN 52 COMP.GEN. 382, 385 (1972):

WE HAVE HELD THAT A PROPOSAL MUST BE CONSIDERED TO BE WITHIN THE COMPETITIVE RANGE SO AS TO REQUIRE NEGOTIATIONS UNLESS IT IS TO TECHNICALLY INFERIOR THAT MEANINGFUL NEGOTIATIONS ARE PRECLUDED.

THUS, THE MERE FACT THAT A PROPOSAL MAY BE TECHNICALLY INFERIOR IN ONE OR MORE RESPECTS-- INCLUDING "INFERIORITY" RELATING TO NONCOMPLIANCE WITH SOME RFP REQUIREMENTS-- DOES NOT NECESSARILY ELIMINATE A PROPOSAL FROM BEING CONSIDERED WITHIN THE COMPETITIVE RANGE.

IN ANY EVENT, AS NOTED ABOVE, THE EVALUATION BOARD SPECIFICALLY FOUND THAT ITT'S PROPOSAL MET OR EXCEEDED ALL RFP REQUIREMENTS ALTHOUGH THE BOARD FOUND THE PROPOSAL TO CONTAIN "SIGNIFICANT WEAKNESSES" IN CERTAIN AREAS. FURTHER, THE BOARD'S FINDING WAS CONFIRMED BY THE COUNCIL'S OBSERVATION THAT NEGOTIATIONS WITH EITHER SYLVANIA OR ITT WOULD BE SUCCESSFUL TO THE END THAT A CONTRACT WOULD BE AGREED TO THAT WOULD MEET THE AIR FORCE'S NEEDS. THUS, WE FIND RATIONAL SUPPORT, BASED ON OUR REVIEW OF THE ENTIRE RECORD, THAT ITT'S PROPOSAL WAS A COMPETITIVE-RANGE PROPOSAL PROPERLY FOR CONSIDERATION FOR AWARD AS WELL AS POST-SELECTION DISCUSSIONS. FURTHER, BASED ON OUR REVIEW OF THE RECORD, WE CANNOT CONCLUDE THAT THE WEAKNESSES-- BOTH AS TO COSTS AND TECHNICAL MATTERS-- IN ITT'S PROPOSAL WERE SUCH THAT DISCUSSIONS-- PRIOR TO SELECTION-- COULD HAVE BEEN HELD WITH THE COMPANY-- WITHOUT VIOLATING THE EXPRESS RESTRICTIONS OF THE DOD PROCEDURE.

AS TO WHETHER SUFFICIENT COST AND TECHNICAL DISCUSSIONS WERE HELD WITH THE OFFERORS, WE NOTE THAT THE SYLVANIA CLAIMS OF LESS-THAN SUFFICIENT NEGOTIATIONS RELATE, ALMOST EXCLUSIVELY, TO THE SUPPOSED LACK OF DISCUSSIONS NOT WITH ITSELF BUT WITH ITT. WE HAVE REVIEWED THE LENGTHY RECORD OF THE DISCUSSIONS HELD WITH ITT. IN OUR VIEW, THERE IS NOTHING IN THE RECORD TO SUGGEST THAT THE DISCUSSIONS WERE OTHER THAN REASONABLE ATTEMPTS TO COMPLY BOTH WITH THE LITERAL REQUIREMENTS OF THE STATUTE AND THE DOD PROCEDURES. FURTHER, IT IS OUR VIEW THAT THE DISCUSSIONS HELD WERE IN FACT REASONABLY COMPLIANT WITH THE GOVERNING STATUTE AND PROCEDURES, RECOGNIZING, UNDER THE ABOVE PRECEDENT, THE BROAD AUTHORITY GRANTED PROCURING AGENCIES TO DECIDE THE NATURE AND EXTENT OF THE DISCUSSIONS NECESSARY TO COMPLY WITH THE STATUTE. CONSEQUENTLY, AND WILL FULL KNOWLEDGE OF THE SIGNIFICANT COST INCREASE NEGOTIATED WITH ITT AFTER SELECTION, WE REJECT SYLVANIA'S ARGUMENT THAT THE AIR FORCE IMPROPERLY DEFERRED TO POST-SELECTION DISCUSSIONS MATTERS THAT SHOULD HAVE BEEN DISCUSSED PRIOR TO SELECTION. WE ALSO FIND-- CONTRARY TO SYLVANIA'S ASSERTION-- THAT PROSCRIBED "LEVELING" DID NOT TAKE PLACE DURING THE POST- SELECTION DISCUSSIONS. MOREOVER, IT IS OUR VIEW THAT SYLVANIA IS ALLEGING LACK OF DISCUSSIONS-- INSOFAR AS ITS OWN PROPOSAL IS CONCERNED-- LARGELY IN THE ABSTRACT BY MERELY CITING THE "BENIGN" CHARACTER OF THE QUESTIONS ASKED OF IT DURING DISCUSSIONS. TO THIS EXTENT, THEREFORE, WE CONSIDER THAT SYLVANIA'S PROTEST IS AKIN TO THE PROTEST IN SPERRY RAND CORPORATION, SUPRA, WHERE, IN DENYING THE PROTEST, WE ALSO OBSERVED THAT THE PROTESTER ALLEGED LACK OF MEANINGFUL DISCUSSIONS "LARGELY IN THE ABSTRACT." CONSEQUENTLY, WE CANNOT CONCLUDE THAT THE AIR FORCE FAILED TO COMPLY WITH THE REQUIREMENT OF 10 U.S.C. 2304(G) IN THIS PROCUREMENT.

# HC758

UNDER THE BROAD UMBRELLA OF ITS ATTACK ON THE WAY THE AIR FORCE IMPLEMENTED THE PROCEDURES, SYLVANIA ALSO QUESTIONS THE SOUNDNESS OF THE AIR FORCE'S COST PROJECTIONS CONCERNING THE LIKELY ULTIMATE COST OF ITS PROPOSAL COMPARED WITH THE PROJECTED COSTS OF ITT'S PROPOSAL. ISSUE IS ALSO TAKEN BY SYLVANIA WITH THE AIR FORCE'S JUDGMENT THAT ITS PROPOSAL WAS PROPERLY RANKED LOWER THAN ITT'S PROPOSAL.

WE HAVE SPECIFICALLY APPROVED THE USE OF THE PARAMETRIC COST EVALUATION TECHNIQUE ADOPTED BY THE AIR FORCE HERE IN EVALUATING PROPOSALS. RAYTHEON COMPANY, 54 COMP.GEN. 169 (1974), 74-2 CPD 137. GIVEN OUR ACCEPTANCE OF THIS TECHNIQUE, OUR APPROVAL OF THE CONCEPT OF UNDISCLOSED COST ADJUSTMENTS TO PROPOSALS FOR USE IN EVALUATION AND POST-SELECTION DISCUSSIONS, AND OUR REVIEW OF THE RESULTS OF THE COST ADJUSTMENTS, WE CANNOT CONCLUDE THAT THE PROJECTED DIFFERENCES IN COSTS BETWEEN ITT AND SYLVANIA LACK A REASONABLE FOUNDATION, NOTWITHSTANDING SYLVANIA'S ALLEGATION TO THE CONTRARY. MOREOVER, AS NOTED ABOVE, THE AIR FORCE'S PRE -SELECTION PROJECTION OF THE COSTS NEEDED TO CORRECT ITT'S DEFICIENCIES WAS CONFIRMED BY THE COST INCREASE ACTUALLY NEGOTIATED WITH ITT DURING POST-SELECTION DISCUSSIONS. ALSO, BASED ON OUR REVIEW OF THE RECORD, WE DO NOT AGREE THAT THE EVALUATED TECHNICAL DIFFERENCES BETWEEN THE PROPOSALS LACK A RATIONAL FOUNDATION. ON THIS POINT WE MUST AGREE WITH THE AIR FORCE'S VIEW THAT SYLVANIA HAS NOT BEEN INFORMED OF ALL THE TECHNICAL DIFFERENCES BETWEEN THE PROPOSALS AND IS THEREFORE NOT IN A POSITION TO REALISTICALLY QUESTION THE EVALUATED DIFFERENCES.

PROTEST DENIED.

/1/ ALTHOUGH SYLVANIA INSISTS THE COMPILER IS "NON-EXISTENT," THE AIR FORCE POSITION IS THAT THE COMPILER IS ACTUALLY A MODIFICATION OF A PREEXISTING UNIT. IN THE ABSENCE OF PROBATIVE EVIDENCE SUPPORTING SYLVANIA'S CONTENTION, A SUFFICIENT BASIS DOES NOT EXIST FOR SUSTAINING ITS POSITION. RELIABLE MAINTENANCE SERVICE, INC.,-- REQUEST FOR CONSIDERATION, B-185103, MAY 24, 1976, 76-1 CPD 337.

/2/ ALTHOUGH SYLVANIA APPARENTLY UNDERSTOOD THESE PROVISIONS AS ABSOLUTELY DENOTING VARIOUS EQUIPMENTS AND PROGRAMMING, THE PROVISIONS DO NOT IN ANY WAY MENTION SPECIFIC EQUIPMENTS AND PROGRAMS. MOREOVER, THE FACT THAT SYLVANIA AND ITT-- BOTH OF WHOM ARE OBVIOUSLY KNOWLEDGEABLE AND EXPERIENCED ELECTRONICS EQUIPMENT MANUFACTURERS AND SUPPLIERS-- ARRIVED AT DIFFERENT CONCLUSIONS ABOUT THE MEANING OF THESE PROVISIONS IS A FURTHER INDICATION THAT THE PROVISIONS DO NOT NECESSARILY DENOTE AN OBJECTIVE LIST OF EQUIPMENT AND PROGRAMS.