MIL-HDBK-61A: CM Guidance for Integration of High Intensity COTS Products

C.2 Principles and Concepts

Among the goals of DoD acquisition reform is the broadening of the industrial base by using performance based acquisition, and the use of commercial-off-the-shelf (COTS) products wherever possible. In addition to the avoidance of new development cost, use of COTS products and acceptance of commercial methods is expected to result in cost savings to the Government. COTS equipment and software are normally designed and manufactured to "best commercial practices" and because they are competition and marketplace driven are often state-of-the-art designs. It is well known that cutting edge technology in many areas such as software, electronics, and especially information technology has an increasing shorter half-life. Using COTS thus enables the DoD to apply or "refresh" the technology in its weapon systems. Buying to Performance specifications, as delineated in Section 2, enables newer technology to be inserted without modifying the basic acquisition documents. This is extremely important when dealing in a commercial marketplace where contractor support of products that they have made obsolete by introducing advanced technology is short lived.

Thus the appropriate use of COTS can accomplish important goals while decreasing both schedule and cost risk. However to experience the benefits, the Government system integrator needs an awareness of the differences between military and commercial acquisition and the potential pitfalls that must be avoided.

Unlike the military acquisition environment, in which the imposition of military standards (e.g., MIL-STD-973) created a more or less level CM playing field, the Integrator (Government or integrating contractor) cannot rely upon commercial vendors having CM maturity. Commercial vendors do not have a CM standard levied upon them and industry standards are voluntary compliance documents. EIA Standard 649, which articulates CM principles and best practices, is relatively new. It provides a benchmark for COTS providers to use, but each enterprise will employ those practices that it perceives to be in its own financial interests. Some have well-defined configuration management processes, while others have what may best be described as ad hoc processes.

The integrator who is concerned primarily with performance and interfaces among a set of COTS products must be able to accommodate the inconsistency in COTS provider CM practice. While less data is required in a nondevelopmental situation such as the acquisition of COTS, there are more complexities introduced into the integrator's process regarding such issues as the identification, operation and servicing, replacement and discontinuance of COTS items and obsolescence of their spare parts.

Configuration management can become a COTS source selection discriminator if a vendor's practices, or lack thereof, are perceived to be an impediment to effective logistic support. CM issues need to be addressed in the vendor and product selection processes. Market analysis surveys in preparation for acquiring COTS items should include CM related questions to give the integrator's CM organization insight into the vendors configuration management practices and an understanding of such vendor practices as serial and part number marking schemes. [Detail: Table C-1. Activity Guide: COTS Vendor CM Questionnaire]

COTS issues related to configuration identification include the choice of acquisition documentation, the baseline for configuration control, and how COTS products are identified and marked.

The use of COTS matches the acquisition reform environment when performance documentation used by the integrator to specify and manage form, fit, function, and interface requirements (FI). Table 3-3 in section 3 of this handbook defines and provides the order of precedence for specification documents to be used for acquisition. Those documents which are performance documents are clearly indicated.

The choice of the most appropriate documentation to use for acquisition of a COTS item varies according to the product end use, supportability requirements, system complexity and many other factors. The specific documentation to use for various types of COTS products can only be determined by understanding the system complexity and the criticality of the COTS product to the program. One method of making this determination is by constructing a decision matrix [Detail: Activity Guide Table C-2.]

Typically the integrator prepares a Commercial Item Descriptions (CIDs) which defines the acquisition performance requirement by FI (Form, Fit, Function, and Interface) and copies vendor data sheet information into a Vendor Item Descriptions (VID) or Source Control Documents (SCD).

Third party vendors generally develop COTS products. Documentation of COTS products is unregulated; therefore, its availability, consistency, and information content may be inconsistent and unpredictable. Data rights are generally not available for use in product design and modification. Additional data required for COTS should be limited to that which is normally provided to commercial buyers. Such data typically includes operating instructions, basic maintenance instructions and parts replacement, which if performed by the user will not invalidate the product warranty. Any additional data can be expensive and is generally unnecessary. Bringing commercial design documentation up to government standard levels, as was often done in the past is a cost that must be avoided. Much of such data can be quickly out-of-date or obsolete. So long as the item meets the verifiable performance requirements, and is supportable in the field using an inventory of spare parts designated by the COTS supplier, the design details should be left to the supplier.

In a performance based acquisition, the Government or its integrating contractor must specify and control to an item's performance rather than to design details. Therefore the only documentation that should be baselined by the integrator should be the performance specification or equivalent document used for acquisition. The COTS contractor may establish design and product baselines for his own convenience. Controlling at the Performance and interface (or interchangeability) level allows the COTS contractor to make changes necessary for technical refreshment and to avoid obsolescence. The contractor may make strategic market driven improvements in his product at the component level, refreshing the technology by substituting improved or later state-of-the-art components without impact to the end user's requirements.

There is little consistency in item identification practices among COTS producers, and often little consistency between two products provided by the same supplier. Vendor supplied part numbers may be of little value beyond the ordering stage; part numbers may be obsolete even before the product. Many vendors do not consistently mark their parts, and some do not mark the parts at all.

This, obviously, makes receiving inspection much more difficult. Because of the dynamic nature of the products, multi-site, multi-unit, and multi-year deliveries are more difficult because each individual installation may contain different revision levels of multiple products, and serialization methods often violate the basic principles of nonduplication [Details: Section 5, 3.6.3, and Activity Guide Table 5-11.] Software licenses, upgrade tapes, and configuration files are difficult to manage because of this lack of consistency between vendors. If sparing is to be done by other than the COTS supplier, it can be a complex issue.

Nonetheless, the integrator can effectively deal with these problems if the enterprise has an effective system that follows basic CM business rules [See C.2.5, Configuration Status Accounting]. The integrator must be allowed the latitude to compensate for inconsistencies and poor practices by the COTS suppliers. Such remedies include auxiliary identifiers and decals applied at the time of incoming inspection for inventory control, serialization, configuration control and accounting.

When managing COTS items, performance specifications (performance baseline) are the key point of control. In fact, they are the only legitimate basis for configuration control that the integrator can use. As pointed out in C.2.3.2, the integrator does not have rights to the design data of a COTS supplier, and cannot direct changes to it. The integrator is an application activity [Ref: 6.1.1.1] with respect to the suppliers product and its documentation, i.e., the integrator may request the supplier to make a change to its product, but does not have the right to direct that change if the supplier is not in agreement. Selection of a COTS item is based in part on life cycle cost considerations; the integrator should be cautious about obviating the cost benefit by attempting to over-control the supplier. The integrator also can choose not to use the suppliers product.

The supplier on the other hand has complete configuration control over the COTS product. The supplier may offer changes (improvements, added features) that are optional at extra cost at any time. On the other hand the supplier may make configuration changes to the product for competitive reasons without any knowledge or compliance by the integrator. COTS suppliers are also subject to unannounced changes by their own suppliers, which may in turn result in changes to the COTS product design. These supplier initiated changes, often improve the product, but are not always made with appropriate modification of technical data or in concert with programmed change activity of the ultimate end user.

Considering the nature of the respective end items, the suppliers standard practices and the competitive environment, requirements for configuration control will vary somewhat from supplier to supplier.

Wherever possible, integrator to COTS supplier configuration control requirements should include the following as a minimum:

Advance notification of design changes that may impact the performance baseline
Advance notification of pending obsolescence
Advance notification of changes to field repairable/replaceable assemblies and spare parts

The integrator can be the recipient of short-term notice of component and sub-component part obsolescence/changes, and is forced into a reactive mode. Without direct control of the product evolution, the integrator must compensate by being aware of pending changes as early as possible and performing change impact analyses that assess alternate solutions to determine what action is in the best interests of the Government.

The impact to the integrator and the Government is minimized by anticipating the likely level of change activity that will occur, including redesign efforts to the prime system to compensate for unplanned COTS iterations. The integrator and the Government must take these "marketplace" considerations into account when planning for and funding COTS projects. Budget reserves for these types of contingencies should be maintained.

The Government must recognize that the "long-lead" change decision and funding process typical of military weapon system programs in the past can seriously erode the savings anticipated from use of COTS. One benefit of controlling the integrator via a performance rather than a detail specification, is the ability for the integrator to react swiftly to implement the compensating changes that do not impact the performance of the prime item.

Obviously, given the many variables discussed in the previous paragraphs, the integrator's configuration status accounting process is the place where the reconciliation between inconsistent COTS supplier CM practices and the clear accountability that is due the Government must take place. Here too there are some pitfalls to be avoided.

Many integrators' current Configuration Status Accounting Systems (CSA) will require modification or enhancement to accommodate the management of COTS products. Most current CSA systems are designed around military standard guidelines. As pointed out in C.2.3, Commercial vendors do not follow the military identification rules in identifying their products. Typically, COTS product and document identifiers often exceed the character size; allowable characters and other format restrictions rigidly enforced via edit checks in CSA systems created for earlier military contracts. Similarly revision identifiers and serial numbers can contain special characters, and exceed the field lengths for many of these legacy CSA systems.

Fortunately, today's information technology provides the means to circumvent most if not all of these inconsistencies. Through the use of relational and object oriented data base tools, bridges can be built between the "legacy" and the reality. An ancillary COTS part identifier can be assigned to the COTS part to establish an alias for the item that can be accommodated within the legacy databases. The integrator-assigned identifier (alias) for the COTS part also can be used to achieve supply support stability by building an interchangeable alternate part data base as the COTS item changes as a result of product/vendor discontinuance and upward compatible vendor changes.

Special consideration should be given to the types of product baselines that need to be established and maintained on COTS software integration projects.

COTS contractor needs to establish and maintain a software product baseline that provides integrity for the contractual developmental effort
A unique baseline for each installation should be established to account for the hardware and software environment differences created by the use of multiple revision levels of COTS products at each installation location.

Contractors need to focus on tracking the versions of COTS tools as they apply to user-developed applications. To manage the relationship between COTS tools and developed applications:

Maintain a meta-file in a software version-control tool identifying all pertinent COTS utilities, operating systems, and compiler version information
Store the files making up the applicable COTS tool, utility or compiler as part of the developmental product within the contractor software version-control system or in a related PDM system.

For correct application of this information, see NOTE on Contents page