The SAE G19A Test Laboratory Standards Development Committee is working on a new standard, AS6171, “Test Methods Standard; Counterfeit Electronic Parts”, intended to support AS5553. My understanding is that AS6171 will eventually replace the product assurance section that currently appears in AS5553A. Thus far, SAE G19A subcommittee has put forth ballots for four test method documents. All of these proposed test methods map test and inspections to one or more of five risk levels and refer to AS6171 for definitions and implementation of these risk levels. AS6171 is in the form of a working draft and has not yet been submitted for balloting.
Within the working drafts of AS6171 I do not find information that addresses a number of questions surrounding the risk levels and how to apply them. The details I find within working drafts of AS6171 and the proposed test methods I have seen thus far indicate to me that significant thought has gone into the depiction of risk levels and their use in risk management activity. Significant, fundamental information, however, does not appear which I believe is central to applying the risk levels to practical risk management applications.
I find insufficient information within these documents to …
- describe how detection (i.e. inspections and tests specified within AS6171) fits within a comprehensive risk management approach,
- define the five risk levels,
- describe guidance on how to apply the levels in quantitative risk management approach,
- identify the probability of various forms of counterfeit part that may escape each of these levels,
- describe how specific tests and inspections were assigned to each risk level,
- map defects and anomalies associated with various forms of counterfeits to specific tests and inspections described in all of the test method documents (currently in the ballot process as well as those being developed).
Without this important information, the risk level convention suggested by AS6171 has limited utility for other than subjective risk assessments.
As it reads now, the risk management approach described in AS6171 assumes that one has already decided to use parts from risker suppliers and that risk management is solely a function of testing and inspecting parts to avoid counterfeits. Given that one avoids the counterfeit parts threat by purchasing from original manufacturers and their authorized suppliers, it makes perfect sense that test and inspection protocols defined in AS6171 are applied to mitigate the risk of receiving counterfeits when riskier suppliers cannot be avoided. The risk management approach implied by AS6171, however, is but one part of a comprehensive risk management approach to avoid counterfeits.
To me, a comprehensive risk management approach includes considering alternatives that would preempt the need to buy parts from riskier suppliers in the first place (hence the “Risk Assessment” block within Figure B-3 of AS5553). In a recent essay, I describe the sort of risk assessment approach that is useful to prime contractors, their customers, and their electronic equipment subcontractors. In addition to test and inspection, the essay also describes other factors necessary to devise a quantitative risk assessment approach – likelihood of counterfeiting across product types, types of counterfeits, “Trusted Supplier” criteria. Members of G19 and other subject matter experts may identify other factors that are necessary to devise a quantitative risk management approach.
Whether the risk level convention within AS6171 aligns with that of DoD’s “Risk Management Guide” or applies its own convention, AS6171 and its test method documents should include more quantitative information and guidance on how best to apply them within specific programmatic constraints and within the context of a comprehensive risk management approach used by DoD and its contractors.
In short, AS6171 should provide a quantitative basis for answering the following question …
“What is the probability of a counterfeit part quality escape associated with each of these risk levels and what are the defects, damage and other anomalies that may not be detected by the test and inspection protocols associated with each of these risk levels?”
If the present and foreseeable state of the art does not allow technical subject matter experts to answer this question backed by defensible estimates at minimum, then perhaps G19A should reconsider this risk level approach for the time being.
Though G19A may not be in a position to quantify the probability of a counterfeit part quality escape associated with each of the proposed risk levels, I anticipate that G19A will be able to answer the following question in the near future …
“What are the defects, damage and other anomalies that are most likely to be detected by each of the test and inspection protocols (or combinations thereof) described within AS6171 and what are the forms of counterfeits that exhibit these defects, damage and anomalies?”
Many of us have seen evolving representations that approach this description put forth by Integra Technologies; The Institute for Defense Analysis; and the Center for Hardware Assurance, Security, and Engineering (CHASE). A mapping of test/inspections to defects, damage anomalies and to forms of counterfeits will be a very important tool that could be used effectively now, and will establish a foundation for developing a more quantitative risk management approach.