Batch measurements

Question
I have a concentration measurement for an ingredient in a bulk final product. This measurement is close to one of the spec limits. Given each measurement has an inherent error (or range), how do I calculate the probability this measurement is actually within the specs? Or, in other words, how do I calculate the probability that I’m right or wrong in accepting or discarding the batch based on that measurement?

Response
The total variability is the combination of process variation and measurement variability. Measurement variability is further defined as the sum of reproducibility and repeatability. Unfortunately, in many situations the measurement and process variation cannot be parsed from the total, especially when only a single value is collected for a measurement. Assuming a measurement study (MSA) was conducted, the repeatability from this study can be used to estimate the confidence interval (or tolerance interval) around a measurement. This would require knowing the degrees of freedom from the estimate of the repeatability from the measurement study to effective estimate the probability.

The reality is that you never know the probability of being right or wrong, as the assumption is that the measurement variability is acceptable, and that any reading close to the specification is not different than any other reading. If the measurement system is capable, we must use the value obtained from the measurement as the true value, without any probability of making a wrong decision.

I will use the following example to clarify the point.

Assume we have a repeatability from a measurement study of 0.25 (standard deviation of sigma error is 0.25) and one with a S=0.05. Assuming an upper specification of 79.25, the probability using the standard normal of a mean exceeding the specification when the observed value is below the specification is seen below.

Observed Value S=0.25 S=.05
79.00 15.9% 0.0%
79.10 27.4% 0.1%
79.20 42.1% 15.9%
79.25 50.0% 50.0%

Assume we have a repeatability from a measurement study of 0.25 (standard deviation of sigma error is 0.25) and one with a S=0.05. Assuming an upper specification of 79.25, the probability using the standard normal of a mean being less than the specification when the observed value is above the specification is seen below.

Observed Value S=0.25 S=.05
79.30 42.1% 15.9%
79.40 27.4% 0.1%
79.50 15.9% 0.0%

Based on the criticality of the measurement, the measurement system should be improved to minimize the probability of making either a Type 1 or Type II error.

Steven Walfish
Secretary, U.S. TAG to ISO/TC 69
ASQ CQE
Principal Statistician, BD

http://statisticaloutsourcingservices.com

Posted in Measuring | Tagged , | Leave a comment

Nonconformance versus CAPA Requests

Question:

I need advice on the use of Nonconformance versus Corrective Action/Preventive Action (CAPA) Requests. I understand and have tried to communicate the low risk and high risk definitions to staff with some understanding. In reporting nonconformance’s some evolve into a root cause analysis which is a positive direction but thought to be a requirement of a Corrective/ Preventative Action. Nonconformance’s are logged on a report and reviewed periodically. CAPA Requests are more elaborate; logged and reported on a metrics with continuous review.

Response:

My answer may seem lengthy but I feel defining things is important. First, here is part of a memo I put together for one company.

ISO terminology and definitions – Corrective action/Preventive action

Some people experience confusion over the differences between corrective and preventive action.

We know that corrective actions are taken to remove the causes of existing nonconformities.

If the nonconformity is detected during production, immediate corrective action is taken to eliminate the problem. In other words, we fix what went wrong. We take preventive action to ensure the same problem does not happen again. However, this is still corrective action because it is based on solving a problem that has already happened.

We might use documents or electronic forms to report/record such actions. Here, caution is advised. For example, if a machinist turns a part undersize, immediate corrective action is taken to fix the mistake and further action is taken so it doesn’t reoccur on subsequent parts. If the original “bad” part was scrap and we record that as a non-conformance in our documentation, with the corrective action noted, we might then close that record. We might then request a follow up with preventive action. That would be a mistake.

Note: Not every problem or non-conformance requires a corrective action. This is determined on a case by case basis, usually by a manager. Each case is different.

Example: A welder accidentally causes weld spatter to fly into a tapped hole. The welder cleans out the B-B’s, re-taps the hole and moves on. Generating a non-conformance form should not be necessary in this case as no product was scrapped or made nonconforming.

Now, let’s say an employee sees a potential problem.

Example: The employee notices the jaws of a turning center are showing very obvious/significant run-out.

This could potentially result in nonconforming product. This is a good case for preventive action. A change request could be generated and when the action is taken, it can be followed up on (verified) and recorded in the appropriate format. In most cases, over an entire year a company will record very few Preventive Action Requests (PAR’s). However, that same organization will register numerous Corrective Action Requests (CAR’s). This is the normal rhythm of things and is what we strive for.

Here are a few definitions for your files. The following Terms and Definitions are taken from ISO 9000:2005:

3.6.4
Preventive action: Action to eliminate the cause of potential nonconformity or other undesirable potential situation.

NOTE 1 There can be more than one cause for a potential nonconformity.
NOTE 2 Preventive action is taken to prevent occurrence whereas corrective action (3.6.5) is taken to prevent recurrence.

3.6.5
Corrective action: Action to eliminate the cause of a detected nonconformity or other undesirable situation.

NOTE 1 There can be more than one cause for a nonconformity.
NOTE 2 Corrective action is taken to prevent recurrence whereas preventive action (3.6.4) is taken to prevent occurrence.
NOTE 3 There is a distinction between correction (3.6.6) and corrective action.

3.6.6
Correction: Action to eliminate a detected nonconformity.

NOTE 1 A correction can be made in conjunction with a corrective action (3.6.5).
NOTE 2 A correction can be, for example, rework.

I hope this has been helpful.

Bud Salsbury
ASQ Senior Member, CQT, CQI
Takco Manufacturing
Phillips, WI
http://www.takcomfg.com

Additional ASQ Resources:

Form by Design
Using flowcharting techniques for robust form design
by Lance B. Coleman

Corrective Action Challenge
How to construct a robust problem-solving process
by R. Dan Reid

CAPA for the FDA-Regulated Industry (book)
Abstract: Medical devices, biopharmaceutical, and traditional drug manufacturing companies devote an important part of their resources to dealing with incidents, investigations, and corrective and preventive actions. The corrective and preventive action system is known as the CAPA system. It is second to none in terms of frequency and criticality of its deviations, and most of the regulatory actions taken by the FDA and foreign regulators are linked to inadequate CAPA systems. This guidance book provides useful and up-to-date information about this critical topic to thousands of engineers, scientists, and manufacturing and quality personnel across the life sciences industries.

Understanding and improving the CAPA system as a whole is the focal point of this book, the first of its kind dealing exclusively with this critical system within this highly regulated industry. By helping those in this industry improve their CAPA systems, it will be a crucial aid in their mission of producing safe and effective products.

Posted in CAPA, Other | Leave a comment

Employee Qualification Audit

Question

I am a Quality Assurance GxP Auditor and I am being told that I cannot perform employee qualification audit.  I am being told that CV/resumes, job descriptions, and training records are confidential and my viewing them would violate an employee’s privacy.  If this is true, how to I prove to my client that the company has qualified personnel?

On the same note, is this also true of an internal or 1st party employee qualification audit where my own company would want me to verify the qualifications of our employees to ensure they meet international FDA/ICH guidelines?

Response

Thanks for contacting ASQ’s Ask the Experts program.

With regard to your question, maintaining confidentiality can be a major concern for the employee, organization and the Auditor.  For this reason, the review of employee files containing private data such as social security numbers, banking, personal contact or other sensitive information should be avoided if possible.

This not only maintains employee privacy, but also reduces the Auditor’s level of exposure to potential liabilities.

So now the question is; how can the Auditor verify employee qualifications and experience? Remember that there is no requirement for an Auditor to review job applications, CV/resumes, or other confidential information.

It’s the organization’s responsibility provide the Auditor with objective evidence that they have established job descriptions for employees performing work activities that affect the quality of the product or services to be provided to the customer (ISO 9001:2008, clause 6.2.1).  This includes providing evidence that the employee’s qualifications, skills, education and any applicable certifications have been verified to meet job description requirements or the need for training has been established to ensure job description requirements are met (ISO 9001:2008, clause 6.2.2, sub., a. b and c).

As you are aware, a job description may be considered as proprietary, but they are seldom considered as private since they don’t contain any personal information.  Some organization’s may require that a nondisclosure agreement (NDA) be signed to protect propriety information such as engineering data, drawings or other methods related to product realization processes.

A record of an organization’s review and verification of employee qualifications should be readily available.  Likewise, training and applicable certification records should be available to provide objective evidence that qualification and/or competency requirements have been met (ISO 9001:2008, clause 6.2.2, sub., e).

Bill Aston
ASQ Senior Member
Managing Director of Aston Technical Consulting Services
Kingwood, TX
http://www.astontechconsult.com

Posted in Auditing | Tagged , , | Leave a comment

ISO 17025 calibration requirements

Question:

I was recently in a discussion with someone about ISO 17025 calibration requirements. It was my understanding that all equipment associated with the tests within our ISO 17025 scope needed to have an uncertainty value reported with each calibration. However, my coworker said only tests that actually use the uncertainty value as a part of their test results calculations require an uncertainty value. Meaning, we have may have tests performed within our ISO 17025 scope but the equipment doesn’t need an uncertainty value?

Could you please provide some clarity on this?

Response:

First let us understand why the measurement uncertainty is required. This is to support the metrological traceability requirement for any measurement made.

The definition of metrological traceability per ISO Guide 99:2007 is:
“Property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty”.

Therefore, if you require your measurements to be traceable, then measurement uncertainty is required. Since this definition of Metrological Traceability was defined per ISO Guide 99 in 2007, there has been confusion on the requirement for reporting measurement uncertainty. The current version of ISO/IEC 17025:2005 does not help by stating “and/or” requirement in Section 5.6.2.1.1 when reporting measurement results with measurement uncertainty.

Whenever such ambiguity exists in the standards, International Laboratory Accreditation Cooperation provides guides and policy documents for clarification for accrediting bodies and accredited laboratories. The ILAC P14:2013 is one such document which provides policy guidance in reporting measurement uncertainties for the laboratories and for accrediting bodies to enforce.

In short, if any equipment (and its associated measurements) requires metrological traceability, then measurement uncertainty must be estimated for that equipment regardless of if it is within the ISO/IEC 17025 scope of accreditation or not.

Dilip A Shah
ASQ CQE, CQA, CCT
President, E = mc3 Solutions
Chair, ASQ Measurement Quality Division (2012-2013)
Secretary and Member of the A2LA Board of Directors (2006-2014)
Medina, Ohio
www.emc3solutions.com

Additional ASQ Resources:

Assessing Performance- Understanding proficiency test result
by Christopher L. Grachanen
Abstract: For many calibration and testing laboratories, routine proficiency testing is a requirement for obtaining and maintaining accreditation status. ISO/IEC Guide 43-1—Proficiency testing by interlaboratory comparisons defines proficiency testing as a “means used in the determination of laboratory testing and measurement performance.

Implementing ISO/IEC 17025:2005 (book)
by Bhavan “Bob” Mehta
Abstract: The purpose of this book is to demystify the requirements delineated within ISO/IEC 17025:2005 while providing a road map for organizations that wish to receive/maintain accreditation for their laboratories. AS9100, ISO 9001, and ISO 13485 are standards that support the development and implementation of effective approaches to quality management and are recognized blueprints for the establishment of a quality management system (QMS) for diverse industries. Although similar to these recognized QMS standards, ISO/IEC 17025 serves a unique purpose: laboratory accreditation. It is not unusual for laboratories to retain dual certification to ISO 9001 and ISO/IEC 17025.

Posted in ISO 17025 - Testing and Calibration Laboratories, Metrology

AS9100 Audit

Question:

I have recently started work at a company that is registered to AS9100. My previous employer was registered to ISO 9001 and I was trained as an internal auditor.

What additional training is required to audit to AS9100? (other than learning the standard).

Does my previous training in internal quality auditing qualify me to audit to the AS9100 standard?

Are the standards for auditor different for AS9100 than ISO 9001?

Response:

The ISO 9001 and AS9100 requirement for internal auditors are the same; that the auditor be competent. The organization determines the competence requirements for its internal auditors. Typically, the competence includes both knowledge of the standard and internal audit methodology.

Buddy Cressionnie
International Aerospace Quality Group Americas AS9100 Lead
Voting member of the U.S. TAG to ISO/TC 176
Southlake, TX

Additional ASQ Resources

AS9100 Keeps Bosch Communications Flying High in Aerospace Industry
by Janet Jacobsen
Abstract: In 2006, the Bosch Corporation acquired Minnesota-based Telex Communications, Inc., a supplier to the aerospace industry. This business became known as Bosch Communications Systems. Boeing, a key customer for Bosch Communications’ aviation headsets, issued a requirement for all suppliers to become certified to AS9100, the international quality management system standard for the aerospace industry. To satisfy Boeing’s requirement, Bosch Communications launched an ambitious initiative to achieve dual AS9100/ISO 9001 certification in less than one year. Bosch contracted with ASQ to provide AS9100 lead auditor and internal auditor training to educate a cross-functional team about the standard and prepare them for the auditing process. In October 2008, just 11 months after launching its certification effort, Bosch earned both AS9100 and ISO 9001 certification.

Road to Revision- The path ahead for updating the AS9100 series of standards
by Buddy Cressionnie
Abstract: The flagship aviation, space and defense quality management system (QMS) standard has started revision activities. AS9100—Quality management systems—requirements for aviation, space and defense organizations is the foundation standard of the International Aerospace Quality Group (IAQG).

The AS9100C, AS9110, and AS9120 Handbook (ebook)
by James Culliton
Abstract: AS9100, AS9110, and AS9120, the quality management system (QMS) standards for the aerospace industry, are written in the most ambiguous language possible. Indeed, they don’t outline how they should be implemented. Those decisions are left to the organization implementing their requirements or, in some cases, to a consultant.

Although some consultant firms for aerospace systems are excellent, there are many that purport to be experts yet proffer systems and processes that are either in contravention to the standards’ requirements or so unwieldy that they render the process impotent.

In an effort to simplify these issues, this book proposes practices that have been described as opportunities for improvement or best practices by registration auditors in the past. It includes a discussion of each of the three standards’ clauses, suggests best practices to comply with them, outlines common findings associated with them, and provides an overview of the changes to AS9100C from AS9100B.

Posted in AS9100 - Aerospace, Auditing | Tagged , ,

Food Safety and Sampling

Question

I like to know how to sample a finished product or ingredient so that the sample to be tested is representative of the product as a whole so it will increase confidence in subsequent test result. This is needed to verify a particular finished product lot or incoming ingredient lot is allergen free.

Response

Sampling is not a simple process of looking up a sample size in a table. There are many factors that influence how you develop a sampling plan. When I develop a QA program, I always try to develop the program to answer a specific question or develop a null hypothesis. Once I have framed the question, I can then develop a sampling plan to help develop the answer.

It appears that the question you would like to ask is the following:

• Is an allergen present in either a lot of finished product or in a lot of ingredient?
This question deals with an attribute issue.

In developing a plan, one needs to take into account a number of statistical assumptions including the following:

• Is the process relatively stable? In statistical process control terms, the process is rarely affected by special or assignable causes of variation. The following is an alternative way to describe a stable process. Is the allergen evenly distributed in the lot or can the allergen be concentrated in one part of the lot? Answering this question helps defines the unit.

• A random sampling plan must be used to select the units to be tested.

• A unit must be defined. The unit must either possess the characteristic or not possess the characteristic. The presence of the characteristic makes the unit defective. Many times in food sampling, a unit may be difficult to define.

• A test must be available that can determine if the unit contains the characteristic. It is permissible to test a portion of the unit as long as long as that portion of the unit correctly identifies whether the unit is or is not defective.

• A sampling plan must be developed in which the units will be collected. Every unit must have an equal chance of being selected for analysis (random sampling).

• The number of units that possess the characteristic must be small (less than 10%) as compared to the number of units that do not possess the characteristic. The removal of the number of units for analysis cannot affect the portion of defective units in the lot.

• The number of “units” in the lot does NOT affect the sensitivity of the attribute sampling plan.

• The portion of units that are defective is critical to the sampling plan. If the portion of defective units declines there needs to be an increase in the number of units sampled to ensure that the sensitivity or power of the sampling plan does not change.

• The total number of units sampled is critical for the sensitivity or power of the sampling plan. The power increases with an increase in number of units sampled and tested to determine if they are defective.

• If the portion of defective units can be estimated, it is possible to calculate the power of the sampling plan using the binomial distribution. Likewise, if a sampling plan is selected, it is possible to calculate the power of the sampling plan for a specific proportion of defective units.

• Need to define the confidence level that is desired to determine whether a lot contains or does not contain the allergen.

• Acceptance number. The smaller the acceptance number, the less of a risk the lot will contain units that are defective. The smaller the acceptance number the more sensitive the sampling plan.

The alternative method is to develop a QA system based on the concepts of process control. A classical approach is to use HACCP.

John G. Surak, PhD
– Providing food safety and quality solutions -

Additional ASQ resources:

Statistical Process Control for the FDA-Regulated Industry
by Manuel E. Peña-Rodríguez
Abstract: The focus of this book is to understand and apply the different SPC tools in a company regulated by the Food and Drug Administration (FDA): those that manufacture pharmaceutical products, biologics, medical devices, food, cosmetics, and so on. The book is not intended to provide an intensive course in statistics; instead, it is intended to provide a how-to guide about the application of the diverse array of statistical tools available to analyze and improve the processes in an organization regulated by FDA.

This book is aimed at engineers, scientists, analysts, technicians, managers, supervisors, and all other professionals responsible to measure and improve the quality of their processes. Although the examples and case studies presented throughout the book are based on situations found in an organization regulated by FDA, the book can also be used to understand the application of those tools in any type of industry.

Readers will obtain a better understanding of some of the statistical tools available to control their processes and be encouraged to study, with a greater level of detail, each of the statistical tools presented throughout the book. The content of this book is the result of the author’s almost 20 years of experience in the application of statistics in various industries, and his combined educational background of engineering and law that he has used to provide consulting services to dozens of FDA-regulated organizations.

The Certified HACCP Auditor Handbook, Third Edition
by ASQ Food, Drug, & Costmetic Division
Abstract: This handbook is intended to serve as a baseline of hazard analysis critical control point (HACCP) knowledge for quality auditors. HACCP is more than just failure mode and effect analysis (FMEA) for food: it is a product safety management system that evolved and matured in the commercial food processing industry allowing food processors to take a proactive approach to prevent foodborne diseases. Both the FDA and the USDA have embraced HACCP as the most effective method to ensure farm-to-table food safety in the United States.

This handbook also assists the certification candidate preparing for the ASQ Certified HACCP Auditor (CHA) examination. It includes chapters covering the HACCP audit, the HACCP auditor, and quality assurance analytical tools.

Posted in FDA regulated industries, Sampling | Tagged , ,

TS16949 Certification Requirements

Question

My company is a fabless semiconductor company and we provide ASIC design and custom IP services, and manage our customers’ supply chain from foundry to assembly and test, and drop ship.

We are gearing ourselves up to supply IC chip to automotive customers and the prerequisites indicate our manufacturing partners must be ISO/TS16949 certified and our company’s QMS compliant to this standard.

There are a lot of documentations to be upgraded and revised, including supplier management specifications. I was thinking instead of repeating what ISO/TS16949 says from A to Z, I thought the best way is to add a clause in my supplier management spec is to specify “Suppliers manufacturing products for automotive customers must be certified to ISO/TS16949 (latest version) by a 3rd Party Registrar”.

Most of our manufacturing suppliers publish/broadcast their certifications in their external webpage and this is how we verify the certification status. Will this approach work from automotive customer’s perspectives?

Looking forward to your guidance and advice.

Response

Thanks very much for your question. A little bit about ISO/TS 16949; if your company doesn’t manufacture anything, then it is not eligible to be registered to ISO/TS 16949. You would, however, be required to be registered to ISO 9001. Double check that your customers are asking for your suppliers to be registered to ISO/TS 16949. That is not a default position. If it’s not in your contract, it won’t be a requirement (because your company is not ISO/TS 16949 registered).

If you are required to be compliant to ISO/TS 16949, make sure that your internal audit function conducts its audits against the ISO/TS 16949 Standard. At the practical level, the biggest impact to your company by being compliant is the requirement for you to use APQP and PPAP. PPAP will also apply to your suppliers.

Be careful about how you try to address supplier management in your QA manual. There’s much more to it than a one line statement. Your customers will expect a rigorous application of supplier selection and management (per ISO 9001) and they will expect you to manage APQP and PPAP with your suppliers. You will still have to be active in managing and being responsible for your suppliers. There will need to be more to your selection and evaluation process than looking at their websites for a certification.

The amount of risk depends on the closeness of the relationship with your “supplier partners”, and whether or not they have a high level of sophistication and experience in the automotive industry. This industry has a very sophisticated supply chain management process and very high performance expectations, with penalties to match.

I hope this answer gets you started.

Denis J. Devos, P.Eng
Fellow of the American Society for Quality
Devos Associates Inc.
Advisors to the Automotive Industry
http://www.DevosAssociates.com

Posted in TS16949 - Automobiles