Zero Acceptance Number Sampling Plans and the FDA

Pharmaceutical sampling

Question

There has been some debate over using the MIL-STD-1916 acceptance sampling plan over the ANSI/ASQ Z1.4-2003 (R2013) sampling plans.  The opinion is that the ANSI/ASQ Z1.4-2003 (R2013) is outdated and no longer an acceptable method of determining a qualification sample plan and the MIL-STD-1916 should be used in place of ANSI/ASQ Z1.4-2003 (R2013). Do you have information around this debate over which sampling plans are acceptable by the FDA?

Answer

FDA does not (and can not) tell you what sampling plan is to be used.  The FDA requirement is that the plan be statistically valid.  As long as you follow the regulation, you are meeting FDA requirements.

In medical device manufacturing the key point is to have the plan accept on zero defectives.  This point is not FDA but legalese.  It is based on past lawsuits.  The plan “Zero Acceptance Number Sampling Plans” by Nicholas L. Squeglia (available from ASQ) has been widely adopted for this reason.

ANSI/ASQ Z1.4 in not outdated and continues to be widely used.  It is the American National Standard Institute (ANSI) version of MIL-STD-105 which the government discontinued maintaining, allowing ANSI to maintain it along with many, many other MIL-STD’s as a government cost reduction.

MIL-STD-1916 can be used but it is not widely used because of its difficulty and practical use.

James Werner

Approved Supplier List 17025 or 9001?

ISO/IEC 17025:2017 General requirements for the competence of testing and calibration laboratoriesQuestion

Every once and while our company will need to find a rare or hard to find item on the web. Due to the rarity of the item we sometimes need to look at sites that are not a typical supplier. So how would you go about approving a supplier such as Amazon or EBay since they are more like a distributor then a supplier and utilize a large pool of other retailers/sellers?

Answer

I think ISO 17025 is not the correct citing; ISO 9001-2015 Section 8.4. would be a better fit.

Approving a distributor is meaningless whether the distributor is Amazon or EBay.  The requirement is under Section 8.4.1 of ISO 9001-2015.  Consider that the supplier is the manufacturer of the item (product) being bought on the web.  The user needs to approve the use of that item – not the supplier. 

The last paragraph under Section 8.4.1 reads, in part: The organization shall determine and apply criteria of the evaluation, selection, and monitoring of performance, and re-evaluation of external providers, based on their ability to provide processes or products and services in according with requirements.  This means that the organization determines the requirements, documents those requirements, and follows the establish requirements.  The requirements here, I suggest, are to approve the supplier based on the supplied item meeting the organization’s needs – specifications.

For example, Home Depot or Lowes is a distributor (source) of a hex-nut that is infrequently used.  The hex-nut has a specification, thread size, length, etc.  The requirement then would be that an inspection of the hex-nut confirms it meets the predetermined requirements. This is all documented.

James Werner

Z1.4 Sample Size

Pharmaceutical sampling

Question

I am trying to determine the sampling size using my ANSI/ASQ Z1.4 table and I wanted to get some clarification. If I am using Table II A and my Sample Size Code letter is D, what would be my sample size? If it falls on an arrow does it mean that I have to change to the next sample size based on where the arrow points?

Answers

From Charlie Cianfrani:

If you are using Z1.4, your sample size is selected based on your lot size.  You would pick the AQL you need based on the risk you are willing to take for the process average of percent defective.  It is important to understand what you are doing when using sampling plans, what they are and the protection you are trying to ensure. Thus, the important step is to determine the AQL. Then you select the sample size to provide the level of protection you are striving to ensure. It is more important to understand the theory behind the tables than to mechanically use the tables.

From Fred Schenkelberg:

Use the sample size where the arrow points. In the 2008 and 2013 versions it explains this in section 9.4, “When no sampling plan is available for a given combination of AQL and code letter, the tables direct the user to a different letter. The sample size to be used is given by the new code letter, not by the original letter.”

From Steven Walfish:

The standard sample size for Code Letter D from IIA is a sample size of 8.  But depending on your AQL, a sample size of 8 would be inappropriate, so the standard has arrows to delineate alternative sample sizes to reach the target AQL.  So, you sample size and accept/reject values are changed.  For example, at an AQL of 0.25, you would move down to a sample size of 50, with an accept/reject of 0/1.  If the lot size is less than 50, you would need to do 100% inspection.  In other words, there is no sampling plan that can give an AQL of 0.25 without a minimum sample size of 50.

From James Werner:

Yes.  When using Z1.4 two items need to be known, lot size and the AQL (Acceptance Quality Limit).  You use Table I – Sample size code letters to determine the Sample size code letter based on the Lot or batch size.  In the question below that was determined to be “D”.  Next step is to use Table II-A to find the sample size related to the sample size code letter – D and the AQL.  On Table II-A go across the table’s row for letter D until it intersect the given AQL column heading.  If an arrow is in that intersection point, follow the arrow then go back to the sample size code letter column to find the actual sample size (if a up/down arrow is in there then you choose).

Example 1.  Code letter is D (as in the question below).  Let’s say the AQL is 0.25.  Starting at code letter D, move across that row until you intersect at the AQL 0.25 column.  There’s a down arrow this row/column intersection.  Follow the arrow downward until the “Ac Re” reads ” 0 1″.  Staying on this row go back to the Sample size code letter column and find Code Letter H and Sample size = 50.  This means for the lot size with code letter D and with an AQL of 0.25 the sample size = 50 and accept the entire lot if no nonconformances were found else reject the entire lot if 1 or more nonconformance were found in the sample.

Example 2.  Let’s say the Sample size code letter was determine from Table I to be “F”.  Looking at Table II-A; If the AQL = 0.65, then the sample size would be 20 and the lot would be accepted zero nonconformance.  But if the AQL = 0.15 then the sample size would be 80.

ASQ/ANSI Z1.4 is available for purchase in PDF as well as hard copy.

Audit Timeline

Question

What is the ASQ recommended time frame between an auditee receiving a final audit plan and the audit commencing at the auditee’s site?

Answers

From Charlie Cianfrani:

ASQ does not have a recommendation!

From George Hummel:

This is not an ASQ requirement.  A CB generally sends an audit schedule/plan three weeks before the audit.

From Jim Werner:

Typically, the final audit plan has been agreed to by both the auditor and the auditee and it includes the date(s) the audit is to take place. This means that the audit plan includes the audit schedule in one document.  There are many books written, with examples, on this topic.  The ASQ Audit Division is a good source.

Is Certification Revocable?

Question

If a company is ISO 9001: 2015 certified, is it revocable?

Answers

From Jim Werner:

A company can indeed have its certification revoked.  Being certified means the company has established a qualify management system that meets the requirements of ISO9001:2015.  The failure of the company to continue to meet those requirements can result in de-certification.

From George Hummel:

Most CBs will revoke a certificate if the client does not answer an audit non-conformance.  Their contract may define other instances.  The questioner should review his or her organization’s contract.

From Charles Cianfrani:

Certified companies receive surveillance audits periodically. If the company fails to maintain compliance with ISO 9001:2015 requirements, eventually (after a series of intermediary steps related to resolution of nonconformity have been unsatisfactorily pursued) their certification can be voided.

ISO Certification and Suppliers

Mr. Pareto Head and Supply Chain comic strip

Question

I work for a small family company that purchases items and potentially processes or packages them into heat protection materials. One of my existing customers is asking for ISO certification for some materials that I will sell to them. The material I’m trying to sell him comes from my supplier who is ISO 9001 certified, but my company is not. How can I show my customer that my supplier is ISO certified without the customer knowing who my supplier is?

Answers

The company is doing a value added process, and not a distributor.  As a result, if the customer is demanding ISO 9001 certification from the company, they need to make the decision, do they want to do business with the company? If so they need to pursue certification. If they do not want to pursue certification, they should tell the customer they do not want to pursue certification.  The customer can make the decision whether they will purchase product from the company.  I have had an experience where I did not want to do an audit with a company.  We told the customer, we will not do it.  The customer responded and came back with a reasonable proposal.  They wanted the business.

John G. Surak, PhD
Surak and Associates
Clemson, SC
A member of Stratecon International Consultants
www.stratecon-intl.com/jsurak.html

First, ISO certification is for a company’s quality management system, not for particular materials.  I would let the customer know, on company letter head, that:  “We certify that the materials we purchased are from ISO 9001 certified suppliers only.  The name of these suppliers is company confidential.”

James D. Werner
Principal Consultant
MDQC
Medical Device Quality Compliance, LLC

Creating a Culture of Quality

ASQ Global State of Quality 2016

Question

I was introduced to Quality Management (& ISO 9001:2015) recently. The culture of the organization that I am concerned with has not embraced Quality Management, and it is often the subject of outright and unprofessional antagonism. I seek direction in order to arm myself with greater knowledge or qualifications as well as change attitudes toward Quality Management at all levels within the organization. I thought that ASQ would be a good resource. Since there are so many channels, a plethora of literature, and various certifications and conferences, I am a bit overwhelmed. I need to focus my efforts, and I hope to be able to do so with some direction from a professional who can relate to such growing pains. Thank you.

Answer 1

Thank you for your question.  I can certainly to relate to you and your plight – I was in a similar circumstance early in my career.   If you were introduced to ISO 9001 this year, I have to assume that your company is not yet registered.   Most manufacturing companies are required by their customers to have registration, but if you are not in that situation, you have to sell Quality Management on its own merit.  The bad news is, that if your senior management doesn’t want a Quality Management System, there is nothing you can do about that.  Now, that being said, you can begin by examining some of the “pain points” in your organization and showing how quality tools can help to solve them.  Management will never embrace quality until they see what is in it for them.   You can start with an analysis of the Cost of Poor Quality.  When your leadership sees the cost of nonconformance, they will be keen to bring those costs down.  COPQ typically includes the cost of external customer complaints, replacing products, late deliveries, and internal costs such as scrap, rework, re-makes etc.  If there has ever been a problem that traces back to not properly understanding a customer’s needs, that is a text-book example of how Quality Management can help.  Start with that.   Look at the costs of poor quality, and sell the idea of using quality tools to bring those costs down.  You will have no chance of selling your management on quality until they can see what’s in it for them.  Good Luck!

Denis J. Devos, P.Eng
A Fellow of the American Society for Quality
Devos Associates Inc.
(519) 476-8951
www.DevosAssociates.com

Answer 2

it sounds like this company needs a culture change. This change can happen only at the direction of the company’s leadership.
Here’s some suggestions:

  1. Each department head has to establish three (3) measurable goals on how his/her department is improving on the quality of their department’s output/work.  These are to be reported at each executive monthly meeting.  Department manager’s must be held accountable for lack of quality improvements.
  1. Every individual’s performance review must include “quality performance.”  This also needs to be measurable (less than last year, improved customer satisfaction from surveys, reduced ‘cost-of-quality’, reduced audit nonconformance observations, etc.)
  1. If the company has a bonus program, individuals/departments bonus is tied into quality performance.  ISO observation means 10% or more cut in bonus.
  1. Have top executives hold meetings on the need for quality and it’s everyone’s responsibility – not just the QA department.  If employees don’t like it they are welcomed to find employment elsewhere.

Jim

Jim Werner
Voting member to the U.S. TAG to ISO TC 176 Quality Management and Quality Assurance
Medical Device Quality Compliance (MDQC), LLC.
ASQ Senior Member
ASQ CQE, CQA, RABQSA Lead QMS Assessor

Terminology for Inspected Material (GMP, ISO 13485)

Pharmaceutical sampling

Q: There is often confusion with the labeling of purchased materials  after they have been “inspected, tested and/or verified” according to good manufacturing practice (GMP)
requirements.  Once out of quarantine, are purchased materials labeled as accepted, approved or released?  I’ve had auditors and inspectors tell me all three.

A: Either term (accepted, approved, or released) is appropriate and commonly used.  It would appear that the auditors are voicing an opinion and shouldn’t be. Neither ISO 13485:2003: Medical devices — Quality management systems — Requirements for
regulatory purposes or FDA’s quality system regulation (QSR) specify what language is to be used.

ISO 13485:2003, clause 7.5.3.3 status identification, states:

“The organization shall identify the product status with respect to monitoring and measurement requirements.  The identification of product status shall be maintained throughout production, storage, installation and servicing of the product to ensure that only product that has passed the required inspections and test … is dispatched, used or installed.”

FDA 21 CFR 820.86 acceptance status requires:

“Each manufacturer shall identify by suitable means the acceptance status of product, to indicate the conformance or nonconformance of product with acceptance criteria. The identification of acceptance status shall be maintained throughout manufacturing, packaging, labeling, installation, and serving of the product to ensure that only product which has passed the required acceptance activities is distributed, used, or installed.”

The requirement should be clear for purchased materials: identify so that only those materials that passed acceptance activities are allowed to be used.  Neither the standard or regulation states how the material is to be identified.  That is up to the manufacturer to define in its operating procedure(s).

My personal recommendation is to use the terms “accept/reject” at receiving and during in-process, then use the terms “release/hold” to mean the final product is or is not to be released for distribution.  But any similar terms are fine as long as they are consistently used throughout the quality system and personnel understand the requirement that they can only use product that passed their acceptance activities.

Jim Werner
Voting member to the U.S. TAG to ISO TC 176 Quality Management and Quality Assurance
Medical Device Quality Compliance (MDQC), LLC.
ASQ Senior Member
ASQ CQE, CQA, RABQSA Lead QMS Assessor

Related Resources

Browse the free, open access articles below, or find more in the ASQ Knowledge Center.

FDA Regulations and Auditing Practices for Software Suppliers at a Pharmaceutical Manufacturer, Software Quality Professional

A review of 17 quality audit reports performed between 1992 and 2003 by an international pharmaceutical company identifies three time frames of audit maturity based on government regulation enforcement patterns, supplier quality practices, and the customer’s changing expectations. Read more.

Statistics in Pharmaceutical Development and Manufacturing, Journal of Quality Technology

An overview is given of the use of statistical thinking and methods in the research and development and manufacturing functions in the pharmaceutical industry. Four case studies illustrate how these issues work in real life settings.  Read more.

Explore the ASQ Knowledge Center for more case studies, articles, benchmarking reports, and more.

Browse ASQ magazines and journals here. 

Establishing and Maintaining a CAPA System

CAPA process, CAPA requestsQ: We have a Corrective Action and Preventative Action (CAPA) system, and we find that CAPAs are almost always completed late — even though we do have an extension request form for CAPAs, and the system sends automated reminders to  employees in advance.

What can we do to resolve this issue and avoid late CAPAs?

A: I will answer this question based on the information provided.

1. Does the CAPA system rank the CAPA based on risk? If not, each CAPA should be ranked either high, medium, or low.

High risks generally mean that the problem behind the CAPA could have a negative affect on the business and put it at risk. For example, in the medical device industry, a high risk CAPA could include a regulation violation, something that can harm a device user or patient, or issues that could result in legal action against the company.

2. Does the CAPA system have a way to involve top management? If not, it should — especially if timely corrective action is not being taken in instances of high risk CAPAs.

3. Does the management review process include a statistical analysis of the time it takes to complete CAPAs?

Often, reports to management include the number of CAPAs greater than 90-days old and greater than 180-days old. In addition to reporting on the number of open CAPAs, also report on the number of CAPAs completed by the due date and the number of CAPAs that are overdue (past the original, assigned completion date).

It is a good idea to also convert these numbers into percentages to make data digestible and to allow for comparison making.

4. Next, discuss with management (if possible) to consider consequences for employees if company problems that result in a CAPAs are not addressed in a timely manner.

With this approach, proceed with caution. You must make certain that the CAPA system is robust. Not every little problem is a CAPA. A good way to weed out the CAPAs from the non-CAPAs is to ask: is this an issue that requires an investigation into the root cause? And, does this problem require corrective action to fix it? If the answers are yes, then it is probably a CAPA.

5. You may want to consider benchmarking how other organizations structure their CAPA system and look to guidance documents for help. The Global Harmonization Task Force published a guidance document help establish CAPA systems. It is for the medical device industry, but it can be applied elsewhere.

Jim Werner
Voting member to the U.S. TAG to ISO TC 176
Medical Device Quality Compliance (MDQC), LLC.
ASQ Senior Member
ASQ CQE, CQA, RABQSA Lead QMS Assessor

Related Content:

Preview a sample chapter from the ASQ Quality Press book CAPA for the FDA-Regulated Industry, along with the full table of contents by clicking here.  Find more information about this book by visiting its ASQ Quality Press webpage.

Restructuring an Internal Auditing Program

Reporting, best practices, non-compliance reporting

Q: For the last 15 years, my company has employed a small cadre of full-time, dedicated safety management system auditors.

A current proposal in our company is to recast those auditors as HES Superintendents under the supervision of an operations or safety manager who has significant management responsibility within the safety management system.  This change will give HES Superintendents (persons performing audits) additional, non-audit tasks for performance on the premises of the auditee immediately before, during or after the audits.  Those non-audit tasks could include workforce training, management mentoring and evaluation, facility inspection, etc. In addition, this change will reduce about 50% of the number of audits performed per person in a given time period.

My concerns are as follows:

•  Supervision of the HES Superintendents (especially assignment, evaluation and compensation determination) by an operations manager, safety manager, or someone under their supervision, could constitute auditee control of the audit program, and a thwarting of the principle of auditor independence.

•  The addition of non-audit tasks to auditors’ work seems to open possibilities for audit conflicts of interest. Since HES Superintendents will participate materially in the ongoing safety management of the company, their independence and impartiality as safety management system auditors would be subject to question.

•  The 50% reduction in number of audits per auditor would result in dilution of auditors’ audit experience and therefore their expertise, leading to attenuation of the company’s capability to audit expertly.

In terms of the principles of management system auditing, are my concerns valid?

Do you know of other instances of this part-time-auditor approach being used in high-risk industries?

Any comment on the wisdom of this proposal?

Occasionally, mutiple experts offer their expertise and viewpoints to assist quality practicioners. Add your voice by commenting on posts!

Bill Aston’s take:

A: You’ve mentioned valid concerns that should be assessed by top management prior to restructuring their organization’s audit program.  As I understand your concerns, they include two primary items:

1.    To ensure that the restructure of the audit program continues to provide auditors with independence, objectivity and impartiality from the processes and process owners to be audited.

2.    Potential result of a 50% reduction of the number of audits conducted per auditor diluting auditor experience and expertise.

With regard to the first item, this is a matter that top management should thoroughly evaluate to ensure that the requirements of ISO 9001:2008 — Quality management systems — Requirements, clause 8.2.2b internal audit, continue to be met.  This clause requires that The selection of auditors and conduct of audits shall ensure objectivity and impartiality of the audit process.  Auditors shall not audit their own work.

In addition, although the requirements in ISO 19011:2011– Guidelines for auditing management systems are not auditable requirements, section 3.1, Terms and Definitions, (note 1), does mention the need for ensuring internal auditor independence.

The key point is that your organization’s registrar will most likely look very closely at how the audit program has been restructured to ensure that auditor independence, objectivity and impartiality have been maintained.

Regarding item number two, although maintaining an auditor’s level of expertise and experience are important, the primary purpose of internal audits is to assess the effectiveness and continual improvement of the quality management system and its processes.  If maintaining auditor expertise and experience becomes an issue due to the reduction in the number of available audit assignments, management should consider adjusting the number of auditors needed to meet the actual workload.

As you’re aware, ISO 9001:2008 requires internal audits to be conducted at planned intervals, but it does not prescribe any frequency for performing audits.  So this area is strictly a decision that must be made by each organization to meet their own specific requirements to ensure the continual improvement of the quality management system (QMS).

In summary, ISO 9001:2008, clause 5.4.2b Quality management system planning, requires top management to ensure that the integrity of the quality management system is maintained when changes are planned and implemented.  This includes the restructuring of processes such as the audit program.  Internal audits are one of the most important tools that an organization has to assess the effectiveness and continual improvement of their quality management system.   Therefore, it’s essential that the personnel performing these audits are trained, experienced and independent of the area being audited.

It has been my experience that there are few organizations that maintain a staff of fulltime QMS auditors.  Most organizations utilize staff personnel who are familiar with the processes to be audited and have been trained and are experienced as auditors.  Although they perform audits, this is usually not their only responsibility.  However, in some cases, large organizations may have one or two fulltime auditors who function corporate-wide and are supported by trained and experienced staff personnel on an as needed basis.

I hope this helps.

Bill Aston
ASQ Senior Member
Managing Director of Aston Technical Consulting Services
Kingwood, TX
www.astontechconsult.com

Thea Dunmire’s take:

A: Given that this question involves audits of a safety management system rather than a quality management system, the more applicable standard would likely be OHSAS 18001:2007 Occupational health and safety management systems – not ISO 9001:2008.  However, OHSAS 18001 also specifically states – “Selection of auditors and conduct of audits shall ensure objectivity and the impartiality of the audit process.”  Although OHSAS 18001 does not include the statement – “Auditors should not audit their own work,” that is definitely true.   As a general rule, auditors should not audit activities for which they are responsible or accountable.

It is common for organizations to utilize individuals as internal auditors who have other staff responsibilities.  Few organizations have dedicated environmental, health and safety management system auditors.  Most internal environmental health and safety (EHS) auditors have other responsibilities.  In addition, based on surveys conducted by the Auditing Roundtable, the overall management of the EHS audit program is often located within the EHS department, not in a separate internal audit function.  This can make ensuring the independence of the EHS audit program very challenging.

The important question isn’t whether specific individuals are auditing full or part time. Instead, it is whether all of the auditors utilized within the audit program have the appropriate independence, competence and resources to conduct the audits they have been assigned.  Independence I have discussed above.  By competence, I mean the general knowledge and skills needed for management system auditing (as set out in clause 7.2.3 Possess appropriate knowledge and skills of ISO 19011) as well as technical expertise appropriate for their audit assignments.  By resources, I mean that there is sufficient support, including adequate time, to conduct the individual audits needed to meet the objectives established for the audit program.

Identifying the resources needed for the audit program is one of the key responsibilities of the person assigned the role of audit program manager (as set out in clauses 5.3.1 Perform audit program management tasks and 5.3.6 Identify program resource requirements  of ISO 19011:2011).  Lack of adequate resources is a common weakness of many internal audit programs.  Often, internal audit programs have very broad and expansively-stated objectives, but lack the resources needed to achieve these objectives.  It is the audit program manager’s responsibility to point out this disparity to top management.  The solution is for top management to either adjust the objectives of the audit program, taking into account the policy commitments made by the organization, or provide more resources for the internal audit program.

A key requirement of a safety management system is identifying the organization’s legal and other requirements to which it subscribes.   These identified requirements must be taken into account when establishing management system programs and procedures.  This includes any legal obligations associated with establishing and maintaining internal audit programs.  For example, for organizations subject to the BOEMRE regulations (offshore oil and gas), the Safety Environmental Management System  (SEMS) regulations require that auditors be qualified and independent (see 30 CFR 250.1926).  Legal requirements, as well as the commitments made by the organization in its occupational health and safety policy (or its sustainability reports), must also be taken into account when identifying the resources needed for the EHS audit program.

Internal audits are one of the important ways of assessing the effectiveness of a management system.  The audit program itself should be reviewed to determine its effectiveness in accomplishing this task.  Changes can, and should, be made to internal audit programs but the potential impacts of proposed changes need to be fully assessed in light of the organization’s policy commitments and its legal obligations.

Here is a link to the Auditing Roundtable survey results I mentioned: AR Member Survey Results – Organizational Location of the EHS Audit Program

Thea Dunmire, JD, CIH, CSP
ENLAR Compliance Services, Inc.
http://www.enlar.com/
Largo, FL

Jim Werner’s take:

A: This is indeed a unique question.  I read and re-read this question over and over, and I have come up with the same opinion – “it depends.”  I am assuming “audit” is referring to an independent review of the quality system.  Some places use the term “audit” to mean an inspection activity.  If the past audits have consistently demonstrated the effectiveness of the quality system, then it is appropriate to reduce the number and frequency of the audits.

As far as the re-organization of the staffing of the auditing function – this is a management decision.

Jim Werner
Voting member to the U.S. TAG to ISO TC 176
Medical Device Quality Compliance (MDQC), LLC.
ASQ Senior Member
ASQ CQE, CQA, RABQSA Lead QMS Assessor

Read more open access content about auditing from the ASQ Knowledge Center archive: