Zero Acceptance Number Sampling Plans and the FDA

Pharmaceutical sampling

Question

There has been some debate over using the MIL-STD-1916 acceptance sampling plan over the ANSI/ASQ Z1.4-2003 (R2013) sampling plans.  The opinion is that the ANSI/ASQ Z1.4-2003 (R2013) is outdated and no longer an acceptable method of determining a qualification sample plan and the MIL-STD-1916 should be used in place of ANSI/ASQ Z1.4-2003 (R2013). Do you have information around this debate over which sampling plans are acceptable by the FDA?

Answer

FDA does not (and can not) tell you what sampling plan is to be used.  The FDA requirement is that the plan be statistically valid.  As long as you follow the regulation, you are meeting FDA requirements.

In medical device manufacturing the key point is to have the plan accept on zero defectives.  This point is not FDA but legalese.  It is based on past lawsuits.  The plan “Zero Acceptance Number Sampling Plans” by Nicholas L. Squeglia (available from ASQ) has been widely adopted for this reason.

ANSI/ASQ Z1.4 in not outdated and continues to be widely used.  It is the American National Standard Institute (ANSI) version of MIL-STD-105 which the government discontinued maintaining, allowing ANSI to maintain it along with many, many other MIL-STD’s as a government cost reduction.

MIL-STD-1916 can be used but it is not widely used because of its difficulty and practical use.

James Werner

Acceptance Sampling Inspection

Automotive inspection, TS 16949, IATF 16949

Question

We have an Acceptance sampling inspection in place where we use the ANSI/ASQ Z1.4 – 2013 standard under Normal Inspection, using General Inspection Level II to drive our samples size and accept, reject criteria. We do not uses switching rules as we have always found them too difficult to manage. I have two questions.

If I have one lot that fails Acceptance sampling and I am trying to bound the issue is it suitable to bound it to the one affected lot if the lot before and after pass or do I need to carry out additional sampling.

My second question is if I have a batch that passes acceptance sampling but at a subsequent downstream process a defect being inspected for by the upstream acceptance sampling inspection is found how do I determine if the lot is acceptable? Do I trust the acceptance sampling inspection or react?

Answer

The first question is not an uncommon one and actually it is a good practice to isolate the lot and do 100% inspection of it.  That way you can estimate the % defective and if another failure occurs in the next 5 lots, then increase the sampling until you have some confidence that the supplier has fixed the problem.  Once that confidence is restored, then you go back to what you inspected originally.

The second question, is one that you have to understand how well do you follow the acceptance sampling process?  If your alpha level is at 95%, 5% of the time, you can accept a bad batch as good. That is the pure definition of the alpha risk.  If this failure falls within the 5%, your process is working and while you sort through the lot, and notify the supplier, it is not something that you over react to.

I hope this helps.

Jim

James Bossert, PhD, MBB, CQA, CQE, CqM/OE
Sr Performance Improvement Consultant

Sampling and Daylight Savings Time

Question

We are wondering if there is any generally accepted procedure to account for seemingly duplicate sample times when operating over the time period when clocks ‘fall back’ one hour for the end of daylight savings time? Our standard practice is to analyze a chemistry sample every half hour, so we foresee two each of the 01:00, 01:30 & 02:00 AM samples this next Sunday morning. Please advise on any generally accepted practice to account for such seemingly duplicate samples.

Answer

I do not know an industry accepted standard for this yet.

If used for control charting, just plot as normal, noting when the sample was taken. For the second 2:30am sample, just note it was after the time change… continue monitoring the process as normal.

If used for lot sampling, analyze the results as normal.

If doing a daily average, then adjust the calculation for the two extra samples, i.e. divide by 26 instead of 24.

At most it may require a slight change to calculations based on the number of samples, otherwise it’s just not a big issue that may require at most a comment or note about the seemingly duplicate sample times or two missing times (in spring).

Cheers,

Fred

Fred Schenkelberg
Reliability Engineering and Management Consultant
FMS Reliability
(408) 710-8248
fms@fmsreliability.com
www.fmsreliability.com
@fmsreliability

Sample Size and Z1.4

Data review, data analysis, data migration

Question

My question is if I’m trying to determine the sample size of migrated data to see if it migrated correctly to the target database, is the Z1.4 table applicable to that?

The scenario is data is being transferred from an old system to a new system and I want to do a quality check on the data in the new database to make sure everything was transferred correctly. I’m hoping to use the Z1.4 table to determine the sample size if its applicable. Is it applicable and if not, do you know of other standards that I should be looking into that is more applicable?

Answer

The movement of a database from one system to another certainly may introduce errors and it may also carry over errors that already exist. In some cases the move may also find and repair errors, yet that generally is done by design.

So, let’s say it’s just a move and you are checking for any new errors that are introduced.

Since you have access to the entire population, the database, in a before (old system) and after the move (new system) and I’m assuming you do not want to check every entry, instead just a sample, then I would recommend using an hypothesis test approach rather than a lot sampling approach.

A hypothesis test based on the binomial distribution may be appropriate as you are checking field entries to determine if they are correct or not (pass/fail).

You can set a threshold defect rate that you want to check the new system is at least this good or better, or you can measure the old system and compare to the new system – it should be equal to the old system as null hypothesis.

You can find a bit more information about a p-test in a good stats book or online at a short tutorial I wrote at https://creprep.wordpress.com/2013/06/01/hypothesis-tests-for-proportion/

The Z1.4 standard would require you to artificially define a lot or consider the entire database as one lot. The standard lot testing approach does not provide the control and statistical power of hypothesis testing, thus my recommendation. With the p-test you can define the confidence, defect rate to detect, and sample size to fit your needs concerning ability to make measurements, cost, and risk.

Cheers,

Fred

Fred Schenkelberg
Reliability Engineering and Management Consultant
FMS Reliability
(408) 710-8248
fms@fmsreliability.com
www.fmsreliability.com
@fmsreliability

Sampling

Question

Is there a sampling plan for determining the number of cases to pull in a batch from which you perform the ANSI/ASQ sampling of individual products?  For example: you receive 550 cases with 145 product vials/case.  Is it proper to sample a total of 500 vials from 25 cases (using square root of n+1) or would applying the ANSI/ASQ single level II be more appropriate?  We would then need to pull 500 vials from 80 cases.  Or is there a better statistical method?

Answer

There are two ways to answer this. One is to follow the standard and take samples from 80 cases until you get 500. It is assumed that the samples are random so that you do not always take the samples from the same location in the case.  That is following the standard.

The second is that you take a sample from 25 cases in a random manner.  That is fine also.  There are no standards for sampling from cases so either way will work.  Years ago, I developed a sampling scheme similar to what is proposed at the employer I was working with at the time.  Sometimes you have to be creative.

Jim Bossert

SVP Process Design Manger, Process Optimization
Bank of America
ASQ Fellow, CQE, CQA, CMQ/OE, CSSBB, CSSMBB
Fort Worth, TX

Sample Size

Manufacturing, inspection, exclusions

Question

If we have a lot size of 27 and we are using a normal inspection level II with an AQL of 2.5. What is the sample size?

Answer

Assuming an attribute is being measured, we use ANSI ASQ Z1.4.2013 to find the sample size.

Given a lot size of 27 we first find in Table I. Sample Size Code Letter that Code letter D represents the sampling plan code letter for lot sizes between 26 and 50 for normal sampling (General Inspection Level II).

The move to Table II-A Single sampling plans for normal inspection to find the row for code letter D and under column for ASQ 2.5 find an up arrow. This indicates that we should use the code letter C which suggests a sampling plan of 5 samples and accept the lot if there are zero defect and reject the lot with one or more rejects.

Hope that helps.

Cheers,

Fred

Fred Schenkelberg
Reliability Engineering and Management Consultant
FMS Reliability
(408) 710-8248
fms@fmsreliability.com
www.fmsreliability.com
@fmsreliability

Food Safety and Sampling

Pharmaceutical sampling

Question

I like to know how to sample a finished product or ingredient so that the sample to be tested is representative of the product as a whole so it will increase confidence in subsequent test result. This is needed to verify a particular finished product lot or incoming ingredient lot is allergen free.

Response

Sampling is not a simple process of looking up a sample size in a table. There are many factors that influence how you develop a sampling plan. When I develop a QA program, I always try to develop the program to answer a specific question or develop a null hypothesis. Once I have framed the question, I can then develop a sampling plan to help develop the answer.

It appears that the question you would like to ask is the following:

• Is an allergen present in either a lot of finished product or in a lot of ingredient?
This question deals with an attribute issue.

In developing a plan, one needs to take into account a number of statistical assumptions including the following:

• Is the process relatively stable? In statistical process control terms, the process is rarely affected by special or assignable causes of variation. The following is an alternative way to describe a stable process. Is the allergen evenly distributed in the lot or can the allergen be concentrated in one part of the lot? Answering this question helps defines the unit.

• A random sampling plan must be used to select the units to be tested.

• A unit must be defined. The unit must either possess the characteristic or not possess the characteristic. The presence of the characteristic makes the unit defective. Many times in food sampling, a unit may be difficult to define.

• A test must be available that can determine if the unit contains the characteristic. It is permissible to test a portion of the unit as long as long as that portion of the unit correctly identifies whether the unit is or is not defective.

• A sampling plan must be developed in which the units will be collected. Every unit must have an equal chance of being selected for analysis (random sampling).

• The number of units that possess the characteristic must be small (less than 10%) as compared to the number of units that do not possess the characteristic. The removal of the number of units for analysis cannot affect the portion of defective units in the lot.

• The number of “units” in the lot does NOT affect the sensitivity of the attribute sampling plan.

• The portion of units that are defective is critical to the sampling plan. If the portion of defective units declines there needs to be an increase in the number of units sampled to ensure that the sensitivity or power of the sampling plan does not change.

• The total number of units sampled is critical for the sensitivity or power of the sampling plan. The power increases with an increase in number of units sampled and tested to determine if they are defective.

• If the portion of defective units can be estimated, it is possible to calculate the power of the sampling plan using the binomial distribution. Likewise, if a sampling plan is selected, it is possible to calculate the power of the sampling plan for a specific proportion of defective units.

• Need to define the confidence level that is desired to determine whether a lot contains or does not contain the allergen.

• Acceptance number. The smaller the acceptance number, the less of a risk the lot will contain units that are defective. The smaller the acceptance number the more sensitive the sampling plan.

The alternative method is to develop a QA system based on the concepts of process control. A classical approach is to use HACCP.

John G. Surak, PhD
– Providing food safety and quality solutions –

Additional ASQ resources:

Statistical Process Control for the FDA-Regulated Industry
by Manuel E. Peña-Rodríguez
Abstract: The focus of this book is to understand and apply the different SPC tools in a company regulated by the Food and Drug Administration (FDA): those that manufacture pharmaceutical products, biologics, medical devices, food, cosmetics, and so on. The book is not intended to provide an intensive course in statistics; instead, it is intended to provide a how-to guide about the application of the diverse array of statistical tools available to analyze and improve the processes in an organization regulated by FDA.

This book is aimed at engineers, scientists, analysts, technicians, managers, supervisors, and all other professionals responsible to measure and improve the quality of their processes. Although the examples and case studies presented throughout the book are based on situations found in an organization regulated by FDA, the book can also be used to understand the application of those tools in any type of industry.

Readers will obtain a better understanding of some of the statistical tools available to control their processes and be encouraged to study, with a greater level of detail, each of the statistical tools presented throughout the book. The content of this book is the result of the author’s almost 20 years of experience in the application of statistics in various industries, and his combined educational background of engineering and law that he has used to provide consulting services to dozens of FDA-regulated organizations.

The Certified HACCP Auditor Handbook, Third Edition
by ASQ Food, Drug, & Costmetic Division
Abstract: This handbook is intended to serve as a baseline of hazard analysis critical control point (HACCP) knowledge for quality auditors. HACCP is more than just failure mode and effect analysis (FMEA) for food: it is a product safety management system that evolved and matured in the commercial food processing industry allowing food processors to take a proactive approach to prevent foodborne diseases. Both the FDA and the USDA have embraced HACCP as the most effective method to ensure farm-to-table food safety in the United States.

This handbook also assists the certification candidate preparing for the ASQ Certified HACCP Auditor (CHA) examination. It includes chapters covering the HACCP audit, the HACCP auditor, and quality assurance analytical tools.

Recommended AQLs for Packaging Material in the Pharmaceutical Industry

Question:

Are there recommended AQLs (critical, major and minor defects) for packaging material (primary and secondary) for the Pharmaceutical Industry?
Thank you

Response:

Though there are no recommended AQLs (or LTPD) for packaging materials, some industry standards have begun to surface. The following table is a guideline that I have seen used successfully for a risk based approach to sampling. Based on the severity and criticality of the packaging materials, these guidelines can be adjusted up to down per your risk management process.  Utilizing a c=0 sampling plan based on the binomial distribution, the sample size can be calculated using the following formula.

 

Primary

Secondary

Confidence (%)

Reliability (%)

Sample Size

Confidence      (%)

Reliability (%)

Sample Size

Critical

99.5

99

527

99

97

151

Major

99

97

151

97

95

68

Minor

97

95

68

95

90

28

I hope this assists you.

Steven Walfish
Secretary, U.S. TAG to ISO/TC 69
ASQ, CQE
Principal Statistician, BD
http://statisticaloutsourcingservices.com

Pesticide Residues Surveillance Program sampling

Question:

I am plant production specialist working in government sector.  I am the manager of pesticides residues surveillance program, on this program we targeting local commodities of fresh fruits and vegetables (F&V) by sampling the targeted numbers and types of F&V in regular basis around the year and we analyze samples and results and establish the annual report. I have checked many similar program in other countries included USDA program but I didn’t find approach methodology or statistical way to identify the sample size to be targeted in the year taking in account type and number of crops, crop production,…etc. to elaborate annual sampling plan. My question here is how can elaborate sampling plan for mentioned program considering all valuable factors?

Your cooperation is highly appreciated.

Thanks in advance…

Response:

Sampling is a method to estimate population parameters. For example, if the goal is to determine the amount of unacceptable residue on store bought apples, and testing every individual apple is impractical, then we use a sample to estimate the proportion with unacceptable residue.

The sample plan must focus on the goal and balance with the resources and technology available. If the goal is to accurately detect a very low proportion with residue, say 1 in 1 million, then the sample size will be larger than if the goal is to detect 1 in 100 with unacceptable residue. The goal to detect 1 in 100 is easier to accomplish (fewer apples tested) yet does not reveal is there is a 1 in 1000 level or not.

A key element is the specific goal for detection and design a sample plan that is capable to detect at or better than the goal’s level. Capable includes the measurement system errors and an understanding of the nature of how failures occur.

Another consideration is the nature of the measurement and goal. If the test is only pass / fail for presence of residue, then we have to use the relatively inefficient sampling plans based on the binomial distribution. If the data is a variable value, such as part per million residue presence, then we can use more efficient sampling plans based on the appropriate continuous distribution. If the testing is destructive to the item being tested that limits the sampling techniques available.

How is the lot defined? If this is an annual report then the lot may be the annual production of a specific fruit or vegetable, say a specific variety of apples. Define the population clearly and any relevant subgroups of interest. If the data is only for an annual report the sampling plan is marked different than if the goal is a monthly monitoring and warning system.

Another consideration is the thresholds along with confidence. For sampling plan creation we use two specific points of interest. The Producer Risk Point (PRP) made up of the Acceptable Quality Level (AQL) and the producers’ risk (Type I risk or Alpha – which is the probability of rejecting a good lot, or in this case stating the residual level is above a specific AQL or value when it actually is not). The second point is the Consumer’s Risk Point made up of the Lot Tolerance Percent Defective (LTPD) and the consumer’s risk (Type II risk or Beta – which is the probability of accepting a bad lot, or in this case stating the residual level is below the LTPD when it is actually is not.)

The closer the AQL and LTPD are the more difficult (more samples) it is to determine an accurate estimate of the population. Likewise the less risk either the producer or consumer desire to incur again results in higher sample sizes.

One more consideration which is often overlooked is the selection of samples for testing. Most sampling plans are based on the assumption that the samples are taken randomly from the entire population. For example with say 50 million apples of a specific variety we would create a system to select samples that each has an equal change of any specific apply being selected. This is not a trivial matter in most cases. The availability and distribution of apples along with storage, shipping and display of apples all contribute to limited or biasing selecting a random sample. If it is not possible to select test items randomly, then study the impact on the study and means to account for a non-random sample.

In summary, for any sample plan:

  • Define the population
  • Define the desire goal of the study
  • Understand the measurement system
  • Use variables data if at all possible
  • Define PRP and CRP
  • Determine capable sampling plan
  • Design method to select random sample

This quick summary of consideration is what I consider the essential elements, yet other may impact the sampling plan. For example, seasonal variations in production, location in supply chain when measurements are made, variations in supply chain impact on presence of residual, differing nature of residue commonly found of different fruit or vegetables, and probably a few more. Understanding the goal, measurement system and random sampling will help determine areas that require consideration.

Cheers,

Fred

 

 

Combating contamination

Q: We want to ensure that we are receiving clean containers to package our products. How can we improve our incoming inspection process?

A: You should encourage your vendor to ship only clean containers. Then, be sure that the shipping and receiving process doesn’t cause contamination. If you can determine the source or sources of the contamination, the best fix is to remove the cause.

If that approach is not possible and you have incoming containers that may have some contamination, then consider the following elements in creating an efficient incoming inspection process.

1) How do you detect the contamination?

Apparently, you are able detect the container contamination prior to filling them, or are able to detect the effect of the contamination on the final product. Given that you are interested in creating an incoming test, let’s assume you have one or more ways to detect faulty units.

As you may already know, there are many ways to detect contamination. Some are faster than others, and some are non-destructive. Ideally, a quick non-destructive test would permit you to inspect every unit and to divert faulty units to a cleaning process. If the testing has to be destructive, then you’ll have to consider lot sampling of some sort.

There are many testing options. One is the optical inspection technique, which may find gross discoloration or large debris effectively. Avoid using human inspectors unless it’s only a short term solution, as we humans are pretty poor visual inspectors.

Another approach is using light to illuminate the contamination, such as a black light (UVA). Depending on the nature and properties of the contamination, you may be able to find a suitable light to quickly spot units with problems.

Another approach, which is more time consuming, is conducting a chemical swab or solution rinse and a chemical analysis to find evidence of contamination. If the contamination is volatile, you might be able to use air to “rinse” the unit and conduct the analysis. This chemical approach may require specialized equipment. Depending on how fast the testing occurs, this approach may or may not be suitable for 100 percent screening.

There may be other approaches for detecting the faulty units, yet without more information about the nature and variety of contamination, it’s difficult to make a recommendation. Ideally, a very fast, effective and non-destructive inspection method is preferred over a slow, error prone, and destructive approach. Cost is also a consideration, since any testing will increase the production costs. Finding the right balance around these considerations is highly dependent on the nature of the issue, cost of failure, and local resources.

2) How many units do you have to inspect?

Ideally, the sample size is zero as you would first find and eliminate the source of the problem. If that is not possible or practical, then 100 percent inspection using a quick, inexpensive, and effective method permits you to avoid uncertainties with sampling.

If the inspection method requires lot sampling, then all of the basic lot sampling guidelines apply. There are many references available that will assist you in the selection of an appropriate sampling plan based on your desired sampling risk tolerance levels.

Another consideration is the percentage of contaminated units per lot. If there is a consistent low failure rate per lot, then lot sampling may require relatively large amounts of tested units. You’ll have to determine the level of bad units permitted to pass through to production. Short of 100 percent sampling, it’s difficult (and expensive) to find very low percentages of “bad” units in a lot using destructive testing.

3) Work to remove original source(s) of contamination to permit you to stop inspections.

I stress this approach because it’s the most cost effective in nearly all cases. In my opinion, incoming inspection should be stopped as soon as possible since the process to create, ship and receive components should not introduce contamination and require incoming inspection to “sort” the good from the bad.

Fred Schenkelberg
Voting member of U.S. TAG to ISO/TC 56 on Reliability
Voting member of U.S. TAG to ISO/TC 69 on Applications of Statistical Methods
Reliability Engineering and Management Consultant
FMS Reliability
www.fmsreliability.com

Related Resources:

Digging for the Root Cause, Six Sigma Forum Magazine, open access

Many Six Sigma practitioners use the term “root cause” without a clear concept of its larger meaning, and similar situations occur in Six Sigma training programs. As a result, many practitioners overlook root causes. Read more.

The Bug and the Slurry: Bacterial Control in Aqueous Products, ASQ Knowledge Center Case Study, open access

When a customer reported a problem using the polycrystalline diamond (PCD) slurry supplied by Warren/Amplex, the company traced its product through the supply chain in order to identify the cause and quickly implement a solution. Read more.

Explore the ASQ Knowledge Center for more case studies, articles, benchmarking reports, and more.

Browse articles from ASQ magazines and journals here.