Could Null Hypothesis State Difference?

Q: Does a null hypothesis always state that there is no difference?  Could there be a null hypothesis that claims there is? 

In the U.S. legal system, the null hypothesis is that the accused is assumed innocent until proven guilty.  In another legal system, there might exist the possibility that the accused is assumed guilty until proven innocent.  In our system, a type 1 error would be to find an innocent man guilty.  What would be considered a type 1 error if the null hypothesis was assumed guilt?

A: Sir Ronald Fisher developed this basic principle more than 90 years ago. As you have correctly stated above, the process is assumed innocent until proven guilty. You must have evidence beyond reasonable doubt. An alpha error (type 1) is calling an innocent person guilty. Failure to prove guilt when a person really did commit a crime is a Beta error (type 2).

What can null hypothesis tell us?  Does the confidence interval include zero (or innocence in the court example)? Instead of asking, “can you assume guilt and prove innocence?” — turn the question around and ask “does the confidence interval include some value that is guilty?”

For example, let’s say a process has an unknown mean and standard deviation, but it has customer specifications from 8-12 millimeters. Your sample measures 14 millimeters. Clearly, your sample is guilty by customer specifications. We need to prove beyond reasonable doubt that the confidence interval of the process, at some risk level (alpha), does not include guilty material. This is done by measuring the process for control.  If it is in control and not meeting customer specifications, either move the distribution, reduce the variation (through Design of Experiments, or other methods), or through some combination of both.

If the new confidence interval does not include guilt, the argument would be that you have proven, beyond reasonable doubt, that the confidence interval does not include the out-of-spec material. Under this circumstance, a type 1 error (alpha error) would be a process  mean less than the upper specification, but the confidence interval included the specification.

For further reading on this material, refer to the following text:

Testing Statistical Hypothesis, E. L. Lehmann and Joseph Romano,  January 2005.

 Bill Hooper
ASQ Certified Six Sigma Master Black Belt
President, William Hooper Consulting Inc.
Naperville, IL


ISO 17025 Clause 5.4.2 – Selection of Methods

ISO/IEC 17025:2017 General requirements for the competence of testing and calibration laboratoriesQ: We are working with the Mexican Accreditation Entity (EMA) for certification to ISO/IEC 17025:2005 General requirements for the competence of testing and calibration laboratories. Clause 5.4.2 states: The laboratory shall confirm that it can properly operate standard methods before introducing the tests or calibrations.

We are a testing laboratory and work with Method 21 – Determination of Volatile Organic Compound, EPA 40 CFR Ch.1 ( 01/07/04 Edition ) Test: Monitoring of Fugitive Emissions.

The question is: What would be the best way or a way to confirm the method? Or, to put it another way, how can we satisfy the requirements in clause 5.4.2 ?

A: The questioner is referring to clause 5.4.2 from ISO/IEC 17025:2005. An excerpt of this clause is below. Please refer to ISO/IEC 17025:2005 for the full clause.

5.4.2 Selection of methods

“…Methods published in international, regional or national standards shall preferably be used….. Laboratory-developed methods or methods adopted by the laboratory may also be used if they are appropriate for the intended use and if they are validated…. The customer shall be informed as to the method chosen. The laboratory shall confirm that it can properly operate standard methods before introducing the tests or calibrations.…”

Since the questioner is using the published methods, there is no need for validation of the method unless the method is modified.

However, the proficiency of being able to apply the published method needs to be demonstrated. This can be demonstrated by a documented Gage R & R study, Analysis of Variance (ANOVA) or Design of Experiments (DOE) study as appropriate to show proficiency in being able to utilize the test method properly.

The results from these studies may also be used to estimate the uncertainty of measurement for the tests. Reporting uncertainty of measurement with both test and calibration results is a requirement in ISO/IEC 17025:2005. The ILAC P14 document is a good guidance document on reporting uncertainty.

Dilip A Shah
President, E = mc3 Solutions
Chair, ASQ Measurement Quality Division (2012-2013)
Secretary and Member of the A2LA Board of Directors (2006-2014)
Medina, Ohio

Design of experiments (DOE)

Q: I am looking for research articles or review papers on Design of experiments (DOE) specially focused on Response surface methods, Split Plot designs, MANOVA, and Repeated measures designs and analysis.  Any help to locate these articles will be greatly appreciated.

A: Thank you for contacting ASQ and the Quality Information Center. I received your request for articles on the topic of Design of experiments (DOE).  Design of experiments is defined as “a method for carrying out carefully planned experiments on a process.  By using a prescribed plan for the set of experiments and analyzing the data according to certain procedures, a great deal of information can be obtained from a minimum number of experiments” (from The Quality Toolbox, 2nd Ed. by Nancy R. Tague, Quality Press, 2005).

I searched the ASQ Knowledge Center for articles that focus on the specific areas you mentioned you were interested in.  I pulled some articles from my search results and have listed them below.  I have also included a link to the full search results so that you can take a look at all of the articles I found.

Response Surface Methods articles (Here is the link to the search results I received):

Split Plot Designs: (Full search results)

 MANOVA: (Full search results)

Repeated Measures designs and analysis: (Full search results)

I hope that this information is helpful.  Please let me know if you have any questions or if you need additional assistance.

Best regards,

ASQ Research Librarian
Milwaukee, WI