Defining Qualification, Verification, and Validation

Q: I understand the hierarchy, but I would be hard pressed, if asked, to give a clear definition of the terms: qualification, verification, and validation. Can one of the experts help explain these terms? Thank you.

A: This is a great question and I hope I’ll be able to help you.

To begin, I refer you to ISO 9000:2005 Quality management systems – Fundamentals and vocabulary.  As you may already know, this document is used to define/describe many terms used in the ISO 9000 series, including the three words you question.

In 9000:2005, under clause 3.8 Terms relating to examination, we find:

3.8.4 verification
Confirmation, through the provision of objective evidence, that specified requirements have been fulfilled
NOTE 1  The term “verified” is used to designate the corresponding status.
NOTE 2  Confirmation can comprise activities such as
–          performing alternative calculations,
–          comparing a new design specification with a similar proven design specification,
–          undertaking tests and demonstrations, and
–          reviewing documents prior to issue.

3.8.5 validation
Confirmation, through the provision of objective evidence, that the specified requirements for a specific intended use or application have been fulfilled
NOTE 1 The term “validated” is used to designate the corresponding status.
NOTE 2 The use conditions for validation can be real or simulated.

Validation definition, as provided by ASQ's Quality Glossary.

3.8.6 qualification process
Process to demonstrate the ability fulfill specified requirements
NOTE 1 The term “qualified” is used to designate the corresponding status.
NOTE 2 Qualification can concern person, products, processes or systems.
EXAMPLE  Auditor qualification process, material qualification process.

I’ll try to expand on these definitions in hopes of making things a bit more clear.  Keep in mind that qualification, verification, and validation are individual processes, but the explanations below (from Boston Scientific) should help you recognize their individuality as well as their interdependence.

Validation is an act, process, or instance to support or collaborate something on a sound authoritative basis.

Verification is the act or process of establishing the truth or reality of something.

Qualification is an act or process to assure something complies with some condition, standard, or specific requirements.

For example:

A design verification verifies that a frozen (static) design meets top level product specifications.

A process validation validates that the on-going (dynamic) manufacturing process produces product that meets product/print specifications and consist of installation qualifications, operational qualifications, process performance qualifications, a product performance qualification and perhaps process verifications.

An installation qualification qualifies that equipment was installed correctly and are a subset of a process validation (or possibly a test method validation).

Validation Examples:
•         Design validation, sterilization validation, test method validation, software validation, and process validation.

Verification Examples:
•         Design verification and process verification.

Qualification Examples:
•         Installation qualification, operational qualification, process performance qualification, product performance qualification, and supplied material qualification.

After reading all of this, I am confident you would be able to explain qualification.  An old and trusty phrase to help summarize the other two is: Validation – Are we producing the right product?; Verification – Are we producing the product right?

Bud Salsbury
ASQ Senior Member, CQT, CQI

Ask A Librarian

ISO 9001 & Time to Retrieve Records

Q: I am looking for an interpretation for ISO 9001:2008 Quality management systems–Requirements, clause 4.2.4 Control of records: “Records shall remain legible, readily identifiable and retrievable.”

What is considered readily retrievable (i.e., 24 hrs, 48 hrs, 8 hrs, 1 hr)? I have a customer who thinks traceability records should be available within an hour of a request. I interpret readily as 24 hrs. The current ISO and TS specifications do not indicate a time, so a reasonable time to me is 24 hrs to pull the information together.

In addition, the customer’s supplier requirements also do not have any specified time for document retrieval. I did contact our third party registrar auditor and he indicated that 24 hrs would be considered readily retrievable as long as there were no customer specific requirements.

A: There appears to be some confusion between records being “readily retrievable” vs. a customer’s request for the delivery of copies of records.  These are two separate issues.

The first issue:  What is meant by “readily retrievable?”  ISO 9001 does not prescribe any specific timeline or define the term “readily retrievable.”  However, the intent of this requirement is to ensure that objective evidence is available to provide proof of conformance or evidence that requirements have been met.  If the organization is unable to provide records upon request during an audit, the auditor will very likely document this as a nonconforming condition. Records must be available upon demand.

The second issue is response time to customer requests for records.  Although records or evidence of conformance may be “readily retrievable” within the organization,  the response time needed for an organization to provide copies of records to a customer may vary based upon the organization’s work load and availability of resources.   So, it may take an organization an hour, a day or a week to deliver copies of records to a customer.  In the event that the timely delivery of records is critical, requirements for the delivery of records should be stated in a contract or in a PO to provide a timeline or a delivery schedule.  The delivery of copies of records or documents to customers is not addressed in ISO 9001, clause 4.2.4.

Bill Aston
ASQ Senior Member
Managing Director of Aston Technical Consulting Services
Kingwood, TX
www.astontechconsult.com

ISO 9001 and CMMI Certifications

 

Manufacturing, inspection, exclusions

Question

Our company is working toward certification to ISO 9001:2008 Quality management systems–Requirements and Capability Maturity Model Integration (CMMI) certifications.

I have studied  ISO 9001 and mapped it to CMMI goals and practices. It appears to me that some sections of ISO point to CMMI level 3 process areas and practices, e.g.:

  • Clause 5.6.1 Management review – General relates to organizational process areas
  • Clause 7.2.1 Determination of requirements related to the product is relative to requirements development, which is a level 3 process area
  • A large part of clause 7.3 Design and development maps to CMMI level 3 process areas

My question is:

Does an organization need to be at CMMI level 3 in order to be ISO 9001:2008 certified? I am not saying certified CMMI level 3, but capable of performing at CMMI level 3?

Thank you so much.

Answer

Although the guidelines contained in CMMI may help to prepare an organization toward ISO 9001 certification,  there are several major differences between CMMI and ISO 9001.

ISO 9001 is an internationally recognized standard for quality management systems.  While CMMI is a Carnegie Mellon University registered trade mark.

ISO 9001 has specific requirements for documented procedures for the control of documents, control of records, control of nonconforming products, internal audits, corrective actions and preventive actions.  In addition, a quality policy, measurable objectives, and management reviews are required.

CMMI is focused on process improvement, while ISO 9001 focuses on customer satisfaction, process improvement, product conformity and the continual improvement of the quality management system.  An organization could be CMMI certified or “capable” as mentioned in the inquiry, but still be some distance way from readiness for ISO 9001 certification.

I hope this helps.

Bill Aston
ASQ Senior Member
Managing Director of Aston Technical Consulting Services
Kingwood, TX
www.astontechconsult.com

Here’s more information about ISO 9001.

OHSAS 18001 and ISO 9001 Work Environment Requirements

Workplace safety, OHSAS 18001, work environments

Q: We had the opportunity to get the certification for OHSAS 18001:2007 Occupational health and safety management systems — Requirements. While looking at the clause interaction between ISO 9001 Quality management systems–Requirements and OHSAS 18001 given at the end of the standard, I did not find any interaction between the standards for clause 6.4 work environment in ISO 9001.

Am I missing anything or is there any reason for it?

A: I am a U.S. Technical Expert for ISO 9001 and associated  quality management system (QMS) standards and have been involved with QMS standards since 1975.

In my opinion, the answer to your question is that the developers of OHSAS 18001:2007 did not feel that ISO 9001 clause 6.4 related to 18001. This, incidentally, I find puzzling.

The requirement in ISO 9001:2008 Quality management systems–Requirements clause 6.4 reads: The organization shall determine and manage the work environment needed to achieve conformity to product requirements.

In other words, you should make sure that your employees have an adequate work environment for producing your products. They should have adequate room temperature, lighting, and etc.

The 2005 report: Integrated Management Systems (IMS) – Potential Safety Benefits Achievable from Integrated Management of Safety, Health, Environment and Quality (SHE&Q) from Environment Directorate, Organisation For Economic Cooperation And Development, Paris, includes the following which might be of interest to you:

“OHSAS 18001 and National Standards

During drafting of the original BS 8800 a major division of opinion arose as to whether or not independent assessment and certification of an organisation’s OSHMS should be encouraged, as for QMS and EMS.  Some viewed such certificates as valuable, particularly in the context of effective supply chain management, others believed that existing certification processes: added minimal value, required excessive resources and resulted in unused manuals – so new certification processes should be resisted.  It proved impossible to reconcile these views within BS8800, which was structured and published as a non-certifiable standard.

As a result, an international consortium of certification bodies, including the commercial arm of BSI, produced the OHSAS 18001 specification in 1999, followed by implementation guidelines OHSAS 18002 in 2000.  Neither document is an official British Standard, but OHSAS 18001 either is, or is likely to become, a national standard in other countries, notably in Pacific Rim.  A recent survey by BSI identified that over 8000 OSHMS certificates have been issued in 70 countries, to many different standards and guidance, and that some 46% are to OHSAS 18001.

With the revision of BS 8800, from which it is derived, it might be presumed that OHSAS would be updated automatically.  A review is indeed planned, but the decision on when to publish a revision will take into account other factors, including the needs of current new users to have time to ‘bed down’ their internal processes before revising them to meet an improved standard.  When a revision is agreed, it is likely to include some alignment with other high-quality national standards such as AUS/NZ 4801, to aid recognition as a truly global standard.

A new US standard was published in 2005: ANSI/AIHA Z10 – Occupational Health and Safety Systems.  The format includes both a standard and associated guidance, but is not intended as a basis for certification.  It is fully compatible with ISO 9001/14001 and takes account of the other national/global OSHMS documents outlined in this section.”

OHSAS 18001:2007 is not an ISO standard. It appears to be simply an update of OHSAS 18001:2000. Its development was driven by the British Standards Institute which publishes the standard and profits directly from its distribution and sales.

Part of the answer to your question is to evaluate for yourself:

1) Why did you go to the expense to be certified to 18001:2007 and who were the customers that you were satisfying by doing this?

2) What is the expectation of these customers?

From a practical standpoint, consider embracing the concept in ISO 9001 clause 6.4. I would expect that providing your employees adequate conditions for producing products can only improve your product offerings and help to enhance customer satisfaction.

Joe Tsiakals
Voting member of the U.S. TAG to ISO/TC 176 (ASQ)
Voting member of the U.S. TAG to ISO/TC 210 (AAMI)

ISO 9001 Management Representative and Reporting Structure

Inspection, Management, Management Representative

Q: Please define the preferred method of meeting the requirements of the management representative in regards to clause 5.5.2.b in ISO 9001:2008 Quality management systems–Requirements.

My organization has reorganized, and I find the role of management representative somewhat detached. I work for a military organization that would like this role to be several layers below the base captain vs. the open door policy for the management representative used by previous commands. Should the management representative have direct access to the top?

A: There is no defined or preferred method for addressing the reporting arrangement for the management representative to top management. Your organization defines and deploys the approach that is workable for the management representative to report to top management.

Charlie Cianfrani
Consulting Engineer
Green Lane Quality Management Services
Green Lane, PA
ASQ Fellow; ASQ CQE, CRE, CQA, RABQSA Certified QMS-Auditor (Q3558)
ASQ Quality Press Author

For more information about management representatives, view the information here.

ISO 9001 Clause 7.4.1, Supplier Control

Mr. Pareto Head and Supply Chain comic strip

Q: My interpretation of  ISO 9001:2008 Quality management systems–Requirements regarding supplier control as addressed in clause 7.4.1 Purchasing process is that suppliers who would require evaluation, selection and registry, would be those who supply products (or services) which affect subsequent product realization, or the final product.

Excellent examples for our organization would be vendors providing raw material, tool/dies, surface preparation or calibration services.

I also believe that the “extent of control” exercised by the organization, could, in fact, mean that certain suppliers are not controlled (evaluated, selected and registered), due to their lack of impact on product realization.

Good examples here would be stationery or sanitation supplies.

After conferring with several colleagues, we are all puzzled to see freight companies (UPS, FedEx) included as controlled suppliers and nonconformance reports written for failure to comply with the standard if they are not included on our approved suppliers list.

I understand the standard is written to provide a framework, and not examples, however I find this interpretation to be too broad for the intended purpose.

A: Thank you for contacting ASQ’s Ask the Experts program.  The intent of ISO 9001:2008, clause 7.4.1 is to ensure suppliers are selected based upon their ability to meet the organization’s requirements, which generally include quality and delivery of product or service intended for the customer.

As you mentioned, suppliers of office supplies such as paper, printer toner and etc. are not usually included on an approved suppliers list since they have zero impact on the organization’s ability to meet customer requirements.

However, some registrars may consider trucking firms or delivery services such as UPS and FedEx as suppliers of services that could impact an organization’s  ability to meet requirements, such as on time delivery and the delivery of product in an acceptable condition to the customer.

Most registrars welcome rebuttals from their clients regarding audit findings.  This could be an excellent opportunity for your company state its position to the registrar and to understand their rationale as to why they believe UPS and FedEx must be on the approved suppliers list.

The bottom line is that your registrar determines how its auditors interpret audit criteria such as clause 7.4.1.

If it is decided to add these companies to the approved supplier list, it should be a painless process since your company probably already has an established performance history for them.

I hope this helps!

Bill Aston
ASQ Senior Member
Managing Director of Aston Technical Consulting Services
Kingwood, TX
www.astontechconsult.com

Does ISO 9001 Clause 7 Apply to Processes?

Manufacturing, inspection, exclusions

Q: Does clause 7 Product Realization in ISO 9001:2008 Quality management systems–Requirements apply to the design and development of manufacturing processes?

We have four facilities that are ISO 9001 certified under one certificate. One location designs the product, and the other facilities manufacture it. In the “design facility” we follow the requirements of clause 7. In the manufacturing facilities, we currently do not apply clause 7 for the process of developing the manufacturing processes.

A: ISO 9001 clause 7.3 is applicable to the design and development characteristics of a product.

ISO 9001:2008 clause 7.1 (Planning of Product Realization) and its reference to clause 4.1 (General Requirements) is more specific to product planning to ensure that the product quality objectives and the processes/resources are available to produce a product that will meet defined quality requirements as specified during design and development in clause 7.3.

Clause 7.1 requires that the planning process include identification of the inter-related processes (i.e., monitoring, inspection, product quality objectives, testing, records of conformity needed to verify the product requirements have been achieved.

The bottom line:  the product characteristics, quality objectives and inter-related processes must be documented.  If this is not fully achieved in the design and development process (clause 7.3), it must be included in the product planning process (clause 7.1). Please see clause 4.1.

Please keep in mind that your company’s ISO registrar will require evidence of conformity (records/documentation) to verify the requirements of clauses 4.1, 7.1 and 7.3 have been met.

Bill Aston
ASQ Senior Member
Managing Director of Aston Technical Consulting Services
Kingwood, TX
www.astontechconsult.com

ISO 17025 Certified Facility

ISO/IEC 17025:2017 General requirements for the competence of testing and calibration laboratories

Q: We have a specification that states test reports shall be from an facility certified to ISO 9001:2008 Quality management systems–Requirements. Our test reports are from a facility certified to ISO/IEC 17025-2005: General requirements for the competence of testing and calibration laboratories.

Isn’t ISO 17025:2005 under the ISO 9001:2008 umbrella?

A: Your interpretation is, indeed, correct. Actually, for a testing lab, accreditation to ISO/IEC 17025 is superior to registration to ISO 9001! As you know, your accreditation agency actually observed your personnel performing tests. They had to demonstrate competency. This was in addition to the verification that you had a working management system in place (that’s why they call it accreditation and not registration. We won’t even get into the misuse of the word certification).

To make sure your customer gets the assurance they want, I recommend you contact your accreditation agency. Ask them for a letter that states this equivalency. That will probably blow your customer away – or at least amaze them! Unless you can show the text you provided to ASQ was from one of the ISO or ANSI standards-writing committees, as an official interpretation, it probably holds little weight.

Your customer is right to monitor your performance this way. Recent food safety issues, prominent in the news, have a common element to them — insufficient attention to supplier performance. Expect to see more of this as the manufacturers and distributors pay more attention to their supply chain. I expect you are or will be doing the same for your critical sub-suppliers. Remember too, there are many ways to monitor supplier performance. Registration/accreditation is one of the ways.

Dennis Arter
ASQ Fellow
The Audit Guy
Columbia Audit Resources
Kennewick, WA
http://auditguy.net

Framework to Integrate ISO Standards and Non-ISO Standards

Reviewing confidential files, training records, human resources files

Q: I have a few questions about integrating standards for one of the experts:

1) Will registrars (in addition to BSI, who wrote it) accept a documented quality management system organized around the framework suggested in PAS 99:2006 – Specification of common management system requirements as a framework for integration, given there is adequate audit evidence that the requirements of both of the integrated standards have been addressed and have been implemented?

2) Is PAS 99 only for ISO-related standards, e.g.,  ISO 9001:2008 Quality management systems–Requirements and  ISO 14001-2004: Environmental management systems – Requirements with guidance for use, or can other combinations be made – e.g., ISO 9001 and American Institute of Steel Construction-Bridge and Highway AISCQC028?

AISCQC028 is not an ISO or ISO sector-specific standard, although the framework and structure is very similar. The AISC has its own certification body (registrar) and would insist that their auditors conduct a certification audit even though an organization has been previously ISO registered. AISC does not object to an integrated system that integrates/combines ISO 9001 with one of their certification standards as long as AISC certification requirements have been addressed.

The integration of ISO 9001 and 14001 is becoming common place and I’m fairly certain that PAS 99 is an acceptable format in those cases. I’m more interested in other industry standards and requirements not generally considered ISO-related that are being demanded by certain customer segments and integrating them in a system that must also be acceptable to ISO registrars because of other customer segments who are demanding ISO registration by their suppliers.

A: This is an excellent, and timely, question.

More and more organizations are developing integrated management systems based on multiple specification standards – such as ISO 9001, ISO 14001 and OHSAS 18001.   In addition, there are more and more management system standards being developed.  This includes both ISO standards and non-ISO standards – such as OHSAS 18001, Responsible Recycling (R2) and, based on your question, AISCQC028.

It is not even clear how many different management system specification standards there are. What one individual considers a guidance document; someone else insists is a specification standard suitable for certification.

So when you are developing documentation for an integrated management system, how should it be organized?

There are several options:

•    One option is to choose one of the standards as the primary high-level structure – say, ISO 9001:2008 – and address the requirements of the other standards within that structure.

•    PAS 99:2006 offers a different option for a high-level framework for organizing the management system documentation for an integrated management system.  (As you correctly point out in your question, PAS 99 cannot be used as a replacement specification standard for any of the discipline-specific management system standards.)

•    Another option is to establish a high-level structure that makes sense for your organization.

There is no required framework for organizing management system documentation.  You can use whichever overall structure and numbering scheme works for your organization.

ISO has recognized that having different high-level structures for its various management system standards may be problematic for organizations that are implementing integrated management systems that are intended to meet the requirements of multiple specification standards.  As a result, in February 2012, the ISO Technical Management Board (TMB) approved a guide for ISO standard writers that specifies a common structure and definitions to be used for all new and future revisions of ISO management system standards.  This was circulated as ISO Guide 83. This action by ISO highlights the primary issue with using PAS 99:2006.  It is out-of-date.

First, the normative references listed in PAS 99:2006 are not the current versions for some of the standards (notably ISO 9001 and OHSAS 18001).  Second, the high-level structure set out in PAS 99:2006 is not consistent with the common structure recently approved by ISO.

The key to establishing an integrated management is NOT the use of a particular organizing framework or high-level structure.  How you organize your management system documentation needs to fit the needs of your organization – not the desires of a particular registration auditor.

What is important is being able to clearly explain how your management system meets the requirements of each of the specification standards to which you want to become certified.  This requires clearly written documentation that defines the links to the requirements you are addressing within your management system.  It may also require discussion with your registrar and/or the use of reference tables – similar to those set out in the Annexes of ISO 9001, ISO 14001, OHSAS 18001 – and PAS 99:2006.

Thea Dunmire, JD, CIH, CSP
ENLAR Compliance Services, Inc.
Thea’s Blogs:
http://www.OHSAS18001expert.com
http://www.managementsystemexpert.com

ISO 9001 7.1 Product Realization

Suppliers, supplier management

Q: Our company, certified to ISO 9001:2008 Quality management systems–Requirements, is experiencing quite a bit of supplier non-conformance.

An option we are interested in is to have a set of manufacturing drawings for our suppliers and a set of inspection drawings. Either the manufacturing set would require tighter tolerance than the inspection set, or the inspection set would have looser tolerances than the manufacturing set.

What would the criteria be to introduce this theory into our procedures?

A: I suggest that you consider the point of ISO 9001:2008 Quality management systems–Requirements, clause 7.1, regarding the development of product objectives. Your planned approach is similar to what is intended by this clause.

The organization should set its product requirements during the product planning stage. Product requirements should be based upon design inputs, outputs, verifications and validations. This provides the essential measurements (tolerances) required for the product to function as designed or intended.

The requirements sent to suppliers, or to the shop floor, should be within the design tolerances or criteria, but not necessarily the same. However, in the event that inspection or a supplier identifies/provides a product that is outside the drawing requirements, it would be up to the engineer or the designer to decide if the product still meets the design criteria. If so, the product would be disposition “accept as is” and would still function as planned.

If repetitious non-conformance is encountered and the product is still within the design criteria, then changes to the supplier and inspection criteria should be considered to prevent continued non-conformance. If the non-conformance does not meet the product requirement or the design criteria, corrective action should be taken with the supplier.

Bill Aston
ASQ Senior Member
Managing Director of Aston Technical Consulting Services
Kingwood, TX
www.astontechconsult.com