Cpk of 1,33 is not enough!

Updated: Oct 6

A deep dive into sample sizes for process validation


Are you questioning why a Cpk of 1,33 should not be enough, although it means only 66 ppm are defective? You're right, but a Cpk of 1,67 isn't enough either; neither is anything above that.

So, it seems that the number of defects is not the issue here, but what's the problem? Unfortunately, many suppliers and manufacturers we see in the medical device industry are not compliant with the regulations.

This post talks about:

  • Current regulatory requirements around the justification of sample sizes

  • How to come up with a valid rationale

  • Tolerance Interval Approach (k-value)

  • Confidence Statement

  • What is the difference between attribute and variable data?


What does the regulation say?

ISO 13485:2016, 7.5.6 – Validation of processes for production and service provision requires the organization in bullet point d) to, as appropriate, document statistical techniques with rationale for sample sizes [2]. Other standards such as the ISO 11607:2019 have similar requirements. ISO 11607:2019 Part 1, 4.3 – Sampling requires that sampling plans are based upon statistically valid rationales [3]. But not only do ISO standards require valid statistical rationale, but the U.S. FDA also has a similar requirement in TITLE 21 CFR 820.250 – Statistical techniques – and a very serious about it, as seen here.


What is a valid rationale?

The question remains to be answered of how to rationalize a sample size. The first and simple answer is if there are standards that require a specific sample size. Unfortunately, very few standards really call out specific sample plans. The second and more complicated answer can be found in the ISO 13485. ISO 13485:2016, 4.1.2 requires in bullet point b) that the organization shall apply a risk-based approach to the control of the appropriate processes needed for the quality management system. This is already the answer to our question. A valid rationale (sometimes referred to as justification) is based on the risk (or severity) and thus on the intended use of the product we are considering - easy, right?

Wait…you don't know how? This does not ring a bell, and you still have no clue how you should do it? That is not too surprising!


Tolerance Interval Approach

To explain the connection to risk management, we first need to look at the common approach used for the statistical interpretation of data. ISO 16269 is a standard that talks about statistical interpretation of data, part 6 of this standard explain the “Determination of statistical tolerance intervals”. Again, something you do not know what it is? Check out our blog post about statistical tolerance intervals. ISO 16269-6 provides k-values for the statistical tolerance interval analysis based on the type of distribution (attribute or variable), confidence levels (90%, 95%, 99%), probability (also reliability) levels (again 90%, 95%, 99%), and sample sizes.

According to ISO 14971, an organization needs to define its risk acceptability for both individual risks and overall residual risk [6]. While this definition connects certain probability (also reliability) levels with product risks, a minimum of 95% confidence must be used unless a justification is provided [1]. Now that we have a confidence level and a probability level (also reliability), we are done if we have attribute data. In the case of variable data, we only must decide on a sample size n to get the k-value. The table provided by ISO 16269-6 shows that the k-value decreases with an increasing sample size n [5].

After all this effort, we are back at the initial question: How many samples? But this time, we have much more than a number of samples to test. We have a confidence level and a probability level that allow us to make a confidence statement. The sample size is now less critical as the k-value increases with a smaller sample size n. Nevertheless, it is essential to have a representative sample.


Representative sample

A sample is representative when it represents the population being studied and can either be a random sample, stratified or periodic sample. In a random sample, each sample is equally likely to be selected. In a stratified sample, an equal number of units are selected from each collection tray, and every nth unit is selected in a periodic sample.

So, I assume you are still asking yourself, "Does that mean I can take only 3 samples?”. And the answer is: “It depends!”

Assuming we have a 95% confidence level and a 95% probability level, and we are assessing:

  • Attribute data (e.g., go/no-go); ISO 16269-6 tells us to take 59 samples, and all must pass [5]. To answer the above question: No, you cannot simply take 5 samples!

  • Variable data (e.g., 1.78mm); ISO 16269-6 gives us multiple options, and we must decide whether we choose, e.g., 3, 30, or 100 samples. 3 samples will most likely not be representative for an injection molding process to allow temperatures, pressures speeds, etc., to vary over their entire range, nor can you prove a normal distribution with 3 samples. To answer the above question: Again, it depends – Evaluate the risk of factors affecting the process, relate the average and standard deviation to the specifications, and do some math! It might be challenging to show that the data is normally distributed.


Confidence Statement

The confidence level (C) and probability level (P) allow us confidence statements like “With C% confidence, more than P% of units conform to requirements”.


What is the difference between attribute and variable data?

The distinction between attributive and variable data is essential but also quite simple.

  • Attribute Data: Go/No-Go, Pass/Fail, good/bad; the result is qualitative

  • Variable Data: continuous values like 1.78N, 2.45mm, 1.09bar; the result is quantitative

While fewer samples can be tested with variable data, attribute data is easy to document and quick to evaluate.

Are you concerned about the increased requirements due to MDR (Medical Device Regulation; 2017/745) and already behind schedule? Contact us today, and we'll take the burden off your shoulders and help you make your supply chain compliant.


References

[1] Taylor, Wayne (2017). Statistical Procedures for the Medical Device Industry. Taylor Enterprises, Inc., www.variation.com

[2] ISO 13485:2016 Medical devices — Quality management systems — Requirements for regulatory purposes

[3] ISO 11607-1:2019 Packaging for terminally sterilized medical devices — Part 1: Requirements for materials, sterile barrier systems and packaging systems

[4] TITLE 21 CFR820.250

[5] ISO 16269-6 Statistical interpretation of data — Part 6: Determination of statistical tolerance intervals

[6] ISO 14971:2019 Medical devices — Application of risk management to medical device

148 views0 comments