You may be thinking, "OMG, what more do we need to validate? Is this still not enough?"
Well, no, it isn't.
Fortunately, we really need to validate Test Methods, not only because they provide us with another level of confidence but because they are mandated by regulations.
Yes, you read correctly – Test Method Validation (TMV) is mandated by regulations.
This post talks about:
What is Test Method Validation for Medical Devices?
What is the Basic Idea of Test Method Validation (TMV)?
Why Do We Have to Perform Test Method Validation (TMV)?
What Types of Test Method Validations Do We Have to Distinguish?
Where Do Measurement Errors Come from?
What is Test Method Validation for Medical Devices?
Validation of a test method provides a thorough understanding of the uncertainty of the method used.
A validated test method ensures that the method is appropriate and that the data generated are reliable and repeatable.
Most of the information and data we collect during process validation, monitoring, or other measuring activities depend on the measurement process.
If the measuring process varies, the results we get from it will do so too.
The quality of the measured values is determined by many values measured using a measuring system that operates under stable conditions [3].
Suppose a measurement system records multiple measurements of a specific characteristic, and all are close to the target value of the characteristic. In that case, the data quality is said to be high and vice versa [3].
Since we want to obtain high-quality data, validation of the test method is even more helpful.
What is the Basic Idea of Test Method Validation (TMV)?
Imagine that the diameter of a rod is to be measured with a caliper.
If you have three people measure it, you will get three different results.
There are multiple reasons for deviating results; Ranging from the force applied to the caliper, the position of the measurement (the rod is most likely not ideally round), the number of digits on the caliper, and many other reasons.
To assess these effects for their importance in the deviation from the actual result, we need to understand what problems we have and where they come from.
With Test Method Validation, we try to achieve exactly that.
We want a Test Method that gives us the same result regardless of who measures the sample and when.
Why Do We Have to Perform Test Method Validation (TMV)?
The requirement to perform Test Method Validation can be found throughout various requirements.
The US FDA requires under 21 CFR Part 820.72 a) Control of inspection, measuring, and test equipment:
Each manufacturer shall ensure that all inspection, measuring, and test equipment, including mechanical, automated, or electronic inspection and test equipment, is suitable for its intended purposes and is capable of producing valid results [5].
The keywords relevant to us are "…test equipment, is suitable for its intended purposes…" and with the definition of validation under 21 CFR Part 820.3 z) Validation means confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use can be consistently fulfilled – we have a requirement for Test Method Validation [5].
ISO 13485 requires under 7.4.3:
The organization shall establish and implement the inspection or other activities necessary for ensuring that purchased product meets specified purchasing requirements.
For those arguing that it is difficult to interpret the requirement for Test Method Validation here – how do you ensure you do not have false positive/negative results?
Other Documents like the MDSAP audit approach even state:
(...) the medical device organization may need to validate the test method used for incoming acceptance to ensure the test method is actually capable of identifying nonconforming product [6].
We often also hear statements like: "We follow a standardized method. We do not need to validate!"
Look into ISO 11607-2:2019, 4.4.3:
All test methods … shall be validated … by the laboratory performing the test [4].
Understandably, regulations "motivate" some organizations, yet we should not forget the benefits of Test Method Validation, as mentioned in the previous section.
What Types of Test Method Validations Do We Have to Distinguish?
When we want to perform a Test Method Validation, we first must distinguish what kind of data our measurement system can provide.
We already talked about attribute and variable data in our “Cpk of 1,33 is not enough!” blog post.
Let’s quickly repeat the most essential things:
Attribute Data: Go/No-Go, Pass/Fail, good/bad; the result is qualitative.
Variable Data: Continuous values like 1.78N, 2.45mm, 1.09bar; the result is quantitative.
The following distinction we must make is whether the test is destructive or non-destructive.
Based on these two questions, we get the following decision tree:
Figure 1 TMV Decision Tree
Where Do Measurement Errors Come from?
You might have come across the term Gage (or Gauge) R&R (Repeatability and Reproducibility); if not, check out our blog post about Gage R&R (with Minitab)
Repeatability is the range of variation resulting from repeated measurements of the same unit by the same person, on the same day, and with the same instrument – it is the best the test method can do [2].
On the other hand, reproducibility is the variation that arises due to operators, days, setups, calibrations, environmental conditions, instrument conditions, and differences between instruments.
Repeatability and Reproducibility together are referred to as Precision [2].
Another source of measurement is called bias or lack of accuracy.
Accuracy refers to getting the correct values, as opposed to getting the same value.
However, bias is controlled by calibration of the instrument with its clear requirements by the US FDA (21CFR820.72) and ISO 13485:2016 (7.6).
This blog provides an overview of the topic of Test Method Validation.
Check out our blog posts on how to perform different kinds of Test Method Validation:
We discuss how to conduct a Gage R&R (with Minitab) study, Destructive Test Methods and Pass/Fail Test Systems (Attribute Agreement Analysis) systems.
Author: Simon Föger
Are you concerned about the increased requirements due to MDR (Medical Device Regulation; 2017/745) and already behind schedule?
Contact us today, and we'll take the burden off your shoulders and help you make your supply chain compliant.
PS: Check out our SIFo Medical YouTube channel if you want to learn more about Quality Management in MedTech.
References
[1] ISO 13485:2016 Medical devices — Quality management systems — Requirements for regulatory purposes
[2] Taylor, Wayne (2017). Statistical Procedures for the Medical Device Industry. Taylor
Enterprises, Inc., www.variation.com
[3] Measurement System Analysis – 4th Edition (2010), Automotive Industry Action Group
[4] ISO 11607-2:2019 Packaging for terminally sterilized medical devices — Part 2: Validation requirements for forming, sealing and assembly processes
[5] TITLE 21 CFR820.72
[6] MDSAP Audit Approach https://www.fda.gov/media/157947/download