Considering Production Equivalence during Test
Guidelines for production equivalence from one industry applies to many
What is production equivalence? It’s a term used a lot in medical device manufacturing, but it applies to many industries. Having worked in the quality field for years, I think the principles and philosophes of quality and reliability engineering apply to a broad range of topics. The methods cross boundaries between service types (like hospitals and schools) or manufacturing industries, and also between production types. And by that, I mean you could be making medical devices, lawn chairs or lawn ornaments, clothing – similar or the same quality and reliability methods and philosophes would apply to all of these product types.
I mention this because I’m going to bring up a concept that’s well known and regulated in the medical device area, but the concept applies to all sorts of production. A lot of my career has been in manufacturing and in medical devices. In the United States, the FDA (Food and Drug Administration) is a regulating body for food, drugs, biologics, and devices. If you want to sell any of those things in the US, your business and products need to meet the regulations of the FDA. The FDA has specified their own regulations for quality systems for these industries, also known as the Current Good Manufacturing Practices (or CGMP). The Current Good Manufacturing Practices for medical devices first took effect in 1978 (that’s just to give you an idea about when this all came to be). Here is a link to the FDA’s CGMPs.
In the FDA’s CGMP, there is a lot to unpack about product design control procedures. Within the design cycle there are two types of tests. There is a design verification to ensure that design outputs meet the design inputs, like requirements and specifications. There’s also design validation, which is product testing done to ensure that the device meets user needs and intended uses. The CGMP touches on something called production equivalence. Here’s what the CGMP says: “Design validation shall be performed under defined operating conditions on initial production units, lots or batches, or their equivalents.” Generally, to meet all the CGMP regulations, medical device manufacturers also ensure that their products for design verification are also production or production equivalent.
Why understanding production equivalence is important before test
What is production equivalence and why is it important to be clear about what it is? Because when you’re in the middle of a project, managing activities and forging ahead toward a deadline, production equivalence can get a little muddy unless your careful. Manufacturers like to start creating parts in a manufacturing cell or in the final factory location as a limited run and there are lots of benefits to doing this. A couple of the benefits are tweaking your process in an environment where you can make changes quickly, and to really ensure that your product, as designed, is going to pass the test. So then, if we’re making parts and testing them, can’t they just count toward our design testing? Maybe, but maybe not. It depends if it’s production equivalent.
First let’s consider production. Understanding production is sort of easy. It’s the initial product units, lots or batches…it’s formal manufacturing. Some of us call it the production department. What would make something production equivalent? Sometimes, and in this case, I think it’s easier to think about what it’s not. To get there, let’s more fully explore what production means from four different areas: process, people, places, and things.
In a production run, manufacturers make products using a process that’s defined. The methods of manufacturing are defined within set specifications and those are inspected and controlled. In production, the people involved are operators who have the skills to make the product or to run that equipment, or they have the right level of training. A production-type place is an environment that’s comparable to where it’s going to be made in the long term. And the things used to make the product are tools and equipment that are available to a typical operator, not a specialized piece of equipment.
Examples and pitfalls of production equivalence using four parameters
Now thinking about what production really means from process, people, places and things, we can think about what production equivalence would need to look like. Let’s review some common pitfalls of production equivalence within these four parameters.
- Process: We may not have a production equivalent process if we’re assembling components that have just been sitting around. Are we sure of their ordering history or who manufactured them? Have they gone through an incoming inspection procedure? We may not have a production equivalent process if we are creating a device or component without a drawing or process document. We’re not inspecting our product when it’s done to make sure that it was made right and not asking ourselves, “Did we make the right thing?”
- People: We may have trouble claiming production equivalence if our operators were not properly trained or if our operators are over-qualified. For example, an engineer with ten years of experience soldering leads assembles a component. But, in production, it would really be an operator with two weeks of training experience and that’s not their normal job. In this case, we would not have production equivalence. The different techniques may produce products with a different level of quality.
- Places: We may have trouble with production equivalence if we put together our parts on a clean workbench, but in a factory, it’s going to be assembled in the same room as a powdered metal press. There have also been some issues where plastic injection molding was done in a clean room for prototype, but in the actual manufacturing facility the garage doors were open because the facility got too hot. The place of manufacture can matter in not just cleanliness, but also with temperature, humidity, and other environmental controls.
- Things: And finally, the things used to make the product: we may have trouble claiming production equivalence if the part tested was a printed 3D part, but the production part will be injection molded. The materials and failure mechanisms of the different manufacturing methods will produce different test results. Or maybe it was assembled by our team gently with a high precision tool on an engineering bench, but in production it will be lined up and snapped into place.
With these examples, we can envision how production or not production equivalent can affect our product tests. On the one hand, our prototype products might perform with better results than the production parts. That would be a problem because now we don’t truly know how the product is going to perform. Does its output meet the inputs? Can it perform as intended? On the other hand, our prototype products might perform worse, in which case we can’t move forward with the design. Perhaps the biggest reason is that we don’t know what we don’t know. Not using production or production equivalent parts introduces an unknown variable. Since we don’t know it, then we have no way or intention of controlling it and that introduces performance risk.
When to test prototype vs. production parts
Now all of this is not to say that we don’t test prototype parts. Yes, test them! Use them in preliminary usability engineering studies, try them on the engineering bench. Test to see if their performance is a factor of safety above how they need to perform. Use them in a fixture-like capacity to test other features. For example, test the electrical continuity of your system while using a 3D printed housing. But prototypes can’t stand-in for production-type parts when we’re doing our final checks to make sure everything performs as we want it to.
In conclusion, production equivalence is that middle ground between prototype and full production. But we need to use it strategically and in a smart way, and we need to be able to justify our decision when using it, which is fair. And I think that goes across any design process in any field. It’s a general good manufacturing practice.
Dianna Deeney is president of Deeney Enterprises, LLC and founder of Quality during Design. She has worked in manufacturing for over 25 years, developing an engineering career from the manufacturing floor to manufacturing process engineering, product design, quality and reliability engineering, and quality management systems. Through her experience, she recognized a gap in understanding between people that design products and the quality techniques and input from quality professionals. She knows that bridging this gap could make a significant, positive difference in the outcome of the design and its design cycle. It can also save on total costs by solving design questions earlier.
Dianna founded Quality during Design with a mission of using the company as a communication tool, to bridge product managers and designers (entry-level to seasoned) to the world’s quality initiatives and quality-minded people. Her vision is a world of products that are easy to use, dependable, and safe – possible by strategically using Quality during Design for products others love, for less. Visit her at www.QualityDuringDesign.com.