Frame It: How to Scope High-Stakes Design Decisions Before You Test Anything

I just published Pierce the Design Fog, which focuses on the early-stage work: getting cross-functional teams aligned on customer needs and experiences and developing clear design inputs. That’s the front end: the fuzzy, collaborative work of defining WHAT to build.

But once you have design inputs and start execution, you face different uncertainty: ‘Will this technical approach work? How confident are we? What do we test first?’ Different kind of problem, different tools needed.

This three-month series addresses the back end: how to make high-stakes technical decisions once concepts are defined. Starting with: how to frame the problem before you test anything.

In my mechanical, quality, and reliability engineering work, I’ve watched hundreds of decisions. What the ones that succeed have in common is the team acknowledges what they don’t know, designs tests or gathers information to reduce specific uncertainties, and makes trade-offs based on updated confidence levels. The ones that fail? They skip straight to arguing about solutions before agreeing on the problem.

This month: how to scope high-stakes design decisions before we test anything. Let’s frame the problem so testing actually helps.

I joined a call with a potential client to talk about FMEA. They were adopting FMEA practices and asked to talk. They started to do FMEAs and had some hang-ups. FMEA (failure mode and effects analysis) is a tool to identify potential failures, surface the unknowns, and plan to reduce the uncertainty, the impact, or control the risk. Were they doing the right FMEA (design vs manufacturing process)? What was a failure, and what was a cause?

After talking about FMEA and what they had done, the frustrated lead engineer took over. He shared his drawings and talked about his specific use case. They had used a 3D printed part to test the viability of a design. The 3D printed part was the weak point. Now, how do they design the part to not be 3D printed? What should the material choice and manufacturing method be? Do they need to re-validate? We finally got to the root of the hang-up, and it wasn’t FMEA execution.

This is the trap: when the pressure is on, we either freeze in analysis paralysis or make a gut call and hope. Neither works for high-stakes decisions.

The team’s instinct was right. They needed risk analysis. But they were trying to use FMEA (a comprehensive tool for finding hidden risks) when they already knew the risk. What they needed was a way to frame the decision so they could systematically reduce the uncertainty.

They had a known unknown but were using the wrong tool.

Here’s the right tool: The Impact vs. Likelihood Matrix. It helps us assess the situation using risk-based thinking and decide what to do next.

To use it, you need to do four things:

Identify your unknown.
Assess the impact to the project.
Assign a likelihood that your current design approach is correct.
Compare the impact and likelihood to determine next steps.

These resulting matrix actions are general guidelines, not hard and fast rules. Once you’ve identified the known unknown, named the impact, and assigned a likelihood, you have the information needed to make better design decisions.

Identify your unknown

The first step is to ensure that your problem is a known unknown that you can affect with your decisions.

In product development, we have two types of unknowns: unknown unknowns and known unknowns.

Unknown unknowns are usually part of early phases of development. We don’t know what we don’t know. The risks are unclear and the design is ambiguous. This is where we apply risk tools that help us systematically identify unknown risks. Preliminary hazard analysis, FMEA, and the Symptom-Impact Model (from Pierce the Design Fog) are ways that we can evaluate the product scenarios and identify potential risks. This is an important first step in managing risks.

Other unknown unknowns are things like supplier bankruptcy, the competitor’s surprise launch, and changing regulations. You can’t test for these. Instead, you build organizational resilience. Acknowledge these other unknowns and push them into the appropriate system.

In later phases of development, we’re mostly dealing with known unknowns. Known unknowns are questions we can write down. “Will this bearing last 10,000 cycles?” “Will customer pay $50 more for this feature?” They pose the greatest threat to our success. We can test them, and we can further name them as a problem and define its scope.

To use the Impact vs. Likelihood Matrix, you must have a known unknown. Out of scope are unknown unknowns that you can handle differently.

Make a statement about your known unknown.

Now make a hypothesis. Put a stake in the ground. What is the problem, and what are you worried about?

Our unknown: Our new injection molded component will perform at least as well as the 3D printed part under field use stresses under specified use conditions.

Assess the impact to the project

The design engineer was under a lot of pressure. Timelines could be screwed if he made the wrong material choice or didn’t add the right ribs for strength. Budgets could explode if the part design was wrong and they needed another mold. Both time lines and costs could exceed the project limits if he had to do extensive retesting.

All of these could be impacts on a design decision if we’re wrong.

Let’s name them and get them out in the open. It may feel uncomfortable, but being honest about the situation is usually best. Not only will you “get it off your chest”, you’ll also be better able to engage the rest of the team to help handle this known unknown.

If only one person knows the impact of getting this wrong, you’ve got an organizational problem, not a technical one. Get your team involved in defining, and acknowledging, the impact of a decision if it’s the wrong one. Ensure you engage them in feedback loops and communication. Look for ways that politics may affect the impact.

Assign the likelihood that the right decision is made

We want to assess our current likelihood and confidence that our design decision is going to be right.

Confidence is a measure of our uncertainty. We’re rarely starting from zero confidence. We have our experience, lessons learned, research, and knowledge about the current project. All of this culminates in our confidence about a design decision.

Generally,

High likelihood means that we are confident that our current design or belief is correct. We have some initial evidence or a strong past precedent.
Low likelihood means we are NOT confident in the current belief/design decision. We have conflicting data, no data, or a high degree of technical uncertainty.

No matter what, you’ll need to assign a confidence in your current belief about a design decision and its impact.

That’s right. There’s no calculation. There’s no testing to define it (yet). You’re assigning it. It’s just that you’re using a number to describe it.

What is the likelihood that our new injection molded component will not perform at least as well as the 3D printed part, and cause all these negative impacts to our project?

If the idea of assigning a likelihood number makes you uncomfortable, try something like this:

I’m 40% confident that the performance life of our new injection molded components will be at least as good as the 3D printed part.

40% is low. Our next natural question is, “What can we do to improve this?” This is where we want to start angling our discussions.

Or, you can take it a step further and describe it like this:

I’m 90% confident that the performance life of our new injection molded components will be between 30% to 70% of the 3D printed part.

Just getting to that point is very telling about the situation. 30% to 70% of the performance life is not only a wide margin, but it’s also pretty low. It shows the level of uncertainty heading into this decision.

Do you notice, also, how it’s easier to assign a likelihood to something if we have it well defined? “Our new injection molded component will perform at least as well as the 3D printed part under field use stresses under specified use conditions.” Given that, we can assign a likelihood range to that statement, with a level of confidence.

Why assign a likelihood? It more easily communicates to others the level of heartburn related to this decision. It immediately paints a picture of where we are right now, given the information we have. And it is a baseline in following phases of making design decisions.

Compare the impact and likelihood to determine next steps.

We defined the design decision impact to the project. We assigned a likelihood and confidence that our decision is right. Now, we map it on our 2×2 matrix.

High impact and low likelihood is a Critical Unknown. Slow down and investigate. Prioritize data gathering to shift the likelihood to high.
High impact and high likelihood is a Calculated Certainty. Proceed with implementation but immediately focus on minimizing the downside. Define and monitor expected loss and build in technical fail-safes.
Low impact and high likelihood is an Easy Win. This is a low-risk decision. Delegate and move fast.
Low impact and low likelihood is probably a Resource Sinkhole. Evaluate if you really want to invest time or money to investigate. Either ship the simplest possible version, or de-scope it entirely to eliminate waste. Don’t over-engineer it: focus on quick, targeted testing and ensuring rapid reversibility.

The value in this process and tool is to flush out important details, set us up for a baseline of later decisions, be able to compare this problem with other problems for trade-off decisions, and get agreement and alignment with the project team.

Example: The 3D Printed Part Decision

The 3D printed component team’s instinct was right – to perform a risk analysis to help them decide. Let’s apply this risk-based tool to frame it.

Known Unknown: Will an injection-molded part perform as well as our 3D-printed prototype under field conditions?

Impact if Wrong: High

Timeline: 8-week delay for mold revision
Cost: $45K for new tooling
Risk: May need complete revalidation ($120K, 12 weeks)

Current Likelihood: Low, 40% confident it will perform adequately

Why so low? Material properties differ significantly
No stress testing on molded version yet
Geometry changes (ribs, draft angles) alter load paths

Matrix Position: High Impact, Low Likelihood = Critical Unknown

Next Action: Slow down. Design tests and gather data that move this from 40% to 80%+ confidence before cutting the mold.

In next month’s post, we’ll focus on the critical unknowns in Phase 2: How to design tests that actually move your confidence needle, and when to stop testing.