Failure modes are part of the design

Context

Failure modes are not bugs you catch later. They are the assumptions a design quietly depends on, and the ways those assumptions can stop being true. A design is not finished when it works on the cases you thought of; it is finished when you have named the cases where it would not work, and decided what to do about each one.

Treating failure modes as a first-class part of the design, rather than something the QA step will find, is what separates a system you can defend from one that only looks good on a slide.

Pharma teams already do this work; they often call it something else. The applicability domain of a model, the qualification report for a PBPK platform, the contraindications and “use with caution” lines on a label, and the post-marketing surveillance plan are all the same idea: name the conditions under which the answer is no longer trusted, and decide what to do about each one. This note uses engineering language because the pattern is general, but the pharma vocabulary is interchangeable.

What is a failure mode?

A failure mode is the specific way a system breaks when an assumption stops holding. A non-technical example: a delivery route optimizer picks the fastest path across town because it assumes every road on the map is open. The failure mode is not a bug in the algorithm; the algorithm did exactly what it was asked. The failure mode is the gap between the model of the world and the world itself.

The pharmacology equivalent is straightforward: a dose recommendation is “correct” inside the model’s applicability domain and quietly wrong outside it. Same shape, different vocabulary.

The route is still “optimal” according to the model, and completely useless in practice. Every engineered system has a list like this. The useful question is whether the list is written down.

How to think through how systems break

A simple frame holds up across very different systems. First, list the assumptions the design depends on, not the equations, the assumptions. Linear behavior in some range. A parameter estimated from a particular population. A sensor that reports at a particular rate. A user who reads the warning. Second, for each assumption, ask what happens if it is wrong, by how much, and in which direction. Third, trace the consequence one or two steps forward and ask how you would know before it propagates.

Most of the value is in the first step. Once the assumptions are written down, the failure modes mostly write themselves.

In pharma language: step one is writing down the model’s applicability domain (population, covariates, dose range, formulation, co-medications considered). Step two is sensitivity and what-if analysis on the assumptions most likely to be wrong in the real patient. Step three is the monitoring and surveillance plan, the lab values, the contraindications, the post-marketing study, that would catch the failure before it reaches a patient.

Example: dose optimization from a PK model

A pharmacokinetic (PK) model describes how a drug moves through the body over time, relating dose to exposure metrics like AUC and Cmax. A common use is to optimize a dosing regimen: pick the dose and schedule that keeps exposure inside a therapeutic window.

Suppose a team builds such a model, fits it to healthy-volunteer data, and runs an optimizer that maximizes time-in-window subject to a peak-concentration cap. The optimizer happily suggests a dose slightly above the standard regimen. On paper, the AUC and Cmax both look clean.

The healthy-volunteer fit implies a roughly linear relationship between dose and clearance through CYP3A4. In a patient with reduced CYP3A4 activity, a known genotype, a co-administered inhibitor like a common antifungal, or simply an older liver, clearance drops, and it drops non-linearly. The same “optimal” dose now produces a peak concentration well above the cap the optimizer was told to respect.

The optimizer did its job. The model just never saw that patient.

A clinical pharmacologist would phrase the same thing as: the proposed regimen sits outside the model’s qualified applicability domain. The math is identical; the word “failure mode” just makes the engineering origin explicit.

Why this is a design problem, not a modeling accident

It is tempting to file this under “the model needs more data” and move on. That misses the point. The failure mode was predictable from the assumptions the moment the model was built: a single-population fit, no inhibitor term, no covariate for hepatic function. Each of those choices was reasonable in isolation; together they defined the boundary of where the optimizer’s answer can be trusted.

The design failure is not that the boundary exists. It is that the boundary was not part of the deliverable. An honest version of the same work ships the dose recommendation and the conditions under which it stops being valid, in the same document.

In regulated work, this is what a good label section or a clear “limitations of the analysis” paragraph in a modeling and simulation report is supposed to do, ship the answer and its boundary in the same document.

Review steps that actually help

A few practices tend to earn their keep:

• Sensitivity analysis on the assumptions you suspect most, not all parameters equally, but the ones whose plausible range moves the answer.
• Boundary-condition tests: rerun the optimization for the elderly patient, the impaired patient, the patient on the obvious co-medication, and see whether the recommendation survives.
• A short, living “known issues” list tied to the model version, so the next person inherits the boundary rather than rediscovering it.
• A human plausibility check between the optimizer’s output and any decision it drives.
• The discipline to separate model failure from optimizer failure when something goes wrong, because the fix is different in each case.
• Treat model qualification, applicability domain documentation, and the limitations section of the M&S report as part of the deliverable, not a separate compliance artifact.
• Tie the boundary to clinical action: contraindications, dose adjustments in special populations, therapeutic drug monitoring triggers, and pharmacovigilance signals that would surface the failure mode in practice.

None of these are exotic. They just need to be planned for, not bolted on after the first surprise.