Optimization is not just maximizing a number

The hard part often comes before the optimization

When people hear “optimization,” it is natural to think about maximizing profit, minimizing cost, finding the best design, or tuning a model until a metric improves. That is part of it, but it can make optimization sound simpler than it is.

In practice, many useful optimization problems are not hard because the math is impossible. They are hard because the objective is unclear, the constraints are incomplete, the tradeoffs are hidden, or the answer is highly sensitive to assumptions.

Before asking a system to maximize or minimize something, we have to decide what the “something” means.

The objective function is a choice

An objective function is the thing the optimization process is trying to maximize or minimize. But the objective function is not handed down from nowhere. Someone chooses it.

Common choices include:

• minimize cost
• maximize yield
• maximize likelihood
• minimize error
• minimize time
• maximize expected return
• minimize energy use
• maximize throughput
• minimize risk
• maximize product fit
• balance multiple competing goals

The objective function shapes the answer. If the objective is poorly chosen, the system may optimize perfectly and still produce a bad decision.

If a manufacturing process only optimizes throughput, it may ignore quality or reliability. If a financial model only optimizes revenue growth, it may ignore cash burn. If a machine learning model only optimizes prediction accuracy, it may ignore interpretability, fairness, or deployment cost.

Optimization is usually about tradeoffs

Real optimization problems rarely involve one clean goal. They involve competing priorities.

• speed vs. accuracy
• cost vs. reliability
• performance vs. manufacturability
• sensitivity vs. specificity
• yield vs. purity
• complexity vs. maintainability
• model fit vs. generalizability
• automation vs. human review
• short-term efficiency vs. long-term robustness

When someone says they want the “best” answer, the next question should be: best according to what tradeoff?

The word “optimal” can hide a value judgment. An optimal answer is not universally optimal. It is optimal relative to a selected objective, constraints, weights, and assumptions.

Constraints define the feasible world

Constraints describe what the solution is allowed to do. In engineering and technical systems, constraints can be just as important as the objective.

• budget
• time
• material limits
• physical laws
• safety limits
• manufacturing capability
• regulatory requirements
• data availability
• compute resources
• integration requirements
• acceptable error
• human review capacity

Without constraints, an optimization problem can produce answers that look mathematically good but are practically useless.

Local optimum vs. global optimum

A local optimum is the best answer nearby. A global optimum is the best answer across the entire search space.

A useful way to think about this is to imagine walking through hills and valleys in the fog. You may find the lowest valley around you, but another valley somewhere else may be lower. That nearby valley is a local minimum. The lowest valley anywhere is the global minimum.

Many systems can get stuck in local optima:

• model training
• process tuning
• business decisions
• engineering design
• product workflows
• career decisions
• financial assumptions

Sometimes the problem is not that the system cannot optimize. The problem is that it optimizes from a limited starting point or within a narrow search space.

Global optimization asks whether there may be a better solution somewhere else in the search space. But global search is often more expensive, uncertain, or computationally difficult. In practice, people often combine local search, heuristics, restarts, sensitivity analysis, and judgment.

Where log-likelihood fits

Maximizing log-likelihood is a specific example of optimization used in statistics and machine learning. The goal is to find model parameters that make the observed data most probable under the model.

If a model has parameters, likelihood asks: given these parameters, how likely is the data we observed? Maximizing likelihood means choosing the parameters that make the observed data most plausible.

Log-likelihood is often easier to work with mathematically and computationally because products of probabilities become sums of log probabilities. Maximizing likelihood and maximizing log-likelihood lead to the same parameter values because the logarithm is monotonic.

But this is still only one kind of objective function.

Even in likelihood-based modeling, the result depends on the model structure, assumptions, data quality, parameterization, and whether the model is appropriate for the question. A model can have a high likelihood and still be a poor decision tool if it ignores the variables that matter for the real-world system.

Simple models are often the first optimization tool

Before building a complex optimizer, it is often useful to build a simple model that clarifies the relationship between inputs and outputs.

• y = mx + b
• cost = fixed cost + variable cost
• profit = revenue − cost
• risk = probability × impact
• throughput = rate × uptime
• expected value = probability-weighted outcome
• model output = function of assumptions

A simple model helps identify what matters, what can be ignored, and what needs to be measured better.

The purpose of a simple model is not to capture everything. It is to make assumptions visible. Once the simple model is clear, optimization becomes more meaningful because we can see what variable is being optimized, what assumptions are driving the result, and where the result may break.

Sensitivity analysis asks what the answer depends on

Sensitivity analysis tests how much the output changes when inputs or assumptions change. The questions it tries to answer:

• What happens if this assumption is 10% higher or lower?
• Which variable changes the result the most?
• Does the optimal answer stay the same across reasonable assumptions?
• Is the decision robust, or does it depend on one fragile estimate?
• Which assumption should we measure more carefully?

Sensitivity analysis is one of the most practical ways to test whether an optimization result is trustworthy.

A few familiar examples:

• In a financial model, the result may be most sensitive to churn, pricing, or customer acquisition cost.
• In a process model, the result may be sensitive to yield, cycle time, or defect rate.
• In a pharmacokinetic model, exposure predictions may be sensitive to clearance, volume of distribution, binding, or absorption assumptions.
• In an AI workflow, performance may be sensitive to retrieval quality, prompt design, input structure, or evaluation criteria.

A practical pattern

This is a reasoning pattern, not a formal universal framework. It is the order I tend to walk through when an optimization question shows up in real work.

1. Define the decision

What decision is the optimization supposed to support?

2. Define the objective

What are we trying to maximize, minimize, or balance?

3. Name the constraints

What limits the feasible options?

4. Build a simple model

What variables and relationships matter most?

5. Identify the search space

What options or parameter ranges are actually being considered?

6. Solve or compare

Use calculation, simulation, heuristics, local search, global search, or model fitting as appropriate.

7. Test sensitivity

What changes the answer?

8. Check practical meaning

Is the “optimal” answer usable, robust, explainable, and aligned with the real decision?

The point is not to make everything mathematically perfect. The point is to make the decision better. This does not eliminate judgment.

When optimization goes wrong

A. Optimizing the wrong objective

The metric improves, but the real goal does not.

B. Ignoring constraints

The answer is mathematically attractive but impossible to implement.

C. Overfitting the model

The model fits observed data but performs poorly outside it.

D. Treating assumptions as facts

The result appears precise because uncertain inputs were treated as certain.

E. Getting stuck in a local optimum

The best nearby answer is mistaken for the best possible answer.

F. Optimizing before understanding

The system produces an answer before the problem is properly defined.

Optimization can create false confidence when the structure of the problem is weak.

Define before optimizing

Optimization is powerful, but it is not magic. It depends on the objective, the constraints, the model, the data, the assumptions, and the search space. A model is useful only if we understand its assumptions. The hard part is often not solving the equation, but defining the objective.

I am less interested in optimization as a way to produce a single impressive number, and more interested in optimization as a disciplined way to clarify tradeoffs, test assumptions, and make better technical decisions.