When Metrics Become the Target

Metrics and analytical models are a double-edged sword. They make complex systems easier to compare, automate, and reason about. But once a metric becomes important, people start optimizing for it. Not for the reality behind it. For the number itself.

This is Goodhart’s law in practice: when a measure becomes a target, it stops being a good measure.

The Incentive Problem

GDP is useful as a rough signal of economic activity. Turn it into the main way a country’s performance is judged and the incentive changes. The goal is no longer to describe the economy as accurately as possible. The goal is to make the number look good.

The same happens with inflation. The basket of goods is supposed to represent everyday costs. But the moment the inflation number becomes politically important, the composition of that basket stops being a neutral technical detail. It becomes part of the game.

You can argue about methodology forever. But the pattern is simple: if people understand the scoring system, they adapt their behavior to the scoring system.

Games Make It Obvious

Computer games show this in a cleaner form because the rules are explicit. Players dissect mechanics, build spreadsheets, calculate damage formulas, find optimal builds, exploit AI behavior, and farm the safest resources. They optimize.

Sometimes that’s the fun. But often it leads to a strange result: the best way to play the game is not the most enjoyable way to play it.

The design says: explore, experiment, immerse yourself. The incentives say: repeat this boring loop because it gives 12% more XP per hour.

The Algorithmic Escape

Large internet platforms learned this lesson. If search ranking, video recommendations, or social feeds are too easy to understand, people optimize directly against them. SEO farms, clickbait thumbnails, engagement bait, keyword stuffing, link networks - every transparent rule becomes an attack surface.

So companies like Google moved toward something closer to a god algorithm: opaque, constantly changing, and impossible to fully reverse engineer. It decides what appears in search results, what shows up on your home page, what gets recommended next. It can reward you one month and bury you the next, often without a clear explanation.

From the platform’s point of view, this makes sense. Uncertainty is a defense mechanism. If nobody knows exactly how the system works, it is harder to game.

The Black Box Problem

Opacity solves one problem by creating another.

How do you judge the quality of a system whose explicit strategy is to be unreadable? How do you audit bias, manipulation, ranking quality, political influence, or commercial pressure when the decision process is hidden behind layers of models, experiments, and internal metrics?

The platform can always say the algorithm is too complex to explain. Users can only observe outcomes. Creators learn superstition. Businesses reverse engineer shadows. Regulators ask for accountability and get dashboards.

This is the uncomfortable tradeoff. Transparent systems are easier to exploit. Opaque systems are harder to trust.

No Clean Escape

We should be honest about what happens after a metric becomes important. It stops being a passive observation and becomes part of the environment. People react to it. Institutions bend around it. Entire markets form around exploiting it.

The more power a metric or algorithm has, the more skepticism it deserves.