Product Metrics for Engineers

The problem

When the conversation turns to product metrics, a lot of technical people react in one of two bad ways.

First way:

“that is PM stuff”

Second way:

“let’s measure everything and open a dashboard”

Both are weak.

In the first case, engineering makes implementation decisions without understanding what needs to move.

In the second, the team collects numbers without criteria and pretends to be data-driven.

A good metric is not meeting decoration.

It is a decision tool.

Mental model

Think about it like this:

a product metric exists to answer whether the change is generating value and at what cost.

That second part matters a lot.

Because almost every main metric needs guardrails so it does not turn into a distorted incentive.

Simple example:

• you increase activation
• but error rate gets worse
• or retention gets worse
• or operational cost goes up

If only the main metric appears, the team celebrates too early.

Breaking the problem down

Adoption is not the same as activation

These concepts are constantly mixed together.

Adoption usually answers:

• how many people used or touched this capability?

Activation usually answers:

• how many reached the first real value?

Example:

• the user opened the automation feature
• that is adoption

But did they manage to configure the first working automation?

• that is activation

If you only measure screen opens, it looks like the feature did well. But maybe nobody reached the part that actually matters.

Retention answers a different question

Retention does not want to know whether the person tried the feature.

It wants to know whether they came back or kept using it repeatedly.

That difference changes everything.

An onboarding feature may improve short-term activation and do nothing for retention.

An essential recurring-use feature may barely affect activation but strongly affect retention.

Without that separation, every discussion becomes “usage went up.”

And “usage went up” by itself says very little.

Guardrails protect against false wins

A guardrail is the metric you do not want to sacrifice in order to improve the main one.

Examples:

• errors per session
• response time
• cancellations
• complaints
• support contacts
• cost per operation

If you improve conversion while brutally worsening errors, latency, or later friction, you probably only moved the problem somewhere else.

Guardrails exist to stop that self-deception.

Metrics need to match the right unit

This detail changes the interpretation.

You can measure by:

• user
• account
• session
• request
• order

Each choice tells a different story.

If one enterprise account has hundreds of users, “per account” and “per user” can change the conclusion a lot.

Good analysis starts by asking:

which unit best represents the behavior we want to understand?

Not every metric needs to become an official KPI

This is another common exaggeration.

Some numbers serve as local learning indicators.

They do not need to become a scoreboard for the whole company.

If everything becomes a KPI, nothing carries real weight.

Operational metrics, exploration metrics, and central metrics are different things.

Simple example

Imagine a new checklist flow inside the product.

Weak main metric:

• page views of the screen

That tells you almost nothing about value.

A better reading might be:

• adoption: users who opened the checklist
• activation: users who completed the first checklist
• retention: users who came back to use the checklist in another week
• guardrails: average time to complete, save errors, increase in support tickets

Now there is a more honest narrative.

You are not measuring curiosity alone.

You are measuring:

• entry
• initial value
• continuity
• side cost

What usually goes wrong

• Measuring clicks and calling that success.
• Mixing entry metrics with value metrics.
• Choosing the easiest number to see, not the most useful one for decisions.
• Failing to define guardrails and only noticing the damage later.
• Turning any small increase into proof of success.
• Building the dashboard before clarifying the question.

How someone more senior thinks

A stronger person usually organizes measurement around four questions:

1. which main behavior do we want to move?
2. how do we know that created value, not just activity?
3. what could get worse if this metric goes up?
4. which slice really matters for the decision?

That kind of question improves the conversation a lot between engineering, product, and leadership.

Because the focus moves away from isolated numbers and toward interpretation.

Interview angle

This topic shows up in questions like:

• “how would you measure the success of this feature?”
• “which metrics would you track after launch?”
• “which number would you choose as a guardrail?”

The interviewer usually wants to see whether you:

• understand the difference between usage and value
• protect against blind optimization
• connect implementation to real impact

Weak answer:

I would look at page views, time on screen, and number of clicks.

Strong answer:

I would separate adoption from activation and retention, because opening the feature does not prove value. I would also add guardrails for error rate, latency, or support load depending on the flow, so we do not call it a success when the main number moved up while the experience got worse.

Closing

A good metric does not need to look sophisticated.

It needs to help answer:

• did this really improve?
• for whom?
• at what cost?

If the answer still depends on heroic interpretation, the problem is almost never the dashboard.

It is the reasoning that chose the wrong metric.