How to Measure Developer Productivity Without Being Evil
ScrumPoi · · 11 min read
How to Measure Developer Productivity Without Being Evil
If your team’s “productivity” is measured in story points per sprint, you’re not measuring productivity. You’re measuring how good people are at gaming a broken system.
Developer productivity is one of the most abused concepts in software. Leaders want numbers. Developers want autonomy. Product wants outcomes. And somewhere in the middle, someone decides that counting tickets closed is a good idea.
It isn’t.
Let’s talk about how to measure developer productivity in a way that:
- Actually improves delivery
- Doesn’t destroy trust
- Doesn’t turn your team into KPI-chasing robots
The Core Problem: You’re Measuring the Wrong Thing
Most attempts to measure developer productivity fail for one simple reason:
They measure activity, not outcomes.
Lines of code, number of PRs, story points completed, tickets closed — these are activity metrics. They’re easy to collect, easy to dashboard, and easy to weaponize.
They’re also easy to fake and strongly correlated with:
- More bugs
- More burnout
- Worse architecture
- Less collaboration
What Developer Productivity Really Is
Developer productivity is not “how fast people type code.”
At a team level, productivity is the ability to reliably turn ideas into working, maintainable software that delivers value.
That means your metrics should reflect:
- Flow – How smoothly work moves from idea to production
- Quality – How often things break and how painful it is to fix them
- Sustainability – Whether the team can keep this pace without burning out
- Value – Whether what you ship actually matters to users or the business
If your “productivity” metrics don’t connect to at least two of those, you’re probably measuring the wrong things.
A Better Lens: Team-Level Flow, Not Individual Output
Stop trying to rank individual developers. The team is the unit of delivery.
Why Individual Metrics Are Toxic
When you measure individuals, you:
- Incentivize hoarding work (“I’ll keep this ticket so I get the credit”)
- Discourage pairing and mentoring
- Punish people who tackle hard, ambiguous problems
- Reward shallow, low-impact work
I’ve seen teams where a senior engineer spent a week refactoring a gnarly module that unblocked everyone else — and got questioned about their “low velocity” in the retro. That’s what bad metrics do.
Shift to Team-Level Metrics
Instead of “How many tickets did Alice close?”, ask:
- How quickly can this team turn a well-shaped idea into production?
- How often does their work cause incidents or regressions?
- How predictable are their delivery timelines?
- How often are they blocked by dependencies, environments, or unclear requirements?
This is where flow-based metrics come in.
The Core Metrics That Actually Help
You don’t need 20 metrics. You need a small set that you actually use to make decisions.
1. Lead Time for Changes
Definition: Time from “work started” to “running in production.”
Why it matters:
- Short lead time means you can respond to change quickly.
- Long lead time usually means bottlenecks: reviews, environments, approvals, testing.
How to use it:
- Track median and 85th percentile, not just average.
- Slice by type of work: features vs bugs vs tech debt.
- If lead time is high, ask:
- Are PRs sitting unreviewed?
- Are we waiting on other teams?
- Is our testing pipeline slow or flaky?
Actionable steps:
- Limit WIP (work in progress) — fewer parallel tasks, faster flow.
- Reduce PR size — aim for “small enough to review in 15 minutes.”
- Automate regression tests so reviews focus on design, not syntax.
2. Deployment Frequency
Definition: How often your team successfully deploys to production.
Why it matters:
- Higher frequency usually means smaller changes, fewer surprises.
- It correlates with lower change failure rates (DORA research).
How to use it:
- Track deployments per week per team, not per individual.
- Watch for long periods with no deploys — that’s risk building up.
Actionable steps:
- Move toward trunk-based development or short-lived branches.
- Automate deployment as much as possible.
- Make deployments boring and reversible (feature flags, blue-green, etc.).
3. Change Failure Rate
Definition: Percentage of deployments that cause incidents, rollbacks, or hotfixes.
Why it matters:
- High frequency + high failure rate = chaos.
- High failure rate often signals rushed work, weak testing, or unclear requirements.
How to use it:
- Track incidents linked to recent changes.
- Look at patterns: certain services, teams, or types of changes?
Actionable steps:
- Add lightweight pre-deploy checks (smoke tests, contract tests).
- Improve observability so you catch issues early.
- Use post-incident reviews to improve process, not assign blame.
4. Mean Time to Recovery (MTTR)
Definition: How long it takes to restore service after an incident.
Why it matters:
- Incidents will happen. MTTR shows how resilient your team and systems are.
- Faster recovery = less stress, less business impact.
How to use it:
- Track from detection to mitigation, not to “perfect fix.”
- Look at whether delays are technical or organizational (waiting for sign-off, unclear ownership, etc.).
Actionable steps:
- Invest in good logging, metrics, and alerts.
- Practice incident response (on-call rotations, runbooks).
- Make rollbacks and feature toggles easy.
What Not to Do: Common Measurement Mistakes
Here’s where most teams go wrong.
Mistake 1: Using Story Points as a Productivity Metric
If you’re comparing velocity across teams or individuals, you’ve misunderstood story points.
Problems:
- Teams inflate estimates to look “faster.”
- People avoid complex stories that would slow them down.
- Story points become a negotiation tool, not a planning tool.
Use story points for capacity planning within a team, not for performance comparison.
Mistake 2: Counting Tickets Closed or PRs Merged
You get:
- Lots of tiny, low-impact tasks
- Shallow fixes instead of root-cause work
- Resistance to helping others (“you close your own tickets”)
Tickets are not equal. A one-line config change and a week-long architectural change are not comparable units of “productivity.”
Mistake 3: Ranking Developers by Lines of Code
This is the classic “being evil” metric.
High LOC can mean:
- Overly verbose code
- Lack of reuse
- Poor design
Some of your best engineers will delete thousands of lines of code and simplify things. That’s productivity.
Mistake 4: Secretly Monitoring People
Tracking keyboard activity, window focus, or “online time” is a great way to:
- Destroy trust
- Encourage fake busyness
- Drive your best people away
If you can’t trust your developers to manage their time, you have a hiring or leadership problem, not a metrics problem.
How to Measure Without Being Evil: Principles
Use these principles as guardrails.
Principle 1: Transparency Over Surveillance
- Be explicit about what you’re measuring and why.
- Share the dashboards with the team.
- Involve developers in defining metrics and thresholds.
If you’re afraid to show your metrics to the team, you’re measuring the wrong things.
Principle 2: Team-Level First, Individual Only When Needed
Default to:
- Team metrics for flow and outcomes
- Individual feedback via:
- Code reviews
- Pairing sessions
- 1:1 conversations
- Peer feedback
Use individual metrics sparingly and qualitatively, not as hard rankings.
Principle 3: Metrics Are Conversation Starters, Not Judgment Hammers
Treat metrics as:
- Signals to investigate, not verdicts
- Inputs to retrospectives and experiments
Bad: “Your lead time is too high; work faster.”
Good: “Our lead time jumped from 2 to 6 days. What changed? Is something blocking us?”
Principle 4: Balance Speed with Quality and Sustainability
If you only optimize for speed, you’ll burn out your team and your system.
Balance your metric set across:
- Delivery (lead time, deployment frequency)
- Quality (change failure rate, escaped defects)
- Sustainability (on-call load, overtime, satisfaction)
Practical, Actionable Ways to Measure Developer Productivity
Let’s make this concrete. Here’s how to set this up without a six-month “metrics initiative.”
Step 1: Start with Three Core Metrics
For each team, track:
- Lead time for changes
- Deployment frequency
- Change failure rate
Keep it simple:
- Pull from your CI/CD system and incident tracker.
- Show trends over time, not just raw numbers.
- Review them in sprint reviews or ops reviews, not every day.
Step 2: Add Qualitative Feedback Loops
Numbers alone are dangerous. Combine them with:
-
Retrospectives – Ask:
- What slowed us down this sprint?
- Which bottlenecks were in our control vs outside?
- Did we ship things we’re proud of?
-
Developer satisfaction pulse (monthly or quarterly):
- “Do you feel you can do your best work here?”
- “What slows you down the most?”
- “How confident are you in the quality of what we ship?”
Look for patterns, not individual complaints.
Step 3: Make Bottlenecks Visible
Use simple visual tools:
- Kanban board with WIP limits
- Clear columns for “Waiting for review,” “Waiting for QA,” “Blocked”
Then track:
- How long work sits in each column
- How often stories are blocked and why
This gives you specific interventions:
- Too long in “Waiting for review”? Introduce review SLAs or rotate “reviewer of the day.”
- Too many “Blocked” items? Address dependencies or clarify ownership.
Step 4: Measure Technical Health Indirectly
Don’t try to invent a “code quality score.” Instead, track signals:
- Time spent on unplanned work (incidents, urgent bugs)
- Rework rate (stories reopened, bugs in recently changed areas)
- Cycle time for changes in “legacy” vs “new” areas
If new features in a certain module always take longer than expected, that’s a sign of technical debt hurting productivity.
Step 5: Align Metrics with Business Outcomes
Developer productivity is only meaningful if it supports business goals.
Connect the dots:
- “We improved lead time from 7 days to 3 days” → “We can now test 2x more experiments per quarter.”
- “We reduced change failure rate from 25% to 10%” → “Fewer incidents, less time firefighting, more time building.”
This prevents the “metrics for metrics’ sake” trap.
Examples of Healthy vs Unhealthy Metric Use
Unhealthy Scenario
- Leadership sets a target: “Increase story points per sprint by 30%.”
- Teams inflate estimates.
- Developers slice work into tiny tickets to look productive.
- Hard, foundational work gets postponed.
- Velocity goes up; morale and quality go down.
Healthy Scenario
- Leadership sets a goal: “Reduce lead time for changes from 5 days to 2 days over the next quarter.”
- Teams analyze where time is lost:
- PRs waiting 2 days for review
- Manual regression testing before deploy
- They run experiments:
- Smaller PRs, review rotation
- Add automated smoke tests
- Lead time drops, deployment frequency increases, and developers feel less blocked.
Same desire for improvement. Completely different impact.
Tools Can Help, But They’re Not the Point
You don’t need an expensive “developer productivity platform” to start. Use:
- Your existing issue tracker for flow data
- CI/CD logs for deploy and failure metrics
- Lightweight tools for better collaboration and retrospectives
For example, if you want more honest retros about what’s slowing developers down, tools like ScrumPoi (a free planning poker and retrospective tool with anonymous voting and no signup friction) can surface real pain points without people worrying about politics.
Use tools to support better conversations, not to micromanage keystrokes.
The Bottom Line
If you measure developer productivity by counting tickets, story points, or lines of code, you will:
- Get worse software
- Burn out your team
- Still not know who’s actually effective
If you instead:
- Focus on team-level flow and quality
- Use a small set of meaningful metrics
- Combine numbers with honest conversations
- Make bottlenecks visible and fixable
…you’ll get faster delivery, better morale, and a culture where “productivity” means building the right things well — not just looking busy.
Measure what matters. And if a metric makes your team feel watched instead of supported, change the metric, not the team.