Measuring dev productivity
Measuring dev productivity
Are we being obtuse?
As I write this, an article by James Shore is on the front page of Hacker News. It caught my eye this morning as I waited for my coffee. The article is called “A Useful Productivity Measure?”
El Dilbertismo
As Shore points out in the article, productivity is well-known as impossible to measure. Kent Beck and Martin Fowler have articles stating exactly that. Further, Beck argues that once you try to improve a dev productivity metric, it backfires.
But Shore’s article shows something else: Maybe we engineers are caught in Dilbertism. All this talk about productivity metrics is distracting us from addressing the problem in front of us. In other words, when the CEO wants a productivity metric, maybe they just want to know that somebody knows where the money is going and can manage things.
Muda
In his presentation, Shore shows where developer time is spent: value-add work or “deferred maintenance, bugs, on call, incident response, deployments, and so forth.” He called those non value-add activities muda, the Japanese term for waste in the Lean movement. He committed to a metric goal: doubling the time spent on value-add work.
While I think his measurement is a good start, eventually, he will need to land on where Arty Starr got to in her book Idea Flow: The bottleneck for development (and likely any knowledge work) is the amount of time a dev has an understanding of the work loaded in their mind. Our work happens in our minds, so any impediment to understanding the task is time we can’t spend on productive work. That means that if you get stuck while working on a value-add feature, that’s muda, too.
Carrying cost
Shore also wants to measure ROI on development work. That reminds me of a team I worked on who was under management pressure to deliver faster. The higher-ups wanted more metrics because it seemed to spiral out of control. “Why aren’t you delivering faster?” But all software has a carrying cost. Nowhere in the management hierarchy were they looking at the sheer volume of dinero that flowed through our system. I should have shown them how much they would lose monthly if we stopped maintaining our centrally placed system. I bet they’d invest a lot more in our team and stop asking us to rush out new features at the expense of existing income.
Gaming metrics
Will Shore’s metric be gamed, as Beck predicts? I’ve seen gaming happen in three ways. The first is that the people reporting the metric are so interested in making the numbers look right that they lie. The metric becomes worse than noise—it’s outright disinformation. Things look like they’re getting better when they’re getting worse. This is a danger, but it looks like Shore is aware of it and will avoid it.
The second way is that people focus on the short-term metrics at the expense of the long-term. For instance, they stop fixing bugs because that time is counted against them. Just do feature work, and you get 100%! Meanwhile, your system is falling down.
The third way is to make the process worse. Fixing bugs is considered rework, so we want to avoid releasing bugs. Let’s write more automated tests, do a code review by two developers, and have Quality Assurance check it before we deploy it. All we’ve done is added more non-value-add work to reduce non-value-add work. You need a counterbalancing metric or ideal to favor effective waste reduction.
Consider an opposing ideal, such as “no hand-offs.” If you can’t have hand-offs, you can’t hand the release to QA to test, so that at least avoids that. The ideal has to be intrinsically good. “No hand-offs” works because hand-offs are known to cause problems. Another ideal would be “one-piece flow,” which means no waiting or batching. These ideals make excellent engineering values for a team.
Conclusion
I’m excited by the idea that we can find ways of communicating with management about productivity. Have we been under the spell of Dilbertism for decades? And though I like applying the idea of muda to our work, I hope it gets applied as in Idea Flow. Finally, I think there are ways to find creative tension between different ideals to look for lateral thinking.