A friendly discussion about Agile Estimating

It started in the AgilePDX Slack as a bit of a dustup over Henrik Kniberg’s Product Ownership in a Nutshell video. Jean Richardson (azuregate.net) was concerned that he conflated velocity with capacity (there are those who consider them sufficiently indistinguishable and those that don’t), something she had seen others do recently. The discussion veered into estimating in general and why teams should do it and what value is really derived from it. If that’s your sort of discussion, you can easily imagine that it wasn’t very satisfying in that medium.

It is my sort of discussion and I suggested we do a pop-up meeting and anyone who was available should join. We could throw down over estimating and see what we could come up with. It’s a complex topic and sitting around a table in the coffee shop with some sticky notes and being able to dig deeper into each other’s thinking might take us somewhere interesting. Not surprisingly, we didn’t fix this (or we’d be writing a book instead of a blog post) but it was an interesting discussion and we did come to some agreements (among the 4 of us: Jean Richardson, Matthew InmanCochrane and Ken Aust — let us know what you think in the comments).

tl;dr

The existing material on the topic is sufficient — if you’ve chosen well. After reviewing several definitions of velocity, we agreed Mike Cohn’s best captures our concerns: “Story points are estimates of effort as influenced by amount of work, complexity, risk and uncertainty. Story points are a unit of measure for expressing an estimate of the overall effort that will be required to fully implement a product backlog item or any other piece of work.” [emphasis mine because I’ve seen plenty of folks refer to it simply as a measure of complexity which is insufficient since that means velocity then becomes “how much complexity we typically deliver”, which doesn’t seem interesting or useful]. The point here is to be crystal clear what you are measuring so that metrics about it are meaningful.

So a Story Point is a multi-faceted measure of effort. That means Velocity maps well to the overall effort (presuming you use Story Points as defined above, more on that below) that your team typically completes per iteration making it a useful predictor of how much effort you should be able to complete in future iterations — all other things being equal. I think that’s where Jean’s concerns about conflating velocity with capacity comes in. Her take on capacity is a combination of time and capability available for the iteration (and this is consistent with multiple sources). With summer coming on, the capability part may become important to consider for teams that aren’t yet as diverse (or generally specialized, see links at end) as they might like to be. Some stories, no matter how important, may be difficult to complete against their estimate because all other things are not equal — you may be missing someone you really need to get it done.

We only poked at a question that brought us together (and may poke at some more in a future session): How can you tell that you are spending the right amount of effort estimating or How do you know if your estimating approach is valuable to your team? The #noestimates folks aren’t sure you should spend much time estimating at all — just get work done. At the other end, some teams may become more concerned about predictability than productivity, spending more effort estimating and planning work when they could just be delivering value. What determines how good or loose your team’s estimates need to be to deliver value reliably (and as much of it as you can)? We discussed some team/organization/environmental concerns that might influence where you choose to be on that spectrum (described briefly below) but did not have time to get beyond that.

One of our “mini-whiteboards” of sticky notes (thanks for Jean’s penmanship)

Our discussion ran for almost 2 hours and was largely discovery in nature — sharing experiences, looking for themes, hoping to find some insight that we’d all overlooked. [It was fun and I hope we do more of these.] I think a fair summary of the discussion would include:

• You need to estimate at least your Stories (or whatever you call your backlog items) and you should have an overall estimate of effort and Business Value. Otherwise it becomes difficult to determine if the story is worth doing at all, much less where it should be in the backlog. Maximizing work not done helps teams focus on delivering the most valuable bits. Having this data also allows you to bargain hunt your backlog. The act of creating an estimate the team can agree on is likely to force valuable discussion of what the story really means (and as with many Agile activities, that discussion can be at least as valuable as the reason you are having it).
• We decided that breaking Stories into Tasks is in itself an act of estimation. It’s not uncommon to find work as you go that wasn’t accounted for during sprint planning. This is where determining your capacity and planning to a reasonable percentage of it really matters if you want to complete the committed stories. There’s a tension here between finding all of the tasks (to limit found work) and getting going, knowing that it’s going to happen anyway. A suggestion here is to tag found work so that you can reflect on how much of that happens and if you need to do anything about it. For example, if you are finding as much work as you planned, perhaps your sprint planning meeting should dig a bit deeper.
• Velocity is the rate at which you consume your backlog (and is typically defined as the number of Story Points you complete per iteration). Some teams have taken to breaking stories into similarly sized pieces of effort, so they just count the number of stories completed. This is essentially the approach described in the Kniberg video — though in the video, the idea of a half-story and double-story creeps in, so while not quite Story Points in their Fibonacci form, it’s clearly not an all stories are the same size approach, either. Just like in grade school math, watch your units. Nice explanation of that showed up here recently. And Mike Cohn writes about the main benefit of Story Points and what to know about your team’s velocity.
• Estimating tasks and assigning them to people during sprint planning doesn’t work for all teams (and can lead to a bit of an anti-pattern). Jean shared a story of a team where doing this made them less productive. When they looked at their backlog items and gut-checked on what they thought they could deliver, things when really well. When they experimented with the more traditional approach, it didn’t fit well and the sprints tended to be less productive. On the anti-pattern side, I’m aware of a team where members get an attaboy if they complete all of their tasks (which totally flies in the face of team responsibility for the sprint delivery).
• The estimating activity is part of your system of work, not a one-size-fits-all activity. The quality and importance of estimation depends on how you handle and evaluate your work (your estimation tolerance). We didn’t have time to go deeper on this though we captured some of our thoughts in the stickies. For example, if you deliver continuously in small batches, you probably don’t even think in terms of iterations, so effort-against-capacity doesn’t likely concern you. Whether you think it will take 2 days or 4 days, it’s the next most important thing to do and you’re going to do it and ship it. If you are on 4 week iterations, you probably need to work harder to avoid found work as it could really pile up on you, blowing out your best laid plans. Hopefully we’ll make time to go deeper on this topic soon.

Fundamentally, prefer progress to activity (your situation will inform which is which). Don’t be afraid to continuously question and try different approaches with your team to find the sweet spot. We’re sure you have thoughts on this, please share them!

Of possible interest:

• I couldn’t decide which of these links I liked best about generalizing specialist (so choose for yourself): https://blog.codinghorror.com/swiss-army-knife-or-generalizing-specialist/. https://www.jrothman.com/htp/hiring-strategy/2014/11/five-tips-to-hiring-a-generalizing-specialist/, or http://blog.projectconnections.com/alan_koch/2016/01/agile-teams-need-generalizing-specialists.html
• Aspects of this piece reflect my own journey and wrestling with how much effort to spend on estimation. It has an interesting, all-star comments section.