backlog

Objective Driven Agile, BLDD & Reactive styles of development

Looking across the state of teams today I can clearly see three different styles of agile: Backlog Driven Development, Reactive and Objective Driven Agile. All have their place, all have their uses but the first two of these are a dead end. I’ve been speaking about the third, which I’m calling Objective Driven Agile, for years but in the last few months, since writing Succeeding with OKRs in Agile and speaking to many people about how OKRs and agile fit together, it has become clear to me that ODA allow us the industry, and teams, to address a number of problems that have arisen with agile in recent years.

1) Backlog Driven Development – BLDD. Teams are typically working in a Scrum fashion with a backlog and short sprints. Such teams are sometimes called “feature factories” because they are aiming to “do the backlog” – and the backlog is inevitably a lis of features.

While the Scrum these teams practice is a lot more agile than what came before they haven’t really moved very far from the big requirements documents of old. Backlogs are frequently bottomless pits, the team can never reach done. The loss of change review board might well be making things worse because Product Owners often lack the authority to say “No” or remove items from the backlog.

Still, not only are these teams agile but in many places this is success. The team are churning out stories, customers can see progress and are persuaded to part with cash. If the backlog can be tamed they might even declare done one day. But then, if the team is being operated by a consultancy or other outsourcer for a client the last thing they want to happen is for work to be done.

2) Reactive. Reactive teams gravitate towards Kanban but some are compelled to work in a Scrum fashion which doesn’t match their real work. Such teams may see backlogs are an anathema because they do “what is needed, right here, right now.” Team members here will say “true agility is listening to what customers want and getting it done as quickly as possible.” And in a way they are right.

Again, these teams are a success story, particularly if they are supporting a live system, running DevOps or if customers previously had to wait months for results.

The problem here is that these teams don’t have much capacity for doing the backlog or anything else and yet they are frequently still expect to “do the backlog”. If such a team have been tasked with immediate customer support then that isn’t a problem but frequently these teams face competing demands.

3) Objective Driven Agile, ODA: this is the approach I’m focused on at the moment. With Objective Driven Agile teams have a mission and a higher purpose, something more than “do the backlog” or “keep the system working.” Such teams might be considered “Empowered” and might practice “Dual track agile.” The key thing is, they are not beholden to a prepared list of things to do. The team are responsible for deciding what customers need, what will add value and what will meet team and organisational objectives, and then, delivering that thing.

These teams have “Product Owners” who are more than mere Backlog Administrators – I’ve started calling them “Product Specialists.” These people are tasked with understanding what will add value for customers, users and other stakeholders. Importantly they have the power to decide what gets done and to say “No”. With that power comes responsibility: responsibility to the team, to stakeholders and to other leaders in the organization. That means Product Specialists can explain – in a business case or in a star-chamber – how work on the product adds benefit and how the desired outcome – the objective – is moves things forward.

All three of these styles have their advantages and disadvantages but the real problems occur when the styles are not clearly stated.

Consider the team trying to deliver the backlog BLDD style but who have to “keep the lights on” and support a live system. It is mathematically impossible to make accurate forecasts about delivery when events can derail you. It is also nye on impossible to deny customer requests when they are waiving a large sum of money or escalating their ask through your organization. But again, this can blow up any plan or roadmap.

Nor is is possible to pursue objectives when teams are committed to supporting live or delivering a backlog. At best the objectives duplicate the role of the backlog and at worst increase work-in-progress and add stress to individuals on the team.

So my advice: decide which type of team you are and focus.

In the last few months I’ve been speaking a lot both about objectives and “just in time backlogs”. I’m fired up and want to pursue this idea, if you are interested please let me know and I’ll write more.


Subscribe to hear more from Allan Kelly and download Continuous Digital for free

Backlogs, #NoBacklog(s) and comfort blankets

At the first Agile on the Beach in 2011, I had dinner with Mary and Tom Poppendieck. As we talked about the conference, agile, lean and everything else I distinctly remember Mary saying “From a lean point of view backlogs are a problem. In a lean environment when you have a backlog you want to eliminate it.” (Postscript: Mary has spoken about her views publicly “Why You Should Just Burn Your Backlog“.)

Just then, I was then working with a client with a runaway backlog. I had calculated the backlog growth and found it was regularly greater than the work done “velocity.” It was like a mortgage were the monthly payments didn’t even cover the interest. If I remember correctly, backlog growth, interest, averaged 8.5% per month. I suggested to the client and their supplier that they throw the backlog away. They, not for the first time, thought I was mad. Since then I’ve voiced the same opinion with other clients and got the same response. But the opinion is gaining ground and I’ve even encountered a couple of places were they have moved away from the backlog.

To be fair, backlogs are useful – they can have a useful role to play but that role is akin to a child’s comfort blanket. There comes a time to say goodbye to childish things.

(By the way, I’m really discussing what Scrum calls “the product backlog” rather than the ever changing “sprint backlog.” So I mean: the stuff we might work on in future and not: the stuff we are working on this week, and next.)

I’m not, yet, ready to join Vasco Duarte in declaring #NoBacklogs but my “nuke the backlog” comment in a podcast with Jenny Herald said pretty much the same thing. That comment has attracted a lot of attention and seeded good discussions. I like those discussions and thats why I resist about #NoBacklog. When we started using #NoProjects it was good for conversation, but then a few people drowned out the discussions with calls of “You are mad”, they never considered our arguments and closed down discussion for others.

So what is the backlog problem?

First, as the graph above shows: backlogs don’t “burn down” they tend to grow, and often grow faster than work is done. That becomes a problem because many people expect the backlog to reduce to zero over time and organizations consider success (“Mission accomplished”) to be a completed backlog. The cost of adding something to the backlog is near zero, but the cost, to the product owner and/or team, of refusing a backlog item can be high – all the time spent explaining why something won’t be included.

The desire to “do the backlog” leads to the second problem: an emphasis on doing backlog over delivering value. It becomes more important to do backlog items (output quantity) then deliver benefits (outcomes.)

That, combined with the ever increasing number of items, leads to problem three: a loss of strategy and sense of purpose. This is classic “can’t see the wood for the trees.” There are simply so many backlog items to do it is hard to see what should be done. (The whole “twice the work in half the time” idea that surrounds Scrum makes this worse still. Raising the outcome over output question will also get you called mad.)

Along the way a lot of stakeholder problems get created because people believe that an item in the backlog is in some way promised when it isn’t. Product Owners and Teams accept items into the backlog for an easy life even when they know it is unlikely to ever get done. This stores up future problems because stakeholders start complaining when they fail to get their items. That damages trust in the team and a vicious circle ensues.

It gets more and more difficult to prioritise anything: more items to consider, more stakeholders to placate, more promises to (pretend to) keep. This makes it increasingly difficult to follow the benefit and change course and act on product feedback.

One of the reasons I came to like OKRs, despite my initial scepticism, was that they provided a means to think about the backlog and eventually move away from it. Another reason why I am avoiding #NoBacklog is because I want to be able to offer an alternative rather than just rubbish the backlog. At the moment I think OKRs are that alternative but I’m a little bit reluctant to force OKR Kool-Aid on people.

I tell a story in Succeeding with OKRs in Agile about a little experiment I conducted with another agile coach. The question was “Are OKRs written based on the backlog you intend to do in the next quarter? Or, are OKRs set in respect of business strategy and product priorities and backlog items selected, or even created, to meet the OKR?”. The experiment showed us that OKRs should come first and the backlog was secondary.

Perhaps backlogs are, as I think Vasco thinks, a hang over from the project days. A similar point is hiding inside Project Myopia: in the project model success is doing all the backlog, but if you prioritise by business value, if you are responsive to customers, if you are practicing dual-track agile and product discovery then you may well find that not everything in the backlog is valuable or even wanted by customers.

Increasingly I view Product Backlogs as comfort blankets – what psychologists call transitional objects. Like children’s toys they help us move from one view of the world to another. When starting with an agile style of working backlogs provide comfort by mapping work to the traditional (project) model, but in time, as you become better at listening to customers and responding they are a hinderance.

I’ll be talking more about backlogs and comfort blankets in an online presentation next week to Berlin Product People, join me there to hear more. (5 September 2022).


Subscribe to my blog newsletter and get Continuous Digital for free

How fresh is your backlog?

Do you struggle with an overwhelming backlog?

Do you count the number of product backlog items in your backlog in tens? hundred? or thousands?

Does your backlog contain many stories which have been there for months, if not years, and yet never raise to the top of the backlog?

Is your success judged on your ability to do the backlog?

Backlogs were a good idea when they were introduced a bit over 20 years ago but today many teams slaves to the backlog – see my posts on the Tyranny of the backlog and Purpose over backlog. One of the benefits I’ve called out for OKRs is the ability to move away from backlog driven development (BLDD).

In Succeeding with OKRs in Agile I ask suggest you either need to prioritise your backlog over OKRs (in which case OKRs are derived from the backlog you intend to do) or OKRs over backlog (in which case OKRs are derived from strategy and the backlog plays a supporting role.) In my podcast with Jenny Herald earlier this year I even say “Let OKRs drive… nuke the backlog.”

Filipe Albero Pomar recently shared his backlog freshness blog which I think is great. Freshness is a great way to think about the state of the backlog that separates the size of the backlog from the relevance of the backlog.

Filipe’s idea is simply to talk about the backlog in terms of freshness – you have a fresh backlog if your backlog items are fresh: written recently, relate to current opportunities, problems and things people currently want.

And of course, the opposite of fresh: stale, stories that have been sitting in the backlog for months, even years, stories that relate to yesterday’s problems and project, stories which people wanted last year. The existence a big backlog of stale stories means the team is seen to be not delivering, the end-date is far off because people still expect all the work to be done.

Filipe suggests backlog freshness can be measured:

1. Set a cut-off date

2. Categorise stories as fresh or stale: fresh stories have been written since the cut-off date, those which are older are stale

3. Calculate freshness as a percentage of fresh from the total: if 25 stories out of 60 have been written in the last month then the backlog is 41% fresh, and 59% stale

Thats a useful metric, I think we can do better, look at the graphic above. I group backlog items into age groups and graphed them. For completeness I added a line to indicate average story age. Clearly this backlog is not fresh – nearly half the stories are over a year old.

I like the idea of graphing backlog freshness because it is easy to understand and makes an impact. In the graph above I’ve categorised backlog items into age groups and added an average line. Clearly this is not a fresh backlog. Whether this is the way to demonstrate backlog freshness I’m not sure – I’m playing with a histogram and quartile ranges.

With some clients I’ve thought of the backlog like a mortgage. There is the principle (the existing backlog), the interest rate (the growth rate of the backlog) and the monthly repayments (stories reaching done). Unfortunately when you do this you sometimes find the mortgage will not be paid for many years, and perhaps never. (Don’t worry about estimating the size of stories, for this sort of analysis the number of stories will get you started, and if your backlog is measured in hundreds of items the small will offset the large.)

I’d love to talk more about this and experiment with some ideas, I think it could be a very useful way of thinking about the backlog.


Subscribe to my blog newsletter and get Continuous Digital for free

Purpose over Backlog

Backlogs are a good idea. Backlogs ease the transition from the old “requirements up front” world to the new more dynamic agile world. Backlogs provide a compatibility layer for agile teams to interface to more traditional project management and governance. Backlogs even allow you to take a stab at done date!

Backlogs allow you to even out work between the quiet periods and the busy times. Backlogs give you a place to store good ideas which you can’t do just now. And because stakeholders can see their request is not forgotten they don’t need to shout for it today.

Yes backlogs are good. I’ve seen them work well myself and I’ve taught many teams to effectively use backlogs.

But – you knew there was a but coming didn’t you? – but…

Backlogs have problems, too many teams are labouring under the Tyranny of the Backlog, they have become backlog-slaves and practice something we might call BLDD – Back Log Driven Development.

(To be clear, when I say “backlog” I am primarily thinking of the product backlog – the long list of all the things the team (might) do in the future. This is different to the sprint backlog (iteration backlog). The sprint backlog is a shorter list of things the team aims to do this iteration. I am using Scrum terminology but the ideas are pretty much “generic agile” and I’m thinking more broadly than Scrum. Many implementations of Kanban feature a product backlog of sorts so while Kanban is less prone to these problem it is not immune.)

1) Lump of Work Fallacy

There is usually an assumption that the backlog represents all the work to be done – an impression reinforced by early implementations of Scrum. In the short term that leads to agile teams being seen as inflexible and prioritising process over need because new work is not allowed in.

In some cases teams even struggle to get started on work because a big-up-front requirements gathering and analysts activity is required to create a backlog. In the worst cases that work is even estimated and end-dates forecast before a line of code is cut or developers hired.

In the longer term it is simply unrealistic to assume the backlog is fixed. Even with more and better analysis it is impossible to foreseen future requests. The agile adage “it is in doing the work that we understand the work” cuts both ways: coders understand what they need to build and customers/stakeholders/analysts understand what they want.

Work will arrive after you begin, any system that does not incorporate that truth will fail one-way or another.

2) Bigger than you think

Not only does the backlog grow with completely new work the work in it changes – and grows. There are many reasons this happens: new opportunities appear, hidden ones become clear, requests require more work than expected and so on.

Humans are very bad at estimating, especially about the future, and, it turns out, they are also very bad at estimating time spent in the past. If you want accurate forecasts you need to invest in them, you need to make structural changes and you need to use statistics.

However, because of the lump of work fallacy and the belief that humans can make estimates, poor end-date projections get made and when they are missed (because they were wrong to start with) everyone gets upset.

3) Fallacy of Done

Backlogs come with burn-down charts and an assumption that there is an end; and that end is when everything is “done.” The team will be done when the backlog is empty. That assumption is baked into BLDD, traditional project management and even governance.

I have long argued that software is never done. I’ll accept that I might be wrong, but in the digital age, when business runs on digital technology (i.e. software) your products are only done when you business is done. The technology is the business, and the business is the technology. Stop the backlog growing, stop growing you technology and you kill the business.

4) Backlog Bottomless pit

Put all those reasons together and the backlog becomes a bottomless pit. In the early days of agile, when I managed teams myself, the backlog would often sit on my desk, written out on index cards and held together with rubber bands. I could get a sense of how big the backlog was my looking.

Today everyone uses electronic tracking systems. Not only do these allow an infinite number of items they rob us of perspective. To paraphrase Comrade Stalin: “2 outstanding backlog items is tragedy, 200 is a statistic.”

5) Backlogs obscure strategy & purpose

With so many backlog items it is easy to get lost – you can’t see the wood for the trees. Arguments over what will be done next start to resemble deciding who should get a lifeboat place on a sinking ship, add in the demands “when will you be done?” (plus explaining why the date has changed) and “the bigger picture” gets lost.

In Back Log Driven Development the sense of purpose and strategic goals is lost as teams struggle with the day-to-day demands of just doing stuff.

6) Powerless product owner (i.e. backlog administrators)

Tyranny of the backlog seems worst where product owners lack real authority and skills. They are little more than backlog administrators. They spend most of the week adding requests to the backlog, then passing a few chosen items to developers in planning meetings. A vicious circle develops, the product owner can’t win so people trust them less, their authority wanes, and the backlog spirals.

Few organisations give product owners the power needed to get a grip on this situation. Indeed, many product owners are plucked from the ranks for development or support and given a battlefield promotion to product owner but lack the skills required. (See The problem with Product Owners.)

A solution?

For years I’ve been suggesting teams throw away the backlog – you will not forget the important things. But then how do you know what to do?

Take a step back, start with your purpose, your mission, the reason you team, your company, your organisation exists. What should you be doing? How can you fulfil that purpose and serve your customers?

This is where I see a role for OKRs and jobs to be done. Both these techniques – together, or separately – can be used as story generators. Every time you need to more work, more stories, you return to your OKRs and ask “what can we do now to move us towards our objective?”

When writing Succeeding with OKRs in Agile I became more and more convinced this is the path to take. Increasingly I sum this up as Purpose over Backlog.

Post publication addition: great suggestion fro Jari Mäkelä on Twitter to call this: Purpose Driven Development, PDD.

Step 1: Clarify your purpose – what is your overarching reason for existing?
Step 2: Clarify how your existing strategy builds towards that purpose, and if you don’t have a strategy create one.

Repeat steps 1 & 2 annually.

Step 3: Think broadly, set your OKRs as a team so you build towards your purpose by following your strategy.
Step 4: Spend the next 12 weeks executing against those OKRs

Repeat steps 3 & 4 every 3 months.

Step 5: In each planning meeting take stock of what you have done and progress against OKRs
Step 6: Ask “what do we need to do next to move towards the OKRs?”

Repeat steps 5 and 6 every 2 weeks

And if you are Kanban’ing then keep steps 1, 2, 3 and 4, adjust 5 and 6 as appropriate.

Having finished, completed, published Succeeding with OKRs I really wish I had been clearer in the book. The ideas are there but with time they have become so much clearer… maybe I need another book.

Buy Succeeding with OKRs in Agile at Amazon today.


Subscribe to my blog newsletter and download Project Myopia for Free

Verified by MonsterInsights