backlog

Big and small, resolving contradition

Agile, Continuous / / / ,

Have I been confusing you? Have I been contradictory? Remember my blog from two weeks back – Fixing agile failure: collaboration over micro-management? Where I talked about the evils of micro-management and working towards “the bigger thing.” Then, last week, I republished my classic Diseconomies of Scale where I argue for working in the small. Small or big?

Actually, my contradiction goes back further than that. It is actually lurking in Continuous Digital were I also discuss “higher purpose” and also argue for diseconomies of scale a few chapters later. There is a logic here, let me explain.

When it comes to work, work flow, and especially software development there is merit in working in the small and optimising processes to do lots of small: small stories, small tasks, small updates, small releases, and so on. Not only can this be very efficient – because of diseconomies – but it is also a good way to debug a process. In the first instance it is easier to see problems and then it is easier to fix them.

However, if you are on the receiving end of this it can be very dispiriting. It becomes what people call “micro management” and that is what I was railing against two weeks ago. To counter this it is important to include everyone doing the work in deciding what the work is, give everyone a voice and together work to make things better.

Yet, the opposite is also true: for every micro-manager out there taking far too much interest in work there is another manager who is not interested in the work enough to consider priorities, give feedback or help remove obstacles. For these people all those small pieces of work seem like trivia and they wonder why anyone thinks they are worth their time?

When working in the small its too easy to get lost in the small – think of all those backlogs stuffed with hundreds of small stories which nobody seems to be interested in. What is needed is something bigger: a goal, an objective, a mission, a BHAG, MTP… what I like to call a Higher Purpose.

Put the three ideas together now: work in the small, higher purpose and teams.

There is a higher purpose, some kind of goal your team is working towards, perhaps there is more than one goal, they may be nested inside one another. The team move towards that goal in very small steps by operating a machine which is very effective at doing small things: do something, test, confirm, advance and repeat. These two opposites are reconciled by the team in the middle: it is the team which shares the goal, decides what to do next and moves towards it. The team has authority to pursue the goal in the best way they can.

In this model there is even space for managers: helping set the largest goals, working as the unblocker on the team, giving feedback in the team and outside, working to improve the machine’s efficiency, etc. Distributing authority and pushing it down to the lowest level doesn’t remove managers, like so much else it does make problems with it more visible.

Working in the small is only possible if there is some larger, overarching, goal to be worked towards. So although it can seem these ideas are contradictory the two ideas are ultimately one.

Big and small, resolving contradition Read More »

Nuke your backlog?

Agile / / / , ,

When I deliver “Honey, I shrunk the backlog” and when I tell people “Nuke the backlog” there are a few questions and talking points which come up again and again. So, if you read my last post and have been asking yourself how you live with a backlog you want to nuke… read on…

Lead with goals

“My boss won’t stand for me deleting the backlog” I empathise. I know it happens.

At the same time I wonder: does your boss really care? – I’m sure some of them do, I see many Product Owners who are really “Backlog administrators.” Their boss is certainly leaning over their shoulder checking on what being done. If this is your case then your boss is the real Product Owner, sorry to say this but you are really a goffer.

In this case you want to educate your boss, you want to start having discussions about a better way. This is going to be a long and hard path. There is no sure fire advice I could give you here, except to suggest you give me a call.

Assuming your boss give you some leeway then go and start a conversation about your bigger goals. Beyond “delivering backlog items” what are the goals your goals? More specifically, what are driving at for the next 10 weeks?

Start to have a conversation about goals bigger than backlog items. Build a routine, a super cycle, around your sprints with the boss and team to discuss bigger goals. Then, during the cycle drive with the goals. If there is a suitable backlog items that contributes towards the goal(s) then do it. If not, write it out and do it immediately.

Either way, whether you are doing this to work around a boss or as a mechanism to tradition to a backlog-less world the same idea applies: create a super cycle, set goals every 10 weeks (approximately) and then drive through the goals rather than the backlog.

Drive with the goals and put the backlog in the back seat, it is secondary. The aim is to avoid nuking the backlog by letting it fad into irrelevance.

Write an “use by” expiry date on new backlog items.

For any item you do add to the backlog make sure it has an expiry date. That is: a date after which it will be removed from the backlog. Giving every backlog item a life expectancy won’t help you today, but it does mean that in the months ahead some items will “self delete” from the backlog.

After a while you might want to visit older items in the backlog and (if you can’t delete them immediately) assign them “use by” date.

I am sure a few people will say “O this need will live for ever.” In which case you can put a long date on it, say 10 years. But that also tells you: there is no urgency. You can do this item anytime in the next 10 years and add value, it can wait.

Better still, write a “best before” and a “use by” date on the item.

The best before date will tell you the date by which an item should be done to maximise benefit. After that date the benefit will decline, it might still be worth doing but it is not as beneficial. The “use by” date now tells you the date on which it has no benefit.

Now when you are reviewing the backlog you can see which items can be pushed back and which items need doing sooner. This is a little bit of a double edged sword for the requester, if they say “If I don’t have it in 2 months it will start loosing value” then it is more urgent, but also if it doesn’t make the cut soon then it can be removed soon.

Keep an ideas list

When you have a great idea, or when someone suggests something you could do add it to an ideas list – not the backlog. In fact, don’t show the list to people, don’t promise anything to anyone. This is your list for things you don’t trust to your memory.

Some people operate their backlog like this already but many people assume that if an item is in the backlog it will be done one day. Others measure the backlog and forecast dates. But ideas may never be done so they complicate this thinking.

Personally I like to think that great ideas will either get remembered or rediscovered. That said, I also write down lots of ideas. However, I frequently throw my ideas lists away. So if at 11pm I think of something I’ll write it down, three weeks later if it is just moving from one list to another I’ll cross it off. I rewrite my “todo” and “priorities” lists regularly.

If it helps then just keep ideas some other place. One team I worked with created a “Sprint 99” – a sprint so far off in the future it was never going to done. They parked all the good ideas there so they had them for reference and if they needed them but there was no suggestion they would ever be done.

You will also want to think carefully about what you tell people when they say “I’ve got a great idea, can you add it to the backlog?” You want to be honest, but you don’t want to create a long conversation. So you might want to say something like “Great, thanks, I’ll add it to our ideas list, if it becomes a block please come back to me and we can talk some more.” In other words, put the ball back in their court to show the value of the idea.

I’ll admit I’m nervous about this suggestion. Part of me things it will inevitably become to be seen as a backlog. I think important things will get remembered, and things which aren’t important can just as easily get lost when put among 1000 other things. Still, I know some people will take comfort in this idea so give it a try and let me know.

This pieces was first published on Medium, Nuke the backlog.

Nuke your backlog? Read More »

Objective Driven Agile, BLDD & Reactive styles of development

Agile / / / , , ,

Looking across the state of teams today I can clearly see three different styles of agile: Backlog Driven Development, Reactive and Objective Driven Agile. All have their place, all have their uses but the first two of these are a dead end. I’ve been speaking about the third, which I’m calling Objective Driven Agile, for years but in the last few months, since writing Succeeding with OKRs in Agile and speaking to many people about how OKRs and agile fit together, it has become clear to me that ODA allow us the industry, and teams, to address a number of problems that have arisen with agile in recent years.

1) Backlog Driven Development – BLDD. Teams are typically working in a Scrum fashion with a backlog and short sprints. Such teams are sometimes called “feature factories” because they are aiming to “do the backlog” – and the backlog is inevitably a lis of features.

While the Scrum these teams practice is a lot more agile than what came before they haven’t really moved very far from the big requirements documents of old. Backlogs are frequently bottomless pits, the team can never reach done. The loss of change review board might well be making things worse because Product Owners often lack the authority to say “No” or remove items from the backlog.

Still, not only are these teams agile but in many places this is success. The team are churning out stories, customers can see progress and are persuaded to part with cash. If the backlog can be tamed they might even declare done one day. But then, if the team is being operated by a consultancy or other outsourcer for a client the last thing they want to happen is for work to be done.

2) Reactive. Reactive teams gravitate towards Kanban but some are compelled to work in a Scrum fashion which doesn’t match their real work. Such teams may see backlogs are an anathema because they do “what is needed, right here, right now.” Team members here will say “true agility is listening to what customers want and getting it done as quickly as possible.” And in a way they are right.

Again, these teams are a success story, particularly if they are supporting a live system, running DevOps or if customers previously had to wait months for results.

The problem here is that these teams don’t have much capacity for doing the backlog or anything else and yet they are frequently still expect to “do the backlog”. If such a team have been tasked with immediate customer support then that isn’t a problem but frequently these teams face competing demands.

3) Objective Driven Agile, ODA: this is the approach I’m focused on at the moment. With Objective Driven Agile teams have a mission and a higher purpose, something more than “do the backlog” or “keep the system working.” Such teams might be considered “Empowered” and might practice “Dual track agile.” The key thing is, they are not beholden to a prepared list of things to do. The team are responsible for deciding what customers need, what will add value and what will meet team and organisational objectives, and then, delivering that thing.

These teams have “Product Owners” who are more than mere Backlog Administrators – I’ve started calling them “Product Specialists.” These people are tasked with understanding what will add value for customers, users and other stakeholders. Importantly they have the power to decide what gets done and to say “No”. With that power comes responsibility: responsibility to the team, to stakeholders and to other leaders in the organization. That means Product Specialists can explain – in a business case or in a star-chamber – how work on the product adds benefit and how the desired outcome – the objective – is moves things forward.

All three of these styles have their advantages and disadvantages but the real problems occur when the styles are not clearly stated.

Consider the team trying to deliver the backlog BLDD style but who have to “keep the lights on” and support a live system. It is mathematically impossible to make accurate forecasts about delivery when events can derail you. It is also nye on impossible to deny customer requests when they are waiving a large sum of money or escalating their ask through your organization. But again, this can blow up any plan or roadmap.

Nor is is possible to pursue objectives when teams are committed to supporting live or delivering a backlog. At best the objectives duplicate the role of the backlog and at worst increase work-in-progress and add stress to individuals on the team.

So my advice: decide which type of team you are and focus.

In the last few months I’ve been speaking a lot both about objectives and “just in time backlogs”. I’m fired up and want to pursue this idea, if you are interested please let me know and I’ll write more.

Subscribe to hear more from Allan Kelly and download Continuous Digital for free

Objective Driven Agile, BLDD & Reactive styles of development Read More »

Backlogs, #NoBacklog(s) and comfort blankets

Agile / / / , , , , / 3 Comments

At the first Agile on the Beach in 2011, I had dinner with Mary and Tom Poppendieck. As we talked about the conference, agile, lean and everything else I distinctly remember Mary saying “From a lean point of view backlogs are a problem. In a lean environment when you have a backlog you want to eliminate it.” (Postscript: Mary has spoken about her views publicly “Why You Should Just Burn Your Backlog“.)

Just then, I was then working with a client with a runaway backlog. I had calculated the backlog growth and found it was regularly greater than the work done “velocity.” It was like a mortgage were the monthly payments didn’t even cover the interest. If I remember correctly, backlog growth, interest, averaged 8.5% per month. I suggested to the client and their supplier that they throw the backlog away. They, not for the first time, thought I was mad. Since then I’ve voiced the same opinion with other clients and got the same response. But the opinion is gaining ground and I’ve even encountered a couple of places were they have moved away from the backlog.

To be fair, backlogs are useful – they can have a useful role to play but that role is akin to a child’s comfort blanket. There comes a time to say goodbye to childish things.

(By the way, I’m really discussing what Scrum calls “the product backlog” rather than the ever changing “sprint backlog.” So I mean: the stuff we might work on in future and not: the stuff we are working on this week, and next.)

I’m not, yet, ready to join Vasco Duarte in declaring #NoBacklogs but my “nuke the backlog” comment in a podcast with Jenny Herald said pretty much the same thing. That comment has attracted a lot of attention and seeded good discussions. I like those discussions and thats why I resist about #NoBacklog. When we started using #NoProjects it was good for conversation, but then a few people drowned out the discussions with calls of “You are mad”, they never considered our arguments and closed down discussion for others.

So what is the backlog problem?

First, as the graph above shows: backlogs don’t “burn down” they tend to grow, and often grow faster than work is done. That becomes a problem because many people expect the backlog to reduce to zero over time and organizations consider success (“Mission accomplished”) to be a completed backlog. The cost of adding something to the backlog is near zero, but the cost, to the product owner and/or team, of refusing a backlog item can be high – all the time spent explaining why something won’t be included.

The desire to “do the backlog” leads to the second problem: an emphasis on doing backlog over delivering value. It becomes more important to do backlog items (output quantity) then deliver benefits (outcomes.)

That, combined with the ever increasing number of items, leads to problem three: a loss of strategy and sense of purpose. This is classic “can’t see the wood for the trees.” There are simply so many backlog items to do it is hard to see what should be done. (The whole “twice the work in half the time” idea that surrounds Scrum makes this worse still. Raising the outcome over output question will also get you called mad.)

Along the way a lot of stakeholder problems get created because people believe that an item in the backlog is in some way promised when it isn’t. Product Owners and Teams accept items into the backlog for an easy life even when they know it is unlikely to ever get done. This stores up future problems because stakeholders start complaining when they fail to get their items. That damages trust in the team and a vicious circle ensues.

It gets more and more difficult to prioritise anything: more items to consider, more stakeholders to placate, more promises to (pretend to) keep. This makes it increasingly difficult to follow the benefit and change course and act on product feedback.

One of the reasons I came to like OKRs, despite my initial scepticism, was that they provided a means to think about the backlog and eventually move away from it. Another reason why I am avoiding #NoBacklog is because I want to be able to offer an alternative rather than just rubbish the backlog. At the moment I think OKRs are that alternative but I’m a little bit reluctant to force OKR Kool-Aid on people.

I tell a story in Succeeding with OKRs in Agile about a little experiment I conducted with another agile coach. The question was “Are OKRs written based on the backlog you intend to do in the next quarter? Or, are OKRs set in respect of business strategy and product priorities and backlog items selected, or even created, to meet the OKR?”. The experiment showed us that OKRs should come first and the backlog was secondary.

Perhaps backlogs are, as I think Vasco thinks, a hang over from the project days. A similar point is hiding inside Project Myopia: in the project model success is doing all the backlog, but if you prioritise by business value, if you are responsive to customers, if you are practicing dual-track agile and product discovery then you may well find that not everything in the backlog is valuable or even wanted by customers.

Increasingly I view Product Backlogs as comfort blankets – what psychologists call transitional objects. Like children’s toys they help us move from one view of the world to another. When starting with an agile style of working backlogs provide comfort by mapping work to the traditional (project) model, but in time, as you become better at listening to customers and responding they are a hinderance.

I’ll be talking more about backlogs and comfort blankets in an online presentation next week to Berlin Product People, join me there to hear more. (5 September 2022).

Subscribe to my blog newsletter and get Continuous Digital for free

Backlogs, #NoBacklog(s) and comfort blankets Read More »

How fresh is your backlog?

Agile / / / , , , / 1 Comment

Do you struggle with an overwhelming backlog?

Do you count the number of product backlog items in your backlog in tens? hundred? or thousands?

Does your backlog contain many stories which have been there for months, if not years, and yet never raise to the top of the backlog?

Is your success judged on your ability to do the backlog?

Backlogs were a good idea when they were introduced a bit over 20 years ago but today many teams slaves to the backlog – see my posts on the Tyranny of the backlog and Purpose over backlog. One of the benefits I’ve called out for OKRs is the ability to move away from backlog driven development (BLDD).

In Succeeding with OKRs in Agile I ask suggest you either need to prioritise your backlog over OKRs (in which case OKRs are derived from the backlog you intend to do) or OKRs over backlog (in which case OKRs are derived from strategy and the backlog plays a supporting role.) In my podcast with Jenny Herald earlier this year I even say “Let OKRs drive… nuke the backlog.”

Filipe Albero Pomar recently shared his backlog freshness blog which I think is great. Freshness is a great way to think about the state of the backlog that separates the size of the backlog from the relevance of the backlog.

Filipe’s idea is simply to talk about the backlog in terms of freshness – you have a fresh backlog if your backlog items are fresh: written recently, relate to current opportunities, problems and things people currently want.

And of course, the opposite of fresh: stale, stories that have been sitting in the backlog for months, even years, stories that relate to yesterday’s problems and project, stories which people wanted last year. The existence a big backlog of stale stories means the team is seen to be not delivering, the end-date is far off because people still expect all the work to be done.

Filipe suggests backlog freshness can be measured:

1. Set a cut-off date

2. Categorise stories as fresh or stale: fresh stories have been written since the cut-off date, those which are older are stale

3. Calculate freshness as a percentage of fresh from the total: if 25 stories out of 60 have been written in the last month then the backlog is 41% fresh, and 59% stale

Thats a useful metric, I think we can do better, look at the graphic above. I group backlog items into age groups and graphed them. For completeness I added a line to indicate average story age. Clearly this backlog is not fresh – nearly half the stories are over a year old.

I like the idea of graphing backlog freshness because it is easy to understand and makes an impact. In the graph above I’ve categorised backlog items into age groups and added an average line. Clearly this is not a fresh backlog. Whether this is the way to demonstrate backlog freshness I’m not sure – I’m playing with a histogram and quartile ranges.

With some clients I’ve thought of the backlog like a mortgage. There is the principle (the existing backlog), the interest rate (the growth rate of the backlog) and the monthly repayments (stories reaching done). Unfortunately when you do this you sometimes find the mortgage will not be paid for many years, and perhaps never. (Don’t worry about estimating the size of stories, for this sort of analysis the number of stories will get you started, and if your backlog is measured in hundreds of items the small will offset the large.)

I’d love to talk more about this and experiment with some ideas, I think it could be a very useful way of thinking about the backlog.

Subscribe to my blog newsletter and get Continuous Digital for free

How fresh is your backlog? Read More »

Purpose over Backlog

Agile / / / , , / 2 Comments

Backlogs are a good idea. Backlogs ease the transition from the old “requirements up front” world to the new more dynamic agile world. Backlogs provide a compatibility layer for agile teams to interface to more traditional project management and governance. Backlogs even allow you to take a stab at done date!

Backlogs allow you to even out work between the quiet periods and the busy times. Backlogs give you a place to store good ideas which you can’t do just now. And because stakeholders can see their request is not forgotten they don’t need to shout for it today.

Listen to Allan Kelly reading Purpose over Backlog

Yes backlogs are good. I’ve seen them work well myself and I’ve taught many teams to effectively use backlogs.

But – you knew there was a but coming didn’t you? – but…

Backlogs have problems, too many teams are labouring under the Tyranny of the Backlog, they have become backlog-slaves and practice something we might call BLDD – Back Log Driven Development.

(To be clear, when I say “backlog” I am primarily thinking of the product backlog – the long list of all the things the team (might) do in the future. This is different to the sprint backlog (iteration backlog). The sprint backlog is a shorter list of things the team aims to do this iteration. I am using Scrum terminology but the ideas are pretty much “generic agile” and I’m thinking more broadly than Scrum. Many implementations of Kanban feature a product backlog of sorts so while Kanban is less prone to these problem it is not immune.)

1) Lump of Work Fallacy

There is usually an assumption that the backlog represents all the work to be done – an impression reinforced by early implementations of Scrum. In the short term that leads to agile teams being seen as inflexible and prioritising process over need because new work is not allowed in.

In some cases teams even struggle to get started on work because a big-up-front requirements gathering and analysts activity is required to create a backlog. In the worst cases that work is even estimated and end-dates forecast before a line of code is cut or developers hired.

In the longer term it is simply unrealistic to assume the backlog is fixed. Even with more and better analysis it is impossible to foreseen future requests. The agile adage “it is in doing the work that we understand the work” cuts both ways: coders understand what they need to build and customers/stakeholders/analysts understand what they want.

Work will arrive after you begin, any system that does not incorporate that truth will fail one-way or another.

2) Bigger than you think

Not only does the backlog grow with completely new work the work in it changes – and grows. There are many reasons this happens: new opportunities appear, hidden ones become clear, requests require more work than expected and so on.

Humans are very bad at estimating, especially about the future, and, it turns out, they are also very bad at estimating time spent in the past. If you want accurate forecasts you need to invest in them, you need to make structural changes and you need to use statistics.

However, because of the lump of work fallacy and the belief that humans can make estimates, poor end-date projections get made and when they are missed (because they were wrong to start with) everyone gets upset.

3) Fallacy of Done

Backlogs come with burn-down charts and an assumption that there is an end; and that end is when everything is “done.” The team will be done when the backlog is empty. That assumption is baked into BLDD, traditional project management and even governance.

I have long argued that software is never done. I’ll accept that I might be wrong, but in the digital age, when business runs on digital technology (i.e. software) your products are only done when you business is done. The technology is the business, and the business is the technology. Stop the backlog growing, stop growing you technology and you kill the business.

4) Backlog Bottomless pit

Put all those reasons together and the backlog becomes a bottomless pit. In the early days of agile, when I managed teams myself, the backlog would often sit on my desk, written out on index cards and held together with rubber bands. I could get a sense of how big the backlog was my looking.

Today everyone uses electronic tracking systems. Not only do these allow an infinite number of items they rob us of perspective. To paraphrase Comrade Stalin: “2 outstanding backlog items is tragedy, 200 is a statistic.”

5) Backlogs obscure strategy & purpose

With so many backlog items it is easy to get lost – you can’t see the wood for the trees. Arguments over what will be done next start to resemble deciding who should get a lifeboat place on a sinking ship, add in the demands “when will you be done?” (plus explaining why the date has changed) and “the bigger picture” gets lost.

In Back Log Driven Development the sense of purpose and strategic goals is lost as teams struggle with the day-to-day demands of just doing stuff.

6) Powerless product owner (i.e. backlog administrators)

Tyranny of the backlog seems worst where product owners lack real authority and skills. They are little more than backlog administrators. They spend most of the week adding requests to the backlog, then passing a few chosen items to developers in planning meetings. A vicious circle develops, the product owner can’t win so people trust them less, their authority wanes, and the backlog spirals.

Few organisations give product owners the power needed to get a grip on this situation. Indeed, many product owners are plucked from the ranks for development or support and given a battlefield promotion to product owner but lack the skills required. (See The problem with Product Owners.)

A solution?

For years I’ve been suggesting teams throw away the backlog – you will not forget the important things. But then how do you know what to do?

Take a step back, start with your purpose, your mission, the reason you team, your company, your organisation exists. What should you be doing? How can you fulfil that purpose and serve your customers?

This is where I see a role for OKRs and jobs to be done. Both these techniques – together, or separately – can be used as story generators. Every time you need to more work, more stories, you return to your OKRs and ask “what can we do now to move us towards our objective?”

When writing Succeeding with OKRs in Agile I became more and more convinced this is the path to take. Increasingly I sum this up as Purpose over Backlog.

Post publication addition: great suggestion fro Jari Mäkelä on Twitter to call this: Purpose Driven Development, PDD.

Step 1: Clarify your purpose – what is your overarching reason for existing?
Step 2: Clarify how your existing strategy builds towards that purpose, and if you don’t have a strategy create one.

Repeat steps 1 & 2 annually.

Step 3: Think broadly, set your OKRs as a team so you build towards your purpose by following your strategy.
Step 4: Spend the next 12 weeks executing against those OKRs

Repeat steps 3 & 4 every 3 months.

Step 5: In each planning meeting take stock of what you have done and progress against OKRs
Step 6: Ask “what do we need to do next to move towards the OKRs?”

Succeeding with OKRs in Agile

Repeat steps 5 and 6 every 2 weeks

And if you are Kanban’ing then keep steps 1, 2, 3 and 4, adjust 5 and 6 as appropriate.

Having finished, completed, published Succeeding with OKRs I really wish I had been clearer in the book. The ideas are there but with time they have become so much clearer… maybe I need another book.

Buy Succeeding with OKRs in Agile at Amazon today.

Subscribe to my blog newsletter and download Project Myopia for Free

Purpose over Backlog Read More »

Verified by MonsterInsights