Agile

This blog has been noticeably quieter than usual over the last couple of months, and whats more, when I have posted something up its often been of the “this is what I’m doing” type rather than the “here’s something to think about” type post. Well…

In the last few months I’ve been putting my energy into other mediums, hence A little book of requirements and user stories and the managing beyond projects workshop I’ve mentioned before – the dates for the workshop has moved back to January by the way.

The big news this week is that I am serialising my #NoProjects presentation as mini-videos on YouTube. Various conference videos are already online but this is a series of short videos, check out the Managing without Projects playlist. (There are 7 episodes so far, more coming soon.)

I also added a new newsletter earlier this month and I expect this will in future carry material which would have been in this blog otherwise. So if you want the latest, freshest stuff I suggest you subscribe to the newsletter.

And here is the piece that the newsletter carried at the start of this month….

What’s next? – after Agile?

As this is the first of my new, occasional, newsletter it seems a great chance to look to the future. So lets ask that question that comes up regularly:

“What comes after Agile?”

Sometimes only asked rhetorically by agilistas to prove they have the inside track on what is really happening!

I had a go at answering it myself a few years ago with a presentation entitled “The future of Agile”. To cut a-long-story-short, I thought the next thing would be Lean. Six and a bit years on I think I was right but perhaps not in the way I thought it might be. I saw Lean displacing Agile as the predominant approach to software development, or at least the buzzword. Well it hasn’t, Agile is still the buzzword it was.

But Lean has permeated more and more thinking on Agile. If you look at Agile almost every idea can be traced – in some cases literally in other case philosophically to Lean thinking. Anyone who goes a bit deeper into Agile quickly runs across the Lean roots. Over time we’ve seen more and more Lean ideas adopted under the Agile umbrella.

My own Xanpan book is an example of that: hard core XP agile infused with Lean Kanban thinking.

Talking of Kanban…. six years Kanban was still new, and it seemed for a while that the Kanban community didn’t want to be part of the Agile community. They wanted to be seen as Lean. (One supposes they thought Lean was superior, or at least more marketable.) That seems to have changed too, I now hear expressions like “Kanban is an alternative path to Agile.” I feel there is a broader understanding that while Kanban is different to Scrum it is not so different that it isn’t part of the Agile movement.

(Perhaps the greatest contribution the Kanban insurrection made has been breaking the Scrum hegemony. Scrum is still often used as a common synonym for Agile but it is no longer seen as the only player. Because of Kanban people are more aware of differences.)

Back to my original question.

When people ask “what is next?”I usually to sense there is something else they are not saying, perhaps what they are saying is: “Agile destroyed the waterfall, what is coming along to destroy Agile?” – after all this is the tech industry were new technologies come along regularly. Cynically I sometimes wonder if these people are thinking “Maybe I can ignore Agile and get with the next thing” or even “Agile is a fad, give it a few months and the wheel will turn, we’ll be back to waterfall.”

Notwithstanding cynicism, it is wrong to think something will come along and disrupt or destroy Agile, the way Agile did Waterfall. Many more innovations build on what has gone before than destroy what has gone before. The question “What comes after Agile?” is better asked “How will Agile evolve to next?”

That is an easier question to answer because the future is here: *Continuous Delivery, Mob Programming and No Projects*. All three build on Agile and takes it to a higher level

Continuous delivery: Teams delivering not just at the end of the iteration but all the way through it. In some cases many times a day or even many times an hour.

Unfortunately for those people who were hoping to leapfrog Agile and get with the next big thing the first step to doing Continuous Delivery is doing Agile well. If your team can’t release a product update at least every two weeks then you aren’t even at the start of Continuous Delivery.

The path to the continuous delivery thing lies through Agile.

Then there is Agile’s cutting edge, the new ideas which will build on Agile and take it further.

On the technical side mob programming and BDD (behaviour driven development) continue to build.

On the process and organization side “#NoEstimates” continues to stir up controversy although I fear it does as much damage as it good.

And of course there is #NoProjects which I am closely associated with.

My thinking on #NoProjects continues to develop and it continues to be a popular conference talk. Now I want to try something different. Next month I intend to run a one day workshop on the subject to help teams explore how they may get away from project thinking. More details and how to book on EventBrite.

As mentioned a few weeks ago I’m compiling my recent Agile Connection series on User Stories and Requirements into a LeanPub book.

I’ve now released the first couple of chapters, you can get them from LeanPub right now!

I intend to add more chapters over the coming weeks. The bulk of the material will be straight from the Agile Connection series, perhaps with a little more editing. There will also be some off cuts, bits that didn’t make it to Agile Connection series either because they were blew the word limit on another piece, were too short to stand lone, or didn’t quite fit in. There is stuff you can do in a book because it is large and different.

My intention is to up the price a bit as I add chapters. Its really annoying now that the UK (and I think all of Europe) charge VAT on e-books. Seems a bit unfair that printed book don’t attract VAT but ebooks do.

If you have any comments on the book, the content or things you think are missing please let me know.

A few weeks ago I tweeted:

“Large companies trying to be Agile remind me of middle aged men buying sports cars”

I wasn’t saying large companies couldn’t be Agile – heaven knows most are trying and a few have successful software teams but on the whole the successes are few and far between. My thinking has nothing to with whether they can succeed (they clearly can, but most fail), or whether we should try and help them (yes, I’ve helped a few in my time, and I’ve seen a few failures) but rather my thinking comes from Peter Drucker:

“Large organizations cannot be versatile. A large organization is effective through its mass rather than through its Agility.” Peter Drucker, Age of Discontinuity

Peter Drucker wrote that in 1968, long before anyone ever thought of applying the word “agile” to software or business.

What I’m getting at is: for most large businesses the things Agile requires go against the grain of what has traditionally made them successful. For example…

Big businesses use economies of scale to extract favourable terms from suppliers. But software development doesn’t have economies of scale, rather it has diseconomies of scale. Applying economies of scale thinking to software development and agile hinders it.

Big businesses (to achieve economies of scale and management) often favour standardised procedures and processes: these bring some benefits but they reduce variation (not good in software development), reduce experimentation (required if you are going to try something new) and discourage risk taking. Sure standardisation brings benefits but the benefits they bring are opposite of what agile thinking would suggest.

Big businesses have lots of customers who are customers because they are customers – inertia. For example changing your bank is so much trouble and effort very few people do it past the age of 25. All that Agile talk about MVP, product management, customers and so on doesn’t matter when you are dealing with existing customers who stay through inertia. You biggest risk is upsetting them enough to make them move.

Big businesses are inherently risk averse: the risk of upsetting customers, suppliers and shareholders is greater when you have more of them. Yet many agile practices look very risky at first, even if you understand logically that they are risk reducing simply doing something your peers don’t is a risk.

For example, image a manager who buys into the logic of continual delivery (CD) and sees how it reduces the risk of releasing software. If he alone adopts CD for his team – when all the other managers stay with the annual release – and something goes wrong then he will be seen as at fault. It leads to defensive decisions and an aversion to change because change is risk

And big companies have more people so there can be a herd mentality. Consider that example again, it there are only two development managers in the company and one decides to adopt CD then half the managers do CD and half don’t. If however he has 10 peers then there are 10 who take a different decision, i.e. stick with the thing that always “worked” in the past and one, him, who changes.

Big businesses often grow through acquisition. Acquisition is often the modus operandi of big companies: buy similar companies, buy their customers, extract economies of scale, install standardised procedures and remove differences. Thats how you make a merger work if efficiencies are your goal. But those things fly in the face of what an Agile company would do.

Some big businesses have got to be big by squeezing the fat out of processes and inventories, i.e. remove the slack. But without the slack there is no room to manoeuvre – remember queuing theory, once utilisation rises delays increase and variation is bad. That makes change more difficult and also makes practices like reducing WIP difficult to implement.

One way big companies succeed is to define roles and responsibilities, indeed hand-in-hand with economies of scale goes economies of specialisation. That is, as organizations (even teams) get bigger it becomes economic to have specialists, like business analysts, in particular roles. But again this cuts against what an agile company would do. Agile companies value multi-skilled/multi-functional people who can adapt to what is required. This approach also helps deal with bottlenecks; specialisation often creates bottlenecks (even met a test manager who carefully schedules his testers so nobody is ever underused?)

I could go on but hopefully you see what I’m saying here: the things that make a company successful at being big are often the same things that make it difficult for the company to be agile and, importantly, vice versa.

Size is inherently un-agile.

Thats not to say size is bad, size brings many advantages but these are not the advantages of agile. Trying to be big and agile is an example of having your cake and eating it. The few companies manage to achieve both really are the exception that proves the rule.

Hence: a company gets big and successful, like a middle aged man who has a good job, and in an effort to reclaim the nimbleness of youth embarks on a quest for a fast car and younger wife (agile), while forgetting that middle age brings its own benefits, notably in this context knowledge and experience.

Now, the question that lurks here is: if a large company becomes “agile”, that is, if it changes many of the things that have made it successful as a large company in order to achieve agility, then: does make sense to remain a large company?

Possibly by dismantling the things that make big successful and replacing them with practices and processes which allow agile to be successful the company has turned itself into little more than a disjoint conglomerate. In which case it may find better economies, and a better stock market valuation, by splitting itself up.

And that in turn suggests that any framework for “scaling agile” has a number of challenges to overcome if it is to succeed.

Finally, I don’t mean to be critical of people who try to help large companies, in many ways these people are taking on the most difficult tasks and deserve our support.

And for completeness, I should say: if you are a big company out there wrestling with these problems please feel free to call me, my rates are very reasonable!

In the last two posts I’ve discussed some problems with of self-organizing teams and highlighted the need to be clearer about what is actually meant when talking of, that is naming, self-organizing teams. At a minimum the labels need clear definition (I suggested some definitions and I hope someone knows some better ones.)

I went further and I called for individuals and teams to be given authority so that timely decisions can be made with maximum knowledge. I’m sure everyone agreed with me up until that point and then I said we should stop “empowering” people, I said I hated the term empowerment and that probably confused a lot of people.

Now I’d better explain myself.

Actually my thinking comes directly from Henry Mintzberg so I’ll let him explain:

“empowerment [does] not change that [participation] because the term itself indicate[s] that the power remain[s] with the manager” Mintzberg, Managing, 2009

Mintzberg argues that the practice of empowerment leaves real authority, real power with the manager, when a manager “empowers” a worker he is offering him a gift, or rather a loan. It is the manager who chooses to empower the worker, by implication the manager may choose not to empower the worker or to remove the power at a later date. The very act of empowering a worker actually emphasises the lack of worker power.

Elsewhere in the same book Mintzberg makes a good point:

“Truly empowered workers, such as doctors in a hospital, even bees in the hive, do not await gifts from their managerial gods; they know what they are there to do and just do it. … people who have a job to do shouldn’t need to be empowered by their managers”

A little later Mintzberg offers a key insight:

“a good deal of what is today called ‘empowerment’ is really just getting rid of years of disempowerment.”

Its not about empowering software engineers to improve the build system, or testers to automate scripts, or analysts to do stakeholder mapping; its about trusting them to do their job right and letting them do it. These are the experts in their field, let them do what they know to be right, indeed, go further: insist they do the right thing.

So if you are a manager stop empowering people and teams, instead stop disempowering them. Give them the trust, authority and support to do their work.

And if you are a worker just reach out and take authority. One of MIntzberg’s better known peers, Tom Peters, like to quote the American soap star Rossanne Barr:

“Nobody gives you power, You just take it” Rossanne Barr quoted in Re:Imagine! by Tom Peters

Let me leave you with a classic pattern from Joe Begin, Do The Right Thing:

Things are bad. Really bad.

        •        When things are bad it is really tough and bad things happen.

        •        When things get better the bad stuff doesn’t happen any more and you feel good. Really good.

Therefore: Do the right thing. Make the bad thing better.

The result: Things are good. Really good.

Known Uses:

        •        When you were small your father would make the Monsters Under the Bed go away just by sticking his head in your room. He did the right thing.

        •        When you are really sick, eat your Mom’s chicken soup. Only your Mom’s. Only she knows how to do the right thing.

Don’t wait for someone to give you authority or empower you, do the right thing.

Quick as a flash Eben Halford on Twitter pointed out the mistake with my last blog (Question for self-organizing teams). I was mixing up self-organizing teams with self-directing teams. Well maybe I was… much of my post still stands either way, and as Eben himself pointed out, we might be trying to split a hair here.

Frankly, I suspect many people think the whole “self-organizing team” thing is one-size. The idea that there are actually “self-organizing” and “self-directed” and possibly “self-managing” hasn’t occurred to most people and, if they have noticed, they don’t know the difference. In fact, I’m not sure I know the difference!

If you take time to look at some of the literature on self-organizing/directing teams you find that very little of the pre-agile stuff talks about self-organizing teams, rather the common term in management literature is self-managing teams, most of the serious research relates to this rather than the (to my mind) weaker idea of self-organization.

To untangle this mess lets define a hierarchy here:

• A self-organizing team is a team where team members get to decide among themselves who does what, the team gets to work on problems and have some power to remove their own blockages. Clearly there are teams who are more self-organizing than others and teams which have more authority than others.
• In a self-managing team there is no active day-to-day management of the team. The team are effectively left to manage their own work. To my mind this is a stronger form of self-organizing.
• A self-directed team is a team which sets its own goals, decides its own objectives and determines its own priorities.

Does anyone know of any serious literature (i.e. research led) which defines these terms better? I’d love to have some better, clear, more well defined, separation.

Clearly some, but not all, of the questions I posed in my last blog relate to self-directed teams rather than mere self-organizing teams.

But while I have, belatedly, made this distinction it seems to me that not everyone does. It seems to me that making this distinction would help by removing a lot of the confusion that sounds self-organizing teams. And making this distinction would actually help with the questions I posed in the last blog post because teams and their managers would be able to draw lines and discuss where authority and responsibilities lie.

You see, one of the problems with labels, especially labels like self-organanizing teams, is that they mean different things to different people. Indeed different authors define them differently.

It is because of this confusion that I prefer to avoid using these labels and prefer to leave the idea of self-organizing teams alone.

Instead I prefer to talk about authority, I want team members and teams, however they are organized and managed, to be given authority. Sometimes authority is vested in the team members equally, so for example each member has the authority to make a decision. In other case authority is vested in the collective team, in that case as long as the team can agree a decision they can make it,

In other cases authority is vested in specific individual team members. This usually occurs where specialist skills or knowledge are needed. The Product Owner role is a great example here: product owners need to have the authority to make decisions on prioritisation, they need authority to make trade-offs and make compromises. The more authority you give the product owner the better they can do their job.

I want teams and individuals to have authority for two reasons. I believe the soft, fuzzy, reason that is usually sighted: people get more satisfaction from work when they have autonomy (i.e. they can make their own decisions) and thus are more motivated and more engaged as workers.

The second reason is very hard, not fuzzy: when developing software there are thousands of decisions that need to be made. Many of the decisions require specialist knowledge and the people with the greatest knowledge, both of the technology and the situation in hand right now, are the people doing the work. The testers and programmers at the code face have more information about what needs to be decided than anyone else.

Pushing decisions down to the lowest level means decision making can be made more efficiently, i.e. in a more timely fashion. But in order to do that the workers need authority.

Now notice I’m saying authority, I’m deliberately avoiding the word “empowerment”. Empowerment is a horrid word and it describes a horrid concept. Don’t talk to me about empowerment, its nasty.

On that cliff hanger, if you want to know why I don’t like empowerment, well tune in next time, the post is already drafted.

Try this thought experiment.

You are a software development manager. You learn about agile and you think it is good. You adopt agile and you make all your teams into self-organizing teams. (Leave aside the question of whether you then quit in a fit of “no managers needed” – we can talk about that later.)

Most of your teams work well. Productivity goes up, people are happier. They organise their own work. Occasionally teams come to you and say they would like extra people on the team. Provided they can back this up with evidence that they are productive and add value, and that adding more members will increase the value they deliver then they are allowed to expand.

But, one team doesn’t perform as expected. They don’t achieve the productivity of the other teams, they don’t demonstrate the value other teams do.

Question: Do you intervene?

If you intervene you are not allowing them to self-organise, you are playing big bad manager.

If you intervene in one team what will the other teams think?

Perhaps its not as clear as that. Perhaps the team are performing but it becomes clear they have achieved a local optimum and are not improving like other teams.

Question: How do you nudge the team to move to a higher performance level?

And what if when you challenge them about things like pair programming, honouring deadlines, maybe predictable velocity, etc. etc. they tell you they have tried these things, or at least talked about them, and they have decided that these are not good for them.

Question: What do you do if they self-orgianise to not do the things you hoped they would? Things you think might improve performance?

(Potentially if you allow a team to self-organize they might even start requesting complete requirements documents up front, they may resist change requests.)

Now suppose that rather than play big bad manager you play facilitator, Uber-Scrum-Master, you go to the team and say: “I think this team could perform better, I’d like to help you improve.”

And someone on the team says back: “Could you please give us an example of when you think we could have performed better?”

Now you have a specific instance in your mind where the other teams (the performing teams) think this one messed up. Do you tell them? To tell them is negative, to tell them may make them defensive, to tell them really make you seem like a big bad manager.

You could hire an agile coach, scrum master or such and ask them to do pose these questions and make these changes. But isn’t that just delegating? You are still the power behind the action.

You could go to the other teams and ask them to challenge the under performing team and raise concerns with them directly but that might be seen as manipulative, and now you are interfering in multiple teams. Are you allowing them to self-organise?

Now suppose things take a turn for the worse. The company looses money and the shareholders ask for redundancies.

Do you communicate this to the teams and ask for volunteers? – you may hit employment law problems here.

Do you ask everyone (yourself included) to take a pay-cut?

What is someone crunched the figures and found it was the underperforming team that was dragging everyone down, and cutting the whole team could return you to profit. Should you ask the teams to do this themselves? Or do you do it?

If the team crunched the figures would the underperforming team decide to offer themselves up for redundancy? Would it be fair it you – or the teams themselves – decided to cut one person from each team instead?

In short: if you have an team where the performance isn’t satisfactory what do you do?

I suppose the ideal is that someone on the team, probably a product owner/analyst type person, is regularly analysing the benefit delivered by the team and the cost of the team. If the benefits fall below the cost and the team can’t come up with a plan to change it then the team members request dissolution. But has anyone ever seen that?

I pose these questions because I don’t know the answers. It is questions like this that add to my scepticism over self-organizing teams, or rather, cause me to see limits in the model.

I find it hard to image any team offering to dissolve themselves. O, there might be the odd story of teams doing it but I find it hard to image this is the norm. I find it easier to image teams getting more defensive and seeking to explain away their failings by reference to others (I’m avoiding the word blame.)

Arie de Geus in The Living Company says that companies, and by extension teams, are living beings, they exist to continue their own existence. In capitalist coiuntries markets will eventually force poor performing companies out of business. In the closed ecosystem inside a company how do we replicate that mechanism?

I see limits in self-organzing teams: One of manager’s roles is to determine these limits and act at the limits.

One more thing, run this thought experiment again, but this time imagine that you (the enlightened manager) chose to dismiss yourself once you have set up the teams. The questions still stand, the problems still exist, but removing the manager removes the last person who might take action.

In the summer my training and consulting business gets quiet. That allows me to take some holiday and to pursue some stuff I don’t get much time for otherwise. This summer I’ve picked up a programming idea I started last summer, worked on over Christmas and still haven’t finished. This means I’m programming!

It is getting close to 10 years since anyone paid me to programme but I dabble a bit, I keep my hand in (a little). Right now I’d like to share some observations I’ve made. Programmers out there – please agree or disagree, just leave comment below. And if you haven’t programmed for a while then think about these points.

A bit of background, I’m working in Python (2.7) for the Google App Engine. It is about 2,300 lines of HTML and 4,000 of code and unit tests. I’ve set myself a challenge of doing this without a debugger. Just unit tests and some debug logging/printf’s if needed. I won’t have an IDE either, I’m using TextMate. Giovanni Asproni persuaded me to try PyCharm but the thought of learning another IDE stopped me installing it.

So,programming…

1. Its addictive – hard to stop, when you see your code working its hard to step back, not to play with the kids, not to eat and even not to go the toilet!
2. Sometimes it feels like your brain will explode, you are just trying to keep so many things in mind at one time and make sense of something (or somethings) new, and understand how it all hangs together and…. you just feel full.
3. It scares me – knowing that I will be so immersed scares me, I don’t pick it up and do little bits when I can be cause I know I’ll get sucked in.
4. It requires large chunks of time – its not like answering e-mail, its not something you can do 5 minutes here, and 20 there, you need a few hours at a time, partly because of #1 and #2 above.
5. Them tests – TDD development makes it so much more enjoyable, yes its hard to keep yourself honest but when you see a little bit more working it feeds the addiction; it also means you spend less time with those annoying little bugs and trivialities. Once or twice I’ve strayed from writing tests for too long and ended up in a mess.
6. Trivial tests Yes! – when I evangelise TDD I often get asked by people “But surely you don’t need to write tests for all the small, obvious stuff, getters and setters say”. (Leaving aside the “you should’t have getters and setters” argument for a moment). Its hard to argue with them. But its perhaps with the trivial stuff that I find TDD makes the biggest difference, perhaps in part because I’m in Python and not a compiled language. Its those occasions when you think “this is trivial I’ll just do it” that often catch you out. I am not thinking, I am on auto pilot, or, heaven forbid, I’ve done a bit of cut and paste.
7. No you can’t TDD everything (particularly in the UI or a third party system) but that means I go slower there, I’m taking more risk. I wish could test that stuff. Why should “not being able to test everything” be an objection to “testing everything you can?”
8. Bigger problems, multiple things: some of the bigger problems you face are where several things come together, and sometimes it far from obvious that they have! (This is the same as my entry in 97 Things Every Programmer Should Know, Two Wrongs Can Make a Right.)
9. Taking a break: sometimes when you are in the middle something, particularly if your brain is about to explode (#2), and/or you are convinced you are on the edge of completing something, or accomplishing a breakthrough (particularly if something is not working like you expect it to) then… you just can’t take a break. Not so much addition but the feeling that if you let go now you will never get it all back into your head again. But, big but, a break is just what you need. Only when you come back fresh can you actually make it work. Only a break will let you step back and think without the overload.
10. Nothing else: When I’m programming, when I’m on a roll, I don’t want to write more Xanpan, or continue the User Stories series, or chase down some leads for work. Yes I should do all these things but… programming is good. There is only so much time in the day. Wouldn’t it be nice if I had someone else to take care of those things? If I was a team I could…. all those programmers who want to do away with managers should think about this. If you do away with managers much of that management work still needs doing and now the programmers are going to have to do it!

Its easy to see what attracted me to programming all those years ago. The deep immersion and challenge, man v. machine, it soaked up so many of my teenage days – particularly summer holidays. Its perhaps fitting that it is mainly during the quieter summer months – when my training and consulting work isn’t so busy – that I get time to do a bit of programming.

Something else thats funny, when I’m programming I really like listening to old 70s and 80s music, stuff I listened too when I was a teenager immersed in programming. Guess I’m just a middle-aged man seeking his youth!

I’ve updated the Team Start Dialogue Sheet. Many thanks to Marta Kolenda who used the sheet with two of her new teams and sent me some feedback. Incorporating her feedback gave me a chance to make some more changes to the sheet too.

I’m always glad to have feedback from people and teams who try any of my Dialogue Sheets – getting feedback from Agile teams can be like getting blood out of a stone!

Plus I’m always keen to sit in and observe teams using the dialogue sheets – particularly the still new Customer Discussion sheet – so please invite me!

And while you are looking at the sheets… you might notice that Software Strategy – my company – has a shiny new website. Bblogs and books don’t pay mortgages so Software Strategy is my commercial side. If you know anyone who wants some advice or training on software development or Agile then point them here!