Agile

Continuing my theme of Principles, I’d like to outline my own principles of “Agile Software Development”. As I implied last time, these may be simply “Software development principles” but right now I’ll keep the “Agile” word.

1. Highly adaptability over highly adapted

Strive to make you processes, practices, team, code and environment adaptable rather than adapted. The future is uncertain and shows no signs of getting more predictable. Rather than try and predict the future and be adapted to it, strive to be adaptable to what ever may come.

Code design and architecture is a good example of this. Rather than spending a lot of time designing a system for the requirements you think the business wants – they may even aware blind this is what they want and sign in blood – accept that things will change.

Adaptability comes not from exhaustive contingency measures but from simplicity, reflection, learning and change.

Plan, design, for the short term but put in place mechanisms to roll with change. Don’t spend weeks designing, spend hours, be prepared to refactor, put in test frameworks to allow change. Sometimes there are architectural things you can do to allow for change – for example the plug-ins pattern – but these are fewer than you think.

Don’t implement this or any other architecture until you actually see the need for it. Don’t build it because you have a hunch.

2. Start small, get something that works, grow

In keeping with the principle #1 above and dis-economies of scale always start as small as you can, if things work then grow. For example: Start with the minimal viable product, if customers like you product add to it. Start with the smallest team you can, if the team delivers good software people use then grow the team

3. Need to think

Agile is not for those who want to follow a flow chart or for those who want to leave their brains at home and follow procedures. Indeed, if you are such a person then working in IT is probably a bad move in the first place.

Not every Agile method or practices will work were you work. Agile experts and methods disagree. You need to think about this.

You also need to think about what you are actually doing, what you have been asked for, and how you can work most effectively. Unfortunately economies of scale thinking has created a lot of companies whose modus operandi seems to be to stop staff from thinking.

1. 4. Need to unlearn

Doing Agile practices is the easy bit. The hard bit is unlearning: the things you have to stop doing. If you have burn down charts you don’t need Gantt charts; if you have an automated test suite you don’t need a regression test process; if you have supplies you trust you might not even need a contract; etc. etc.

Things that have made us successful in the past are included here. Teams advance, companies change, to much baggage from the past is carried forward making life in the new world expensive. Remember the words of John Maynard Keynes:

“The difficulty lies, not in the new ideas, but in escaping from the old ones, which ramify, for those brought up as most of us have been, into every corner of our minds.” The General Theory of Employment, Interest and Money (1935)

5. Feedback: getting and using

Agile doesn’t work out of the box, as the two points above say: you have to find what works for you. Some of this might be obvious at first but the more you get into this the more you are dependent – in all sorts of ways – on feedback.

Feedback about the thing you are building. Feedback about the way you are working – as team and as individuals. Feedback about how your customers are responding. etc. etc. etc.

6. People closest to the work make decisions

The people doing the work are usually in the best position to make decisions about the work itself. They have the most knowledge about the work and they have the most immediate need of a decision.

Every time a decision traverse up a decision tree information is lost and delay is introduced.

7. Know your schedule, fit work to the schedule not schedule to work

Easy really. Know when something is needed by and fit the work to that schedule.

In part this principle is a reaction to Kelly’s First Law of Project Complexity – coming in the next blog entry.

8. Some Agile practices will fix you, others will help you see and help you fix yourself

Some Agile Practices – like planning meetings – will administer a fix and you will get a little bit better immediately. Other practices – like retrospectives – will not fix anything but will help you see what is happening and find your own solutions.

Most Agile Practices are somewhere in the middle. Iterations and stand-up meetings are good examples. They may fix some problems immediately – they will improve focus and communication for sure. More importantly they will help people see what is happening and will, in some cases, force you to get better at what you do.

I should have been at Tom Gilb’s annual gathering this week – GilbFest as some people call it. Instead I’m sitting at 36,000 feet somewhere over Newfoundland heading for Chicago. I don’t often get out of Europe but maybe my fame is spreading.

The theme at GilbFest this year was Principles – defined by my dictionary as “a fundamental truth or proposition that serves as the foundation for a system of belief or behavior or for a chain of reasoning.” In fact principles is something that has been on my mind a lot over the last year or so. My “What and Why of Agile” which I gave to BCS London a few months ago featured a few thoughts on principles and I started a blog entry on the subject but never finished it.

Now is the time to finish it. Had I been at GilbFest this is what the audience would have heard about, or rather, this blog entry is the first of three instalments on what I would have said.

Let me say I don’t care whether you call these Agile or not, since we can’t define just what Agile is I’m prepared to accept they may just be principles. That said, I will divide the list into “Software Development Principles” and “Agile Software Development Principles” because I think the first set are universally applicable to software development while the second set require you to at least accept the idea of Agile.

I’ll publish these as separate blog entries, there is a lot here. After that I’ll also return to Kelly’s Law which I penned some years ago – before blogs!

Software Development Principles:

Principle 1: Software Development exhibits Diseconomies of Scale

Many, if not most, of us have been brought up with the idea that if we “do” bigger things get cheaper. Buying 2 litres of milk is cheaper than buying 1 litre. Building 1,000,000 identical cars is cheaper than building 10 different cars 100,000 each.

In software this isn’t true. Bigger teams are more difficult to manage, more expensive and less productive per head than smaller teams. This effect is so pronounced that really large teams might be less productive in total than small teams.

For example: a team of five will, per head, be more productive than a team of 25. Still the team of 25 will achieve more in total than the team of 5. However a team of 50 might be less productive than a team of 25 in total.

And its not just teams. Large software releases are more expensive than lots of small releases. Producing software to satisfy 100 users is more expensive than producing software to satisfy 10, or 1.

The effect appears again and again. Its why Lean folk like to emphasis small batch sizes. Unfortunately post-industrial society has internalised the concept of mass production and economies of scale. You, me, everyone, needs to purge themselves of economies of scale thinking and embrace dis-economies of scale if you are going to be be successful in this world.

By the way, I suspect this applies to other industries, more than we realise, however, I am a software guy, I can talk with authority about software so that I’ll stick to there.

Principle 2: Quality is essential – quality makes all things possible

By quality I’m really thinking bugs, I want to see bug-free software. I definitely do not want to see gold plating, I have no time for reusable code (as I said in an earlier blog).

Philip Crossby said it best: “Quality is Free” – Neils Malotaux puts it more accurately if less dramatically “Quality is cheaper.” The basic message is the same: pay attention to quality, rid yourself of rework and it will work out better. I said more about this in my “How Much Quality can we afford?” presentation.

And when we get to Agile I’m quite clear: if you don’t build in quality I don’t see how you can get iterations to work and I have no hope you will ever truly achieve Agile.

I’ll finish here for now, I’ll continue with the Agile Principles in the next entry.

To finish I should say these principles are a work in progress. That doesn’t mean that I intend to change them when things get tough. Rather I mean a) there may be some more I haven’t get identified, b) there might be some even deeper underlying truth below some of these principles.

One of the closing slides in my Agile Foundations course includes a quote from Ken Schwaber saying that only 30% of teams who attempt Scrum will be successful. What I find interesting about this quote is that it aligns with many other change management studies. Researchers like Harvard Professor John Kotter regularly say 70% of major change efforts fail.

On his blog Ken Schwaber says he doesn’t remember this and instead suggests only 30% will become “excellent development organizations.”

Either way, the prognosis isn’t optimistic. A few months ago, at the end of the course, someone asked the obvious question, a question so obvious I wonder why nobody has asked it before: “What can we do to ensure that we are in the 30% who make it?”

Given that I had the Managing Director, the Director of Technology and most of the technology team in the room it was an excellent opportunity to set the change agenda. And I fluffed it, despite having written a book on the subject I didn’t have a quick answer to hand. But it set me thinking: “What are the 10 things a team can do to make Agile (any flavour) stick?”

Here then is that list, the team in the room will recognise the first three, it was after that that I had to think.

1) Use a physical board: over the last year I have become convinced that the single biggest difference between teams which successfully adopt Agile working and those who try, fail, or end up stuck is the use of an actual physical board.

I know some teams find this difficult, I know some teams are distributed, I know there is technology out there to do this for you but I stand by my point. If you can make it physical, in a place where many, if not all, can see, then you are more likely to succeed.

2) Start collecting and using statistics and other data: velocity, burn-down, bugs identified, bugs logged, etc. etc. Metrics have a bad name in software development – rightly in most cases. But that only means that have been badly collected, managed and used, it doesn’t mean they aren’t useful. At the very least measure your velocity and create a burn-down chart or cumulative flow diagram of the work to do or arising.

3) Engage a coach/consultant: at the risk of being accused of trying to make work for myself I should say you can adopt Agile all by yourself. You can read the books, you can experiment, you can go on courses. But doing it without help makes the whole process slower and increases the risk that you won’t make it to the 30%.

Personally, I find it difficult to know just how an Agile Coach differs from an Agile Consultant. What ever you call the role you want someone who can:

• Provide advice on which practices and process to adopt, and how to best adopt them
• Offer examples of what they have seen work, and not work, elsewhere, and how other team tackle similar issues
• Observe, examine, query and challenge your thinking on what you are doing
• Challenge your thinking and point out opportunities and idea that you haven’t seen yet

You may need to work with multiple advisors since few will be able to cover all process, practice, technology, product and strategy bases. On very large team it might be worth having full-time consultants although the model I have had most success with is light-touch coaching in tandem with a pull-change model (below).

I don’t believe such an advisor needs to be full time. I practise, and have written before about, light-touch Agile coaching, in this model I return to companies at intervals, perhaps monthly, perhaps more frequently, sometimes less frequently and continue the discussion.

4) Action over talking: action speaks louder than words, until you start trying to do Agile you can’t foresee all the issues and questions which will arise. The longer you spend talking about doing it, and not actually doing it, the more it anticipation will build up, the more more it will look like jus another management fad.

By all means talk about it, plan a bit but there is no real substitute for just getting stuck in and doing it. In particular do not spend your time agonising over whether to do XP or Scrum, or Lean or FDD, or DSDM or Kanban. They are all pretty much of a muchness and you will end you up crafting your own hybrid anyway.

Likewise, discusses a few weeks ago: don’t waste your time looking for evidence, make your own.

Planning your way to Agile is anathema, just do it – JFDI.

5) Give everyone training and start group wide discussions: teams don’t get to be Agile by management deeming “thou shalt be Agile” – although plenty of managers and team leaders have tried the approach. Reading books works for some people but most books go unread, or the words go in one eye and out the other.

If you want to be Agile then invest in taking the time to explain to people what it is. But don’t stop there, make time for people to talk about what Agile is to them, what they like, don’t like, will do, won’t do. Agile is a team sport and unless the team have a shared understanding they will be playing different games.

6) Enthuse, Pull, don’t Push: Anyone who has worked around companies for a few years will have seen management pushing the latest change initiative: ISO 9000, Sig Sigma, CRM, ERP, etc. etc. Someone dreams up these ideas and then a change machine sets about pushing them out.

Apply a lean principle: Pull, don’t push. Forbid the words “change management.” Enthuse individuals and teams, have them ask for Agile. And when they ask give them the help and support they need. This works at the individual level, at the team level, at the company level.

If you are in management this means you need to engineer a pincer movement: you want enthusiasm for change coming from the bottom up to meet your support coming top down. Introducing Agile top-down alone is, in my opinion, as quite likely to kill it – employees are, rationally, skeptical of top-down management change. We live in a post-modern, post-BRP, post-layoffs, post-recession, post-everything world. Employees aren’t children they’ve heard what happens.

Rather than impose change from the top down managers need to build, kindle people’s curiosity, get people asking questions and for help, create bottom-up change initiative and support it. Do everything build the fire without extinguishing it.

The good news is the Agile marketing machine may already have got there ahead of you. People may already be curious about Agile, or even keen to try it – they may even be doing it when you are not looking. If not then find ways to stir interest. When they come asking for support – for budget for speaker, trainers or coaches – or time to go to conferences, give it, give it generously. Offer more, ask when else they need, and above all else: learn to change your own behaviours to match.

7) Be clear on Why you are going Agile: what ever level you are, engineer, tester, project manager, director, look beyond the Agile hype. What is the problem you want “Agile” to fix for you? Understand why you want change and what you expect from it.

Don’t just “get Agile” because it is this month’s fashion, get “Agile” to achieve something more important.

8) Process and technical, Adopt technical side as well as process side: don’t think you can just change the process and it will all be all right. You need to address the technical side too, you need to improve quality, you need to support the engineers, testers and others who are at the code face doing the work.

I’ve come across big companies who view the technical side as somehow dirty: the attitude seems to be “thats technical” or “ they get their hands dirty” or “we can ship it to [Low cost country of choice this week]”.

Get your hands dirty, talk to engineers, adopt Test Driven Development, refactoring, shun big up front design architecture, learn to live with rough designs and evolving architecture. There are real feedback loops here.

9) Get Product Management/Owner flow to developers clear and clean: it isn’t just about fixing the coding side, the requirements side needs to be addressed to. Specifically there needs to be a clear path from someone who represents requirements – typically called a Product Owner or Product Manager and frequently staffed with a Business Analyst – and the development team. Far more negotiation is going to happen over “what” then “when”. Someone needs to represent – and have authority – over that side of things.

10) Structural changes – Functional groups: Staff your teams to do the work for which they are responsible, end functional groups – i.e. database developers and UI developers in separate teams. This is just the first of more structural changes you will need to make. But if you fail at this you won’t get to play again.

There you go, each of those items could be an entry in its own right, maybe one day they will be. Thats enough to get you started. If there was an eleventh is would be: let go of the past, things change, Agile isn’t purely additive. If you don’t stop doing some of your current things you will never see the full benefit. But 11 can wait, those 10 will get your a long way.

With the points-mini-series still fresh in the mind now seems a good time to say publicly something which I’ve been saying privately for a long time.

Avoid points based contracts. i.e. don’t outsource work, or undertake work, on the basis of points – be they story points, abstract points, nebulous units of time or any other name you give them.

I have one client at the moment who wants their software supplier to sign a points based contract, I’ve advised against it. Another client is trying to sell points based contracts to their clients, while they are having some success – I think they rushed in before they had enough data to understand the implications.

Why do I say this? Well three reasons

First is Goodhart’s Law: “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” Put it another way: any measurement metric will change behaviour once it is used for control.

In this context it means: points are very good at measuring story size and team velocity, they can be accurate at predicting when a piece of work will be done. But, if you use them for other purposes – like regulating a contract and making payments – they will change their behaviour. They won’t be so useful for predicting end dates, or for controlling contracts for that matter.

This is a problem many readers will be familiar with from traditional time estimation. Estimates are nominally sought to determine how long a piece or work will take, and thus how much it will cost. But they are also used as a means of targeting and for control. They are used as a proxy for commitment and they are gamed (i.e. changed for specific ends) when they don’t give the time/cost numbers desired. (This is something Esther Derby discusses recently in her blog, Estimating is often helpful, estimates are not.)

In fact, time estimates show the same range of problems present in the corporate budgeting processes and which has given rise to the beyond budgeting movement.

One direct result of Goodhart’s Law in this context is…

Inflation, the second reason to avoid points based contracts: points are subjective, they are not grounded in time, complexity, function point analysis, lines of code or any other objective measurement. They are in fact like a fiat currency: they are worth what you can buy with it. If people don’t believe in it, or believe the value will change then it will change. Check out rational expectations theory if you want to understand why.

Overtime points can devalue with the result that point scores increase. Actually, I believe the free floating nature of points is one of the strongest reasons for using them but in terms of signing a contract it makes them useless.

Most teams I see work in low points: 1, 2, maybe a 5, rarely an 8, they score 10, 12 or maybe 20 an iteration. One team I saw worked in tens, and scored hundreds each iteration. It was like one of those old Space Invaders machines were the last digit was a hard coded “0”.

The team’s project manager finished planning meetings with an call for the team to work harder next iteration, to reclaim the lost time. Iteration on iteration velocity increased. Inflation was rampant.

Finally there is practicality. As the recent posts from myself and Vasco Duarte demonstrate points, there is still a lot of debate over points. Personally I have come to the conclusion that exactly how you run iterations and count points makes a big difference. While if you agree with Duarte you might as well dispense with points and sign story based contracts.

Then there is the team: only the team which will do the work can accurately say how many points a piece of work will, or did, take; and then only when they have experience of doing the work. So you shouldn’t sign a points based contract unless you have the team in place and they have done some of the work.

Even a relaxed interpretation of that last point should lead you to conclude you should only sign a points based contract when the team is experienced in using points and you have historical data. If you feel you must sign a points based contract then only do it when you have data.

Still, I’d rather you didn’t do it in the first place.

As anyone with time to spare will know I’ve recently spent a lot of time thinking, and writing about story points. This was in response to Vasco Duarte’s Story Points Considered Harmful blog from a month or two back. For completeness here are the links:

• Vasco Duarte: Story Points Considered Harmful
• Story points considered harmful? 1 of 4 – Journey’s start
• Story points 2 of 4 – Duarte’s arguments
• Story points 3 of 4 – An example
• Points 4 of 4 – Breakdown

Some conclusions I draw from this:

• There is far more work to do on Abstract/Story Points than we, as a community have done to date
• There are many more nuances to the assignment of points, the breakdown of work and the management of the outcomes than I think I previously realised
• I must go and see what Mike Cohn actually says about Story Points before I say any more about his approach
• Stable teams are crucial – but then I’ve been saying this all along

Given all this I’d like to pose a few questions and hypothesis of my own

Q1: When does the correlation between story points and number of cards become stable?

Hypothesis: I would expect a team new to “Agile”, stories and points to start off with erratic point scores and number of stories complete per sprint. Thus I would not expect the correlation to be stable. As a team settles down I would expect points to become stable, then stories completed and thus establish a correlation.

Q2: Is there any serious research into story points out there?

In the same line as my recent post “Agile: Where’s the Evidence?” it would be interesting to know if anyone has examined the use and accuracy of story points. Again, I should seclude myself in an academic library and review the data. But again, I have to find time.

More problematic, I suspect, OK another hypothesis, that some of the reason why story points work – which I listed in part 3 of my posts – will make it very difficult to determine if they are accurate because the thing story points are measuring will change.

This entry directly continues from three earlier ones:

• Story points considered harmful? 1 of 4 – Journey’s start
• Story points 2 of 4 – Duarte’s arguments
• Story points 3 of 4 – An example

Duarte’s analysis, and my response, has got me thinking. And I think it would be useful – to me at least, maybe to some readers! – to explain why I think story points, or rather the “Abstract Points” that I prefer, are still useful, and why I advise teams to break down Blues – stories, possibly User Stories.

Why do I advise teams to break down Blues/Stories?

My background is as a C++ programmer, I worked on financial, telecom, and other systems. A business story, a Blue, would frequently be bigger than a developer could manage in an iteration – particularly if you have a legacy system. Thus I would break Blues down to Whites. (See Blue-White-Red (PDF) if you want to know more about this approach.)

Blues mean something to the business, Whites mean something to developers. I think this situation still holds for many developers in many environments. This has several advantages:

• Whites are smaller pieces of work, they flow through a system more easily. Progress can be seen, tasks tracked, velocity calculated.
• “The Business” aka Product Owner/Manager/BA are not always good at delivering small stories, breaking a blue down gives the developer a chance.
• On some teams the business have been beaten up by development to request really small stories. However these stories lack business value. Because Blues are going to be broken down they can be large enough to have value even if that means that can’t be completed in one iteration/sprint.
• (Yes, you heard that right) Whites are completed during the iteration, when all the Whites, or the essential ones, are completed then the Blue is completed.
• Breaking Blues down to Whites is as much a design exercise as it is an estimation and scheduling one. This allows teams to engage in design and create a shared understanding.
• Breaking Blues down to Whites frequently reveals functionality or assumptions about the Blue requirement which can be removed or postponed.
• Having the Product Manager/Owner/BA in the room during this break down allows for requirements elaboration and knowledge mining.
• Work can be rolled from one iteration to the next. I’m very relaxed about carry over work and I think for a new team its almost unavoidable. However doing it this way allows some points to be counted and illustrates what is happening.

(Of course the break down does create some problems: a Blue can only be done when all the Whites, or some done and other cancelled, which mean tracking becomes more complex. It might also break the Lean idea of “single piece flow” but I’m not sure.)

Next, why, given Duarte’s analysis, do I still advise teams to estimate their work?

• I have seen the breakdown and estimate approach. As detailed previously in one case it allowed a team to forecast to the day.
• Estimating work allows teams, and individual team members, to raise a warning when work is not understood, defined or involves a lot of risk. For example, a team estimating with planning poker will normally settle on an “average size” of task, e.g. 3 or 5 points. When they suddenly assign 13 or 20 points to a task something is wrong.
• And just in case the warning is ignored the team, the people at the code face, the people doing the work, have a control mechanism. No matter how much the business or a manager bully a team they can still assign a high point score.
• Equally, when differences in estimation appears it is a trigger to discussion, to learning, to understanding. This is desirable.

Finally, there is one more reason why I will continue to advise to point their work, and its one I don’t normally admit to but, well Vasco, you win…..

Placebo effect.

Managers, particularly trained project managers, find it alien to not estimate. Actually they are not alone, I’ve seen plenty of developers and testers who think the Kanban craze of not estimating work is nutty. Going through the rituals of pointing and planning poker provides at least the appearance of doing “the right thing.”

Asking these folk to go cold turkey on planning and estimation is tough.

Likewise, asking them to give up Gantt charts can be tough, so we offer them burn down charts. To be honest I find intra-sprint burn-down charts useless. Even the efficacy of pan-iteration burn-down charts surprised me at first. I now see they can be very useful and recommend their use. (Intellectually I prefer Cumulative Flow Diagrams but they are more difficult to get your head around and more difficult for the casual viewer to understand.)

A mature team is, almost by definition, beyond needing placebos. In a mature team I would expect the business to be requesting small stories which do represent value and do fit within an iteration. Thus I would expect Duarte’s analysis to hold up and a mature team might well decide to go without points and use cards.

However, for team at the beginning of its Agile journey I don’t expect these conditions hold.

Finally, for this instalment, I’ve started to wonder about Blue-White-Red again. I’ve long regarded Blue-White-Red as a Scrum/XP hybrid – closer to XP than Scrum if I’m honest. While I’ve been asked to write more about it in the past never have. Over time I have refined my thinking about it. I’m now wondering if Blue-White-Red is actually something more different than I’ve ever appreciated.

Maybe someone who has used Blue-White-Red can answer than one.

This entry continues from two earlier ones:

• Story points considered harmful? 1 of 4 – Journey’s start
• Story points 2 of 4 – Duarte’s arguments

I’ve got some (abstract) points data of my own. Not as much as Duarte’s but some. One team in particular is interesting. The development manage said a few months ago “We can deliver to the day.” But actually, when you look at the data the velocity looks quite variable. Whats going on?

Well two things, at least. First, when you average the data out it is no where near as variable – thats what averages do. I’m reminded of the old economists warning: “Do not pay to much attention to one month’s figures [GDP/GNP/Inflaction/etc]. Look at the trend.”

So yes, velocity iteration to iteration changes but over a longer period it is meaningful.

Second, this is a team I regard as stable. I learned a long time ago that if you don’t have a stable team your velocity data is meaningless. The velocity is delivered by the team members, if you change the team you can’t get a meaningful velocity.

While I regard this team as stable when I looked close, looked at the data and dredged my memory, this was not a stable team. One member retired, one member joined, the team was joined by another person to tackle a specific sub-set of work, the team adopted TDD a few months after moving to iterations, later still they tried doing pair-programming.

Somewhere along the line the hardware team joined the iterations, added to the velocity, then, after a while left. It didn’t work as well as hoped. When you look at the data you can see this: alone the software team can have a standard deviation as low as 3.6 on an average velocity of 62 (over 5 iterations) , with the hardware team added that goes to over 17 on a velocity of 60.

(Velocity falls after team expansion is a phenomenon I’ve seen in two other data sets I’ve got. Brooks’ Law doesn’t completely explain this, other factors are at work which I will discuss another day (i.e. when I understand them more fully!))

In other words, the team wasn’t stable. In fact, given all that change I’m surprised velocity was as stable as it was!

I think a third factor was at work. Once a team have put a point score on a card, say they point it to 5. Then there is a mild incentive to finish the card in something that feels like 5 points. Not the strong commitment of Scrum mythology, more a pride in ones own skills, and perhaps, a desire to score points at the end of the iteration.

Fourth: its not just development estimates that are helping the team hit dates. Armed with this data scope can be fine tuned, teams can take decisions on when to do refactorings and so on.

Fifth: once a team has velocity data and can forecast dates it it can negotiate on features and deliveries. This echo’s Duarte’s story but is more fine grained. Of course this won’t help if the end customers/users/clients/stakeholders aren’t prepared to engage.

Given all this I believe abstract points and graphs are helpful, not harmful.

Perhaps one day I’ll be able to publish this data. Its just one team but it shows that velocity and point scoring can work.

This blog entry follow directly from the previous Story Points – Journey’s Start.

Key to Duarte argument, and something I didn’t originally appreciate from his initial Tweets is: he is not saying story points are rubbish, forget about them. What he is saying is: it is simpler and equally accurate to just count the stories as atomic items. This is equivalent to saying “All stories are 1 point” and having done with it.

For a mature team with a good relationship with its stakeholders I could see this working well. However, for less mature teams (who have difficulty agreeing among themselves) or a team with bully-boy stakeholders (or bully boy anyone else for that matter) then I think being able to put a higher point score on the card serves as a useful warning mechanism.

Duarte says in his blog “the best predictor of the future is your past performance!” On this I couldn’t agree more. He then poses three questions – and answers – which I think are worth reviewing.

Q1: Is there sufficient difference between what Story Points and ’number of items’ measure to say that they don’t measure the same thing?

Here he finds there is a close correlation. I’m not surprised here, in fact I would expect this to be the case. Teams are encouraged to write small stories, in fact Scrum almost mandates this because work should be completely done at the end of a sprint. In effect there is an upper bound placed on the size of a story.

Actually, I’m not so keen on this rule. I allow work to be carried from iteration to iteration but I only allow points to be scored when the work is done. Thus I encourage stories to be completed in an iteration but I don’t mandate it. One of the exercises I do with teams on my courses actually sets out to illustrate this point.

At the very least I would expect teams to settle on an “average story size” implicitly. Notice also that the correlation applies whether all stories are of size 1 or of size 2, 3 or any other number. Its a correlation between two series of numbers.

However, given all this Duarte has a point: if your stories are clustered around an average size then you might as well count the stories.

Q2: Which one of the two metrics is more stable? And what does that mean?

Duarte’s analysis says that both stories and story points have similar standard deviation. Thus they are of similar stability. Since these two are closely correlated this isn’t a surprise. In fact, given the correlation, it would be a surprise if one was notably more stable.

Q3: Are both metrics close enough so that measuring one (number of items) is equivalent to measuring the other (Story Points)?

Duarte’s data seems to measure the same thing – again, if they are closely correlated then this is exactly what you would expect. You can write out the equation:
        Story Points ~= (Correlation Co-efficient) x (Number of Stories)

(The ~= is supposed to mean approximately equal.)

With this out of the way Duarte moves on to consider Mike Cohn’s claims for story points.

Claim 1: The use of Story points allows us to change our mind whenever we have new information about a story
Duarte says: Story Points offer no advantage over just simply counting the number of items left to be Done.

I agree here. I’ve long encouraged teams to move away from story pointing work in the distant future. Yes I encourage them to story point some stories in the backlog – say a few months work – and story point tasks – for the next iteration. But for stuff that is “out there” or has just arisen my advice is usually: just assign it your average story point value.

In other words, assume your average story point value is your Correlation Co-efficient. When work gets close then estimate it traditionally, you might find the value changes. When it gets really close break it down into tasks.

Claim 2: The use of Story points works for both epics and smaller stories
Duarte says: there is no significant added information by classifying a story in a 100 SP category

Again I agree. To be honest I’m not a fan of Epics and while some of the teams I work with use them I often encourage teams to dump them. To me an epic is just a collection of stories around a theme.

Actually, what Cohn and Duarte are saying are not at odds here. Cohn doesn’t (seem to) make any additional claims. Its just a scaling question.

Claim 3: The use of Story points doesn’t take a lot of time        
Duarte says: In fact, as anybody that has tried a nontrivial project knows it can take days of work to estimate the initial backlog for a reasonable size project.

Here I have issues with both Cohn and Duarte.
If you are estimating stories then it does take time. Fast as it is, even planning poker takes time. However there is also a lot of design and requirements discussion going on in that activity. Therefore I don’t see this as a problem. In fact I see it as an important learning exercise.

True, on a none trivial project it will take time to estimate a large backlog. But a) that is valuable learning and b) I won’t try. I’d either estimate it in chunks or I’d apply an average estimate to work which wasn’t going to happen anytime soon – see claim #2.

I deliberately delay estimation as long as possible to allow more information to arrive and because work will change. It might be changed out of all recognition or it might go away completely.

Claim 4: The use of Story points provides useful information about our progress and the work remaining
Duarte says: This claim holds true if, and only if you have estimated all of your stories in the Backlog and go through the same process for each new story added to the Backlog.

Again I agree. However I doubt the usefulness of the concept of “work remaining”. Its only work remaining if you think you have a lump of work to do. In my experience work is always negotiable. Its just that people don’t want to negotiate until they accept that they won’t get everything.

One of my clients has gone through the very expensive exercise of estimating all the work they might do. Earlier this year they realised they had 3000 points to do by Christmas. They also realised they had capacity to do less than 1000. This brought home to fact that they couldn’t do everything – something many people on the project had long known or suspected. The company are still working through this issue but at they are having the discussion now, in March and April not September and October.

Claim 5: The use of Story points is tolerant of imprecision in the estimates
Duarte says: there’s no data [in Cohn’s book] to justify the belief that Story Points do this better than merely counting the number of Stories Done. In fact, we can argue that counting the number of stories is even more tolerant of imprecisions

Again I agree. But then, if there is a high correlation between story points and stories then this is self-evident. And again, as I said before: we need to work with aggregates and averages.

Claim 6: Story points can be used to plan releases
Duarte says: Fair enough. On the other hand we can use any estimation technique to do this, so how would Story Points [be better than counting the number of stories]

Again, agreement, and with correlation its self evident.

Duarte goes on to give a worked examples in which a project does not achieve the desire velocity and gets cancelled. His story makes no use of story points, simply stories. To be honest I’m missing something here. True the stories in his story have no points, but I don’t see where that makes a difference. What he describes is exactly the way I would play the scenario although I would have story points in the mix.

His conclusion: “Don’t estimate the size of a story further than this: when doing Backlog Grooming or Sprint Planning just ask: can this Story be completed in a Sprint by one person? If not, break the story down!”

This is interesting because while Duarte is working at the story level this pretty closely models the way I advise teams to work. I always tell teams:

• “I’d like a story to be small, to fit in one iteration but that isn’t always the way.”
• “In my experience stories need to be broken down, both so they can get done but also so you get flow and as part of a design exercise.”

I then have teams break down stories – which I call blues – into (developer) tasks – whites – this follows my Blue-White-Red process from a few years back. I then put points on whites – at this point you probably start to see why I prefer the term Abstract Points not Story Points because Whites aren’t stories.

When a team have a feel for points then I will have them put points – the same units, just bigger – on Blues. For Blues that won’t be done for a while then I’m happy to assign averages, and for Blues that are further out then I’m happy to leave them unpointed.

So, thank you for reading my analysis of here and staying all the way.

My conclusion? I think I agree with Duarte but I don’t agree with him. I think I actually disagree with Mike Cohn but I’d have to go back and look at what he says himself.

I think the way I estimate with teams, the way I used to do it when I ran teams and the way I teach clients to estimate is more different than perhaps I appreciated. My method grew from my interpretation of Kent Beck’s XP planning game and velocity. However I now think my approach has drifted from this. The result of my experience, seeing what works and what doesn’t has refined my approach.

The approach I’ve ended up with has similarities with what Duarte describes but is also different.

Despite lots of authors attempts to describe Scrum/XP planning and estimating I still find myriads of minor variations. Some improve things, some not. Until this moment I’ve always believed my approach only differed in minor ways. Now I’m thinking….

And what of the Duarte’s harmful claim? Actually, although he uses the word in the title I don’t see any discussion of the harm they cause.

Story points might be pointless but do they do any harm? I don’t really see it, they may be a waste of time, there might be more effective ways of doing the same thing but that’s not the same as harmful. Story points might mislead, there is little evidence here so I’ll hold judgement on that one.

To be continued….

A few months ago Vasco Duarte, with a little help from Joseph Pelrine started a discussion entitled “Story Points considered harmful.” They, or at least Vasco, has given this as a conference keynote and has blogged about it.

(Warning: this is the first of 4 blog entires, its quite a long post. Plus I think there are two appendix blog entries to follow up.)

Now I’ll admit, when I first heard this argument I thought “Well I can guess where they are coming from” – I have sympathy with the argument, I’ve always considered story points as suspect myself. I also thought “But I don’t think they are right”. Specifically I know a team who use story points and claim to be able to schedule delivery “to the day.”

I’ve collected some data of my own from teams I’ve worked with, am working with or at least in contact with and done a bit of amateur analysis. I’m also taking time to go over Vasco and Joseph’s arguments.

This is a big topic and its going to take me a while to get to the bottom of the data, what I think and the pro and con arguments. So please forgive me, this is going to take a few, possibly long blog articles to go through.

Lets get a couple of things on the table to start with.

Firstly I don’t believe Story points are a Scrum technique. Yes they have been subsumed into Common Scrum but I believe they originally originated with Extreme Programming. Where they originated isn’t really important because I believe story point estimation and tasking (i.e. estimating work to do with story points and then scheduling a number of story points) is at odds with Scrum Commitment.

I’ve blogged about this before, Two Ways to Fill and Iteration, so I won’t repeat myself. Just say, in my book commitment and story pointing are alternatives.

By the way, the name Story Points comes from Mike Cohn, I prefer to call them Abstract Points, and I’ve heard others call them “Nebulous Units of Time”.

Second, I don’t believe story points can ever be stable if you don’t have a stable team. If you remove people from a team I expect it to slow down, if you add people to the team I expect it to slow down too – at least in the short term. In the longer term you might increase capacity but frankly I don’t know how long that will take.

I few years ago I worked with one team which had story/abstract points which appeared to be random. When I adjusted form changes in team staffing they average was constant. That said, I don’t expect points to remain constant, they might do for a while but I expect them to fluctuate at the very least.

It goes without saying that I expect sprint/iteration length to be stable too.

Which brings us to the third thing: with points, stories and projections, they only work at the average and aggregate level. They are a good predictor over several sprints but they offer no guarantees for the next sprint/iteration.

Finally, for now, something I do agree with Duarte on: we can’t estimate. By “we” I mean humans. Last year I devoted several blog entries to the subject, Humans Can’t Estimate.

A few weeks ago I was presenting at the BCS SIGIST conference – another outing for my popular Objective Agility presentation. Someone in the audience asked: “Where is the evidence that Agile works?”

My response was in two parts. First although it sounds like a reasonable question I’ve come to believe that this is a question that is asked by those who don’t believe in Agile, those who want to stall thing. It is rarely a question aimed at a rational decision.

Second I said: lets turn the question around, Where is the evidence for Waterfall? – as far as I know there is none, although there are plenty of cases of failed projects.

Now this was a pretty flippant answer, I was in show mode and as someone later asked: “Do you really recommend we ridicule people who ask for evidence?”, to which the answer is certainly No.

So let me try and answer the question more seriously.

Lets start with “Agile.” How do we define Agile? Scrum? XP? Where does Agile start and Lean begin, or vice-versa? Or, as I believe, Agile is a form of Lean?

We have a definition problem. Agile is poorly defined. I e-mailed a friend of mine who is undertaking a PhD in Architecture and Agile and asked him: Can you please point me at a literature review?

(Notice, if I was a serious researcher I would closet myself in the library for days or weeks and do the literature review myself. I’m not and I need to earn money so I I didn’t.)

His answer: “that’s an interesting question because a lot of research simply assumes that agile (as a whole) is better, without any evidence for it.”

Michael also pointed out there is a context issue here. Embedded systems? Web development? Finance? Health care?

And how do you define better? Better architecture? Better requirements? Better code quality?

And Better compared to what? Chaos? Good waterfall? Bad waterfall? CMMI level 1? 2? 3? 4? 5?

You see, context.

Despite this one study claimed Scrum resulted in productivity improvements of as much as 600% – Benefield, “Rolling Out Agile in a Large Enterprise”. I’ve even heard Jeff Sutherland verbally claim Scrum can deliver 1000% improvement. To be honest I don’t believe this figures. If they are possible then I think it says something about chaotic state the organisations started in. In these cases my guess is following any process or practice would be an improvement. Standing on one leg every morning would probably have generated a 50% improvement alone.

There is a trap for Agile here. Much traditional work has defined better as: On schedule/time, on budget/cost with the desired features/functionality. But Agile doesn’t accept that as better, Agile negotiates over features/functionality and aims for business value. A report from Cranfield University a few years ago suggested than much traditional IT work failed to truly capture business value because people focused on: time, budget, features.

Then there is a question of bugs. Traditional development has been very accepting of bugs, Agile isn’t.

Maybe asking for evidence about Agile is aiming for too much. Maybe we should look at the practices instead. Here there is some evidence.

Over the years there have been various studies on pair programming which have been contradictory. Since most of these studies have been conducted on students you might well question the reliability.

Test Driven Development is clearer. As I blogged two years ago there is a study from Microsoft Research and North Carolina University which is pretty conclusive on this, TDD leads to vastly fewer bugs.

Keith Braithwaite has also done some great work looking at Cyclomatic Complexity of code and there seems to be a correlation between test coverage and better (i.e. lower) cyclomatic complexity. Keith is very careful point out that correlation does not imply cause although one might hypothesis that test driven development leads to “better” design.

TDD and source code are relatively easy to measure. I’m not really sure how you would determine whether Planning Meetings or User Stories worked. Maybe you might get somewhere with retrospectives. Measure how long a team spends in retrospectives, see if the following iteration delivers more or less as a result.

For their book Organizational Patterns of Agile Software Development Coplien and Harrison spent over 10 years assessing teams. This lead to a set of patterns which describe much of Agile software development. This is qualitative, or grounded, research rather than qualitative but is just as value.

At this point I probably should seclude myself in the British Library for a week to research each practice. Maybe I should, or maybe you want to call me an charlatan. Next year, maybe.

If we switch from hard core research to anecdotal evidence and case studies things become easier. As many readers know I’ve been working with teams in Cornwall for over 18 months. On my last visit we held a workshop with the leaders of the companies and software teams. Without naming names some comments stood out here:

• “The main benefit [of Agile] was time to market… I don’t know how we would have done it without Agile”
• “Agile has changed the way we run the company”
• “It is hard to imagine a world without Agile”

That last company is now finding their source code base is shrinking. As they have added more and more automated tests the design has changed and they don’t see the need for vast swaths of code. Is that success? Each line of code is now far more expensive and they have less. (Heaven only knows what it does to function point analysis.)

If you want something a little more grounded there was a recent Forrester report (sorry, I can’t afford the full report) which said: “Agile enables early detection of issues and mid-course corrections because it delivers artefacts much sooner and more often. Finally, Agile improves IT alignment with business goals and customer satisfaction.”

Before this there was a Gartner report in which said: “It’s a fact that agile efforts differ substantially from waterfall projects. It’s also a fact that agile methods provide substantial benefits for appropriate business processes.” (Agile Development: Fact or Fiction, 2006).

But I’m back to that nebulous thing “Agile.”

All in all I can’t present you any clear cut evidence that “Agile works”. I think there is enough evidence to believe that Agile might be a good thing and deserves a look.

What I haven’t done is look for, let alone present, any evidence that Waterfall works.

Frankly I don’t believe Waterfall ever worked. Full stop. Winston Royce who defined “the waterfall” didn’t so why should I? (You have to read to the end of the paper.)

OK, sometimes, to save myself form a boring conversation or to cut to the chase I’m prepared to concede that: Waterfall worked in 1972 on developments using Cobol on OS/360 with an IMS database. But this isn’t 1972, you aren’t working on OS/360 and very few of you are using Cobol with IMS.

(I have seen inside an Z-OS Cobol IMS team more recently and something not unlike waterfall kind of worked, but the company concerned found it expensive, slow and troublesome and wanted them to be “Agile.”)

Even if I could present you with some research that showed Agile, or Waterfall, did work then it is unlikely that the context for that research would meet you context.

(Thus, if you do want to stall the Agile people in your company, first ask “Where is the evidence Agile works?”. When they produce it ask “How does this relate to our company? This is not our technology/market/country/etc. etc.”)

I think it might come down to one question: Is software development a defined process activity or an empirical process activity?

If you believe you can follow a defined process and write down a process to follow and the requirements of the thing you wish to build then waterfall is probably for you. On the other hand, if you believe the requirements, process, technology and a lot else defies definition then Agile is for you.

Finally, the evidence must be in the results. You organisation, your challenges, your context, are unique to you. So my suggestion is: make your own evidence.

Set up two similar teams. Tell one work Waterfall and one to Agile and let them get on with it. Come back every six months and see how they are doing.

Anyone want to give it a try?

And if anyone knows of any research please please please let me know!

(Many thanks to Michael Waterman for a very thoughtful e-mail which filled in some of the missing pieces for this blog.)