A provocative piece from me in this month’s Agile Journal – Requirements Come Second.
If you’ve not checked out the Agile Journal recently you might want to have a look, they’ve given the site a make-over and have added blogs, forums and all that community stuff.
Requirements come second Read More »
At the end of last year I was lucky enough to hear Dan Jones give a keynote talk at XP Day 2008. Dan is one of the authors of The Machine that Changed the World and as such is one of the fathers of Lean. Or at least father of the term Lean, I’m sure Dan would point out that he (and his co-authors) didn’t so much invent Lean as reveal it to the world.
Dan opened his talk by addressing the difference between Agile and Lean directly. And in his view there is no difference. Perhaps someone else who was in the room can help me with the exact quotes but this is how I remember him putting it:
• When Dan, Jim Womack and Daniel Roos wrote did the research that resulted in The Machine that Changed the World they had to think of a term to describe what they found. Toyota Production System was too Toyota specific. They eventually came up with Lean. In the process they considered the name Agile but rejected it – “because it wouldn’t sell” he quibbed.
• Later (someone who’s name he gave but I don’t remember) decided there should be an “American version of Lean” and decided to use the term Agile. As far as Dan can tell it is this source that the original Agile software people (those who wrote the Agile manifesto I guess) picked up on.
• As far as Dan is concerned he doesn’t care which term we use, its all the same.
• (And he’s glad to see IT folk taking an interest because traditionally the Lean manufacturing movement has found IT a block to Lean adoption.)
I’m glad to hear Dan say this, certainly my flavour of Agile has long been influenced by Lean – even before the Poppendieck’s excellent Lean Software Development. I now feel more able than ever to talk about them as one.
That said I think there is a subtle distinction, its actually a distinction that can be helpful on some context. Its also a distinction which helps explain why Agile is not XP even though XP is Agile and why there is more to Agile than Scrum.
It helps to look at this diagram – if you’ve seen it before you’ll notice I’ve extended it recently:
At the top there are specific Agile methods like XP, Scrum, Crystal, etc. These are quite prescriptive in their ways – some more than others. Each is an Agile method but none are Agile.
Agile is a toolbox, these methods have chosen elements from the toolbox. Agile is more general, Agile is more of a value system, a philosophy.
Agile itself is Lean but it is more prescriptive than Lean. Few Agile teams would not do Test Driven Development for example. So while Agile is more general than XP or Scrum it is less general than Lean.
Lean itself is more philosophical still and contains fewer specific practices. You have to apply Lean thinking in your own context. Lean is a journey.
Still further down we find Organizational Learning. All these methods are based on continual learning – systems thinking should probably be included in here. But organizational learning is almost entirely values, philosophy and academic research, it is almost devoid of specific practices. (I could get into the Knowledge Management debate here but not today.)
Recently I’ve added RUP and Kanban to this diagram.
RUP shares a lot in common with Agile methods but it comes from a very different place. It comes from traditional software engineering, it comes from the unification of object orientation and it comes from the commercial world. RUP can (I am told) be made to look and feel like Agile but this doesn’t change where it comes from.
Then there is Kanban. Kanban is the new kid on the Agile block but it is derived more direct from Lean thinking. So for me it doesn’t inhabit that top apex of the pyramid, it draws on Agile and Lean. And this is where Dan Jones point is very appropriate: it doesn’t matter, at this level Agile and Lean are merging. So if you want to argue that it should be up there with XP and Scrum I don’t mind.
Agile and Lean – the same but different Read More »
Bristol BCS are holding a series of sessions entitled “Spring School – An Introduction to Agile Development”. I will be presenting the fourth and final session (23 March) on the “Future of Agile”.
I’ve not written it yet but it will probably include plenty about Lean and Kanban, the Change Problem and the importance of Business Analysts and Product Managers.
Fellow ACCU member Giovanni Asproni is doing the session on 16 March on Agile testing and TDD.
If your in the Bristol and Bath area I’m sure it will be worth the £30-£50 charge.
BCS Bristol Spring school Read More »
After slogging my way into London’s through the worst snow in 18 years on Monday and Tuesday this week to deliver a Agile Project Management course, I was a little relieved when Kingston & Croydon BCS cancelled last nights presentation – because of the weather. Still I was a sorry to miss the evening.
If you were one of those who planned to come along all I can say is Sorry, I believe it will be rescheduled for a later date.
But, when I opened my e-mail I had a very nice surprise…
My book, Changing Software Development: Learning to become Agile is going to be translated into Japanese. I can’t wait to see it. This was quite unexpected and I think goes to show how well the book has been received.
Now if any Japanese readers would like me to do a talk in Japan I’m more than happy to oblige, although I will have to ask for expenses.
Taking Japan by storm (and not taking Kingston) Read More »
Writing that last blog entry on my introduction of Kanban also led me to reflect a bit on my Blue-White-Red Agile method.
I’ve long felt Blue-White-Red was largely derived from XP, thats what the developers were excited about. I called it Blue-White-Red because it was a cut down XP, I was introducing the ideas incrementally. Also, the word “extreme” frightened the managers and “pair programming” frightened most developers.
In the last few years I seen more and more Scrum in BWR. At the time I “invented” it I hadn’t read much about Scrum books so maybe it is surprising how much like Scrum it looks. But, big but, I think this also goes to show how little difference there is between XP and Scrum. I’d have to go back and read the original XP book to be sure but it seems to me the main difference is XP’s emphasis on engineering practices (TDD, refactoring, etc.) while Scrum emphasis the project management bit (just without the project manager).
The other influence was Lean. I’d started reading about lean sometime around 2000. In fact I wrote about “Thinking Beyond Lean” in the April 2002 issue of Overload. I’d also been exposed to a lot of lean on my MBA course.
I distinctly remember telling people the white board was a “Kanban board” and when people asked “What if I have too many cards to put see the board?” | telling them “That is a sign you have too much work on the go” – I was attempting a crude form of work in progress limits.
So what would I recommend to teams in future?
Well as always it depends. But I’m going to favour Kanban. I think Scrum (and BWR) still have their place, they are more suited to environments were large project specifications exist up front. In addition, Scrum has the advantage when the organization needs to see external validation and BWR has the advantage because there is less baggage. There is less written about it.
That last bit might seem a little off but I mean it. The fact that it is less well knows means it brings fewer assumptions, fewer stories about how it failed and perhaps a cleaner sheet. Kanban has many of those advantages today but I don’t expect it to last, success will remove them.
Reflections on Blue-White-Red and Kanban Read More »
I’ve mentioned the Kanban software development method in this blog before. For those who don’t know its “the new kid on the block” in Agile circles – although the originator (David Anderson) would be quick to point out it is designed to be a Lean development method.
Last year I did some consulting with a large online travel agency. I was involved with helping five teams “get Agile.” This presented an opportunity to try out Kanban in the wild. What I found was: it works, and I feel it is a better models of my own approach to software development than other methods.
What made this engagement more interesting was that I was able to start one team using the Blue-White-Red Agile (BWR) development method I’ve described before and two teams using Kanban so making for an interesting comparison.
The BWR team had been tasked with a project for which a large functional specification had been written. The spec wasn’t perfect but the organization expected them to complete it. BWR allowed the team to break the spec down and attempt it piecemeal. Some upfront estimates were made and a burn-down chart created.
(It almost looked Scrum like, that is, if you ignored the Project Manager, Development Manager, Team Leader, Architect and second Project Manager who joined the project later – and lets not forget me as Agile Coach. Its hard for a team of five developers (and a Product Mmanager) to be self-organizing when there is an equal number of self-organizers.)
This worked well. In the first few weeks I took a lot of heat because I refused to let a completion date for the work be given. Simply we did not have enough date to go on. Once the data started to come through I could do that.
The two Kanban teams were in a different position. Although they had “projects” to do most of their work was sustaining work. There were systems in place and issues needed fixing. Many of the things the company called “projects” were really to small to be worthy of the name – but that’s another story.
As a result the teams had more diversity in their work and more variability. Work would just appear – and disappear. They would have been hard pressed to keep to an unchanged task list (backlog) for a one week iteration, let alone a four week sprint.
This is the first good point about Kanban: its easier to handle changing work loads.
The teams still did “iterations” but the planning meetings quickly changed. Some work would be planned out but this could be usurped at any time. The iteration became the basic unit of comparison, allowing two periods to be compared. This meant that the start/end of iterations meetings were rather more of a review and retrospective session than planning session.
In future I think I would nominally keep iterations for this reason. They are useful time periods to review work and provide a regular review/retrospective point.
Second good point about Kanban: its easier to pick up.
On the whole I found the two Kanban teams needed less training, instruction and coaching than the BWR team (some of whom had been on Scrum training courses.) However, there was some work needed to map out the correct process flow for the Kanban teams to have on their boards.
For anyone creating a Kanban board some advice: big boards a better, involve the team and use coloured tape – insulation tape works well. In the past I’ve just drawn lines on the white board, with Kanban there is more movement and the lines are lost so I used insulation tape to create hard barriers.
A Kanban board
I expected the work in progress (WIP) limits in Kanban to pose a problem. I was expecting managers and others to want to break the WIP limit. This didn’t happen but I found developers breaking the WIP limits.
Developers broke the limits not because they disagreed but because they were accustomed to starting new work when they hit a problem. For example, a developer is working on item X, they come to a block on X so start on Y. In the extreme they block on Y and start on Z. You now have three partially done pieces of work. Trying to instil a “resolve the block” approach was difficult – largely because this organization was overloaded with managers and developers had been “taught” not to resolve these type of issues themselves.
In this organization lots of things blocked. Blocked on other teams. Blocked on IT support. Blocked on administration, etc. etc. I didn’t like it but I added a “blocked” column to the Kanban board. I also added a tally sheet to identify the biggest reoccurring blocks. I then tried to have the managers feel responsible for resolving the blocks. This was only partially successful because managers were themselves blocked because there were so many other “managers” who wanted their time. (The organization used talk as a substitute for action.)
All the teams held daily stand-up (Scrum) meetings and I encouraged the team manager, product manager and BAs to have their own pre-meeting to ensure the prioritisation queue was full and correct. This worked well.
I had the teams tracking cards by making a note on the back of the date as it progressed across the board. The date when it entered the priority queue, the date when work started and stopped, the date when it went for sign-off or test and the date when it was “done.” This data (when complete) was fascinating and started to reveal issues and opportunities, and give an idea of flow. In fact there was so much date I was overwhelmed.
In conclusion I find myself remembering Karl Scotland’s comment from a few months back. Karl had a quote which was something like: “The emphasis in Scrum is on being Agile and improvement follows; the emphasis in Kanban is on improving and being Agile follows.”
In the past I’ve shied away from labelling a team as doing “Scrum” or “XP” because I was aware they were not going it completely and we weren’t seeing all the benefits. I don’t feel that about Kanban, I’m happy to call these teams Kanban. Yes they had problems but we were resolving them. The teams were on a journey.
As anyone who has read Changing Software Development will know, I’m an advocate for incremental improvement. While Scrum (and XP, etc.) hold out the promise of overnight Agile – one big bang and your there – I see Agile as a journey not a destination.
Notes on a Kanban software development experience Read More »
This is the third in a short series of blog posts which started with me explaining why I don’t like Gantt charts as a “project” planning tool and then suggesting an alternative approach.
Short term project planning (1 to 6 months)
A regular release cycle is the first step to delivering. Decide what your cycle is, 1 month, 2 months – probably no more than 3 months and do work in those chunks. Size the work to fit in the cycle. Within the release cycle have several work cycles (iterations) of several weeks length. The more variable or unknown your work the shorter your cycles.
Its OK to sketch out what will be in upcoming iterations and cycles but don’t get too attached to what is allocated where. It will change. And for that reason don’t go into too much detail – certainly no estimates.
All work that goes into the iteration or cycle is rigourously prioritised – 1, 2, 3, … to N. (As mentioned in Run Scope Creep backwards recently.) At the start of each cycle, then each iteration, the team – and business owner – can have detailed conversations about what is to be done and how it is to be done. It doesn’t make sense to have these conversations earlier – because less is know about the need and thing might change. And it doesn’t make sense to have these conversations later because that is too late.
These items aren’t done until they are completely done. All or nothing, no bugs left to fix, no UI to complete, no db schema to revisit, etc. Done as in ready to go to customer.
Ideally each release should have its own small goal but sometimes it ends up being a hotch-potch of things. And ideally the team should commit to doing the work. Many Agile books and methods talk about the importance of commitment, I agree but… its hard to get one day-1, it comes with time.
Roadmapping: Medium term (3 months to 1 year)
For the 3 months to 1 years period create a near term product roadmap. The roadmap is a living document which should be changing. It should be marked with the time buckets of your cycles and no finer. Things can and should move between buckets but everything except the bucket dates is flexible.
A roadmap is not a direct substitute for a project plan. Roadmaps are different. They are need driven not date driven. They are an artefact of product management not project management. They are a learning tool not a commitment. They create discussion not enforcement.
And yes, sanitised version might be shown outside the company to customer but only with the caution that it might change.
Roadmapping: 1 year to 5 years
All roadmaps are rolling but beyond the next year it doesn’t make sense to have so much detail so have another roadmap for the mid-term. You can get more general, more speculative simply because more things can change. So keep your near term and long term roadmaps separate.
This roadmap should show major events inside and outside your company: planned IPO, proposed legislation changes, anticipated competitor products and major event with your major customers.
So why are the roadmaps I describe better than project plan Gantts? Well…
• They are designed to keep the debate open and keep discussion on what is needed
• They do not forecast into the future they are one scenario
• Items in on the roadmap turn into goals supporting the larger visions for the product and company
• Dates are kept vague and planning is separate from commitment
See my Product Roadmap pattern for more about roadmaps, this is part of my business pattern series.
Beyond 5 years: Strategy and scenario planning
These technique tends to work best in the short term. In the long long term where you are going, what your product looks like are a matter of business strategy. Top management should pass down some direction here. Planning for the longer term needs different tools. For the long term – 5 years or more I suggest you start with some scenario planning.
There you go, a rough guide to project planning. Some people would call it Agile project planning, personally I don’t care what you call it. If you want more details it depends on your situation so get in touch.
• Project plans (part 1 of 3): Why I don’t like Gantt charts
• Project plans (part 2 of 3): Turn the question around
• Project plans (part 3 of 3) – Planning cycles
• Run Scope Creep backwards
Project plans (part 3 of 3): Planning cycles Read More »
As my last post was a moan I promised to give some advice on how to go about project planning. There is a lot to this and I can’t possibly give all the details in a short blog so I’ve split this into three blog posts. Still there is a lot more I could say.
Turn the question around
Instead of a confrontational question and defensive answer have a rich discussion.
When someone asks “when will it be ready?” turn the question around – (1) When is “it” needed by?
When does your business need a product to use? What are the important dates for the company – end of quarter? end of year? trade show? competitors? contract signing?
I always prefer running a project were there are hard business driven milestones to make. Dates which mean something.
Next (2) What is needed on these dates?
Unfortunately too many conversations in the software business seem to involve someone saying “I need everything and I need it yesterday” – not a very useful position, and if its true you should start looking for a new job because there is no way anyone can make that happen.
Look at each of those date and define what is needed. What are the qualities of the thing that is needed? What is the minimum marketable product or feature for that date?
At this point you know: what is needed, how long you have, and who you have to do it.
Yes, I slipped that last one in because its true: you have the people you have and in the short run your resources are constrained because of Brook’s Law and because hiring people takes time.
So within these parameters the engineering team can now come up with options. How can this need be met within this time with these people? This activity separates the real engineers from the rest. Engineers create solutions within these parameters.
Wikipedia (today) says: “Engineers are concerned with developing economical and safe solutions to practical problems, by applying mathematics and scientific knowledge while considering technical constraints. As such, the work of engineers is the link between perceived needs of society and commercial applications.”
And if you cant? Then go back to the “customer” and discuss it. Find a smaller feature, extend the time (shoot for a different trade show), find an alternative way of delivering.
(Tom Gilb has some good examples of engineering solutions into tight constraints and his impact estimation tables can come in useful here.)
Vision
All work needs to be guided by a bigger vision. You can call it a vision or a goal if you wish I don’t mind. Personally I like the term BHAG – Big Hairy Audacious Goal. You’ve got to know where you are going to give context to everything else.
There are two ways to run a project. The first – exemplified by Gantt charts – is to get all the pieces of string you think need to do the project, lie them end-to-end and see where you get to. If you read my previous post you’ll see why I don’t like that approach.
The second way, I’ll call it the BHAG way, is what I discuss above. Decide where you want to be, when you want to be there, what is will look like and then find a way of getting there. How you get there is called engineering. It is what engineers do.
My favourite example of the BHAG way? The Digital OpenVMS team committed to a delivery schedule and said “We don’t know how to achieve this, but we commit to finding a way.”
• Project plans (part 1 of 3): Why I don’t like Gantt charts
• Project plans (part 2 of 3): Turn the question around
• Project plans (part 3 of 3) – Planning cycles
• Run Scope Creep backwards
Project plans (part 2 of 3): Turn the question around Read More »
The first of three posts: this one sets the problem and the next two attempt a solution…
A friend e-mails to say that his team have almost finished their project plan. There is something about the words “project plan” which are like a red flag to this bull… while I’m prepared to admit I may over react sometimes I don’t think I’m completely crazy and I think I can put some logic behind my dislike of “project plans”.
Lets start by defining what I mean. Its not so much a “project plan” – after all I think planning is a great way to explore your problem. As a mental exercise a team planning what they are going to do is good. And I’m all for a bit of critical path analysis to work out what the dependencies are. And I think scenario planning is great. From what I’ve heard war-gaming is another good for of planning.
I should admit to an assumption. When this friend says “project plan” I’m immediately imagining Gantt charts and PERT diagrams. There are many other forms of plan which make a lot of sense. I think product road-mapping is an essential activity. Its specifically Gantt and PERT charts that make me see red – and as layman in the field I don’t see much difference between Gantt and PERT to be honest.
So what is it about Gantt charts? Here’s some:
• The charts are built on estimates and estimates are just that, estimates. They are not commitments, they are not fact, they are not enforceable. Therefore Gantt charts are not an accurate way to plan a project.
• The charts create illusion of control; here is a piece of paper which says the project will be done on 1 May 2010 – or whenever. It won’t. That date is the one date the project will not be ready. It might be ready before, more likely it will be ready after but that one date is fantasy.
• Many – too many – discussions now centre on “why are you deviating from the plan” – “why are the estimates wrong?” – “how can we get back on plan?” – arguments centre over the plan not the thing being produced.
• There is now work to do “correcting” the plan; updating it for what has happened and attempting to re-forecast. This makes work but does not add value.
• The charts are never accurate but they are treated as such.
• The “plan” is taken as a form of commitment.
• The further out the plan the more inaccurate it is: planning for next month is one thing but 5 years out? There is simply not enough information to do it accurately. Plus, things change too much, the further out you look the more things can (and will) change.
Basically, what I’m driving at is: once the plan is in place disucssion moves from what is important (what are we building? when do we need it by?) to the plan. Its goal deferment.
There is a lot more I could say here but I’d be repeating myself. If you are interested read An alternative view of design (and planning) and the section in Changing Software Development.
Fortunately I showed this to my friend and he says they’ve not drawn a Gantt chart, phew! But before he tries I need to give some advice, the next two post will try and do that.
• Project plans (part 1 of 3): Why I don’t like Gantt charts
• Project plans (part 2 of 3): Turn the question around
• Project plans (part 3 of 3) – Planning cycles
• Run Scope Creep backwards
Project plans (part 1 of 3): Why I don't like Gantt charts Read More »
One of the criticisms levied at Agile software development is that of Scope Creep. Project managers seem particularly sensitive to this issue, probably because its been fatal to many projects before and thus Project Manager are trained to watch for it and come down hard on it.
However, I’ve come to the conclusion that in a lot of Agile work the secret is to run Scope Creep backwards. Rather than have a project get bigger as more and more requests are added to it what we need to do is have it get smaller over time. Let me explain.
Typically a product/project gets specified and then work proceeds, as it does more and more “change requests” appear, things people hadn’t thought of, ideas for what else it could do – scope creep. (See my piece, Why do requirements change from a few years ago if you want to know more about the why.)
As projects grow in size the chances of them meeting any deadline diminish. This is why traditional Project Managers look at Agile and throw their hands up in the air seeing rampant Scope Creep.
But if you run it backwards, write the spec, write it as big as you like, who cares how big you write it or how many features you put in. The bigger it is the less people will remember. (See my piece on the Documentation myth for why big documents don’t work.)
Allowing everything into the spec means you don’t have to have those hard conversations about what’s in and what’s out. Everyone gets to feel there thing is in.
Now prioritise. If its really big make a first pass with MoSCoW rules. Anything which isn’t MUST have can be ignored for now – you’ve probably got more must’s than you have time for anyway. Generally I’m not a fan of Moscow rules but they are OK as a starting point when you have a lot of requests.
(Note: that’s MoSCoW rules, not Moscow rules – personally my only Moscow rule is avoid the city, my last trip there was awful so future visits are restricted to changing planes.)
Now within the MUSTs prioritise 1, 2, 3…. N. No duplicate priotities. This way when something increases in priority ther things fall automatically, people immediately see the consequences of their decisions.
What criteria you use for prioritisation is up to you. Doing it by value makes a lot of sense, so does doing it by risk. It also makes sense to get some early wins under your belt.
Next work through the items in priority order.
Anything that comes in later (Scope Creep) has to be slotted into these priorities. Add a new item with priority 3 and everything 4..N moves down one.
Notice I’m not interested in change control. I don’t make any effort to restrict the work that is asked for. All I do is demand it is prioritised along with everything else. Valuable ideas can arrive at any point in the work so why draw an arbitary line to differenciate what was “in scope” and what was “out of scope.” The objective is deliver value.
Now as you work through the list you will find that with items 1..N’ you have a a valuable product. Something you can at least show to customers, something people might even pay money for today.
Once you start getting real feedback – especially the sort represented by money – everything changes. Once you know what real customers want you are, literally, in business. You’ll want to add new items and look again at your priorities.
At this point you’ve saved yourself the work N’..N plus all the Should Have’s, Could Haves, etc. This is running Scope Creep backwards.
What all of this highlights is that building a software product isn’t a binary decision. It isn’t “all of this stuff” or nothing. Its an analogue thing not a digital thing. Traditional projects spend a lot of time trying to prevent scope creep through tough change control procedures but really it doesn’t matter when a request comes in. What is important is what is the value it will bring.
Value, not arbitary dates, should be the criteria for deciding what to build.
Running Scope creep backwards Read More »
