allan kelly

There is a re-occuring question I am asked from time to time, and I see on discussion forums. It again appeared last week:

“We are new to Agile and are looking to adopt a tool for managing the process… which tool would you recommend?”

There are variations on this question (replace Agile with Scrum/XP/Kanban/whatever, or replace “managing the process” with requirements/development/testing) or maybe its put as “Does TFS work with Scrum?” or “Is Target Process the right tool for XP?” but its the same basic question.

Anyway, I try not to take the bait on these but last week I couldn’t resist and answered it. My answer solicited a positive response from several people so I thought I’d share my answer with more of you and expand on it a little:

For anyone new to Agile (all flavours, Scrum, XP, etc.) I strongly advise avoiding all software tools for managing the process until you get the hang of working in the new manner manually. You will have a faster and more fulfilling learning experience if you do it by hand – pens, cards and white board – for a few weeks before you look at a tool.

There are several dangers with tools:

• You may mistake the tool for the process: e.g. using Rally doesn’t make you Agile
• The tool might not work quite the way you need to work: in your company the tool might not do something you need to do
• The tool might be too strict: when your learning you need a bit of slack
• People may communicate through the tool rather than face-to-face
• You may spend time customizing the tool when you should be focusing on the actual work
• And, when you find things don’t work quite the way you want them to it is much easier to change a card or a board than change the software.

Something magical happens when people work with cards. Abstract “work” becomes real. We relate to it in different ways when it has a physical presence and you can move it about. You don’t get that with a tool.

Once you have a tool the tool easily becomes an impediment to further change:

• To change you need to change the tool
• You need time to reconfigure the tool (if you can)
• You need time to move your data to another tool (if you can)
• You need to justify the change of tool to those who paid for it: “We paid $10,000 for these licenses 6 months ago, what do you mean you aren’t using it any more?”
• You need to justify spending money on another tool: “And you want me to spend another $5,000 on another tool? Will you use it this time?”
• And you have invested yourself in the tool, you don’t want to admit you got it “wrong”.

Nobody ever got fired for spending too much on index cards, or for throwing a pack away.

Yes I know there are genuine reasons people want to use a tool to help with their work. But choosing the tool before you change is putting the cart before the horse. In my opinion people and teams which spend time evaluating and deciding on their tool are resisting change.

Damn the torpedos just do it!

Do it by hand to start with, give it about 3 months like then then look at some free tools, don’t waste time on customizing anything. Give it another 3 months then you should have the experience to find the best tool for your team.

Don’t be scared to change tools – and don’t invest so much in one tool that you can’t change in a few months when you realise its not right.

I’m not saying “Don’t use tools”. To be honest my preference is not to use them, stick to paper, pens and boards. However, I recognise that on some occasions they can help (remote working springs to mind.) But, before you start using one be clear what you need to get from it.

The thing is: Agile is about learning, and you have a much better learning experience when you are manually managing the tasks. If you need a tool to manage them you probably have an over complicated process. When you insert a tool between you and the process you loose sight of what is happening. The tool is a box labelled “magic happens here” and complexity can set in.

There is a story. Its about Jack Kilby – not heard of him? You should have done, he was co-inventor of the Silicon chip. A few years after inventing the chip he helped Texas Instruments develop the calculator. But he regretted the passing of the slide rule.

He said: the slide rule gave engineers the numbers but they needed their intuition to know were the decimal point went. Was the answer 123,456,78 or 123.45678? The calculator took that away and he thought that made for poorer engineering.

Using an tool to manage your Agile process is like that. It removes part of your intuition, part of your understanding.

I stopped reviewing books in this blog a while ago but, having said a few words about Domain Driven Design Using Naked Objects last time I want to say a few words about another book I’ve just finished reading: Wired for Innovation: How Information Technology Is Reshaping the Economy.

For anyone who is concerned with the future of IT, technology in general, and improving the state of the art this book is well worth reading. At a little over 100 pages and £10 it is worth the investment.

The book is written by two academics, and it is a bit academic in tone and approach but there are some gems in here. Not only have the authors studied how IT is used they have made serious attempt to measure it. In fact, if you have read anything by Brynjolfsson before there is a good chance it was his work on the “productivity paradox.”

The authors know about measuring IT and have some fascinating statistics. For example, did you know that in the last couple of decades “IT intensive” companies have grown much much faster than those which are not IT intensive?

I don’t want to review the book but I do want to highlight some of the points the authors make for the benefit of those who won’t read the book. In particular, there are some facts here which need to be considered by those who advocate a focus on “Business Value” when developing software (of which I include myself).

Firstly the book resolves the “productivity gap”. This was the observation in the late 1980’s that despite all the investment in productivity enhancing IT there was no resulting increase in productivity in the production and GDP statistics.

The explanation turns out to be: time. There is a time lag between investment in information technology and when the benefits are seen. This has closed somewhat in recent years, instead of decades it is down to years but it still exists.

So: problem number 1 for the business value crowd, How do you measure business value when the benefit will not be seen for two or three years? Or longer still? How do you create a feedback loop when your developers are working today and you can’t measure the value for two years?

The second issue we need to pay attention is “the complementary nature of IT.” It turns out, when you look at the numbers that IT by itself doesn’t delivery a whole load of benefit. To see real benefit you have to combine it with other practices. There are seven:

• Move from analogue to digital processes
• Open information access
• Empower the employees
• Use performance based incentives
• Invest in corporate culture
• Recruit the right people
• Invest in human capital

If you want to increase “business value” then do all these things as well as your IT work. Trouble is, many of these things fall outside the bounds of your typical IT project. (And in part doing these is why there is a time lag.) If you don’t do these things then you will fall a long way short of maximising business value.

Put these together and you discover: a successful IT project requires organisations to change their processes. Which leads to another issue the book brings out: should business processes be counted as assets for a company?

After all, we count the hardware and software used for the process, why not the actual process? We invest in it, we spend money on it, why not count it?

This may sound irrelevant but it is important because: failure to count processes as assets means companies don’t invest in them. It processes were counted it would become easier to justify the seven things listed above. If processes were assets then money spent on them would come off one account and appear on another so it is easier to justify.

Where all this is leading is a stark reminder, next issue, that in many businesses, and among many managers, IT is considered intangible. The authors don’t challenge this assumption, to be honest, there is good reason why IT is seen as intangible. But it does mean that someone who talks about “Business Value” looks a bit out of place when “everyone knows” its intangible.

(Why IT is seen as intangible is probably another blog entry.)

Next, a lot of IT has delivered “value” it isn’t countered. Take for example a Google search. How much is a search worth to you? How much time and effort would you need if you had no Google?

But, while Google’s revenue is countered by the Government those searches aren’t. In fact, nobody, anywhere, attempts to put a value on Google searches.

Or take Wikipedia. How much is it worth? If I want to know how the steam engine works I can go out and buy a book. That will be counted. But if I read it on wikipedia it isn’t.

These are just examples of how IT systems change things and add value which isn’t countered. It is the kind of challenges we are up against when we try and measure “business value” in IT work.

And its why, for anyone who claims to know about measuring business value in IT, this book is worth reading.

A recent McKinsey quarterly has a piece on organizational Agility, “Competing through organizational agility” by Donald Sull. It s a reminder that “agile” means something outside of the software world.

That said, the piece is a little disappointing. The piece doesn’t say anything particularly new. The only insight I gained from it was separation of agility into: Strategic agility, Portfolio agility and Operational agility. It would have been nice to know if you can have one without the others.

The main article also contains a sidebox which suggests some companies are centralizing decision making to improve agility. Personally I find that idea a little crazy. As far as I am concerned centralized decision making runs counter to agility for two reasons.

First it moves power away from those closest to the action. Those who deal with the customers, market and problem are the best place to make decisions. Moving decision making to the centre is disempowering and adds several layers of message passing.

Second centralized decision making is too often slow, bureaucratic or politicized. It takes time for the centre to realise a decision needs making, time to gather the information, time to make the decision and time to send out the decision.

Interestingly some of the examples given in the piece (e.g. Zara) are the same example cited by the Lean guys. More evidence that Agile is Lean by a different name.

An interesting prediction from Kevin Rutherford for 2010: “I predict that 2010 will be the year that eXtreme Programming returns to centre stage.”

I tend to agree with Kevin, while his logic is sound I would offer an alternative rationale for the return of XP. It goes like this…

XP can be viewed from two perspectives: project or engineering practices.

The first of these is basically Scrum. The cognoscenti might debate which came first and which borrowed from which but basically the approach is the same: short time-boxed iterations. Yes Scrum adds in Scrum Masters, Product Owners, self-organizing teams and such but while XP’s description is more basic they have broadly the same approach.

As to engineering practices, there is a growing Software Craftsmanship movement which is aiming to put these back on the agenda. While Scrum has been centre stage in the Agile world many of the engineering practices have been absent. Uncle Bob Martin has been talking about this for a couple of years and I’ve observed it myself (see my post about the Scrum Wall.)

Scrum gained a lot of popularity during 2008 and 2009 but too many of those teams adopted the project management bit without the engineering practices. While that approach can bring benefits it also stores up problems for the future. As more teams take this approach more teams will see these problems and more teams will need to add back the engineering practices and take craftsmanship seriously.

Add engineering practices to a Scrum team and it isn’t very different from XP.

The question Kevin leaves unanswered is: XP 1 or XP 2? For me its Extreme Programming, first edition, “the old testament.”

Now, to extend Kevin’s prediction. I hope we will see the return of “requirements” in 2010, or rather, more focus on “business value” delivery.

The “what are we building?” question has been underplayed in Agile too date. This means the Product Owner, Customer, Business Analysts and/or Product Manager roles (call it what you will) needs to have more attention. Fixing development is the easy bit, the difficult bit is doing the right thing. But, you can only really address that question once you can deliver.

Living and working in London means a lot of the software people I meet and talk to work in financial services. It amazes me the number of programmers etc. that banks employ. And you hear stories, some really awful stories about the state of IT in banks. And I’ve seen a few in my time.

So, for all those readers out there who have to work with banks and other financial services companies I recommend you get hold of a copy of this weeks Economist and read “Silo but deadly.” This article lends some weight to those stories I hear in pubs, at conferences and during training courses.

Some highlights:

• Financial services companies are estimated to spend $503bn on IT this year, thats more than Governments, more than manufacturing and more than any other sector
• The chief risk officer at Deutsche Bank says: “Banks are essentially technology firms”
• SOA architecture is a problem, that’s Spreadsheet Oriented Architecture
• Data quality is a recognised problem

The articles suggests that the woeful state of IT in banks may have contributed to the recent problems. This is something I’ve suggested other links before now, e.g. the Moody’s bug.

Personally, I think one of the reasons why banks and other financial companies have such poor IT goes back to that quote above: Banks are technology firms but they don’t view themselves that way. As a result the way they don’t necessarily take the right approach to people or projects, and they don’t take the right short-term v. long-term view and so on.

Lest we get too negative about our industry you might also want to read this recent survey from McKinsey which suggests IT isn’t doing such a bad job, and outside the IT department corporations are pretty happy with IT on the whole.

Twice this year I have visited companies which want to try a more Agile way of working but who have requirements coming of their ears. I’m talking hundreds of pages of requirements. Sometimes as use cases and scenarios. Sometimes as big meat documents written by expensive consultants and bearing names like “Requirements Specification” or “Business Design.”

Its hard to argue with such requirements documents, in part because its hard to read them and understand what they are about. The thing they have in common is a level of detail. They go into great detail on what needs doing and how you might go about doing it.

Its not the first time I’ve seen such documents. I’ve been on similar projects before. One variation is a project I saw last year were the documents were not requirements documents but strategy documents. Several big name consultancies had been into this (media) company and do strategic work around a new product. Lots of presentations, lots of SWOT analysis and so on.

What all these documents have in common, apart from being expensive, is that they are worse than useless. They give the impression that things are know but they aren’t know. They knowledge walked out the door, what you have is a snapshot of someone’s understanding. But its a snapshot that nobody can really take in because its so big. And it is a snapshot that is rapidly aging.

As I’ve said before: the bigger a document the less likely it is to be read, and if it is read, then the bigger it is the less that will be remembered of it.

To which we might add: the bigger the document is the more there is to go out of date.

Yet because these documents exist people feel compelled to use them. They may even be mandated to use them. They don’t feel they can say to their boss: this was a useful learning exercise, no we will do it for real. So we then embark on a process that try to use documents which only get in the way.

Having someone investigate requirements or strategy can be useful. But think of it as a learning and feasibility exercise. And above all: don’t let that person leave the project.

At best these documents are historical reference source; as worst they are an obstruction.

Set a clear goal for the work and work out what you need to do to meet it. As Tom Gilb says, this should fit on one page of A4.

As you work towards this goal you will uncover requirements. You might want to note them down but remember there is no point in running far ahead of the development team.

Once you have something up and running – even in demo – what people think they want will change. So start small, get something working and get a team that delivers.

Just say No to big requirements documents. Say Yes to short, concise, goal to aim at.

Continuing about patterns… At the beginning of the month I went out to Bulgaria to teach pattern writing to some academics and students at the University of Sofia. It was the first time I’d been to Bulgaria so I was interested to see the country which physically, historically and culturally sited between Europe and Russia.

We made the class very practical, no PowerPoint, in fact apart from a 5 minute video it was computer free. Yes we talked, yes we used flip charts but most of all it was practice, this was about writing after all. Only about an hour in and we had our participants putting pen to paper to write some patterns.

For this we used a video about milk packaging from TetraPak, there is an English version on You Tube, as it happens we used the short German version but it was the pictures not the words which were important. After watching this we did some brainstorming to identify what was happening in the video, i.e. what solutions were being used to solve which problems.

Initially we asked the participants to think about three things: the state before the solution, what the solution was and how things were different after the solution. Once the groups had grasped this we expended these into the traditional parts of a pattern: context, problem forces; solution and implementation; consequences and all the other bits (known uses, etc.)

During the writing myself and my co-presenters (EuroPLoP regulars Klaus Marquardt and Didi Schueltz) circulated between the groups to answer questions, provide advice and deliver a bit of shepherding.

And then we repeated the exercise with different subject material. We did some brainstorming to identify ideas which the participants would like to write patterns about, problems in their own domain. We repeated the writing process and finished the day with a writers workshop.

So in the space of one day we covered all the pattern basics. Although not in as much depth as maybe we could have. We carried on into a second day but only used half of the day. Next time I’ll be sure to use two whole days.

On the second day we covered the ideas behind shepherding, talked some more about pattern form, answered some questions and ran a second writers workshop, this time on an existing pattern by an experienced writer.

More than ever I left feeling that patterns are a generally applicable knowledge management technique. Not everything belongs as a pattern but the form is useful in many domains. And pattern thinking (or perhaps pattern analysis is a better term) is useful even when the thing you are writing (or just analyzing) is not a pattern.

And what did I learn? Well…

• I really enjoyed teaching pattern writing and hope to do it again
• Using a video to provide source material worked well
• Not using slides worked well, it requires confidence but the result was better
• You still need to give the attendees something to take away so having support material is necessary. Still, I’m always disappointed with slides which are little more than bullet points after the class. We did hand out some of my writing about patterns and patterns form, some patterns about patterns and some links to websites with more information and patterns.
• Having co-presenters was good. This was an intensive class so it needed more than one person
• You really need two days to do this properly
• Having attendees experience what you do was far more valuable than just lecturing them

Next time I would like to include a pattern reading exercise too. Take a well written pattern and read it through as a group. I’m going to be delivering a software patterns course this week and I plan to try this exercise in miniature.

In my last blog (Magic happens here – ERP, CRM, SAP, BPM) I mentioned change programmes in passing but didn’t elaborate. So I will now. This topic isn’t unique to the BPM work I have discussed in the last few blog entries but it is related.

When I think of “Change Programmes” I am thinking of the scenario where an organization decided something must change. Someone – consultants, managers, senior folk – decide what need sot change and how, e.g. introduce a new IT system, change the business processes, etc. – and then set about making it happen.

From time to time I run across articles online, in blogs and in prints journals and books which address the question like:

• how to you motivate people to change?
• how do you overcome resistance to change?
• why do so many change projects fail?
• why do people resist change?

And so on. The common theme is that a group of people, lets call then Workers, are required to change their behavior in response to some initiative from other people, lets call them Managers – although they often go by the name of Consultants.

This is a problem many IT projects of all shapes and sizes face at times. Its also a problem faced within IT when Managers want to change the way the team works.

I don’t believe people resist change. I think people, for Workers are always People, are happy to embrace change when they see the advantage, when they have choice and when they are involved actively in the change.

For example: many people are only too happy to get a new mobile phone when their supplier tells them they can have a free upgrade.

Change problems only exist when one group of people want to impose change on another group. Its not resistance to change, its it resistance to being changed. Been told what to do differently.

As long as one group of people think they know best and try to impose change on another group people will continue resist change, articles like I mention above will continue to be written and change initiatives will continue to fail.

The solution is to change the way you deal with the problem. Those who want change need to involve those who need to change. If you are a Manager who wants a team to improve their performance then ask team what they could do to improve. Ask the team what they would do differently. Take the teams ideas and help the team turn them into action.

If you are an IT expert charged with delivering a new IT system – to support business change or otherwise – then ask the people who will use the new system what it should do, give them demonstrations as the system developers, ask their opinion, deploy the system early and incorporate the feedback of the Workers.

I’m not saying you ignore long term issues, strategy, corporate objectives and such. Tell your Workers about these things and harness their ideas. Integrate the ideas and goals into a shared understanding.

This advice applies to all change initiatives and all IT change projects. (My “Agile approach to BPM” entry a few weeks ago put this in the context of Agile and BPM.)

Too often companies use IT as a blunt instrument to impose a change in process, behavior and working practices. I sometimes imagine it as a dice game. Faced with imagined resistance to change and complex systems Managers commission an IT projects in the hope that it will create change. They grab the dice – very expensive dice I should say – and roll them in the hope that they will roll a six and win.

Surely we can do better and move beyond luck for success.

I want to draw a line under the BPM/SAP mini-series of blogs. I am more aware than ever about the need to learn more in this area and actually I already know more thanks to people’s responses to these blogs – a few comments and a few e-mails. I hope to spend time in future finding out more about this arena and why it presents so many difficulties.

(And I’m hoping to see Jez Higgins talk at ACCU 2010, Wrestling with Giants.)

However, I continue to have this feeling that the SAP folks just don’t explain themselves. It feels like people are constantly telling me: “you don’t understand…”. What I would like is someone to explain please? Dont just tell me I don’t understand. Please, tell me: what is it that is different? What am I missing here?

Until that happens the BPM/SAP world will continue to feel like a secret society of people who “understand it” but will not explain it.. Until then all this is a mystery, SAP will be a box marked “magic happens here.” I’ve looked a some websites on SAP and I’ve looked at some books. The emphasis on results rather than how only deepens the mystery.

So, let me close by posing a hypothesis, maybe some researcher can pick this up and explore it.

I suggest companies buy big application suites one the assumption that:

Cost of Purchase + Cost of Customisation (programming) + Cost of Change Programme

is less than <

the Cost of Developing a Product from scratch

And that:

Cost of ongoing licencing + Cost of ongoing Maintenance

is less than <

Cost of Maintaining a Development Team to maintain a bespoke product

You might also add in a saving in management time, headaches from managing software developers, speed to market and a few other variables but that is the basic equation.

My belief – and I have no evidence for this so it is conjecture (don’t you just love blogging?) – is that companies underestimate the cost of installing and customising these big packages, and simultaneously overestimate the cost of developing their own solution.

That second part is probably the most controversial bit. I suggest that when run well these costs are not excessive and not run away. The current crop of development techniques (yes, the Agile and Lean ones) together with modern tools and existing libraries can help contain these costs.

A couple of cases might demonstrate my point….

As it happens, I do know of a company in London which has taken this route. They have four developer using PHP to continually develop their CRM system and other business functions.

A few years ago a company I worked with introduced SalesForce to help the sales team and started to extend SalesForce to marketing and support. At first it looked like a large part of my project would be thrown away as SalesForce replaced it. After many weeks of talking to the third party integrator it became clear there was something we needed and the system wouldn’t do. We developed the functionality ourselves for a fraction of the cost and time it would have taken to have the third party customise (programme) SalesForce. but then, it was a software company which knew how to do these things.