allan kelly

Tom Gilb sent me a link to this presentation from Ryan Shriver about the use of Scrum with Tom’s EVO techniques. Very interesting, and it returns me to the subject of IT/business alignment – which in part is also a discussion of where Agile software development is wrong.

My blog from last week touched on this but I think its worth spelling this out. There are two problems with Agile, and two answers:

Question 1: Why do Agile techniques work? – forget the technical explanations of why TDD is a good thing, why RUFD is preferable to BUFD or how value stream mapping helps. Think for a moment: there are a lot (an awful lot) of software development groups which are not effective.

Problem 1: How can it be that the same set of (Agile) practices can help all these groups?

Answer 1: Because these groups all have similar problems, probably because too many people were taught (and believed) the same set of practices that have created the problems.

Question 2: How can Agile work without business alignment? The basic Agile techniques pay little attention to what the business wants. The XP onsite customer is really lightweight, XP assumed the customer knows what they want but often they don’t. Scrum is better but the product owner is still left to their own devices, Scrum says little about how the product owner knows what’s what.

Problem 2: How can Agile be effective without business alignment?

Answer 2: It doesn’t have to be. The MIT Sloan Review piece shows that just getting to be effective is such a massive improvement over being ineffective that it is a success. You don’t have to be aligned with the business to show improvement.

In the long run, for organizations which have effective Agile teams these problems will show up. And in time, as more companies achieve Agility, the competitive advantage of Agile Software Development will diminish.

The advantage will move to the companies that address these second order problems. Such companies need to: learn how to improve Agile practices beyond what we currently know, and, get IT aligned with the business.

In an Agile world this means pushing Agile further. Tom would call this Evolutionary, I call it learning, it amounts to the same thing in the end.

The Financial Times carried a piece on Wednesday entitled ‘Agility: Flexibility takes over from planning’. It starts off about agile business but actually spends most of piece talking about agile software development. Both the presence of the piece, and the evidence in it suggests agile is getting more prevalent.Nice definition of company agility from Professor Donald Sull of London Business School:

“he defines [agility] as a company’s ability consistently to identify and seize opportunities more quickly and effectively than rivals.”

What is interesting about this definition is that it says nothing about what you do, or how you do it. Instead it measures agility by comparison with your competitors.

Slightly disappointing that the examples the article cites are the same examples that come up again and again, Zara, BT and perhaps slightly new, Borland.

If you have the time, this piece from the same issue is also worth a look: ‘Why do so many technology projects end in failure?’ This picks up on the EDS / BSkyB court case I commented on a couple of weeks ago.

Its a shame the editors didn’t link the two pieces. In the long term Agile is about seizing opportunities before your competitors, in the short term it is about delivering IT projects on time, on budget and without court cases.

I have spent the last two days at the XP Day 2008 conference here in London. A very good conference and highly recommended. Unusually for me I wasn’t speaking at this one which made for a nice change. No worries about presentations, rooms and being in the right place at the right time.I picked up a whole bunch of ideas, and had another bunch of thoughts. Before I forget them I want to quickly record them – some of these ideas are probably worth an entire blog on their own.

Multi-media keynotes: Both keynote speakers incorporated multi-media elements in their presentation. Jeff Patton included music and Yvonne Rogers video. I think this is a sign of two things: multi-media is becoming the norm; second, people are increasingly bored with the PowerPoint (’death by’) and presenter talking at you (chalk-and-talk) type presentations.

Media companies are more receptive to Agile: A lot of the people attending XP Day had connections with media companies, the BBC and BSkyB were both represented, as were set-top box manufactures, advertising companies and others in the media. Of course there were old industry (pharma, banks, etc.) there too but they have always around.

This is a trend I’ve observed elsewhere. It seems to me that media companies are more receptive to Agile software development than others. There are perhaps two forces at work here. Firstly media companies are fairly new to software as a key part of their business. Yes they always had IT systems but not as part of the product. Agile is the current ‘best practice’ when the BBC, Sky, etc. started to development it was already the norm.

Perhaps more importantly, the second force: media companies expect just in time delivery. You can’t delay the 6 O’clock news because it isn’t ready, you know well in advance you have 30 minutes to fill, and you have to prioritise. The nature of media companies is closer to Agile development so they are more receptive.

Process as a substitute for team work? Of cause Agile software development is all about development process and practice. But scratch deeper and everyone agrees development depends on good people and team working. In fact team working has got more important as software has got bigger and more complex.

It seems to me that a lot of what we call process is actually trying to substitute for team work. We spend time talking about the process rather than the team, we devise process rather than build teams. Somehow we expect the process to substitute for a good team.

Another Agile failure mode: I forget who mentioned this but someone pointed out that Agile teams often fall apart when someone leaves, particularly if that individual is forces to leave by redundancy. This kind of fits with my experience. When an Agile team works well the team bond. The departure of one person is likely to trigger others to leave of their own accord.

Agile Business: This is a topic I’ve been thinking about a lot, so has Chris Matts and others. David Stoughton and Nick Coutts did a great session on the future of business. Not necessarily Agile but a lot of the trends they see in society and the economy suggest the successful business of tomorrow look a lot like Agile organizations.

One point that stuck in my mind above all else was the suggestion that we have moved from a supply economy to a demand economy. Rather than successful companies being those who could produce things the cheapest, and persuade the most people to buy them – a push system – the successful companies of the future will operate a pull system. They will produce what the customer wants, when the customer wants it. Rather than being cost driven they will be value driven.

For my money this was the best session at the conference.

Future of Agile: There are many many software development organizations that are a mess. A little bit of Agile can fix make a great improvement. Agile software development thinking has a long way to run and an awful lot of implementation to deliver. However, Agile has flaws.

One of these flaws is that if you examine in too closely it collapses. The Agile manifesto is actually pretty meaningless, nobody could really disagree with it. If you try and write down a high level value statement of what Agile delivers it sounds pretty much like all the other methodologies and tools that have proved to be snake-oil in the past. Even some of the Agile practices don’t stand up to examination, e.g. pair programming – most developer hate it! Retrospective? – most companies don’t do them.

Agile is still state-of-the-art, the best we have.

It is also a marketing term; a term that collects together a bunch of ideas and allows them to be labelled and grouped together. As a marketing term it is open to abuse.

Increasingly people are increasingly asking: what next?

The people asking this question most often are those with the most experience so some of this questioning is simply boredom, they are looking for the next thing. Some of it is very real and necessary.

(There is some excitement about a new Agile methodology called Kanban, I don’t know much about it and I don’t think this is the next big thing. Something here, but I haven’t had a chance to read it myself.)

So what next?

To me the next big thing will represent a break with Agile. The Agile community came from the Patterns community, Agile is not Patterns. The Patterns community came from the OO community. Patters are not OO. So we shouldn’t expect the next thing to be Agile+.

The next big thing will be People and Team. Software development always comes back to people. Each successive move (OO –> Patterns –> Agile) takes us closer to people. As I said above, sometimes we use process as a substitute, sooner or later we need to address the real issue.

The trick to improving your software development efforts is quite simple: improve your people. Expose them to new ideas, get them learning, help them put those new ideas and learning into action.

This is the second of two follow ups replying to some comments on the blog. What’s interesting about this one is that its from someone who I don’t know, or rather has chosen to hide their name. Still they have a great memory because they’ve linked something I said a few days ago with something I said last year. Now there’s paying attention! And its good for me because it helps me improve my own understanding.Anyway this comment asked how I reconciled my comments in the personal retrospective entry where I said I should have done more quicker, with my comment last year (last thing your people need is a strategy) where I suggested managers should spend time observing before acting. Good catch who ever you are.

So, at the risk of sounding like a politician trying to wiggle out a difficult question I’ll try and explain and perhaps I’ll shift my position a little. There is no simple answer so please bear with me…

To some degree it is a question of context. One size does not fit all, and it is important to look at the problems facing an organization before acting. The trouble in the ‘solutions’ oriented world is that you risk applying the wrong solution if you are not careful. Just as it is wrong to apply a predetermined solution before looking at the problem it is wrong to use a predetermined time frame before acting.

Which leads to the question: how do you know what the context is? Well you have to look around, observe what is happening and talk to people. I still believe the people on the ground want to do the best job possible and most likely know what needs doing. So I think its more important to talk to the people doing the tasks than it is to those who nominally control what is going on.

If the people doing the tasks, and those in leadership positions, aren’t seeing the same problems, and aren’t trying to go in the same direction then they positions need to be reconciled. That might mean explaining to the leadership why their model of the world is wrong. Or it might mean explaining to the workers that the company doesn’t want that.

The key point both ways round is not to assume you know all the answers, keep an open mind yourself. Collectively the team will know the answers, and one person may know more than most. I see a manager’s role as unlocking those ideas.

Going back to my personal retrospective post, one thing I didn’t mention was that during my first week at the client I held a reverse retrospective with the development team. This was a little like a project retrospective but since there was no single project to review, and others on the team were also new we had nothing to retrospect about. I call it a “future-spective.”

I gathered the team together for an afternoon and we talked about problems we faced and the problems the management saw. We also talked about improvements we could make and created a long list of things we could do. We then collectively prioritised them and worked out what actions were needed.

One way I would go faster in future is to implement the top priorities on that list faster. I realised when I did my personal retrospective that it took us months to implement some of our ideas, and as a result it took longer to see the benefit and prevented us from moving onto other initiatives.

In future I’ll probably use the future-spective technique but I’ll implement the suggestions sooner. And I’ll go further, I allowed myself to get caught in the ‘no time to improve’ trap in places and after 3 or 4 future-spectives stopped doing them.

As my anonymous commenter pointed out there was quite a gap in the three-month period I talked about last year and the one-week period I talked about recently. To be honest that gap surprises me, I’m not sure why I said 3 months last year, it does seem quite long given what I learned in my personal retrospective. However, I’m also surprised that it only took me a week to diagnose so many issues. So perhaps one week is a minimum and 3 months a maximum, ask me again after I have some more data points.

Again its a matter of context, I’d like to think it is always less than 3 months but I’ll be surprised if its as short as 1 week. In general its is better to delay making decisions until you are sure of the situation but once those decisions are made it is better to act quickly.

One way to ensure that decisions are acted on quickly is to involve more people in making the decision. Although having more people involved may delay the decision when one is made these people share the decision and share the responsibility for acting.

I still stand by my comment that when you are new the last thing your people need is your strategy (actually stole that idea from Lou Gerstner in Who Says Elephant Can’t Dance). However, you do need your own personal strategy. You need to know what you are doing and what you are aiming for.

That personal strategy needs to have a short term element (meet, watch and listen), a medium term element (find problems and opportunities, get peoples ideas, get agreement) and a long term element (do whatever it is you were asked to). And your personal strategy needs to tell you when to wait, watch and learn – something my friend Klaus Marquardt refers to as Proactive waiting.

I don’t know how many readers this blog has but I don’t think it s many. In fact I tend to think I know them all. Well, that’s a illusion I can be pretty certain to dispel. I know the ones I know, because they comment on the blog, or they tell me that read it, but by definition I can’t know who I don’t know – if you get my drift. If I now find someone who reads my blog who I didn’t know read it I still don’t know how many reads I have, I just know more of them.What is gratifying is that although the reader base may be quite small I get quite a few comments. And sometimes I get comments in e-mail not as comments on the blog. So it was that Mick Brooks e-mailed to point out an ambiguity in my blog posting, Software as an Asset, I wrote:

“Your software is worth $10,000,000. Next year, with inflation and depreciation the software may only be worth $9,000,000. So there is an incentive to invest up to $1,000,000 in improving the software, increasing functionality, or code quality, or making it easier to use.”

And Mick asked:

“I don’t understand why there is an incentive to invest *up to* $1,000,000 – why
not twice that, or more?”

Well he’s right, I my thinking was sloppy. I skipped several steps in and jumped to a conclusion. The nature of blogging is that it is meant to be fast, what you think of now, not too polished. Anyway, let me think about this a bit slower…

What I was trying to say is: if your software depreciates by $1,000,000 over the course of a year there is a strong incentive to invest in it. At the moment software isn’t carried as an asset on the books, therefore it is worth $0 this year and $0 next year. Therefore, why invest in it?

Now suppose we wish to keep the software worth $10,000,000 each year. We now have to do something to add 10% value to the software just to stand still. Therefore, anything we can do that costs less than $1,000,000 is worth considering. What we really want to do is spend as little as possible to increase the value by as much as possible, i.e. maximise our rate of return.

There are two possible ways value the software. Firstly, remember I imagined a tool that could value our software? Well, what we need to know is what parameters that tool looks at, then by targeting those parameters we can increase the value.

For example, from a product perspective: if adding some new functionality to the system will add value we might do that. Or, from a programmer perspective, if improving the code quality, increasing cohesion and reducing coupling say, will increase the value we could do that. In this way the parameters the tool uses will guide us what changes we make.

The second way we might value the improvements to the software is to look at the effort expended in enhancing it. It is already possible to capitalise the cost of research and development projects, i.e. show expenditure on R&D as an investment on the balance sheet. So, we could argue that spending $100,000 on a programmer to make changes to the software increases the value of the software by $100,000.

We have to be careful with this line of arguing because we could end up counting every bug fix as an asset. WorldCom did something similar with their telephone network and look what happened to them.

Which ever way we value the software the trick is to find the way of increasing the value as much as possible by paying as little as possible – the rate of return. Once you know this you can compare such a project against others in your organization. So, if spending $100,000 to improve the software results in an extra $1,000,000 of asset on the balance sheet it has paid back 10 times.

Conversely, if spending $5,000,000 on a replacement piece of software results in an asset worth $10,000,000 (and the old software is binned) the return is only twice.

But, doing nothing, let the software stand still looks less attractive than it did when the software was not counted as an asset.

Thus, valuing software as an asset gives us a new way of thinking about how we manage the lifetimes of our software.

Back to Mick’s point: my example was a little brief. The real test is whether we can see the value of our software asset increasing faster than other assets. If it is then it is worth investing in. Whether this is a $100,000, $1m a $2m investment.

As I said in my previous blog entry I recently finished a major assignment with a start-up client. The company had a lot of problems when I started, and the development group had its share of bad problems. I think I’ve left them in a much better shape than when I arrived.Last Friday I took some time today to reflect on what it was I did and how I did it. A personal retrospective if you like. This took place between me and my word processor. I just wrote and wrote…. I tried to understand what I did right, that is what I would do again. I tried to understand what I did wrong, things I would avoid in future. And most of all I tried to understand what I would do differently.

To help me I had my personal journal, a diary of events, thoughts and reflections I’d kept during the assignment. The entries were perhaps not as frequent as I would have liked. It started well but after the first few weeks I got very erratic until close to the end. At the time the journal was useful for thinking things through in my own head, understanding what was happening and finding courses of action.

What I found striking was that in the first week I had diagnosed most of the major problems with the development group and company. Some of these I had managed to fix, some remained problems when I left – you can’t fix everything I’m afraid. But it was just striking how much I had worked out so soon.

I normally get sceptical about the expression ‘hit the ground running.’ I’m not sure if anyone really does it. But in this case I’m glad to say I was up and running within a week of getting there.

So what were my big insights from this personal reflection? Two really. Although I consider myself a good communicator (think of this blog, think of the book, etc. etc.) I really could have communicated even more. Particularly upwards.

My second finding: I wish I’d done things faster. There are a few decisions I seemed to hold off doing for weeks but in retrospect worked out for the best.

Any note to myself for the next assignment needs to read: be fearless, go faster.

An interesting piece from Monday’s Financial Times – Study urges IT valuation rethink. What this study says is: IT is an asset, therefore companies should value the asset show it as an asset on the books. This isn’t such a new idea, some people have been arguing this for a while but INSEAD and Micro Focus have written a report and got some publicity for the idea. (See also the Micro Focus press release.)

I have to say I agree with the central argument being made here. IT systems are an asset to a business, they allow the business to operate, they cost money to develop and without them there would be a big hole in the organization. Companies count machines, buildings and even brands as assets but not IT systems. In most companies if the IT system disappeared tomorrow it would need to be replaced and that would cost money.

Actually companies do carry some IT elements as assets – hardware. Mainframes, PCs, printers etc. are bought, counted as assets and devalued over time. So actually, when we talk about putting IT on the balance sheet we are really talking about the software.

Now this has some interesting consequences – not all of which my younger self would be happy to hear. Suppose for the moment software is considered an asset, what happens?

Well firstly you get an increase in your balance sheet. This is good, it shows the company is worth more. It also means that software can be depreciated over time, as with hardware, we can aim to write it off over time. This might make it easier to end of life a system after a period of time. Given that software deteriorates over time this could be a good thing.

However, this also means you can’t dispose of it overnight. How many developers faced with a poor system cry: “We can’t maintain it! We need to re-write it” ? How many managers accept this argument. If software is an asset this decision is changed significantly.

Neither could you throw away failed projects so easily. Many companies bury failed projects. Rather than admit to an embarrassing failure they simply don’t and loose the costs in expenditure. However if you are building an asset and suddenly it disappears then you will have some explaining to do.

For both these reasons (and others) putting software on the balance sheet will make it more difficult to get rid of existing systems. This is the bit my younger self would hate.

When I was still programming I saw lots of really bad software. I always wanted to re-write it and sometimes I did. However rewriting is no panacea. It is often (usually) better for the business to continue with the old system. But in doing so you have to improve it. You have to move it in the right direction or else it will become impossible to work with.

This is what interests me now: how can a business maintain legacy systems? This is the challenge I now embrace as a manager.

Now, if we start counting software as an asset there will be more incentive to maintain the software and keep it alive for longer. This means we will invest in the software to keep the asset. Of course we’re going to need tools to value the software – INSEAD suggest one tool and my guess is Micro Focus can sell you a product to do so.

So what is the net effect?

Well I think this approach would allow better reasoning about the software life-cycle. If a piece of software is only going to be used once then it isn’t going to be an asset and we treat it as such. However if we are going to count it as an asset we can decide whether we are going to devalue it over time, and plan in advance for retirement or replacement. Or if we are going to maintain it and keep (or increase) the value on the books then we need to invest in it. Either way we get to reason about the future of the software more logically.

Now imagine we had a tool that assess software value. Point it at the source code control system, put in some economic numbers and bingo! Your software is worth $10,000,000. Next year, with inflation and depreciation the software may only be worth $9,000,000. So there is an incentive to invest up to $1,000,000 in improving the software, increasing functionality, or code quality, or making it easier to use. That changes the game.

(I’m sure Micro Focus, who specialise in tools for legacy (and particularly COBOL) systems are well aware of this. I for one hope we hear more about this idea.)

We tend to think our software will have a short life. But we already know that some software systems are 20, 30 or 40 years old. (Remember the millennium bug?) These are major engineering achievements.

We take it for granted than major engineering achievements are still there. The Forth Rail bridge was opened in 1890. Nearly 120 years later it still works, it is an asset to the nation. St Pancras station in London was reopened yesterday after refurbishment, originally opened in 1877 it is still an active part of the economy 130 years later and it is an asset to National Rail.

On this basis there is every reason to believe that software being written today will be in in use in 2127. Isn’t it about time we started treating it as such?

I’m just coming to the end of The Innovator’s Solution (Christensen and Raynor, 2003) – OK, I admit it, I can’t be bothered to read the last couple of chapters. Like so many books the first few chapters are very interesting, then the tend to get a little bit boring. I suppose this is because the first few contain the ideas while the later chapters are more about implementation. Still, I always make a point of reading the last chapter as this usually restores some of the feel of the early chapters.That said, its a good book and interesting. I suppose more people have heard of Christensen earlier book The Innovators Dilemma. So why did I choose to read this one and not the earlier one? Well I guess because it was a) newer, and b) be seemed to offer a solution where the other offered a problem. Having read this I wonder if I would have been better reading the first book after all. That’s not to say this isn’t a good book.

Clayton Christensen is most famous for giving the world the theory of disruptive technologies. Such technologies are ones which, well, disrupt, the current market. Current market leaders fall, distribution channels change and small companies get to grow fast. All good stuff.

What gets less publicity is the alternative to disruptive technologies, namely sustaining technologies. These are technologies that can be used by the current market incumbents to maintain their market. In this book Christensen and Raynor suggest that in many instances companies can choose to use a new technology as disruptive or sustaining, it depends in part on how you package and market it, what market you are in now and which market you want to be in.

They also suggest that in an established market the incumbents will always win a straight fight with new competition. The only chance a new entrant has is to use their technology as a disruptive influence. Conversely incumbents need to use technology (maybe the same technology) to sustain their position. So if you are a new entrant don’t both trying build a better mouse trap, and if you are an incumbent don’t bother trying to disrupt the market.

What is nice about this book, although it does make it longer in place, is that the authors understand that a different context needs a different solution. Unlike many books which say: faced with problem X you should do Y; these authors say, if you face problem X and you are an incumbent than do Y, if you are a new entrant do Z.

The formula for a new entrant with a disruptive technology seems quite simply. Enter the market at the low end, offer a cheaper product than the current incumbents. Rather than compete head on target people who don’t buy the current products (non-competition) with a cheaper, less functional product. Sell to people who don’t buy the current product or are over served by it. Then as your technology improves expand this base. Quite often the current incumbents will move out of your way because they don’t want the low end of the market – that is if they notice you at all!

Of course I summarise and miss out many of the details but you get the idea. If you want to know more then read the book!

Interestingly much of the book’s key message is shared with Crossing the Chasm, they have the same idea but come at it from different perspectives. The key idea is: start with a small niche and grow it.

The advice in Crossing the Chasm is for small high-tech companies who wish to increase sales of their product. Moore suggests you define your own niche, nominate it then grow the niche. In this case the new technology is assumed to be superior – in some way – to the incumbents.

The advice in Innovator’s Solution is aimed at existing, probably large, corporations who want more innovation. Here the niche is always at the low end of the market, it is always a niche defined by cost and price. In this case the technology is superior through a lower cost structure, therefore price can be lower. Over time the niche is expanded by moving the product up market.

Most of the small software companies I see are so concerned with getting a sale, sometimes at any price, that the idea of niche marketing is foreign to them. Sales (and any marketing) tends to be a shot gun affair where any sale is what counts.

Indeed the whole idea of marketing is pretty foreign. Increasingly I see the existence of well functioning marketing as the key differentiator between successful and not successful technology companies. When I say marketing I mean both outbound marketing – telling people you have a product, advertising, etc. etc. and (the lesser known) inbound marketing – understanding your customers, knowing their problems. For both you need to define your target market.

Trouble is: you don’t need a marketing department to start a high-tech company. You do need techies, scientists and engineers to design and build the product. You do need sales people, someone to get out and sell the thing that has been built. If you lack either of these you don’t have a company. So you always find engineers and sales in any company.

But you don’t need marketing. You can carry on with the sales and engineers for quite a while without marketing. But if you want to grow you either need marketing or you need a lot of luck.

This is a mistake made by many many software companies. Which might just explain why Crossing the Chasm, The Innovators Dilemma and Solution sell so well.