October 2007

I’m just coming to the end of The Innovator’s Solution (Christensen and Raynor, 2003) – OK, I admit it, I can’t be bothered to read the last couple of chapters. Like so many books the first few chapters are very interesting, then the tend to get a little bit boring. I suppose this is because the first few contain the ideas while the later chapters are more about implementation. Still, I always make a point of reading the last chapter as this usually restores some of the feel of the early chapters.That said, its a good book and interesting. I suppose more people have heard of Christensen earlier book The Innovators Dilemma. So why did I choose to read this one and not the earlier one? Well I guess because it was a) newer, and b) be seemed to offer a solution where the other offered a problem. Having read this I wonder if I would have been better reading the first book after all. That’s not to say this isn’t a good book.

Clayton Christensen is most famous for giving the world the theory of disruptive technologies. Such technologies are ones which, well, disrupt, the current market. Current market leaders fall, distribution channels change and small companies get to grow fast. All good stuff.

What gets less publicity is the alternative to disruptive technologies, namely sustaining technologies. These are technologies that can be used by the current market incumbents to maintain their market. In this book Christensen and Raynor suggest that in many instances companies can choose to use a new technology as disruptive or sustaining, it depends in part on how you package and market it, what market you are in now and which market you want to be in.

They also suggest that in an established market the incumbents will always win a straight fight with new competition. The only chance a new entrant has is to use their technology as a disruptive influence. Conversely incumbents need to use technology (maybe the same technology) to sustain their position. So if you are a new entrant don’t both trying build a better mouse trap, and if you are an incumbent don’t bother trying to disrupt the market.

What is nice about this book, although it does make it longer in place, is that the authors understand that a different context needs a different solution. Unlike many books which say: faced with problem X you should do Y; these authors say, if you face problem X and you are an incumbent than do Y, if you are a new entrant do Z.

The formula for a new entrant with a disruptive technology seems quite simply. Enter the market at the low end, offer a cheaper product than the current incumbents. Rather than compete head on target people who don’t buy the current products (non-competition) with a cheaper, less functional product. Sell to people who don’t buy the current product or are over served by it. Then as your technology improves expand this base. Quite often the current incumbents will move out of your way because they don’t want the low end of the market – that is if they notice you at all!

Of course I summarise and miss out many of the details but you get the idea. If you want to know more then read the book!

Interestingly much of the book’s key message is shared with Crossing the Chasm, they have the same idea but come at it from different perspectives. The key idea is: start with a small niche and grow it.

The advice in Crossing the Chasm is for small high-tech companies who wish to increase sales of their product. Moore suggests you define your own niche, nominate it then grow the niche. In this case the new technology is assumed to be superior – in some way – to the incumbents.

The advice in Innovator’s Solution is aimed at existing, probably large, corporations who want more innovation. Here the niche is always at the low end of the market, it is always a niche defined by cost and price. In this case the technology is superior through a lower cost structure, therefore price can be lower. Over time the niche is expanded by moving the product up market.

Most of the small software companies I see are so concerned with getting a sale, sometimes at any price, that the idea of niche marketing is foreign to them. Sales (and any marketing) tends to be a shot gun affair where any sale is what counts.

Indeed the whole idea of marketing is pretty foreign. Increasingly I see the existence of well functioning marketing as the key differentiator between successful and not successful technology companies. When I say marketing I mean both outbound marketing – telling people you have a product, advertising, etc. etc. and (the lesser known) inbound marketing – understanding your customers, knowing their problems. For both you need to define your target market.

Trouble is: you don’t need a marketing department to start a high-tech company. You do need techies, scientists and engineers to design and build the product. You do need sales people, someone to get out and sell the thing that has been built. If you lack either of these you don’t have a company. So you always find engineers and sales in any company.

But you don’t need marketing. You can carry on with the sales and engineers for quite a while without marketing. But if you want to grow you either need marketing or you need a lot of luck.

This is a mistake made by many many software companies. Which might just explain why Crossing the Chasm, The Innovators Dilemma and Solution sell so well.

It is not often I feel sorry for EDS. Early in my career I had contact with EDS and I was not impressed. Since then nothing I have heard about them has caused me to change my opinion. So it is pretty unusual that I ever give them the benefit of the doubt, let alone feel sorry for them. But this time maybe I do…

Just at the moment EDS are being sued by BSkyB – part of News International – for failure to deliver on a project. In 2000 BSkyB agreed to pay EDS £48m to develop a new ‘state of the art’ customer service system. Unfortunately the project collapsed in 2003, BSkyB completed the project without EDS at a cost of £265m and are now suing them for £709m. The case is being covered by the FT if you want to know more.

Basically EDS claim that BSkyB didn’t know what it wanted. It seems the requirement was simply: a ‘state of the art’ customer service system to provide BSkyB competitive advantage over their competitors. BSkyB respond that EDS didn’t follow any recognisable project planning system.

I don’t know the rights and wrong of the case, what I do know is that businesses frequently don’t know what they want. I’ve written about this several times before (Software requirements and strategy and Requirements: A dialogue not a document for example). This case is about EDS and BSkyB so it is high profile but the same debate is acted out between thousands of IT providers and their customers every week. Because these names are less well know, and because these companies can’t afford to go to court we never heard about them and some agreement is reached.

This happens internally too. Companies with their own IT groups often get into this mess, the business can’t tell the IT people what they want. Perhaps it is because I’m in London but most of the stories I hear involve banks. Sometimes being part of the same company can help, and sometimes it makes things worse.

Rather than ask whose fault is it? we need to ask How can we fix it?

Well there are two parties here and both need to be involved.

The business side needs to set the overall goals – ‘build a state of the art customer service system to produce competitive advantage’. And the IT side – whether in house or outsourced – needs to implement what was asked for an no more. But in between there is a massive, massive, gap.

• What does a state of the art customer service system look like?
• What is competitive advantage in customer service?
• How do you avoid spending so much on ‘state of the art’ that you loose competitive advantage?

These aren’t easy questions to answer. Neither side can answer them alone and neither side can expect the other to answer them. You need co-operation, you need an on going dialogue.

Business needs to be able to articulate what it wants but it is wrong to think it can articulate everything up front. It can set goals but in achieving those goals there are thousands, indeed millions, of options and decisions to be made. The devil really is in the detail. Business and IT need to make choices together.

Making those decisions requires people in the role (business analysts or product managers), it requires IT people who understand business goals, it requires business people who understand how IT is created and how to recognise the benefits, and it requires a process to keep talking and making decisions.

In this case it is wrong to think BSkyB can just throw the problem ‘over the wall’ to EDS. And it is wrong of EDS to expect fully defined requirements documents up front. But, the nature of competitive business encourages people to take this approach.

I don’t know how EDS and BSkyB structured this project but here some advice:

• Start small: IT projects are often far too big, if you can find the real benefit it then stick to delivering just that, keep it small. Inside every large project is a small one struggling to get out.
• Requirements have to be driven by the customer, who needs to set business objectives and vision. However further elaboration needs to come from close collaboration between the business and the supplier.
• If you intend to build a ‘state of the art’ system, and gain competitive advantage then know your enemy, know what state of the art means and know what other systems do.

Finally, make sure you are building your ‘state of the art’ system for the right reason. Will a state of the art system really deliver competitive advantage? Maybe you don’t need ‘state of the art’, maybe you just need to use an ‘average’ system better. Maybe you don’t need to build a system at all, maybe you can just take one off the shelf. This won’t be cheap – especially if you have to customise it – but it will be a lot cheaper than starting from scratch.

As a PC user there was only one game in town for office software – Microsoft Office. Yes there was other software but very few people used it. Everyone used Word, Excel, etc. and if you used anything else people thought you were strange.Since moving to the Mac I’ve discovered the hegemony doesn’t exist here. I’ve discovered people who use NeoOffice and as Mark points out in his comment today there is Apple software too.

Funny thing is I first met Mark when we both worked for a company which produced UNIX based office automation software, Quadratron. In fact the company had made a good job of missing the PC market and realised it needed to catch up. Somehow Mark, Adrian, myself and a couple of others weren’t going to make it happen. Once it became clear the company had no future we jumped ship.

But now it appears the office automation market might be on the move again.

Yesterday I used the Google spreadsheet and word processor for the first time. In fact I was using them in collaboration mode with Till Schuemmer as we started the process or organising EuroPLoP 2008. They worked well – despite my very slow ADSL connection – and I was impressed. However shortly after we finished the ADSL connection collapsed again so while Google Apps might be the future they are not the present.

Also this week I learned that IBM is launching, or maybe re-launching, a office automation suite called Symphony. Lotus Symphony was an integrated office package in the days of 8Mhz 80286 CPUs with 640k RAM. It worked but not well enough too make much of a mark.

Why has IBM done this?
Will Google Apps (or ADSL) ever be stable enough to challenge desktop software?
Will NeoOffice or Apple displace Microsoft on the Mac?

I don’t know the answers to these questions but I do see the start of a change.

Back in April and May I discussed failure modes in Agile software development. Well I’ve found another. That is to say I’ve found another way in which an Agile development practise can go wrong. I’ve seen this before but I didn’t recognise it as a failure mode, I saw it as a little local difficulty now I’ve seen it a second time I understand it a bit better. The trouble with this failure is that at first it looks a lot like what should be happening.

One of the core ideas behind Lean and Agile is improving quality of code. Improving code quality enabled several other benefits. Improved quality means fewer bugs, so future development is less disrupted because there is less rework. Improve quality means less time spent in test at the end of a development. Thus short iterative cycles become possible. And improved quality means you can deliver (or just demo) any time. So quality improvement is important.

One way in which we seek to improve quality is through automated unit testing. This is usually undertaken as Test Driven Development or Test First Development. Now there is a whole host of reasons why it can be difficult but that is another blog. With automated unit tests fine grained functionality can be tested very rapidly which provides for rapid regression testing and increased confidence in new code.

This is where the failure comes in.

We actually want the tests to fail if we change something because they prevents errors slipping into the system. So we add a test for new code, write the new code and by the time we check the code it is finished and the test pass. But, if later something changes which will break the code then we want the test to fail in order to catch the error that has been introduced. In this case we want the test to be fragile so that it catches anything which is now wrong.

The problem occurs when the test is fragile in the wrong way. In the case I’ve just observed the test was fragile when data in the database changed. Previously when I saw this the tests were fragile around COM. The test was/is not sufficiently isolated from its environment.

True dyed-in-the-wool Agile/TDD practitioners will immediately jump to say: you shouldn’t do it that way. And they are right. These fragility points should be addressed. This might be by stubbing code out, using mock objects or by tearing-down and restoring some element – such as a database.

As a consequence instead of the tests helping you validate code and move faster they become a hindrance. The need to update and change the tests regularly adds to the work load and complexity. Instead of the test showing everything is good they add to the maintenance burden, the exact opposite of what was desired.

The problem is that those trying to take the Agile/TDD approach have good intentions by they don’t have the experience, but how do you get the experience without doing it?

Well there are three answers here: first the organization should have invested in training for these people. Second the organization could have hired someone with experience to help them, perhaps as a part-time coach or perhaps as a full time co-worker. And third, the management could have helped the teams learn faster and better and encouraged them to reflect more on what they were doing.

Trouble is Agile development is often introduced bottom up, so people do just jump in. And even where management introduce it then such training and coaching can be expensive – that is if you know who to call and where to buy it in the first place. (If you don’t know who to call then let me know, I know people who do this stuff.)

So although it is well intentioned and the ‘just jump in’ / ‘just do it’ approach can get you started but it can also start building up bigger problems further down the line. If you know what to look for you can spot the problems and – at least try to – take corrective action. If you don’t then it is quite likely you’ll give the whole Agile experiment a bad name,