allan kelly

Lean Product Development references

After yesterdays blog posting I had a comment from ‘Brave Heart’ asking for some references on Lean Product Development. I’m more than happy to help and normally I’d reply direct to Brave Heart. But… I can’t find your e-mail address on your posting or blog. So that means everyone is going to see my recommendations…

The only book I’ve read which is dedicated to Lean Product Development is Michael Kennedy’s Product Development for the Lean Enterprise. There is some additional information in Machine that Changes the World (little bit dated now), Lean Solutions and Thinking Beyond Lean (also getting a bit old) but not a lot. I believe there are a couple of other books dedicated to the Toyota Product Development System but I haven’t read them.

I think you would benefit from looking at some of the literature from the software community. In particular Mary and Tom Poppendieck’s books Lean Software Development and Implementing Lean Software Development.

From there you have to decide if you want to look at Agile software development. I think of Agile as an application of Lean, which I talk about in my own book, Changing Software Development. If you do then SCRUM is going to be of more interest than XP or some of the others.

This is because SCRUM has its roots in the knowledge management – specifically the theories of Nonaka – and these link up with the knowledge thinking behind Lean Product Development. Which brings us full circle back to Kennedy. As Kennedy points out, Lean Product Development might be better described as Knowledge Based Product Development.

As for research, you might want to go and watch some Agile or Lean teams in action. Its probably easier to find teams which consider themselves Agile than Lean. You’ll want to look at how their activities differ from ‘traditional’ product development.

I hope that helps.

Brave Heart, please drop me e-mail (with your address) if I can be of any more help.

Lean Product Development references Read More »

News, Conferences and a Project Management revelation

This blog has been rather quiet for the last couple of weeks, and most likely it will be quiet for the next couple of weeks. This hasn’t been for a lack of interesting things to say, quiet the opposite. In the last few weeks I’ve had insights coming left, right and centre. However, I haven’t had much time for blogging.

As I said at the start of the year I want to divert more of my writing into other projects. So this entry is here as a place holder and a preview of some stuff to expect. So please forgive me for breaking my one entry, one idea rule.

Those of you who know me off-blog will know I spent most of last year working with an Internet TV (IPTV) technology company. Some consultancy actually continued into the year, mainly with the CEO. Well now I’m please so say I’m going to get involved in IPTV again. I’m going to be doing some work with one of the major players to developer a completely new IPTV platform. Should be fun.

After XP Day last year I suggested that media companies are more open to Agile methods many other companies, specifically, banks. I am more and more convinced of this fact. Underpinning this is that fact that companies that use the traditional (broken) methods are still in business, these methods have brought them some success and fit with their business model – or their business model has been changed. Therefore these companies have little reason to go through a difficult change.

At ACCU London last month someone asked me: Why do banks have so many bad systems? It was that moment I realised: Because they can afford it. I hope to expand on this topic soon.

I’ve been busy with EuroPLoP 2008 and I’m at SPA 2008 next week, both look like being a good conferences. Then I’m at ACCU 2008 a couple of weeks later. Both of these will produce some reflections in this blog.

I’m going to be writing more about Project Management in the near future. Readers of this blog, and those who know me, will know I have been critical of them in the past. And I promise to carry on being so. But…

During the last two week I have taken Project Manager training, not just any training, PRINCE 2 certified training. Yes you read that right. Barring any surprised form the exam I took today I expect to be PRINCE 2 certified in the next few weeks.

So, why did I do it? Did I have a labotomy? Well it wasn’t an easy decision to spend nearly £1500 on the course but here’s why:
• I’ve criticised Project Managers in the past but this has been unfair, I haven’t really known what they should be doing, only what they do
• Increasingly I find that I’m asked to do Project Management roles, so the training should help there.
• More organizations look to PRINCE 2 to show you know about Project Management. To impress them you need to tick that box.
• I’d really like to understand how formal Project Management compares to Agile, and whether it is possible to bridge the divide.

I’ll write more about this soon but some immediate thoughts.

I had to do two courses for two exams, one last week and one this week. Consequently I’ve met over a dozen other people taking the same course and exams. Almost everyone of these people was an independent (contract) Project Manager in the IT sector. They were sending their own money to go on the course, and foregoing even more money by being there. Why?

Because increasingly PRINCE 2 is becoming a necessary qualification, if you don’t hold it your CV won’t get beyond the recruiter or HR department. PRINCE 2 fans are probably delighted to hear that. PRINCE 2 is wining! But actually I think its a very dangerous development.

This is because PRINCE 2 is massively risk averse. Of course PRINCE 2 practitioners would probably say “No its not, it just identifies and managers risk, you decide the risk level, isn’t that sensible?” but actually the very mechanism is risk averse.

Why is this dangerous? Because most companies – unlike the Government who developed PRINCE 2 – want to make a profit. Economists tell us “Profit is the return for Risk”. So, remove the risk and you remove the profit… more on this shortly.

Interestingly nobody on the course expected to actually use PRINCE 2. Even those sent by their companies (about 2) didn’t. So I don’t think PRINCE 2 is increasing in use, its just the qualification that is being asked for more and more.

Second insight. PRINCE 2 is not reflective, it considers the cost of projects, the cost of work but not the cost of itself. To run a full blown (or even half blown) PRINCE 2 is going to cost you a lot in administration and paperwork. That’s before you count the missed opportunities.

A few years ago I found myself introducing Agile at a company which hired a PRINCE 2 manager. We didn’t get on. We were both trying to do good for the company, and in many ways our prescriptions matched but … well I’ll explain where PRINCE 2 and Agile match soon, and I’ll explain why they are incompatible too.

Somehow I got cast as the zealot. Some people came to believe that I wanted Agile, Agile, Agile. He never got cast as a zealot, that only dawned on me this week. Actually advocating PRINCE 2 and its practices the way he did was every bit, more so, as zealous as I was. But…

PRINCE 2 is so respectable nobody will call you a zealot for advocating its use. Its like suggesting you play Bach or Mozart. How can anyone object? Its respectable. Agile on the other hand is punk, its putting a safety clip in your nose and taking risk. No wonder I was seen as the bad guy! Now, once my certification comes through….

News, Conferences and a Project Management revelation Read More »

High Church, Second System effect and re-writes

After ACCU last last Thursday I was in the pub with Bronek Kozicki – and many others. The subject of system re-writes came up. This is a perennial subject, for example have a look at Alan Griffith’s Overload 79 editorial or Tim Penhey’s CVu editorial from the same month.

Recently I’ve been consulting with a company which has a truely awful code base. The developers want to re-write it, naturally. Unfortunately the the code base isn’t the only problem, the company has other problems and matters aren’t helped by Kissinger-esque linkage with other issues. I’m not going to say anything else about them except to say Conway’s Law was clearer than ever.

And its not just software people who stuggle with the question: Is it better to re-wright, or not to re-write? This piece from last week’s FT shows English lawyers and tax authories have the re-write problem.

Bronek sent me a link to Portrait of a Noob by Steve Yegge – not a blogger I read but I’ve been sent several of his posts by people in the past so I know of him. In many ways I find this reminiscent of my own High Church C++ pieces from a few years back. Both pieces point out that quality code is subjective, what you think of as good code in one context is not the same as what you consider in another context. Steve is right to point out that your persepctive on code changes. In particular, as you get older and acquire more experience not only does you coding style change but the way you approach problems changes.

As someone pointed out on Thursday night, you also need to remember Fred Brook’s and the Second System Effect:

‘This second is the most dangerous system a man ever designs. When he does his third and later ones, his prior experiences will confirm each other as to the general characteristics of such systems, and their differences will identify those parts of his experience that are particular and not generalizable.

The general tendency is to over-design the second system, using all the ideas and frills that were cautiously sidetracked on the first one.’

I’ve long wanted to conduct a little experiment. Take a developer, take a system he developed in the first few years of his life. Wait long enough for him to forget the details, say 10 years. During that time let him learn, gather experience and continue developing systems. Now ask him to make changes to the system from his early years – with all the identify removed of course. My theory is, he will proclaim it unmaintainable and in need of a rewrite.

That is a little unfair actually because one of the things you learn as you get more experience – OK, get older, lets not beat around the bush – is that all systems are maintainable. The other thing you learn is that rewrites usually end in failure.

Our education system has it wrong. At school and college we learn to new write new code. But most developers spend most of their time maintaining old code. Our education trains us to believe a lie, that we write new systems. For the first few years of a graduate engineer’s career he is disappointed, he searches for ‘green field’ development and faced with an old system he throws his arms up – ‘Re-write! We must re-write!’

Not only are young developers sadly disappointed by the state of the world they are a risk. Sometimes people believe them and let the re-write the system. (Believe me, I was once that developer. I persuaded my employer to let me re-write a system. It was a mistake. )

Colleges need to recognise this and start setting assignments were people must maintain code written by others. Systems should be handed from one year to the next for changes.

As a developer gets more experience they learn how to work with existing systems, when to re-write parts and when to work around. The hallmark of an experienced developer is that they do not jump to say ‘Re-write!’ – they can at least present options.

In fact, the hallmark of the best developers is that do work with legacy systems. Anyone can write a new system, we get trained for it in college, its what the books describe, there are few limitations. You can create your own mess, and you usually know your way around your own mess.

But the best developers can work with someone else’s mess and still find gold.

High Church, Second System effect and re-writes Read More »

The Location Myth – why location it matters

The internet, we were told, would make location irrelevant. But it hasn’t. Some examples.

• My brother lives in the UAE so we talk by phone. We would use Skype but the local telephone company blocks Skype and various other services.

• I have several kitchen containers from a New Zealand company. I bought them in the UK and would happily buy more but they are no longer stocked in any shop I know – and believe me I’ve tried. I could order them online from the US but the American shops won’t ship outside of the USA and Canada.

• When I’m searching the web, particularly if I’m looking to buy something, and I find the kind of site I’m looking for I want to know where are they based? London? Cardiff? Chicago? Mumbai? I have nothing against India companies, or American companies (provided they will ship abroad) but when I’m faced with a site I’ve never heard of I want some kind of reassurance. Their location is part of that. OK, there are fraudsters in London as there are in Chicago, I’m unlikely to chase them myself, and they could be lying about where they are, but it all adds to my understanding.

In short: the internet makes many things possible over long distanced but location still makes a difference.

The same is true in software development teams.

You can have people located in different offices, different parts of the world, and you can use telephones, wiki’s, Skype, video-conference kit, etc. etc. to work but it isn’t the same as having everyone in the same place.

When you locate people elsewhere you incur extra costs. Obviously the cost of telephone calls or bandwidth but these are a small part. As you use more technology (conference calls, video conferencing, shared whiteboards, etc. etc.) you incur more and more direct costs. Even wiki’s take time to installl, and time to use.

You also incur costs because conversation often need to be arranged. So I might IM someone and say ‘Can you talk?’ and 20 minutes later they get back and say ‘In 5’. The need synchronise before talking slows down the process and there is a cost of asking.

Then there is the cost of missed conversations. We make small talk, trivia, at the water cooler, or walking to lunch, or because someone is there and we want to moan. When someone is remote these conversations are lost. We incur cost because we don’t get the benefits of these costs.

Then there is the cost of setting up meetings to compensate. A few years ago Lise Hvatum and myself ran a focus group at EuroPLoP entitled ‘What do we think of Conways Law now’. One of the recommendation we came up with was: when you remove an informal communication channel you need to replace it with a formal one.

So when we have lots of remote workers you need more formal, arranged in advance, meetings. This ads to the costs again.

And when you do talk the level of information transfer is less. Even with video conferencing your primary communication mode is voice. Communication is centred on explicit words. Face to face much communication is implicit, its in the tone of the voice, the facial expressions, the body language and how other react. You loose all this when you are remote.

Lets face it, software developers don’t have a great reputation for communicating well anyway so why put obstacles in the way?

Every time we put an obstacle in the way we reduce information transfer. This goes for cubical walls and offices too, open plan offices are my preference. Every barrier, cubical partition, different floors, different buildings, different continents, reduces information flow. The bigger the barrier the bigger the loss.

And for a development team – focus on that word: team. Who ever heard of a football team which played on different pitches?

Team work is about just that, team work. You can’t do it when you are alone. Every barrier makes team work more difficult.

It is an illusion to think that you can develop software when your team is spread around a single building let alone the world.

This is especially true for people who are in leadership positions in an organization. You can’t have a CEO, a team leader or an architect who is removed from those he leads.

Sure there are exceptions. A few companies are completely distributed, Sleepycat software – before Oracle bought them – springs to mind, as do Open Source projects. On the whole these work because they are born on the internet.

The other reason remote working teams work is because they aren’t so much a team but a bunch of individuals, each working on their part of the system and seldom interacting with each other. I think of these as Swordfish teams – a bunch of developers flying in formation. As a result (Conway’s Law again) you are likely to end up with a very diverse system architecture.

Don’t get me wrong, I’m a big fan of working form home once in a while. The odd day spent working at home can be really productive – precisely because I have few conversations and can focus on one or two things. Working from home is great for reflecting and preparing oneself.

And it is acceptable for large teams to have the odd person working remotely. But the smaller the team is greater the need for them to be in one place.

Maybe once upon a time developers could work alone, remotely. They could put on headphones and switch off to the rest of the world. But today developing software is a team sport and needs to be played like one.

I say that as one who has never liked team sports such as football, rugby, cricket, etc. I’ve always preferred solo sports like skiing, running and swimming. I have to adopt and change too.

The Location Myth – why location it matters Read More »

Developers are not the only fruit

 

My younger self would be horrified to hear me say this but, when you develop software you need people who are not software engineers.

Don’t get me wrong, coders/engineers/programmers are the most important people because these are the people who actually produce the thing. As such they are central to any development effort, you can’t write software with a collection of managers, analysts and testers any more than you can build a ship without shipwrights.

Coding is where the metal is bent, shaped and welded. When coding there is no room for ambiguity or fudge, the code doesn’t lie and it doesn’t take to ambiguity. Thus there is a lot of details in coding which aren’t obvious before you start. So your developers need to be able to call on Product Managers for more detail and, if you have any architects, you need them at the code face coding with your developers.

See Jim Coplien and Neil Harrison’s patterns Work Flows Inward and Architect Also Implements for more discussion of this topic.

One of the worst projects I ever worked on was completely top heavy with managers (programme, project, customer, delivery, team – you name it we had a manager for it), testers, analysis, admin staff – yes you needed a lot of admin staff when the project was over 120 people. But only about 12 actual coders. Crazy. The overhead was there because… well, the ISO-9000 procedures needed a lot of managers, and it gave the consultancy implementing the project a lot of bodies to bill for, and it gave the Government (the people who were ultimately paying for it all) a lot of reassurance that it would be done one time, to budget and spec. Except, it was late, the spec trimmed left, right and centre, and if there ever was a budget we blew it.

So you see I’m no fan of carry extra weight on a project.

But, you need people who don’t code.

I’ve written before about the role of the Product Manager and why product management is important, and why you need them. So lets say we accept that Product Manager are needed.

In an idea world we wouldn’t need software testers either. And I expect one day to work somewhere where there are no testers. I don’t believe this is an impossible dream. But with the current state of play we need software testers in most organizations.

One role I don’t think we need is Software Architect. On the whole I think most architects are mislabelled. They should be Product Managers, Business Analysts, Senior Software Designers/Engineers or even Project Managers. Sometimes this is because the organization misuses the title, sometimes its because they don’t understand it (after all, what is software architecture?) but most of the time its because of title inflation. You couldn’t give someone a pay rise so you gave them a nice title.

Although I don’t read Joel Spolsky very often I do agree with him about Astronaut Architects. The moment your architects stops coding its time to fire them. Once they stop coding they are a burden not an asset.

Project Managers are another group I don’t think we need. I accept teams need some leadership but Project Managers are completely the wrong people to. Project Managers are trained to analyse, dissect, use whips (badly) and emit hot air. Their are not leaders by nature or training.

The Agile world is re-inventing project management as coaching but unfortunately the Agile world is pretty messed up about coaching. Coaches are useful when you they get it right and they are genuinely improving the team rather than acting as Project Managers with a different title.

But, and this is where my younger self will have a fit: you do need someone to oversee and manage the whole development process. The big problem is very very few people can do this job right.

By way of explanation to my younger self: done badly this role is worse than useless, done right it can really improve things. Unfortunately most people do it badly.

Its done badly for several reasons.

Firstly, most Software Managers have never been trained in the role – or management at all. They ‘make it up as they go along.’

Second, some of these managers don’t understand software development, they come from outside the domain. Such people try and make software development fit their existing model of management but it never will. Therefore they are constantly trying to make a square fit in a round hole.

Third, many of the managers who come from inside the domain, and know how to develop software can’t give up coding software. To be a good software manager you have to stop being a coder. You have to learn to look at the world differently, you have to realise you can’t intervene to fix problems, you can only arrange things so other people can fix them.

When I was younger I once worked for a bank. The code I was working on was terrible and I had no support from the organization. I really really wanted my manager, and his manager, to come and code with me. I thought their big mistake was to stop coding. I was wrong.

Their biggest mistake was not to stop coding but to stop communicating with those who did code. The big mistake was to put too much space between themselves and the building process. Had they involved themselves more they would have better understood the problems and would have been able to arrange things so I could fix them.

If you are managing a software team you should be talking to them every day. You should be improving their working conditions every day. You should be allowing work to flow to them.

When you move from coding to management you have to give up part of yourself, you have to trust the people you manage. It is their code now, not yours.

So, in conclusion: it is wrong to think you only need developers to create software. It is also wrong to think you can code and do other things – you need the separation. But it is also wrong to have too big a separation.

Make any of those mistakes and you just moved one more duck out of line.

 

Developers are not the only fruit Read More »

The lining up your ducks theory of software development

Software development is easy. Don’t listen to what people say. I was 12 when I started programming and I picked it up no problem. Businesses are full of people who programme without even knowing it: Excel is programming by another name, and how many people write macros?

Yes, programming is really easy. You can buy ‘Programming for Dummies in …’ – well just about any language you want. People can, and do, create small world changing programs without ever really being taught how to program.

That’s why software development and new businesses go hand in hand. Its one of the reasons why so much innovation today appears as software.

But….

Software development is also one of the hardest things. In fact, developing good software is so complex it might be the most complex thing man has ever attempted.

Moon landing? – it used software, compared to the complexity of the systems in the international space station I’m sure Apollo looks simple.

Nuclear power? – compared to the maths used by the finance industry today it looks simple. And that is all in software.

The thing is, writing a small piece of software can be (but not always) very easy. Problems set in when you want to grow the software, want more people to use it, want to sell it, want to package it, want to re-use the code or ideas, take bugs out of the system, add features, make it run 24×7.

It is very very easy to write simple software and have it do something useful. It is an order of magnitude harder to get it to commercial quality and keep doing it. And, to make it harder, there is no single accepted method for doing this and getting it right every time.

It sometimes amazes me that we get any of this right. More amazing is that software which is fundamentally flawed works, or at least seems to work. Stuff which by any engineering criteria is broken is used by companies every day, they even base their business on it. (There is another blog entry in this.)

Developing good software, delivering on time, producing with sufficient quality, etc. etc. – all the things you expect from ‘professional systems’ is a matter of getting your Ducks in a Row. What I mean is…

Creating good software, meeting expectations, delivering on time and the rest is not just a matter of getting one thing right. It is a matter of getting many things “right” – or at least workable. You can get some of these wrong and still delivery something – maybe a little late, or lacking features but you will do something.

Each time you do something badly, each time one of your ducks is out of position, you won’t break the whole thing. This isn’t a chain, instead of breaking you will reduce your productivity, or move away from the target, or reduce quality. With each duck that moves out of position it gets worse but it still works, somehow.

Put it another way. If creating ‘the best software ever’ means scoring 100%. Each time you move a duck out of position you remove 10%. So:
• don’t co-locate your developers, loose 10%
• start work without talking about design, loose another 10%
• work to a fixed specification or work to no specification loose 10% as well.

Down to 70%.

• Fire developers half way through the project, loose 10% for lost productivity, loose 10% for moral.
• Projects looking late, add more developers, loose 10% for violating Brooks Law.

40% doesn’t look good does it?

But there aren’t many case where you need to score 100%. 90% will do, even 80%, 70% often, some customers will be happy with 40% – or less.

The people who get upset most by this?

Not customers, they get annoyed but quite often there is nothing they can do about it.

It is the developers themselves. They see each loss, they know where all those 10% are being lost. And these guys are problem solvers, they want to fix these problems.

Now go back to my falling off a log theory of business.

I said: you do something right. Not 100%, maybe you just need 10%. But that one thing is about doing something useful, something other people want. Its about benefit and effectiveness.

Viability of your software product/company =
benefit of your software
x
effectiveness of your organization

If your software doesn’t bring many benefits then you need an effective organization to stay in business. You have to be better than the rest.

If your software brings lots of benefits then it doesn’t matter how ineffective your organization – shoot the ducks! – you will stay in business.

If your software has few benefits and you can’t deliver, then Game Over.

Which leaves… lots of benefits and a highly effective organization. Then you have a world beater.

Unfortunately few companies fall into the last category which leave most people working in one of the first three.

The lining up your ducks theory of software development Read More »

'In search of mediocracy' – The falling off a log theory of business

When I was at business school my classmates and I read lots of good stuff about how to run a good company. Books like ‘In Search of Excellence’ and ‘Built to Last’. But I remember some of my classmates asking: ‘Why aren’t the companies I’ve worked for like these?’ To most, if not all, my classmates these books described something that was not the case.

That was five years ago. Since then I have met many people, few if any of them have worked for ‘Excellent’ companies.

Indeed most of the people I meet (particularly in software development) tell me crazy things that have happened at their companies. Decisions to close down offices just before a contract is won. Objective setting sessions were the objectives set are meaningless. Management which does the opposite of what they know is best.

One company I worked with in the last five years came closer to ‘excellence’ than any other company I know. Still, it got into trouble and has had three rounds of downsizing in the last two years. From what I hear things are pretty crazy now. Apparently in the last round they paid off four engineers, soon discovered they needed them, tried to hire them back, HR got involved and slowed the whole thing down during which time three of them got new jobs.

Another company I worked with was perpetually shooting themselves in the foot. So much so that they even let go the one person go who could make sense of the development team. Just this week I learned the same company which is letting one of their key managers go.

I end up hoping the end comes soon but these companies survive. The books say they should go out of business but they don’t.

So it is that I find myself asking ‘where are the excellent companies?’ And I feel pain when someone else tells me another story of the stupid things their employer is doing.

I’ve come up with a theory which explains it. I apply it only to the software industry because this is what I know but I suspect it might be more universal. It goes like this…

Like having a baby there is no law against starting a company. Actually there are a few laws, and in some countries it can be hard, but in most Western countries, provided you do not have a criminal record you have a right to start a company. In the UK it can take less than a day to form a company.

Most of these companies will go out of business in under a year. If I remember the statistics less than 10% survive year 1.

This 10% have done something right. Maybe they hired the right people, maybe there were in the right place at the right time. Maybe their competition was more of a mess than they were. Perhaps they really did have ‘first mover advantage.’ In other words, many of these companies survive purely on luck. They got something right. That doesn’t guarantee they got anything else right.

Some of these companies will go out of business in the years that follow but the longer a company remains in business the greater its chances of surviving even longer.

Over time these companies will gain some customers. Now customers can be surprisingly resilient, not because they are ‘loyal’ but because switching costs are great and because they have to decide to go elsewhere. Remember, these companies may be dysfunctional themselves so changing isn’t easy.

When you have a few customers, particularly big ones, you can make the money go a long way. Software companies which pull in their wings and go into maintenance mode can shed most of their staff and still stay in business for years. In fact, I would argue that almost any company you think is about to fail can last another year.

Notice there is nothing here which says the company will be ‘excellent’, or even that it will do what the management text books say. All I’m saying is that crazy mixed up companies can survive for a long time when they shouldn’t. Many of these companies have little right to exist by business school standards but they do.

As a consequence many, if not most, companies are, to some degree dysfunctional. Many people work for these companies. Many people are abused by these companies. And these companies are enough to put you off companies altogether.

I call this my ‘Falling off a Log Theory of Companies’: Starting a company is as easy as falling off a log. Most companies which come into being will fail quickly, the majority of those fail will be to some degree dysfunctional but will survive. Consequently most of the people I know will work for dysfunctional companies.

I think this theory applies especially to software companies for two reasons. Firstly there are still many many opportunities in software to be found and exploited, i.e. you can still get lucky. Secondly, the IT industry, and those who fund it, are massively optimistic. We accept failure too easily.

A few months ago I explained this theory to someone not in software. He said he’d seen the theory in action too. He also suggested the book these companies need is not ‘In Search of Excellence’ but ‘In Search of Mediocracy.’

'In search of mediocracy' – The falling off a log theory of business Read More »

Christmas reading: Classic essays on software development

 

There are many, many books about software development. The technical ones alone (e.g. Java, C++, Apache, .NET, etc. etc.) take up meters and meters of shelf space. The ones about project management take up a bit less space but they are still plentiful, and the ones about people in the process take up even less space but are longer lasting.

It is the later category that interests me most theses days. Some books seem perennially relevant and may never go out of print, books like:

• Mythical Man Month by Fred Brooks
• Psychology of Computer Programming by Gerry Weinberg
• Peopleware by Tom DeMarco and Timothy Lister

But often you don’t need a full book to make a point. On these occasions an essay or journal article will do. Often they make the point better simply because they are shorter. Unfortunately these tend to be overlooked too often – books get all the publicity. I would like to highlight half a dozen essays which I think are perennially relevant and often overlooked. In some cases this is because the articles can be hard to get hold of or, more likely, people don’t know they exist.

1. How do committee invent? by Melvin Conway – the origin Conway’s Law. Although written in 1968 this piece is truer than ever. Arguments made by Conway explain much of what actually happens in software development. I’ve examined and looked at this subject myself elsewhere.

Conway suggests that organization that create systems (not specifically software systems) will create copies of their organizations. So, if you have one developer writing the entire system it will be all entangled, few interfaces and difficult for anyone else to work with. And if you have half a dozen developers who don’t communicate you will get six different coding styles, and six versions of most common functions.

What Conway didn’t foresee is that many of system now create the reverse force. Organizations are limited by the systems they use, thus their structures become copies of the systems in use.

2. Worse is Better by Richard ‘Dick’ Gabriel – this started as a keynote talk, became an essay and then a series of letters in which Dick argued anonymously with himself.

If you are one of these developers who think the best code will always win the day you need to read this. Dick suggests that sometimes the solution which is technically worse is economically superior. In a similar vein see Brian Foote’s Big Ball of Mud pattern.

3. No Silver Bullets from the author of Mythical Man Month, Fred Brooks. This piece came out in the mid 1980’s. The Wikipedia summary is here, otherwise you will need to by the Anniversary edition of Mythical Man Month to read it.

Brooks argues that software development is hard, and there is no ‘silver bullet’ which will make it easier. No language, no operating system, no methodology or any other tool will fix it. I have been told that Brooks has since said the if there is a Silver Bullet it is Agile Software Development but I can’t find this quoted anywhere so I don’t completely believe it.

4. A Rational Development Process and How to Fake It by Dave Parnas. Originally published by the IEEE this also appear in Software Fundamentals: The Collected Papers of David L. Parnas. You might also find some copies stashed on the Internet. Dave Parnas deserves to be more widely read by software engineers today so it you haven’t read much of his work buy the collected papers, well worth reading.

In this paper Parnas argues that while we may want to use a rational development process, and while it makes sense to do so it is impossible. This is because software development requires intuition and inspiration – or as I would put it learning. And since you can’t schedule these things or guarantee they will happen the chances of following a rational process are negligible.

However, in order to make our work accessible to those who come after us we need to fake the development process so it looks like we followed a rational process.

I like the logic, I agree with much of it but I have one disagreement. Each generation of software engineers comes along and believes the previous generation were rational and get confused by what they find. They slowly learn that they cannot be rational – causing a lot of angst. They also wonder how the previous generation made such a mess. So frustration is built up.

5. Cathedral and the Bazaar – by Eric Raymond: an explanation of Open Source software. And an explanation of why software created in corporations is frequently a mess. I don’t think Open Source is the answer to all our problems, but I think it is an answer to some and is here to stay.

6. Enrollment Management by Peter Conklin: until recently this was one you really have to hunt down, but it is worth it. Originally published in the Digital Technical Journal (Digital as in DEC or Digital Equipment) in 1992 it was republished by the IEEE in July 1996. HP now have back issues of the Digital Journal online so you can read it here.

This piece tells the story of the Alpha AXP programme in the late 1980’s and early 1999’s. In theory, on paper, on GANTT charts the project couldn’t be done. The author tells how enrollment management was used to engage those working on the project. Although written in 1992 this paper is about Agile development. And it is about Agile development in the large – over 2,000 engineers worked on the project.

I wrote about this article before, a few years go. One of the small stories in this paper always brings a smile to my face. The OpenVMS team committed to a delivery schedule and said ‘We don’t know how to achieve this, but we commit to finding a way.’

So there you have it. Six essays I recommend you read. And if you only have time for one? Well, chances are you know about Open Source and the Silver Bullet so I recommend you read Enrolment Management or A Rational Process, and if you push me more I’ll say: Enrolment Management.

 

Christmas reading: Classic essays on software development Read More »