allan kelly

I’ve been thinking a lot about Scrum in the last few weeks. Scrums done a fantastic marketing job for itself, so much so that Scrum is now the poster child for Agile.

However there are aspects of Scrum I find troubling. One of these is the self-organizing team. Its not that I have anything against a self-organizing teams, in fact I’m all for them. And although I don’t talk about them in Changing Software Development much of what is written there implicitly advocates teams taking on organization for themselves.

No, what troubles me is the system within which self-organizing teams are expected to exist.

My first worry is that such teams are culturally incompatible with many organizations. Many (perhaps most) organizations are set up as command-and-control structures. As much as I’d love to have a revolution and change the whole organization I do not expect this to happen overnight. Self-organizing teams will often need to exist within command-and-contol organizations.

Some organizations will be open the occasional self-organizing team and will tolerate them. The team itself needs to recognise the boundaries of its self-organization and while it might seek to spread the idea it shouldn’t expect the whole organization to understand, the team needs to learn to interface to a hierarchical structure.

And in other companies the organizational immune system will actively try to ejects the self-organization team as counter-cultural.

A while ago I heard about a French bank in London which tried Agile development – it may have been Scrum I don’t know. Things went well to start with then the Business Analysts in the organization rebelled. They didn’t like the way the team was working and told management so. The experiment was closed down.

My second worry is that we might be asking too much of self-organizing teams within a hierarchical environment. Specifically, teams can (and do) change what is within their control but they exist within a system and they have limited power to change the system.

As W Edward Deming would point out: we should not blame the worker, or look for the worker, to fix a problem which is actually part of the system. The system needs to create the environment in which workers can do their best work. When the system stops the best possible work then it is they system that is at fault.

This is the thinking behind Deming’s point 10:

“Eliminate slogans, exhortations, and targets for the work force asking for zero defects and new levels of productivity. Such exhortations only create adversarial relationships, as the bulk of the causes of low quality and low productivity belong to the system and thus lie beyond the power of the work force.”

So, if we place a self-organizing team (or even a semi-self-organizing team) in a system which creates problems we cannot expect the team to fix the problems. There are limits to what the team can do. It is management’s job to create the environment.

I have been a vocal advocate of people trying to fix these kind of problems in their organizations. Indeed I actively urge people to try and address these problems. Until you try and fix a problem you don’t know if you can fix it or not.

Those of us who help organizations improve the way they work – including the adoption of Agile methods – need to recognise when a team can change the way they work and when they system the team exists in needs to be changed. And when the system needs changing we need to decide how far we go in changing that system.

I make no apologies for blogging again about Product Management because it is important and because, on the whole. As I said in my previous entry, Agile methods have a very simplistic view of determining what needs doing.

In the short run the Alignment Trap tells us that it is better to focus on doing things right, but in the long run Lean thinking tells us we have to do the right thing – remember 80% of features aren’t used. So Product Management is a long run play.

That is one of two reasons why Agile methods tend to underplay requirements and “Product Ownership” – because you get a lot of benefits by ignoring them to start with. The other is that Agile methods largely originated with developers who generally tend to underplay the role of requirements. (The exception to the rule may be EVO which is quite keen to understand exactly what people want.)

The trouble with Product Management is that everyone has a view on what needs doing, thus, in the absence of a Product Manager (or BA if your that type of company) other people will fill the void –

• Developers sometimes try and fill the void but developers have empathy with the code and find it hard to untangle their feelings about the code from what the customer wants.
• Architects claim to fill the void because they have the “bigger view” but architects are – almost by definition – uber-developers so have the same problem as developers. But architects are also tasked with looking at technology therefore any requirements they come up with are likely to be technology not market based.
• Project Managers often try and fill the void but their training and inclination is very different. They have a tendency to view things through the prism of dates rather than business value.

In the UK confusion between Project and Product management is rampant. It is slowly getting better but many companies can’t tell the two apart. This is really sad but also really dangerous.

Project Management can always expand to fill more time: you can always redraw a PERT chart, or level it again, you can always add another risk to the risk log (“An asteroid may hit Earth and delay the project”) or you can always go and ask a developer “Are we there yet?”

Product Management on the other hand can always be shrunk to squeeze in. After all we all have views on what the product should do so stick a finger in the air and guess. (And the more senior the person doing the guessing the more likely their guess will be taken as truth.) It takes time to visit customers, talk about their needs, time to go to trade shows, investigate competitors, time to calculate ROI and so on. Far quicker to stick a finger in the air.

My personal rule is: one Product Manager to every three to seven developers. If your product is stable, changing little and in a quiet market then one Product Manager may be enough for seven developers. But if you product is developing, the technology is innovative and the market changing fast you need one Product Manager for every three developers.

Its a false economy to skimp on Product Managers, you may save money but you may well create the wrong thing. Far better to go a little slower and create the right thing.

Given how much work a Product Manager has to do its natural to look for ways to split the role up. The first split is to hive off outbound marketing to Product Marketing. In a small company or team a Product Manager may do both inbound and outbound marketing but once it grows you should split the two roles.

Another division is between Tactical and Strategic, and with this split the role fits well with the Agile/Scrum idea of Product Owner. The Tactical Product Manager takes on the Product Owner role.

I hadn’t thought much about this until a friend of mine brought some blogs from Eigen Partners to my attention. These guys explain it well themselves so I’ll leave you to read Agile/Scrum Software Development and Product Management part 1, part 2, part 3 and part 4.

Ryan Shriver – aka The Agile Engineer – has some interesting comments on the requirements in Scrum projects – and Agile more generally. I tend agree with Ryan’s comments for a couple of reasons. Firstly I don’t think Scrum (or other Agile methods) really understand them. Secondly there is a lot more to requirements than is commonly realised.

Because Agile largely comes from the developer view we tend to neglect the bit that comes before but actually that is even more important in the long run. In the short run it is not important, in the long run it is vital. Agile methods have too simplistic a view of requirements, its all “user stories”. As Ryan says, there is more to it than that.

Andrés Taylor has some nice things to say about my book on his blog this week. Most of his blog is expressing concern about distributed teams and the discussion that follows is quite interesting too – Keith Briathwaite has continued on his blog. In my view distributed teams are generally a sub-optimal way of arranging your development process. However you may choose to organise in this way for several reasons. The main ones are:

1. You have no choice, for example: you need an expert in, say, database transaction processing on embedded systems and there are three guys in the world none of whom want to move to you Halifax Nova Scotia office.
2. You need your development team close to your customers and your customers are in different places.
3. Somebody somewhere has done a calculation that says the lost productivity is acceptable because the savings elsewhere. For example: having two developers in London and two in Bangalore is so much cheaper than having four in London that the 10% productivity loss is acceptable.

A few years ago we ran a focus group at EuroPLoP on Conways Law (What do we think of Conways Law now?) One of the insights I got from the focus group was:

• When ever you remove an informal communication channel create a formal one.

So…. Whatever the reason the important thing is to compensate for the distance. If you are saving so much money by spreading people out spend some of the money on flying people around to meet one another. Spend some of it on teleconferencing and video conference kit. Open the firewall to IM and Skype traffic.

Consequently companies that decide to spread their developers around should not then complain about travel budgets. If you are saving with the left hand you need to spend with the right so budget for it.

Andrés Taylor comments remind me to confess to something. The title of the book is wrong.

The book, as published by Wiley is entitled: Changing Software Development: Learning to become Agile.

The title I thought agreed to was: Changing Software Development: Learning to be Agile. I’m not upset with Wiley or my editor, I only noticed this myself a few weeks ago (about a year too late). I’m sure if you look back through my blog entries you’ll see the mistake for yourself. I assume the small change was to improve the grammar (not my strong point.)

But, for me it changes the title quite significantly. Learning to Become Agile implies you learn then you are Agile. End of story. Learning to Be Agile – in my mind at least – can be read two ways. The first is as I just said, you learn then you are Agile.

The second is more subtle, and I’m not sure if I can express it. This is the “to be, or not to be” version. The implication goes both ways: true Agile implies you learn, learning implies you are Agile, think of it as “I learn therefore I am Agile” – in order to be Agile you must learn.

Maybe this second version is a bit of a stretch – does anyone else get it? – but for me its actually more important than the other reading and in many ways this is the true title of the work. Shame I never noticed the subtle change.

This might be a little off-topic for this blog but a) its a damn big story, b) says a lot about business and c) a lot of software developers are connected with banks. That’s my excuse so….

There are a couple of aspects of the bank rescue plans that trouble me.

First one: Now is the ideal time to open a bank. Does anyone have a few million to invest?

I say this in all seriousness because at the moment banks won’t lend to one another because they don’t know if the other bank is holding any “toxic” assets. The public is nervous about investing with banks because there have been several failures. Yet plenty of people still want to lend from banks.

Thus, my New Bank Ltd. will have no toxic assets because it has no assets to start with. We will ensure it takes on no toxic assets. Therefore, the public will save with us, other banks will lend to us and we can make loans (i.e. acquire assets) which other banks can only dream of just now.

Rather than prop up failing banks Governments should be encouraging new banks to set up.

Next: Banks which are too big to fail. So what do the we do? Merge them with other (bigger) banks, e.g. BNP buys part of Fortis, Lloyds TSB buys HBOS, etc. As a result we have one bank which is much bigger than the bank which just failed. So, if this bank gets into trouble…

See what I mean? If a bank is “too big to fail” and we put it with another big bank we have a potentially bigger problem.

Put these two ideas together and I think it makes sense to re-create the lost smaller banks. All the big banks are home to banking brands which are still remembered by the British public and I think the same is true elsewhere.

Royal Bank of Scotland could devolve Natwest – or even go further and recreate the National and the Westminster bank. Barclays could return Martins bank and the Woolwich, HBOS could spin off Halifax (as a mutal?) and Lloyds TSB could split back to Lloyds and TSB.

The parent bank would be left as a “Bad Banks” holding toxic debt and owned by their current shareholders. They would exist to manage what is left and salvage what they can. The reborn Natwest and TSB would be clean banks. And all these banks would be small enough to fail without bring down the whole system.

I am speaking at the BCS SPA London group next month. It only seems like 5-minutes ago I was at the BCS building talking to the BCS PROMS-G group but this is a different group, a different audience and a different topic.

Actually, its getting hard to find big new things to say about “Agile” so I’ve entitled my talk Reflections of An Agile Advocate. I intend to look at the state of the “Agile”, identify some trends and suggest were we are heading next.

If you want to come along it is free but you will need to register.