Agile

It happened in an unexpected way – I won’t go into the details but about 6-8 weeks ago I had an opportunity to move from the development side of things to the product side of things.

Has taken me a while to get my head around it all – I still am in fact. I spoke to a couple of the product managers here to get a better idea of what I should be doing but it wasn’t until I spoke to my long time friend Richard Hall that I really made progress on understanding what I was doing.

Richard’s been a Product Manager for many years, when I decided to do an MBA a couple of years he said:
“You’ll become a Product Manager, all MBAs become Product Manager!”
And I said:
“No I won’t! I’ll still be a developer.”

Well, I got my MBA and I stayed a developer. Now he’s been proved right, for a while I didn’t want to admit it to him – silly of me I know. I spoke to him at the weekend and yes he laughed when I reminded him what he said – note, I had to remind him, he’d forgotten.

I picked his brains a bit and I got some good information. In particular he pointed me at the Silicon Valley Product Group (http://www.svproduct.com.) They have some really good papers on their website, I recommend them to any new (or existing) PMs.

And this reminded me of something else. When I went to Silicon Valley I discovered all these Product Managers. Not marketing people, but people concerned with what should be in the product.

In the UK you never meet a Product Manager. You meet lots of “Business Analysts” but they aren’t the same. Over time I came to appreciate their role and I saw that the existence of this role, and good people to fill it, was one of the differentiating factors in the Valley.

Now I’m working for a company in London which is doing well, and guess what? Unlike most British companies they actually have Product Managers. Actually, Richard is back in the UK too and is a Product Manager for another successful British technology company. So maybe the idea is spreading.

I think I’m going to like being a Product Manager.

Last night I ran an organizational mapping exercise at the Extreme Tuesday Club. What, you may ask is that?

Well, I learnt the technique from observing Jim Coplien apply it. Jim and Neil Harrison have been running these workshops for some years, their aim was to better understand development organizations and produce the patterns in their book “Organizational Patterns for Agile Software Development.”

Typically they run the workshop with a software development team, at the end of the sessions they have a collection of index cards they can analyze and produce a map of your organization. This map can then guide you through their patterns telling you which patterns your organization is using, which is isn’t using and which it should think about using.

This is a very powerful technique. It will tell you a lot you didn’t know, much that is obvious once it is pointed out and confirm a lot of suspicious various people have. As a tool for change its a great motivator: these guys come in, spend a day with you and can describe you back to yourself.

If you want to improve your software development efforts I strongly recommend getting them to run one of these studies.

Anyway, that is something else.

The sessions I ran with help from Giovanni Asproni and Rachel Davies was not really what Jim and Neil do, it was more inspired by Jim’s workshop. And what we didn’t do was go anywhere near the patterns. In effect, it was a demo of how you might go about mapping the interactions in your team.

So, what did I learn form this?

First thing I learned was that running this in a pub, with people drinking and eating, joining late and leaving early is far from ideal!

Second thing is that the people in the room hadn’t actually worked in a team together. So, we where trying to map an organization that didn’t actually exist. Naturally, each individual had different expectations of what they expected other people to do. The guy playing the “Architect” role had a different understanding of what an Architect would do to the person playing the Customer role.

To make matters more complicated, this was XTC, by definition, the people in the room believed in Agile development, Extreme Programming, Lean Software and all that jazz. Nothing wrong with that in itself but it made things even more complicated. There are few organization which are actually out and out XP, or Lean, or Agile. Everyone has a different take. For example, the person playing the “Project Manager” understood this as an enabler role, more of a coach, while others where expecting him to manage a project.

The other problem we faced was that time was limited – OK, time is always limited but the pub was going to close at 11pm. Having seen the technique used a few times before I knew where time would be wasted so I tried to hurry it along at stages. I probably over did this a little bit but it did allow us to finish in less than 2 hours so leaving time for a discussion.

The discussion that followed was very interesting. We discussed the technique, our companies, our experiences of agile, and how the technique had shown some things that we saw in real life (e.g. a frustrated developer got fed up of project management and testing and decided to talk to the user directly.)

Actually, discussions at XTC are always interesting but I don’t always feel I come away knowing something new. This was different. I got some insights from the exercise – I might write them up in more detail in the next few days or weeks.

The other thing I should say is that although this idea has come from software development its not confined to software development. It could be used to map any organization or process. (Interestingly, a lot of Jim and Neil’s patterns also apply outside of software development.)

Actually, in retrospect, since my objective was just to demonstrate the idea of the technique it may have been better to choose something from outside the software domain, say, getting a car services.

In the middle of the workshop I did have that “O no! Its all gone wrong, I can’t do this” feeling but in the end most people thought it was a great success.

(Who was it who said “In the middle of any change it feels like failure?” – I can’t remember, I think it was a Harvard Business School professor?)

I think I agree with them, it was a success for me because I got to try moderating this technique, it was a success for the participants because they go to see what an organization mapping technique could look like and it was a success for everyone because we did get some insights into the software process.

They may not tell their managers, supervisors or leaders but they will talk about it with others in the pub, the coffee shop, or at the water cooler, or some other place where it is safe to moan.

It is the work of a good leader to give people a safe place to express these problems in an environment where they can be dealt with. It is the leaders responsibility to listen, understand and see what can be done.

It doesn’t mean it is the leaders responsibility to do something about it. They may decide they need to act, but more often it is their responsibility to ensure the people facing these problems can act to solve them.

All too often people think “nothing can be done” so they moan about it to one another in the pub and absolve themselves from fixing it. Sometimes they don’t feel they can fix it, sometimes they don’t think they are allowed to fix it. In an enabled organization you want people to fix the problems as and when they find them.

I think it was Gerry Weinberg (Secrets of Consulting, 1985) who pointed out that problems can’t survive in the light of day. Once a problem is brought out into the light – or named as Gerry puts it – then your half way to fixing it.

The second half is more difficult: you have to want to fix the problem.

All too often people don’t actually want to fix the problem. Maybe they are scared of the solution, or maybe they fear the consequences of fixing it, or maybe one of many other reasons.

Two things you should know before I get into the meat of this Blog.

First, I have a lot of time for the writings of John Seely Brown. If you’ve not heard of him before get yourself a copy of “The Social Life of Information”.

Second, Wharton Business School at the University of Pennsylvania run an interesting e-magazine called “Knowledge @ Wharton”. Of the newsletters I’m signed up for it is one of the few I actually look forward to receiving.

In this month’s Knowledge@Wharton there is an interview with John Seely Brown, together with John Hagel he has a new book out “The Only Sustainable Edge: Why Business Strategy Depends on Productive Friction and Dynamic Specialization.”

You can get a feel for what the book is about from the interview and it sounds like it might be quite interesting. I’ll add it to my “I should read” list.

Anyway, the reason for mentioning all this comes towards the end of the interview and John Hagel says:

“Ultimately what we see is the re-conceiving of the role of the firm. Traditionally the role of the firm has been to increase the efficiency of transaction costs, whereas we see more and more that the firm has to provide opportunities for capability building of the people within the
firm. If the firm cannot do that, people will leave and seek out environments that can help them accelerate capability building better. It’s a very different way of thinking about what the firm needs to provide to its employees, and the role of the employees within the firm.”

Wow! Your not kidding, this is re-conceiving the firm. I’ve been giving some thought to this since I read it a day or so ago and I think he’s right. Lets take it a bit at a time.

Yes, firms have traditionally existed to produce efficiency in transaction costs. A car manufacture has all the designers, engineers, production staff, distribution, etc. etc. in one entity because they is how they can use them efficiently, no need to ask yourself “Can I trust this supplier?” when the supplier is yourself, no need to go hunting for trucks to deliver your cars when you own a fleet of trucks yourself – you get the idea.

Only building companies is hard. And we know from experience that competition drives down costs – this one of the reasons why US capitalism beat Soviet communism. But if competition results in the lowest costs and prices why don’t we compete for everything? Why shouldthat car manufacture have a fleet of trucks when if he adds competition his costs will be reduced?

Answer: transactions costs, because the savings from setting one trucking firm against another are less than the costs of finding the firm, negotiating a contract, monitoring the contract and paying the bills.

But this is changing, trends in outsourcing – often enabled by IT – mean that transactions costs are falling and it becomes worthwhile to use competition to reduce costs. We are already starting to see the emergence of virtual companies where almost all the activities are
outsourced.

You could argue that in future we won’t need firms because they can outsource everything. But hang on, some firms must exist because they will be the outsourcers!

So, if you are a firm that exists, say you are an outsourcer so you have to do what-ever-it-is you are going to need people. Why would these people work for you?

Well, one answer is they need to pay the rent or the mortgage. But if this is the only reason people work for you the chances are they aren’t going to be very motivated, if you are using the stick rather than the carrot people may go through the motions, they may do something, it mayeven be the thing you want them to do but it isn’t going to be very efficient.

Contrast that firm with the one that does uses the carrot. They create a place where people want to work, where people are enthusiastic, where they are motivated. Suppose they are doing the same – or similar task. Who is going to be the most profitable?

The answer to that question kind of depends on how you see the world and what evidence you decide to look at. For me the answer is the second firm. Trouble is, the second type of firm is much more difficult to organize, operate and manage. It also is also he opposite of the macho images we get from TV and Hollywood: “Do it or I shoot you”, “Make it so”, “I could tell you but I would have to kill you.”

But back to the plot…

Now remember, these firms are outsourcers, other people are giving them business to transact. So, which firm is going to win the bulk of the contracts? Right, the second firm.

So the question boils down to: how do I get these enthusiastic people to work for me? (So I can have a productive workplace and win those outsourcing contracts.)

This is where the Brown-Hagel argument really makes sense to me. It is the firms who give their people the opportunity to grow. Notice the word is grow, not challenge, not highest paid, but growth.

When I think about my own career is a search to improve myself, to learn more and, yes, grow. Sometimes I’ve done this by earning lots of money and using this to travel, buy a house so I can have my own space, buy car so I can experience things; and sometimes I’ve grown by working abroad, and sometimes I’ve just grown by working on an interesting project.
Once I’m not growing I’m not interested in work and my productivity goes down, I start asking: what next?

And then I come home at night. First I might cook – I find growth in cooking, new recipes, new variations on recipes I know, practicing a familiar one or inventing something from what I have in. And then often enough (too often?) I lock myself in my study to work on some pattern paper, or a magazine article, or (re)organising some body I’m involved in, why I might even write a blog!

Or maybe I settle down with a book.

If bed-time comes and I’ve not grown a little, learned a little something, thought about things I kind of feel I’ve wasted my time.

Maybe not everybody will buy into the Brown-Hagel argument, growing isn’t for everyone. But, where you do need them, and where you can take this advantage then I think your onto a winner.

I’m from software development, when I was an undergraduate I used to love the fantasies my lecturers used to tell me about Formal Methods. If only everyone could specify their program formally then everything would be good. I loved these stories. And then I tried it.

They didn’t work like I was told they would, they didn’t solve anything, they moved the problem somewhere else. Now later in life I don’t believe they ever could – you see, I’ve discovered the soft side of software development.

After formal methods fell from the pedestal I read a lot on software process practice. I could see how you could make it all work like clockwork. This was good. And then I worked for a company certified to ISO 9001 – it was hell.

I never got started with CMM before I worked for a company that was introducing it. There is a lot of good in the CMM literature but it is a ruler to measure things by, not a thing that stands high by itself. Anyway, its upside down. The top layer is “self improving” and this is what you need right at the bottom.

Now I’ve been to business school and have a qualification to prove it. So I’ve worshipped at the feet of great companies, GE, Sony, JCB and others – my personal favourite was SAS Institute. I’ve read how it should be –Porter, Hamel, Levitt, Womack and others. (At least Tom Peters is entertaining – particularly since he’s happy to be inconsistent; may favourite is Henry Mintzberg, he knows things are not as they are “supposed to be” and he’s happy to say it. His great insight: things change over time, ideas emerge.)

Yet I work in business, and the company I work for is pretty enlighten, intelligent people and, at least on the face of it, its managed well. Still, I can find dysfunctional areas. Does this mean the company is doing something wrong? Is there some great thinker/author they’ve missed out on?

But then, every company I’ve ever work for has had some degree of success. Even one or two really really bad places have had successes, or at least a successful history. So why is it so difficult to find a company with gets it right?

Does your employer get it right?

Where is the company that knows its customer, isn’t stuck-in-the-middle, practices lean manufacturing, values innovation, develops software in a Agile way with CMM level 5?

I’m forced to the conclusion that such places don’t exist.

If such placed did exist we need so much literature?

And actually, if such a place did exist wouldn’t it be a bit boring to work at?
Somewhere in all companies there is dysfunction. This brings opportunities and challenges. It doesn’t necessarily mean the company is doomed – although it may be. The important thing is
self-awareness, is the company and employees aware of a problem?
(Problem: its difficult to admit your dysfunctional when your interviewing someone. So you paint a rosy picture, and when they join they get disappointed.)

It maybe they are aware, they are trying to fix it, or they are routing around it – who cares if your front line people can’t get their innovations accepted up the chain if the R&D department have great pipeline of new ideas?

No, its when people don’t know about dysfunctional that there are real problems. How can you deal with an issue if you are blind to it?

I’m sure I’ll return to this topic as I write more in this Blog.

Have you ever read a specification document? I did today and I was reminded why I don’t like them.

They are boring. Even when they are telling you something you didn’t know they are boring.

They are boring because they follow the “form follows function” mantra. The function of a specification document is to communicate what one person wants (or thinks they want) to a second person who is responsible for implementing it. But there is more, the first person wants to specify exactly what it is they want, so thinks are set out in simple fashion, e.g. bullet points, no prose. Supposedly this limits ambiguity.

It also provides an audit trail so they can go back and see what has been done and what isn’t.

Yet in communicating from A to B they fail. They fail because they are boring and person B, the receiver, is going to switch off. The receiver is likely to read what they want to read, they will quickly get a feel for what they think the document says and proceed to read it like that. (Remember, the meaning of a message is decided not by the sender but by the receiver.)

Ambiguity still exists, in fact, because the document is so boring it can be difficult to pay attention enough to spot the ambiguity. And because everything is presented as bullet points it can be difficult to understand how the different parts hang together.

I suppose it does provide an audit trail but this is more damming than it is positive. We create an audit trail because we expect things to go wrong, we expect to need to trace back and apportion blame. So, we set up an adversarial relationship.

And when you’ve worked with a few requirements specification you know they are frequently the subject of battles so you don’t approach them with any enthusiasm. So you find them even more boring.

The result? Requirements documents fail on all counts; they fail to communicate, they fail to provide an mechanism for getting things done, they fail to enthuse, and they set up a bad work environment – so things are more likely to go wrong.

How do we fix this?

Well, I don’t have any hard tried and tested solution so these are just ideas:

• On site customer, this is what Extreme Programming recommends and it seems to bring benefits
• Close in Product Manager, if you can’t have a customer have a Proxy Customer, a Product Manager
• Use verbal communication in addition or instead of written, involve everyone
• Rather than try to specify everything down to the last detail allow people to engage on a voyage of discovery, involve them in finding the requirements
• Paint pictures and visions, allow people to flesh out the detail themselves
• Tell Stories

Stories are something that interests me. They are a topic I expect to return to over the course of this blog. In the meantime I’ll just recommend one book. It is The Springboard by Stephen Denning.

Maybe it’s odd that I’m writing a second Blog entry in one day – I’ve just finished the first! – but I have something else to say.

I’m just in the process of updating my website to refer to my Blog, which means I have to take down a page that said:

• “Strange but true. OK maybe not strange to you but some people have asked me if I keep a Blog and I say No. I then go home, think about it and decide it would be a good idea but I still don’t do it. Why not?”

Well, as you can see I do now. So I should answer the two reason’s I gave for not Blogging:

• “Firstly I keep a work diary, by keeping it closed I can think more freely. I think it was Dag Hammarskjold (second secretary general of the UN) who advocated “open agreements arrived at by secret means”; (In contrast to Wood Wilson who called for “open agreements openly arrived at”.) His logic was that sometimes you needed to change your position, and this was easier done in secret. I feel the same way.”

Well, maybe I’m making a rod for my own back and I’ll regret it later. For the moment I’d like to try Blogging, lets see what happens.

• “Secondly, my writing is too poor to keep a Blog. I do publish – on this website and elsewhere – a lot of writing. But this has usually been much edited and changed before it sees the light of day. This allows me to exercise my thoughts and get them into clear(er) English.”

In writing this Blog I’m going to attempt to address this problem. Hopefully, I can keep my ideas to “bite sized chunks” which don’t need much editing. Secondly, I beg the readers forgiveness.

Lets see what happens.

As some people know I’ve not been particularly keen on blogging, so, since this is a blog, indeed my blog, that raises the question: Why?

In a future entry I’ll write about why I don’t like the word why but for now lets rewrite the question: What do I hope to achieve by writing a blog?

Well, I first got thinking about blogging about 18 months ago when I was at XTC (Extreme Tuesday Club) when Chris Matts asked “Do you blog?”

So, I’ve spent 18 months thinking about this and now I have a need. For about 5 or 6 years now I’ve been writing magazine pieces. Mainly these have appeared in ACCU Overload. They started with technical pieces and moved through an enquiry phase and into a research based new paradigm phase (influenced by my MBA course). Lately have become more opinion pieces.

After six years its time I tried something new, a forum that is more opinion based may just be what I’m looking for.

Next I’m quite taken with the idea of personal journals, or diaries if you prefer, or learning diaries if you want to give them a longer title. My interest in these was started when I kept one as part of my MBA course. Since then I’ve kept a professional diary quite regularly. I use it as a sense making and thinking mechanism in work.

I’ve always said that the advantage of a private diary over a public blog is that I can say things privately that I can’t say publicly, e.g. my boss is ??? However, I’ve also noticed a tendency in my own diary to sometimes become a frustration vent that can be less than constructive. So, this blog is an experiment: What advantages does a public blog have? Well, I’m about to find out.

(I suppose a legitimate question is: what makes me think that anyone would want to read my ramblings? Well my website is now taking over 10,000 hits a month so maybe some people are interested in my views.)

So now I blog. This is Entry One.

What is this blog to be about?

I’ve been politically active in the past but I’m not any more so I don’t think this will be a political blog.

There will be lots of software development stuff in here, that’s my background, although its something I’m trying to get away from – I’ll write another entry about this and identity sometime. When I write about software development I’ll probably be writing about Agile development and Lean software, I’m a big fan of both of these.

There will be some business stuff – I’m interested in business so I’ll pull that in.

Now there is an intersection here. Two in fact.

The first is Change. This is the real problem that both software developers and businesses face. How do I change my development team to Agile practices? How do my customers need to change? How must my business change?

Second, there will be a lot of stuff about Patterns. I came to patterns through software development (you know, Gang-of-four stuff) but my current interest is in applying pattern techniques to the business domain. When I say “business patterns” I don’t mean the kind of “IT business patterns” that some people propose as “business patterns” – I mean IT free patterns of business strategy, operations, etc. Take a look at my business patterns page if you want to know more.

There you have it, another blog is born.