Agile

I shouldn’t do this, it raises my blood pressure, I have better things to do, I have better things to blog about but I can’t help myself….

I made the mistake of opening a mail from a recruitment agent, the title: “Agile Project Manager/Scrum Master”, let me dissect it for you…

“My client based in the West Midlands are looking for an Agile Project Manager/Scrum Master for a long term project opportunity starting in May.”

Agile Project Manager/Scrum Master? – this is a pretty confused role, it is one thing or another, its one thing for a good project manager to act as a facilitator or for a Scrum Master to take on project responsibilities but to start out like this, well is just messed up.

Long term project? – there is no such thing, projects are temporary!

“The successful Agile Project Manager will be responsible for the planning, organising and managing of IT projects within this matrix resource environment. My client is looking for a technical Project Manager with strong experience in leading and managing multiple complex projects in agile environments. To be suitable you MUST have strong process and project management skills with experience in SCRUM.”

Planning, organising and managing of IT projects…? – what happened to self-organizing, empirical control?

And note “projects” not project (as it was before). Multiple project is a bad sign.

Matrix resource environment? – Matrix management, this organization has shot itself in the foot already.

Technical Project Manager? – so this person is a Scrum Master, a Project Manager and Technical, do they also make the tea and wash up?

And why all the SHOUTING? – Scrum is spelt Scrum, as in Rugby, it is not S.C.R.U.M. or SCRUM.

“As the Agile Project Manager/Agile Coach you will have ownership of the full life cycle project process, including the Software development lifecycle.”

Coach too? – as well as a Scrum Master, Project Manager and Technical? I’m not even complaining about the use of Agile and Scrum as if they a synonymous.

Ownership of the full life cycle… including the software development lifecycle, really? – except for the fact that the contract is for 6 months, the company is matrix managed, there are multiple projects and that some bastard hotchpotch process of SCRUM and traditional project management has been mandated.

In other words: you have as much ownership as a homeowner who’s house has just been repossessed by the bank.

Anyone taking this job would have their hands ties behind their back on day one.

“Key requirements:

• Demonstrable experience managing complex Projects in fully Agile environments
• SCRUM Master with previous experience of running Agile project teams
• Excellent presentation skills & competence in project management tools
• Full life cycle delivery experience
• Ensure all SCRUM ceremonies are adhered to throughout project lifecycle
• Collaborate with project teams (both on and offshore), create and maintain project plans and schedules”

Fully Agile environments don’t have projects, Scrum Masters don’t run Agile teams and “maintain project plans and schedules” sounds distinctly Gantt chart-ish.

Saying “all SCRUM ceremonies” sounds like the people doing the asking don’t actually know what the ceremonies are, just they must be “adhered to”. Anyway, Scrum isn’t a full lifecycle model so good luck here!

What is interesting here is what is not said:

• No mention of facilitation, unblocking, coaching
• No mention of technical skills (TDD, ATDD, BDD, refectoring, etc); this is supposed to be a “technical project manager”
• No mention of empirical process control or forecasting
• No mention of value
• No mention of prioritisation
• No mention of the Product Owner/BA/Product Manager

Normally I would be pleased that the recruiter wasn’t asking for a Scrum Master Certification, a Prince 2 certificate, or, heaven forbid, a “Agile Project Manager” qualification but on this occasion I read it more as a sign that the company doing the recruiting don’t actually know what they want.

Similarly, no mention of tools or technologies – Agile in PHP is very different Agile in embedded C, let alone the ERP Agile I’m working on now.

Finally,

“If you are interested to find out more, please reply…”

Should read:

If you are a traditional project manager (who used to code in the dim and distant past) but can’t get work now the world has gone agile, but think you know what agile is (mini-waterfalls) then please give me a ring, you probably know more than the people doing the interviewing anyway.

Bottom line:

1. Unfortunately this is the state of our industry, it is what many organization do and want. But it is far far from what the best teams do.
2. Don’t blame the recruiter, their industry is based on failure, most recruitment agents are failed real estate agents.
3. The organization requesting this role have such a poor understanding of what they need they shouldn’t actually be allowed to hire anyone.

Taking a break from my recent series of blogs about management, I’d like to discuss something that has come up with my current client – OK, I’m bending my rule to and blog about live events but this isn’t about the client, its about an idea.

Opportunistic salvage

Many readers will have seen this scenario before: there is an existing system, it has been developed under a previous approach, a non-Agile, an approach that didn’t give quality a high priority. Some believe that the new work, based on an Agile approach, can take the current system “as is” and build on it, maybe some patches will be needed but why throw away something that has already been built?

Something customers have? Something you’ve paid for? – they have a point.

And there are other people who believe, perhaps with good reason, the existing system has poor code quality, poor architecture, significant defects (bugs), and in general takes more effort to maintain than it would to rebuild.

As always the truth lies somewhere in between, and as always, determining what the truth is is not easy. The best that can be said is: some parts of the system are good and can be built upon, and some parts of the system bring more costs than they do benefits.

Anyone who has ever owned an old car will know the problem: you take the car to the garage and you are told “Ooo…. this is expensive to fix” – not as expensive as a new car but expensive. Nor it this the first time you’ve taken the car in, this time its the steering, last time it was the electrics, next time… the clutch? the gear box? And maybe there is rust.

You have to make a decision: should you spend the money to keep the car running or invest the money in a new car?

This is the question we face. Our solution is: Opportunistic Salvage.

We will look at each part of the system on a case-by-case basis and decide whether we can bring it across or whether we rebuild.

Critically this is driven by tests, automated acceptance tests to be specific.

Faced with a request, a requirement, we develop a series of acceptance tests – based on the acceptance criteria that go with the requested story.

These tests will be implemented in an automated ATDD fashion using a tool like Cucumber, SpecFlow, FIT or Selenium. We will then take the part of the existing system and run it through the tests.

Two points to note at here:

• Building an automated acceptance test system is needed for a proper Agile approach. So building this keeps with the new approach. (If you are not automating your tests to build fast feedback loops its unlikely you really are doing Agile anyway.)
• Some work may be required to make the section(s) of code testable, i.e. make it a module with a well defined interface. If the code is already a well defined module then good; if not then it should be. Going forward, in an Agile approach, requires good code so if an investment is needed then the investment is made.

Now one of two things will happen.

Possibly the lifted code will pass the tests, this is good and shows that the people who said we should build on the existing were right. We can take a quick look inside the code to make sure its good quality but it probably won’t need much work – after all it just passed the tests. If it happens that a bit of work, possibly refactoring is required then since there are now tests this is quite doable.

On the other hand, if the module fails the test… well, it is a good thing we found out now and didn’t leave it till later! (One wonders how the current system is working but lets not worry about that right now.)

The code has failed tests so obviously needs some work. Nobody wants to ship code which is known to be faulty.

Now we look inside the code, and we make a professional judgement: is it better to patch up this code and make do? Part of this judgement included remember the product is expected to continue for some time to come.

These decisions are made on a case-by-case basis. There is no blanket decision to move all the old system, nor is there a blanket decision to write everything from new. What is good is kept, what is OK is patched up and what is bad is thrown away.

The people to make the judgement call are the people who are going to be working on the code: not the managers, not the business analysts, not the project managers, not the product managers, not even the testers. All these people may have a view but there is one group of people who are recruited and employed because of their professional skills with code.

The programmers, aka software engineers, aka software developers.

Given a code module that is failing tests and is expected to have a long life these people need to make a professional decision on whether the better course of action is to patch and make do or to rewrite.

In making this decision we are accurately aware that sometimes writing from scratch can be faster than patching up something that already exists – faster both in the short run and in the long run when ongoing maintenance costs are considered.

This is especially true in software, especially true when the people who originally wrote the code are long gone and especially true when the original code was written without tests.

These decisions are made on a case-by-case basis. Where code can continue it is salvaged and reused. Where isn’t sensible then it isn’t.

So… no debate about legacy or new build: Opportunistic salvage driven by tests.

If those who believe the legacy product provides a solid base are right then there will be little new code written, sure you will need to write some tests but you will want to do that anyway.

If those who believe the legacy product should be rewritten are right then many tests will fail, code will be found to be crummy and it will be replaced.

I’ve been meaning to write about the managers and Agile software development for a long time. And, apart from a few asides, I haven’t.

Why not?

Well partly because the topic is difficult, or rather large, but mostly I’ve not written it because I’m fearful of the flames that will come down on me.

You see I think managers have a role to play in Agile, but I am acutely aware that many people don’t.

So am I steeling myself, I’d love to come up with a “grand theory of management” but I’ve been trying to do that for years! I think instead I’d start blogging about my thoughts on management, and hopefully a grand unified theory of management will emerge.

Still, I expect many people will not agree with me – there is one of them I can see right now, he’s almost standing in front of me. The funny thing is, I was a professional programmer, I got interested in management as a way of ensuring I didn’t become the type of poor manager I’d worked for.

I took time out of my programming career to study management. What nobody told me was that if you study management programmers are more suspicious of you than ever, non-developer management types will always see you as a programmer (perhaps a jumped up programmer) and programmers who have fallen into management (those with no qualification and probably no time to study the subject) don’t trust you. Add in the fact that the academic community is split over management qualifications and it’s a right mess.

Unfortunately managers and programmer are locked in a decades old existential fight.

Many on the management side of IT dream of a world where programmers can be replaced by a software tool. That isn’t going to happen, all it does it move the point at which the programming happens. If you replace a programmer with a software tool the person who uses the tool is now a programmer – whether you call them a programmer, analyst, consultant or serf.

On the other side programmers dream about replacing managers with “self organizing teams.”

Both sides want victory by proving the other need not exist – at which point they will disappear in a puff of smoke. (Sometimes it seem to be the old worker-capitalist class struggle.)

My logic is this:

• Management is a skill like any other, like writing Java or like writing test scripts
• Many people who find themselves managing don’t have the skill and some don’t bother to learn
• Consequently many “managers” aren’t actually very good at managing
• While there are a few “natural” managers – like there are natural Java developers(?) – one can learn to be a better manager

The state of management in the IT world is so bad that frequently removing the manager altogether makes things a better than they were.

Oh, and the misuse of the word “manager” doesn’t help.

The thing is…

I’m an ex-programmer, I still consider myself an engineer (another controversy) and I also think management can play a useful role in engineering software development.

It took me 10 years to become a software engineer and I never stopped learning, 34 years later I’m still learning to be a better programmer.

I also hold an MBA degree, it took me a year of hard work to get it.

1 year, not 10 years and certainly not 34 years.

Actually, it has taken three generations to make me an engineer: my farther was an engineer, and my grandfather too. I was going down engine rooms when I was 7 or 8, I’ve watched engines being overhauled. I know what being in dry dock means and I know commercial considerations and engineering considerations are often in conflict, but engineers not people in suits make it work.

Being an engineer and a manager is not a contradiction. I know that because my farther and grandfather were Chief Engineers: they took years to obtain that rank, and when they did they still went down the engine room, they still mucked in sometimes, but they also spent a lot of time “managing” the work in the engine room and on deck.

You see: it takes an engineer to manage engineering.

Summary

• The general quality of management in the IT world is poor
• Management itself is a skill, one which few take time to learn about
• Managing software engineering well requires one to understand software engineering: it takes and engineer to manage engineering

I’ve done a few agile introductions in my time, in fact I’ve started to feel I could almost write a book entitled “agile by numbers”. So yesterday when this question appeared on some LinkedIn group I thought I’d give it a quick go:

“I am working with an organization which wants to explore agile adoption. What are the some of the best ways for start the process. Is it to start with agile training and then help set up with an agile agile coach to take it forward ? Or first do an assessment to understand why they want to adopt agile and than suggest the best approach : if it’s scrum, Kanban or XP.”

My answer went something like this:

First, don’t get distracted by researchershing options, don’t worry about Scrum v. Kanban v. XP v. What-Ever. Just get on and do it. (Of course I should have plugged my own book and said “Do Xanpan!”)

1. Find someone you can trust who knows about this stuff (perhaps I should have said someone like me!)
2. Get them to give an 1 or 2 hour (max) talk to interest people
3. Ask the team leads if they would like to volunteer their team or upcoming work, then talk to the people who would be involved, choose/make a small self contained team (i.e. coders, testers, product person) (Note: if you only have one team then you can skip steps 2 and 3 because you are doing to work with this team.)
4. When you have found your guinea pigs give them a day or two of rehearsal (i.e. a training session where they get to practice what they do)
5. Start doing weekly iterations immediately: no waiting for “the right time”, start while things are fresh
6. Arrange for the person who gives the training session to come back each week, then throttle back to less often when the time is right
7. Engage a technical coach/trainer and after several iterations run some technical training (TDD or BDD, take your pick). Have this person come back for some one-on-one coaching (i.e. pair programming), allow one day per coder plus time with testers if you have them
8. See what happens, review, keep going, expand

I did notice that a few of the other replies recommended teams start with coaching rather than training. I’ve done it that way, in fact thats how all the early teams did it. I’ll also say that in my experience – and yes, I’m biased, I sell training – giving people some training up front works best. Perhaps thats because the way I do training is to give people a chance to experience working in an Agile setting in the classroom, so perhaps what I’m saying is: all training is not equal.

This little recipe takes you so far. I’ve stopped with the technical stuff. The process side will get you so far but if you don’t do the technical stuff – lots of automated testing! – then you will only go so far.

The other thing I haven’t said here at all is: this recipe is only for the supply side, once the supply side starts to shape up the focus needs to shift to the demand side, those people you call Product Owner, Business Analyst, Product Manager or something like that. Thats where it gets difficult, not because these people are themselves more difficult but because you are now wrestling with the wider organization and changing a bigger mindset.

Another problem that sets in here is that companies can loose the will to change. I’ve worked with a few companies who apply this prescription but they get so much improvement the immediate pain goes away and they loose the motivation to change further.

This prescription can very quickly makes things get better. But if you want the full benefit you need to keep taking the technical medicine and supplement it with demand side medicine too. Those changes take longer to make a improvement, they are a slow burn.

I sometimes come across teams who are wrapped up in discussing what they should be building. OK, I suppose this is because I hang out with too many management types. Sometimes there are only a couple of programmer, occasionally one, or maybe they haven’t got that far.

Sometimes I come across pre-teams, people who will form a team to do some work but haven’t started yet. I guess the project folk would call this “pre-project phase” and there is a lot of discussion: “Should we do Agile or Waterfall? Scrum or Kanban?” and “Which outsourcer will we use?” and “If we are agile how will we….”

In short these people are wrapped up in agonising about what might happen.

On these occasions I’m reminded of a story I once heard about the actors Laurence Olivier and Dustin Hoffman. In the story Hoffman, who is a famous method actor, says to Olivier something like:

“Mr Olivier, I am honoured to be working with you, but please tell me how do you learn you character? I mean do you use the method or some other approach? What should I do?”

To which Olivier says:

“My dear boy, … have you tried acting?”

This is what I feel, but usually don’t, to these teams, managers, analysts, whoever:

“Have you actually tried developing and delivering some software?”

Some planning, some forethought, some agonising is good. It helps you think things through. But it quickly becomes a game of diminishing returns. Its the old Eisenhower quote again:

“Plans are worthless, but planning is everything.”

Which is a natural follow on from Helmuth von Moltke’s observation:

“No plan survives contact with the enemy.”

Peter Drucker has a similar, though to my mind better, quote too:

“Plans are only good intentions unless they immediately degenerate into hard work.”

So let me say:

“No plan for software development survives contact with the code. (Double so for legacy code.)”

“No requirements roadmap ever survives contact with the market.”

(According to wikipedia the Olivier-Hoffman exchange happened during the filming of Marathon Man and isn’t exactly true but does contain a grain of truth.)

One of the reasons this blog has been quiet recently is that I’ve been putting my efforts into video: The #NoProjects Video series.

This is now available as 16 short episodes on YouTube based on my popular #NoProjects/Beyond Projects conference presentation. If you have seen the conference presentation you know what to expect although there are some changes. My hope is that by making the presentation available as 16 short (I think the longest episode is 5 minutes) pieces the argument will reach a wider audience.

There are a few questions I’ve been meaning to clarify about #NoProjects / Managing Without Projects and now is as good a time as ever:

Q: Isn’t “No” rather negative?

A: Yes it is, I’ve had a go at changing it but attempts so far haven’t caught the same feel. Perhaps this is because “No” is so definite or because “Beyond” is 4 characters more and 4 characters on Twitter is a big deal.

For a while I preferred the name “Beyond Projects”

Lately I’ve been calling it “Managing without Projects”

But perhaps “Software without Projects” might be better still, that would lend itself to the hashtag #SW2Prjs

Q: Is #NoProjects actually #NoEstimates?

A: No

The hashtags and ideas originate with different people. There are parallels and the ideas might be symbiotic but while I know some #NoEstimates people would like to see #NoProjects as part of the same idea I don’t.

Estimates are a complex area and right now its difficult to have a rational conversation about them – perhaps it was ever so. But, I still see value in “estimates” in some context. Its all a question of “What question are you really asking?” using estimates might help or it might now.

While most of the conversation on “estimates” implies “time estimates” there is another form of estimation: “value estimates”. These don’t get anywhere near enough attention or use. I use them and I help my clients use them but I don’t see them elsewhere very often.

I should expand on this another time.

Q: Does #NoProjects mean #NoManagement, aka “Self managing teams” ?

A: No – I have no argument with management itself. I have an argument with the concept of projects which I think leads to poor management of software development.

I see management as a skills, the same as Java, SQL, Testing or Analysts. Unfortunately very few people actively learn about management or try to master it as a skill in the same way they might Python or Java.

When they do try and learn “management skills” people often start with “project management.” I sometimes thing of “project management” as “management lite.”

As a result an awful lot of “management” of technology (specifically software development) is poor. In such conditions removing management (i.e. self management) can actually improve things.

Again, I should really write an extended piece on this subject.

Q: What about Project Managers?

A: I treat each and every Project Manager as an individual. They can bring valuable skills and experience to a team. Unfortunately many of them don’t and add little. In part this is because they are asked – both by their employer and their professional bodies – to manage using a model that is inappropriate to the software domain.

So I don’t see mass redundancies of project managers, rather I see the good ones being allowed to manage properly using a better model.

Q: Isn’t this a pretty radical position when so many companies revolve around the project model?

A: Yes, my hope is to get people to talk about the issues I raise. Even if companies don’t abandon the project model completely I hope they can get better by finding their own ways to address these issues.

Besides, I’m a consultant and I’m only too happy to talk about these issues myself and give advice. Just call me.

As a consultant few of my clients actively set out to become a NoProjects company and I don’t try and make them so. They and I want them to do better. I find that if I think in a NoProjects way I can understand the client better, and if they still have projects I can then map the ideas across.