Management

Rational irrationality and anarchy in the workplace

Agile, Management / / / , , / Leave a Comment

This is not the post I intended to open 2026. In recent weeks several threads have come together to challenge my thinking and I feel compelled to share with my readers. So, a slightly long and philosophical start to the new year!

Garbage can

A while back I read Amethodical systems development (Truex, Baskerville and Travis, 2000), it has stayed with me and has been a major influence on my reasoning about software. The authors argue that every development, contains unique aspects and is not replicable. However, by focusing on “a method” engineering has elevated the processes used to a privileged position and neglected what actually happens. To some degree the emergence of methods based on experience (e.g.XP) in the years after the paper addressed some of this concern but not all. It also means that Scrum, XP, PRINCE2, SSADM, SAFe – or any other brand names method – overlook many important factors. (Hence why I always describe Xanpan as a model of what you can create yourself.)

Rational irrationality

For a few years now I’ve been meaning to write a blog about Rational Irrationality. This is a phenomenon I’ve seen again and again inside corporate environments. In a nutshell this is interplay of rational processes which combine to produce irrational systems. Rational processes and systems are put in place with the best intentions but once you have a few, independently rational, processes in place the interaction of these becomes irrational.

Perhaps the simplest example I remember was a Senior BA who refused to let his analysts look at user requests until the Technical Architects had proposed a design. He reasoned that his BAs were stretched, many user request never went anywhere so until the Architects had given it their backing he wasn’t going to allocate any BA time. Meanwhile, the Lead Technical Architect, quite rationally, didn’t want his people designing systems which hadn’t been scoped, how could the design something if they didn’t know what it was? Both were acting rationally but the result was irrational.

The two times were irrational rationality seems to peak are around project inception and kick-off, then when moving to live production environments, however they everywhere. Perhaps the problem is not with irrational processes and corporations but with me – and maybe you. The problem could be less these systems but our engineering brains which expect there to be a rational, systematic, logical, way through this problem.

Just because my brain can see these systems interlocking, connecting, blocking and deadlocking doesn’t mean others do. Perhaps it is because I’m an engineer, or perhaps because I’m dyslexic and visualise, I can see these things like machines and gears in my brain, the same way I used to imagine code working.

Garbage Can Model

I revisited these ideas a few months ago when I discovered The Garbage Can Model – I must think Mark Smalley and his book AI and the Being Between Us.

The garbage can model goes a long way to explaining rational irrationality and how it comes about: despite what an organization says, and despite artefacts like org charts, the organization is anarchy. Attempts to control it as a rational thing don’t work.

Now I think back to my experience, organized anarchy is probably a better mental model than rational entity in many places. The interplay of all those rational processes creates irrationality and disconnects people. The more people try to join up work the harder it gets, too many connections, Declare independence and you are seen as disruptive and “not team players”, the corporate anti-bodies come out.

The garbage can holds a collection of problems. These may get resolved, delayed or subsumed into something else. These problems are complicated by fluid engagement from stakeholders (they only sometimes join meetings), unclear technology and problematic preferences.

There are solutions too, although not necessarily solutions to the problems in the garbage can. These solutions are products, perhaps backed by vendors but not necessarily so. These solutions are looking for problems they can be applied to. Once in a while decision opportunities arise – IME typically when money needs allocating or a deadline hits. Still, delaying a decision means problems remain.

What ideals are lying around?

The economist Milton Friedman once said: “Only a crisis—actual or perceived—produces real change. When that crisis occurs, the actions that are taken depend on the ideas that are lying around. That, I believe, is our basic function: to develop alternatives to existing policies, to keep them alive and available until the politically impossible becomes politically inevitable.”

Friedman is arguing for the creation of products (policy alternatives) which then wait until a decision point. Friedman is telling us how to work in the garbage can, whether economics, systems development, or geo-politics.

From a amethodoical view engineers are trying to create rational solutions and processes while others are biding their time until their product/solution can have its day.

Fake it

Of course, when we look back and try to explain it – or when someone says “Can you do the same for me?” – we rationalise it. We don’t admit it was a garbage can or lacked method, that some stakeholders never turned up or decisions were delayed until they became irrelevant. We explain it as if it was meant to be, or as Dave Parnas put it “A rational design process: How and why to fake it.”

After all, who would admit their process was amethodical and wasn’t the result of apply career enhancing frameworks? Or that their decision making was little more than a garbage can that only produced decisions when crisis hit?

Now, when someone believes their organisation is rational, maybe they think it follows SAFe, or maybe they the hierarchy works, they treat it like such. But their mental model does not reflect the reality. Consequently the system doesn’t respond as they expect, even if it isn’t anarchy it looks like it.

So what good is this?

I would like to think that I will stop looking for engineering solutions. That I will accept more of the rational irrationality in corporations. That I will practicing patience.

I think it is more likely that simply knowing my engineering brain is wrong to expect a well oiled machine I will be more tolerant. At the very least, I will tell myself to tolerate more. Still, I will endeavour to make my bit of the world a little better, and a little more rationale. Perhaps knowing the world is irrational will help me be rationale.

Signup for the my latest posts by e-mail and download a free book

Rational irrationality and anarchy in the workplace Read More »

Agile: not Dead, but evolving

Agile, Agile Process, Management / / / Leave a Comment

Sorry. I’ve deliberately avoiding the click-bait “Agile is Dead” topic, until now.

For the last few years I’ve delivered a lecture on Agile to Oxford University students and this year the tutor specifically asked me to say something about the state of agile. When I looked over last years slides I see I was already talking about this. I’ll write more about this soon, if you can’t wait checkout “Xanpan 2021” from Frug’Agile en Arménie.

So, is Agile Dead?

Clearly not. (Albeit agile mania probably is.)

Agile is all around us. Teams work in sprints, hold daily “stand up” meetings, tools like Jira continue to sell, requirements documents are full of user stories, business journals regularly talk about “agile” and “agility” without any reference to software.

That doesn’t mean the result is perfect. The “agile” which prevails today falls short of what I and others in the community dreamed. As I’ve said before, Agile won the war but lost the peace.

Right now, agile isn’t getting attention because AI is. AI is soaking up all the discretionally time and budget so agile is squeezed out. Ironically, to get the most from AI you need the learning processes embedded in Agile to find better ways. Right now we don’t now the best way to use AI. We are in a vast experimental phase and we need more of the learning and feedback in agile.

Back to the question, is Agile Dead?

The common agile that prevails is a watered down, corporately acceptable version that is still a lot better that went before.

But then, most people don’t remember before agile. The Big Up Front practices which gave massive requirements and functional specifications; the defined process and ISO-9000 process audits, the guilt of “not doing it properly” and the inability of those doing the work to influence how it was done.

While many of those problems have resurfaced under other names in the agile world things are still a lot better. If we had stayed with that approach there would be no automatic updates to the apps on your phone, no digital business, nor much of the other technology that surrounds us. Maybe Apple and Google would be OK but legacy banks, airlines, telecos and Governments would be even worse than they are now and a million start-ups would never have started.

In truth, many of the “waterfall” processes were never followed. I worked on exactly one project that did it by the book, Railtrack Aplan. Officially it was a success, it went live. But what went live was a shadow of what was supposed to be delivered.

Everywhere else did something that (kind of) worked and then felt guilty for not doing “properly”. When I was at Reuters they tried to force us to work by the book, they destroyed much of their own capability in the process.

What has agile ever given us?

Agile showed there was another way and added democracy by opening the debate on “how we work”. The Internet help agile spread and opened up the debate in a way that had never been possible before.

If nothing else Agile gave us a better reference model, a better way of describing our work.

Actually, it gave us several reference models, Scrum, XP, DSDM, etc. Always, and everywhere, people adapt, when processes work they use them, when defined process don’t work they work around them. For a while agile licensed that working around, experiments were everywhere.

Agile was not so much new in itself as a new combination of ideas which were lying around.

The engineering practices in XP descend from the 1970s quality movement based on the work by Phil Crosby and W.Edwards Deming.

The self-organizing teams in Scrum drew on the sociotechnical systems. First recognised in the 1940s and 1950s by the Eric Trisk – then at P&G and Topeka and the genesis of Senge’s organizational learning.

The inspect and adapt philosophy in Tom Gilb’s Evo and then Scrum comes from Stafford Beer and management cybernetics.

Lean thinking draws on many of these ideas directly but lean also begat its own software process in Kanban.

As for the Frankenstein’s monster that is SAFe… you can decide for yourself whether SAFe is agile but it is definitely not lightweight. Because of its size alone it is hard to adapt SAFe and involve the workers.

The return of Agile?

Can we expect Agile to return to its previous permanence? Will the day come when everyone wants to hire a Scrum master? No.

That has passed. Organisations have ticked the Agile box – if only because they have moved on to AI. The days of big agile transformations are largely over because companies have declared success.

Put it another way: Management fads don’t return.

Imperfect agile is here, hopefully enough of it has been adopted that companies will continue to improve.

More importantly, those ideas underlying agile – quality, sociotechnical, cybernetics, learning – are still valid and will continue to have influence. Some companies will embrace them and get a lot from them, some will continue to reject and most will dip-in-and-out. These ideas will return, albeit in a different package and with a different name.

But none of that means agile is dead. Agile mania might be over but agile is continuing to evolve out of sight. Agile wasn’t the first coming of these ideas and it won’t be the last. Next post I’ll talk more about how I see it evolving.

Signup for the my latest posts by e-mail and download a free book

Thanks to Fritz Geller-Grimm for the parrot picture under CC license

Agile: not Dead, but evolving Read More »

What I’ve been getting wrong about PDCA

Agile, Agile Process, Continuous, Management / /

I’ve been teaching planning lately and once again it seems to me that the PDCA cycle – aka Shewhart or Deming cycle – is pretty much the core of all planning. Or rather, it is the basis for all mutli-pass planning – when iteration is allowed. (One-pass planning, big-up-front-design “BUFD”, is fine for trivial situations but alway has problems in complex situations.)

So, again I’m reminded of why I don’t like PDCA. Two reasons.

Adjust over Act

When the fourth step is labeled “Act” it fails to speak to me. “Act?” I ask, “Didn’t we just DO?” Easily fixed, label step-4 “Adjust” – many people do. Now it says, “Plan a bit, do a bit, check the results, now adjust the plan or the way you are working.” That makes more sense to me.

4 unequal steps

Secondly, the typical presentation – like my diagram above – makes it look like the four steps are equal and that is not the case. Just in terms of the time they take the fourth is almost always the shortest. Which of the other steps dominate is going to depend both on the planning culture where you are and the amount of work that needs doing.

Many places will put a lot of time and effort into planning. While this can entirely justified if you are building something that lives depend on, planning suffers from diminishing returns. It is often far better to plan a little, run around the circle and plan some more. Planning is learning but so is doing, you can learn more by a few minutes of doing than hours of planning.

Other places will skip planning altogether and launch into doing. While over-planning is problematic jumping straight in is also a problem. Either way, in terms of the PDCA cycle, planning is not an equal element.

Now when planning is skipped or rush the doing phase is going to expand. In fact, on a really big endeavour which needs a lot of planning the doing phase can also be very large. Of course, its entirely possible that your planning is so excellent that you see an quick way to deliver. But again, doing is not an equal quarter.

Test-fix-test-fix-test doom loop

The same does for the check (or test) phase. It can be long or short. If your planning was good, and your doing was quality then you can hope that the check phase is really small. It does happen. But too often there is little quality in planning so you actually end-up with a short-circuit as the checks fail and more doing is needed to fix things. (This is the test-fix-test cycle that can destroy any schedule.)

I wouldn’t expect Plan, Do and Check to be equal sizes, depending on your organisation, culture and the nature of the thing you are doing I would expect one to dominate.

But I don’t see that in Adjust. Adjust is the forgotten child. Indeed, in many projects, especially at the end, everything just goes hell for leather do-check-do-check-…. Adjust, and even planning, goes out the window.

Using PDCA successfully

Even in the best places Adjust is always going to be the shortest of the four. The irony is that it is probably the most important. It is the step where reflection and improvement happen.

The truth is I’ve always struggled to apply the PDCA cycle formally. But when I look back at almost every single engagement the actual work can be mapped to the PDCA. It is fundamental, whether building product, running sprints, setting and executing OKRs or almost any other non-trivial work. Its just that the steps are not equally time consuming or equally respected.

And the secret to making it work well? Simple, go around fast. A little planning, a little doing, quick check, small adjustments and go again. Learn in the planning, learn in the doing, learn all the way round and put that learning into action.

What I’ve been getting wrong about PDCA Read More »

Patterns for building a learning organization

Management / /

The question is: “What do companies who learn actually do to learn?“

Yes, we all know about retrospectives but there is a lot more too it than that. This was the question Tsvetelina Plummer and I set out to answer in what became our paper to EuroPLoP this year: “Learning to learn: patterns for building the learning organisation”

While we know this isn’t a complete answer – there is more work to do – I’m immensely proud of the patterns here. We put a lot of effort in before the conference, first by ourselves and then with MaryLynn Manns as our shepherd. And in the last few weeks a lot more effort to incorporate the workshop comments and get the paper ready for the proceedings.

So here is the paper, we’d love to know what you think.

Even better, if you have example of what you and your company do, a case study, an activity, anything really, then please please please get in touch and tell us about it.

The official version will appear in a few months in the conference proceedings by Springer Nature but only the formatting changes really (and this version is better.)

Patterns for building a learning organization Read More »

4 quick fixes to bring order out of choas

Agile Process, Management / /

I have been working with a coach recently to think about what I do. Odd as it might sound I have real trouble articulating what I do. This is particularly true when I’m employed to work on team performance and processes. In my mind I turn up, I talk to people and they then makes things better. Can I take the credit for that?

One of the reoccurring comments I get from clients is that I bring order from chaos. While I’m immensely proud of that it is a hard sell when trying to articulate the value companies can get from hiring me. That is because it implies you already have chaos, and who wants to admit that?

Whether chaos or not I’ve been running over my past engagements and a few things stand out. These are things that I have repeatedly done with teams, things which have made work better (and therefore more productive, efficient and fulfilling.)

Most of these suggestions are quick fixes: you can introduce them pretty quickly and see results. But they do not detract from long term improvement. Even if you have time these are a good place to start.

Triage processes

These come in various shapes and sizes. Typically this means appointing one or two people to quickly review work requests and decide if they need immediate action, can wait a little while or be pushed back.

In one case this mean that all urgent support requests or bug fixes were added to the teams work board in a special column. A few minutes before the daily “stand-up” the Product Owner and Team Lead would review these and decide if it was immediate action, push-back or hold for the next regular planning meeting.

(In the most urgent cases work might begin as soon as the request was made but that decision was reviewed just before the stand-up the next day.)

In another case I convened a weekly meeting to review all the urgent work requests which had some in in the last week (mostly things people called bugs.) Collectively we would agree the prioritisation (1, 2, … 5) and then, for just the priority 1s list them in priority order. Unless the customer service manager and myself (as Development Manager) agreed something new became more important during the week we stuck to this order.

There are plenty of other examples I can give but you get the idea. These triage processes work wonders at controlling task switching, confused priorities, decibel management and other sins. Consequently they boost productivity, work order and satisfaction.

Sacrifice one

Hand-in-hand with triage I often sacrifice one team member to urgent but unplanned work: bug reports, customer service requests, and so on. Even on a team of 3 or 4 having one person designated at the urgent work or fixer can payback both in addressing issues and allowing others to work uninterrupted. This designated person isn’t assigned to any other work, or at least nothing time critical.

There are a few engineer who relish this role and love nothing more than a stream of “small fixes.” Most engineers would rather work on bigger items and stick with them for a few days. So unless you are blessed with one of these people rotate the role between team members.

Routines

I’ve no claim to originality here, this is 101 in the agile processes playbook. Still it is effective. Choose the routines and ceremonies that make sense for your team: daily stand-up, regular planning/replenishment (I prefer weekly, most teams do it fortnightly, just please avoid 3-week sprints), retrospectives (not necessarily every iteration, perhaps monthly, at least quarterly) and regular releases. I’d love to work with a high performing team who do many releases a day but the only time I’ve seen teams doing such is with bug fixes. Again, weekly, fortnightly, monthly, these can all bring more structure than you have now.

And of course, if you are using OKRs create a cadence and put the dates in your calendar.

Create focus

Taken together those three suggestions: triage, sacrifice and routines all aim to create more team focus. Focus is one of the most powerful tools you can use. These three changes start to create focus.

Beyond them there is so much more that can be done to create focus: backlog structuring and prioritisation, OKRs, testing approaches, various graphs and measurements, …

Statistical forecasting

Even if a team is making effort estimates – in time, story points, tea bags or whatever – start collecting data. Actually, since most teams these days are using electronic tracking systems you probably have an archive of data which you can mine today.

Go and analyse the data.

How many “stories” do you do a week?

What is the cycle time for a story? What is the average? The longest? The shortest?

What is the average time a story spends in the backlog?

How many things are in flight at the same time?

What is the bug count? And what direction is it going in?

What is the team “velocity” ? More importantly, what is the backlog growth rate?

There are many more question you can ask your data. Not infrequently you find that the data you have is not good enough to answer these questions but that itself is information.

When you start to answer these questions you can start to forecast based on data not guesswork.

And by the way: I’m more than happy to help you analyse your data, just get in touch and we can work something out.

Quality

So far all my suggestions can be done relatively quickly. The next one is a longer burn: improve quality.

When you quality is low a defect – a bug – can raise a hand at any time and create chaos: disrupt a release, distract and engineer, panic a customer, etc. etc, Defects can be high profile, very disruptive and difficult to close. Defects can destroy any schedule or plan no matter how carefully constructed.

The solution is simple: improve initial quality.

To do this install test-first working, code reviews, frequent customer review, shorten feedback cycles, etc. The catch is these can take a little time before they pay back.

Two I haven’t mentioned

Now there are two things I haven’t mentioned so far and will have some readers wondering why I haven’t: reduce work-in-progress and introduce outcome oriented working.

Reducing WIP can have wonderful effects but explicitly reducing WIP requires authority or credibility – and usually the credibility is required to persuade someone with authority. Introducing triage, sacrifice and routines will have the effect of reducing WIP but without the overhead of persuading people that “doing less produces more”.

So I only target WIP directly once I have picked some other low-hanging fruit. Plus, having the data to back up my case will really help. Attempts to reduce WIP usually start with a screening of Stockless Production.

Similarly, outcome driven working requires a certain credibility and trust to pull off. So often people are just drowning in work which OBVIOUSLY NEEDS DOING. Even if the engineers aren’t then there are plenty of other people saying “Just F…ing do it!” So asking people to take a step back and think about the desired outcome can be hard. Better build up some credibility first.

That is my list, what would you suggest I add to it?

And if you would like to discuss these ideas further please let me know.

4 quick fixes to bring order out of choas Read More »

Are Product Owners set up to fail?

Agile, Management, Scrum / /
Product Owner choosing postits

I’ve long seen two big problems with Product Owners.

First, Product Owners don’t talk to customers as often as they should.

Second, the Product Owner role is frequently set up to be little more than a Backlog Administrator.

Now these two problems aren’t completely unrelated. If you are simply a Backlog Administrator what is the point of talking to customers?

For a while now I’ve taken a broad interpretation of the Product Owner role. Yes, it was introduced by Scrum, and Scrum defines a very powerful Product Owner (which I agree with). But, the Product Owner role has become a general term for “the one who should decides what needs doing next.” Simultaneously the role has become undermined and very few Product Owners seem to have the power to decide much more than the next couple of items.

The Product Owner role seems to make Product Managers and Business Analysts nervous. It shouldn’t, both Product Managers and Business Analysts are types of Product Owner. After all, while the Scrum definition calls for a powerful PO it says little about how the Product Owner knows what they need to know to make decisions. In my book – and yes, I cover this in Art of Agile Product Ownership – that knowledge usually comes from being a Product Manager or Business Analyst.

Recently I’ve been taking talking to my subscriber base and the issue of Product Owner comes up again and again. While nobody thinks Product Owners should be Backlog Administrators most Product Owners in the wild are just that, Backlog Administrators.

In some cases people think it is because of their situation. “In the public sector Product Owners are not honoured…”, “In Germany, Product Owners are not valued”, “In the outsourced development market the contract ties the PO’s hands.”

Actually, its everywhere. Product Ownership are failing everywhere.

In many cases I’ve seen Product Owners who are “battlefield promotions”. A coder is picked to be Product Owner on the grounds that Scrum demands a product owner and they are “The best coder” (or perhaps the worst), “they understand the system”, “its a technology project” or some other weak criteria. Real power continues to be vested elsewhere.

In other cases the Product Owner comes from “the business side” and has little understanding of the technology, how technology teams work or what is expected of them. Consequently the technology group run circles round the Product Owner.

There is a reoccurring tendency for Product Owners to become the Product Dogsbody and pick up the things that nobody else wants to do. Other times the Product Owner is seen as the team leader by those outside the team, however they hold little real sway inside the team (because they are a Dogsbody, Backlog Administrator, coder who is acting up or new to technology).

But the thing is: we can’t blame the Product Owner for any of this. They have been set up to fail by people who don’t understand or respect the Product Owner role, real power is withheld from the PO. Possibly because giving the PO power can seem like reducing ones own power.

As a result the role as an operational necessity (because Scrum or SAFe say there should be one) rather than the Strategic Role which is should be.

Perhaps one mistake I’m making here is: using the term Product Owner. Perhaps it is too connected with Scrum, and in the same way that Scrum has been devalued maybe Product Owner has too.

So, how do we fix this?

I’m not sure. Perhaps we should dump the title Product Owner, maybe everyone should use Product Manager – although that title too is misunderstood. Product Leader could be an option but I’ve also heard complains about that. Product Expert? Product Analyst? Product Person? – would any other name help?

In the past I’ve run a Strategic Product Owner workshop which aims to give POs the skills to operate more strategically. But for this to happen someone at a more senior level has to ask for the workshop and ask POs to be more strategic.

One option, is for Product Owners to meet customers. This will both inform their decision making and give them the legitimacy that comes from customer contact.

A brave PO can simply start thinking and acting more strategically, they can put themselves in the driving seat and start making decisions. While I think this would often work it requires a brave person who is willing to risk their job. (And please, meet some customers before you grab the controls!)

Ultimately the good, knowledgable, powerful, product people are key to achieving agility, failure to give such leaders power puts limits on success.

Subscribe to be the first to hear of new posts

Plus download Continuous Digital for free get occasional discounts and gifts

Are Product Owners set up to fail? Read More »

Software has diseconomies of scale – not economies of scale

Agile, Management / / / , ,

“Practical men, who believe themselves to be quite exempt from any intellectual influence, are usually the slaves of some defunct economist.”

John Maynard Keynes

Most of you are not only familiar with the idea of economies of scale but you expect economies of scale even if you don’t know any ecoomics. Much of our market economy operates on the assumption that when you buy/spend more you get more per unit of spending.

At some stage in our education — even if you never studied economics or operational research — you have assimilated the idea that if Henry Ford builds 1,000,000 identical, black, cars and sells 1 million cars, than each car will cost less than if Henry Ford manufactures one car, sells one car, builds another very similar car, sells that car and thus continues. The net result is that Henry Ford produces cars more cheaply and sells more cars more cheaply, so buyers benefit.

(Indeed the idea and history of mass production and economies of scale are intertwined. Today I’m not discussing mass production, I’m talking Economies of Scale.)

Software Milk
Software is cheaper in small cartons, and less risky too

You expect that if you go to your local supermarket to buy milk then buying one large carton of milk — say 4 pints in one go — will be cheaper than buying 4 cartons of milk each holding one pint of milk.

Back in October 2015 I put this theory to a test in my local Sainsbury’s, here is the proof:

Collage of milk prices
Milk is cheaper in larger cartons
  • 1 pint of milk costs 49p (marginal cost of one more pint 49p)
  • 2 pints of milk cost 85p, or 42.5p per pint (marginal cost of one more pint 36p)
  • 4 pints of milk cost £1, or 25p per pint (marginal cost of one more pint 7.5p) (January 2024: the same quantity of milk in the same store now sells for £1.50)

(The UK is a proudly bi-measurement country. Countries like Canada and Switzerland teach their people to speak two languages. In the UK we teach our people to use two systems of measurement!)

So ingrained is this idea that when it supermarkets don’t charge less for buying more complaints are made (see The Guardian.)

Buying milk from Sainsbury’s isn’t just about the milk: Sainsbury’s needs the store, the store needs staffing, it needs products to sell, and they need to get me into the store. All that costs the same for one pint as for four. Thats why the marginal costs fall.

Economies of scale are often cited as the reason for corporate mergers: to extract concessions from suppliers, to manufacture more items for lower overall costs. Purchasing departments expect economies of scale.

But…. and this is a big BUT…. get ready….

Software development does not have economies of scale.

In all sorts of ways software development has diseconomies of scale.

If software was sold by the pint then a four pint carton of software would not just cost four times the price of a one pint carton it would cost far far more.

The diseconomies are all around us:

Small teams frequently outperform large teams, five people working as a tight team will be far more productive per person than a team of 50, or even 15. (The Quattro Pro development team in the early 1990s is probably the best documented example of this.)

The more lines of code a piece of software has the more difficult it is to add an enhancement or fix a bug. Putting is fix into a system with 1 million lines can easily be more than 10 times harder than fixing a system with 100,000 lines.

Projects which set out to be BIG have far higher costs and lower productivity (per unit of deliverable) than small systems. (Capers Jones’ 2008 book contains some tables of productivity per function point which illustrate this. It is worth noting that the biggest systems are usually military and they have an atrocious productivity rate — an F35 or A400 anyone?)

Waiting longer — and probably writing more code — before you ask for feedback or user validation causes more problems than asking for it sooner when the product is smaller.

The examples could go on.

But the other thing is: working in the large increases risk.

Suppose 100ml of milk is off. If the 100ml is in one small carton then you have lost 1 pint of milk. If the 100ml is in a 4 pint carton you have lost 4 pints.

Suppose your developers write one bug a year which will slip through test and crash users’ machines. Suppose you know this, so in an effort to catch the bug you do more testing. In order to keep costs low on testing you need to test more software, so you do a bigger release with more changes — economies of scale thinking. That actually makes the testing harder but…

Suppose you do one release a year. That release blue screens the machine. The user now sees every release you do crashes their machine. 100% of your releases screw up.

If instead you release weekly, one release a year still crashes the machine but the user sees 51 releases a year which don’t. Less than 2% of your releases screw up.

Yes I’m talking about batch size. Software development works best in small batch sizes. (Don Reinertsen has some figures on batch size in The Principles of Product Development Flow which also support the diseconomies of scale argument.)

Ok, there are a few places where software development does exhibit economies of scale but on most occasions diseconomies of scale are the norm.

This happens because each time you add to software work the marginal cost per unit increases:

Add a fourth team member to a team of three and the communication paths increase from 3 to 6.

Add one feature to a release and you have one feature to test, add two features and you have 3 tests to run: two features to test plus the interaction between the two.

In part this is because human minds can only hold so much complexity. As the complexity increases (more changes, more code) our cognitive load increases, we slow down, we make mistakes, we take longer.

(Economies of scope and specialisation are also closely related to economies of scale and again on the whole, software development has diseconomies of scope (be more specific).)

However be careful: once the software is developed then economies of scale are rampant. The world switches. Software which has been built probably exhibits more economies of scale than any other product known to man. (In economic terms the marginal cost of producing the first instance are extremely high but the marginal costs of producing an identical copy (production) is so close to zero as to be zero, Ctrl-C Ctrl-V.)

What does this all mean?

Firstly you need to rewire your brain, almost everyone in the advanced world has been brought up with economies of scale since school. You need to start thinking diseconomies of scale.

Second, whenever faced with a problem where you feel the urge to go bigger run in the opposite direction, go smaller.

Third, take each and every opportunity to go small.

Four, get good at working in the small, optimise your processes, tools, approaches to do lots of small things rather than a few big things.

Fifth, and this is the killer: know that most people don’t get this at all. In fact it’s worse…

In any existing organization, particularly a large corporation, the majority of people who make decisions are out and out economies of scale believers. They expect that going big is cheaper than going small and they force this view on others — especially software technology people. (Hence Large companies trying to be Agile remind me of middle aged men buying sports cars.)

Many of these people got to where they are today because of economies of scale, many of these companies exist because of economies of scale; if they are good at economies of scale they are good at doing what they do.

But in the world of software development this mindset is a recipe for failure and under performance. The conflict between economies of scale thinking and diseconomies of scale working will create tension and conflict.

Originally posted in October 2015 and can also be found in Continuous Digital.

To be the first to know of updates and special offers subscribe — and get Continuous Digital for free.

Software has diseconomies of scale – not economies of scale Read More »

Fixing agile failure: collaboration over micro-management

Agile, Management / / / , ,

I’ve said it before, and I’m sure I’ll say it again: “the agile toolset can be used for good or evil”. Tools such as visual work tracking, work breakdown cards and stand-ups are great for helping teams take more control over their own work (self-organization). But in the hands of someone who doesn’t respect the team, or has micro-management tendencies, those same tools can be weaponised against the team.

Put it this way, what evil pointed-headed boss wouldn’t want the whole team standing up at 9am explaining why they should still be employed?

In fact, I’m starting to suspect that the toolset is being used more often as a team disabler than a team enabler. Why do I suspect this?

Reason 1: the increasing number of voices I hear criticising agile working. Look more closely and you find people don’t like being asked to do micro-tasks, or being asked to detail their work at a really fine-grained level, then having it pinned up on a visual board where their work, or lack of, is public.

Reason 2: someone I know well is pulling their hair out because at their office, far away from software development, one of the managers writes new task cards and inserts them on to the tracking board for others to do, hourly. Those on the receiving end know nothing about these cards until they appear with their name on them.

I think this is another case of “we shape our tools and then our tools shape us.” Many of the electronic work management tools originally built for agile are being marketed and deployed more widely now. The managers buying these tools don’t appreciate the philosophy behind agile and see these tools as simply work assignment and tracking mechanisms. Not only do such people not understand how agile meant these tools to be used they don’t even know the word agile or have a very superficial understanding.

When work happens like this I’m not surprised that workers are upset and demoralised. It isn’t meant to be this way. If I was told this was the way we should work, and then told it was called “agile” I would hate agile too.

So whats missing? How do we fix this?

First, simply looking at small tasks is wrong: there needs to be a sense of the bigger thing. Understand the overall objective and the you might come up with a different set of tasks.

Traditionally in agile we want lots of small work items because a) detailed break down shows we understand what needs to be done, b) creating a breakdown with others harnesses many people’s thinking while building shared understanding, c) we can see work flowing through the system and when it gets stuck collectively help.

So having lots of small work items is a good thing, except, when the bigger thing they are building towards is missing and…

Second, it is essential teams members are involved with creating the work items. Having one superior brain create all the small work items for others to do (and then assign them out) might be efficient in terms of creating all the small work items but it undermines collaboration, it demotivates workers and, worst of all, misses the opportunity to bring many minds to bear on the problem and solution.

The third thing which cuts through both of these is simple collaboration. When workers are given small work items, and not given a say in what those items are, then collaboration is undermined. When all workers are involved in designing the work, and understanding the bigger goals, then everyone is enrolled and collaboration is powerful.

Fixing this is relatively easy but it means making time to do it: get everyone together, talk about the goals for the next period (day, week, sprint, whatever) and collectively decide what needs doing and share these work items. Call it a planning meeting.

The problem is: such a meeting takes time, it might also require you to physically get people together. The payback is that your workers will be more motivated, they will understand the work better and are ready to work, they will be primed to collaborate and ready to help unblock one another. It is another case of a taking time upfront to make later work better.

Fixing agile failure: collaboration over micro-management Read More »

Reworking the risk equation

Business, Management / / /

It is hard to argue with the old project management equation:

Risk exposure = Risk impact x Risk probability

Risk impact is often take to be money, say $1,000,000 and probability a percentage, say 50%. So exposure is also in money, $500,000. There is something rather obvious about it. So, while I’ve long disliked the equation I’ve not taken it to task, I never quite put my finger on it, or perhaps, I couldn’t really articular a winning argument.

That changed last week when I took “Papa” Chris Matt’s dog for a walk, or rather, he walked the dog I just tagged along and discussed the state of the world, and the thing we call agile.

Chris has a better equation:

Risk exposure = Money invested x Time to payback

Probability is out, after all, if something fails then it all fails. You were never going to loose $500,000, although you might loose the whole million. Probability is something of a red herring.

Chris’ insight here is that there is risk until the moment when you start seeing payback, i.e. until the investment starts pay is proven. In retrospect I should have realised this before. I read How Big Things Get Done a few months back and Bent Flyberg has the same idea. He points out that the time of biggest risk starts when the big money gets spent and continues until the project is delivered. Flyberg uses this logic to argue that long, detailed, upfront planning is good – because it is cheap and allows problems to be identified and avoided). Once you start the real work then do it as fast as possible; keep moving fast even if it costs you more because until the window is closed everything is as risk. (A highly recommended book by the way.)

The lesson many people take from this is: spend along time in planning and make sure you are certain about what you are doing before you spend the big money. I suggest a more important lesson is: once you start spending the money get to the point of return as fast as possible. Slowing spending does not reduce risk or save money, it increases risk and reduces benefit.

I’ve pointed out before that one way to increase the return on investment (significantly) is to make sure work starts paying back sooner. For example, make sure that the team deliver something every month (which itself reduces risk because something – anything! – is actually being delivered) and have that thing earn some revenue even if it is less than the full amount you want. When you do return on investment calculations correctly (i.e. account for time) then bringing forward the point where some (even a little) money is returned causes ROI calculations to jump.

Although it might seem too good to be true, the same logic will reduce risk. Both on paper and in practice.

Everyone who has ever argued for measuring and reducing cycle time will be happy right now. But Chris has a second point: today’s endeavours often involve multiple teams, and until all those teams deliver there is no return.

Think of it like ships in a convoy: they move at the speed of the slowest ship. None of the ships arrive until they all do, until they do they are all at risk, and since they are moving slow there is more risk. It also means, that reducing risk, speeding delivery, increasing ROI is going to depend on speeding up the slowest ship. If that sounds familiar its because it is what theory of constrains says: the system is constrained by its most significant bottleneck.

Reworking the risk equation Read More »