Management

Software has diseconomies of scale – not economies of scale

“Practical men, who believe themselves to be quite exempt from any intellectual influence, are usually the slaves of some defunct economist.”

John Maynard Keynes

Most of you are not only familiar with the idea of economies of scale but you expect economies of scale even if you don’t know any ecoomics. Much of our market economy operates on the assumption that when you buy/spend more you get more per unit of spending.

At some stage in our education — even if you never studied economics or operational research — you have assimilated the idea that if Henry Ford builds 1,000,000 identical, black, cars and sells 1 million cars, than each car will cost less than if Henry Ford manufactures one car, sells one car, builds another very similar car, sells that car and thus continues. The net result is that Henry Ford produces cars more cheaply and sells more cars more cheaply, so buyers benefit.

(Indeed the idea and history of mass production and economies of scale are intertwined. Today I’m not discussing mass production, I’m talking Economies of Scale.)

Software Milk
Software is cheaper in small cartons, and less risky too

You expect that if you go to your local supermarket to buy milk then buying one large carton of milk — say 4 pints in one go — will be cheaper than buying 4 cartons of milk each holding one pint of milk.

Back in October 2015 I put this theory to a test in my local Sainsbury’s, here is the proof:

Collage of milk prices
Milk is cheaper in larger cartons
  • 1 pint of milk costs 49p (marginal cost of one more pint 49p)
  • 2 pints of milk cost 85p, or 42.5p per pint (marginal cost of one more pint 36p)
  • 4 pints of milk cost £1, or 25p per pint (marginal cost of one more pint 7.5p) (January 2024: the same quantity of milk in the same store now sells for £1.50)

(The UK is a proudly bi-measurement country. Countries like Canada and Switzerland teach their people to speak two languages. In the UK we teach our people to use two systems of measurement!)

So ingrained is this idea that when it supermarkets don’t charge less for buying more complaints are made (see The Guardian.)

Buying milk from Sainsbury’s isn’t just about the milk: Sainsbury’s needs the store, the store needs staffing, it needs products to sell, and they need to get me into the store. All that costs the same for one pint as for four. Thats why the marginal costs fall.

Economies of scale are often cited as the reason for corporate mergers: to extract concessions from suppliers, to manufacture more items for lower overall costs. Purchasing departments expect economies of scale.

But…. and this is a big BUT…. get ready….

Software development does not have economies of scale.

In all sorts of ways software development has diseconomies of scale.

If software was sold by the pint then a four pint carton of software would not just cost four times the price of a one pint carton it would cost far far more.

The diseconomies are all around us:

Small teams frequently outperform large teams, five people working as a tight team will be far more productive per person than a team of 50, or even 15. (The Quattro Pro development team in the early 1990s is probably the best documented example of this.)

The more lines of code a piece of software has the more difficult it is to add an enhancement or fix a bug. Putting is fix into a system with 1 million lines can easily be more than 10 times harder than fixing a system with 100,000 lines.

Projects which set out to be BIG have far higher costs and lower productivity (per unit of deliverable) than small systems. (Capers Jones’ 2008 book contains some tables of productivity per function point which illustrate this. It is worth noting that the biggest systems are usually military and they have an atrocious productivity rate — an F35 or A400 anyone?)

Waiting longer — and probably writing more code — before you ask for feedback or user validation causes more problems than asking for it sooner when the product is smaller.

The examples could go on.

But the other thing is: working in the large increases risk.

Suppose 100ml of milk is off. If the 100ml is in one small carton then you have lost 1 pint of milk. If the 100ml is in a 4 pint carton you have lost 4 pints.

Suppose your developers write one bug a year which will slip through test and crash users’ machines. Suppose you know this, so in an effort to catch the bug you do more testing. In order to keep costs low on testing you need to test more software, so you do a bigger release with more changes — economies of scale thinking. That actually makes the testing harder but…

Suppose you do one release a year. That release blue screens the machine. The user now sees every release you do crashes their machine. 100% of your releases screw up.

If instead you release weekly, one release a year still crashes the machine but the user sees 51 releases a year which don’t. Less than 2% of your releases screw up.

Yes I’m talking about batch size. Software development works best in small batch sizes. (Don Reinertsen has some figures on batch size in The Principles of Product Development Flow which also support the diseconomies of scale argument.)

Ok, there are a few places where software development does exhibit economies of scale but on most occasions diseconomies of scale are the norm.

This happens because each time you add to software work the marginal cost per unit increases:

Add a fourth team member to a team of three and the communication paths increase from 3 to 6.

Add one feature to a release and you have one feature to test, add two features and you have 3 tests to run: two features to test plus the interaction between the two.

In part this is because human minds can only hold so much complexity. As the complexity increases (more changes, more code) our cognitive load increases, we slow down, we make mistakes, we take longer.

(Economies of scope and specialisation are also closely related to economies of scale and again on the whole, software development has diseconomies of scope (be more specific).)

However be careful: once the software is developed then economies of scale are rampant. The world switches. Software which has been built probably exhibits more economies of scale than any other product known to man. (In economic terms the marginal cost of producing the first instance are extremely high but the marginal costs of producing an identical copy (production) is so close to zero as to be zero, Ctrl-C Ctrl-V.)

What does this all mean?

Firstly you need to rewire your brain, almost everyone in the advanced world has been brought up with economies of scale since school. You need to start thinking diseconomies of scale.

Second, whenever faced with a problem where you feel the urge to go bigger run in the opposite direction, go smaller.

Third, take each and every opportunity to go small.

Four, get good at working in the small, optimise your processes, tools, approaches to do lots of small things rather than a few big things.

Fifth, and this is the killer: know that most people don’t get this at all. In fact it’s worse…

In any existing organization, particularly a large corporation, the majority of people who make decisions are out and out economies of scale believers. They expect that going big is cheaper than going small and they force this view on others — especially software technology people. (Hence Large companies trying to be Agile remind me of middle aged men buying sports cars.)

Many of these people got to where they are today because of economies of scale, many of these companies exist because of economies of scale; if they are good at economies of scale they are good at doing what they do.

But in the world of software development this mindset is a recipe for failure and under performance. The conflict between economies of scale thinking and diseconomies of scale working will create tension and conflict.


Originally posted in October 2015 and can also be found in Continuous Digital.

To be the first to know of updates and special offers subscribe — and get Continuous Digital for free.

Software has diseconomies of scale – not economies of scale Read More »

Fixing agile failure: collaboration over micro-management

I’ve said it before, and I’m sure I’ll say it again: “the agile toolset can be used for good or evil”. Tools such as visual work tracking, work breakdown cards and stand-ups are great for helping teams take more control over their own work (self-organization). But in the hands of someone who doesn’t respect the team, or has micro-management tendencies, those same tools can be weaponised against the team.

Put it this way, what evil pointed-headed boss wouldn’t want the whole team standing up at 9am explaining why they should still be employed?

In fact, I’m starting to suspect that the toolset is being used more often as a team disabler than a team enabler. Why do I suspect this?

Reason 1: the increasing number of voices I hear criticising agile working. Look more closely and you find people don’t like being asked to do micro-tasks, or being asked to detail their work at a really fine-grained level, then having it pinned up on a visual board where their work, or lack of, is public.

Reason 2: someone I know well is pulling their hair out because at their office, far away from software development, one of the managers writes new task cards and inserts them on to the tracking board for others to do, hourly. Those on the receiving end know nothing about these cards until they appear with their name on them.

I think this is another case of “we shape our tools and then our tools shape us.” Many of the electronic work management tools originally built for agile are being marketed and deployed more widely now. The managers buying these tools don’t appreciate the philosophy behind agile and see these tools as simply work assignment and tracking mechanisms. Not only do such people not understand how agile meant these tools to be used they don’t even know the word agile or have a very superficial understanding.

When work happens like this I’m not surprised that workers are upset and demoralised. It isn’t meant to be this way. If I was told this was the way we should work, and then told it was called “agile” I would hate agile too.

So whats missing? How do we fix this?

First, simply looking at small tasks is wrong: there needs to be a sense of the bigger thing. Understand the overall objective and the you might come up with a different set of tasks.

Traditionally in agile we want lots of small work items because a) detailed break down shows we understand what needs to be done, b) creating a breakdown with others harnesses many people’s thinking while building shared understanding, c) we can see work flowing through the system and when it gets stuck collectively help.

So having lots of small work items is a good thing, except, when the bigger thing they are building towards is missing and…

Second, it is essential teams members are involved with creating the work items. Having one superior brain create all the small work items for others to do (and then assign them out) might be efficient in terms of creating all the small work items but it undermines collaboration, it demotivates workers and, worst of all, misses the opportunity to bring many minds to bear on the problem and solution.

The third thing which cuts through both of these is simple collaboration. When workers are given small work items, and not given a say in what those items are, then collaboration is undermined. When all workers are involved in designing the work, and understanding the bigger goals, then everyone is enrolled and collaboration is powerful.

Fixing this is relatively easy but it means making time to do it: get everyone together, talk about the goals for the next period (day, week, sprint, whatever) and collectively decide what needs doing and share these work items. Call it a planning meeting.

The problem is: such a meeting takes time, it might also require you to physically get people together. The payback is that your workers will be more motivated, they will understand the work better and are ready to work, they will be primed to collaborate and ready to help unblock one another. It is another case of a taking time upfront to make later work better.

Fixing agile failure: collaboration over micro-management Read More »

Verified by MonsterInsights