Technical

Debt help sign

Technical Debt: Engineers, you are not alone

I don’t read many books about software or technology these days, I tend to read outside the domain: economics, business and management – which after all is much of what I do in the technology world these days.

Recently I’ve been reading Winning now, winning later by David Cote and find really interesting. He hardly mentions software and never mentions agile but he is giving me a new perspective on technical issues, particularly technical debt (or technical liabilities as I prefer to call them). He talks about issues which have similar characteristics to tech debt but are completely different, legal issues for example. He sees these issues as conflicts between short-term thinking and long-term thinking.

Cote’s argument is that short term actions should support, not conflict, with long term goals. I agree. It might not be easy but if actions in the hear-and-now conflict with longer term goals then the chances of reaching those goals is diminished.

Cote is writing about his time as CEO of Honeywell – a US industrial conglomerate if you don’t know. Unusually Cote is honest about many of the dirty problems the company faced when he took over – a lot of business books glossy over such problems or talk about “challenges” or “opportunities”.

For example, Cote describes how Honeywell managers were chasing numbers and targets every quarter. They had no time for long term improvements because they were so busy “making the numbers”. One of his managers cut down a forest to sell as timber in order to make the end of quarter numbers. Sales people would give products away to new customers or offer large discounts at the end of the quarter. However customers knew this would happen so delayed orders until they were sweetened.

Making the short term numbers meant the company undercut itself so lost revenue next quarter. Management time was spent finding accounting tricks to “make the numbers” rather than improving the business. And since targets ratcheted up the next quarter was more difficult and required more diversions.

Other examples included legal cases Honeywell was fighting: spending time and money on lawyers, building up bad will with customers, politicians and local people. This in turn made it more difficult to get support when the company needed it.

I read these examples, and others, and I hear an engineer saying “Technical debt.” That is exactly what it is.

A software engineer who does a dirty job on a code change because they feel under pressure stores up problems for themselves and future engineers who need to do the next change. Which is exactly the same as a factory which dumps waste into a lake as a quick fix and then needs to clean up the lake later.

Actually economists have a term for this: externalities. These are the costs which are forced onto other payer, e.g. the factory saves money on waste disposal but the local government has to pay to clean the lake. I’ve long thought a lot of “technical debt” could be considered an externality because it pushed the cost onto someone else.

Today it is probably harder than ever to escape these cost – in code, in law, in financing – because there are more and more people out there looking for these things. Environmentalists look at waste in lakes, society expects companies to pay if they pollute and courts make companies pay. Smart investors will look closely at a firms accounts and discount the firm, or short them, if they see dubious practices.

This is Cote’s argument: in the short term it might save or generate money to fight legal cases (deny deny deny), sell off forests, discount sales and such, but, in the longer term – and the longer term might just be weeks – it will costs. And when it costs it will damage growth.

Doesn’t that sound just like technical debt/liabilities?

Naturally it is hard to see a company that chases numbers, pollutes and fights all legal claims caring about the quality of code. Engineers will have a hard job fighting for technical excellence there.

Cote argues, and I agree with him, that it doesn’t have to be this way. Acting responsibly and thinking about tomorrow – whether that is pollution, sales, accounting, code quality – will make it easier to grow later. Just because it is difficult to act in a manor that meets todays needs and make the world better for tomorrow does not allow use to ignore it: all of us need to think harder and find a solution that doesn’t mortgage tomorrow.

And sometimes the right answer is to accept the slow path, take it on the chin, pay the cost you’ve been avoiding. For Cote that mean settling legal cases and accepting some costs, for software teams that means doing the refactoring, rewriting a module or just saying No to more changes.

As I’ve said before: in software the long term comes along very soon.

As as I’ve blogged before there is no such thing as quick and dirty, only dirty and slow.

We might talk about debt/liabilities but really we are talking short-term v. long-term, a pay-day loan v. investment. Engineers have an unfortunate habit of talking about technical debt as a binary good v. evil debate with no other options.

Finding these less obvious paths which satisfy the short-term and long term is hard(er) but also offers the opportunity for higher, and longer, term improvement, something which is itself a competitive advantage.


Subscribe to this blog by e-mail and download Project Myopia ebook for Free

In the beginning there is architecture, well maybe

PlansiStock_000005364702Small-2019-07-26-14-59.jpg

I was on a panel last year when someone asked that old chestnut:

“Surely before you start coding anything you need to design an architecture?”

As regular readers might guess I took a very simple stand:

“No, you need to get something that works, you need to show you can address the problem in a way that delivers business benefit. You can retrofit architecture when you have shown that your thing is useful and valuable. Sure spend some time thinking about design and architecture but this should be measured in hours not days. As the system grows return to these discussions but keep them to hours not days. Embrace emergent design and refactoring.”

Another panellist disagreed:

“And…”

In 2018 you can tell someone is about to disagree with you because they don’t start “No” or “But” they start “And”

“Companies have databases, and data centres, and standards, and you need to spend some time thinking about them. You need to integrate to existing systems. Changing database schema’s, let alone database application servers is big work and you want to avoid it if you can.”

OK, I can’t remember his exact words but that is the tone I recall. I was told, I was wrong.

Despite all those very real concerns I stand by my position. OK, maybe I need to elaborate a bit so here goes…

The thing is, I have nothing against architecture. When you build a software system you will – before very long – have an architecture. It may not be a very good one but you will have one.

I encourage all developers to study software design and architecture. Coders make hundreds, even thousands, of decisions everyday when they are coding and there is an architectural angle to every decision. Coders need to be versed in good design practices so they make good decisions.

Nor do I have anything against guiding the architecture in such a way that it becomes better with time – indeed I think that is a necessity. Neither do I disagree with architectural conventions in a system or documentation, again they become more valuable with time.

What I’m not in favour of is: believing that you can set it from the start. I believe the more you try to predetermine the more time you will waste.

Architecture is nine-tenths what you have, and one-tenth what you think you have, or should have. Architecture exists in the minds and shared models of those who maintain the system as much as it does in the code.

There are some occasion when there is a completely blank sheet of paper to start with. New start-ups spring obviously, but even inside a large corporation there are occasions when it happens; say when the company wants to experiment with new technologies and approaches, or hedge options. There are two priorities here:

  • Be different, differentiate the solution from what already exists
  • Prove that this new solution delivers something of business benefit

So: pick the solutions that seem right to you now – thats “you” plural, you and the team you are working with.

Maybe spend a little time considering your options (Oracle? MySql? CouchBase? Mongo?) but don’t spend too long, hours not days. Accept that you will learn when you start using your chosen option and accept that you may change it, or that you may make a bad decision and be cursing you decision for ever more.

The thing is: all that time spent designing the architecture, researching options and making decisions is wasted if you do not deliver business benefit. If nobody uses it then having the perfect product, perfect architecture, is pointless.

Anyway, even if you spend months and agree the perfect architecture it will not survive. Once you start work you will find assumptions invalidated, you will find things don’t work as you expected, and you will find that the cheap code monkeys you hired to implement you wonderful design do things you didn’t expect.

Now consider the case where you start a new initiative in an existing environment. There are existing databases and systems to work with. Again you don’t need to spend a lot of time looking at options because you don’t have them. Instead you have constraints.

If the company standard is Oracle for databases you are going to use Oracle.
If you need to integrate with SalesForce then you need to integrate with SalesForce.

You might believe that CouchBase is a better solution than Oracle from the word go. In which case you can either:

  • Accept that right now Oracle is the standard and use it even if it is an inferior solution. In time if you demonstrate business benefit than you will be in a stronger position to change. Or you might find that even an inferior solution is good enough.

Or

  • Argue your case and hope to win: if you don’t win then you are back to the previous option, if you do then great.

What I would not do is: embark on a lengthy examination of all the possible options with the aim of deciding which is best. This takes time and means you will deliver later, which means cost-of-delay will reduce the value you deliver. Sure the great design might generate slightly more business benefit, or get sightly more performance from your development team but you are also delaying.

If you start with a good-enough solution you will learn. That learning may change your opinion, or it might confirm your initial opinion.

Remember: You are not building the ideal system. You are building the best system you can within constraints. The best system you can with the time and money available. In an existing environment many of those constraints are usually given before you start.

Unfortunately, us development folk tend to limit options thinking and exploring to the initial stages of a development effort – usually before any code is written. Once work is underway they don’t consider options often enough, we tend to jump at the first solution we think of.


Like this post? – Like to receive these posts by e-mail?

Subscribe to my newsletter & receive a free eBook “Xanpan: Team Centric Agile Software Development”

Check out my latest books – Continuous Digital and Project Myopia – and the Project Myopia audio edition

Principles of software development revisited

BeachSpainLite-2017-08-15-10-04.jpg

Summer… my traditional time for doing all that stuff that requires a chunk of time… erh… OK, “projects” – only they aren’t well planned and they only resemble projects in the rear-view mirror.

Why now? Why summer? – Because my clients are on holiday too so I’m quiet and not delivering much in the way of (agile) training or consulting. Because my family is away visiting more family. Thus I have a chunk of time.

This year’s projects include some programming – fixing up my own Twitter/LinkedIn/Facebook scheduler “CloudPoster” and some work on my Mimas conference review system in preparation for Agile on the Beach 2018 speaker submissions.

But the big project is a website rebuild.

You may have noticed this blog has moved from Blogger to a new WordPress site, and attentive readers will have noticed that my other sites, allankelly.net and SoftwareStrategy.co.uk have also folded in here. This has involved a lot of content moving, which also means I’ve been seeing articles I’d forgotten about.

In particular there is a group of “The Nature of Agile” articles from 2013 which were once intended to go into a book. Looking at them now I think they still stand, mostly. In particular I’m impressed by my 2013 Principles of Software Development.

I’ll let you can read the whole article but here are the headlines:

Software Development Principles

  1. Software Development exhibits Diseconomies of Scale
  2. Quality is essential – quality makes all things possible
  3. Software Development is not a production line

Agile Software Principles

  1. Highly adaptable over highly adapted
  2. Piecemeal Growth – Start small, get something that works, grow
  3. Need to think
  4. Need to unlearn
  5. Feedback: getting and using
  6. People closest to the work make decisions
  7. Know your schedule, fit work to the schedule not schedule to work
  8. Some Agile practices will fix you, others will help you see and help you fix yourself

The article then goes on to discuss The Iron Triangle and Conway’s Law.

I think that essay might be the first time I wrote about diseconomies of scale. Something else I learned when I moved all my content to this new site is that the Diseconomies of Scale blog entry is by far my most read blog entry ever.

I’m not sure if I’m surprised or shocked that now, four years later, these still look good to me. I really wouldn’t change them much. In fact these ideas are all part of my latest book Continuous Digital.

I’m even starting to wonder if I should role those Nature of Agile essays together in another book – but thats bad of me! Too many books….

Join Allan Kelly’s mailing list to get free updates on blog post, insights and events.