Code is the problem

At the Norwegian Smidig conference this year Dag Sjøberg held an interesting lightning talk (in norwegian): Viktigste faktorer for å redusere teknisk gjeld. Dag presented findings from an experiment into the effects of technical debt. After submitting tenders, 4 software houses had been asked to actually build the same system. These 4 systems were then put into production. Users were then randomly assigned to one of these systems each time they logged in. After a couple of years an independent expert was asked to review the code in each system and assess its maintainability. Finally, the same changes were implemented in all of the systems.

So, what were the results from this small experiment? It turns out that the expert was not that good at predicting which system would be easiest to maintain. Another surprising fact was that the level of technical debt in a system was also a poor predictor of maintainability. There was only one aspect than correlated very closely with the time spent implementing the changes: the amount of code in a system. The system with the smallest codebase required the least amount of time to implement the changes despite the fact that it was not very well written.

Now, you may object that a sample size of 4 and 1 maintenance iteration hardly makes for a scientific study. Unfortunately, this is about as good as it gets in computer “science”. This study does not prove that technical debt is irrelevant or that the size of the codebase is the only thing you should care about. Use the results with caution. For me, this study is one of many things that point towards an inverse relationship between the size of a system and its maintainability. The concept of competence debt is closely tied to the size of a system.

If the size of a system is important the question becomes: how do you keep it as small as possible while still being able to provide a lot of value? The answer is as obvious as it is sublime. The code should be succinct and a system should only do what it must be able to do. The hard part is achieving these goals… a post about that will follow

Niklas Björnerstedt

One obvious point I forgot to mention when this was posted is that refactoring out technical debt leads to a smaller codebase. So even if it did turn out to be the case that code quality never has an impact on maintainability the reduction in amount of code should have an impact.

Posted on 2013.11.22 at 13:57

Comments are off for this post.

2 Responses to Code is the problem

  1. allan says:

    This idea that less code leads to more maintainability keeps going around and around in my head.

    Do you have any references for this? – In English that is

    Assuming this is true then maybe there is something in all those Lines of Code counts. Maybe KLoc isn’t such a meaningless metric after all.

    It also begs the question: “Does that mean Ruby is more maintainable than Cobol?” – which is the implication because we can do a lot more in one line of Ruby (or another modern language) than older languages (like Cobol).

    And if true it would help resolve the “High Church C++” conundrum I posed nearly 14 years ago ago.

  2. smalltalk80 says:

    I found a number of references to Dag Sjøbergs work (in English) here: I have not had time to read them but from the titles a few of the papers look relevant.

    I do not believe in measuring lines of code. It measures the wrong things. The number of distinct programmatic entities is more relevant.

    There certainly is an argument to be made for using higher level languages when appropriate. I remember a story from the nineties. Two projects were working in parallel, one in C++ and one in Smalltalk. A manager first visited the C++ project. The developer’s discussions were about memory leaks and pointer problems. He then visited the Smalltalk project. The discussions there focused on things like: “What do you think the customer wants to happen in this situation?”.


About Niklas

Born in Sweden, grew up in New York, lives in Norway. Yes, I have problems with identity, but I do think that my background helps me see things from a different perspective.

Read more