Software is grown, not built

Hello, world!

Here is a new blog, probably one among a few dozens created this very minute on the web. What will make it special, and I hope successful, is its strong focus on one single topic: the challenges and strategies of growing software over time.

But maybe this concept of "growing" code is not familiar to you. Let me try to explain.

Until recently, the prevailing approach for building software was the "fixed time / fixed price / fixed scope project" approach: define exactly what you want to build, create a blueprint for the structure of the code, and then start implementing that structure. Wait a few months, integrate all the developments, throw in a few testers, fix a few hundred bugs, and that's it! Your software is ready to deploy - and who knows, it’s maybe even usable.

The limits of this approach, from a project management perspective, have been largely documented by proponents of agile methodologies. What I would like to focus upon here is not the dynamics of the project, but rather those of the software construction itself.

For me, the main problem with this approach is the underlying vision of how software is built. In many such projects, software is seen as a product that is meant to be "finished", like a car or a watch. This way of seeing things is not completely wrong: you do need to have something finished for a v1.0 one time or another. The trouble is, this finished product metaphor breaks as soon as you realize that just after that v1.0 you will be asked for a v1.1, a v1.2, and later on a v2.0. Ten years later, you might even expect a v9.0 or a v12.0. This is not something you ask from a given car or a watch once it is created and shipped.

What I have observed in too many projects is that after a few releases the code base becomes a complete mess, and the team ends up spending most of its time just fighting the system. If you look at the code, you will find something akin to the car in this picture:

If you ask developers why their software has become a mess, they will tell you that the initial design did not take into account the needs of the many features that were added after that famous v1.0. They will tell you that they had no time to change the design, so they figured out solutions to add the features one way or the other (‘Where’s the shoehorn?”). After a few years of this treatment, new features become very expensive, and the work is so tedious that the best developers leave the project, making the situation ever worse. If this was not an in-house development project, users also start looking elsewhere.

There is another way of building software. A way in which the design evolves gracefully over the years. A way in which the product becomes more powerful release after release, with a clean design that allows developers to introduce powerful concepts that were simply not available or relevant when the application was still small. In this vision of software development, software is grown, always with an emphasis on the long-term consequences of the programming decisions that are made.

This vision of software development is not new at all. In 1975, Frederick Brooks was writing in his mythical book “The Mythical Man-Month”:

"The brain alone is intricate beyond mapping, powerful beyond imagination, rich in diversity, self-protecting and self-renewing. The secret is that it is grown, not built. So it must be with our software systems." (p.201)

With the advent of agile methodologies such as Extreme Programming or Scrum, this approach is becoming more and more popular these days. This is a good thing, since these approaches acknowledge the true nature of software. The bad news is that growing code is not that simple - even worse, it might kill the project even faster if the team lacks the corresponding skills and thinks that growing code is just an other name for quick and dirty programming.

The whole purpose of this blog is to discuss the various pitfalls that a team will have to face when growing its own software, along with the many strategies available to escape them. We’ll talk about OO design, patterns, code usability, Christopher Alexander and his theory of centers.

Interested? Stay tuned!