Millions of Choices
Every piece of software is the incarnation of thousands, even millions, of choices. From the choice of programming language and the overall architecture down to the minutiae of the build scripts.
Complex, interconnected parts; subtle, long-term decisions; consequential but often unplanned interactions: all arise from the work of the software designer, the programmer, the architect.
As every programmer knows, they don’t simply spring into existence fully formed, but are shaped through experience and experimentation.
A single software craftsman might once have spent years on a significant program, perhaps a compiler or a database system, to produce a beautiful work of art: a complex piece of software born of unbounded intent.
That was no easy task, and eventually the grim reality of industry demanded that a full team be devoted to developing a software product.
When you bring in a team, you know that some decisions will be made locally. The chief architect of a software product, or the benevolent dictator for life of an open-source project, can devote only a limited amount of time to each minor choice.
Yet the choices are made, and they count. They become stratified and ultimately shape the product.
Who makes these choices?
The small decisions usually fall to the programmer writing a particular function or subsystem.
Sometimes these questions are escalated, especially when the best choice is unclear. Meetings are held. Discussions take place. Sometimes the chief architect is involved, but not always; that would not be feasible.
Take the Linux kernel, for instance: imagine the millions of choices that converged to produce the current source tree. Some of the most important ones can be traced back to Linus himself, while many others were made and on occasion reversed by brilliant kernel developers over the course of decades.
That’s why large-scale software is not merely assembled mechanically, like a puzzle whose final shape is known in advance: it is grown like a living organism or written like a novel.
You can’t design a software product in all its complexity in one go. Iteration is essential.1
This brings us back to the question with which we implicitly began: what is software?
And if software is the sedimentation of accumulated choices, what is the role of code in all this?
As a programming language nerd, of course I believe that code, i.e., the material instructions for the machine2, plays a central role, and that programming languages should be powerful and expressive. But I don’t buy the idea that writing code is the be-all and end-all of creating software.
Abelson and Sussman put it memorably: “Programs must be written for people to read, and only incidentally for machines to execute.”3
Now, writing code can be a chore for a working programmer or, indeed, a blessing for those naturally inclined to it. But every piece of code embodies a choice, even when that choice emerges through exploration.
Somehow, we tend to forget this. Somehow, even some of today’s best programmers talk as though “code” and “software system” were interchangeable.
But code is the body, not the soul.
The ghost in the machine, if there can be one, is the accumulated history of thoughts, decisions, failed experiments, brilliant ideas, fragmentary design notes on paper, concepts, discussions, conclusions, sketches, diagrams, and jottings that underlie a software system.
Software without the ideas conceived, the choices made, and the decisions reached is nothing.
It is not even code. It simply cannot exist.
Notes
The waterfall model was an illusion of sorts, a powerful illusion now returning after the waning of so-called Agile, but that is material for another post. ↩︎
The reader will forgive the simplification. ↩︎
See the preface to the first edition of the beautiful Structure and Interpretation of Computer Programs (SICP). ↩︎