Small modular parts

The key to habitability?

Our last episode was with David Nolen. We talk about his development process, his origin, and his philosophy. The next episode is on Tuesday, April 22 with special guest Fogus. Please watch us live so you can ask questions.

I have finally released the new version of Introduction to Clojure, my flagship module in Beginner Clojure, my signature video course. This update is long overdue, but it makes up for its tardiness with fully updated content, modernized for current idioms and better teaching.

If you have the previous edition, you can already find the new edition in your dashboard. You get the upgrade for free as my thanks for being a part of this crazy journey.

If you buy Beginner Clojure now, you’ll also get the new version. Because it’s such a major upgrade, I’m going to raise the prices soon. If you want it, now is the time to buy. It will never be this cheap again.

---

Small modular parts

I’ve been seeping in the rich conceptual stews of Patterns of Software. In it, Richard Gabriel explores how the ideas of Christopher Alexander apply to software engineering (long before the GoF Design Patterns book). One of the early ideas in the book is that of habitability, the characteristic of a building to support human life. Architecture needs to provide the kinds of spaces humans need, and also be adaptable to changing life circumstances. A house must support time together, time alone, and time alone together (sharing a space but doing different things). But it also must allow adding an extra bedroom as your family grows.

Habitable software is analogous. Programmers live in the code. They must feel comfortable navigating around, finding what they need, and making changes as requirements change. Christopher Alexander says that it is impossible to create truly living structures out of modular parts. They simply don’t adapt enough to the circumstances.

However, we know that’s not entirely true. Bricks are modular parts, and many of the living examples he gives are buildings made of bricks. It must be that the modules need to be small enough to permit habitability. You can’t adjust the size of a wall if the wall is prefabricated. But you can adjust the size of the wall if the bricks are prefabricated to a resolution that is just right.

This is true in software as well. Large modules are not as reusable as small ones. Take classical Java. I think the language gets the size of the abstractions wrong. The affordances of the language are the for/if/… statements, arithmetic expressions, and method calls, plus a way to compose those up into a new class. It goes from the lowest level (basically C) to a very high level, with very little in-between.

Contrast that with Clojure, which gives you many general-purpose abstractions at a higher level than Java (map/filter/reduce, first-class functions, generic data structures), and then almost nothing above it. Just the humble function to parameterize and name a thing. Lambda calculus (basically first-class functions) goes a long way. Java’s methods and classes give you a way to build procedural abstractions over data storage and algorithms, but the language offers torturous facilities for control flow abstractions. First-class functions can abstract control flow. I think Clojure got the level right.

Except maybe Clojure’s standard library overdoes it. I’m a big fan of map/filter/reduce. You can do a lot with them. But then there are others. For instance, there’s keep, which is like `map` but it rejects `nil`s. And there’s `remove`, which is the opposite of `filter`. Any call to keep could be rewritten:

(keep {:a 1 :b 2 :c 3} [:a :b :c :d])

(filter some? (map {:a 1 :b 2 :c 3} [:a :b :c :d]))

Those two are equivalent. `remove` can also be rewritten:

(remove #{:a :b :c} [:a :b :c :d])

(filter (comp #{:a :b :c}) [:a :b :c :d])

I do use keep and remove sometimes, when I think about them. But how much do they really add? Is the cost of learning these worth it? How often do you have to switch it back to map and filter anyway to make the change you want?

Here’s what I think: keep is just slightly too big. It’s a modular part that does just a tad too much. map is like a standard brick. keep is like an L-shaped brick that’s only useful at the end of a wall or on a corner. Useful but not that useful, and certainly not necessary. The same is true of remove. It’s not useful enough.

I think Clojure did a remarkably good job of finding the right size of module. They feel human-scale, ready for composition in an understandable way. It makes programs of medium to large size feel more habitable. I see this about what little Smalltalk code I’ve read: Smalltalk’s classes are small, highly general modular units, like Point and Rectangle, not UserPictureDrawingManager.

One aspect of habitability is maintainability—the de moda holy grail of software design. Back in 1996, when Patterns of Software was published, Gabriel felt the need to argue against efficiency as the reason for software design. Somewhere between efficiency’s reign and today’s maintainability, code size and then complexity ruled.

Long-time readers may guess where I’m going: These characteristics all focus on the code. Abstraction gets talked about in terms of its (excuse me) abstract qualities. An abstraction is too big or small, too high- or low-level, too shallow or deep. At best, we’re talking about something measurable in the code, at worst, some mental structures only in the mind of the guru designer who talks about it.

I want to posit domain fit as a better measure—one that leads to habitability—and that is also objective. Domain fit asks: “How good is the mapping between what your code represents and the meanings available in the domain?” That mapping goes both ways. We ask both “How easily can I express a domain situation in the code?” and “How easily does the code express the domain situation it represents?” Fit covers both directions of expressivity.

I believe that domain misfit causes the most difficulties for code habitability. If your code doesn’t fit well with the domain, you’ll need many workarounds. Using reusable modules is only a problem because they don’t adapt well to the needs of your domain—not because they’re too big. It just so happens that bigger modules are harder to adapt. It’s not that a wall module is bad, per se, just that it’s almost never exactly the right size, and so you make compromises.

It’s not that the components in Clojure are the right size. It’s that Clojure’s domain—data-oriented programming—is the right size for many problems. It allows you, the programmer, to compose a solution out of parts—like bricks in a wall. And Clojure’s code fits the domain very well. Tangentially: It makes me wonder what the domain of Java is. I guess what I’m saying is that using a vector graphics API to do raster graphics is going to feel uninhabitable. But you can’t say it’s because vector graphics is a bigger abstraction than raster. It’s more about having the right model.

Now, Alexander might disagree that a pre-fab wall of exactly the right size is okay. He believes that there’s something in the handmadeness of things, too. It’s not just that the wall is the wrong or right size. Even if it were perfect, the perfection itself doesn’t lend itself to beauty. Geometrically precisely laid tiles cross some threshold where you don’t feel comfortable anymore. Ragged symmetry is better. We want bricks but they shouldn’t be platonic prisms.

So this is where I conclude and tease the next issue. I started this essay thinking size was important. I thought that Clojure got it right by finding a size of composable module that was a sweet spot. But now, I think it’s not about size. I don’t even know what size means anymore. It’s more about domain fit than ever. Perhaps I’m digging in deeper to my own biases (and please, I’m relying on you the reader to help me realize if I am). But this is what my reading is leading me to—the importance of building a domain model. When we talk about domain models, we often think of these jewel-like abstractions with perfect geometry. But this is a pipe dream. Our domains are too messy for that. In the next issue, I want to explore the dichotomy of geometric and organic adaptation.