Lisp as an OO language

Deep into Alan Kay's cryptic definition of OOP

Apologies! Last week’s issue was misconfigured. It only went out to paid subscribers. You can still read it online.

Nobody cares about software design

Please consider buying Grokking Simplicity for your team. It makes a great holiday gift. Nothing says “I appreciate you” than a thoughtful book on functional programming with some subtle and slightly dark humor.

---

Lisp as an OO Language

It’s natural when we want to understand a term to seek a definition from the person who coined the term. When people want to understand object-oriented programming, they turn to Alan Kay. Here’s what he said once about what he meant by the term:

OOP to me means only messaging, local retention and protection and hiding of state-process, and extreme late-binding of all things. (link)

Now, this doesn’t sound too far off from what you expect from Java, Ruby, C++, etc. If you squint, “messaging” is methods, “local retention and protection and hiding of state-process” is encapsulation, and “extreme late-binding of all things” is some kind of dynamic mumbo-jumbo that Smalltalk had that we might compile away these days, perhaps combined with polymorphism.

He continues:

It can be done in Smalltalk …

Understood, Alan.

and in LISP…

Wait, what? I thought Lisp was functional.

There are possibly other systems in which this is possible, but I'm not aware of them.

Wait! When was this written? In 2003? What about Java? And Ruby? And C++? And Objective-C? What???

But he also said this:

I made up the term 'object-oriented', and I can tell you I didn't have C++ in mind. (link)

Ok. Now we should all be confused—unless we’ve fallen down this rabbit hole before and come out the other side. If this is your first time, welcome.

What we shouldn’t do is just say “huh, that’s weird” and continue to use Alan Kay’s description of OOP without further analysis (as is done in countless OO talks, including this recent one). It’s clear to me, after many years of studying Alan Kay’s output and programming languages that Alan Kay’s conception of OOP is a completely different thing from our current, mainstream use of the term. It’s not just a little different, like a popular conception evolving from an original conception. No, they are totally different things that happen to use the same term:

Someone asked me what I was doing, and without thinking, I said “object-oriented programming”. (A very bad choice as it turned out, for many reasons.) (link)

The rabbit hole is deep, so let’s jump now. Here’s the weird stuff so far:

• C++ is not object oriented

• Lisp is object oriented

• messaging

• state-process

• late-binding

When I first encountered Smalltalk and the rest of Alan Kay’s work, I believed for years that the real difference between Smalltalk and Java (I’ll pick on Java as a representative of OOP languages) was that Smalltalk was a live system. That would explain why Smalltalk and Lisp were both OO, and it would also explain the late-binding stuff. Late binding means you can redefine things on the fly, which is essential for live coding. I believed that for years.

But it eventually didn’t sit right with me. I started seeing other things he was talking about, particularly about writing software at internet scale (“every object should have an IP address”) and trying to incorporate lessons from the internet into programming. I started coming to a much different conception, which I think brings all of these “weird” things together.

Here’s my attempt to explain it all in one paragraph. I’m sorry if I don’t have quotes for every little piece. I’m happy to discuss any parts of this in the comments. Please ask your questions:

Object-Oriented Programming (OOP)—as originally conceived by Kay—was an attempt to scale software by conceiving of it as a complex adaptive system of interconnected, adaptive, and self-regulating meaning-making machines.

Let’s go through Kay’s original formulation with this in mind. Here it is again:

OOP to me means only messaging, local retention and protection and hiding of state-process, and extreme late-binding of all things. (link)

Messaging means that the sender has to write down in some format what it would like the request from the receiver. This is very different from a method call in Java, which is essentially a subroutine that runs in the namespace of the callee. By writing it down, you are allowing the receiver to interpret it (i.e., make meaning). That’s where Lisp comes in, by the way. The Lisp interpreter is an interesting example of a meaning-making system.

Local retention and protection and hiding of state-process is quite a mouthful. But it is essential to complex adaptive systems. Notice that there are three verbs happening to the same object—state-process. What is “state-process”? I’ve thought long about it and my current understanding is that it is state changing over time. The object must maintain invariants in its own state. To do that it must protect it. The state should belong to it (retention). And it should be hidden because if other objects can see it, they will start to rely on it. This forces everything to go through that layer of interpretation. In other words, the object should have complete control of its own state.

Having control of its own state is vital to scaling complex adaptive systems. Sequential algorithms can easily reach a size that is difficult to understand. If you break up the work into a distributed system, now you have distributed systems problems. The object has to maintain its own state to guard against data integrity problems common in parallel systems. The object becomes a unit of atomic consistency. But distributed systems have other issues, like messages happening out of order, messages failing to arrive, nodes crashing, etc. You can’t centralize solving those issues. Each object must maintain the state for its own solutions, including retry strategies, idempotence enforcement, message reordering strategies, etc.

Finally, late binding of all things is a way to make the system adaptable yet never have to shut down. You’re not going to get it perfect the first time. We’re going to learn better strategies for computing. You want the system to keep running even as you’re replacing the parts. Late binding makes everything replaceable at runtime.

Well, that’s my current understanding. Please do leave some questions and try to pick it apart. It’s the ultimate compliment.

Rock on!
Eric