Lisp, Smalltalk, and the Power of Symmetry

Like many hackers, my first real programming language love was Lisp. Paul Graham, who inspired my own explorations of the language, is a particular advocate and has written quite a bit about Lisp and what makes it different from other programming languages. So what does make Lisp different? Why does Lisp continue to be one of the most powerful, flexible, and concise programming languages in existence, despite the fact that it was invented in 1958–making it the second-oldest high-level programming language in the world?

Paul Graham’s answer is macros:

Many languages have something called a macro. But Lisp macros are unique. […]

Lisp code is made out of Lisp data objects. And not in the trivial sense that the source files contain characters, and strings are one of the data types supported by the language. Lisp code, after it’s read by the parser, is made of data structures that you can traverse.

If you understand how compilers work, what’s really going on is not so much that Lisp has a strange syntax as that Lisp has no syntax. You write programs in the parse trees that get generated within the compiler when other languages are parsed. But these parse trees are fully accessible to your programs. You can write programs that manipulate them. In Lisp, these programs are called macros. They are programs that write programs.

There’s an interesting implication hidden in PG’s argument here. What he’s saying is that macros make Lisp powerful because they allow you to write programs that themselves write programs. But that’s what macros are, not what enables them–Lisp’s macros aren’t the cause of its power, they’re a symptom of it. What makes Lisp powerful isn’t its macros, it’s the fact that Lisp runs in the same context it’s written in. It’s s-expressions all the way down. This leads to an interesting possibility: could there be other ways to achieve similar power? Are macros the only possible way to make programs that write programs, or might there be others?

When I studied computer languages in college, the second language we looked at after Lisp was Smalltalk. I had been so impressed with the power and flexibility of Lisp that the first question out of my mouth that day was “does Smalltalk have macros?” My professor, after thinking it over for a second, replied that “Smalltalk doesn’t need macros.” Huh? Smalltalk doesn’t need macros? Why not? What does it have instead? It took me three years to figure it out: Smalltalk doesn’t need macros because it has classes instead.

It may seem strange to compare object-oriented classes to Lisp macros. Aren’t classes and class structures notoriously rigid, brittle, and prone to spaghetti code? Isn’t all that the opposite of a Lisp macro?

It’s true that the usual flavor of Object-Oriented Programming (“OOP,” as in “OOPs what was I thinking when I designed this class hierarchy”) has issues. Languages such as Java, C++, and even Python seem to think that “object-oriented” means mostly “classes and inheritance.” Which is sort of like saying that “driving” means mostly “buttons and pedals.” What most of these languages seem to miss is that Smalltalk’s class system, like Lisp’s macro system, is a symptom of the power already available in the language, not its cause. If it didn’t already have it, it wouldn’t really be that hard to add it in yourself.

Without Smalltalk’s underlying fundamentals, class inheritance becomes nothing more than a tool for code reuse. As such, it is neither the only nor necessarily the best such tool. The real power of Smalltalk’s object system, including its classes, is not inheritance–it’s reflection. Just as Lisp macros are powerful because they can operate on any Lisp code, including themselves, Smalltalk classes are powerful because they themselves are objects. Smalltalk, like Lisp, runs in the same context it’s written in. It’s objects all the way down.

Lisp is powerful because all Lisp programs are also Lisp data–everything that can be run can be written (and read) as an s-expression. Macros in Lisp are simply what happens when you apply this relationship inductively: they are code that manipulates data that is itself code.

Smalltalk is powerful because all Smalltalk data are programs–all information is embodied by running, living objects. Class programming in Smalltalk is simply data manipulating programs that are themselves data–it’s the inverse of the Lisp philosophy, but the end result is the same. It’s what enables the Smalltalk debugger to freeze, dissect, modify, and resume programs mid-execution. It’s what enables the browser to instantly find all objects that respond to a given message, or all superclasses and subclasses of a given object, or every running instance of a given class. It’s why the Smalltalk IDE isn’t just written in the language, it quite literally is the language.

Lisp, as PG mentions above, has effectively no syntax: because Lisp source code is expressed in the same form as running Lisp code, which is expressed in the same form as Lisp data, the three are interchangeable. Programs can write data which can be run as programs which can write data which can…

Smalltalk goes one further than Lisp: it’s not that Smalltalk’s source code has no syntax so much as Smalltalk has no source code. “Source code,” after all, just means “program that isn’t running,” and there is no such thing as a Smalltalk program that isn’t running! Because there is absolutely nothing in Smalltalk except “data that runs”–which is what an object is, after all–then there is no distinction in Smalltalk between data and programs. Data (objects) can write programs (objects) which write data (objects) which write programs (objects) which…

So Lisp macros aren’t the only way that programs can write programs, after all. The only thing that seems to be required for a language to allow this is a pervasive symmetry between programs and data. If a language allows programs and data to be treated as the same thing, then that language becomes easily and infinitely extensible–a language of the gods. S-expressions, it turns out, are not the only way to do this–you can do it with objects as well. I wonder if there are still more ways we haven’t yet tried?

Advertisements

12 Comments

Filed under Programming

12 responses to “Lisp, Smalltalk, and the Power of Symmetry

  1. fede

    Well, actually there ARE other ways. Lisp macros take structured data, but many other languages can work at the source level (lisp too, through readers). Look at SNOBOL (and Pearl), or REBOL. I think your point is mainly about homoiconicity. A property that I think is very well implemented in Io (which is inspired in both Lisp and Smalltalk, BTW)

  2. Alpheus

    Actually, I’ve noticed this pattern myself. I first started out with BASIC, and when my father learned I wanted to program computer games, he suggested C, but I stumbled onto C++. When I started my first year of college, I became interested in computer languages, and received all sorts of recommendations to look at different languages. In that first year, I learned something interesting: the languages that I found most interesting departed from C in dramatic ways, while those others suggested, such as Ada, Pascal, and Modula-2, were like C, except that they did stupid things like denote blocks with BEGIN and END instead of { and }.

    To this day, I have noticed the pattern continue. I appreciate C for its “simplicity”, but would prefer Python as the representative for Algol-based languages; but the most powerful languages continue to be those like Lisp/Scheme/Clojure, Smalltalk, Forth, and perhaps APL/J, Haskell/Erlang/OCaml, and Prolog. These programming languages also have a common thread: they are all (well, maybe not Haskell and OCaml, so much) very simple ideas that have far-reaching ramifications…

  3. Pingback: Lisp, Smalltalk, and the Power of Symmetry | ExtendTree

  4. See Forth, and concatenative programming languages for a completely different way of doing the same thing. In those systems, you build up the language as you go.

  5. Love your article, Malcolm. It’s one of the very best I’ve ever read on the subject of Lisp and Smalltalk.

    I’d like to point out a broken link in your article. The link to Alan Lovejoy’s “Getting The Message” no longer exists. However, it is still available at the Internet Archive: https://web.archive.org/web/20150908201317/http://www.smalltalk.org/articles/article_20100320_a3_Getting_The_Message.html

    In addition to the Internet Archive, it is also available on my blog: https://medium.com/smalltalk-talk/getting-the-message-667d77ff78d

    I love your article so much, I’d like to republish it on my blog, with your permission, of course. Please let me know your answer; I’ll check back here shortly. Thanks.

    • Thanks for your comment! Sorry it took me so long to approve–we had a baby last year and this blog has been sorely neglected. It looks like the broken link was fixed during that time…

      I am happy to grant permission, if you’d still like to republish, although for future reference all original content on this blog is licensed under Creative Commons, so no express permission is needed 🙂

  6. Pingback: Lisp, Smalltalk, and the Power of Symmetry | Copy Paste Programmers

  7. As a long time Smalltalk zealot and Lisp fan, I enjoyed the article. Amusing that the way to do objects (which define behaviour/process/algorithm and data structure separately) really well is to have an environment where they are “equal” and accessible to each other. To the question of whether there are other environments where this can be achieved: I agree with Rahul on Forth which is a very interesting stack based language. To this I would add (gasp!) assembler and macro-assembler. The advantage of Lisp and Smalltalk is that they do it in a way that allows it to be portable across architectures and to create DSLs appropriate to the problem at hand trivially. The Smalltalk keyword syntax is particularly adept here.

  8. Pingback: Why Use Smalltalk – Smalltalk In China

  9. Pingback: 为什么使用Smalltalk – Smalltalk In China

  10. Pingback: 面向对象编程中的「对称性」 – noiter

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s