concept Lisp in category dsl

This is an excerpt from Manning's book DSLs in Action. Login to get full access to this book.

Some languages offer compile-time metaprogramming. When you’re developing a DSL using this kind of language, you build metastructures as part of your source code, which get translated to the normal forms of the language before it runs. Lisp supports this technique through macros that get expanded to normal Lisp forms during the macro expansion phase (I discuss this in more detail in appendix B). For these languages, there’s an intermediate stage where you have source code translation before the byte code is generated for the virtual machine.

Lisp has the most sophisticated and complete support for compile-time metaprogramming, through its system of macros. Unlike C macros that have limited expressive power and that operate based on textual substitution, Lisp macros are powered by the full extensibility of the language.

When you have a macro call in your Lisp expression, the Lisp compiler doesn’t evaluate the arguments. Instead, it passes them unevaluated to the macro code. The macro code is processed and returns a new Lisp form that’s evaluated in place of the original macro form. This entire transformation of macro calls runs at compile time, generating code that consists of valid Lisp forms that are integrated with the AST of the main program. Look at figure B.3 for an illustration of the compile-time metaprogramming support that Lisp offers.

Figure B.3. Lisp uses macros to provide compile-time metaprogramming support

By using the QUOTE special form, data syntax can be embedded easily into the code syntax. Lisp macros are good examples of this philosophy. In fact, Lisp extends this paradigm of data as code and offers a full-blown template mechanism for writing metaprograms. In Common Lisp, we call this quasiquotation. In Clojure, the same feature is implemented through syntax quote, unquote, and splicing unquote. Here’s the definition of the macro defstruct in Clojure:

(defmacro defstruct
 [name & keys]
 `(def ~name (create-struct ~@keys)))

copy

The syntax quote character, represented by a backquote (`) causes the form that follows it to be interpreted as Lisp data and works like normal quoting. But within the syntax quoted form, the unquote character (~) turns off quoting and lets Lisp compute that form. A similar technique in Common Lisp, the quasiquotation, lets you write templates of data in which some parts of the data are fixed, but the others are computed. It’s pretty much a complete template sublanguage embedded within the syntax of Lisp.

Lisp is a language with minimal syntax. The Lisp parser is so simple because all it needs to parse are lists! Both the data and code syntax are represented uniformly using list structures. The Lisp macro body is also a list structure.

Lisp is a homoiconic language, and as you now know, it has powerful compile-time metaprogramming abilities. You might be wondering how this relates to the awesomeness of Lisp in implementing DSLs. The answer is simple: stick to the Lisp philosophy of representing your DSL as a list structure and organize repeatable constructs and patterns as macros. If you design your DSL this way, you won’t need a separate parser for your DSL. You can use the Lisp parser to parse something that isn’t even a valid Lisp form and extend the language with new syntax and semantics that speak the vocabulary of your domain. Look at figure B.4 for a visual of the Lisp-as-the-DSL metaphor.