Showcasing the progn! macro (part 1)

Manipulating symbol values is so common in emacs-lisp that you can almost always expect the bodies of functions and macros to be encased by let or let*, special forms that bind symbols to values. In practice, this adds what is essentially a default level of additional indentation to virtually any reasonably sized block of lisp code you write. And while this may not be such a big issue in isolation, in combination with the abundant body-enclosing macros in emacs-lisp--save-excursion, save-restriction, save-match-data, unwind-protect, condition-case and eval-after-load just to name a few–it is that much easier for nesting to get out of hand. Even worse is using let and let* in combination with closely related symbol-binding macros such as cl-labels, cl-flet, and cl-letf. Each of these constructs adds its own boilerplate as well as additional level of nesting making the resulting form progressively more unwiedly and cumbersome to read.

Consider the following form as an example. It does two very typical things: it binds symbols--a b and c–with let and it binds symbols--plus and minus–to functions via cl-flet. And yet before we have even reached the main body we are already at two levels of nesting.

(let ((a 1)
      (b 2)
      (c 3))
  (cl-flet ((plus (x y) (+ x y))
            (minus (x y) (- x y)))
    (minus (plus b c) (plus a b))))

To be clear, I am not demonizing nesting itself. Proper nesting actually makes code more readable by conveying information about what pertains to a specific form. And, objectively speaking, the nesting here is logical and consistent with the nesting of any typical body-enclosing lisp form. Nevertheless, the similar nature of the symbol-binding lisp constructs, the frequency of their use, as well as the verbosity that arises from using them in tandem presaged the need for an abstraction. Accordingly, I devised progn!.

To see how it works consider the initial example below re-written using progn!. Even with a cursory glance it is apparent how much cleaner and less cluttered the form is compared to its counterpart.

(progn!
  (set! a 1)
  (set! b 2)
  (set! c 3)
  (flet! plus (x y) (+ x y))
  (flet! minus (x y) (- x y))
  (minus (plus b c) (plus a b)))

Among the first things you will notice is the appearance new macros: set! and flet!. The former is a wrapper around setq and the latter just ignores its arguments and expands to nil. Primarily, these macros–especially for flet! which otherwise does nothing–is to indicate to progn! where in the body it should either record information or modify the body itself. The idea of using certain lisp patterns to indicate some action is not a novel one: it was inspired by loopy an iteration macro that in turn was inspired by cl-loop.

(defmacro set! (var value)
  `(setq ,var ,value))

(defmacro flet! (name args &rest body)
  "Define a local function definition with `cl-flet'.
NAME, ARGS and BODY are the same as in `defun'.
Must be used in `progn!'."
  (declare (indent defun))
  (ignore name args body))

During macro-expansion, progn! loops¹through its body detecting forms that match the patterns (set! SYMBOL VALUE) and (flet! NAME . ARGUMENTS). When it finds the former it collects SYMBOL into a list of symbols to be implicitly let-bound. When it finds the latter (shown below) it replaces it as well as subsequent forms, REST, with (cl-flet ((NAME . ARGUMENTS)) REST). After finishing going through the body progn! let-binds the symbols it collected from set! patterns around the resulting body. The result is the following macro-expansion² which is virtually equivalent to the initial example.

(let ((a nil)
      (b nil)
      (c nil))
  (set! a 1)
  (set! b 2)
  (set! c 3)
  (cl-flet ((plus (x y) (+ x y)))
    (cl-flet ((minus (x y) (- x y)))
      (minus (plus b c) (plus a b)))))

It should be noted that abstracting the code in this way does come with a drawback. Namely, you lose out on some of the precision and flexibility you have to specify the scope of forms. In the following example for instance, the form (+ 1 1) is in the scope of let but not cl-flet whereas (minus (plus b c) (plus a b)) is in both. With progn! this is impossible. However, with over a year of using progn!, I have not found a case where I actually need this level of control.

(let ((a 1)
      (b 2)
      (c 3))
  (+ 1 1)
  (cl-flet ((plus (x y) (+ x y))
            (minus (x y) (- x y)))
    (minus (plus b c) (plus a b))))

By working in combination with these indicator macros progn! facilitates writing more direct and concise code. Previously, I felt discouraged from using cl-flet, cl-labels and cl-letf despite their palpable usefulness due to their unruly verbosity; now I find myself using such constructs all the time.