Books of Note

Practical Common
LispThe best intro to start your journey. Excellent coverage of CLOS.

ANSI Common
LispAnother great starting point with a different focus.

Paradigms of Artificial Intelligence
ProgrammingA superb set of Lisp examples. Not just for the AI crowd.

Sunday, January 09, 2005

Keyword parameters in macro expansions 

I was hacking a macro last night (some work on state machines again) and discovered a subtle point that I hadn't realized before. It seems that the default values of keyword arguments are evaluated at the time a macro is expanded, rather than remaining unevaluated like keyword parameters in the macro call. Let's give an example here:

First, let's look at a simple macro:

(defmacro foo1 (x) `(+ ,x 1))

From the REPL, we can see what this expands to with various arguments using MACROEXPAND.

CL-USER> (macroexpand '(foo1 10))
(+ 10 1)
T
CL-USER> (macroexpand '(foo1 (+ 1 9)))
(+ (+ 1 9) 1)
T

Notice that when we call the macro with a complex form as the argument, the macro doesn't evaluate the argument. Rather, it simply sets X to "(+ 1 9)" and that is then inserted into the expansion at the right place. The fact that forms are not evaluated before the macro runs allows us to create some complex macros with interesting behavior. See AND and OR, for example, which evaluate their arguments one-by-one, as needed to calculate the value of the entire expression. Once AND has detected a single false value or OR has detected a single true value, the rest of the parameters are left unevaluated, resulting in the "short-circuit evaluation" for these expressions that we're all used to (in both Lisp and C).

Now, things get interesting when we have a keyword parameter in a macro:

(defmacro foo2 (&key (x (+ 1 9))) `(+ ,x 1))

In this case, we're telling the macro expander that x is an optional parameter, but if it's not there, we want it's value to be "(+ 1 9)". Lets see how this expands with a couple of cases:

CL-USER> (macroexpand '(foo2))
(+ 10 1)
T
CL-USER> (macroexpand '(foo2 :x (+ 1 9)))
(+ (+ 1 9) 1)
T

Okay, so what is going on here?? In the first case, the macro expands with the default value for X, but in this case it looks like the expansion has already evaluated the "(+ 1 9)" that we specified for the default. In the second case, we provide the value and the macro does not evaluate X the argument but binds X to the form "(+ 1 9)".

Now, in this simple case, it really isn't a big deal. The compiler and optimizer should crunch all this down to the same value. The case that bit me last night is when I wanted to specify a parameter with a default functional value. For instance:

(defmacro test-it (x y &key (test #'eql)) `(funcall ,test ,x ,y))

So this simple macro allows you to compare two values. If the user doesn't specify a value with the TEST keyword parameter, it defaults to EQL. Let's see how this expands:

CL-USER> (macroexpand '(test-it 'a 'a :test #'eql))
(FUNCALL #'EQL 'A 'A)
T
CL-USER> (macroexpand '(test-it "a" "a" :test #'string=))
(FUNCALL #'STRING= "a" "a")
T

So far, so good. Just what we expect. Now, let's just go with the default value:

CL-USER> (macroexpand '(test-it 'a 'a))
(FUNCALL #<FUNCTION "top level local call EQL" {1076EAD}> 'A 'A)
T

WHOA! What's going on here. Well, as we saw previously, the default value of TEST is being evaluated before the macro expansion takes place. In this case, the expression "#'EQL", which is shorthand for "(FUNCTION EQL)", is being evaluated. This produces the actual function object associated with EQL, which is unprintable. Thus, when the macro expands, this gets stuffed into the middle of the resulting expression. Now, it turns out that in the SBCL REPL, this actually works as you might expect:

CL-USER> (test-it 'a 'a)
T
CL-USER> (test-it 'a 'b)
NIL

We'll run into problems, however, when we try to compile a file that uses this macro, with COMPILE-FILE, for instance. In this case, SBCL doesn't know how to serialize the raw function reference into the resulting FASL file.

So, how do we fix this? Well, the lesson is that we want to prevent evaluation of default values in this case. We can do that with a quick QUOTing of the value:

(defmacro test-it (x y &key (test '#'eql)) `(funcall ,test ,x ,y))

Notice the extra single-quote in the "'#'EQL" form. This results in:

CL-USER> (macroexpand '(test-it 'a 'a :test #'eql))
(FUNCALL #'EQL 'A 'A)
T
CL-USER> (macroexpand '(test-it "a" "a" :test #'string=))
(FUNCALL #'STRING= "a" "a")
T
CL-USER> (macroexpand '(test-it 'a 'a))
(FUNCALL #'EQL 'A 'A)
T

Now, everything works as expected. So, note to self, remember to quote default optional and keyword parameters in macro definitions when we don't want them to be evaluated.


Comments:


That's actually pretty disturbing. Thanks for pointing it out!

--Randall Randall
 


how about this...

CL-USER> (defmacro foo2 (&key (x '(+ 1 9))) `(+ ,x 1))
FOO2
CL-USER> (macroexpand '(foo2))
(+ (+ 1 9) 1)
T
 


(Sorry, indentation isn't working - this code will look ugly.)

Remember that a macro is really just generating a regular function.

So this:

(defmacro foo2 (&key (x (+ 1 9)))
`(+ ,x 1))

Turns into something like this:

(defun foo2-macro-expander (whole-expression environment)
(declare (ignore environment))
(destructuring-bind (&key (x (+ 1 9))) whole-expression
`(+ ,x 1)))

Now if you see how the macro expression is passed into the function there and look at the behavior of DESTRUCTURING-BIND it might help you understand why the default arguments get evaluated.

Besides, if they too were unevaluated, this would not work:

(defmacro foo3 (&key x (y x))
`(+ ,x ,y))

(macroexpand '(foo3 :x (+ 1 9)))
=> (+ (+ 1 9) (+ 1 9))
(macroexpand '(foo3 :x (+ 1 9) :y something-else))
=> (+ (+ 1 9) SOMETHING-ELSE)

-bcd
 

Post a Comment


Links to this post:

Create a Link

This page is powered by Blogger. Isn't yours?