Common Lisp For Mac

LispWorks for Macintosh is a full native implementation of ANSI Common Lisp for Mac OS X. There are separately-licensed 32-bit and 64-bit products. LispWorks (32-bit) for Macintosh supports Intel-based Macintosh computers and is available in five Editions. Clozure CL (often called CCL for short) is a free Common Lisp implementation with a long history. Some distinguishing features of the implementation include fast compilation speed, native threads, a precise, generational, compacting garbage collector, and a convenient foreign-function interface.

Now it's time to start writing your own macros. The standard macros Icovered in the previous chapter hint at some of the things you can dowith macros, but that's just the beginning. Common Lisp doesn'tsupport macros so every Lisp programmer can create their own variantsof standard control constructs any more than C supports functions soevery C programmer can write trivial variants of the functions in theC standard library. Macros are part of the language to allow you tocreate abstractions on top of the core language and standard librarythat move you closer toward being able to directly express the thingsyou want to express.

Perhaps the biggest barrier to a proper understanding of macros is,ironically, that they're so well integrated into the language. Inmany ways they seem like just a funny kind of function--they'rewritten in Lisp, they take arguments and return results, and theyallow you to abstract away distracting details. Yet despite thesemany similarities, macros operate at a different level than functionsand create a totally different kind of abstraction.

Once you understand the difference between macros and functions, thetight integration of macros in the language will be a huge benefit.But in the meantime, it's a frequent source of confusion for newLispers. The following story, while not true in a historical ortechnical sense, tries to alleviate the confusion by giving you a wayto think about how macros work.

The Story of Mac: A Just-So Story

Once upon a time, long ago, there was a company of Lisp programmers.It was so long ago, in fact, that Lisp had no macros. Anything thatcouldn't be defined with a function or done with a special operatorhad to be written in full every time, which was rather a drag.Unfortunately, the programmers in this company--thoughbrilliant--were also quite lazy. Often in the middle of theirprograms--when the tedium of writing a bunch of code got to be toomuch--they would instead write a note describing the code they neededto write at that place in the program. Even more unfortunately,because they were lazy, the programmers also hated to go back andactually write the code described by the notes. Soon the company hada big stack of programs that nobody could run because they were fullof notes about code that still needed to be written.

In desperation, the big bosses hired a junior programmer, Mac, whosejob was to find the notes, write the required code, and insert itinto the program in place of the notes. Mac never ran theprograms--they weren't done yet, of course, so he couldn't. But evenif they had been completed, Mac wouldn't have known what inputs tofeed them. So he just wrote his code based on the contents of thenotes and sent it back to the original programmer.

With Mac's help, all the programs were soon completed, and thecompany made a ton of money selling them--so much money that thecompany could double the size of its programming staff. But for somereason no one thought to hire anyone to help Mac; soon he was single-handedly assisting several dozen programmers. To avoid spending allhis time searching for notes in source code, Mac made a smallmodification to the compiler the programmers used. Thereafter,whenever the compiler hit a note, it would e-mail him the note andwait for him to e-mail back the replacement code. Unfortunately, evenwith this change, Mac had a hard time keeping up with theprogrammers. He worked as carefully as he could, but sometimes--especially when the notes weren't clear--he would make mistakes.

The programmers noticed, however, that the more precisely they wrotetheir notes, the more likely it was that Mac would send back correctcode. One day, one of the programmers, having a hard time describingin words the code he wanted, included in one of his notes a Lispprogram that would generate the code he wanted. That was fine by Mac;he just ran the program and sent the result to the compiler.

The next innovation came when a programmer put a note at the top ofone of his programs containing a function definition and a commentthat said, 'Mac, don't write any code here, but keep this functionfor later; I'm going to use it in some of my other notes.' Othernotes in the same program said things such as, 'Mac, replace thisnote with the result of running that other function with the symbolsx and y as arguments.'

This technique caught on so quickly that within a few days, mostprograms contained dozens of notes defining functions that were onlyused by code in other notes. To make it easy for Mac to pick out thenotes containing only definitions that didn't require any immediateresponse, the programmers tagged them with the standard preface:'Definition for Mac, Read Only.' This--as the programmers were stillquite lazy--was quickly shortened to 'DEF. MAC. R/O' and then'DEFMACRO.'

Pretty soon, there was no actual English left in the notes for Mac.All he did all day was read and respond to e-mails from the compilercontaining DEFMACRO notes and calls to the functions defined in theDEFMACROs. Since the Lisp programs in the notes did all the realwork, keeping up with the e-mails was no problem. Mac suddenly had alot of time on his hands and would sit in his office daydreamingabout white-sand beaches, clear blue ocean water, and drinks withlittle paper umbrellas in them.

Several months later the programmers realized nobody had seen Mac forquite some time. When they went to his office, they found a thinlayer of dust over everything, a desk littered with travel brochuresfor various tropical locations, and the computer off. But thecompiler still worked--how could it be? It turned out Mac had madeone last change to the compiler: instead of e-mailing notes to Mac,the compiler now saved the functions defined by DEFMACRO notes andran them when called for by the other notes. The programmers decidedthere was no reason to tell the big bosses Mac wasn't coming to theoffice anymore. So to this day, Mac draws a salary and from time totime sends the programmers a postcard from one tropical locale oranother.

Macro Expansion Time vs. Runtime

The key to understanding macros is to be quite clear about thedistinction between the code that generates code (macros) and thecode that eventually makes up the program (everything else). When youwrite macros, you're writing programs that will be used by thecompiler to generate the code that will then be compiled. Only afterall the macros have been fully expanded and the resulting codecompiled can the program actually be run. The time when macros run iscalled macro expansion time; this is distinct from runtime,when regular code, including the code generated by macros, runs.

It's important to keep this distinction firmly in mind because coderunning at macro expansion time runs in a very different environmentthan code running at runtime. Namely, at macro expansion time,there's no way to access the data that will exist at runtime. LikeMac, who couldn't run the programs he was working on because hedidn't know what the correct inputs were, code running at macroexpansion time can deal only with the data that's inherent in thesource code. For instance, suppose the following source code appearssomewhere in a program:

Normally you'd think of x as a variable that will hold theargument passed in a call to foo. But at macro expansion time,such as when the compiler is running the WHEN macro, the onlydata available is the source code. Since the program isn't runningyet, there's no call to foo and thus no value associated withx. Instead, the values the compiler passes to WHEN arethe Lisp lists representing the source code, namely, (> x 10)and (print 'big). Suppose that WHEN is defined, as yousaw in the previous chapter, with something like the following macro:

When the code in foo is compiled, the WHEN macro will berun with those two forms as arguments. The parameter conditionwill be bound to the form (> x 10), and the form (print'big) will be collected into a list that will become the value ofthe &restbody parameter. The backquote expression willthen generate this code:

by interpolating in the value of condition and splicing thevalue of body into the PROGN.

When Lisp is interpreted, rather than compiled, the distinctionbetween macro expansion time and runtime is less clear becausethey're temporally intertwined. Also, the language standard doesn'tspecify exactly how an interpreter must handle macros--it couldexpand all the macros in the form being interpreted and theninterpret the resulting code, or it could start right in oninterpreting the form and expand macros when it hits them. In eithercase, macros are always passed the unevaluated Lisp objectsrepresenting the subforms of the macro form, and the job of the macrois still to produce code that will do something rather than to doanything directly.

DEFMACRO

As you saw in Chapter 3, macros really are defined with DEFMACROforms, though it stands--of course--for DEFine MACRO, not Definitionfor Mac. The basic skeleton of a DEFMACRO is quite similar tothe skeleton of a DEFUN.

Like a function, a macro consists of a name, a parameter list, anoptional documentation string, and a body of Lispexpressions.¹However, as I just discussed, the job of a macro isn't to do anythingdirectly--its job is to generate code that will later do what youwant.

Macros can use the full power of Lisp to generate their expansion,which means in this chapter I can only scratch the surface of whatyou can do with macros. I can, however, describe a general processfor writing macros that works for all macros from the simplest to themost complex.

The job of a macro is to translate a macro form--in other words, aLisp form whose first element is the name of the macro--into codethat does a particular thing. Sometimes you write a macro startingwith the code you'd like to be able to write, that is, with anexample macro form. Other times you decide to write a macro afteryou've written the same pattern of code several times and realize youcan make your code clearer by abstracting the pattern.

Regardless of which end you start from, you need to figure out theother end before you can start writing a macro: you need to know bothwhere you're coming from and where you're going before you can hopeto write code to do it automatically. Thus, the first step of writinga macro is to write at least one example of a call to the macro andthe code into which that call should expand.

Once you have an example call and the desired expansion, you're readyfor the second step: writing the actual macro code. For simple macrosthis will be a trivial matter of writing a backquoted template withthe macro parameters plugged into the right places. Complex macroswill be significant programs in their own right, complete with helperfunctions and data structures.

After you've written code to translate the example call to theappropriate expansion, you need to make sure the abstraction themacro provides doesn't 'leak' details of its implementation. Leakymacro abstractions will work fine for certain arguments but notothers or will interact with code in the calling environment inundesirable ways. As it turns out, macros can leak in a small handfulof ways, all of which are easily avoided as long as you know to checkfor them. I'll discuss how in the section 'Plugging the Leaks.'

To sum up, the steps to writing a macro are as follows:

1. Write a sample call to the macro and the code it should expandinto, or vice versa.
2. Write code that generates the handwritten expansion from thearguments in the sample call.
3. Make sure the macro abstraction doesn't 'leak.'

A Sample Macro: do-primes

To see how this three-step process works, you'll write a macrodo-primes that provides a looping construct similar toDOTIMES and DOLIST except that instead of iterating overintegers or elements of a list, it iterates over successive primenumbers. This isn't meant to be an example of a particularly usefulmacro--it's just a vehicle for demonstrating the process.

First, you'll need two utility functions, one to test whether a givennumber is prime and another that returns the next prime numbergreater or equal to its argument. In both cases you can use a simple,but inefficient, brute-force approach.

Now you can write the macro. Following the procedure outlinedpreviously, you need at least one example of a call to the macro andthe code into which it should expand. Suppose you start with the ideathat you want to be able to write this:

to express a loop that executes the body once each for each primenumber greater or equal to 0 and less than or equal to 19, with thevariable p holding the prime number. It makes sense to modelthis macro on the form of the standard DOLIST and DOTIMESmacros; macros that follow the pattern of existing macros are easierto understand and use than macros that introduce gratuitously novelsyntax.

Without the do-primes macro, you could write such a loop withDO (and the two utility functions defined previously) like this:

Now you're ready to start writing the macro code that will translatefrom the former to the latter.

Macro Parameters

Since the arguments passed to a macro are Lisp objects representingthe source code of the macro call, the first step in any macro is toextract whatever parts of those objects are needed to compute theexpansion. For macros that simply interpolate their argumentsdirectly into a template, this step is trivial: simply defining theright parameters to hold the different arguments is sufficient.

But this approach, it seems, will not suffice for do-primes.The first argument to the do-primes call is a list containingthe name of the loop variable, p; the lower bound, 0;and the upper bound, 19. But if you look at the expansion, thelist as a whole doesn't appear in the expansion; the three elementare split up and put in different places.

You could define do-primes with two parameters, one to holdthe list and a &rest parameter to hold the body forms, and thentake apart the list by hand, something like this:

In a moment I'll explain how the body generates the correctexpansion; for now you can just note that the variables var,start, and end each hold a value, extracted fromvar-and-range, that's then interpolated into the backquoteexpression that generates do-primes's expansion.

However, you don't need to take apart var-and-range 'by hand'because macro parameter lists are what are called destructuringparameter lists. Destructuring, as the name suggests, involves takingapart a structure--in this case the list structure of the formspassed to a macro.

Within a destructuring parameter list, a simple parameter name can bereplaced with a nested parameter list. The parameters in the nestedparameter list will take their values from the elements of theexpression that would have been bound to the parameter the listreplaced. For instance, you can replace var-and-range with alist (var start end), and the three elements of the list willautomatically be destructured into those three parameters.

Another special feature of macro parameter lists is that you can use&body as a synonym for &rest. Semantically &body and&rest are equivalent, but many development environments will usethe presence of a &body parameter to modify how they indent usesof the macro--typically &body parameters are used to hold alist of forms that make up the body of the macro.

So you can streamline the definition of do-primes and give ahint to both human readers and your development tools about itsintended use by defining it like this:

Mac printer driver for hp 1020. In addition to being more concise, destructuring parameter lists alsogive you automatic error checking--with do-primes defined thisway, Lisp will be able to detect a call whose first argument isn't athree-element list and will give you a meaningful error message justas if you had called a function with too few or too many arguments.Also, in development environments such as SLIME that indicate whatarguments are expected as soon as you type the name of a function ormacro, if you use a destructuring parameter list, the environmentwill be able to tell you more specifically the syntax of the macrocall. With the original definition, SLIME would tell youdo-primes is called like this:

But with the new definition, it can tell you that a call should looklike this:

Destructuring parameter lists can contain &optional, &key,and &rest parameters and can contain nested destructuring lists.However, you don't need any of those options to writedo-primes.

Generating the Expansion

Because do-primes is a fairly simple macro, after you'vedestructured the arguments, all that's left is to interpolate theminto a template to get the expansion.

For simple macros like do-primes, the special backquote syntaxis perfect. To review, a backquoted expression is similar to a quotedexpression except you can 'unquote' particular subexpressions bypreceding them with a comma, possibly followed by an at (@) sign.Without an at sign, the comma causes the value of the subexpressionto be included as is. With an at sign, the value--which must be alist--is 'spliced' into the enclosing list.

Another useful way to think about the backquote syntax is as aparticularly concise way of writing code that generates lists. Thisway of thinking about it has the benefit of being pretty much exactlywhat's happening under the covers--when the reader reads a backquotedexpression, it translates it into code that generates the appropriatelist structure. For instance, `(,a b) might be read as(list a 'b). The language standard doesn't specify exactlywhat code the reader must produce as long as it generates the rightlist structure.

Table 8-1 shows some examples of backquoted expressions along withequivalent list-building code and the result you'd get if youevaluated either the backquoted expression or the equivalentcode.²

It's important to note that backquote is just a convenience. But it'sa big convenience. To appreciate how big, compare the backquotedversion of do-primes to the following version, which usesexplicit list-building code:

As you'll see in a moment, the current implementation ofdo-primes doesn't handle certain edge cases correctly. Butfirst you should verify that it at least works for the originalexample. You can test it in two ways. You can test it indirectly bysimply using it--presumably, if the resulting behavior is correct,the expansion is correct. For instance, you can type the originalexample's use of do-primes to the REPL and see that it indeedprints the right series of prime numbers.

Or you can check the macro directly by looking at the expansion of aparticular call. The function MACROEXPAND-1 takes any Lispexpression as an argument and returns the result of doing one levelof macro expansion.³ Because MACROEXPAND-1 is afunction, to pass it a literal macro form you must quote it. You canuse it to see the expansion of the previous call.⁴

Or, more conveniently, in SLIME you can check a macro's expansion byplacing the cursor on the opening parenthesis of a macro form in yoursource code and typing C-c RET to invoke the Emacs functionslime-macroexpand-1, which will pass the macro form toMACROEXPAND-1 and 'pretty print' the result in a temporarybuffer.

However you get to it, you can see that the result of macro expansionis the same as the original handwritten expansion, so it seems thatdo-primes works.

Plugging the Leaks

In his essay 'The Law of Leaky Abstractions,' Joel Spolsky coined theterm leaky abstraction to describe an abstraction that 'leaks'details it's supposed to be abstracting away. Since writing a macrois a way of creating an abstraction, you need to make sure yourmacros don't leak needlessly.⁵

As it turns out, a macro can leak details of its inner workings inthree ways. Luckily, it's pretty easy to tell whether a given macrosuffers from any of those leaks and to fix them.

The current definition suffers from one of the three possible macroleaks: namely, it evaluates the end subform too many times.Suppose you were to call do-primes with, instead of a literalnumber such as 19, an expression such as (random 100)in the end position.

Presumably the intent here is to loop over the primes from zero towhatever random number is returned by (random 100). However,this isn't what the current implementation does, asMACROEXPAND-1 shows.

When this expansion code is run, RANDOM will be called each timethe end test for the loop is evaluated. Thus, instead of loopinguntil p is greater than an initially chosen random number,this loop will iterate until it happens to draw a random number lessthan or equal to the current value of p. While the totalnumber of iterations will still be random, it will be drawn from amuch different distribution than the uniform distribution RANDOMreturns.

This is a leak in the abstraction because, to use the macro correctly,the caller needs to be aware that the end form is going to beevaluated more than once. One way to plug this leak would be to simplydefine this as the behavior of do-primes. But that's not verysatisfactory--you should try to observe the Principle of LeastAstonishment when implementing macros. And programmers will typicallyexpect the forms they pass to macros to be evaluated no more timesthan absolutely necessary.⁶ Furthermore, since do-primes is builton the model of the standard macros, DOTIMES and DOLIST,neither of which causes any of the forms except those in the body tobe evaluated more than once, most programmers will expectdo-primes to behave similarly.

You can fix the multiple evaluation easily enough; you just need togenerate code that evaluates end once and saves the value in avariable to be used later. Recall that in a DO loop, variablesdefined with an initialization form and no step form don't changefrom iteration to iteration. So you can fix the multiple evaluationproblem with this definition:

Unfortunately, this fix introduces two new leaks to the macroabstraction.

One new leak is similar to the multiple-evaluation leak you justfixed. Because the initialization forms for variables in a DOloop are evaluated in the order the variables are defined, when themacro expansion is evaluated, the expression passed as endwill be evaluated before the expression passed as start,opposite to the order they appear in the macro call. This leakdoesn't cause any problems when start and end areliteral values like 0 and 19. But when they're forms that can haveside effects, evaluating them out of order can once again run afoulof the Principle of Least Astonishment.

This leak is trivially plugged by swapping the order of the twovariable definitions.

The last leak you need to plug was created by using the variable nameending-value. The problem is that the name, which ought to bea purely internal detail of the macro implementation, can end upinteracting with code passed to the macro or in the context where themacro is called. The following seemingly innocent call todo-primes doesn't work correctly because of this leak:

Neither does this one:

Again, MACROEXPAND-1 can show you the problem. The first callexpands to this:

Some Lisps may reject this code because ending-value is usedtwice as a variable name in the same DO loop. If not rejectedoutright, the code will loop forever since ending-value willnever be greater than itself.

The second problem call expands to the following:

In this case the generated code is perfectly legal, but the behaviorisn't at all what you want. Because the binding ofending-value established by the LET outside the loop isshadowed by the variable with the same name inside the DO, theform (incf ending-value p) increments the loop variableending-value instead of the outer variable with the same name,creating another infinite loop.⁷

Clearly, what you need to patch this leak is a symbol that will neverbe used outside the code generated by the macro. You could try usinga really unlikely name, but that's no guarantee. You could alsoprotect yourself to some extent by using packages, as described inChapter 21. But there's a better solution.

The function GENSYM returns a unique symbol each time it'scalled. This is a symbol that has never been read by the Lisp readerand never will be because it isn't interned in any package. Thus,instead of using a literal name like ending-value, you cangenerate a new symbol each time do-primes is expanded.

Note that the code that calls GENSYM isn't part of theexpansion; it runs as part of the macro expander and thus creates anew symbol each time the macro is expanded. This may seem a bitstrange at first--ending-value-name is a variable whose valueis the name of another variable. But really it's no different fromthe parameter var whose value is the name of a variable--thedifference is the value of var was created by the reader whenthe macro form was read, and the value of ending-value-name isgenerated programmatically when the macro code runs.

With this definition the two previously problematic forms expand intocode that works the way you want. The first form:

expands into the following:

Common Lisp Macros

Now the variable used to hold the ending value is the gensymedsymbol, #:g2141. The name of the symbol, G2141, wasgenerated by GENSYM but isn't significant; the thing thatmatters is the object identity of the symbol. Gensymed symbols areprinted in the normal syntax for uninterned symbols, with a leading#:.

The other previously problematic form:

looks like this if you replace the do-primes form with itsexpansion:

Again, there's no leak since the ending-value variable boundby the LET surrounding the do-primes loop is no longershadowed by any variables introduced in the expanded code.

Not all literal names used in a macro expansion will necessarilycause a problem--as you get more experience with the various bindingforms, you'll be able to determine whether a given name is being usedin a position that could cause a leak in a macro abstraction. Butthere's no real downside to using a gensymed name just to be safe.

With that fix, you've plugged all the leaks in the implementation ofdo-primes. Once you've gotten a bit of macro-writingexperience under your belt, you'll learn to write macros with thesekinds of leaks preplugged. It's actually fairly simple if you followthese rules of thumb:

• Unless there's a particular reason to do otherwise, include anysubforms in the expansion in positions that will be evaluated in thesame order as the subforms appear in the macro call.
• Unless there's a particular reason to do otherwise, make suresubforms are evaluated only once by creating a variable in theexpansion to hold the value of evaluating the argument form and thenusing that variable anywhere else the value is needed in theexpansion.
• Use GENSYM at macro expansion time to create variable namesused in the expansion.

Macro-Writing Macros

Common Lisp For Mac Download

Of course, there's no reason you should be able to take advantage ofmacros only when writing functions. The job of macros is to abstractaway common syntactic patterns, and certain patterns come up againand again in writing macros that can also benefit from beingabstracted away.

In fact, you've already seen one such pattern--many macros will, likethe last version of do-primes, start with a LET thatintroduces a few variables holding gensymed symbols to be used in themacro's expansion. Since this is such a common pattern, why notabstract it away with its own macro?

In this section you'll write a macro, with-gensyms, that doesjust that. In other words, you'll write a macro-writing macro: amacro that generates code that generates code. While complexmacro-writing macros can be a bit confusing until you get used tokeeping the various levels of code clear in your mind,with-gensyms is fairly straightforward and will serve as auseful but not too strenuous mental limbering exercise.

You want to be able to write something like this:

and have it be equivalent to the previous version ofdo-primes. In other words, the with-gensyms needs toexpand into a LET that binds each named variable,ending-value-name in this case, to a gensymed symbol. That'seasy enough to write with a simple backquote template.

Note how you can use a comma to interpolate the value of theLOOP expression. The loop generates a list of binding formswhere each binding form consists of a list containing one of thenames given to with-gensyms and the literal code(gensym). You can test what code the LOOP expressionwould generate at the REPL by replacing names with a list ofsymbols.

After the list of binding forms, the body argument towith-gensyms is spliced in as the body of the LET. Thus,in the code you wrap in a with-gensyms you can refer to any ofthe variables named in the list of variables passed towith-gensyms.

If you macro-expand the with-gensyms form in the newdefinition of do-primes, you should see something like this:

Looks good. While this macro is fairly trivial, it's important tokeep clear about when the different macros are expanded: when youcompile the DEFMACRO of do-primes, thewith-gensyms form is expanded into the code just shown andcompiled. Thus, the compiled version of do-primes is just thesame as if you had written the outer LET by hand. When youcompile a function that uses do-primes, the code generatedby with-gensyms runs generating the do-primesexpansion, but with-gensyms itself isn't needed to compile ado-primes form since it has already been expanded, back whendo-primes was compiled.

Beyond Simple Macros

I could, of course, say a lot more about macros. All the macrosyou've seen so far have been fairly simple examples that save you abit of typing but don't provide radical new ways of expressingthings. In upcoming chapters you'll see examples of macros that allowyou to express things in ways that would be virtually impossiblewithout macros. You'll start in the very next chapter, in whichyou'll build a simple but effective unit test framework.

¹As with functions, macros can also containdeclarations, but you don't need to worry about those for now.

²APPEND, which I haven't discussed yet, is a functionthat takes any number of list arguments and returns the result ofsplicing them together into a single list.

Since the arguments passed to a macro are Lisp objects representingthe source code of the macro call, the first step in any macro is toextract whatever parts of those objects are needed to compute theexpansion. For macros that simply interpolate their argumentsdirectly into a template, this step is trivial: simply defining theright parameters to hold the different arguments is sufficient.

But this approach, it seems, will not suffice for do-primes.The first argument to the do-primes call is a list containingthe name of the loop variable, p; the lower bound, 0;and the upper bound, 19. But if you look at the expansion, thelist as a whole doesn't appear in the expansion; the three elementare split up and put in different places.

You could define do-primes with two parameters, one to holdthe list and a &rest parameter to hold the body forms, and thentake apart the list by hand, something like this:

In a moment I'll explain how the body generates the correctexpansion; for now you can just note that the variables var,start, and end each hold a value, extracted fromvar-and-range, that's then interpolated into the backquoteexpression that generates do-primes's expansion.

However, you don't need to take apart var-and-range 'by hand'because macro parameter lists are what are called destructuringparameter lists. Destructuring, as the name suggests, involves takingapart a structure--in this case the list structure of the formspassed to a macro.

Within a destructuring parameter list, a simple parameter name can bereplaced with a nested parameter list. The parameters in the nestedparameter list will take their values from the elements of theexpression that would have been bound to the parameter the listreplaced. For instance, you can replace var-and-range with alist (var start end), and the three elements of the list willautomatically be destructured into those three parameters.

Another special feature of macro parameter lists is that you can use&body as a synonym for &rest. Semantically &body and&rest are equivalent, but many development environments will usethe presence of a &body parameter to modify how they indent usesof the macro--typically &body parameters are used to hold alist of forms that make up the body of the macro.

So you can streamline the definition of do-primes and give ahint to both human readers and your development tools about itsintended use by defining it like this:

Mac printer driver for hp 1020. In addition to being more concise, destructuring parameter lists alsogive you automatic error checking--with do-primes defined thisway, Lisp will be able to detect a call whose first argument isn't athree-element list and will give you a meaningful error message justas if you had called a function with too few or too many arguments.Also, in development environments such as SLIME that indicate whatarguments are expected as soon as you type the name of a function ormacro, if you use a destructuring parameter list, the environmentwill be able to tell you more specifically the syntax of the macrocall. With the original definition, SLIME would tell youdo-primes is called like this:

But with the new definition, it can tell you that a call should looklike this:

Destructuring parameter lists can contain &optional, &key,and &rest parameters and can contain nested destructuring lists.However, you don't need any of those options to writedo-primes.

Generating the Expansion

Because do-primes is a fairly simple macro, after you'vedestructured the arguments, all that's left is to interpolate theminto a template to get the expansion.

For simple macros like do-primes, the special backquote syntaxis perfect. To review, a backquoted expression is similar to a quotedexpression except you can 'unquote' particular subexpressions bypreceding them with a comma, possibly followed by an at (@) sign.Without an at sign, the comma causes the value of the subexpressionto be included as is. With an at sign, the value--which must be alist--is 'spliced' into the enclosing list.

Another useful way to think about the backquote syntax is as aparticularly concise way of writing code that generates lists. Thisway of thinking about it has the benefit of being pretty much exactlywhat's happening under the covers--when the reader reads a backquotedexpression, it translates it into code that generates the appropriatelist structure. For instance, `(,a b) might be read as(list a 'b). The language standard doesn't specify exactlywhat code the reader must produce as long as it generates the rightlist structure.

Table 8-1 shows some examples of backquoted expressions along withequivalent list-building code and the result you'd get if youevaluated either the backquoted expression or the equivalentcode.²

It's important to note that backquote is just a convenience. But it'sa big convenience. To appreciate how big, compare the backquotedversion of do-primes to the following version, which usesexplicit list-building code:

As you'll see in a moment, the current implementation ofdo-primes doesn't handle certain edge cases correctly. Butfirst you should verify that it at least works for the originalexample. You can test it in two ways. You can test it indirectly bysimply using it--presumably, if the resulting behavior is correct,the expansion is correct. For instance, you can type the originalexample's use of do-primes to the REPL and see that it indeedprints the right series of prime numbers.

Or you can check the macro directly by looking at the expansion of aparticular call. The function MACROEXPAND-1 takes any Lispexpression as an argument and returns the result of doing one levelof macro expansion.³ Because MACROEXPAND-1 is afunction, to pass it a literal macro form you must quote it. You canuse it to see the expansion of the previous call.⁴

Or, more conveniently, in SLIME you can check a macro's expansion byplacing the cursor on the opening parenthesis of a macro form in yoursource code and typing C-c RET to invoke the Emacs functionslime-macroexpand-1, which will pass the macro form toMACROEXPAND-1 and 'pretty print' the result in a temporarybuffer.

However you get to it, you can see that the result of macro expansionis the same as the original handwritten expansion, so it seems thatdo-primes works.

Plugging the Leaks

In his essay 'The Law of Leaky Abstractions,' Joel Spolsky coined theterm leaky abstraction to describe an abstraction that 'leaks'details it's supposed to be abstracting away. Since writing a macrois a way of creating an abstraction, you need to make sure yourmacros don't leak needlessly.⁵

As it turns out, a macro can leak details of its inner workings inthree ways. Luckily, it's pretty easy to tell whether a given macrosuffers from any of those leaks and to fix them.

The current definition suffers from one of the three possible macroleaks: namely, it evaluates the end subform too many times.Suppose you were to call do-primes with, instead of a literalnumber such as 19, an expression such as (random 100)in the end position.

Presumably the intent here is to loop over the primes from zero towhatever random number is returned by (random 100). However,this isn't what the current implementation does, asMACROEXPAND-1 shows.

When this expansion code is run, RANDOM will be called each timethe end test for the loop is evaluated. Thus, instead of loopinguntil p is greater than an initially chosen random number,this loop will iterate until it happens to draw a random number lessthan or equal to the current value of p. While the totalnumber of iterations will still be random, it will be drawn from amuch different distribution than the uniform distribution RANDOMreturns.

This is a leak in the abstraction because, to use the macro correctly,the caller needs to be aware that the end form is going to beevaluated more than once. One way to plug this leak would be to simplydefine this as the behavior of do-primes. But that's not verysatisfactory--you should try to observe the Principle of LeastAstonishment when implementing macros. And programmers will typicallyexpect the forms they pass to macros to be evaluated no more timesthan absolutely necessary.⁶ Furthermore, since do-primes is builton the model of the standard macros, DOTIMES and DOLIST,neither of which causes any of the forms except those in the body tobe evaluated more than once, most programmers will expectdo-primes to behave similarly.

You can fix the multiple evaluation easily enough; you just need togenerate code that evaluates end once and saves the value in avariable to be used later. Recall that in a DO loop, variablesdefined with an initialization form and no step form don't changefrom iteration to iteration. So you can fix the multiple evaluationproblem with this definition:

Unfortunately, this fix introduces two new leaks to the macroabstraction.

One new leak is similar to the multiple-evaluation leak you justfixed. Because the initialization forms for variables in a DOloop are evaluated in the order the variables are defined, when themacro expansion is evaluated, the expression passed as endwill be evaluated before the expression passed as start,opposite to the order they appear in the macro call. This leakdoesn't cause any problems when start and end areliteral values like 0 and 19. But when they're forms that can haveside effects, evaluating them out of order can once again run afoulof the Principle of Least Astonishment.

This leak is trivially plugged by swapping the order of the twovariable definitions.

The last leak you need to plug was created by using the variable nameending-value. The problem is that the name, which ought to bea purely internal detail of the macro implementation, can end upinteracting with code passed to the macro or in the context where themacro is called. The following seemingly innocent call todo-primes doesn't work correctly because of this leak:

Neither does this one:

Again, MACROEXPAND-1 can show you the problem. The first callexpands to this:

Some Lisps may reject this code because ending-value is usedtwice as a variable name in the same DO loop. If not rejectedoutright, the code will loop forever since ending-value willnever be greater than itself.

The second problem call expands to the following:

In this case the generated code is perfectly legal, but the behaviorisn't at all what you want. Because the binding ofending-value established by the LET outside the loop isshadowed by the variable with the same name inside the DO, theform (incf ending-value p) increments the loop variableending-value instead of the outer variable with the same name,creating another infinite loop.⁷

Clearly, what you need to patch this leak is a symbol that will neverbe used outside the code generated by the macro. You could try usinga really unlikely name, but that's no guarantee. You could alsoprotect yourself to some extent by using packages, as described inChapter 21. But there's a better solution.

The function GENSYM returns a unique symbol each time it'scalled. This is a symbol that has never been read by the Lisp readerand never will be because it isn't interned in any package. Thus,instead of using a literal name like ending-value, you cangenerate a new symbol each time do-primes is expanded.

Note that the code that calls GENSYM isn't part of theexpansion; it runs as part of the macro expander and thus creates anew symbol each time the macro is expanded. This may seem a bitstrange at first--ending-value-name is a variable whose valueis the name of another variable. But really it's no different fromthe parameter var whose value is the name of a variable--thedifference is the value of var was created by the reader whenthe macro form was read, and the value of ending-value-name isgenerated programmatically when the macro code runs.

With this definition the two previously problematic forms expand intocode that works the way you want. The first form:

expands into the following:

Common Lisp Macros

Now the variable used to hold the ending value is the gensymedsymbol, #:g2141. The name of the symbol, G2141, wasgenerated by GENSYM but isn't significant; the thing thatmatters is the object identity of the symbol. Gensymed symbols areprinted in the normal syntax for uninterned symbols, with a leading#:.

The other previously problematic form:

looks like this if you replace the do-primes form with itsexpansion:

Again, there's no leak since the ending-value variable boundby the LET surrounding the do-primes loop is no longershadowed by any variables introduced in the expanded code.

Not all literal names used in a macro expansion will necessarilycause a problem--as you get more experience with the various bindingforms, you'll be able to determine whether a given name is being usedin a position that could cause a leak in a macro abstraction. Butthere's no real downside to using a gensymed name just to be safe.

With that fix, you've plugged all the leaks in the implementation ofdo-primes. Once you've gotten a bit of macro-writingexperience under your belt, you'll learn to write macros with thesekinds of leaks preplugged. It's actually fairly simple if you followthese rules of thumb:

• Unless there's a particular reason to do otherwise, include anysubforms in the expansion in positions that will be evaluated in thesame order as the subforms appear in the macro call.
• Unless there's a particular reason to do otherwise, make suresubforms are evaluated only once by creating a variable in theexpansion to hold the value of evaluating the argument form and thenusing that variable anywhere else the value is needed in theexpansion.
• Use GENSYM at macro expansion time to create variable namesused in the expansion.

Macro-Writing Macros

Common Lisp For Mac Download

Of course, there's no reason you should be able to take advantage ofmacros only when writing functions. The job of macros is to abstractaway common syntactic patterns, and certain patterns come up againand again in writing macros that can also benefit from beingabstracted away.

In fact, you've already seen one such pattern--many macros will, likethe last version of do-primes, start with a LET thatintroduces a few variables holding gensymed symbols to be used in themacro's expansion. Since this is such a common pattern, why notabstract it away with its own macro?

In this section you'll write a macro, with-gensyms, that doesjust that. In other words, you'll write a macro-writing macro: amacro that generates code that generates code. While complexmacro-writing macros can be a bit confusing until you get used tokeeping the various levels of code clear in your mind,with-gensyms is fairly straightforward and will serve as auseful but not too strenuous mental limbering exercise.

You want to be able to write something like this:

and have it be equivalent to the previous version ofdo-primes. In other words, the with-gensyms needs toexpand into a LET that binds each named variable,ending-value-name in this case, to a gensymed symbol. That'seasy enough to write with a simple backquote template.

Note how you can use a comma to interpolate the value of theLOOP expression. The loop generates a list of binding formswhere each binding form consists of a list containing one of thenames given to with-gensyms and the literal code(gensym). You can test what code the LOOP expressionwould generate at the REPL by replacing names with a list ofsymbols.

After the list of binding forms, the body argument towith-gensyms is spliced in as the body of the LET. Thus,in the code you wrap in a with-gensyms you can refer to any ofthe variables named in the list of variables passed towith-gensyms.

If you macro-expand the with-gensyms form in the newdefinition of do-primes, you should see something like this:

Looks good. While this macro is fairly trivial, it's important tokeep clear about when the different macros are expanded: when youcompile the DEFMACRO of do-primes, thewith-gensyms form is expanded into the code just shown andcompiled. Thus, the compiled version of do-primes is just thesame as if you had written the outer LET by hand. When youcompile a function that uses do-primes, the code generatedby with-gensyms runs generating the do-primesexpansion, but with-gensyms itself isn't needed to compile ado-primes form since it has already been expanded, back whendo-primes was compiled.

Beyond Simple Macros

I could, of course, say a lot more about macros. All the macrosyou've seen so far have been fairly simple examples that save you abit of typing but don't provide radical new ways of expressingthings. In upcoming chapters you'll see examples of macros that allowyou to express things in ways that would be virtually impossiblewithout macros. You'll start in the very next chapter, in whichyou'll build a simple but effective unit test framework.

¹As with functions, macros can also containdeclarations, but you don't need to worry about those for now.

²APPEND, which I haven't discussed yet, is a functionthat takes any number of list arguments and returns the result ofsplicing them together into a single list.

Common Lisp Format Flush

³Another function, MACROEXPAND, keepsexpanding the result as long as the first element of the resultingexpansion is the name of the macro. However, this will often show youa much lower-level view of what the code is doing than you want,since basic control constructs such as DO are also implementedas macros. In other words, while it can be educational to see whatyour macro ultimately expands into, it isn't a very useful view intowhat your own macros are doing.

Common Lisp Macrolet

⁴If the macroexpansion is shown all on one line, it's probably because thevariable *PRINT-PRETTY* is NIL. If it is, evaluating(setf *print-pretty* t) should make the macro expansion easierto read.

⁵This is from Joel on Softwareby Joel Spolsky, also available athttp://www.joelonsoftware.com/articles/LeakyAbstractions.html. Spolsky's point in the essay isthat all abstractions leak to some extent; that is, there are noperfect abstractions. But that doesn't mean you should tolerate leaksyou can easily plug.

⁶Of course, certain forms are supposedto be evaluated more than once, such as the forms in the body of ado-primes loop.

⁷It may not be obvious that thisloop is necessarily infinite given the nonuniform occurrences ofprime numbers. The starting point for a proof that it is in factinfinite is Bertrand's postulate, which says for any n > 1, thereexists a prime p, n < p < 2n. From there you canprove that for any prime number, P less than the sum of the precedingprime numbers, the next prime, P', is also smaller than the originalsum plus P.