Origin of Advice

Apparently, Gary King needs advice a lot. This reminded me that I posted an article about the origin of advice some time ago in the AOSD mailing list. I think it's a good idea to repost it here to make it available to others as well, so here we go. (I have links for almost all the literature references at the bottom of this article.)

The notion behind advice can be traced back to a paper by Oliver Selfridge [Selfridge 1958]. He introduced the notion of demons that record events as they occur, recognize patterns in those events, and can trigger subsequent events according to patterns that they care about.

A first software system that was obviously heavily influenced by that paper was called PILOT and is described in Warren Teitelman's PhD thesis [Teitelman 1966]. The PhD thesis was supervised by Marvin Minsky, but additionally Warren Teitelman mentions Oliver Selfridge as a strong influence in his acknowledgements. Marvin Minsky and Oliver Selfridge worked both at MIT back then. Anyway, that is the work in which the notion of advice, very similar to before and after advice as we know them today, was actually first introduced. Warren Teitelman later added the concept of advice to BBN Lisp, which was then bought/licensed (?) by Xerox PARC and became Interlisp.

Later, the notion of demons was mentioned in a seminal paper by Marvin Minsky [Minsky 1974] that spawned an interest in framework-based knowledge representation systems. Among others, Daniel Bobrow and Terry Winograd developed and described KRL [Bobrow, Winograd 1976], which was based on the ideas in Marvin Minsky's paper. If I understand correctly, before/after demons played an important role in such systems.

A little bit later, Howard Cannon developed Flavors at MIT, the first object-oriented extension for Lisp, strongly influenced by Alan Kay's Smalltalk. Howard Cannon had written a very influential paper [Cannon 1979-2003] that was, unfortunately, never officially published. He explicitly mentions before/after demons, as do other publications about Flavors, for example [Weinreb, Moon 1980].

I have a copy of Howard Cannon's paper availabe, and I have asked him to make it publicly available, but he still hasn't done this (yet). His is a mind-blowing paper that introduces multiple inheritance, method combinations based on macros - i.e. before and after demons and a first precursor to around methods -, and the notion of meta-protocols that obviously later on turned into metaobject protocols.

The experiences with KRL and Flavors had then been integrated at Xerox PARC into LOOPS (Lisp Object-Oriented Programming System), foremostly by Daniel Bobrow and Mark Stefik, implemented in Interlisp. There is a nice overview page about LOOPS and a download page for the papers mentioned there.

Flavors and LOOPS were chosen as the main bases for the Common Lisp Object System (CLOS) as part of the ANSI standardization of Common Lisp. CLOS was developed by representatives of the various existing object-oriented extensions for Lisp. LOOPS / Xerox PARC was represented by Daniel Bobrow and Gregor Kiczales. This was around 1986 - 1988.

CLOS has before/after/around methods. I haven't been able to spot when around methods entered the scene, whether this was already part of LOOPS or whether this was an addition in CLOS. In Flavors, there were only before/after methods, but there was an extra concept called wrappers that effectively allowed one to express the same thing as around methods in CLOS.

One of the most impressive outcomes of the efforts behind LOOPS and CLOS is the book The Art of the Metaobject Protocol [Kiczales, des Rivières, Bobrow 1991], which I think is one of the most important books in the history of computer science (and Alan Kay seems to agree).

Crista Lopes' paper [Lopes 2002] describes the subsequent history how metaobject protocols were turned into what we think of as aspect-oriented programming today. The main motivations, as far as I understand them, were a) to move from a runtime-based approach, which is natural for metaobject protocols, towards a compile-time based approach and b) to make some of the benefits of being able to manipulate the meta-level available to purely base-level code. Advice play an important role in aspect-oriented programming, but instead of advising just single functions, you can advise whole pointcuts, which are essentially sets of functions described in (more or less) declarative ways.

Robert Hirschfeld, myself and others have taken a different turn with Context-oriented Programming, and focus on a more dynamic approach again. We have taken the idea of crosscutting concerns that emerged in the AOSD community, but dropped the idea of pointcuts, and instead concentrated on new and interesting ways to dynamically activate and deactivate layers, which are potentially crosscutting behavioral program variations. Since you can add new layers at any point in time, you can also effectively add new levels of before/after/around methods at runtime as needed, something that can be achieved in plain CLOS only statically through new user-defined method combinations, or requires recompilation in aspect-oriented language extensions like AspectJ. Here are some links for Context-oriented Programming:

• Context-oriented Programming at Hasso-Plattner Institut.


• My research page with sections on ContextL and Context-oriented Programming.


• The ContextL project page.

I agree that advice are an important concept in programming, but we have still not seen all the possible and interesting variations yet. Although they have a long history already, there is still a future ahead for them.

References

• [Bobrow, Winograd 1976] Daniel Bobrow and Terry Winograd, An Overview of KRL, A Knowledge Representation Language.


• [Cannon 1979-2003] Howard Cannon, Flavors - A Non-Hierarchical Approach to Object-Oriented Programming. Unpublished draft.


• [Kiczales, des Rivières, Bobrow 1991] Gregor Kiczales, Jim des Rivières, Daniel G. Bobrow, The Art of the Metaobject Protocol, MIT Press, 1991.


• [Lopes 2002] Cristina Videira Lopes, AOP: A Historical Perspective.


• [Minsky 1974] Marvin Minsky, A Framework for Representing Knowledge.


• [Selfridge 1958] Oliver G. Selfridge, Pandemonium, Mechanization of Though Processes: Proceedings of a Symposium held at the National Physics Laboratory on 24-27 Nov. 1958, volume I., (NPL Symposium no. 10, HMSO, 1959).


• [Teitelman 1966] Warren Teitelman, PILOT: A Step Toward Man-Computer Symbiosis.


• [Weinreb, Moon 1980] Daniel Weinreb and David Moon, Flavors: Message Passing in the Lisp Machine.

Reflection in Potsdam

Charlotte Herzeel and I had been invited by Robert Hirschfeld to give presentations at the Hasso-Plattner-Institut in Potsdam, Germany about the CLOS Metaobject Protocol and 3-Lisp about two weeks ago. These presentations have been recorded and are now online for your viewing pleasure:

• Pascal Costanza, The CLOS Metaobject Protocol (in German)


• Charlotte Herzeel, Reflecting on 3-Lisp (in English)

You need RealPlayer to see this.

European Lisp Symposium mentioned in Linux Magazin

The 1st European Lisp Symposium is mentioned in Linux Magazin online (in German language).

European Lisp Symposium 2008 - Call for Papers

1st European Lisp Symposium

Bordeaux, France, May 22-23, 2008

LaBRI, Université Bordeaux 1


Important Dates

• Submission of research papers: February 11, 2008


• Work-in-progress papers: March 24, 2008


• Author notification: April 7, 2008


• First final versions due: April 28, 2008

Accepted research papers will be invited for a special issue of the Journal of Universal Computer Science (J.UCS). See the symposium website for more details.

Scope

The European Lisp Symposium 2008 invites high quality papers about novel research results, insights and lessons learned from practical applications, and educational perspectives, all involving Lisp dialects, including Common Lisp, Scheme, ISLISP, Dylan, and so on.

Topics include, but are not limited to:

• Language design and implementation techniques


• Language integration, interoperation and deployment


• Experience reports and case studies


• Reflection and meta-level architectures


• Educational approaches


• Software adaptation and evolution


• Configuration management


• Artificial intelligence


• Large and ultra-large-scale systems


• Development methodologies


• Development support and environments


• Persistent systems


• Scientific computing


• Parallel and distributed computing


• Data mining


• Semantic web


• Dynamic optimization


• Innovative applications


• Hardware and virtual machine support


• Domain-oriented programming

We also encourage submissions about past approaches that have been largely forgotten about, as long as they are presented in a new setting.

We invite submissions in two categories:
original contributions and work-in-progress papers.

• Original contributions have neither been published previously nor are under review by other refereed events or publications. Research papers should describe work that advances the current state of the art, or presents old results from a new perspective. Experience papers should be of broad interest and should describe insights gained from substantive practical applications. The program committee will evaluate each contributed paper based on its relevance, significance, clarity, and originality.
  
  Accepted papers will be published in the Journal of Universal Computer Science (J.UCS). Authors of accepted papers are expected to present their work at the symposium main track in Bordeaux on May 23, 2008.



• Work in progress describes ongoing work that is not ready for publication yet, but would benefit strongly from feedback by other researchers, practitioners and educators. Such contributions will not be published in the symposium proceedings, but will be made available at the symposium website. The work-in-progress track will be organized as a series of writers' workshops where authors work together to improve their papers. Some authors who submit papers for the main track will be suggested to contribute their work in this track instead, if the program committee decides that their submission is not yet ready for a publication.
  
  The writers' workshops will take place at the symposium in Bordeaux on May 22, 2008.

Submissions

Papers for the main track must be submitted electronically, preferably as PDF or PostScript file (level 1 or 2). However, submissions in RTF or Word format are also accepted. Initial submissions may not exceed 15 pages in the J.UCS style, including all appendices. (Invited papers for the journal publication will have a page limitation of 25 pages in the same format.) See the symposium website for more details, including about the submission procedure.

Papers for the work-in-progress track may be in PDF, PostScript level 1 or 2, RTF or Word, and may not exceed 25 pages. There are no further requirements on their format. Papers for the work-in-progress track must be sent via email to pascal.costanza@vub.ac.be.


Program Chair

• Pascal Costanza, Vrije Universiteit Brussel, Belgium

Program Committee

• Marco Antoniotti, Universita Milano Bicocca, Italy


• Marie Beurton-Aimar, Université Bordeaux 1, France


• Jerry Boetje, College of Charlston, USA


• Theo D'Hondt, Vrije Universiteit Brussel, Belgium


• Irène Durand, Université Bordeaux 1, France


• Marc Feeley, Université de Montréal, Canada


• Erick Gallesio, Universite de Nice / Sophia Antipolis, France


• Rainer Joswig, Independent Consultant, Germany


• António Leitão, Technical University of Lisbon, Portugal


• Henry Lieberman, MIT, USA


• Scott McKay, ITA Software, Inc., USA


• Ralf Möller, Hamburg University of Technology, Germany


• Nicolas Neuss, Universität Karlsruhe, Germany


• Kent Pitman, PTC, USA


• Christophe Rhodes, Goldsmiths College, University of London, United Kingdom


• Jeffrey Mark Siskind, Purdue University, USA


• Didier Verna, EPITA Research and Development Laboratory, France

Lisp success story from Hungary

[Levente Mészáros posted the following Lisp success story to comp.lang.lisp. I reproduce it here with kind permission so that it is also available as a blog entry. In other words, the following is not written by me, but by Levy.]

We are a small group of friends (3-5) who left the "professional" programming scene about one and a half year ago to start our own Lisp business.

In July, 2007 we started working on a project for the Hungarian government with a very tight deadline. In mid October the system showed first life signs and now we can say it is extensively used by thousands of users. In spite of the short timeframe and the remaining glitches it was a surprise even for us that the system is able to handle such a load.

Unfortunately the application is not publicly accessible, so we can't provide a URL, but we can give some of the details. It is used for gathering data from the Hungarian communes to help the budget planning.

The entire system is based on opensource software, it runs on multiple SBCLs behind an apache load balancer. The data is stored in a PostgreSQL server.

We have around 4000 registered users, average 300 online and more than 500 at peak times. There are 7000-10000 logins per day.

We use the following Common Lisp libraries. Some of which are ours and some of which are built by others. Many thanks for their efforts, especially to the SBCL developers!

Data storage/persistence
- cl-perec
- cl-rdbms
- cl-postgres (postmodern)

Web GUI
- cl-dwim (the umbrella project)
- UCW
- yaclml
- parenscript
- rfc*

PDF exporting
- cl-pdf
- cl-typesetting

Unit testing
- stefil
- SLIME and emacs with customizations

MOP
- computed-class
- contextl

Misc
- bind
- alexandria
- iterate
- cl-def
- cl-l10n
- cl-graphviz
- cl-serializer
- defclass-star
- local-time
- cffi
- babel
- verrazano

and more...

Main software components:
- Ubuntu linux 6.10
- PostgreSQL 8.2
- SBCL 1.0.10
- Apache2 (due to various headaches soon to be replaced by a lisp based load balancer)

Hardware:
- 10 nodes with two dual core x86-64 CPUs summing up to 40 cores altogether, each node has 3 Gbytes ram.
- Currently we use one of the nodes as database server but soon it will be replaced with another server with 2 or 4 quad core x86-64 CPUs, 300 GByte disk in raid 5 and 16 GBytes ram.

Some people say Lisp is dead but seems like there's life after death... ;-)

ContextL Survey September 2007

Context-dependent behavior is becoming increasingly important for a wide range of application domains. Unfortunately, mainstream programming languages do not provide mechanisms that enable software entities to adapt their behavior dynamically to the current execution context. In collaboration with various researchers, we have developed a new programming technique called "Context-oriented Programming" (COP).

ContextL is our first fully implemented and currently most mature programming language extension for COP and is built on top of the Common Lisp Object System (CLOS). ContextL has first been made available to the public in early 2005, and has already been adopted by a number of programmers. We would now like to assess how well ContextL has been received so far.

Please consider participating in our first survey about ContextL - this will help us a lot to develop ContextL and related projects further (like Closer to MOP, etc.).

You can find the survey and more information about it at http://prog.vub.ac.be/~pcostanza/COP/survey.html.

ECOOP'07 Lisp Dinner

As a social event attached to the 4th European Lisp Workshop, we are planning to have a dinner on Monday, the 30th of July in Berlin, starting at 19:30 hours. We have not yet decided what the location will be, and this depends on how many people will participate. The dinner will be open to anyone.

If you want to attend, please send a short email notice to Hans Hübner. Also let him know if you have special dinner needs or wishes.

Lisp meeting in The Hague this Sunday

I am forwarding this announcement from Tayssir John Gabbour:

We're hosting a Lisp meeting in The Hague this Sunday. At least ten
people should be here, with a broad range of experiences and
interests.

For those interested in Lisp businesses, this meeting will be at the
offices of a growing Lisp-using company which already has multiple
commercial projects. You'll be able to see up-close what sorts of
issues arise. I personally hope that these meetings won't just be
episodic affairs, but offer a practical service to interested Lisp
users.

To counterbalance boring business issues, Pascal Costanza is offering
an entertainingly philosophical talk about the history of reflection
and metaprogramming in the Lisp family. (If you were ever curious what
3-Lisp and its "infinite reflective tower" is all about, you'll no
doubt find this interesting.)


This is an informal meeting, so no registration is needed: feel free
to come as you wish, and I hope you enjoy it. If you need directions
or a hotel room, there's more info at: http://wiki.alu.org/BeNeLux.

Tayssir

AMOP at Google Book Search

While checking some links for my previous posting I have found out that Google Book Search provides the full text of The Art of Metaobject Protocol online. That's nice.

Classes in the Metaobject Protocol

Dr. Dobb's Portal has republished another classic Lisp article, again by Nick Bourbaki (a pseudonym of Richard Gabriel). It's an introduction to the CLOS Metaobject Protocol which is more technical than others I have read - so may be harder to understand for newbies - but at the same time provides more insight into some of the internal details of the CLOS MOP. Although this is an old article, it's very close to the actual CLOS MOP specification, with only one serious deviation as far as I can tell. The deviation is the fact that slot-value-using-class is specialized on slot names in that article, while they are actually specialized on slot definition metaobjects in the "real" CLOS MOP specification. The reason for this deviation is that for most of the time during the standardization of CLOS and its MOP, slot-value-using-class was indeed supposed to be specialized on slot names (like slot-unbound and slot-missing in ANSI Common Lisp), but that was changed only very late in the game before publication of The Art of the Metaobject Protocol.

Anyway, the information in that article is still valid, so I can recommend it.

Dr. Dobb's Classic Lisp Articles

Recently, I have noticed that the online version of Dr. Dobb's Journal published an article about multitasking in Common Lisp. What was weird to me was that it reads like a rather old article mentioning techniques that, if I understand correctly, were popular in the 1980's, and especially mentioning a Common Lisp implementation that has not been maintained for probably over a decade. So I asked around in comp.lang.lisp whether anyone had an idea what this was about.

Surprisingly, Jon Erickson, editor of Dr. Dobb's Journal, responded to that question and explained that they routinely get requests for classic articles that are 15-20 years old, and so they indeed regularly republish such articles from the past. Since they are responsible for the archives of a number of publications, such as AI Expert, C/C++ Users Journal, BYTE, and Computer Language, there are probably quite a few nuggets to be rediscovered here.

For example, earlier this year they have already published an article by Paul Graham about graphic objects, originally from 1988. Just yesterday, they have published a very interesting article by Nick Bourbaki about dynamic programming. Nick Bourbaki is actually Richard Gabriel's pseudonym that he used in some of his writings in the early 1990's - see Worse Is Better and also Nickieben Bourbaki.

European Lisp Workshop '07 - news

As already announced, this year's European Lisp Workshop will be held in Berlin on July 30, again in conjunction with ECOOP. There are two noteworthy news items:

• We are happy to announce that Alexander Repenning will be giving a keynote presentation about Antiobjects: Mapping Game AI to Massively Parallel Architectures using Collaborative Diffusion.


• The submission deadline for papers, essays and breakout group proposals has been extended to May 31. Notifications of acceptance will be sent out on June 8, one week before the early registration deadline.

See the workshop website for more information.

ELW'07

There will be a new instance of the European Lisp Workshop at ECOOP 2007 in Berlin, Germany on July 30. The deadline for submission of long papers, short essays and/or proposals for breakout groups is May 13. Please see the ELW'07 website for more information.

Reflective Layer Activation in ContextL

I have just uploaded the paper Reflective Layer Activation in ContextL to my website that we have presented at this year's ACM Symposium on Applied Computing.

This paper describes some aspects of the metaobject protocol of ContextL. This is not exactly the same version as presented at the symposium, but the paper has been updated to reflect the new API in ContextL 0.4. However, apart from the resulting changes of function names and a few other minor edits, the contents have remained the same.

ContextL 0.40.

I have just released version 0.40 of ContextL. The idea is that this version is feature-complete with regard to the feature set of the upcoming 1.0 release. The goal is now to test this version extensively, and to write a documentation of the current features and the API of ContextL before version 1.0 is eventually released.

There have been some changes in the API which are listed at the ContextL homepage. These are mostly cosmetic changes, like better names for functions, etc.

However, there are also some new features. The most interesting ones are probably several readers for the main ContextL metaclasses, such that the introspective part of ContextL is now complete, and a function current-layer-context. The latter can be used to capture the set of currently active layers, which can later be reinstalled for example with funcall-with-layer-context. Current-layer-context should be interesting for example to "inherit" active layers to/from threads in multithreaded Common Lisp implementations, or to capture active layers alongside continuations such that they can both be reinstalled together later on. This could, for example, be interesting in conjunction with continuation-based web frameworks.

For more details, see the ContextL homepage and announcements/discussions in the closer-devel mailing list.

My impressions of ILC'07

I am posting this as an individual member of the Lisp community. I am not speaking for the ILC'07 organizers. (Oh man... ;)

I am very happy that ILC'07 turned out to be so successful. This hasn't been always clear during the preparations, so I guess all the organizers breathed a sigh of relief when the numbers of registrations started to increase (and we suddenly had to come up with ideas how to manage a situation in which we nearly had too many registrations ;).

Among the "generic" highlights, I personally enjoyed the atmosphere a lot. I am convinced that this is mainly due to the fact that almost everyone had their accommodation at the same place. In my experience, conferences where this is the case are typically more successful - you already see the participants in the morning during breakfast, you have a lot more time to talk to each other, and so on.

For me, it was also nice to see so many people again. On average, the number of smart people is a lot higher at Lisp meetings than elsewhere, so there is always a chance to have some very interesting discussions with very different perspectives provided by everyone.

About the conference itself, I am happy that the program chair (JonL White) has carefully selected a very interesting set of presentations. Almost all talks were of high quality, covering a broad range of topics with only a couple of downers. My personal highlights have been:

• Antonio Leitao's work on translating Java source code to idiomatic (!) Common Lisp sources. He has already done the reverse (translating a subset of Common Lisp to Java) in the past and is now heading towards a solution that potentially improves the situation on the library front for Common Lisp. The interesting aspect here is the focus on generating idiomatic code, i.e., code as if it had been written manually by experts in the respective target languages.


• Mike Sperber's illuminating talk about R6RS Scheme. I had some fears wrt having this presentation for an audience that mostly consists of Lispers, but non-Schemers. However, Mike has given a very entertaining talk and helped everyone to have a better idea about the goals of R6RS. (IMHO, it's important that Schemers and Lispers talk more to each other because both sides can learn from each other. I'm glad that ILC'07 has helped improve the situation a little bit.)


• Manuel Serrano has given a very impressive presentation about his HOP framework for developing web applications in Scheme. It's good to see that there are a couple of approaches that head in the right direction of enabling the creation of GUIs that makes users' lives easier instead of focusing on solving technicalities of accidental elements in the underlying technology (for example, the infamous "back" button). I see HOP in the same line as Marc Battyani's (unfortunately unpublished) web framework, but as far as I can tell, Manuel has gone a few steps further.


• Alexander Repenning had a paper about X-expressions in XMLisp, which sounds a lot more boring than what he actually presented. This is not yet another library for dealing with XML, but actually modifies the Common Lisp reader (parser) to treat XML as syntax for literal CLOS objects. The applications that he has built on top of that technology are very impressive indeed.


• Clemens Fruhwirth's Liskell is actually a very simple idea. He takes GHC - a widely used implementation of Haskell - and replaces the Haskell syntax with s-expressions. This enables, among other things, the inclusion of macros. I have always wanted to take a closer look at Haskell, but its syntax always put me off. Now I don't have an excuse anymore. ;)

For obviously personal reasons, I am also very happy that Charlotte Herzeel's presentation about HALO is so well-received - she got a lot of positive feedback for her presentation. However, it is only an overview of HALO, there is still more to come. I hope that we can report on some exciting results soon.

On the negative side about ILC'07, it seems to me that the program was maybe a little too packed - we had to get up very early in the morning (very hard for me ;), and there was maybe not enough time to engage in discussions or even do some work with other attendees. We have tried to establish a breakout group format, but apparently we haven't found the right approach yet. Ralf Mattes suggested a birds-of-a-feather-sessions format to me, so maybe that's something we should look into for the future.

A real downer was the annual meeting of the Associations of Lisp Users (ALU). It wasn't the first time they promised to become more "professional", but their presentation to the ALU members was worse than average. I hope they will manage to get their act together. Otherwise, it will be hard for them to be taken seriously by the Lisp community in the long run. It is worth noting, though, that they are responsible for backing the International Lisp Conference, so they must be doing something right... ;)

Fortunately, the rest of the conference didn't suffer from the ALU meeting - the day after was as good as, or maybe even a little more exciting than, the other days on all levels.

Next on the schedule is the European Lisp Workshop, co-located with ECOOP 2007 and taking place in Berlin on July 30, and most likely sometime early next year another instance of the European Common Lisp Meeting.

Lisp Bleep Survivalism

A few tidbits:

• Registrations for the International Lisp Conference 2007 are coming in like crazy, and we are soon running out of rooms at the Clare College. There are only three days left until the early registration deadline with reduced fees, so there are enough reasons to better hurry up. ;)


• New videoclips by Motor and Nine Inch Nails.


• This is the last posting about music to this blog, since the postings here occur, for example, at Planet Lisp. So in order to stop boring Lispers about music they are not interested in, I will post all music-related ramblings to the freshly brewed infox_blog instead.
  

What is the point of macros?

One of the distinguishing features of Lisp and Scheme is the ability to define macros that allow you to extend the base language with new language constructs. Here are two examples what this actually means and why this is a good idea.


Let's take a common idiom from Java: Whenever you want to open a file, process its contents, and close it again afterwards, you have to write code roughly like this:

FileInputStream in = new FileInputStream(filename);
try {
   doSomething(in);
} finally {
   in.close();
}

Note how the orange code is the code that you are actually interested in: That's the part that states which file you want to open, that you want to open it for reading, and that you want to do something with the file. The rest is just idiomatic code that you have to write just to make the machinery work: You declare a variable outside of the try block (this is important for the visibility of the variable!), and you have to put the call of the close method in a finally block to ensure that the file is always closed even when an exception occurs.

The problem here is that you have to write this code over and over and over again, and everytime you write this code, you have to get the details right. For example, when I was still a Java hacker, I always have put the variable declaration inside the try block, just to get the error message from the compiler that this is wrong. This is annoying - it would be nice if I didn't have to think about this. (I have actually gotten it wrong again in the first version of this posting. Go figure.)

Here is the same code in Common Lisp:

(let ((in (open filename)))
  (unwind-protect
      (do-something in)
    (close in)))

This is not yet an improvement: The orange part of the code is still deep down in other code that just ensures that the machinery for opening and closing the file works. I still have to declare the variable first (here with a let), I still have to use the Lisp-equivalent of try-finally (here it's called unwind-protect), and I still have to ensure that the closing of the file happens as part of the clean-up step of an unwind-protect.

However, Lisp gives us a way to get rid of the idiomatic code. What we would actually like to write is this:

(with-input-file (in filename)
  (do-something in))

This code states exactly what we want and nothing else: We want to execute some code with an open file bound to some variable, and we implicitly want to ensure that it is closed again at the end of this code fragment.

Here is a macro that defines this construct:

(defmacro with-input-file ((variable filename) &body body)
  `(let ((,variable (open ,filename))
     (unwind-protect
         (progn ,@body)
       (close ,variable))))

Here is an explanation of the code:

• The macro is defined with defmacro and we name it like the language construct we want to build. It takes two variables: a variable name and a piece of code that will give us the filename in the resulting code. It also takes a body of code for the with-input-file construct.


• The macro creates a piece of code that the Lisp compiler should use instead of the original code that uses the with-input-file construct: We basically define a code template in which the variable, filename and body spots are filled in with the parameters that the macro receives.


• Some minor details: The progn groups a number of expressions into one - this is similar to grouping code in a pair of curly braces in Java. The backquote, comma and comma-at notations are there to clarify how the parameters should be inserted - that's something that you can better learn in good Lisp tutorials.

Although writing macros takes a little practice, it is very straightforward once you are used to doing this. Macros help you to avoid typing a lot of idiomatic code: whenever you notice some coding pattern, you can put it in some macro and then use it instead.

Syntactic Abstractions

Let's get a little closer to the heart of what macros actually are: They provide you with a way to define syntactic abstractions. Abstractions are used for hiding away implementation details that client code should not be interested in. For example, in classes you can define private fields and methods for internal use and define a public interface that client code has to go through to ensure that the internal details are always used properly. Likewise, with functional abstractions - closures - you can pass around a function that can refer to variables in its lexical environment without giving anyone direct access to these lexical variables.

Many implementation details can be hidden very well with such abstraction mechanisms. However, others cannot be hidden well, and this is when macros become interesting.

For example, assume that your language doesn't provide any iteration constructs but requires you to use recursion instead. You can relatively easily build your own iteration constructs - for example a while function - like this in Lisp:

(defun while-function (predicate block)
  (if (not (funcall predicate)) (return))
  (funcall block)
  (while-function predicate block))    

This function works as follows: It takes a predicate function and a block of code, also provided as a function. When calling the predicate (with funcall) returns false, the while-function returns. Otherwise, the block is executed (again with funcall) and then the while-function calls itself again.

Here is an example of using the while-function:

(let ((i 0))
  (while-function (lambda () (< i 10))
    (lambda ()
      (print i)
      (incf i))))

This code fragment increments i from 0 up to 10 and prints i at each step. This piece of code doesn't look very nice. One reason is that Lisp's lambda construct is somewhat lengthy. Smalltalk, Ruby, and other languages have nicer syntax for lambda expressions, and would make this code shorter. However, even then there is a problem here: We still have to write idiomatic code to write a while loop although we are actually not interested in the idiom. Here, the idiomatic element is the use of a lambda expression to ensure that some code is not immediatily executed, but only later under the control of the while-function. However, what we actually want to say is this, which doesn't contain any idiomatic code at all:

(let ((i 0))
  (while (< i 10)
    (print i)
    (incf i)))

And, as you might have guessed, here is a macro to implement this while construct:

(defmacro while (expression &body body)
  `(while-function (lambda () ,expression)
     (lambda () ,@body)))

This macro uses the while-function in the code that it creates. This is actually one of the typical ways to write Lisp code: We first define the functional (or object-oriented, or imperative, or whatever) abstractions, and then we add some syntactic layer on top to make the code look nicer, and especially to hide away implementation details that we cannot hide otherwise.

Why is it so interesting to hide away implementation details, like the use of lambda expressions to delay evaluation? Well, we could actually decide not to use lambda expressions at all in our expansion. Here is an alternative implementation of the while macro:

(defmacro while (expression &body body)
  `(tagbody
     start (if (not ,expression) (go end))
           ,@body
           (go start)
     end))

Yes, Common Lisp provides the tagbody construct inside which you can use go, i.e., a goto statement! Gotos are not a very good idea to use in your own code, but gotos are very useful for generating efficient code in macros. In this second implementation of the while macro, there is for example no need anymore to allocate space for the closures that the two lambda expressions create, because we can simply jump around the code to delay its execution.

Of course, this is a toy example, so the little gain in efficiency here is probably not worth the effort. However, what is important is that the "API" for the while macro hasn't changed at all compared to the first version. You still write this:

(let ((i 0))
  (while (< i 10)
    (print i)
    (incf i)))

That's one of the important advantages of abstractions: You can change internal implementation details while all the client code can be left unchanged!

And this is the whole point of macros: You can abstract away code idioms that you cannot astract away in any other way, and you effictively have a lot more options to change implementation details without affecting any clients.

Macros are also one of the fundamental reasons for why Lispers like Lisp's strange syntax so much: The syntax is a direct representation of what is elsewhere called the abstract syntax tree, and the fact that the AST is directly available for manipulation as a very simple list data structure makes it so straightforward and convenient to implement and use macros.

If you now have gotten the taste of this, you may want to read some more introductory material about Lisp, like my own opinionated guide, Peter Seibel's excellent book about Common Lisp, or any of the other available tutorials.

Call for Participation: ILC'07

INTERNATIONAL LISP CONFERENCE 2007

Clare College, Cambridge, England - April 1-4, 2007

In cooperation with ACM SIGPLAN

Sponsored by The Association of Lisp Users

The Association of Lisp Users is pleased to announce the 2007 International Lisp Conference will be held in Cambridge, England at Clare College from April 1st to 4th, 2007. This year's program consists of tutorials at beginners' and advanced levels, prominent invited speakers from the Lisp and Scheme communities, an excellent technical session, tours of Central Cambridge, Anglesey Abbey and Ely, and a quintessential English experience: a traditional dinner served in the college's Great Hall. The advance registration deadline is March 11th. The ILC'07 programming contest is also still running until March 3rd.

Schedule

• Saturday, March 31st
  
  
  
  
  • Optional tour of Cambridge
  
  



• Sunday, April 1st
  
  
  
  
  • Tutorials and workshops
    
    
    
    • Ernst van Waning, Extended Tutorial: Common Lisp in One Day
    
    
    • Pascal Costanza, Context-oriented Programming in Common Lisp
    
    
    • Richard Brooksby, Improve your Lisp using the Memory Pool System
    
    
    • Duane Rettig, Optimizing and Debugging Programs in Allegro CL
    
    
  
  



• Monday, April 2nd
  
  
  
  
  • Invited presentations
    
    
    
    • Christian Queinnec, Teaching CS to undergraduates at UPMC
    
    
    • Michael Sperber, It's All about Being Right: Lessons from the R6RS Process
    
    
    • Herbert Stoyan, Lisp: Themes and History
  
  
  • Presentations of accepted papers
  
  



• Tuesday, April 3rd
  
  
  
  
  • Invited presentations
    
    
    
    • Jans Aasman, Scalable Lisp Applications
    
    
    • Ralf Moeller, Building a Commercial OWL Reasoner with Lisp
    
    
    • Manuel Serrano, HOP: An Environment for Developing Web 2.0 Applications
  
  
  • Presentations of accepted papers
  
  
  • Annual meeting of the Association of Lisp Users
  
  
  • Conference banquet
  
  



• Wednesday, April 4th
  
  
  
  
  • Invited presentations
    
    
    
    • Richard Jones, Dynamic Memory Management
    
    
    • John Mallery, Lisp/CL-HTTP
  
  
  • Presentations of accepted papers
  
  



• Thursday, April 5th
  
  
  
  
  • Optional tour of Anglesey Abbey and Ely
  
  

Conference Registration

Conference registration is now open. The advance registration deadline is March 11th. You can get further discounts as an ACM/SIGPLAN and/or ALU member. Registration includes: access to all events, morning and afternoon teas / coffees, self-service lunch, banquet (Tuesday April 3rd), proceedings and hopefully a conference t-shirt. Accomodation is available in Clare College's "Memorial Court". Credit cards and PayPal are accepted, as are cheques (sterling or US dollars) and international bank transfers.

Organizing Committee

• Co-Chairs: Carl Shapiro (SRI International), Pascal Costanza (Vrije Universiteit Brussel)


• Members: Rusty Johnson (ALU), Peter Lindahl (ALU)


• Program Chair: JonL White (The Ginger Ice Cream Factory / ALU)


• Local chair: Nick Levine (Ravenbrook / ALU)


• General correspondence: ilc07-organizing-committee at alu.org

Mailing Lists

General conference announcements are made on a very occasional basis to the low-volume mailing list ilc07-announce. If you're thinking of participating in ILC 2007, you should either join this list or take an occasional look at the archives.

Let's Lisp again!

There are several news about this year's International Lisp Conference to be held in Cambridge, UK in the first week of April.

1. The Register has published a press release about the conference. Nick Levine told me that it's a good idea to "digg" this article (whatever that means ;). Apparently, you can do this directly via this entry at digg.


2. There is a conference poster available for download. Please print it off and distribute it everywhere.


3. Finally, you can show off your Lisp programming skills by providing implementations for the conference's programming contest.

Best Albums of 2006

Here is my list of music recommendation based on what I perceived being the most interesting albums that I have listened to in 2006. This time, it is not a "best of" list, like last year, but a more careful breakdown.

First of all, the best albums that were released this year are as follows.

1. The Knife - Silent Shout.

It took me a while to get into this album. I had seen recommendations in a few places that I check every now and then, and since they are on Mute Records in the US they caught my attention. However, listening to clips of this album didn't help me at first. The first track, "Silent Shout", sounded to me like a uninspired copy of Kraftwerk, and the second one, "Neverland", like an uninspired piece of electronic punk. However, when I have seen the video clip for "We Share Our Mother's Health" in Kevin Holy's Ten Best Music Videos of 2006, I got it. I bought the album, and since then I almost exclusively listen to it. I can strongly recommend "The Captain", "We Share Our Mother's Health" and "Marble House".

2. Kate Havnevik - Melankton.

Another act from Sweden. I have listened to this a lot and considered this to be the best album of 2006 until very recently. This one is especially interesting because it is the very first album I have bought just because of a recommendation at the iTunes Music Store - I have never heard of this singer before. I can especially recommend the following tracks: "Unlike Me", "Nowhere Warm" and "Suckerlove".

3. Motor - Klunk.

I have already discussed this album before. They have supported Nitzer Ebb on their reunion tour, and Motor have been fabulous.

4. Lou Rhodes - Beloved One.

The first solo album from the singer of Lamb. Recommended tracks: "Treat Her Gently", "No Re-Run" and "Beloved One".

There were also new releases Laibach ("Volk") and Tanya Tagaq (Sinaa - I haven't heard of her before, but she was mentioned at Björk's website). Both are quite interesting and impressive, but I still have to make up my mind before final judgment.

Underworld have already started the "RiverRun Project" in 2005, but I only heard about it in 2006. It is a series of mp3 files (together with comprehensive artwork) that you can buy at their website. Since these are not commercially released on CD, they are quite free-form, which is good. I can especially recommend Lovely Broken Thing, which is part 1 in that series.

Apart from these albums, I have finally managed to get a better overview of Swans. Love of Life has always been one of my favorite albums for about ten years now, but I never took the time to take a look at their other releases (which are quite numerous). I have now listened to quite a few of their albums during this year, and I can recommend Children of God / World of Skin as a good starting point because it captures the different musical styles of Swans quite well.

Two good "best of" collections that were released in 2006 are Collected by Massive Attack and "Body of Work" by Nitzer Ebb. Apart from that, I currently listen to Tekel, who capture the sensibilities of 80's electronic pop music quite well.

That's it for now. Looking forward to new releases by Skinny Puppy, Komputer, Nine Inch Nails and Recoil in 2007...