November 03, 2007
Lisp success story from Hungary
We are a small group of friends (3-5) who left the "professional" programming scene about one and a half year ago to start our own Lisp business.
In July, 2007 we started working on a project for the Hungarian government with a very tight deadline. In mid October the system showed first life signs and now we can say it is extensively used by thousands of users. In spite of the short timeframe and the remaining glitches it was a surprise even for us that the system is able to handle such a load.
Unfortunately the application is not publicly accessible, so we can't provide a URL, but we can give some of the details. It is used for gathering data from the Hungarian communes to help the budget planning.
The entire system is based on opensource software, it runs on multiple SBCLs behind an apache load balancer. The data is stored in a PostgreSQL server.
We have around 4000 registered users, average 300 online and more than 500 at peak times. There are 7000-10000 logins per day.
We use the following Common Lisp libraries. Some of which are ours and some of which are built by others. Many thanks for their efforts, especially to the SBCL developers!
Data storage/persistence
- cl-perec
- cl-rdbms
- cl-postgres (postmodern)
Web GUI
- cl-dwim (the umbrella project)
- UCW
- yaclml
- parenscript
- rfc*
PDF exporting
- cl-pdf
- cl-typesetting
Unit testing
- stefil
- SLIME and emacs with customizations
MOP
- computed-class
- contextl
Misc
- bind
- alexandria
- iterate
- cl-def
- cl-l10n
- cl-graphviz
- cl-serializer
- defclass-star
- local-time
- cffi
- babel
- verrazano
and more...
Main software components:
- Ubuntu linux 6.10
- PostgreSQL 8.2
- SBCL 1.0.10
- Apache2 (due to various headaches soon to be replaced by a lisp based load balancer)
Hardware:
- 10 nodes with two dual core x86-64 CPUs summing up to 40 cores altogether, each node has 3 Gbytes ram.
- Currently we use one of the nodes as database server but soon it will be replaced with another server with 2 or 4 quad core x86-64 CPUs, 300 GByte disk in raid 5 and 16 GBytes ram.
Some people say Lisp is dead but seems like there's life after death... ;-)
September 03, 2007
ContextL Survey September 2007
ContextL is our first fully implemented and currently most mature programming language extension for COP and is built on top of the Common Lisp Object System (CLOS). ContextL has first been made available to the public in early 2005, and has already been adopted by a number of programmers. We would now like to assess how well ContextL has been received so far.
Please consider participating in our first survey about ContextL - this will help us a lot to develop ContextL and related projects further (like Closer to MOP, etc.).
You can find the survey and more information about it at http://prog.vub.ac.be/~pcostanza/COP/survey.html.
July 26, 2007
ECOOP'07 Lisp Dinner
If you want to attend, please send a short email notice to Hans Hübner. Also let him know if you have special dinner needs or wishes.
June 28, 2007
Lisp meeting in The Hague this Sunday
We're hosting a Lisp meeting in The Hague this Sunday. At least ten
people should be here, with a broad range of experiences and
interests.
For those interested in Lisp businesses, this meeting will be at the
offices of a growing Lisp-using company which already has multiple
commercial projects. You'll be able to see up-close what sorts of
issues arise. I personally hope that these meetings won't just be
episodic affairs, but offer a practical service to interested Lisp
users.
To counterbalance boring business issues, Pascal Costanza is offering
an entertainingly philosophical talk about the history of reflection
and metaprogramming in the Lisp family. (If you were ever curious what
3-Lisp and its "infinite reflective tower" is all about, you'll no
doubt find this interesting.)
This is an informal meeting, so no registration is needed: feel free
to come as you wish, and I hope you enjoy it. If you need directions
or a hotel room, there's more info at: http://wiki.alu.org/BeNeLux.
Tayssir
June 25, 2007
AMOP at Google Book Search
Classes in the Metaobject Protocol
Anyway, the information in that article is still valid, so I can recommend it.
June 03, 2007
Dr. Dobb's Classic Lisp Articles
Surprisingly, Jon Erickson, editor of Dr. Dobb's Journal, responded to that question and explained that they routinely get requests for classic articles that are 15-20 years old, and so they indeed regularly republish such articles from the past. Since they are responsible for the archives of a number of publications, such as AI Expert, C/C++ Users Journal, BYTE, and Computer Language, there are probably quite a few nuggets to be rediscovered here.
For example, earlier this year they have already published an article by Paul Graham about graphic objects, originally from 1988. Just yesterday, they have published a very interesting article by Nick Bourbaki about dynamic programming. Nick Bourbaki is actually Richard Gabriel's pseudonym that he used in some of his writings in the early 1990's - see Worse Is Better and also Nickieben Bourbaki.
May 17, 2007
European Lisp Workshop '07 - news
- We are happy to announce that Alexander Repenning will be giving a keynote presentation about Antiobjects: Mapping Game AI to Massively Parallel Architectures using Collaborative Diffusion.
- The submission deadline for papers, essays and breakout group proposals has been extended to May 31. Notifications of acceptance will be sent out on June 8, one week before the early registration deadline.
See the workshop website for more information.
April 24, 2007
ELW'07
Reflective Layer Activation in ContextL
This paper describes some aspects of the metaobject protocol of ContextL. This is not exactly the same version as presented at the symposium, but the paper has been updated to reflect the new API in ContextL 0.4. However, apart from the resulting changes of function names and a few other minor edits, the contents have remained the same.
April 21, 2007
ContextL 0.40.
There have been some changes in the API which are listed at the ContextL homepage. These are mostly cosmetic changes, like better names for functions, etc.
However, there are also some new features. The most interesting ones are probably several readers for the main ContextL metaclasses, such that the introspective part of ContextL is now complete, and a function current-layer-context. The latter can be used to capture the set of currently active layers, which can later be reinstalled for example with funcall-with-layer-context. Current-layer-context should be interesting for example to "inherit" active layers to/from threads in multithreaded Common Lisp implementations, or to capture active layers alongside continuations such that they can both be reinstalled together later on. This could, for example, be interesting in conjunction with continuation-based web frameworks.
For more details, see the ContextL homepage and announcements/discussions in the closer-devel mailing list.
April 09, 2007
My impressions of ILC'07
I am very happy that ILC'07 turned out to be so successful. This hasn't been always clear during the preparations, so I guess all the organizers breathed a sigh of relief when the numbers of registrations started to increase (and we suddenly had to come up with ideas how to manage a situation in which we nearly had too many registrations ;).
Among the "generic" highlights, I personally enjoyed the atmosphere a lot. I am convinced that this is mainly due to the fact that almost everyone had their accommodation at the same place. In my experience, conferences where this is the case are typically more successful - you already see the participants in the morning during breakfast, you have a lot more time to talk to each other, and so on.
For me, it was also nice to see so many people again. On average, the number of smart people is a lot higher at Lisp meetings than elsewhere, so there is always a chance to have some very interesting discussions with very different perspectives provided by everyone.
About the conference itself, I am happy that the program chair (JonL White) has carefully selected a very interesting set of presentations. Almost all talks were of high quality, covering a broad range of topics with only a couple of downers. My personal highlights have been:
- Antonio Leitao's work on translating Java source code to idiomatic (!) Common Lisp sources. He has already done the reverse (translating a subset of Common Lisp to Java) in the past and is now heading towards a solution that potentially improves the situation on the library front for Common Lisp. The interesting aspect here is the focus on generating idiomatic code, i.e., code as if it had been written manually by experts in the respective target languages.
- Mike Sperber's illuminating talk about R6RS Scheme. I had some fears wrt having this presentation for an audience that mostly consists of Lispers, but non-Schemers. However, Mike has given a very entertaining talk and helped everyone to have a better idea about the goals of R6RS. (IMHO, it's important that Schemers and Lispers talk more to each other because both sides can learn from each other. I'm glad that ILC'07 has helped improve the situation a little bit.)
- Manuel Serrano has given a very impressive presentation about his HOP framework for developing web applications in Scheme. It's good to see that there are a couple of approaches that head in the right direction of enabling the creation of GUIs that makes users' lives easier instead of focusing on solving technicalities of accidental elements in the underlying technology (for example, the infamous "back" button). I see HOP in the same line as Marc Battyani's (unfortunately unpublished) web framework, but as far as I can tell, Manuel has gone a few steps further.
- Alexander Repenning had a paper about X-expressions in XMLisp, which sounds a lot more boring than what he actually presented. This is not yet another library for dealing with XML, but actually modifies the Common Lisp reader (parser) to treat XML as syntax for literal CLOS objects. The applications that he has built on top of that technology are very impressive indeed.
- Clemens Fruhwirth's Liskell is actually a very simple idea. He takes GHC - a widely used implementation of Haskell - and replaces the Haskell syntax with s-expressions. This enables, among other things, the inclusion of macros. I have always wanted to take a closer look at Haskell, but its syntax always put me off. Now I don't have an excuse anymore. ;)
For obviously personal reasons, I am also very happy that Charlotte Herzeel's presentation about HALO is so well-received - she got a lot of positive feedback for her presentation. However, it is only an overview of HALO, there is still more to come. I hope that we can report on some exciting results soon.
On the negative side about ILC'07, it seems to me that the program was maybe a little too packed - we had to get up very early in the morning (very hard for me ;), and there was maybe not enough time to engage in discussions or even do some work with other attendees. We have tried to establish a breakout group format, but apparently we haven't found the right approach yet. Ralf Mattes suggested a birds-of-a-feather-sessions format to me, so maybe that's something we should look into for the future.
A real downer was the annual meeting of the Associations of Lisp Users (ALU). It wasn't the first time they promised to become more "professional", but their presentation to the ALU members was worse than average. I hope they will manage to get their act together. Otherwise, it will be hard for them to be taken seriously by the Lisp community in the long run. It is worth noting, though, that they are responsible for backing the International Lisp Conference, so they must be doing something right... ;)
Fortunately, the rest of the conference didn't suffer from the ALU meeting - the day after was as good as, or maybe even a little more exciting than, the other days on all levels.
Next on the schedule is the European Lisp Workshop, co-located with ECOOP 2007 and taking place in Berlin on July 30, and most likely sometime early next year another instance of the European Common Lisp Meeting.
March 08, 2007
Lisp Bleep Survivalism
- Registrations for the International Lisp Conference 2007 are coming in like crazy, and we are soon running out of rooms at the Clare College. There are only three days left until the early registration deadline with reduced fees, so there are enough reasons to better hurry up. ;)
- New videoclips by Motor and Nine Inch Nails.
- This is the last posting about music to this blog, since the postings here occur, for example, at Planet Lisp. So in order to stop boring Lispers about music they are not interested in, I will post all music-related ramblings to the freshly brewed infox_blog instead.
February 27, 2007
What is the point of macros?
Let's take a common idiom from Java: Whenever you want to open a file, process its contents, and close it again afterwards, you have to write code roughly like this:
FileInputStream in = new FileInputStream(filename);
try {
doSomething(in);
} finally {
in.close();
}
Note how the orange code is the code that you are actually interested in: That's the part that states which file you want to open, that you want to open it for reading, and that you want to do something with the file. The rest is just idiomatic code that you have to write just to make the machinery work: You declare a variable outside of the try block (this is important for the visibility of the variable!), and you have to put the call of the close method in a finally block to ensure that the file is always closed even when an exception occurs.
The problem here is that you have to write this code over and over and over again, and everytime you write this code, you have to get the details right. For example, when I was still a Java hacker, I always have put the variable declaration inside the try block, just to get the error message from the compiler that this is wrong. This is annoying - it would be nice if I didn't have to think about this. (I have actually gotten it wrong again in the first version of this posting. Go figure.)
Here is the same code in Common Lisp:
(let ((in (open filename)))
(unwind-protect
(do-something in)
(close in)))
This is not yet an improvement: The orange part of the code is still deep down in other code that just ensures that the machinery for opening and closing the file works. I still have to declare the variable first (here with a let), I still have to use the Lisp-equivalent of try-finally (here it's called unwind-protect), and I still have to ensure that the closing of the file happens as part of the clean-up step of an unwind-protect.
However, Lisp gives us a way to get rid of the idiomatic code. What we would actually like to write is this:
(with-input-file (in filename)
(do-something in))
This code states exactly what we want and nothing else: We want to execute some code with an open file bound to some variable, and we implicitly want to ensure that it is closed again at the end of this code fragment.
Here is a macro that defines this construct:
(defmacro with-input-file ((variable filename) &body body)
`(let ((,variable (open ,filename))
(unwind-protect
(progn ,@body)
(close ,variable))))
Here is an explanation of the code:
- The macro is defined with defmacro and we name it like the language construct we want to build. It takes two variables: a variable name and a piece of code that will give us the filename in the resulting code. It also takes a body of code for the with-input-file construct.
- The macro creates a piece of code that the Lisp compiler should use instead of the original code that uses the with-input-file construct: We basically define a code template in which the variable, filename and body spots are filled in with the parameters that the macro receives.
- Some minor details: The progn groups a number of expressions into one - this is similar to grouping code in a pair of curly braces in Java. The backquote, comma and comma-at notations are there to clarify how the parameters should be inserted - that's something that you can better learn in good Lisp tutorials.
Although writing macros takes a little practice, it is very straightforward once you are used to doing this. Macros help you to avoid typing a lot of idiomatic code: whenever you notice some coding pattern, you can put it in some macro and then use it instead.
Syntactic Abstractions
Let's get a little closer to the heart of what macros actually are: They provide you with a way to define syntactic abstractions. Abstractions are used for hiding away implementation details that client code should not be interested in. For example, in classes you can define private fields and methods for internal use and define a public interface that client code has to go through to ensure that the internal details are always used properly. Likewise, with functional abstractions - closures - you can pass around a function that can refer to variables in its lexical environment without giving anyone direct access to these lexical variables.
Many implementation details can be hidden very well with such abstraction mechanisms. However, others cannot be hidden well, and this is when macros become interesting.
For example, assume that your language doesn't provide any iteration constructs but requires you to use recursion instead. You can relatively easily build your own iteration constructs - for example a while function - like this in Lisp:
(defun while-function (predicate block)
(if (not (funcall predicate)) (return))
(funcall block)
(while-function predicate block))
This function works as follows: It takes a predicate function and a block of code, also provided as a function. When calling the predicate (with funcall) returns false, the while-function returns. Otherwise, the block is executed (again with funcall) and then the while-function calls itself again.
Here is an example of using the while-function:
(let ((i 0))
(while-function (lambda () (< i 10))
(lambda ()
(print i)
(incf i))))
This code fragment increments i from 0 up to 10 and prints i at each step. This piece of code doesn't look very nice. One reason is that Lisp's lambda construct is somewhat lengthy. Smalltalk, Ruby, and other languages have nicer syntax for lambda expressions, and would make this code shorter. However, even then there is a problem here: We still have to write idiomatic code to write a while loop although we are actually not interested in the idiom. Here, the idiomatic element is the use of a lambda expression to ensure that some code is not immediatily executed, but only later under the control of the while-function. However, what we actually want to say is this, which doesn't contain any idiomatic code at all:
(let ((i 0))
(while (< i 10)
(print i)
(incf i)))
And, as you might have guessed, here is a macro to implement this while construct:
(defmacro while (expression &body body)
`(while-function (lambda () ,expression)
(lambda () ,@body)))
This macro uses the while-function in the code that it creates. This is actually one of the typical ways to write Lisp code: We first define the functional (or object-oriented, or imperative, or whatever) abstractions, and then we add some syntactic layer on top to make the code look nicer, and especially to hide away implementation details that we cannot hide otherwise.
Why is it so interesting to hide away implementation details, like the use of lambda expressions to delay evaluation? Well, we could actually decide not to use lambda expressions at all in our expansion. Here is an alternative implementation of the while macro:
(defmacro while (expression &body body)
`(tagbody
start (if (not ,expression) (go end))
,@body
(go start)
end))
Yes, Common Lisp provides the tagbody construct inside which you can use go, i.e., a goto statement! Gotos are not a very good idea to use in your own code, but gotos are very useful for generating efficient code in macros. In this second implementation of the while macro, there is for example no need anymore to allocate space for the closures that the two lambda expressions create, because we can simply jump around the code to delay its execution.
Of course, this is a toy example, so the little gain in efficiency here is probably not worth the effort. However, what is important is that the "API" for the while macro hasn't changed at all compared to the first version. You still write this:
(let ((i 0))
(while (< i 10)
(print i)
(incf i)))
That's one of the important advantages of abstractions: You can change internal implementation details while all the client code can be left unchanged!
And this is the whole point of macros: You can abstract away code idioms that you cannot astract away in any other way, and you effictively have a lot more options to change implementation details without affecting any clients.
Macros are also one of the fundamental reasons for why Lispers like Lisp's strange syntax so much: The syntax is a direct representation of what is elsewhere called the abstract syntax tree, and the fact that the AST is directly available for manipulation as a very simple list data structure makes it so straightforward and convenient to implement and use macros.
If you now have gotten the taste of this, you may want to read some more introductory material about Lisp, like my own opinionated guide, Peter Seibel's excellent book about Common Lisp, or any of the other available tutorials.
February 21, 2007
Call for Participation: ILC'07
INTERNATIONAL LISP CONFERENCE 2007
Clare College, Cambridge, England - April 1-4, 2007
In cooperation with ACM SIGPLAN
Sponsored by The Association of Lisp Users
The Association of Lisp Users is pleased to announce the 2007 International Lisp Conference will be held in Cambridge, England at Clare College from April 1st to 4th, 2007. This year's program consists of tutorials at beginners' and advanced levels, prominent invited speakers from the Lisp and Scheme communities, an excellent technical session, tours of Central Cambridge, Anglesey Abbey and Ely, and a quintessential English experience: a traditional dinner served in the college's Great Hall. The advance registration deadline is March 11th. The ILC'07 programming contest is also still running until March 3rd.
- Saturday, March 31st
- Sunday, April 1st
- Tutorials and workshops
- Ernst van Waning, Extended Tutorial: Common Lisp in One Day
- Pascal Costanza, Context-oriented Programming in Common Lisp
- Richard Brooksby, Improve your Lisp using the Memory Pool System
- Duane Rettig, Optimizing and Debugging Programs in Allegro CL
- Tutorials and workshops
- Monday, April 2nd
- Invited presentations
- Christian Queinnec, Teaching CS to undergraduates at UPMC
- Michael Sperber, It's All about Being Right: Lessons from the R6RS Process
- Herbert Stoyan, Lisp: Themes and History
- Presentations of accepted papers
- Invited presentations
- Tuesday, April 3rd
- Invited presentations
- Jans Aasman, Scalable Lisp Applications
- Ralf Moeller, Building a Commercial OWL Reasoner with Lisp
- Manuel Serrano, HOP: An Environment for Developing Web 2.0 Applications
- Presentations of accepted papers
- Annual meeting of the Association of Lisp Users
- Conference banquet
- Invited presentations
- Wednesday, April 4th
- Invited presentations
- Richard Jones, Dynamic Memory Management
- John Mallery, Lisp/CL-HTTP
- Presentations of accepted papers
- Invited presentations
- Thursday, April 5th
Conference Registration
Conference registration is now open. The advance registration deadline is March 11th. You can get further discounts as an ACM/SIGPLAN and/or ALU member. Registration includes: access to all events, morning and afternoon teas / coffees, self-service lunch, banquet (Tuesday April 3rd), proceedings and hopefully a conference t-shirt. Accomodation is available in Clare College's "Memorial Court". Credit cards and PayPal are accepted, as are cheques (sterling or US dollars) and international bank transfers.
Organizing Committee
- Co-Chairs: Carl Shapiro (SRI International), Pascal Costanza (Vrije Universiteit Brussel)
- Members: Rusty Johnson (ALU), Peter Lindahl (ALU)
- Program Chair: JonL White (The Ginger Ice Cream Factory / ALU)
- Local chair: Nick Levine (Ravenbrook / ALU)
- General correspondence: ilc07-organizing-committee at alu.org
Mailing Lists
General conference announcements are made on a very occasional basis to the low-volume mailing list ilc07-announce. If you're thinking of participating in ILC 2007, you should either join this list or take an occasional look at the archives.
February 05, 2007
Let's Lisp again!
- The Register has published a press release about the conference. Nick Levine told me that it's a good idea to "digg" this article (whatever that means ;). Apparently, you can do this directly via this entry at digg.
- There is a conference poster available for download. Please print it off and distribute it everywhere.
- Finally, you can show off your Lisp programming skills by providing implementations for the conference's programming contest.
January 21, 2007
Best Albums of 2006
First of all, the best albums that were released this year are as follows.
1. The Knife - Silent Shout.
It took me a while to get into this album. I had seen recommendations in a few places that I check every now and then, and since they are on Mute Records in the US they caught my attention. However, listening to clips of this album didn't help me at first. The first track, "Silent Shout", sounded to me like a uninspired copy of Kraftwerk, and the second one, "Neverland", like an uninspired piece of electronic punk. However, when I have seen the video clip for "We Share Our Mother's Health" in Kevin Holy's Ten Best Music Videos of 2006, I got it. I bought the album, and since then I almost exclusively listen to it. I can strongly recommend "The Captain", "We Share Our Mother's Health" and "Marble House".
2. Kate Havnevik - Melankton.
Another act from Sweden. I have listened to this a lot and considered this to be the best album of 2006 until very recently. This one is especially interesting because it is the very first album I have bought just because of a recommendation at the iTunes Music Store - I have never heard of this singer before. I can especially recommend the following tracks: "Unlike Me", "Nowhere Warm" and "Suckerlove".
3. Motor - Klunk.
I have already discussed this album before. They have supported Nitzer Ebb on their reunion tour, and Motor have been fabulous.
4. Lou Rhodes - Beloved One.
The first solo album from the singer of Lamb. Recommended tracks: "Treat Her Gently", "No Re-Run" and "Beloved One".
There were also new releases Laibach ("Volk") and Tanya Tagaq (Sinaa - I haven't heard of her before, but she was mentioned at Björk's website). Both are quite interesting and impressive, but I still have to make up my mind before final judgment.
Underworld have already started the "RiverRun Project" in 2005, but I only heard about it in 2006. It is a series of mp3 files (together with comprehensive artwork) that you can buy at their website. Since these are not commercially released on CD, they are quite free-form, which is good. I can especially recommend Lovely Broken Thing, which is part 1 in that series.
Apart from these albums, I have finally managed to get a better overview of Swans. Love of Life has always been one of my favorite albums for about ten years now, but I never took the time to take a look at their other releases (which are quite numerous). I have now listened to quite a few of their albums during this year, and I can recommend Children of God / World of Skin as a good starting point because it captures the different musical styles of Swans quite well.
Two good "best of" collections that were released in 2006 are Collected by Massive Attack and "Body of Work" by Nitzer Ebb. Apart from that, I currently listen to Tekel, who capture the sensibilities of 80's electronic pop music quite well.
That's it for now. Looking forward to new releases by Skinny Puppy, Komputer, Nine Inch Nails and Recoil in 2007...
October 22, 2006
International Lisp Conference '07 - Call for Papers
September 20, 2006
Freely downloadable books by Niklaus Wirth.
These books have influenced me a lot. Niklaus Wirth's main strength is a focus on the essentials, which is especially apparent in his concise programming style. I can especially recommend Compiler Construction which basically contains all you need to know about compilers in a very thin book (unless you decide to make this your field of expertise), and Project Oberon which contains the complete source code of a full operating system (including a file system, a garbage collector, a GUI, and so on).
See also the ETH Oberon Home Page for more information about Oberon and related materials.
CDR 0 & 1 finalized.
These CDRs were mostly test runs of the CDR process, although they are of course also useful in their own right. (The CLOS MOP had already been published over 15 years ago and has long been proven useful, so didn't really need an official "stamp.")
See the CDR website for more information.
Furthermore, we have set up a CDR Blog where we will post CDR-related announcements - actually the same ones as in the cdr-announce mailing list.