Wednesday, September 29, 2004
Bill Clementson had an interesting write-up about some of the things discussed recently at the Amsterdam Lisp User Group meeting. Among them was the proposal to create a Common Lisp Request For Implementation (CLRFI) process that would facilitate the evolution of Lisp without having to drag everybody into a heavyweight standards body like ANSI.
I suggested something along these lines a few months ago, and the idea has only gotten more firmly etched in my brain the more code I write in CL.
For example, I have been playing around with some amount of network application code. Just about every CL implementation supports a different network interface library. That's just painful. When I complained about this on c.l.l way back when, well-meaning people suggested that I write my own adapter library between some internal, private network API that I should design to whatever the specific implementation API happens to be. While that can be done, the problem here is that it's just painful. People said that I should keep separate in my mind the concepts of a programming language (Lisp) and anything that is OS-specific (like network APIs). The C programming language doesn't specify a network API, they said. Yes, that's true, but I have to believe that was mainly because when C was defined many machines didn't actually have network connections and most programs were not written to be networked. Clearly, the world has changed regarding networking capabilities in the last 20 years. However, C did, from it's earliest days, define a stdio library for file I/O (as documented in K&R). In any case, I'm a firm believer that modern programming languages need some standardized, basic infrastructure in order to facilitate the development of widely-interoperable libraries of application-level functions.
The problem I generally have isn't taking my Common Lisp programs and running them on other platforms. I rarely have to do this. It's being able to take other peoples library source and use it on whatever Common Lisp system I'm currently using. In practice though it's not that big a problem. There are defacto standards around for a lot of things and some very portable libraries. But if the CLRFI implementations could be shipped with the Common Lisp implementations themselves then there would be no porting necessary and that would be a great thing.
In any case, a lightweight process like CLRFI is perfect. Frankly, I understand the arguments where people say that going back to ANSI to extend CL is a mistake. I have dealt with many standards bodies over the years (including ANSI when I was working on FDDI) and the concerns are well-founded. For those that don't know and haven't been exposed to this, there are many types of standards bodies, from heavyweight to lightweight. ANSI is at the heavyweight end of the scale. It's the official standards body for the USA and is the member body representing the USA at ISO. This means lots of policies, procedures, and a general lack of speed. If you want to standardize something like power sockets, fine, ANSI is a great place. Knock yourself out. Fast-moving technology, it's a pain.
At the other end of the scale, we have something like SRFI or CLRFI: just a bunch of like-minded people putting out memos to each other, capturing good ideas and making sure they are recorded in a common location where everybody can know where they are. It's low-overhead and you can make a lot of progress very quickly. No, there's nothing to force anybody to comply with a memo posted on a certain web site, but you'd be amazed how just writing something down and putting some letters on it like "CLRFI" has the power to change things. It's funny that the IETF originally used to be very lightweight, just an organization for assigning numbers and publishing Request For Comments (RFCs) in an orderly way. As the Internet grew and the infrastructure became more important, the IETF started that gradual slide toward bureaucracy.
So, we'll see if the CLRFI idea makes much headway. Personally, I hope it does. I think that CL needs to start adding some basic infrastructure to itself in a standard way. Something like the CLRFI process could start that ball rolling. Who knows, if we're successful, in 20 years we may be as bureaucratic as the IETF is now. We should only be so lucky.
Friday, September 24, 2004
I have finally been getting some good Lisp hacking in. I feel like I have climbed my way out of total "newbieness" and I'm in beginner mode now. My hacking is still not very smooth: lots of trips back to CLHS or CLtL to look up function details, or even to try to find out whether there is a standard function that does what I need. There is nothing like writing a really nice Lisp function for something, only to discover 10 minutes later that Lisp hackers 20 years ago thought of the same thing and provided it in the standard library.
Coming from a C/C++/Java background, here are the things that really strike me about Lisp:
- The parenthesis are really no big deal. I don't know how many years I stayed away from Lisp simply because I was intimidated by the parenthesis. Yes, you notice them for the first week or so, then they gradually fade out of your conciousness and you just go with it. Emacs is a huge win here. If you're reading this and you are staying away from Lisp because you heard all the old jokes about "LISP" standing for "lots of irritating silly parenthesis," you'll have to find a new reason. This one doesn't cut it anymore.
- It is really cool that everything is a function. It's nice being able to plop an IF form in the middle of a complex expression to select between a couple of choices. It's interesting to note that C developed the conditional operator ("?:") to reach the same level of convenience because C makes the distinction between statements and expressions. With Lisp, you don't have to do that, and you get things like IF, WHEN, UNLESS, DO, etc., all returning useful values.
- High-order functions are a BLAST! You can get some effect of high-order functions by passing around pointers to functions in C, but it just isn't done very often and you still don't get closures. You really don't realize how useful this is until you try it. Basically, it allows you to start writing very generic algorithms and parameterizing parts of their function in the same way you're used to parameterizing with data values in C functions. This can be as simple and mundane as a :KEY function parameter provided to extract a data value from a structure passed to the POSITION function, or as complex as you want with something passed to a function like MAPCAR. The ability to use high-order functions eliminates so much redundant code, it's amazing.
- Going along with high-order functions, it's really nice to have anonymous functions. High-order functions would be used a lot less, I suspect, if you had to name every function. It's interesting that this is exactly what C forces you to do to use a function pointer. Because functions are therefore more heavyweight, it's harder to justify using the function-call-by-pointer mechanisms in C programs. In contrast, Lisp makes this trivial. The only downside I have found with LAMBDA functions is that it's sometimes hard to debug a stack trace if you have lots of anonymous functions being used. The debugger output isn't too meaningful.
- Going along with high-order functions and anonymous functions, closures just rock! The ability to reach out and grab a free variable that is in scope is just cool. It allows you to easily squirrel something away for later without having to pass it into a LAMBDA function as a formal parameter or create any storage for it explicitly. As a result, code is a lot shorter and less complex than it would be if you had to manage all that yourself.
- Lisp macros are simply mind-blowing. When you realize how underpowered C macros are, you'll never want to go back. Being able to write code at compile time is just cool. The fact that it allows you to actually add language syntax is even better. And, oh, by the way, Lisp macros wouldn't be nearly as powerful without that Lisp code = list data thing that is assisted by all those parenthesis.
- One thing I still can't seem to get my head around is prefix notation for arithmetic expressions. I have no problem with prefix notation for general function calls, but I seem to crave infix for arithmetic expressions. This is particularly odd since my dad worked at HP since I was a little kid and I grew up on RPN calculators. You would think that prefix would be no big deal, but it seems to be 180 degress from RPN and that somehow screws me up. I find that when I'm writing a big arithmetic expression in Lisp that I'm constantly jumping back to the front of the expression to add in another operator that I somehow left out. I suppose that's because I'm still thinking in RPN. Everything that would come at the end of an RPN expression instead comes first in Lisp. Over time, I'm expecting this to get better.
I'm sure I'm missing a few other things, but that's a good start.
Update: Something I totally forgot about at the time I originally wrote this but which is definitely unique in programming languages: symbols. Yes, you're tempted to think of symbols simply as variables, as with other languages, but they're far more than that. Being able to use symbolic representations for various abstractions is very convenient, both when originally doing development, as well as debugging.
So, the new baby has been quite a handful, but we're managing. Three kids is, well, interesting. I'm not sure what I thought it would be like, but this isn't it. Having a baby by itself isn't an issue--we're pros at that now. It's that the other two kids need so much more attention right at the point when you have almost nothing else left to give, that's the problem.
But, things are finally settling down now to the point where I can get some Lisp hacking done.
Tuesday, September 07, 2004
Wow, it has been almost month since my son arrived. Where did it go?
I guess it's true what they say... That jump from two to three kids means going from man-to-man to zone defense. With my wife caring for the littlest one, I'm wrangling the two preschoolers. Luckily, we have had some extended family staying with us for a bit to help out. Otherwise, we'd be goners...
Needless to say, however, I'm not getting much Lisp hacking done.