Friday, December 30, 2005
Okay, it's that end-of-year time agan. Time to reflect back on all that's happened this year and plan for the next. I was going through my weblogs this morning and decided it would be fun to pull out the most-hit pages and make a Letterman-esque Top 10 list. Note that I did a straight ranking of pages, irrespective of the date they were created, ignoring things like monthly index pages or the books page. About half the pages were from 2004. Possibly, this reflects the amount of time these pages have been available; some newer pages suffer because there simply hasn't been enough time for them to garner enough hits and because there are fewer back-links to them. Be that as it may, here's the roundup!
10 -- More RAD Thoughts. A discussion of some feedback on an earlier post about RAD development in Lisp. Also, I cover some thoughts on the various license terms of commercial Lisp implementations.
9 -- XML + ASN.1 = faster? (or Stones Don't Float). A discussion of Fast Infoset and the idea of XML compression in general.
8 -- More REST and continuations. A quick hit announcement of Anton van Straaten's talk about REST and continuations at LL4. One of the comments gives a link to the slides.
7 -- Lisp RAD. The original post wondering about Lisp RAD that prompted the follow-up at #10. Hmmm... I wonder where SK8 is today? I haven't heard anything about it in a while.
6 -- Moving to the darcs side. A discussion about the darcs revision control system and a comparison of it to Arch. I think this post has been popular because it made its way onto the darcs wiki. I should write some more about my experience with darcs. The summary is that it has been quite positive. I use darcs all the time now and I find it quite nice. I don't think it takes the place of Subversion for some larger, more centralized projects, but it's great for a small group of developers or people doing distributed development. I have noticed that a lot of Lisp projects that once used Arch have now gone over to the darcs side, too.
5 -- Darcs and Arch revisited. A follow-up to the original darcs posting with more info about my experiences.
4 -- Web application design: the REST of the story. This posting describes the REST web application architecture in some detail. I think this posting has gotten a bunch of cross-links in the web development community, particularly in the REST vs. continuations debate. Following this, Marco Barringer, the author of Uncommon Web (UCW), and I ended up going back-and-forth regarding REST and continuations in a long email exchange.
3 -- DNS Message Decoding: A Case Study Comparing Java and Common Lisp. Written May of 2004, this posting continues to be a favorite, which is quite remarkable because it's actually a bit burried on the Finding Lisp site, mentioned in just a single blog posting with no top-level links. This posting is a comparison I did of the DNS packet decoding code I wrote in Lisp for my Resolver library versus some similar code I once wrote in Java. I did this comparison to convince myself that what Paul Graham and others have said about Lisp reducing the amount of code you have to write was actually true. In this case, Lisp certainly took down Java.
2 -- ILC 2005, Wednesday report (late). This posting chronicles part of my experience at this year's ILC at Stanford. In short, this was a fun conference, for a couple of reasons. First, many of the talks were quite enjoyable, even (particularly?) the ones with some controversial content (Patrick Dussud's Re-inventing Lisp for Ubiquity comes to mind -- "He's a witch! Burn him!" ;-). Second, I got to meet a bunch of nice people face-to-face (among many others, Mastenbrook, Wiseman, Fix, O'Connor, Tilton, Costanza, and Seibel, whom I had already met at various BA Lispniks events, but who signed my copy of Practical Common Lisp).
1 -- Reddit and Lisp psychosis. Well, what can I say. In spite of being written only a couple weeks ago, this was far-and-away the top runner this year. The response to this post bowled me over. I somehow got to the #1 ranking on Reddit for a couple days (before the copulating deaf couple knocked me out ;-), which garnered a bunch of visibility for this blog, some of which seems to have stuck. Hello to the new readers. Further, it seems to have struck a cord with a bunch of people that have been harboring frustration with Lisp's "freeze-dried-since-1994" state. All the energy surrounding the Reddit event, not necessarily my posting, seems to have erupted into a flurry of activity, including the completion of a port of SBCL to Windows, the CL Gardeners project, and my own work on network programming APIs (and I'm sure even more projects that I haven't even heard about yet).
Here's to an industrious 2006. Happy New Year!
Tuesday, December 27, 2005
This is going slower than I would have liked, but I have a start on a paper proposing a sockets API. Right now, the paper simply analyzes the current sockets APIs of a few CL implementations. There is a "to be written" section where the eventual proposed API will go. I have a start on that, too, but I held it back because I wrote a lot of it half-asleep on the plane back from NYC a couple weeks ago, and I know there are a bunch of blunders in it. The rest of it seems sane enough to start spreading around, however. If anybody has any feedback, please shoot it to me at dave at findinglisp.
Thanks to Thom Goodsell and Iban Hatchondo for their comments.
Friday, December 16, 2005
The other day I posted about Das Keyboard being an interesting twist on keyboards, being unlabeled. At Interop this week, I happened to run into a company called Zeomi, who makes a keyboard with an added tab key. Basically, they put a duplicate tab key under the arrow keys, over by the keypad. They say that if you do lots of data entry into either spreadsheets or forms where tab is used to move among fields, this can really speed things up, presumably because you can be flipping through papers with your left hand while your right hand does everything on the numeric keypad. I did find it humorous that they're trying to patent this.
Also, there were lots of comments on my comments regarding Dvorak keyboard layouts. I should give you some background. I'm already a 70+ word-per-minute Qwerty touch-typist (test yourself over at powertyping.com). Back in 1997 or so, I actually switched to Dvorak for about a month and I got reasonably good (almost over the hump). I found that I kept running into a few of the various standard problems, basically as documented in this page of Dvorak information. In the end, I switched back. My Dvorak never did get quite as fast as my Qwerty, but I only stuck with it for a month or so. I'm sure I would have been at least as fast on Dvorak as I am at Qwerty if I had only kept going for another week or two. I do admit that Dvorak was much more comfortable than is Qwerty. You definitely can type a lot more words without moving your fingers from the home row. In the end, the biggest pain for me was being able to type Ctrl-X, -C, and -V for cut/copy/paste with my left hand while holding the mouse in my right. Dvorak just messes with that and it was always an annoyance. I'd actually be interested in how people cope with things like Emacs with a Dvorak keyboard, where there are lots of keychords.
Anyway, if you have become a Dvorak convert, I can certainly understand why. Every now and then, I remap my keyboard and type in Dvorak for an hour or two. And then I go back to Qwerty. ;-)
Just got back from New York tonight. Interop, the show I was out there for, seemed a bit small, but all the various trade shows have gotten that way. I had a speaking slot this morning, talking about open source networking. Looks like I got out of town just in time, what with the transit strike probably happening Friday morning. It was fun to drop by Rockefeller Center yesterday and see everybody ice skating. Then we ran back to the hotel because everybody's ears were frozen; it was bitter cold. And who could believe we got a sweet deal on hotel rooms at the Intercontinental for only $290 per night. Most everything in Manhatten is going for $300+ per night; even the dumpy hotels.
Thursday, December 15, 2005
Holy exposure, Batman! As if Reddit wasn't enough, while I was away on a business trip this week, I somehow made the Blogs of Note section on the Blogger homepage. I wonder if the copulating deaf couple will knock me out of that, too? ;-)
Saturday, December 10, 2005
Wow! I was just browsing through my web logs and noticed a lot of referrals coming in from Reddit. I went over there and it turns out that my blog entry titled "Reddit and Lisp psychosis" is #1 on Reddit's list. Yikes!
My consulting has got me working in downtown Palo Alto, CA, most days now (at least through the end of December). I read Dan Moniz's blog about being in PA speaking to CommerceNet on Friday but was still on vacation at the time (sorry, Dan, would have loved to meet you). That said, if anybody else works around PA and wants to grab a lunch or a coffee and talk Lisp, drop me a note (dave at findinglisp). I work right downtown, so it's easy enough to walk to just about anyplace on University Ave. or one of the close side-streets.
I get a smile reading just about anything Nikodemus writes. ;-) Often it's because he right.
Friday, December 09, 2005
I went away on a family vacation at the first part of this week and just got back last night. During that time, I completely missed the whole Reddit scandal. It seems like the guys at Reddit ended up choosing to rewrite Reddit in Python because Lisp just wasn't working for them. To some, this seems to be a slap in the face of Lisp. However, if you look at spez's blog entry, you can see that the rationale for choosing Python was pretty sane.
In particular, this paragraph was interesting (also quoted by John at Lemonodor):
Emacs and SLIME are a killer combination, but I develop on a Mac, and reddit.com is a FreeBSD box. On my Mac, my choices of threaded Lisp implementations was limited to OpenMCL, and in FreeBSD it's CMUCL. Because of the low-level socket and threading code we had to write, reddit would not run on my Mac, and I was always tethered to our FreeBSD development server. Not being able to program offline is a pain.
In a comment on Lemonodor, Steve Huffman said:
The biggest trouble that plagued us was that we could never quite get Lisp reddit stable enough to sleep at night. There were weird threading issues that would bring the site to its knees a couple times a day and required constant monitoring.
Now, here's where the Lisp psychosis comes in. Rather than suggesting that the guys at Reddit were dopes, or that they didn't try hard enough, or that they should have done such and such a workaround, blah, blah, I wish Lispers would step up and internalize that the Reddit experience was a great case study and that the community should work to solve the issues it raised. These guys did not have a bad Lisp experience. In fact, they are quite complimentary of Lisp. For example:
Lisp is an amazing language. After spending the entire summer working entirely in Lisp, it's nearly impossible to work in another language and not say to myself, "If only this were Lisp..." Lisp has many goodies that make programming in it a joy: the REPL, macros and the lack of syntax are some. I won't go into the details, but rest assured, it's cool. People become Lisp zealots for a reason.
So here's what I took away from the Reddit feedback:
- Lisp is a great language. Keep this point in mind. The Reddit developers gave Lisp compliments; they didn't 'diss it.
- Lisp has a balkanized feature set. Some (necessary!) things to build modern applications are not cross-platform. When you have to work on multiple machines and environments, which is more and more the norm these days, there are no open source implementations of Lisp that run across the dominant environments without differences. (CLISP comes the closest, but you may or may not be able to tolerate its GPL license terms.) Because these features are not standardized, you're left writing compatibility layers if you want things to work across platforms.
- In particular, networking and threading are problem areas.
- Lisp libraries are scant. This is a well known problem in the Lisp community. The standard, basically valid, response is "Jump in and help us write some more libraries." (Kudos to Kenny Tilton for driving this line hard. He's right, but there's also a bit more to it than that.)
- The library problems are compounded by the balkanization of feature set. In some cases you can find something that sort of works, but it may work on another implementation, not yours. If it has any dependencies on the problem areas of threads and networking, you've got a long road of tweaking ahead to get it to work. This time is better spent getting on with your real task.
- Edi Weitz (yet again) wins the Lisp Superhero award for creating the best libraries out there, bar none. Seriously, if anybody aspires to create libraries that are well-used, go take some cues from Edi. His code is always high quality, his APIs and implementations are always complete, he's absolutely responsive to problem reports, he provides great documentation, and he often tests his code across multiple implementations, trying to make them as cross-platform as possible. In short, you couldn't expect better service and support from a commercial vendor, and Edi releases his code as open source.
Okay, all that said, what's the constructive response here? My suggestions are:
- First, stop grumbling and suggesting workarounds to the Reddit folks. They did the Lisp community a great service by documenting their experiences. Rather than harrassing them, sit down and talk with them to get more info.
- Next, focus on the foundational balkanization problems. In my opinion, the biggest issues with Lisp are the lack of standard (defacto or otherwise) networking and threading APIs. If those were in place, it would be a lot easier to get libraries that worked all over the place. Would this solve everything? No, but it would go a long way and would enable lots of other innovation on top of that foundation instead of having everybody spending time creating compatibility libraries and generally re-inventing the wheel.
- Work on getting a good cross-platform, open source Lisp implementation with a liberal license. Like I said previously, I really like CLISP, and I use it for developing on Windows, but the license is not suitable for all code since it all-but-forces your code to be released as GPL. I'd really love to see SBCL or CMUCL ported to Windows, with the full set of functionality. I'd also love to see SBCL's baseline functionality present on all platforms (currently threading only works on Linux, for instance, not BSD). By the way, this is no knock on any other Lisp implementation. I just happen to use SBCL on Linux and think it rocks. If OpenMCL can make the jump off of Mac onto other platforms, that would be great. Or maybe GCL can do the job. I don't know the exact route, but the fact is the lack of a baseline functionality across all platforms is hurting us.
Can I do all the above? Nope. I've tried grokking the internals of SBCL and I'm only qualified to kibitz around the margins. I stand in awe of guys like William, Christophe, Dan, Nikodemus, Gabor, Juho, Alexey, and many others (Update: I added to this list twice already because I felt so bad for leaving out somebody's name--sorry William and Nikodemus. If you're a significant SBCL contributor--or even and insignificant one--believe me when I say I stand in awe of you too ;-). I do try to help out SBCL by reducing the friction for newbies to get started with Lisp, compiling the RPM binaries that are available on SourceForge and maintaining the FedoraLisp.org Yum repository.
That said, I'm going to start working on the networking API issues. You'll see a document from me posted over the next couple of weeks. It's been in process for a few months, but I finally got a hankering to finish it with this Reddit bru-ha-ha. If you're interested in networking APIs and have a desire to help in such an endeavor, please drop me a line (dave at findinglisp). I'd love to spread the work around. If somebody else wants to grab the threading stuff by the throat that would be great.
Above all, stay productive. Use the Reddit feedback to motivate you to make Lisp better. Whatever you do, stop suggesting workarounds to the Reddit guys or making them feel bad for the choices they made. They seem like smart guys, so let's assume that they did what they did knowing all the options (they do have Paul Graham on the board, so I'm guessing that they talked about this before they did the rewrite). They have moved on and our job is to make any such workaround unnecessary for the next crew that tries to use Lisp.
And before anybody says it, yes I know that the commercial Lisp vendors have solved some of these problems. I have discussed before some of my thoughts about the various license terms the vendors use. It isn't that those terms are bad in the abstract, they just don't allow those implementations to solve the problem I'm interested in. In short, I think it would help drive Lisp adoption if a common, cross-platform, free, open source version existed, in the same way that GCC has helped democratize programming in C. The next-best alternative would be a $99 "Turbo Lisp" environment ala Borland's language products in the 1980s/1990s. Perhaps the Lispworks guys will offer such a thing (hint, hint ;-).