Books of Note

Practical Common
LispThe best intro to start your journey. Excellent coverage of CLOS.

ANSI Common
LispAnother great starting point with a different focus.

Paradigms of Artificial Intelligence
ProgrammingA superb set of Lisp examples. Not just for the AI crowd.

Monday, September 22, 2008

PHP vs. Lisp: Unfortunately, it's true... 

I just found this today while browsing Reddit: Practicality: PHP vs. Lisp?

This is exactly the psychology that goes through the head of everybody who hears about Lisp and decides to check it out. Summary: Lisp is awesome, except for the multitude of trivially annoying things that make it much less so, to the point that everybody just throws in the towel and uses PHP or Ruby or whatever.

I hate reading articles like this because I know that it's basically all true. I also know that within 24 hours, the comments section of that blog post will be filled by Lisp apologists who keep suggesting workarounds and fixes. These are the same people who went nuclear a couple of years ago when Reddit got rewritten in Python, and it's all the same old arguments.

If Brian Carper's blog post annoys you, stop apologizing and start innovating. Languages exist in a market place and right now Lisp marketing itself quite poorly. Yes, I know that "marketing" is rejected by techie types who think that it should never matter, but as a marketing guy who markets to techies, I'll tell you that it absolutely does. And those who reject marketing will flat-out lose to those who understand it.

As a "customer" of a particular language, nobody wants to hear how their particular issue is unimportant in the grand scheme of things. Nobody wants to hear that such and such a problem was "solved" 20 years ago with this particular hack or workaround that only Lisp gurus know about. Nobody wants to hear that it's a simple job to port that particular library over to that particular implementation. People have a ton of free language options these days, and if Lisp isn't going to compete in terms of ease of use and productivity, then people will choose languages that will.

Note that theoretical ease-of-use doesn't count. Everybody will probably admit that Lisp macros are all-that-and-a-bag-of-chips, and building DSLs is way more productive than writing boilerplate code, and nevermind CLOS and closures, etc. Those are GREAT things that Lisp brings to the party, but they are completely useless if a newbie can't crank out a web page in 30 minutes like he can with PHP.

If you step back for a second and really try to listen to the arguments against Lisp, you'll find that there are two classes of gripes: fundamental dislikes and competitive disadvantage. Things like griping about the parenthesis are a fundamental dislike. Some people don't like parenthesis. Fine. Lisp is not for you. Parenthesis and sexprs are pretty fundamental to Lisp. They're part of its DNA. If you don't like them, then you don't like Lisp itself. There's no issue with that. Just use something like Java. Most Lisp programmers hate Java syntax, too. If you're really enlightened, take a look at Dylan. In short, as a guy who likes Lisp, I'll defend Lisp's parenthesis until I die.

The other type of gripes are about competitive disadvantage. These are issues that are not fundamental to the language itself, it's just that the Lisp community hasn't done much to solve them. Things such as having a reasonable set of standard libraries to do programming in today's Internet-connected world have nothing to do with the fundamentals of Lisp. Library documentation and tutorials would help. Having a decent system for the installation of libraries that doesn't depend on little-known and sometimes unmaintained web servers would also help. Everything in this category is hard to defend, and Lispers should universally be ashamed of it. Imagine saying to a "customer" something like, "Yes, I know that everything is undocumented and the installation infrastructure is unreliable, but you know, that's a feature." Rubbish!

In short, while somebody, somewhere, will always be griping about Lisp, the Lisp community, all of us, could go a long way toward making all those gripes fall into the "I hate parenthesis" category rather than the "There aren't any libraries and what libraries do exist don't have any documentation." There is a lot of work to do. Let's get coding...


Comments:


Since I did comment on the other article you referred to, I do not feel it is fair if I comment on your blog as well about that other article (besides I already stated my opinion)

But in my humble opinion I think we should clearly draw a distinction here - in the Lisp world there *are* people who agree with some of the mentioned points, i.e. the part I referred to about the style of the homepages for these languages, some known Lisp guy agreed about the (boring?) homepage of Lisp. (Not that perl's homepage is that much more interesting, but then again perl originated as a now old language too, and lisp is even older)

I believe one huge driving factor behind a language design and the evolution of it are "key people". In python and ruby it is easy to see who are the guys who drive it the most (guido, or matz), with perl this is true too (larry though he seems to be a bit less involved these days with perl than in the past, though i may be wrong here). For php this is partially true too although I guess there were two key guys behind, but for lisp it is a bit hard to find any "key guys" driving it in the year 2008.

It kinda lacks a face-to-show and the homepage is a sign for that (although in fairness I have to point out that a good design need not necessarily originate with the author behind a language, I think the one who designed ruby's current official webpage is not the author of the language for example).

PS: Even GNU folks have not long ago styled up their webpages, so if they can do it, why can't lisp - and most importantly, WHEN will this happen. :D
Even smalltalk's homepage looks http://www.smalltalk.org/main/ a bit more interesting. I know that a homepage does not mean much, but then again a homepage that is "well cared for" shows that there may be a few people who obviously have invested some time to make things "look or seem better", and this is a little token of appreciation. It gives a first impression as well (or in the case of php homepage, a rather usable documentation)
 


I started learning Lisp last year with the SICP book and it really showed me how powerful a language could be. However, I think the problem with Lisp is that there is no momentum going for it. While there are projects like SBCL and Clozure which are still maintained, there are few blog posts or documentation on modern topics of interest (web frameworks, etc).

That said, my CS professors tell me that Lisp is still strong in certain fields, which is great. I think it's fine for languages to find their niche.
 


Is there a "market leader" or "mind share leader" among Lispers? I would guess that SBCL is the mind share leader among free lisps. I would guess that almost everyone would have to throw their weight behind SBCL and really make it useful to average programmers.
 


I currently have a commercial project for which I have attempted to use sbcl with mysql & hunchentoot. Unfortunately, I don't have the time to manage the learning curve needed (I'm fairly experienced in scheme, but not lisp). So this one will most likely end up using php, which I hate, but I've got a business to run here...
 


There are other sides to this story.. just some points to ponder:

1 - if you found a tool that gives you a significant technical advantage over your competitors, would you get on a high-horse and evangelize it?


2- Why is it so important to select a tool that is "popular"?


3- The author of the referenced article had problems with mod_lisp. Does that mean you will, too?


By way of comparison, getting emacs working with Slime requires the addition of maybe three lines in your .emacs file. The author apparently had trouble with that, too. So I ask again, are you going to let this person tell you something is hard to do and not investigate for yourself?


4- There are currently two "market leaders" in the CL world. One is LispWorks: www.lispworks.com. The other is Franz: www.franz.com.

Both have very good environments, although Franz is still stuck in the draconian world of charging royalties for distribution. Not so with LispWorks.

Franz has a web server called AllegroServe, and a pretty good framework called WebObjects. I *think* there is a free, multi-platform version of AllegroServe hosted at SourceForge.

So there you go. IMO it should be pretty obvious by now that people who use CL really don't care whether or not anyone else is playing on their team. At long last, you either understand CL and can use it for something productive...or you don't.


The two commercial vendors I mentioned have found their niche markets, and apparently it is enough to keep them in business and employed, and to have a nice, productive life.


What else do you need?
 


There are a lot of things to like about lisp (and lisp-like languages) but the market for web development vs. PHP (for example) is almost nil. Sadly getting a paycheck often trumps choice.
 


Want to help Lisp? Go implement a Lisp that compiles to python virtual machine. And do A LOT of compromises (farewell CLOS...) to make it completely compatible with python libraries.

Let's face it, there's not enough lisp programmers in the world to have good open source lisp libraries, we might as well steal them from python.

Clozure comes close but it doesn't try hard enough to be compatible with java.
 


Come join us over at newLISP...

http://newlisp.nfshost.com

Why newLISP, why not one of the other standard LISPs?
LISP is an old language born, grown, and standardized in times very different from today, times when programming was for highly educated people who engineered programs. newLISP is LISP reborn as a scripting language: pragmatic and casual, simple to learn without requiring you to know advanced computer science concepts. Like any good scripting language, newLISP is quick to get into and gets the job done without fuss. It has all of the essential elements that made LISP famous: symbol processing, lists, anonymous functions (lambda expressions), and S-expressions as a simple syntax for both data and program.

newLISP has a modern relevant API (Application Programming Interface) with networking, regular expression, advanced math, simulation, statistics, financial, HTTP, and XML functions built right into it. newLISP modules are available that let you connect newLISP to databases, mailservers, and more. newLISP can interface to standard shared libraries on Win32 and Linux/UNIX systems to add even more functionality. The newLISP API is user-driven and adapts over time to the ever-changing requirements of tomorrow's applications.
 


I hate reading articles like this because I know that it's basically all false.

For exmaple, he claims that "MySQL is one simple apt-get away", yet the Lisp frameworks are "nearly impossible to install", even though the very Lisp framework he recommends is also apt-get installable.

The entire rant is full of these kind of misleading claims. It's not even a humorous rant, sadly.

In fact, if you replace "PHP" with, say, "C", his argument works just as well. C is even better-documented, has more libraries, is easier to install (you probably don't even need to), has lousy abstractions, and is generally awful to use. If these arguments are enough to sway someone, then why not go one better and use C for your next web app?

I try not to be a Lisp apologist, for exactly the reasons you mention. But when people oppose your cause by lying, do we have to take them seriously? Look at PHP: every time you use PHP, God drowns a box of puppies. But you don't see the PHP people tripping over themselves to do anything about it.
 


Anonymous: I made a Lisp-based website, and when I tried to install SBCL on my Debian box via the package manager it failed hard due to the way SBCL uses mmap, and I had to patch it and compile it by hand. Google "SBCL VPS", many people have this experience. No such problems with PHP or Ruby on Rails.

Common Lisp is unpopular enough a program that my distro of choice (Gentoo) tends to fall WAY behind on getting Lisp packages into the stable Portage tree. You often have to install things via ASDF or by hand to get the latest versions, or use an experimental (unstable) package overlay. Last I checked there are only a couple of guys working on Lisp for Gentoo. Meanwhile PHP has a small army. I imagine this is similar for other distros; last time I tried some Lisps on Ubuntu, the version in the repo was pretty out of date.

Personally I don't have "a cause" and I don't oppose yours if you have one.
 


@Brian: The packages you get from the Lisp overlay are the latest version available upstream, in other words the same versions you'd get by using asdf-install. There's nothing "experimental" about the overlay, really.
 


Really? I'm always wary of using overlays, perhaps unjustly so in cases like this. Why don't they make it into the tree?
 


My suggestion to the CL community is to:

1. Create a batteries included* CL distribution that works well on both Unix variants and Windows.

2. Document it well, including newbie tutorials.

I don't think Lisp will ever be sold to the average developer. But the productivity boost of Lisp prepackaged with a decent set of libraries would largely eliminate the need for average developers.

*some of those batteries should be things smart people want, like a good scientific library, good graphing, etc.
 


@Brian: I haven't finished the recruiting process yet. Once that's done, the overlay will be merged into portage.
 


Brian: I work at a Rails-based company, and when we upgraded our Ubuntu server from 2.0 to 2.1, it nuked all our gems (different paths, and even the gem syntax changed). I had to patch the gem package files by hand. Google "require_gem undefined"; many people have this experience. No such problems with Lisp.

I've had trouble with out-of-date packages in Ubuntu (never used Gentoo) before, but that's not at all Lisp-specific. It bit my Python programs even worse. Eventually I gave up and went back to Debian unstable.
 


I think this topic need a bit of moderation (no not someone editing postings, I mean common sense and reining in the wild claims).

Yes, there are a lot of things that make a CL solution a bit more cumbersome to set up and deploy than a LAMP solution and popularity has a lot to do with that.

But, like some other poster said, while it might not be a one-click solution it's not *that* hard to install a lisp and a bunch of lisp libraries! If you are computer literate it can be done. Could it be even easier and better documented? Sure, but let's not get carried away. Having a thread filled with complaints isn't going to change the world.

Do CL have a lack of focus? Yes, I think it does. It has no benevolent dictator and will probably never get one. What it does have is a few implementations that are usable on Linux and Windows, which are the two most important platforms. Personally since I prefer Free Software I recommend SBCL. It even powers some really major software projects so it has what it takes.

What could be done then? Well. The Debian project (and maybe rpm based distros as well, it was a while since I used Linux) had a nice idea with apt-get. If you have a bunch of repositories and a clear mechanism for pulling down those packages you need and resolve dependencies we could have our "CPAN" for CL. It would need a few dedicated and mirrored repositories but it could be done. Common-lisp.net is a start. The apt-get replacement could be built upon ASDF and some efforts to that effect have been started.

Is that it? Probably not. Documentation and tutorials would be nice to have, and having them on a logical site like e.g. lisp.org would be a nice idea. How do you get people to write those? I don't know but some kind of karma system, maybe?

I might be rambling a bit, but I think my main point is that the installing problem can be solved, and the complaints that lisp is lacking so much infrastructure is a bit overstated even if it has some substance. I'm no big lisp guru and I have without problems downloaded tarballs and then copied a .asd file to a predefined directory. Much harder than that isn't it, fairly often.
 


Interesting post but I disagree that marketing will turn Lisp into anything other than a niche language. I think Lisp is flat-out confusing to most programmers. That cannot be solved by putting lipstick on the pig, to use a popular expression.

Keep in mind the need to read code as well as write it. Lisp is hard and unpleasant to read (as well as write) for your average corporate developer. That means it's expensive to maintain by anyone other than talented Lisp developers (of which there are relatively few). That spells death for a commercial language.
 


By way of comparison, getting emacs working with Slime requires the addition of maybe three lines in your .emacs file.

err... let me bring up two random things that i think are so typical in the lisp community:


1) these are the default encodings slime tries: '(iso-latin-1-unix iso-8859-1-unix binary)

yes, utf-8 is not the first one, and not, it's not even among them.



2) the default slime-complete-symbol-function is 'slime-simple-complete-symbol

imho, fuzzy completion is one of the top 3 features of slime, and it's also very unique compared to other languages. back, when i was not yet kicked out of slime-devel, i've changed the default to 'slime-fuzzy-complete-symbol, but my commit was rolled back by an important oldtimer. he had no clue about the importance of defaults, though, and he was arguing about how trivial it is to change the completion. yes, he was arguing that the newbcomers should do it, not the oldtimers. noone stood up to help me on the list. today, it was even moved out to a "contrib" that is not loaded by default - one more indirection, one magnitude less probability that a newcomer will meet it in their first week of lisping.

imo, this is why the lisp community is as it is: seemingly it doesn't care about the newcomers at all.

- attila

ps: i've been working with various languages from assembly to slate (a novel lisp-like language) and i was even a java code monkey for a few years to pay the bills... but for the last two years we are working fulltime to build up a common lisp web framework for our freelancer projects. unfortunately it's not trivial to install it yet, but we have numerous TODO entries to change that situation, some of them already done. (for the interested, most of our stuff is linked to from http://common-lisp.net/project/cl-dwim/ )

based on this we are running a live webapp for the hungarian government to manage the municipalities.
 


problem with lisp is that it is never used for real work, yeah php and python aren't as elegant. But they work.

Who ever saw an actual page of HTML rendered with lisp?!? Where are your jQuery examples???

where is the community, the forums, the email lists, the google groups?
 


I recently did a project in Lisp to scrape a website, build a graph, then render it and there wasn't a single library I needed that wasn't a) written and b) simple to install. But I've just done a Ruby project and Rdoc/Rake&Hoe/Gem are still so much easier to use than any Lisp equivalent.

So compared to Ruby's:

1. Decent documentation system.

2. Decent make system.

3. Decent package manager.

The only one of these that is currently good enough in Lisp IMO is 3 with clbuild. ASDF-install is still two requires, a colon and a quote away from being that good. The current documentation generators tend to produce confusing output. And ASDF is powerful but I have to re-learn it each time I use it, it could do with a project generator or something.
 


Lack of a usable production quality debugger. :) Thats my biggest beef with using lisp with non trivial amounts of code. I have tried the debugger available in the Lispworks Edition and at least on my machine it crashed very often. SBCL does have a stepper but its underdocumented. Not being able to set breakpoints and gather the cause of failure is pretty limiting. (Am not repeating the lack of "blessed" libraries issue, where we have a dominant canvas library, a dominant xml parser library ... since its been made elsewhere).
 


Yes! Yes! Yes!

I found Lisp years ago and have used it for exactly one (1) practical real world application since then (which was tiny). For all the reasons you describe. The language itself is great, but it is utterly impractical for real-world use (that isn't being funded by a deep-pocketed government agency who can afford to wait years for a product.) Not because of the language itself, but because of the (lack of) support infrastructure around it.

And the ivory tower academics who make up the core of the Lisp "community", who have rarely spent a day in the real world trenches, who work at research institutes or universities or as high-level employees at Google, flush with cash and time, dismiss these concerns out of hand. They will not lower themselves to think of practical matters, which is all well and good, but I don't have that luxury in the working world.

There is no web stack to speak of, cool experimental stuff notwithstanding - they're not practical. I can't crank out a web app in an hour the way I can with Rails or Merb or even PHP. There is no full stack solution. There are 10 HTML markup libraries. 10 controller libraries, 10 database interfaces, 10 logging packages, etc. Almost all are unmaintained and poorly documented. What documentation there is assumes you are a Lisp expert. Whereas with Rails, you get it all tied up in a neat little package, ready to go. A novice can pick up a tutorial and write a basic working app in less than a day. An advanced user can do much more.

Heck, there is no standard way to split a string in Common Lisp. There is no standard way to open a socket. This is BASIC STUFF. The libraries to do these things are hard to find, hard to install, and poorly documented.

And the community, such as it is, is internally balkanized and openly hostile to newcomers. They embody the worst of the old-school Unix world: "I learned this the hard way, through a long trial-and-error slog, so everyone else should do the same. How dare you be a noob! Quit complaining and spend the next 20 years figuring out everything I know. How dare you ask me to tell you!"

Any time I bring any of this up in a Lisp channel or newsgroup or mailing list, I am drowned out by "Politics! You are bringing up politics! You evil person you!" or else "If you want a standard library/web toolkit/GUI toolkit, write one. Nobody's stopping you." They rather miss the point.

Unfortunately, I am not a tenured professor or comp sci PhD student who can afford to spend the next few years doing that. Much as I'd like to, I have to work for a living, which means delivering working code NOW, not "after I spend a few months polishing up an ad hoc web stack out of this box of parts that the Lisp world handed me."

So I continue to code "real" apps in Ruby, hopelessly awaiting the day when a standard library and web stack and GUI toolkit materializes and the academic elitists writing recursive Fibonacci finders are pushed out of the Lisp community by software engineers writing real-world apps.

With best regards,
-Hoping for a Standard Library and Docs
 


Is it really true that lisp lacks real-world applications? Maybe it's like COBOL. How many have used a COBOL application? Probably not something you do every day, personally, but your bank probably does it and your insurance company and other services you rely on and so on.

I know people talk all the time about how hard it is to find libraries, but since I so often find quite a few I've begun to wonder if the problem is not that they are unknown more than unavailable. Are there no usable stuff for web development here, for example?

http://www.cliki.net/web

I can say nothing about their documentation, though. People seems to go on all the time about web stuff and it's not really my field.

Anyway. I do agree that the situation regarding debuggers could be better. I absolutely hate C, but when I use it gdb is a boon.
 


Hopeful Hobbyist Lisper, what do you mean by by splitting a string?? STRING-TRIM, STRING-LEFT-TRIM, STRING-RIGHT-TRIM isn't doing what you want? MAP, FIND, FIND-IF and POSITION isn't applicable?

I don't understand what you're asking for? Note, this is not an attempt so be a smug lisp weeny. I honestly don't understand.

Socket support is an interesting case, yes. I do agree it's a bit problematic that it's not part of CL. Since CL was standardized before the Internet boom it's not strange, though. But, surely usocket is not that hard to install and understand? (yes I know about IOlib as well)

What I see as a problem is that since the standardization process ended there have been no community consensus about things that suddenly was glaring omissions, like sockets. Now all CL implementations comes with some kind of socket support, but they are all different. I think that some kind of lisp committee, were people who are trusted in the lisp world might be voted in, could help define a common standard for how to package things would be a way to do it. If all implementations shipped with e.g. usocket included everyone could develop for that and know that whatever the implementation it would work. That's a bit like it is in the perl/python world where they only have one implementation to worry about.
 


@Andreas - the functionality I am looking for is Ruby's String#split method (which has analogs in Perl, PHP, Python, etc.)

You have a string and you want to split it into an array, either on some substring or on a regexp.

E.g. in Ruby, "Now is the time".split(" ") -> ["Now", "is", "the", "time"].

I was told on IRC that there is nothing in Common Lisp that does this, and after an hour of digging through Cliki, I found some code snippet that does.
 


That's true. CL does not contain such a function or macro. It's not to hard to write (I think "ANSI Common Lisp" by Graham contains such an example), but sometimes you don't want to do that for every little thing...

I can see why it wasn't included though. Many scripting languages are designed to do specific things and CL is a more general language. C and Fortran don't have a split() either for example.

It would be interesting to collect small "features" like that which are common in, say, web programming.

I guess that is what you need in a "web stack" or whatever someone called it?

BTW, for reference, what was the stuff on cliki called?
 


Earlier in the discussion, anonymous wrote: "Want to help Lisp? Go implement a Lisp that compiles to python virtual machine."

While I'm not so sure about Python as a target, I fully agree with the basic idea, which is to have Lisp implementations which fully integrate with the major platforms/frameworks. This would give Lisp programmers full access to all libraries which exist for that platform, and thereby largely eliminates the library problem.

Clojure follows that approach. It integrates with the JVM; Lisp code and any other code written for the JVM can call each other directly.

My preferred approach is a Lisp implementation on top of .NET's (and Mono's) CLR and DLR. I'm not alone with the idea, apparently; there were and are a number of projects in this direction. See
http://www.clausbrod.de/Blog/DefinePrivatePublic20080312DotNetAndLisp
for a list of them and some more background.
 


Now we have a possible solution to all this illness. Sun have almost ready a new JVM with support for dynamic languages. The world is now waiting for an implementation of Common Lisp under the new DaVinci Machine. Right now, several dynamic languages are being re-implemented under this invokedynamic bytecode. In addition a new Lisp was born: Clojure... then the Common Lisp community must be in hurry or Clojure will eat all them.
 


guys, it's not that hard to use CFFI to call other code. I think the problem is somewhere else.

CL in more places is a goof thing, though. So CL on a JVM or other platform like .NET might be a good idea.

What is a problem, though, is implementations not being compatible when the ANSI standard ends.

That is what I consider a bigger problem, compared to python or perl.
 


> guys, it's not that hard to use
> CFFI to call other code.

Well, to call managed code (such as in the JVM or CLR), it isn't sufficient to hack some trivial glue code and call it via CFFI. For example, you need to control the lifetime of objects created in the CLR with which your Lisp code wants to talk and cooperate. Edi Weitz' RDNZL code, particularly the Managed C++ part of it, gives an impression of what needs to be done.

But even you have something like RDNZL or some other CFFI-based glue code, you still wouldn't be able to do the kinds of things which a Common Lisp compiler generating Java bytecode or .NET IL would give you: You wouldn't have to write any kind of marshaling code at all - parameters could be transferred directly and quickly on the managed stack, classes and interfaces would directly "see" and call each other. Not only would this give you immediate access to all libraries for the platform, you'd also get all the tooling support. (Think Common Lisp in Visual Studio's debugger, for instance.)

Clojure demonstrates
how a platform integration can look like for the JVM.
 


Well, Just don't see the need. CFFI solves my needs.
 

Post a Comment


Links to this post:

Create a Link

This page is powered by Blogger. Isn't yours?