Books of Note

Practical Common
LispThe best intro to start your journey. Excellent coverage of CLOS.

ANSI Common
LispAnother great starting point with a different focus.

Paradigms of Artificial Intelligence
ProgrammingA superb set of Lisp examples. Not just for the AI crowd.

Tuesday, November 09, 2004

Web application design: the REST of the story 

When I first got turned on to Lisp, I quickly discovered both Paul Graham's web site and Chris Double's blog. Since that time, I have been thinking about web design in Lisp. Both Paul and Chris have, at times, espoused the idea that it's pretty cool to use continuations [and here] (or at least closures in Paul's case) to program web applications. You get a nice programming model that is very much like a "normal" application where you can effectively present a series of screens to the user and pick up input data while programming in a very linear way.

A few weeks ago, I found some articles on REST -- representational state transfer. REST is the name of an architectural style that was coined by Roy Fielding in his PhD dissertation to describe the way the web works. Essentially, Fielding argues, the web works by returning to a client a set of representations (HTML pages, typically, but not necessarily) that describe the current state of a web resource named by a URI. Each representation includes links to other interesting resources somehow related to the current resource and the client may access those resources using those links. If you look through either Fielding's dissertation or most of the other REST writings on the web, you'll find a whole bunch of buzzword-compliant, but ultimately vacuous, language about REST and how good it is. I finally boiled it down to the following key points, with the help of some of Paul Prescod's articles and website:

The REST crowd says that these principles are what make the world-wide web the most scalable architecture ever built. Indeed, when you follow these principles, the overall web architecture and infrastructure is working with you, not against you. For instance, caching happens at many points in the web (client, intermediate nodes, and possibly in front of the server in the form of reverse proxy caches). If each resource is uniquely identified by a URI, you don't have problems with the browser back button and people can easily share URIs with others using cut-and-paste from the browser location bar. Because HTTP is a very loosely coupled, late-bound, general-purpose transfer protocol, clients and servers can evolve without the other end of the wire also having to change. Finally, intermediate nodes can interact with data traveling between client and server and participate in the protocol to optimize performance or other characteristics.

When you violate these principles, at least some of the web infrastructure shuts itself off or otherwise isn't working for you. REST specifically argues that the following architectural items are problemmatic in web application design:

REST can be applied to both interactive (browser-based) applications, as well as web services. My take is that there are some drawbacks to applying a pure REST architecture to an interactive application. I think that you can apply much of REST and you'll end up with a great system if you do, but limiting yourself to HTTP authentication will make your application look like it's straight from 1995 (i.e. ugly as sin--you might as well give all your pages a gray background and times-roman fonts). That said, the principles that make up REST are sound and it is advantageous to follow them when you can.

For grins, I took a look at and it's remarkably REST-like for most of the overall interface. Have you ever noticed how you can forward an Amazon link to anybody via email and it just works? There is a little bit of magic happening on the server, but most of it works because of REST principles. Now, Amazon obviously has a lot of customized pages. Rumor has it that those cost them dearly, too, in terms of scalability. They require a lot more server resource to serve that up and the content isn't as cachable as it could be. In their case, however, I'm sure they sell a lot more merchandise because they include that personalization.

From what I can tell, though, REST really shines when creating web services. Indeed, many of the REST resources on the web are devoted to describing why REST makes a far better web services infracture than something based on SOAP. After reading many of these, I think I have to agree. The combination of REST+XML is powerful for a general-purpose web services infrastructure. In fact, I think that REST could be applied to distributed Lisp applications by substituting sexprs for XML and it's far better than having to deal with SOAP and all its baggage. Another realization I came to is what a horrible, aweful thing SOAP is. Simply put, out of control.

So what does this have to do with continuations and web programming? Simply that it seems like continuation-based programming might have some pretty big scalability problems if used too much on a high volume site. Does that mean continuation-based programming is bad? No, just that, like everything, you have to know when to apply it and when you're pushing it beyond its sweet spot. In particular, it seems suited for certain portions of an interactive web application, but probably would not be good to use in a web service design. Further, I'm pretty convinced that programmers should spend more time learning about state machines and how they work. Most of the interactive parts of a web application can be modeled as a state machine. With the syntax transformations afforded by Lisp macros, it should be possible to design event-driven web applications fairly easily and not require the saving of so much continuation state.

Last week, I talked a bit with Paul Graham about Viaweb's architecture. The important items are (some of which have been reported by Paul in his various essays [here, and here]):

So, in the case of Viaweb, Lisp was used for the heavyweight portion of the site and interacted with by a relatively small number of merchants (hundreds, not hundreds of thousands). The data built up there was then used to generate a static site that interacted with CGI scripts to implement the shopping cart itself.

Where is all this going? I'm not quite sure yet. Clearly web application architecture has evolved a lot since Viaweb was founded in 1995. Things like fork-and-exit CGI scripts are a thing of the past on a high volume site, with FastCGI being the minimum for modern efficiency. But it seems like there are some things to be learned from the REST style, too.

Finally, it's important to note that most REST advocates are positioning REST for web services interfaces as an alternative to SOAP/UDDI/etc., not necessarily as the style to use for interactive web applications. That said, the fact that Amazon uses it is very interesting.


You may find this paper [1] interesting. It tackles the problems with state and continuation expiration in a continuation-based web frameworks.

[1] "Automatically Restructuring Software for the Web" by Matthews, Findler, Graunke, Krishnamurthi, Felleisen

This post gave rise to a long email conversation between dave and myself.

Thank you, very interesting!

Thanks to help understand web and programming!

The statement from the email thread that "[The REST guys are] against sessions and consequently generally down on personalization. Their answer is that this isn't REST." Is 100% false.

Well since that comment was made by someone anonymously and pointed to nothing to support the point I guess we can just assume it's bogus and he did correctly represent "the REST guys".

Post a Comment

Links to this post:

Create a Link

This page is powered by Blogger. Isn't yours?