I spent a while last night reading about Tim Bray's adventures with Erlang. Tim started investigating Erlang as a part of his "Wide Finder Project" in which he's looking for programming languages that will help accelerate common tasks on the soon-to-be-very-popular CPUs with many cores but slower clock rates.

Tim works at Sun, and so this question and project makes perfect sense in light of Sun's Niagra and T2 processors with many cores and CMT. It also makes perfect sense in light of Intel's Tera-scale computing initiative where they have demonstrated chips with 80 cores. In short, the future is going to be very, very parallel, and we had better come to terms with that.

Unfortunately, the modern multi-threaded programming paradigms are ill-equipped to take advantage of these modern processors. Most popular programming languages have the same simple threads+locks paradigm that was popularized with pthreads and Java. While this works, it doesn't work well. As with so many things in programming, the threads+locks paradigm forces programmers to remember a whole lot of crufty details, otherwise they will produce code with very subtle bugs that is very, very difficult to debug. Put another way, threads+locks is the parallel programming equivalent of manual memory management. In the same way that GC manages memory in many modern programming languages, we need an equivalent to help programmers manage parallel programming.

Erlang has many features that help it work well on multi-core systems. The language is inherently multi-threaded and concurrent and relies on threading almost down to the core (you can write non-threaded Erlang programs, but the language makes it so easy to use threading that you'd hardly want to). I have talked about Erlang before a few months ago (here and here).

So, here we have Tim Bray asking a perfectly sensible question, "What programming language is going to help us programmers exploit the soon-to-be-commonplace multi-core CPU?" Erlang is certainly a potential answer to that problem. Unfortunately, Bray decided to pick a problem for which Erlang is particularly unsuited and then compares Erlang to another programming language that optimized for the problem, albeit without any parallel programming support--Bray picked a simple web log analysis problem.

The web log analysis problem is one that is well suited to Perl, Python, or Ruby. One might even say that these languages were virtually created to solve problems in this exact domain. If Perl does anything really well, it processes text with regular expressions. Python is a bit more clunky than Perl or Ruby in terms of regex syntax, but still does quite well. Ruby was created to solve many of the same problems as Perl, but with a better object model and saner syntax (IHMO).

Erlang, on the other hand, was created to develop 24x7x365, long-running telecom software. This is stuff that aims to have downtimes measured in a handful of minutes per year and that must be able to be upgraded on the fly and recover from any faults or failures. In short, Erlang aims to help programmers write code that can take a bullet to the head, recover, and keep on doing its thing, later allowing the programmer to find the fault, fix it, and upgrade the system, all while staying up and doing its thing. This is a hugely complex task, I can assure you. And Erlang conquers it in fairly good fashion.

So, when Bray decides to try Erlang on this problem, he naturally finds that it blows chunks. His first attempt doesn't even attempt to use any Erlang threading features, which of course defeats the whole reason for the investigation in the first place. Knowing enough about Erlang to be dangerous, I found myself saying, "Well, duh! Why did you expect that to get great performance?"

Overall, I'm getting pretty tired about these simplistic comparisons that people do between programming languages. It always feels like they're an elaborate sort of "gotcha." Step 1, pick a task that runs particularly well on the evaluator's most familiar language (Bray picks web log analysis, which runs quite well in Ruby, his choice language). Step 2, pick a victim language. When the victim language doesn't measure up, yell "WTF?!?! [insert language] sucks." Now, in truth, Bray didn't do that last part, but you have seen the pattern other times, I'm sure (witness the number of people that list the Computer Language Shootout as justification for almost anything).

What would have made a better comparison is writing a multi-threaded web server in both Erlang and Ruby and see which server is able to deliver the best performance to 10,000 active clients with widely varying download speeds. I'd be willing to bet that Erlang does a better job. No, I wouldn't even suggest writing a 24x7x365 telecom switch in Ruby; as fine as Ruby is, Erlang would win that hands down.

So, rather than making languages do stupid tricks as the basis of comparison, let's acknowledge that there is something that we can learn from just about every language. The fact is, all languages optimize for particular problem domains and I don't think that a universal programming language exists that would perform well on all tasks. Bray rapidly found out that Erlang isn't optimized for doing line-oriented I/O and it's regex library sucks. So what? While those problems could be eliminated from Erlang, the fact that Ericsson has deployed large telecom gear without having to fix those issues means that Erlang is ideally suited to its original programming domain.