Perl trumps Ruby and Erlang in the Wide Finder Project


Tim Bunce points me to this post about Perl being faster than Ruby in Tim Bray's Wide Finder code competition.

The Wide Finder is at heart an Apache log analysis tool to show commonly hit pages, but for purposes of this comparison, it's analyzing 971MB. Bray explains:

It’s a classic example of the culture, born in Awk, perfected in Perl, of getting useful work done by combining regular expressions and hash tables. I want to figure out how to write an equivalent program that runs fast on modern CPUs with low clock rates but many cores; this is the Wide Finder project.

All the talk about Erlang and parallelism makes me want to get back to working through my copy of Programming Erlang. Oh tuits, come to me!


AMD is said to be working chips that would have multiple cores that all work as one, not requiring any special work by the software. Assuming that works out, perhaps none of this will matter.

I'm not surprised that Ruby was slow, since it's pretty well established on various benchmarks, but I am surprised Python didn't do better. Maybe that code will improve.

Wow, that's really interesting. I'm working on re-writing awstats to bring it up to the 21st century, and I was just thinking about this exact sort of problem the other day. This will be handy! :-)


Leave a comment

Job hunting for programmers

Land the Tech Job You Love, Andy Lester's guide to job hunting for programmers and other technical professionals, is available in PDF, ePub and .mobi formats, all DRM-free, as well as good old-fashioned paper.