Rafaël Garcia-Suarez has put out the first release candidate for Perl 5.10.0. This will be the first new release of a production version of Perl in over 2½ years, so well worth taking a look at.

Again, for an introduction to the features in Perl 5.10's new features, see Ricardo Signes' slides for his talk Perl 5.10 For People Who Aren't Totally Insane.

I applaud Michael Schwern's announcement today that he will no longer be supporting Perl 5.5 in any of his modules. Toolchain modules like Test::More and ExtUtils::MakeMaker will be compatible with Perl 5.6.0, and others with 5.8.0. As Schwern puts it, "5.5 is effectively end-of-lifed." And not a moment too soon, I believe. Perl 5.6.0 came out seven years ago, and 5.8.0 five.

Schwern's breaking point was seeing the Perl Survey results that only 6% of respondents use Perl 5.5. Most of all, he points out:

Finally, I'm coming around to chromatic's philosophy: why are we worried about the effect of upgrades on users who don't upgrade? Alan Burlson's comments about Solaris vs Linux are telling: if you're worried more about supporting your existing users then finding new ones, you're dead.

I applaud Schwern's radical break from the past. No longer will he be "hamstrung from using 'new' features of Perl," as he puts it. This will allow him the freedom to do more great things as I fully expect he will.

Most of all, I'm glad that he just did it. No committee, no call for consensus, no poll of people to see what everyone thought. JFDI, baby, JFDI.

Who among us will be the first to write a module that takes advantage of Perl 5.10's new features, urging us all forward, instead of mired in the mud of the past? I can't wait to see it happen.

David Cantrell has come up with his CPAN dependencies and test results checker. It checks out all the dependencies for a distribution, and displays the installation results for each of the dependencies, as as reported by the CPAN testers. It's interesting to see the visual representation of all the modules' test histories, but the idea of multiplying all the "probabilities" of installation is specious.

A few days ago Gerard Goossen released version 1.5 of his kurila project to the CPAN, a fork of Perl 5, both the language and the implementation. I talked with about the history of this new direction.

---

Andy: Why Kurila? Who would want to use it? What are your goals?

Gerard: Kurila is a fork of Perl 5. Perl Kurila is a dialect of Perl. Kurila is currently unstable, the language is continuously changing, and has just started.

There are a few goals, not all of them going in the same direction. One of the goals is to simplify the Perl internals to make hacking on it easier. Another is to make the Perl syntax more consistent, remove some of the oddities, most of them historical legacy.

What is currently being done is removing some of the more object/error-prone syntax like indirect-object-syntax and removing symbol references. Both of these are not yet very radical yet, most modern Perl doesn't use indirect-object-syntax or symbol references.

But I am now at the stage of doing more radical changes, like not doing the sigil-change, so that my %foo; $foo{bar} would become my %foo; %foo{bar} .

Andy: Where do you see Kurila getting used? Who's the target audience for it?

Gerard: Kurila would be used for anything where currently Perl is being used. I am using Perl for large websites so changes will be favored in that direction.

I am working for TTY Internet Solutions, a web development company. We develop and maintain websites in Perl, Ruby and Java. Websites we develop include www.2dehands.be, www.sellaband.com, www.ingcard.nl and www.nationalevacaturebank.nl. Of these www.2dehands.be and www.nationalevacaturebank.nl are entirely written in Perl.

We are not yet using kurila in production, but I have a testing environment of www.2dehands.nl which is running on Kurila. Developing Kurila is part of my work at TTY.

Many of the changes in Kurila are inspired by bugs/mistakes we made developing these sites. It started with the UTF8 flag. We encountered many problems making our websites UTF-8 compatible. In many cases the UTF8-flag got "lost" somewhere, and after combining it with another string, the string got internally upgraded and our good UTF-8 destroyed. Because everything we have is default UTF-8. The idea was simply to make UTF-8 the default encoding, instead of the current default of latin1.

Andy: Did you raise the possibility of changing the default encoding in Perl?

Gerard: The problem is that changing the default encoding the UTF-8 is that is destroys the identity between bytes and codepoints. So it's not a possibility for Perl 5. Like what does chr(255) do? Does it create a byte with value 255 or character with codepoint 255?

I made a patch removing the UTF-8 flag and changing the default encoding to UTF-8 and sent it to p5p.

Andy: What was the response?

Gerard: There was as good as no response to it, I guess because it was obvious that it seriously broke backwards compatibility and the patch was quite big, making it difficult to understand.

About two weeks after the utf8 patch, I announced that I wanted to change the current Perl 5 development to make it a language which evolves to experiment with new ideas, try new syntax and not be held back by old failed experiments. One of the interesting things about Perl is that it has a lot of different ideas and these are coupled to the syntax.

There was of course the question of why not Perl 6.  That it should/could be done in backwards-compatible way. That there is no way of making the Perl internals clean, that is better to start over.

And about half a year ago I announced that I had started Kurila, my proof of-concent for the development of Perl 7. Rewriting some software from scratch is much more difficult then it seems, and I think starting with a well proven good working base is much easier. Perl 5 is there, it is working very good, has few bugs, etc., but it can be much better if you don't have to worry about possibly breaking someone code, and just fix those oddities.

Andy: Do you have a website for it?  Are you looking for help?

Gerard: There isn't a website yet, and also no specific mailing list, currently all the discussion is on p5p. There is a public git repository at git://dev.tty.nl/perl.

Andy: What can someone do if he/she is interested in helping?

Gerard: Contact me at gerard at tty dot nl. Make a clone of git://dev.tty.nl/perl and start making changes.

When in a tight loop of many records from a database, using the quick & dirty solution of calling $sth->fetchrow_hashref can be expensive. I was working on a project to walk through 6,000,000 records and it was slower than I wanted. Some benchmarking showed me that I was paying dearly for the convenience of being able to say my $title = $row->{title};.

When I converted my code to bind variables to the columns in the statement handle, I cut my run time about 80%. It was as simple as adding this line:

$sth->bind_columns( my $interestlevel,
my $av_flag, my $isbn, my $title );

before calling the main loop through the database. Now DBI knows to put the data directly in there, without creating an expensive temporary hash. I also can't make a typo such as $row->{ISBN} inside the loop, so there's a measure of safety as well.

The benchmarks below show the relative speeds of each of four techniques:

hashref     took 31.5048 wallclock secs
array       took 8.83724 wallclock secs
arrayref    took 5.5308 wallclock secs
direct_bind took 4.46956 wallclock secs

Here's the key parts of the benchmark program I used:

use Benchmark ':hireswallclock';
sub hashref {
while ( my $row = $sth->fetchrow_hashref ) {
my $interestlevel = $row->{interestlevel};
my $av_flag = $row->{av_flag};
my $isbn = $row->{isbn};
my $title = $row->{title};
}
$sth->finish;
}
sub array {
while ( my @row = $sth->fetchrow_array ) {
my ($interestlevel, $av_flag, $isbn, $title) = @row;
}
$sth->finish;
}
sub arrayref {
while ( my $row = $sth->fetchrow_arrayref ) {
my $interestlevel = $row->[0];
my $av_flag = $row->[1];
my $isbn = $row->[2];
my $title = $row->[3];
}
$sth->finish;
}
sub direct_bind {
$sth->bind_columns( my $interestlevel,
my $av_flag, my $isbn, my $title );
while ( my $row = $sth->fetch ) {
# no need to copy
}
$sth->finish;
}
for my $func ( qw( hashref array arrayref direct_bind ) ) {
my $sql = <<"EOF";
select interestlevel, av_flag, isbn, title
from testbook
limit 1000000
EOF
# This sub calls the SQL and returns a statement handle
$sth = sqldo_handle( $sql );
my $t = timeit( 1, "$func()" );
print "$func took ", timestr($t), "n";
}

Did you find this article useful? Or does it not belong on Perlbuzz? Let us know what you think.

The first release candidate of Perl 5.10, with the first new syntax and major features since 2002, will be released soon, probably in the next week or two. The code has been in feature freeze for weeks, and only minor patches are being accepted. Lately the VMS porters have been working on compatibility problems with File::Spec and File::Patch, and Jos Boumans and Ken Williams are syncing core CPAN modules with the Perl source trunk.

For a gentle and well-presented introduction to the features in Perl 5.10, see Ricardo Signes' slides for his talk Perl 5.10 For People Who Aren't Totally Insane.

Please note that Perl 5.10 is not Ponie. Ponie was the project that was to put Perl 5.10 on top of the Parrot virtual machine, but Ponie has been put out to pasture.

It's been a while, because I had all kinds of overblown expectations of being able to offer the results in a range of formats, but I have to admit that a month in a developing country with only intermittent satellite internet has bested me.

Therefore, I've decided to release the results of the Perl Survey in PDF format for now, with the HTML version to follow as time permits.

Some points of interest:

• 4% of the Perl community are women
• Average salary for Perl programmers worldwide is $68,687 (USD equivalent)
• 87% of Perl people are using 5.8.X at least some of the time; that means that 13% are still on 5.6.X or lower
• 33% of respondents are members of their a Perl Mongers group
• However, 27% of respondents don't participate in the Perl community at all (or at least not in any way that we asked about)

Find out all about this, and more, at the link above.

Also, the entire data set for the survey has been released under a Creative Commons license, and I'd like to encourage everyone to download it and play with it. A number of people have already contributed their own analyses of certain parts of the data.

I'd like to thank everyone involved for their help, and I hope that this information will be of use to the Perl community going forward.

A recent column in Software Development Times lamented the lack of support for threading in software today. In it, Andrew Binstock says:

Dynamic languages are even further behind. To wit, Python, allows only one thread to run at a time (except for I/O); this means you can have threads but not running in parallel. Ruby can run threads only within the one VM, which is arguably better but nowhere near good enough. And OpenMP, which might be a solution for some, is limited to C++ or Fortran.

Unfortunately, Binstock ignores the dynamic language that should be first on his mind, Perl. Perl's threading got stable back with Perl 5.8.7 in 2005, after a few years of experimental support. Today, Mac OS X and most Linux distributions ship with a threaded Perl enabled.

How can you tell if your Perl supports threads? Look at the output of perl -V for the string "useithreads":

$ perl -V | grep ithreads
config_args='-ds -e -Dprefix=/usr -Dccflags=-g  -pipe  -Dldflags=-Dman3ext=3pm -Duseithreads -Duseshrplib'
usethreads=define use5005threads=undef useithreads=define usemultiplicity=define

While I agree with the basic premise that demand for threaded apps on the desktop is low, the tools for supporting threads are not quite as sparse as Binstock would have us think.

P.S. O'Reilly & Associates is now O'Reilly Media, and has been since April 2004.

Test::Harness 3.00 has finally been released, and it's a huge opportunity for anyone who writes tests with Perl, if only for the ability to run prove -j and run tests in parallel. I took a few minutes as the maintainer of the old 2.xx series to interview Andy Armstrong, the new maintainer of 3.x, about the history of the new Test::Harness, and what it took to get here.

Andy Lester: So, Andy Armstrong, a joyous day has come: Test::Harness 3.00 has been released.

Andy Armstrong: Yes. I was relieved.

Andy Lester: What has it taken to get to this point?

Andy Armstrong: I think we had a fair bit of paranoia about breaking the toolchain for everyone, and thus becoming extraordinarily unpopular. That made us cautious. We spent a lot of time building our own smoke testing setup. And running lots of people's tests against our code.

Andy Lester: Who is the "we" in this? Back in June of 2006, Schwern and I started the kick off to Test::Harness 3.00 at YAPC::NA in Chicago. What's happened since then?

Andy Armstrong: Ovid got the code started and had just about everything in place by April 2007. Around then I volunteered to look at a Windows problem. And I sort of got dragged in. I really liked the code Ovid had written and enjoyed working on it - so that was an attraction. The Windows problem took a few minutes - but I'm still here.

Andy Lester: You've done a lot more than being dragged in. You hosted the Subversion repository, and the mailing list. What else?

Andy Armstrong: My monopolist plan laid bare for all to see... I have a server which is nominally so I can do things like that - so then I have to do them to justify its existence. So I'm hosting the perl-qa wiki, the TAP wiki. Just sites that needed a home. Like an orphanage :)

Andy Lester: You've uploaded T::H 3. Are you now the maintainer? I thought Ovid was going to be the maintainer of T::H3. (I ask both for the benefit of the Perlbuzz readers, and for my own knowledge:-))

Andy Armstrong: I think I made a move on Ovid somewhere back there and he didn't struggle. So now I'm it. I honestly can't remember how that happened.

Andy Lester: Glad to have two Andys maintaining different versions of the same module. :-) So why does someone want to upgrade to Test::Harness 3? What's in it for the average Perl user?

Andy Armstrong: If you do nothing else - just install it - you'll get better looking test reports. Color even :) And when people start writing test suites that use TAP version 13 features you'll get even more informative reports as an indirect result of T::H 3.00.

Andy Lester: And it's completely compatible?

Andy Armstrong: It's very slightly more fussy about completely crazy syntax errors. But generally yes, compatible - foibles and all. That's syntax errors in TAP (Test Anything Protocol) - just for folk who don't know what's going on behind the scenes.

Andy Lester: So what's in the future for Test::Harness and prove, its command-line interface?

Andy Armstrong: Well we're just talking about TSP (Test Steering Protocol) on the perl-qa list. And we need to do something interesting with the YAML diagnostic syntax we have now. I've written a module for TextMate that uses that so that the cursor jumps to the right line in the test program when you click on the diagnostic.

Andy Lester: What's the benefit of TSP? How would a tester use that?

It would give a test suite more active control over its own execution. Particularly in the case of things like user interface toolkits or modules that are highly platform or configuration dependent you may have large number of tests you'd like to skip conditionally. So TSP would be a convenient way to have a single controller program that would decide which other parts of the test suite to execute. And then you'd probably grow on that to expose more control over which tests to run via prove or whatever UI you'd be using.

Andy Lester: So a more advanced version of SKIP blocks, and you wouldn't have to figure out what tests to run when you ran Makefile.PL or Build.PL.

Andy Armstrong: Yes. And an area where people are likely to find new applications too.

Andy Lester: Anything else people should know?

Andy Armstrong: That there's still plenty more to do with T::H and testing in general. And that I'm surprisingly cheap :) I also want to thank people who have worked on Test::Harness 3, in alphabetical order: Sebastien Aperghis-Tramoni, Shlomi Fish, David Golden, Jim Keenan, Andy Lester, Michael Peters, Curtis "Ovid" Poe, Michael Schwern, Gabor Szabo and Eric Wilhelm. All helped immensely - even you :)

Andy Lester: Well, I do want to thank all of you for doing this massive overhaul of Test::Harness. Abandoning the existing code and starting from scratch has given T::H a new lease on life, and a new platform to move forward.

Our own CPAN Watch has the tag line "We watch the CPAN so you don't have to", but if you still want to, then there's a new CPAN feed in town.

search.cpan.org has the Recent Uploads page and feed, but all it shows is the distribution name and description. Daisuke Murase has created his own CPAN Recent Changes site and feed that now shows the relevant parts of the change log, back to the latest non-developer release. At least, that seems to be its intent, although its heuristic seems to not always work.

I'm glad to see the new site in place. I'm sure new additions will be coming, and maybe others will offer their own spin on what's new in CPAN, as well. It's a mutation that will lead to more evolution.

Attention all trend-finders and part-time statisticians: The results of the Perl Survey 2007 are now available.

Michael Schwern presented a basic analysis of the results were recapped at Pittsburgh Perl Workshop this past weekend, and I'm still trying to get a summary to post here. Skud is also working on her own analysis of the data, and will post it to the Perl Survey results page.

No need to wait for them, though. We encourage you to do your own analysis of the data, and tell us and the world what you think it means. The data is released under a Creative Commons license, and I've created Data::PerlSurvey2007 to give you a quick & dirty way to slurp up the data into Perl code and have fun trying to find your own correlations and interesting insights.

It's simple code. It just slurps up the .csv file available on perlsurvey.org (and also included with the distro) and gives you an array of hashrefs for each completed response. They look like this:

{
'Attended Perl Mongers' => '1',
'Attended Perl Mongers (non-local)' => '0',
'Attended conference' => '1',
'Attended conference (non-local)' => '0',
'CPAN modules maintained' => '12',
'Contributed to CPAN' => '1',
'Contributed to Perl 5' => '0',
'Contributed to Perl 6' => '0',
'Contributed to other projects' => '1',
'Contributed to websites' => '1',
'Country of birth' => 'au',
'Country of residence' => 'au',
'Date survey completed' => '2007-07-26 12:49:37',
'ID' => '25',
'Income' => '80000-89999',
'Industry/ies' => [
'Internet',
'Real Estate'
],
'Led other projects' => '1',
'Other programming languages known' => [],
'Perl versions' => [
'5.005',
'5.6.1',
'5.8.4',
'5.8.5',
'5.8.8'
],
'Perlmonks' => '0',
'Platforms' => [
'BSD - FreeBSD',
'Linux - Debian',
'Linux - Ubuntu',
'Mac OS/X',
'Windows XP'
],
'Posted to Perl Mongers list' => '1',
'Posted to other list' => '1',
'Presented at conference' => '1',
'Primary language spoken' => 'English',
'Programming languages known' => [
'JavaScript',
'MOO',
'PHP',
'Ruby'
],
'Proportion of Perl' => '90',
'Provided feedback' => '1',
'Sex' => 'female',
'Subscribed to Perl Mongers list' => '1',
'Subscribed to other list' => '1',
'Year of birth' => '1975',
'Years programming (total)' => '22',
'Years programming Perl' => '11'
},

Nuthin' fancy, but I hope this gives people a starting point without having to worry about the mundane parts of slurping up CSV files.

In the past couple of days, I've seen some counterproductive social behaviors that help scare away community members and lead to boring monoculture: Taking a public dump on the projects of others when they do not directly affect you. It's rude, it discourages future risk taking in everyone, it goes against the very nature of open source that has brought us here today, and it leads to monoculture. I'd like people to stop.

Mutation #1: kurila

Gerard Goossen recently released kurila, a fork of Perl 5 that includes some speedups and tweaks that seem to scratch Gerard's itches, as well as bundling extra modules. I'm right now trying to get an interview with him to find out more about his project and the reasons behind it, because there are probably some interesting lessons in there. However, the disapproval on the Perl 5 Ports list was swift and severe.

All forking based on the Perl 5 syntax and code base, throwing away CPAN compatibility, seems to me to be a complete worthless waste of time.

So what? Who is anyone to say how Gerard is to use his time? Is there any harm here? No? Then leave the guy alone, please.

Mutation #2: lambda

Eric Wilhelm released lambda, a distribution that lets you use the Greek character lambda (&#955) as an alias for sub {...}, apparently as a nod to Python's lambda keyword for anonymous functions. Immediately people jumped on him saying that the module should go into the Acme:: namespace, as if the namespaces of CPAN mean anything in 2007. There was also this cluck-cluck from someone I figured would be more encouraging (and later apologized, as it turns out):

Well, if you want to use it in your own code and your work's code, that's fine (because I'm sure you find typing CONTROL-SHIFT-EL so much easier than "sub {}" :) but if it shows up in your CPAN modules, you might get a few complaints since this sugar, while a really nifty hack, adds nothing complex but does screw up older editors and will confuse the heck out of a lot of maintenance programmers.

Personally, I figure that if someone's a smart enough programmer to do a hack like the lambda module, he or she is also smart enough to figure out potential downsides. And so what if he doesn't? What's the harm here?

Mutation #3: perlbuzz

Perlbuzz itself has always come under this umbrella of disapproval. Even before we announced the site, Skud and I have fended off the comments saying "We already have use.perl.org, we don't need Perlbuzz." Maybe not, but why do you care if we start the site? Why does it bother you? And why do you find it necessary to tell us that we're embarking on a waste of time?

I hope that in the past few months, the work that Skud and I have done have shown you, the reader, that Perlbuzz is a worthwhile addition to the Perl community, and a valuable news source that overlaps other news sources while not being a subset. What if Skud and I had listened to the tsk tsk of the doubters? Perl would be right where we it was before, with nothing new.

Evolution requires mutation

Why are we so quick to take a dump on the projects of others? The only way anything interesting happens is that people try weird, new things and see what sticks. What if Larry had listened to those way back when who said "Ah, we've got Awk and shell tools, we don't need Perl?"

I fear our tendency to monoculture. I want crazy new projects to thrive, not get squashed at their very infancy. Next time someone comes out with a project that you think is silly, congratulate the person rather than scoffing at it. Who knows what it might lead to?

(And a big thank you to Jim Brandt for the "Evolution requires mutation" idea.)

I talked today with Richard Dice, the newly-elected president of the Perl Foundation, about the recent changes in TPF, and what TPF has been working on lately. If you've asked "What does TPF do? Why should I support it?", this interview should help answer that.

Andy: Richard, you're now President of The Perl Foundation, Jim Brandt is Vice-President, and Bill Odom is Chairman. What do these changes mean for TPF and for the Perl community?

Richard: Regarding the first of the two questions you have embedded in there, what it means for TPF, there's a pretty straightforward answer - it means that I am now the person entrusted with the abilities of the President, per Article V, item 5.05 of the Bylaws of The Perl Foundation. From the point of view of TPF being a corporation the abilities of the President are pretty standard President-stuff. It basically means that I'm the guy able to sign contracts and am responsible for the general management of the corporation. The President is also a member of the Board of Directors so I have a voice within that group and a vote in all voting matters of the Board. I don't think that there are too many surprises as far as any of that goes. It's vanilla-corporation-legal stuff.

For the previous two years the TPF President had been Bill Odom. In the past few months Bill had been considering what his own personal strengths, interests and abilities to commit time would be in the future and mostly he was thinking that what we wanted to invest his energies into were Board considerations. That is, organizing how the Board would conduct its responsibilities. And that's a Chairman job. The chairman for the previous few years had been Kevin Lenzo. After all the years Kevin had been involved he felt as though he had done all he wanted and needed to in terms of active participation. So the Board thought that Bill would be the right person to take on that role. I was the Steering Committee Chair of TPF for the almost-two years up to that point. Bill and I did plenty of work and discussion together because of that, and I got a level of familiarity with much of the rest of the Board over that time as well. They thought I would be a good choice to fill the position.

I think that the Board liked one aspect of my thinking in particular pertaining to the Perl community. That is: the Perl community is just fine. Better than fine. The community is great. TPF exists to support the community. So what we have to pay attention to is the areas where the community is not great.

Andy: What areas would those be?

Richard: We need to help the rest of the world understand what Perl has to offer them. We need to talk with the rest of the world and gather together what they have to tell us, organize it, and present it back to the community in a coherent way so that we understand what the perceptions of the rest of the world are regarding what Perl and its community are all about. This kind of communication is a pre-condition for the next step, which is figuring out how the community and the rest of the world can help each other.

Andy: Any plans or grand ideas to share along those lines?

Richard: I'll share one plan that has already come to pass. Forrester Research approached TPF back in April 2007, asking us to participate in a survey of dynamic languages (Perl, Python, Ruby, PHP, Javascript) they were putting together. That was really important. Forrester has a lot of reach into the corporate IT world, at the VP and CIO/CTO level. I thought it was very important for us to get the word "Perl" prominently placed within that survey. What followed was a few weeks of brain-wracking work, not just mine but with a ton of help from about a dozen people inside and outside of TPF. But it was also important for us to participate because just seeing what sorts of considerations Forrester put into the survey were reflection of what they thought their audience was interested in. Participation was an excellent two-way communication opportunity.

Andy: What were the results of their report?

Richard: The results were quite good I thought. Forrester "Wave" surveys have a pretty standard format; in it, Perl was considered a "Leader" in this space. TPF will issue a larger press release about the results of this survey later. (The citation guidelines are complicated and we have to spend some real time in sifting through it all before we can make an official and detailed statements of the results.)

Another project that I'm involved with now is trying to make Perl 5.8.8 an official part of the Linux Standard Base 3.2 spec. This is a really good idea, as it means that any Linux ISVs that make a product that targets LSB 3.2 can assume the presence of a (sane) Perl distribution and so they don't have to ship it themselves.

These two examples suggest what I think will be a theme of the next year, which is TPF working with other organizations in alliances. Everyone is good at something. No one is good at everything. We have to be able to offer our expertise to other organizations, and we have to be willing to work with others in order to take advantage of their expertise. Trying to do things another way is a recipe for frustration and limited results.

Andy: Is the LSB project something that needs to happen at the TPF level? Is this one of those things that couldn't happen if TPF weren't doing it?

Richard: That's a good question. I don't immediately see a reason why TPF would have to be involved. Linux Foundation could have tried doing this without our help. However, this goes back to what I said about everyone having their own areas of expertise. The people in Linux Foundation aren't experts about Perl. From my perspective, the things I helped them with on this are pretty minor. But I saved them a ton of time helping them stay away from blind alleys in where they were going with this. And I could give them confidence that this was an effort that was worth undertaking. If they wanted to include Perl in LSB and they couldn't find a "Perl door" to knock on to get help in what they're doing, maybe they'd think that it wasn't worth the effort because Perl wasn't vibrant, active and supported.

[Note: Allison Randal noted after this interview was published "In fact, the Linux Foundation did try to do do it without our help, but had a hard time figuring out who to talk to in the community." -- Andy]

As I said before, I think TPF has a huge role to fill in interfacing between people on the inside of the community and people on the outside. Perhaps some Perlbuzz readers can't imagine other people thinking that Perl isn't vibrant, active and supportive. But that's exactly my point — without an organization like TPF to speak for Perl in these kinds of situations, that's exactly the kind of impression that would be conveyed. Some aspects of what TPF does are simple, but they're crucial.

Andy: So did the Linux Foundation come to TPF asking about getting Perl in LSB 3.2?

Richard: I'm not quite sure what the mechanics of the engagement were. Allison Randal, a TPF Board member, was the one who started up the discussion with Linux Foundation. Once she made initial contact I inherited the doing of the work.

Andy: So what can Perlbuzz readers do to help out with TPF?

Richard: The first thing I'd urge Perlbuzz readers to do is to be involved with the Perl community. Go to YAPC conferences, go to Perl Workshops and Hackathons, go to your local Perl Mongers meetings. This strengthens the whole Perl community, not just TPF. (And as an aside, it's something I've found enormously personally rewarding and enjoyable. I recommend it to anyone.)

Be eyes and ears on the ground and in the local Perl and IT scenes. If you see something interesting that you think has implications for Perl, let us know. Email pr@perlfoundation.org. Pay attention to websites like news.perlfoundation.org, use.perl.org, perlbuzz.com and yapc.org. Every now and then something can happen where TPF could use specific help. These are the places where the news would first go out to. There is also the #tpf IRC channel at irc.perl.org. If you want to talk to TPF folk, you can look for us there.

Andy: I should note that I am the pr@perlfoundation.org contact, and that Perlbuzz is sort of an outgrowth of my PR role for TPF, although entirely separate from TPF.

Richard: That's it for projects right now, but please track me down for another interview in a few months. We can cover what's been going on then. And thanks for the great work with perlbuzz.com! And while it's separate from your Perl Foundation PR hat, I think the most important thing is that Perl gets promoted! You and Skud are doing this fantastically well with perlbuzz.com so I'm a big supporter.

Andy: Any time you have something to say to the community, Richard, I'm glad to publicize it. Thanks for your time.

The Pittsburgh Perl Workshop is coming soon, October 13-14. If you're not yet signed up, there's still time to register. The price is only $40 for students for two days, or $70 for non-students, a heck of a deal for two days of talks from dozens of speakers. Plan to come early and make the Friday Social. Plans are also afoot for a Saturday Social.

There are also spots available in the full-day From Zero To Perl training class, so you can bring your friends who aren't yet hep to the wonders of Perl.

Finally, if you're going to PPW, and you've always wanted to speak at a conference, but never had enough to say, there is still time to get one third of your fifteen minutes of fame. Submit your lightning talk today.

Chris Dolan recently received a Perl Foundation Grant to write 20 new policy modules for Perl::Critic. I managed to catch up with Chris by email and ask him a bunch of questions about his work on Perl::Critic, the TPF grant, and more.

Kirrily: So, how did you first get involved in Perl::Critic?

Chris: Funny, I had to look back at my email to remember this. You'd think I could remember two years ago...

I read Damian Conway's Perl Best Practices (aka PBP) in summer 2005 and thought, "Finally! The solution to the 'Perl is not readable' myth!" I tried applying all of the ideas manually in a new module I was just about to release to CPAN (Net::IP::Match::Regexp). I found the process rewarding, but a little tedious and error-prone.

After working with the PBP recommendations manually, I thought about writing some code to judge some of the recommendations automatically, but happily I decided to check CPAN first. I found Perl::Metric::Basic and posted some RT reports. Then I found Perl::Critic, which had just been renamed from Perl::Review. I found the code much more approachable that Perl::Metric and soon filed a few RT reports and a cpanratings review. Jeff Thalhammer, the P::C creator and leader, emailed me back personally thanking me for the feedback. After that, we had about a dozen back-and-forths where Jeff sent me pre-release tarballs of Perl::Critic and Test::Perl::Critic to test before sending them to CPAN. Then, I wrote my first from-scratch policy (Modules::RequireEndWithOne) about a month later, and I was hooked.

Kirrily: What prompted you to apply for a TPF grant? How did you convince them to give you money? How did you find the grant process?

Chris: I hope nobody criticizes me for admitting this, but my reason for applying for the grant was self-motivation. Our second child was born Jan 1, 2007 and the company where I worked was struggling, so it was a hard time to find energy for my open source efforts. I usually work best under pressure, so I decided the publicity of a TPF grant would force me to get something done (and the money will be a nice bonus too).

I wrote a ton of grant proposals in my former life as an astronomer and, of course, tons of business proposals as a web/programming consultant, so I found the TPF grant process pretty easy. The positive impact of Perl::Critic is obvious on the community of people who care about good Perl code, so the justification part was straightforward. The harder part was deciding what part of P::C to propose to build.

Kirrily: Is there a theme or themes to the 20 modules you're doing?

Chris: Yep. We keep an extensive TODO.pod file in the P::C svn repository. One portion of that file is devoted to PBP recommendations that seemed feasible to implement in code. I had created that list over a year earlier by paging through the PBP book and brainstorming implementations for each recommendation. Needless to say, some of the PBP recommendations will never make it into P::C -- for example "Never assume warning-free compilation implies correctness" (page 432).

It turned out there were an even 20 PBP-based ideas in the TODO list (vs. about 40 non-PBP ideas). Twenty seemed like a nice round number and a challenging-but-doable quantity. At that time, I was primary author of 27 of the 93 finished policies, so I had a pretty good idea of what was involved.

Kirrily: Why should people use Perl Critic? Why shouldn't they?

Chris: In my opinion, it's good for two things:

1) Finding bugs (real bugs and potential bugs)

2) Making your code more approachable

Both are especially important for open source code that might be used by thousands of other programmers. Why write bad open source code?

One reason not to use Perl::Critic is to put a badge of honor on your code. Achieving P::C compliance is a good thing to announce to the world because it tells users that you care about the quality and consistency of your code. But P::C doesn't help make your code better directly. It just tells you that your code isn't broken in one of about 100+ known ways. And some of those 100+ contradict each other! Compare that to the innumerable ways to write bad code. So, you've got to start with good code and use P::C as a tool to make it a little better.

One piece of advice I like to share is that you should not be afraid to turn off P::C policies that don't work for your code. I think Damian gave similar advice in PBP. In the P::C code itself, we have 98 instances of ## no critic(...) which turns off certain policies. Even Perl::Critic is not 100% Perl::Critic compliant! But 98 out of 27,000 lines of code isn't bad.

Another controversial topic is whether .t files should be P::C compliant. Personally, I don't bother. Writing test code needs to be fast and easy or it falls by the wayside.

Kirrily: Do you know of any really interesting uses of P::C out there? Famous companies using it?

Chris: Good question. We should ask Jeff Thalhammer about that -- as P::C founder, he has a closer connection to our user community. There was brief talk about creating a for-profit consulting entity to help big companies write better code using P::C, but I don't think anything came of that (I was too busy to participate).

What I find interesting is the people/groups/companies who have privately implemented their own policies for P::C. Parrot, for example, had a few custom policies the last time I looked. Another example is MathWorks, who funded the Perl::Critic developers to write some specific policies they desired. Now that's a business model any open source developer craves!

Kirrily: What next after you've implemented these P::C modules?

Hmm, I have no real plans. More policies, I guess. I'd love to get more people writing them. They aren't too hard to create and without them P::C is worthless. If anyone wants to get involved, I'd be happy to be a mentor. Giving fish vs. teaching fishing, y'know.

In my day job, I do a lot of Java coding (and often enjoy it). We use a commercial package called CodePro by Instantiations which is conceptually similar to Perl::Critic but is much more polished. They have over 900 policies to choose from, many of them specifically designed to squelch anti-patterns in the big APIs like Spring, Hibernate, Struts, etc. Jealously, I've often thought that they have an easier job than we do because Java is so much easier to parse than Perl, and the code smells seem to fall in more easily identifiable patterns.

When Perl 6 becomes popular, I'd like to see Perl::Critic be implemented against it. Perl 6 will have a grammar that is much more amenable to grepping. Behind the scenes, I'd like to see the Perl 6 version of P::C work against de-sugared syntax or perhaps even the abstract syntax tree to simplify the policy writing. Unless a huge amount of work goes into improving PPI, that's not going to happen for Perl 5. At one point, we talked about writing P::C policies using an XPath-like notation, but nobody ever championed that goal. I discovered that FindBugs (another Java code analysis tool) took that approach successfully. Maybe P::C can benefit from the Parrot tree grammar engine (TGE). Or the optimizer. After all, the syntax pattern matching that P::C has to do is not that different from the opcode pattern matching that an optimizer performs.

Overall, what I'd really love is for Perl 6 to avoid the "write-only language" moniker that got slapped on Perl 4 and Perl 5. If P::C can help with that, then it has succeeded.

Thanks, Chris!

Find out more about Perl::Critic at PerlCritic.com or listen to this PerlCast interview with Jeff Thalhammer, creator of Perl::Critic.

mod_perl2 User's Guide (modperl2book.org) by Stas Bekman and Jim Brandt has just been published by Onyx Neon, and Amazon says it's shipping. I can't think of the last time I've been as excited to see a Perl book get published. Besides brian d foy's Mastering Perl, there hasn't been a significant Perl book in quite a while. Perlbuzz is going to work with Onyx Neon to have a book giveaway, so watch this space in the next few days.

Jim Brandt took some time to talk with me about the book, why you need to switch to mod_perl 2, and why it took so long to get the book to market.

Exuberant ctags, the standard tags utility on most systems today, has released version 5.7 for download with the following improvements to its Perl support (among many other improvements):

• Added support for 'package' keyword
• Added support for multi-line subroutine, package, and constant definitions
• Added support for optional subroutine declarations
• Added support for formats
• Ignore comments mixed into definitions and declarations
• Fixed detecting labels with whitespace after label name
• Fixed misidentification of fully qualified function calls as labels

If you've never used tags or ctags, start today. Your life will never be the same after.

For example, if you run ctags on a tree of source code, and then run vim -t some_function from the shell, vim will open the file where some_function lives and leave your cursor there. Alternatively, if you're editing a file with vim and you position your cursor over the word some_function and press Ctrl-], vim will jump to the function. Other editors have similar bindings.

Your editor doesn't know Perl, but it relies on ctags generating a tags file, which is in a standard file. Of course, ctags isn't perl itself, so isn't as exact. These improvements in v5.7 will make your tag files more accurate.

If you've got a Perl module, it's easy to add a makefile target for tags. See any of my major modules (WWW::Mechanize, ack, etc) and steal from the Makefile.PL, or just add the following to your Makefile.PL:

sub MY::postamble {
my $postamble = <<'MAKE_FRAG';
.PHONY: tags
tags:
ctags -f tags --recurse --totals 
--exclude=blib 
--exclude=.svn 
--exclude='*~' 
--languages=Perl --langmap=Perl:+.t 
MAKE_FRAG
}

(With ctags 5.7 it turns out, that --exclude=.svn is no longer necessary, as ctags automatically knows to ignore it now.)

search.cpan.org, which for many people is the CPAN, has added a crucial little enhancement. The star ratings given to distributions at cpanratings.perl.org now appear on the search results. This makes it easy to tell at a glance which modules may best serve your needs.

Say you want a module to work with Excel spreadsheets. Now, when you search on "Excel", the star ratings in the results give the searcher starting points.

Thanks to Graham Barr for making this change.

There's very little case law around Open Source software licenses, but this week we've seen an interesting case involving the Artistic License, under which Perl is distributed.

The blog Law and Life: Silicon Valley has a discussion of the case:

The decision makes two important points: (1) the Artistic License is a contract and (2) the failure to include the copyright notices was not a "restriction" on the scope of the license. The first point is important because the Free Software Foundation and some lawyers have taken the position that open source licenses are not contracts. They have good reasons for wishing to avoid some contract formalities, but this position has complicated discussions about the enforceability and remedies for open source licenses.

The second point is very important because it deals with remedies. Generally, the remedy for contract violations under US law is damages, not "injunctive relief" (which means that the court order a party to cease their violation). On the other hand, copyright infringement generally includes a presumption that injunctive relief is appropriate. Thus, the question of whether the violation of a license is a contract violiation or copyright infringement (it can be both) is very important, because licensors would prefer to obtain an injunction prohibiting the breach of the license.

As pointed out in the article linked above, this decision in relation to the Artistic License doesn't apply to other licenses. This will no doubt have some bearing on how Perl chooses to use the Artistic 2.0 license.

An article on use Perl asks:

• Could the same thing happen under the Artistic 2.0 or Will further revisions to the Artistic license be required?
• Should adoption of Artistic 2.0 wait until Perl 5.10?

However, as a District Court case, this decision only applies in California -- admittedly a very influential place in the tech world, but we can still hope that this decision, which many see as a bad one, will be overturned in a higher court.

Further reading:

• use Perl article
• Slashdot article
• Court papers published by the model railway project involved in this court case.

ohloh.net is a social networking site built around open source projects. People give kudos to each other, and there's some sort of ranking system such that I'm #277 of 29,000 users, which shows that their ranking system is fueled by psychedelics.

ohloh has some cool analyses for the projects, since they analyze the public source repositories of the projects they track. For instance, here's the codebase size for Parrot

There are also great tools to look at who's committed to projects. Take a look at the list of committers to the Parrot project. This is a great use of sparklines to reveal the history of the project through the amount of code committed to the project.

See how Leopold Tösch works like crazy for years on Parrot, then disappears.
My Parrot contributions started out modest, and then I lost interest for a year, then came back a year later.
Compare that to my Perl 5 contributions which are more sporadic, but over a longer period of time.

I've never seen analyses like these done before. If there are others, please let me know in the comments.