• Gerard Goossen talks about Kurila, a Perl 5 fork

    A few days ago Gerard Goossen released version 1.5 of his kurila project to the CPAN, a fork of Perl 5, both the language and the implementation. I talked with about the history of this new direction.


    Andy: Why Kurila? Who would want to use it? What are your goals?

    Gerard: Kurila is a fork of Perl 5. Perl Kurila is a dialect of Perl. Kurila is currently unstable, the language is continuously changing, and has just started.

    There are a few goals, not all of them going in the same direction. One of the goals is to simplify the Perl internals to make hacking on it easier. Another is to make the Perl syntax more consistent, remove some of the oddities, most of them historical legacy.

    What is currently being done is removing some of the more object/error-prone syntax like indirect-object-syntax and removing symbol references. Both of these are not yet very radical yet, most modern Perl doesn't use indirect-object-syntax or symbol references.

    But I am now at the stage of doing more radical changes, like not doing the sigil-change, so that my %foo; $foo{bar} would become my %foo; %foo{bar} .

    Andy: Where do you see Kurila getting used? Who's the target audience for it?

    Gerard: Kurila would be used for anything where currently Perl is being used. I am using Perl for large websites so changes will be favored in that direction.

    I am working for TTY Internet Solutions, a web development company. We develop and maintain websites in Perl, Ruby and Java. Websites we develop include www.2dehands.be, www.sellaband.com, www.ingcard.nl and www.nationalevacaturebank.nl. Of these www.2dehands.be and www.nationalevacaturebank.nl are entirely written in Perl.

    We are not yet using kurila in production, but I have a testing environment of www.2dehands.nl which is running on Kurila. Developing Kurila is part of my work at TTY.

    Many of the changes in Kurila are inspired by bugs/mistakes we made developing these sites. It started with the UTF8 flag. We encountered many problems making our websites UTF-8 compatible. In many cases the UTF8-flag got "lost" somewhere, and after combining it with another string, the string got internally upgraded and our good UTF-8 destroyed. Because everything we have is default UTF-8. The idea was simply to make UTF-8 the default encoding, instead of the current default of latin1.

    Andy: Did you raise the possibility of changing the default encoding in Perl?

    Gerard: The problem is that changing the default encoding the UTF-8 is that is destroys the identity between bytes and codepoints. So it's not a possibility for Perl 5. Like what does chr(255) do? Does it create a byte with value 255 or character with codepoint 255?

    I made a patch removing the UTF-8 flag and changing the default encoding to UTF-8 and sent it to p5p.

    Andy: What was the response?

    Gerard: There was as good as no response to it, I guess because it was obvious that it seriously broke backwards compatibility and the patch was quite big, making it difficult to understand.

    About two weeks after the utf8 patch, I announced that I wanted to change the current Perl 5 development to make it a language which evolves to experiment with new ideas, try new syntax and not be held back by old failed experiments. One of the interesting things about Perl is that it has a lot of different ideas and these are coupled to the syntax.

    There was of course the question of why not Perl 6.  That it should/could be done in backwards-compatible way. That there is no way of making the Perl internals clean, that is better to start over.

    And about half a year ago I announced that I had started Kurila, my proof of-concent for the development of Perl 7. Rewriting some software from scratch is much more difficult then it seems, and I think starting with a well proven good working base is much easier. Perl 5 is there, it is working very good, has few bugs, etc., but it can be much better if you don't have to worry about possibly breaking someone code, and just fix those oddities.

    Andy: Do you have a website for it?  Are you looking for help?

    Gerard: There isn't a website yet, and also no specific mailing list, currently all the discussion is on p5p. There is a public git repository at git://dev.tty.nl/perl.

    Andy: What can someone do if he/she is interested in helping?

    Gerard: Contact me at gerard at tty dot nl. Make a clone of git://dev.tty.nl/perl and start making changes.

  • Bind output variables in DBI for speed and safety

    When in a tight loop of many records from a database, using the quick & dirty solution of calling $sth->fetchrow_hashref can be expensive. I was working on a project to walk through 6,000,000 records and it was slower than I wanted. Some benchmarking showed me that I was paying dearly for the convenience of being able to say my $title = $row->{title};.

    When I converted my code to bind variables to the columns in the statement handle, I cut my run time about 80%. It was as simple as adding this line:

    $sth->bind_columns( my $interestlevel,
    my $av_flag, my $isbn, my $title );
    

    before calling the main loop through the database. Now DBI knows to put the data directly in there, without creating an expensive temporary hash. I also can't make a typo such as $row->{ISBN} inside the loop, so there's a measure of safety as well.

    The benchmarks below show the relative speeds of each of four techniques:

    hashref     took 31.5048 wallclock secs
    array       took 8.83724 wallclock secs
    arrayref    took 5.5308 wallclock secs
    direct_bind took 4.46956 wallclock secs
    

    Here's the key parts of the benchmark program I used:

    use Benchmark ':hireswallclock';
    sub hashref {
    while ( my $row = $sth->fetchrow_hashref ) {
    my $interestlevel = $row->{interestlevel};
    my $av_flag = $row->{av_flag};
    my $isbn = $row->{isbn};
    my $title = $row->{title};
    }
    $sth->finish;
    }
    sub array {
    while ( my @row = $sth->fetchrow_array ) {
    my ($interestlevel, $av_flag, $isbn, $title) = @row;
    }
    $sth->finish;
    }
    sub arrayref {
    while ( my $row = $sth->fetchrow_arrayref ) {
    my $interestlevel = $row->[0];
    my $av_flag = $row->[1];
    my $isbn = $row->[2];
    my $title = $row->[3];
    }
    $sth->finish;
    }
    sub direct_bind {
    $sth->bind_columns( my $interestlevel,
    my $av_flag, my $isbn, my $title );
    while ( my $row = $sth->fetch ) {
    # no need to copy
    }
    $sth->finish;
    }
    for my $func ( qw( hashref array arrayref direct_bind ) ) {
    my $sql = <<"EOF";
    select interestlevel, av_flag, isbn, title
    from testbook
    limit 1000000
    EOF
    # This sub calls the SQL and returns a statement handle
    $sth = sqldo_handle( $sql );
    my $t = timeit( 1, "$func()" );
    print "$func took ", timestr($t), "n";
    }
    

    Did you find this article useful? Or does it not belong on Perlbuzz? Let us know what you think.

  • Perl 5.10's first release candidate coming soon

    The first release candidate of Perl 5.10, with the first new syntax and major features since 2002, will be released soon, probably in the next week or two. The code has been in feature freeze for weeks, and only minor patches are being accepted. Lately the VMS porters have been working on compatibility problems with File::Spec and File::Patch, and Jos Boumans and Ken Williams are syncing core CPAN modules with the Perl source trunk.

    For a gentle and well-presented introduction to the features in Perl 5.10, see Ricardo Signes' slides for his talk Perl 5.10 For People Who Aren't Totally Insane.

    Please note that Perl 5.10 is not Ponie. Ponie was the project that was to put Perl 5.10 on top of the Parrot virtual machine, but Ponie has been put out to pasture.

  • Perl Survey results released

    primary_language_spoken.png

    It's been a while, because I had all kinds of overblown expectations of being able to offer the results in a range of formats, but I have to admit that a month in a developing country with only intermittent satellite internet has bested me.

    Therefore, I've decided to release the results of the Perl Survey in PDF format for now, with the HTML version to follow as time permits.

    Some points of interest:

    • 4% of the Perl community are women
    • Average salary for Perl programmers worldwide is $68,687 (USD equivalent)
    • 87% of Perl people are using 5.8.X at least some of the time; that means that 13% are still on 5.6.X or lower
    • 33% of respondents are members of their a Perl Mongers group
    • However, 27% of respondents don't participate in the Perl community at all (or at least not in any way that we asked about)

    Find out all about this, and more, at the link above.

    Also, the entire data set for the survey has been released under a Creative Commons license, and I'd like to encourage everyone to download it and play with it. A number of people have already contributed their own analyses of certain parts of the data.

    I'd like to thank everyone involved for their help, and I hope that this information will be of use to the Perl community going forward.

  • Perl has supported threading for years

    A recent column in Software Development Times lamented the lack of support for threading in software today. In it, Andrew Binstock says:

    Dynamic languages are even further behind. To wit, Python, allows only one thread to run at a time (except for I/O); this means you can have threads but not running in parallel. Ruby can run threads only within the one VM, which is arguably better but nowhere near good enough. And OpenMP, which might be a solution for some, is limited to C++ or Fortran.

    Unfortunately, Binstock ignores the dynamic language that should be first on his mind, Perl. Perl's threading got stable back with Perl 5.8.7 in 2005, after a few years of experimental support. Today, Mac OS X and most Linux distributions ship with a threaded Perl enabled.

    How can you tell if your Perl supports threads? Look at the output of perl -V for the string "useithreads":

    $ perl -V | grep ithreads
    config_args='-ds -e -Dprefix=/usr -Dccflags=-g  -pipe  -Dldflags=-Dman3ext=3pm -Duseithreads -Duseshrplib'
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    

    While I agree with the basic premise that demand for threaded apps on the desktop is low, the tools for supporting threads are not quite as sparse as Binstock would have us think.

    P.S. O'Reilly & Associates is now O'Reilly Media, and has been since April 2004.