Jekyll2022-07-13T23:34:18-05:00https://perlbuzz.com//PerlbuzzNews and notes about Perl 5Andy Lesterfor, map, grep and function calls autovivify arrayrefs2020-02-04T21:36:45-06:002020-02-04T21:36:45-06:00https://perlbuzz.com/2019/03/18/autovivification<p>I was surprised by one aspect of Perl’s autovivification a few weeks ago,
and I’m surprised it’s taken over 20 years for me to run into it.</p>
<p>I’m used to autovivification being a handy thing when working with, for
example, hashes of arrayrefs. Here I’m using autovivification to create
hash entries in <code class="language-plaintext highlighter-rouge">%users_by_category</code> whenever a new one is added.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>my %users_by_category;
for my $user ( @users ) {
my $category = categorize($user);
push @{$users_by_category{$category}}, $user;
}
</code></pre></div></div>
<p>This saves from having to do the tedious equivalent:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>if ( not exists $users_by_category{$category} ) {
$users_by_category{$category} = [];
}
push $users_by_category{$category}, $user;
</code></pre></div></div>
<p>Autovivification doesn’t work just because you dereference an undef value. For example:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>my $x;
my $n = scalar @{$x};
Can't use an undefined value as an ARRAY reference at ...
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">push</code> in the first example works because using
<code class="language-plaintext highlighter-rouge">@{$users_by_category{$category}}</code> in a <code class="language-plaintext highlighter-rouge">push</code> makes it an “lvalue”,
meaning it’s a value that can appear on the left side of an assignment
operator, or something that can be assigned to.</p>
<p>What surprised me is that <code class="language-plaintext highlighter-rouge">map</code>, <code class="language-plaintext highlighter-rouge">grep</code>, <code class="language-plaintext highlighter-rouge">for</code> and function calls are all lvalue context.</p>
<p>They’re lvalues because in all those cases, the value of the array being
passed can be modified. It makes perfect sense when I thought about it,
but I’d never run into it.</p>
<p>Here’s an example with <code class="language-plaintext highlighter-rouge">grep</code>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>my $x;
grep {1} @{$x};
say "x is $x";
x is ARRAY(0x1e29fc8)
</code></pre></div></div>
<p>Or even just a regular function call:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sub foo {}
my $x;
foo(@{$x});
say "x is $x";
x is ARRAY(0x1059fc8)
</code></pre></div></div>
<p>To help protect against autovivification in your code when you’re not
expecting it, you can look at the
<a href="https://metacpan.org/pod/autovivification"><code class="language-plaintext highlighter-rouge">autovivification</code></a> pragma
available on CPAN.</p>
<p>Thanks to Leon Timmermans and Paul Evans for their input.</p>Andy LesterI was surprised by one aspect of Perl’s autovivification a few weeks ago, and I’m surprised it’s taken over 20 years for me to run into it.ack 2.28 has been released with new color charts2019-03-18T13:05:35-05:002019-03-18T13:05:35-05:00https://perlbuzz.com/2019/03/18/ack-2-28-released<p>I’ve released <a href="https://metacpan.org/pod/distribution/ack/ack">ack 2.28 to the CPAN</a>. This release doesn’t add any major
new features, but for those who like to customize things, there are two new help options: <code class="language-plaintext highlighter-rouge">--help-colors</code> and
<code class="language-plaintext highlighter-rouge">--help-rgb-colors</code>.</p>
<p>The <code class="language-plaintext highlighter-rouge">--help-colors</code> option shows a chart of various color combinations using the standard eight ANSI terminal colors.</p>
<p><img src="/assets/images/2019/ack-color-chart.png" alt="ack color chart" /></p>
<p>The <code class="language-plaintext highlighter-rouge">--help-rgb-colors</code> option shows a grid of the 216 colors possible using RGB color specifications.</p>
<p><img src="/assets/images/2019/ack-color-chart-rgb.png" alt="ack RGB color chart" /></p>
<p>Thanks to Bill Ricker for his work in getting the RGB color chart organized in a useful way, via the
<a href="https://metacpan.org/pod/Convert::Color">Convert::Color</a> module.</p>
<p>ack is available on the CPAN, and also as a single-file download at <a href="https://beyondgrep.com">beyondgrep.com</a>.</p>
<p>Let me know if you find it useful.</p>Andy LesterI’ve released ack 2.28 to the CPAN. This release doesn’t add any major new features, but for those who like to customize things, there are two new help options: --help-colors and --help-rgb-colors.perl101.org needs a new maintainer2019-03-10T15:34:55-05:002019-03-10T15:34:55-05:00https://perlbuzz.com/2019/03/10/perl101-org-needs-a-new-maintainer<p>My website <a href="http://perl101.org">perl101.org</a> needs a new maintainer.</p>
<p>I’ve had it up and running since 2007 with the intent of having a site that had everything a beginning Perl programmer
needed to know. The intent was that there be a site where someone learning on the fly could learn important things like
“Don’t use capture variables without checking that the match succeeded” that might not be covered by most online
tutorials.</p>
<p>It’s had very little attention from me for years, and I don’t want to renew the domain when it comes up
for renewal in a few weeks.</p>
<p>According to Google Analytics, the site got 5K users and 6K sessions in the past 28 days.</p>
<p>If you’re interested in taking over the site, please make an issue in the GitHub issue tracker for the site and let me
know what you would to keep it going.</p>
<p>If nobody steps up to maintain it, I’m going to let it expire in a couple of weeks.</p>Andy LesterMy website perl101.org needs a new maintainer.Announcing SQL::Tiny2019-03-10T14:34:55-05:002019-03-10T14:34:55-05:00https://perlbuzz.com/2019/03/10/announcing-sql-tiny<p>I just released a new module, <a href="https://metacpan.org/pod/SQL::Tiny">SQL::Tiny</a>. It’s intended for creating simple SQL
statements where speed is not as important as having maintainable code.</p>
<p>SQL::Tiny is a very simple SQL-building library. It’s not for all your SQL needs, only the very simple ones.</p>
<p>In my test suites, I have a lot of ad hoc SQL queries, and it drives me nuts to have so much SQL code lying around.
SQL::Tiny is for generating SQL code for simple cases.</p>
<p>I’d far rather have:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>my ($sql,$binds) = SQL::Tiny::sql_insert(
'users',
{
name => 'Dave',
salary => 50000,
phone => undef,
status => 'Active',
dateadded => \'SYSDATE()',
}
);
</code></pre></div></div>
<p>than hand-coding:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>my $sql =
'INSERT INTO users (name,salary,phone,status,dateadded) '.
'VALUES (:name,:status,NULL,:salary,SYSDATE())';
my $binds = {
':name' => 'Dave',
':salary' => 50000,
':status' => 'Active',
};
</code></pre></div></div>
<p>or even the positional:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>my $sql =
'INSERT INTO users (name,salary,phone,status,dateadded) '.
'VALUES (?,?,NULL,?,SYSDATE())';
my $binds = [ 'Dave', 50000, 'Active' ];
</code></pre></div></div>
<p>The trade-off for that brevity of code is that SQL::Tiny has to make new SQL and binds from the input every time. You
can’t cache the SQL that comes back from SQL::Tiny because the placeholders could vary depending on what the input data
is. Therefore, you don’t want to use SQL::Tiny where speed is essential.</p>
<p>The other trade-off is that SQL::Tiny handles only very simple code. It won’t handle JOINs of any kind.</p>
<p>SQL::Tiny isn’t meant for all of your SQL needs, only the simple ones that you do over and over.</p>
<p>I looked at the fine <a href="https://metacpan.org/pod/SQL::Abstract">SQL::Abstract</a>, but it was more complex than I wanted to
deal with, and more importantly, I didn’t want to bring in a trail of depencies.</p>Andy LesterI just released a new module, SQL::Tiny. It’s intended for creating simple SQL statements where speed is not as important as having maintainable code.Use Getopt::Long even if you don’t think you need to2018-08-03T11:10:53-05:002018-08-03T11:10:53-05:00https://perlbuzz.com/2018/08/03/use-getoptlong-even-if-you-dont-think-you-need-to<p><i>Note: This is a post from 2008 that I dug up that still applies today.</i></p>
<p>
There's a <a href="http://perlmonks.org/?node_id=687283">thread over on perlmonks</a> talks about Tom Christiansen's
assertion that you should use it, by default, even when you only have one command-line argument to parse:
</p>
<blockquote>
What seems to happen is that at first we just want to add--oh say for example JUST ONE, SINGLE LITTLE -v flag.
Well, that's so easy enough to hand-hack, that of course we do so... But just like any other piece of software,
these things all seem to have a way of overgrowing their original expectations... Getopt::Long is just *wonderful*,
up--I believe--to any job you can come up with for it. Too often <b>its absence means that I've in the long run
made more work for myself--or others--by not having used it originally.</b> [Emphasis mine -- Andy]
</blockquote>
<p>
I can't agree more. I don't care if you use <a href="http://search.cpan.org/dist/Getopt-Long">Getopt::Long</a> or <a href="http://search.cpan.org/dist/Getopt-Declare">Getopt::Declare</a> or <a href="http://search.cpan.org/dist/Getopt-Lucid">Getopt::Lucid</a> or any of the other <a href="http://search.cpan.org/search?query=getopt&mode=dist">variants out there</a>. You know <i>know</i> <b>know</b> that you're going to add more arguments down the road, so why not start out right?
</p>
<p>
Yes, it can be tricky to get through all of its magic if you're unfamiliar with it, but it's pretty obvious when you
see enough examples. Take a look at <i>prove</i> or <a href="https://beyondgrep.com/"><i>ack</i></a> for examples. <a
href="http://search.cpan.org/src/PETDANCE/WWW-Mechanize-1.34/bin/mech-dump">mech-dump</a> is pretty decent as an
example as well:
</p>
<pre>
GetOptions(
'user=s' => \$user,
'password=s' => \$pass,
forms => sub { push( @actions, \&dump_forms ); },
links => sub { push( @actions, \&dump_links ); },
images => sub { push( @actions, \&dump_images ); },
all => sub { push( @actions, \&dump_forms, \&dump_links, \&dump_images ); },
absolute => \$absolute,
'agent=s' => \$agent,
'agent-alias=s' => \$agent_alias,
help => sub { pod2usage(1); },
) or pod2usage(2);
</pre>
<p>Where the value in the hashref is a variable reference, the value gets stored in there. Where it's a <tt>sub</tt>, that sub gets executed with the arguments passed in. That's the basics, and you don't have to worry about anything else. Your user can pass --abs instead of --absolute if it's unambiguous. You can have mandatory flags, as in agent=s, where --agent must take a string. On and on, it's probably got the functionality you need.
</p>
<p>
One crucial reminder: You <b>must</b> check the return code of <tt>GetOptions</tt>. Otherwise, your program will carry on. If someone gives your program an invalid argument on the command-line, then you <b>know</b> that the program cannot possibly be running in the way the user intended. Your program must stop immediately.
</p>
<p>
Not checking the return of <tt>GetOptions</tt> is as bad as not checking the return of <tt>open</tt>. In fact, I think I smell a new Perl::Critic policy....
</p>Andy LesterNote: This is a post from 2008 that I dug up that still applies today. There's a thread over on perlmonks talks about Tom Christiansen's assertion that you should use it, by default, even when you only have one command-line argument to parse: What seems to happen is that at first we just want to add--oh say for example JUST ONE, SINGLE LITTLE -v flag. Well, that's so easy enough to hand-hack, that of course we do so... But just like any other piece of software, these things all seem to have a way of overgrowing their original expectations... Getopt::Long is just *wonderful*, up--I believe--to any job you can come up with for it. Too often its absence means that I've in the long run made more work for myself--or others--by not having used it originally. [Emphasis mine -- Andy] I can't agree more. I don't care if you use Getopt::Long or Getopt::Declare or Getopt::Lucid or any of the other variants out there. You know know know that you're going to add more arguments down the road, so why not start out right? Yes, it can be tricky to get through all of its magic if you're unfamiliar with it, but it's pretty obvious when you see enough examples. Take a look at prove or ack for examples. mech-dump is pretty decent as an example as well: GetOptions( 'user=s' => \$user, 'password=s' => \$pass, forms => sub { push( @actions, \&dump_forms ); }, links => sub { push( @actions, \&dump_links ); }, images => sub { push( @actions, \&dump_images ); }, all => sub { push( @actions, \&dump_forms, \&dump_links, \&dump_images ); }, absolute => \$absolute, 'agent=s' => \$agent, 'agent-alias=s' => \$agent_alias, help => sub { pod2usage(1); }, ) or pod2usage(2); Where the value in the hashref is a variable reference, the value gets stored in there. Where it's a sub, that sub gets executed with the arguments passed in. That's the basics, and you don't have to worry about anything else. Your user can pass --abs instead of --absolute if it's unambiguous. You can have mandatory flags, as in agent=s, where --agent must take a string. On and on, it's probably got the functionality you need. One crucial reminder: You must check the return code of GetOptions. Otherwise, your program will carry on. If someone gives your program an invalid argument on the command-line, then you know that the program cannot possibly be running in the way the user intended. Your program must stop immediately. Not checking the return of GetOptions is as bad as not checking the return of open. In fact, I think I smell a new Perl::Critic policy....ack 2.24 is released, speeding up common cases2018-06-21T18:34:55-05:002018-06-21T18:34:55-05:00https://perlbuzz.com/2018/06/21/ack-2-24-is-released-speeding-up-common-cases<p>
I've just uploaded a new version of ack, version 2.24, <a href="https://metacpan.org/pod/distribution/ack/ack">to the
CPAN</a> and posted it to <a href="https://beyondgrep.com/">beyondgrep.com</a>.
</p>
<p>
This version of ack introduces an optimization to not search most files line-by-line until after ack has read the whole
file and determined that the pattern matches in the file at all. This speeds things up quite a bit. Here are some
timings in seconds for acking against the Drupal codebase. It compares ack 2.22, the just-released 2.24 and the latest
beta of <a href="https://beyondgrep.com/ack3/">ack version 3</a>:
</p>
<pre>
| 2.22 | 2.24 | 3 beta
--------------------------------------------------------------
ack zqj /home/andy/drupal | 1.93 | 1.53 | 1.46
ack zqj-not-there /home/andy/drupal | 1.85 | 1.50 | 1.45
ack foo /home/andy/drupal | 1.99 | 1.66 | 1.62
ack foo -w /home/andy/drupal | 1.94 | 1.63 | 1.57
ack foo\w+ -C10 /home/andy/drupal | 4.54 | 2.13 | 1.97
ack (set|get)_\w+ /home/andy/drupal | 2.01 | 1.79 | 1.79
</pre>
<p>
Of course ack doesn't have the raw speed of a tool like ripgrep, but it's got a different feature set. See this
<a href="https://beyondgrep.com/feature-comparison/">feature comparison chart of greplike tools</a> for details.
</p>Andy LesterI've just uploaded a new version of ack, version 2.24, to the CPAN and posted it to beyondgrep.com.Avoid the vagueness of dies_ok() in Test::Exception2017-06-28T04:53:11-05:002017-06-28T04:53:11-05:00https://perlbuzz.com/2017/06/28/avoid-the-vagueness-of-dies_ok-in-testexception<p>
It's good to check that your code handles error conditions correctly, but <tt>dies_ok()</tt> in
<a href="https://metacpan.org/pod/Test::Exception">Test::Exception</a> is too blunt an instrument to do it.
</p>
<p>
Consider this code that checks that the <tt>func()</tt> subroutine dies if not passed an argument.
</p>
<pre style="code">
#!/var/perl/bin/perl
use warnings;
use strict;
use Test::More tests => 4;
use Test::Exception;
sub func {
die 'Must pass arg' unless defined $_[0];
}
# Test for failures if the arg is not passed.
dies_ok( sub { func() }, '#1: Dies without an argument' );
throws_ok( sub { func() }, qr/Must pass arg/, '#2: Throws without an argument' );
lives_ok( sub { func(42) }, '#3: Lives with an argument' );
# Oops, we made a typo in our function name, but this dies_ok() still passes.
dies_ok( sub { func_where_the_name_is_incorrect() }, '#4: Func dies without an argument' );
</pre>
<p>
In case #4, the call to <tt>func_where_the_name_is_incorrect()</tt> indeed dies, but for the wrong reason. It dies because the function doesn't exist. If we had used <tt>throws_ok</tt> instead of <tt>dies_ok</tt> like so:
</p>
<pre style="code">
throws_ok( sub { func_where_the_name_is_incorrect() }, qr/Must pass arg/, '#4: Func dies without an argument' );
</pre>
<p>
then the test would have failed because the exception was incorrect:
</p>
<pre>
# Failed test '#4: Func dies without an argument'
# at foo.t line 19.
# expecting: Regexp ((?^:Must pass arg))
# found: Undefined subroutine &main::func_where_the_name_is_incorrect called at foo.t line 19.
</pre>
<p>
Why do I post this? I found an example of this in some code I was working with, where the test had been passing for the
wrong reason for the past six years. Take the time to be specific in what you check for.
</p>Andy LesterIt's good to check that your code handles error conditions correctly, but dies_ok() in Test::Exception is too blunt an instrument to do it.Dueling code wizardry is one of the things I love most about Perl2017-06-27T09:03:20-05:002017-06-27T09:03:20-05:00https://perlbuzz.com/2017/06/27/dueling-code-wizardry-is-one-of-the-things-i-love-most-about-perl<p>
At least week's Perl Conference, Damian Conway talked about some new magical awesomeness he created, as he so frequently does. It's <a href="https://metacpan.org/pod/Test::Expr">Test::Expr</a>, and it makes it easier to write tests:
</p>
<pre>
# Write this ... ... instead of this.
ok $got eq $expected; is $got, $expected;
ok $got ne $unexpected; isnt $got, $unexpected;
ok $got == $expected; is_deeply $got, $expected;
ok $got ~~ $expected; unlike $got, $pattern;
ok $got =~ $pattern; like $got, $pattern;
ok $got !~ $pattern; unlike $got, $pattern;
ok $obj->isa($classname); is_ok $got, $classname;
ok $obj->can($methodname); can_ok $obj, $methodname;
</pre>
<p>
It also improves the diagnostics by showing the expression that failed.
</p>
<pre>
# Failed test '$got eq $expected'
# at t/synopsis.t line 13.
# because:
# $got --> "1.0"
# $expected --> 1
</pre>
<p>
Chad Granum, the maintainer of much of Perl's testing infrastructure took that last part as a challenge and overnight created his own magic in response: <a href="https://metacpan.org/pod/Test2::Plugin::SourceDiag">Test2::Plugin::SourceDiag</a>.
</p>
<pre>
use Test2::V0;
use Test2::Plugin::SourceDiag;
ok(0, "fail");
done_testing;
</pre>
<p>
Produces the output:
</p>
<pre>
not ok 1 - fail
Failure source code:
# ------------
# 4: ok(0, "fail");
# ------------
# Failed test 'fail'
# at test.pl line 4.
</pre>
instead of:
<pre>
not ok 1 - fail
# Failed test 'fail'
# at foo.t line 4.
</pre>
<p>
This kind of dueling wizardry is one of the things that I love so much about Perl and its community.
</p>
<p>
Watch Chad's lightning talk:
</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/wGTZCaU_z5U" frameborder="0" allowfullscreen></iframe>Andy LesterAt least week's Perl Conference, Damian Conway talked about some new magical awesomeness he created, as he so frequently does. It's Test::Expr, and it makes it easier to write tests:Improve your test logs with simple distro diagnostics2017-06-11T14:22:24-05:002017-06-11T14:22:24-05:00https://perlbuzz.com/2017/06/11/improve-your-test-logs-with-simple-distro-diagnostics<p>
Automated module testing systems are becoming more and more common. In addition to our long-serving
<a href="http://cpantesters.org/">CPAN Testers</a> service, Perl authors can have their modules tested by
<a href="https://travis-ci.org/">Travis</a> for Linux and <a href="https://www.appveyor.com/">Appveyor</a> for
Windows. CPAN Testers tests each distribution uploaded to PAUSE, whereas Travis and Appveyor keep an eye on your
GitHub account (or other services) and try testing after each push to the home repo.
</p>
<p>
Something that I've found helps out with diagnosing problems is by having a diagnostic dump of modules in the the first test. I'll have a test like t/00-modules.t, like this one from ack:
</p>
<pre><code>
#!perl -T
use warnings;
use strict;
use Test::More tests => 1;
use App::Ack; # For the VERSION
use File::Next;
use Test::Harness;
use Getopt::Long;
use Pod::Usage;
use File::Spec;
my @modules = qw(
File::Next
File::Spec
Getopt::Long
Pod::Usage
Test::Harness
Test::More
);
pass( 'All external modules loaded' );
diag( "Testing ack version $App::Ack::VERSION under Perl $], $^X" );
for my $module ( @modules ) {
no strict 'refs';
my $ver = ${$module . '::VERSION'};
diag( "Using $module $ver" );
}</code></pre>
<p>
Then, when the user or automated tester runs <tt>make test</tt>, the first thing out tells me exactly what we're working with.
</p>
<pre>
[19:15:52] t/00-load.t .................. 1/23 # Testing ack version 2.999_01 under Perl 5.026000, /home/andy/perl5/perlbrew/perls/perl-5.26.0/bin/perl
# Using File::Next 1.16
# Using File::Spec 3.67
# Using Getopt::Long 2.49
# Using Pod::Usage 1.69
# Using Test::Harness 3.38
# Using Test::More 1.302073
</pre>
<p>
This is also very useful for when users have test failures and submit their logs to a bug tracker. It's especially important in this case to show the Getopt::Long version because ack has had problems in the past with some changes in API in the past.
</p>Andy LesterAutomated module testing systems are becoming more and more common. In addition to our long-serving CPAN Testers service, Perl authors can have their modules tested by Travis for Linux and Appveyor for Windows. CPAN Testers tests each distribution uploaded to PAUSE, whereas Travis and Appveyor keep an eye on your GitHub account (or other services) and try testing after each push to the home repo.Perl::Critic 1.128 fixes bugs and works with Perl 5.262017-06-11T13:57:28-05:002017-06-11T13:57:28-05:00https://perlbuzz.com/2017/06/11/perlcritic-1-128-fixes-bugs-and-works-with-perl-5-26I've just released a <a href="https://metacpan.org/release/PETDANCE/Perl-Critic-1.128">new official release of Perl::Critic</a>, the static code analysis tool for Perl. It uses the new version of the PPI Perl-parsing module, and it works with the new Perl 5.26, which does not include <tt>.</tt> in <tt>@INC</tt> by default.
If you've never used Perl::Critic to analyze your code base for potential bugs and stylistic improvements, mostly based on Damian Conway's <i>Perl Best Practices</i>, try it out.
Here's the changelog:
<pre> [Bug Fixes]
* PPI misparsing a module caused an incorrect "Must end with a
recognizable true value." This is fixed by upgrading to PPI
1.224. (GH #696, GH #607)
* A test would fail under the upcoming Perl 5.26 that omits the current
directory from @INC. Thanks, Kent Fredric.
* Fixed an invalid test in the RequireBarewordsIncludes test. Thanks,
Christian Walde. (GH #751)
* If an element contained blank lines then the source "%r" displayed
for a violation was wrong. Thanks, Sawyer X. (GH #702, #734)
[Dependencies]
Perl::Critic now requires PPI 1.224. PPI is the underlying Perl parser
on which Perl::Critic is built, and 1.224 introduces many parsing fixes
such as:
* Fixes for dot-in-@INC.
* Parse left side of => as bareword even if it looks like a keyword or op.
* $::x now works.
* Higher accuracy when deciding whether certain characters are operators or
variable type casts (*&% etc.).
* Subroutine attributes parsed correctly.
[Performance Enhancements]
* Sped up BuiltinFunctions::ProhibitUselessTopic ~7%. Thanks, James
Raspass. (GH #656)
[Documentation]
* Fixed incorrect explanation of capture variables in
ProhibitCaptureWithoutTest. Thanks, Felipe Gasper.
* Fixed incorrect links. Thanks, Glenn Fowler.
* Fixed incorrect example for returning a sorted list. Thanks, @daviding58.
* Fixed invalid POD. Thanks, Jakub Wilk. (GH #735)
* Updated docs on ProhibitYadaOperator. Thanks, Stuart A Johnston. (GH #662)
* Removed all the references to the old mailing list and code repository
at tigris.org. (GH #757)
</pre>Andy LesterI've just released a new official release of Perl::Critic, the static code analysis tool for Perl. It uses the new version of the PPI Perl-parsing module, and it works with the new Perl 5.26, which does not include . in @INC by default.Perl::Critic releases its first new developer release in 21 months2017-05-26T04:57:32-05:002017-05-26T04:57:32-05:00https://perlbuzz.com/2017/05/26/perlcritic-releases-its-first-new-developer-release-in-21-months<p>
I've just released a <a href="https://metacpan.org/release/PETDANCE/Perl-Critic-1.127_02">new developer release of Perl::Critic</a>, the static code analysis tool for Perl, as we work toward its first new release in 21 months. This version of Perl::Critic fixes a few bugs and relies on a new release of the underlying Perl parsing library PPI, which also has had its first new release in a while.
</p>
<p>
This version of Perl::Critic is also ready for the impending release of Perl 5.26, which will no longer include <tt>.</tt> in <tt>@INC</tt> by default.
</p>
<p>
I've been spending some time working through the <a href="https://github.com/Perl-Critic/Perl-Critic/issues">issues in the GitHub project</a>, cleaning up what I can and clarifying others.
</p>
<p>
If you've never used Perl::Critic to analyze your code base for potential bugs and stylistic improvements, mostly based on Damian Conway's <i>Perl Best Practices</i>, try it out.
</p>
<p>
Here's the changelog:
</p>
<pre>
[Bug Fixes]
* PPI misparsing a module caused an incorrect "Must end with a
recognizable true value." This is fixed by upgrading to PPI
1.224. (GH #696, GH #607)
* A test would fail under the upcoming Perl 5.26 that omits the current
directory from @INC. Thanks, Kent Fredric.
* Fixed an invalid test in the RequireBarewordsIncludes test. Thanks,
Christian Walde. (GH #751)
* If an element contained blank lines then the source "%r" displayed
for a violation was wrong. Thanks, Sawyer X. (GH #702, #734)
[Dependencies]
Perl::Critic now requires PPI 1.224. PPI is the underlying Perl parser
on which Perl::Critic is built, and 1.224 introduces many parsing fixes
such as:
* Fixes for dot-in-@INC.
* Parse left side of => as bareword even if it looks like a keyword or op.
* $::x now works.
* Higher accuracy when deciding whether certain characters are operators or
variable type casts (*&% etc.).
* Subroutine attributes parsed correctly.
[Performance Enhancements]
* Sped up BuiltinFunctions::ProhibitUselessTopic ~7%. Thanks, James
Raspass. (GH #656)
[Documentation]
* Fixed incorrect explanation of capture variables in
ProhibitCaptureWithoutTest. Thanks, Felipe Gasper.
* Fixed incorrect links. Thanks, Glenn Fowler.
* Fixed incorrect example for returning a sorted list. Thanks, @daviding58.
* Fixed invalid POD. Thanks, Jakub Wilk. (GH #735)
* Updated docs on ProhibitYadaOperator. Thanks, Stuart A Johnston. (GH #662)
* Removed all the references to the old mailing list and code repository
at tigris.org. (GH #757)
</pre>Andy LesterI've just released a new developer release of Perl::Critic, the static code analysis tool for Perl, as we work toward its first new release in 21 months. This version of Perl::Critic fixes a few bugs and relies on a new release of the underlying Perl parsing library PPI, which also has had its first new release in a while.Speed up DBI reads by binding variables directly2017-04-27T04:38:33-05:002017-04-27T04:38:33-05:00https://perlbuzz.com/2017/04/27/speed-up-dbi-reads-by-binding-variables-directly<p>
If you're using <a href="https://metacpan.org/pod/DBI">DBI</a> directly for your database access, not through some ORM, then <tt>fetchrow_hashref</tt> is probably the handiest way to fetch result rows. However, if you're working on lots of rows and time is critical, know that it is also be the slowest way to do so.
</p>
<p>
Here's a benchmark that shows that binding columns with <tt>bind_column</tt> takes half the runtime of <tt>fetchrow_hashref</tt>.
</p>
<pre><code>
use strict;
use warnings;
use 5.010;
use Benchmark ':hireswallclock';
our $ITERATIONS = 1_000_000;
our $sth;
sub prep_handle {
my $sql = <<"EOF";
SELECT title, author, isbn
FROM title
WHERE ROWNUM < $ITERATIONS
EOF
return sqldo_handle( $sql ); # Calls DBI->prepare
}
sub hashref {
while ( my $row = $sth->fetchrow_hashref ) {
my $title = $row->{title};
my $author = $row->{author};
my $isbn = $row->{isbn};
}
$sth->finish;
}
sub array {
while ( my @row = $sth->fetchrow_array ) {
my ($title,$author,$isbn) = @row;
}
$sth->finish;
}
sub arrayref {
while ( my $row = $sth->fetchrow_arrayref ) {
my $title = $row->[0];
my $author = $row->[1];
my $isbn = $row->[2];
}
$sth->finish;
}
sub direct_bind {
$sth->bind_columns( \my $title, \my $author, \my $isbn );
while ( my $row = $sth->fetch ) {
# No need to copy.
}
$sth->finish;
}
say "Running $ITERATIONS iterations";
for my $func ( qw( hashref array arrayref direct_bind ) ) {
$sth = prep_handle();
my $t = timeit( 1, "$func()" );
printf( "%-11s took %s\n", $func, timestr($t) );
}
</code></pre>
Which gives these results
<pre>
$ ./dbi-bind-bench
Running 1000000 iterations
hashref took 7.37747 wallclock secs ( 4.98 usr + 0.25 sys = 5.23 CPU) @ 0.19/s (n=1)
array took 4.01768 wallclock secs ( 1.68 usr + 0.19 sys = 1.87 CPU) @ 0.53/s (n=1)
arrayref took 3.86365 wallclock secs ( 1.60 usr + 0.16 sys = 1.76 CPU) @ 0.57/s (n=1)
direct_bind took 3.36962 wallclock secs ( 1.13 usr + 0.15 sys = 1.28 CPU) @ 0.78/s (n=1)
</pre>
<p>
When speed is key, bind your output variables directly.
</p>Andy LesterIf you're using DBI directly for your database access, not through some ORM, then fetchrow_hashref is probably the handiest way to fetch result rows. However, if you're working on lots of rows and time is critical, know that it is also be the slowest way to do so.ack 2.18 has been released; ack 3 starting development2017-03-24T10:25:21-05:002017-03-24T10:25:21-05:00https://perlbuzz.com/2017/03/24/ack-2-18-has-been-released-ack-3-starting-developmentI've just uploaded ack 2.18 to CPAN and to <a href="https://beyondgrep.com/">https://beyondgrep.com/</a>.
ack 2.18 will probably be the final release in the ack 2.x series. I'm going to be starting work on ack 3.000 in earnest. Still, if you discover problems with ack 2, please report them to <a href="https://github.com/petdance/ack2/issues">https://github.com/petdance/ack2/issues</a>
If you're interested in ack 3 development, please sign up for the ack-dev mailing list and/or join the ack Slack. See <a href="https://beyondgrep.com/community/">https://beyondgrep.com/community/</a> for details.
<pre>2.18 Fri Mar 24 14:53:19 CDT 2017
====================================
[FIXES]
ack no longer throws an undefined variable warning if it's called
from a directory that doesn't exist. (GH #634)
--context=0 (and its short counterpart -C 0) did not set to context
of 0. This means that a command-line --context=0 couldn't override
a --context=5 in your ackrc. Thanks, Ed Avis. (GH #595)
t/ack-s.t would fail in non-English locales. Thanks, Olivier Mengué.
(GH #485, GH #515)
[ENHANCEMENTS]
--after-context and --before-context (and their short counterparts
-A and -B) no longer require a value to be passed. If no value is
set, they default to 2. (GH #351)
Added .xhtml to the --html filetype. Added .wsdl to the --xml filetype.
Thanks, H.Merijn Brand. (GH #456)
[DOCUMENTATION]
Explain that filetypes must be two characters or longer. (GH #389)
Updated incorrect docs about how ack works. Thanks, Gerhard Poul.
(GH #543)
[INTERNALS]
Removed the abstraction of App::Ack::Resource and its subclass
App::Ack::Resource::Basic. We are abandoning the idea that we'll have
plugins.
Removed dependency on File::Glob which isn't used.
</pre>Andy LesterI've just uploaded ack 2.18 to CPAN and to https://beyondgrep.com/.ack 2.16 has been released2017-03-10T09:35:50-06:002017-03-10T09:35:50-06:00https://perlbuzz.com/2017/03/10/ack-2-16-has-been-releasedack 2.16 has been released. ack is a grep-like tool optimized for searching source code. It's available at <a href="https://beyondgrep.com">https://beyondgrep.com</a>, or via CPAN using App::Ack.
Here are the changes between 2.16 and 2.14.
<pre>2.16 Fri Mar 10 13:32:39 CST 2017
====================================
[CONFUSING BEHAVIOR & UPCOMING CHANGES]
The -w has a confusing behavior that it's had since back to ack 1.x
that will be changing in the future. It's not changing in this
version, but this is a heads-up that it's coming.
ack -w is "match a whole word", and ack does this by putting turning
your PATTERN into \bPATTERN\b. So "ack -w foo" effectively becomes
"ack \bfoo\b". Handy.
The problem is that ack doesn't put a \b before PATTERN if it begins
with a non-word character, and won't put a \b after PATTERN if it
ends with a non-word character.
The problem is that if you're searching for "fool" or "foot", but
only as a word, and you do "ack -w foo[lt]" or "ack -w (fool|foot)",
you'll get matches for "football and foolish" which certainly should
not match if you're using -w.
[ENHANCEMENTS]
Include .cljs, .cljc and .edn files with the --clojure filetype. Thanks,
Austin Chamberlin.
Added .xsd to the --xml filetype. Thanks, Nick Morrott.
Added support for Swift language. Thanks, Nikolaj Schumacher. (GH #512)
The MSYS2 project is now seen as Windows. Thanks, Ray Donnelly. (GH #450)
Expand the definition of OCaml files. Thanks, Marek Kubica. (GH #511)
Add support for Groovy Server Pages. Thanks, Ethan Mallove. (GH #469)
The JSP filetype (--jsp) now recognizes .jspf files. Thanks, Sebastien
Feugere. (GH #586)
Added --hpp option for C++ header files. Thankis, Steffen Jaeckel.
ack now supports --ignore-dir=match:.... Thanks, Ailin Nemui! (GitHub ticket #42)
ack also supports --ignore-dir=ext:..., and --noignore-dir supports match/ext as well
[FIXES]
Reverted an optimization to make \s work properly again. (GH #572,
GH #571, GH #562, GH #491, GH #498)
The -l and -c flags would sometimes return inaccurate results due to
a bug introduced in 2.14. Thanks to Elliot Shank for the report! (GH #491)
Behavior when using newlines in a search was inconsistent. Thanks to Yves Chevallier
for the report! (GH #522)
Don't ignore directories that are specified as command line targets (GH #524)
Fix a bug where a regular expression that matches the empty string could cause ack
to go into an infinite loop (GH #542)
[INTERNALS]
Add minimal requirement of Getopt::Long 2.38, not 2.35, for GetOptionsFromString.
Added test to test --output. Thanks, Varadinsky! (GH #587, GH #590)
Added test to make sure subdirs of target subdirs are ignored if
--ignore-dir applies to them. Thanks, Pete Houston. (GH #570)
Many optimizations and code cleanups. Thanks, Stephan Hohe.
Fixed an out-of-date FAQ entry. Thanks, Jakub Wilk. (GH #580)
[DOCUMENTATION]
Expanded the explanation of how the -w flag works. Thanks, Ed Avis.
(GH #585)
</pre>Andy Lesterack 2.16 has been released. ack is a grep-like tool optimized for searching source code. It's available at https://beyondgrep.com, or via CPAN using App::Ack.Higher-level list utility functions with List::UtilsBy2017-02-13T17:53:43-06:002017-02-13T17:53:43-06:00https://perlbuzz.com/2017/02/13/higher-level-list-utility-functions-with-listutilsby<p>I’m in love with <a href="http://search.cpan.org/dist/List-UtilsBy/lib/List/UtilsBy.pm">List::UtilsBy</a>. It’s one of those “Why
didn’t someone do this earlier?” modules (or maybe it’s “Why didn’t I know about it earlier?”). It replicates much of
the functionality of List::Util and lets them operate on arbitrary blocks.</p>
<h2 id="convenience">Convenience</h2>
<p>I’ve always been annoyed at having to repeat the field name in a <code class="language-plaintext highlighter-rouge">sort</code> sub:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>my @sorted = sort { $a->name cmp $b->name } @users;
</code></pre></div></div>
<p>Now it’s just this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>my @sorted = sort_by { $_->name } @users;
</code></pre></div></div>
<p>I can’t imagine how many times I’ve written something like this to build a hash of the counts of something based on a list:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># Tally up a list of first letters of everyone's name.
my %n;
for my $i ( @users ) {
++$n{ substr($i->name,0,1) };
}
</code></pre></div></div>
<p>Now, with List::UtilsBy, I can do this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>my %n = count_by { substr($_->name,0,1) } @users;
</code></pre></div></div>
<p>How about getting the user with the highest salary?</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>my $highest_paid;
my $max_salary = 0;
for my $user ( @users ) {
if ( (my $val = $user->salary) > $max_salary ) {
$highest_paid = $user;
$max_salary = $val;
}
}
say $highest_paid->name, ' is the highest paid.';
</code></pre></div></div>
<p>Or I can just do this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>my $highest_paid = max_by { $_->salary } @users;
</code></pre></div></div>
<h2 id="performance">Performance</h2>
<p>For most of the functions I’ve tried, List::UtilsBy is slower than the hand-written loop. However, for sorting
<code class="language-plaintext highlighter-rouge">sort_by</code> has been faster because it evaluates the key exactly once for each element of the loop. It was even faster
than rolling my own Schwartzian transform. For complex key methods, the time savings will be dramatic.</p>
<p>I used Benchmark to run a test of 1000 sorts of 5000 input records.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>timethese( 1000, {
# Key comes from a hash lookup.
lookup_raw_____ => sub { @x = sort { $a->{name} cmp $b->{name} } @input },
lookup_schwartz => sub { @x = map { $_->[0] } sort { $a->[1] cmp $b->[1] } map { [$_, $_->{name}] } @input },
lookup_utilsby_ => sub { @x = sort_by { $_->{name} } @input },
# Key comes from a method call.
method_raw_____ => sub { @x = sort { $a->name cmp $b->name } @input },
method_schwartz => sub { @x = map { $_->[0] } sort { $a->[1] cmp $b->[1] } map { [$_, $_->name] } @input },
method_utilsby_ => sub { @x = sort_by { $_->name } @input },
} );
</code></pre></div></div>
<p>Giving these results:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># Key comes from a hash lookup.
lookup_raw_____: 14 wallclock secs @ 72.52/s
lookup_schwartz: 19 wallclock secs @ 53.28/s
lookup_utilsby_: 15 wallclock secs @ 64.47/s
# Key comes from a method call.
method_raw_____: 54 wallclock secs @ 18.64/s
method_schwartz: 22 wallclock secs @ 45.64/s
method_utilsby_: 17 wallclock secs @ 59.59/s
</code></pre></div></div>
<p>In most cases, I’m more than happy to burn a few milliseconds in the name of simplicity and reduced amount of code.</p>
<p>Thanks to Paul Evans for putting this together. And thanks to Dave Rolsky for putting out
<a href="http://search.cpan.org/dist/List-AllUtils">List::AllUtils</a> which brings together
<a href="http://search.cpan.org/dist/List-Util">List::Util</a>, <a href="http://search.cpan.org/dist/List-MoreUtils">List::MoreUtils</a> and
List::UtilsBy under one handy umbrella.</p>Andy LesterI’m in love with List::UtilsBy. It’s one of those “Why didn’t someone do this earlier?” modules (or maybe it’s “Why didn’t I know about it earlier?”). It replicates much of the functionality of List::Util and lets them operate on arbitrary blocks.All about the new Test2 framework and how it will help your tests2016-05-30T09:31:15-05:002016-05-30T09:31:15-05:00https://perlbuzz.com/2016/05/30/all-about-the-new-test2-framework-and-how-it-will-help-your-tests<p>The new Test2 framework has been released after a couple years of development. I wanted to find out about what this means for users of Test::Simple and Test::More, so I chatted with the project leader, Chad Granum (<a href="http://search.cpan.org/~exodist/">exodist</a>).</p>
<p><strong>Andy Lester</strong>: So Test2 has just been released after a couple of years of work, and a lot of discussion. For those of us who haven’t followed its development, what is Test2 and why is it a good thing?</p>
<p><strong>Chad Granum</strong>: The big changes will be for people who write test modules. The old Test::Builder was tied to specific generation of TAP output. That’s been replaced with a flexible event system.</p>
<p>It all started when David Golden submitted a patch to change the indentation of a comment intended for humans who read the test. The change would help people, but meant nothing to the machine. I had to reject the patch because it broke a lot of downstream modules. Things broke because they tested that Test::Builder produced the message in its original form. I thought that was crazy, and wanted to make things easier to maintain, test, and improve.</p>
<p><strong>Andy</strong>: Test::Builder’s internals were pretty fragile?</p>
<p><strong>Chad</strong>: That is true, but that’s not the whole picture. The real problem was the tools people used to validate testing tools. Test::Builder::Tester was the standard, and it boiled down to giant string comparisons of TAP output, which mixes messages for the computer’s use, and messages for human use.</p>
<p>While most of the changes are under the hood, there are <strong>improvements for people who just want to write tests</strong>. Test2 has a built-in synchronization system for forking/threading. If you modify a test to load Test2::IPC before loading Test::More, then you can fork in your tests and it will work in sane/reasonable ways. Up until now doing this required external tools such as Test::SharedFork which had severe limitations.</p>
<p>Another thing I want to note is an improvement in how Test2 tracks file+line number for error reporting purposes. As you know diagnostics are reported when a test fails, and it gives you the filename and line number of the failure. Test::Builder used a global variable <code class="language-plaintext highlighter-rouge">$Test::Builder::Level</code> which people were required to localize and bump whenever they added a stack frame to their tool. This was confusing and easy to get wrong.</p>
<p>Test2 now uses a <code class="language-plaintext highlighter-rouge">Context</code> object. This object solves the problem by locking in the “context” (file + line) when the tool is first called. All nested tools will then find that context. The context object also doubles as the primary interface to Test2 for tool writers, which means it will not be obscure like the <code class="language-plaintext highlighter-rouge">$Level</code> variable was.</p>
<p><strong>Andy</strong>: I just counted 1045 instances of <code class="language-plaintext highlighter-rouge">$Test::Builder::Level</code> in my codebase at work. Are you saying that I can throw them all away when I start using Test2?</p>
<p><strong>Chad</strong>: Yes, <strong>if you switch to using Test2 in those tools you can stop counting your stack frames</strong>. That said, the <code class="language-plaintext highlighter-rouge">$Level</code> variable will continue to work forever for backwards compatibility.</p>
<p><strong>Andy</strong>: Will the TAP output be the same? We’re still using an ancient install of <a href="http://search.cpan.org/dist/Smolder/">Smolder</a> as our CI tool and I believe it expects TAP to look a certain way.</p>
<p><strong>Chad</strong>: Extreme care was taken to ensure that the TAP output did not change in any significant ways. The one exception is David Golden’s change that started all this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Ok 1 - random test
# a subtest
Ok 1 - subtest result
1..1
Ok 2 - a subtest
</code></pre></div></div>
<p>This has changed to:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Ok 1 - random test
# a subtest
Ok 1 - subtest result
1..1
Ok 2 - a subtest
</code></pre></div></div>
<p>That is the change that started all this, and had the potential to break CPAN.</p>
<p><strong>Andy</strong>: So Test2 is all about possibilities for the future. <strong>It’s going to make it easier for people to create new <code class="language-plaintext highlighter-rouge">Test::</code> modules</strong>. As the author of a couple of <code class="language-plaintext highlighter-rouge">Test::</code> modules myself, I know that the testing of the tests is always a big pain. There’s lots of cut & paste from past modules that work and tweaking things until they finally pass the tests. What’s different between the old way of doing the module testing and now?</p>
<p><strong>Chad</strong>: Test::Builder assumed TAP would be the final product, and did not give you any control or hooks into everything between your tool and the TAP, as such you had to test your final TAP output, which often included text you did not yourself produce. In Test2 we drop those assumptions, TAP is no longer assumed, and you also have hooks in almost every step of the process between your tool and the final output.</p>
<p>Many of the actions Test::Builder would accomplish have been turned into Event objects. Test tools do their thing, and then fire events off to Test2 for handling. Eventually these events hit a formatter (TAP by default) and are rendered for a harness. Along with the hooks there is a tool in Test2::API called intercept, it takes a codeblock, all events generated inside that codeblock are captured and returned, they are not rendered and do not affect the global test state. Once you capture your events you can test them as data structures, and ignore ones that are not relevant to your tools.</p>
<p>The Test::Builder::Tester way may seem more simple at first, but that is deceptive. There is a huge loss of information. Also if there are changes to how Test::Builder renders TAP, such as dropping the ‘-‘ then everything breaks.</p>
<h3 id="using-testbuildertester">Using Test::Builder::Tester</h3>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>test_out("ok 1 - a passing test");
ok(1, 'a passing test');
test_test("Got expected line of TAP output");
</code></pre></div></div>
<h3 id="using-intercept-and-basic-testmore-tools">Using intercept and basic Test::More tools</h3>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>my $events = intercept {
ok(1, 'a passing test');
};
my $e = shift @$events;
ok($e->pass, "passing tests event");
is($e->name, "a passing test", "got event name");
is_deeply(
$e->trace->frame,
[__PACKAGE__, __FILE__, 42, 'Test2::Tools::Basic::ok'],
"Got package, file, line and sub name"
);
</code></pre></div></div>
<h3 id="using-test2toolscompare">Using Test2::Tools::Compare</h3>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>like(
intercept {
ok(1, 'a passing test');
},
array {
event Ok => sub {
call pass => 1;
call name => 'a passing test';
prop file => __FILE__;
prop package => __PACKAGE__;
prop line => 42;
prop subname => 'Test2::Tools::Basic::ok';
};
},
'A passing test'
);
</code></pre></div></div>
<p><strong>Andy</strong>: What other features does Test2 include for users who aren’t creating <code class="language-plaintext highlighter-rouge">Test::</code> modules?</p>
<p><strong>Chad</strong>: Test2’s core, which is included in the Test-Simple distribution does not have new features at the user level. However Test2-Suite was released at the same time as Test2/Test-Simple, and it contains new versions of all the Test::More tools, and adds some things people have been requesting for years, but were not possible with the old Test::Builder</p>
<p>The biggest example would be <strong>“die/bail on fail”, which lets you tell the test suite to stop after the first failure</strong>. The old stuff could not do this because there was no good hook point, and important diagnostics would be lost.</p>
<p>It’s as simple as using one of these two modules:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>use Test2::Plugin::DieOnFail;
use Test2::Plugin::BailOnFail;
</code></pre></div></div>
<p>The difference is that <code class="language-plaintext highlighter-rouge">DieOnFail</code> calls <code class="language-plaintext highlighter-rouge">die</code> under the hood. The <code class="language-plaintext highlighter-rouge">BailOnFail</code> will send a bail-out event which will abort the current file, and depending on the harness might stop the entire test run.</p>
<p><strong>Andy</strong>: So <strong>how do I start using Test2</strong>? At my day job, our code base has 1,200 <code class="language-plaintext highlighter-rouge">*.t</code> files totalling 282,000 lines of code. Can I expect to install the new version of Test::Simple (version 1.302019) that includes Test2 and everything will “just work”?</p>
<p><strong>Chad</strong>: For the vast majority of cases the answer is “yes”. Back-compatibility was one of the most significant concerns for the project. That said, some things did unfortunately break. A good guide to what breaks, and why can be found in <a href="https://metacpan.org/pod/distribution/Test-Simple/lib/Test2/Transition.pod">this document</a>. Usually things that break do so because they muck about with the Test::Builder internals in nasty ways. Usually these modules had no choice due to Test::Builder’s limitations. When I found such occurrences I tried to add hooks or APIs to do those things in sane/reasonable ways.</p>
<p><strong>Andy</strong>: Do I have to upgrade? Can I refuse to go up to Test-Simple 1.302019? What are the implications of that?</p>
<p><strong>Chad</strong>: Well, nobody is going to come to you and force you to install the latest version. If you want to keep using your old version you can. You might run into trouble down the line if other <code class="language-plaintext highlighter-rouge">Test::</code> tools you use decide to make use of Test2-specific features, at which point you would need to lock in old versions of those as well. You also would not be able to start using any new tools people build using Test2.</p>
<p><strong>Andy</strong>: And the tools you’re talking about are <code class="language-plaintext highlighter-rouge">Test::</code> modules, right? The command line tool <code class="language-plaintext highlighter-rouge">prove</code> and <code class="language-plaintext highlighter-rouge">make test</code> haven’t changed, because they’re part of Test::Harness?</p>
<p><strong>Chad</strong>: Correct. Test::Harness has not been touched, it will work on any test files that produce TAP, and Test2 still produces TAP by default. That said I do have a project in the works to create an alternative harness specifically for Test2 stuff, but it will never be a requirement to use it, things will always work on Test::Harness.</p>
<p><strong>Andy</strong>: So if I’m understanding the Changes file correctly, Test-Simple 1.302012 was the last old-style version and 1.302014 is the new version with Test2?</p>
<p><strong>Chad</strong>: No, Test-Simple-1.001014 is the last STABLE release of Test-Simple that did not have Test2, then Test-Simple-1.302015 was the first stable release to include Test2. There were a lot of development releases between the 2, but no stable ones. The version numbers had to be carefully crafted to follow the old scheme, but we also had to keep it below 1.5xxxxx because of the previous maintainers’ projects which used that number as well as 2.0. Some downstream users had code switched based on version number and expected an API that never came to be. Most of these downstream distributions have been fixed now, but we are using a “safe” version number just in case.</p>
<p><strong>Andy</strong>: <strong>What has development for this been like?</strong> This has been in the works for, what, two years now? I remember talking to you briefly about it at OSCON 2014.</p>
<p><strong>Chad</strong>: At the point we talked I had just been given Test-Simple, and did not have any plans to make significant changes. What we actually talked about was my project <a href="http://search.cpan.org/dist/Fennec/">Fennec</a> which was a separate Test::Builder based test framework. Some features from Fennec made their way into Test2, enough so that Fennec will be deprecated once I have a stable Test2::Workflow release.</p>
<p>Initially development started as a refactor of Test::Builder that was intended to be fairly small. The main idea was to introduce the events, and a way to capture them. From there it ballooned out as I fixed bugs, or made other changes necessary to support events.</p>
<p>At one point the changes were significant enough, and broke enough downstream modules that I made it a complete fork under the name Test-Stream. I figured it would be easier to make Test::Builder a compatibility wrapper.</p>
<p>In 2015, I attended the <a href="http://act.qa-hackathon.org/qa2015/">QA hackathon in Berlin</a>, and my Test-Stream fork was a huge topic of conversation. The conversation resulted in a general agreement (not unanimous) that it would be nice to have these changes. There was also a list of requests (demands?) for the project before it could go stable. We called it the punch-list.</p>
<p>After the Berlin hackathon there was more interest in the project. Other toolchain people such as <a href="http://search.cpan.org/~haarg/">Graham Knop (Haarg)</a>, <a href="http://search.cpan.org/~bulkdd/">Daniel Dragan (bulk88)</a>, <a href="http://search.cpan.org/~rjbs/">Ricardo Signes (rjbs)</a>, <a href="http://search.cpan.org/~mstrout/">Matt Trout (mst)</a>, <a href="http://search.cpan.org/~ether/">Karen Etheridge (ether)</a>, <a href="http://search.cpan.org/~leont/">Leon Timmermans (leont)</a>, <a href="http://search.cpan.org/~jberger">Joel Berger (jberger)</a>, <a href="http://search.cpan.org/~kentnl/">Kent Fredric (kentnl)</a>, <a href="http://search.cpan.org/~ribasushi">Peter Rabbitson (ribasushi)</a>, etc. started reviewing my code, making suggestions and reporting bugs. This was one of the most valuable experiences. The project as it is now, is much different than it was in Berlin, it is much better from the extra eyes and hands.</p>
<p>A month ago was <a href="http://act.qa-hackathon.org/qa2015/">another QA hackathon, in Rugby UK</a>, and once again Test2 was a major topic. This time the general agreement was that it was ready now. The only new requirements on the table were related to making the broken downstream modules very well known, and also getting a week of extra cpan-testers results prior to release.</p>
<p>I must note that at both QA hackathons the decisions were not unanimous, but in both cases there was a very clear majority.</p>
<p><strong>Andy</strong>: So what’s next? I see that you have a grant for more documentation. Tell me about that, and <strong>what can people do to help</strong>?</p>
<p><strong>Chad</strong>: The Test2 core API is not small, and has more moving pieces than Test::Builder did. Right now there is plenty of technical/module documentation, but there is a lack of overview documentation. There is a need for a manual that helps people find solutions to their problems, and tied the various parts together. This is the first part of the manual docs for tool authors.</p>
<p><a href="http://search.cpan.org/dist/Test2-Suite/">Test2::Suite</a> is also not small, but provides a large set of tools for people to use, some are improvements on old tools, some are completely new. The manual will have a second section on using these new tools. This second part of the manual will be geared towards people writing tests.</p>
<p>The best way for people to help would be to start using Test2::Suite in their tests, and Test2 in their test tools. People will undoubtedly find places where more documentation is needed, or where things are not clear. Reporting such documentation gaps would help me to write better documentation. (<a href="https://github.com/Test-More/test-more">Test::More repo</a>,
<a href="https://github.com/Test-More/Test2-Suite">Test2::Suite repo</a>)</p>
<p>Apart from the documentation, I have 2 other Test2 related projects nearing completion: <a href="http://search.cpan.org/dist/Test2-Workflow/">Test2-Workflow</a>, which is an implementation of the tools from Fennec that are not a core part of Test2, and <a href="http://search.cpan.org/dist/Test2-Harness/">Test2-Harness</a> which is an optional alternative to Test::Harness. Both are pretty much code-complete on GitHub, but neither has the test coverage I feel is necessary before putting them on CPAN.</p>
<p><strong>Andy</strong>: Thanks for all the work that’s gone into this, both to you and the rest of those who’ve contributed. It sounds like we’ll soon see more tools to make testing easier and more robust.</p>Andy LesterThe new Test2 framework has been released after a couple years of development. I wanted to find out about what this means for users of Test::Simple and Test::More, so I chatted with the project leader, Chad Granum (exodist).Template Toolkit’s DEFAULT directive does not short-circuit2014-09-15T05:31:34-05:002014-09-15T05:31:34-05:00https://perlbuzz.com/2014/09/15/template_toolkits_default_directive_does_not_short-circuit<p>Template Toolkit’s DEFAULT does not do short-circuit evaluation like you might think it would.
If you have</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[% DEFAULT foo = user.calculate_foo; %]
</code></pre></div></div>
<p>Then the results of the method call <code class="language-plaintext highlighter-rouge">user.calculate_foo</code> are assigned to <code class="language-plaintext highlighter-rouge">foo</code>, unless <code class="language-plaintext highlighter-rouge">foo</code> already has a true value.
However, whether or not foo already has a true value, <code class="language-plaintext highlighter-rouge">user.calculcate_foo</code> is always invoked.
You might expect the code to look like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$foo = $user->calculate_foo() unless $foo;
</code></pre></div></div>
<p>But it’s really like this;</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$temp = $user->calculate_foo();
$foo = $temp unless $foo;
</code></pre></div></div>
<p>So, if <code class="language-plaintext highlighter-rouge">user.calculate_foo</code> is an expensive function that you don’t want to invoke unless you have to, you’ll have to
resort to an explicit <code class="language-plaintext highlighter-rouge">IF</code> block:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>IF !foo;
foo = user.calculate_foo;
END
</code></pre></div></div>
<p>It’s also worth noting that <code class="language-plaintext highlighter-rouge">DEFAULT</code> is not for checking if a variable is set, but for checking if it is true.</p>Andy LesterTemplate Toolkit’s DEFAULT does not do short-circuit evaluation like you might think it would. If you haveack 2.0 has been released2013-04-19T04:16:09-05:002013-04-19T04:16:09-05:00https://perlbuzz.com/2013/04/19/ack_20_has_been_released<p>
<a href="http://beyondgrep.com/">ack</a> 2.0 has been released. ack is a grep-like search tool that has been optimized for searching large heterogeneous trees of source code.
</p>
<p>
ack has been around since 2005. Since then it has become very popular and is packaged by all the major Linux distributions. It is cross-platform and pure Perl, so will run on Windows easily. See the <a href="http://beyondgrep.com/why-ack/">"Why ack?"</a> page for the top ten reasons, and dozens of testimonials.
</p>
<p>
ack 2.0 has <a href="http://beyondgrep.com/ack-2.0/">many changes from 1.x</a>, but here are four big differences and features that long-time ack 1.x users should be aware of.
</p>
<ul>
<li>By default all text files are searched, not just files with types that ack recognizes. If you prefer the old ack 1.x behavior of only searching files that ack recognizes, you can use the <tt>-k/--known-types</tt> option.</li>
<li>There is a much more flexible type identification system available. You can specify a file type based on extension (.rb for Ruby), filename (Rakefile is a Ruby file), first line matching a regex (Matching <tt>/#!.+ruby/</tt> is a Ruby file) or regex match on the filename itself.</li>
<li>Greater support for ackrc files. You can have a system-wide ackrc at /etc/ackrc, a user-specific ackrc in ~/.ackrc, and ackrc files local to your projects.</li>
<li>The <tt>-x</tt> argument tells ack to read the list of files to search from stdin, much like xargs. This lets you do things like <tt>git ls | ack -x foo</tt> and ack will search every file in the git repository, and only those files that appear in the repository.</li>
</ul>
<p>
On the horizon, we see creating a framework that will let authors create ack plugins in Perl to allow flexibility. You might create a plugin that allows searching through zip files, or reading text from an Excel spreadsheet, or a web page.
</p>
<p>
ack has always thrived on numerous contributions from the ack community, but I especially want to single out <a href="http://hoelz.ro/">Rob Hoelz</a> for his work over the past year or two. If it were not for Rob, ack 2.0 might never have seen the light of day, and for that I am grateful.
</p>
<p>
A final note: In the past, ack's home page was betterthangrep.com. With the release of ack 2.0, I've changed to <a href="http://beyondgrep.com/">beyondgrep.com</a>. "Beyond" feels less adversarial than "better than", and implies moving forward as well as upward. beyondgrep.com also includes a page of <a href="http://beyondgrep.com/more-tools/">other tools that go beyond the capabilities of grep</a> when searching source code.
</p>
<p>
For long time ack users, I hope you enjoy ack 2.0 and that it makes your programming life easier and more enjoyable. If you've never used ack, give it a try.
</p>Andy Lesterack 2.0 has been released. ack is a grep-like search tool that has been optimized for searching large heterogeneous trees of source code. ack has been around since 2005. Since then it has become very popular and is packaged by all the major Linux distributions. It is cross-platform and pure Perl, so will run on Windows easily. See the "Why ack?" page for the top ten reasons, and dozens of testimonials. ack 2.0 has many changes from 1.x, but here are four big differences and features that long-time ack 1.x users should be aware of. By default all text files are searched, not just files with types that ack recognizes. If you prefer the old ack 1.x behavior of only searching files that ack recognizes, you can use the -k/--known-types option. There is a much more flexible type identification system available. You can specify a file type based on extension (.rb for Ruby), filename (Rakefile is a Ruby file), first line matching a regex (Matching /#!.+ruby/ is a Ruby file) or regex match on the filename itself. Greater support for ackrc files. You can have a system-wide ackrc at /etc/ackrc, a user-specific ackrc in ~/.ackrc, and ackrc files local to your projects. The -x argument tells ack to read the list of files to search from stdin, much like xargs. This lets you do things like git ls | ack -x foo and ack will search every file in the git repository, and only those files that appear in the repository. On the horizon, we see creating a framework that will let authors create ack plugins in Perl to allow flexibility. You might create a plugin that allows searching through zip files, or reading text from an Excel spreadsheet, or a web page. ack has always thrived on numerous contributions from the ack community, but I especially want to single out Rob Hoelz for his work over the past year or two. If it were not for Rob, ack 2.0 might never have seen the light of day, and for that I am grateful. A final note: In the past, ack's home page was betterthangrep.com. With the release of ack 2.0, I've changed to beyondgrep.com. "Beyond" feels less adversarial than "better than", and implies moving forward as well as upward. beyondgrep.com also includes a page of other tools that go beyond the capabilities of grep when searching source code. For long time ack users, I hope you enjoy ack 2.0 and that it makes your programming life easier and more enjoyable. If you've never used ack, give it a try.Debunking the “five weekends every 823 years” myth with Perl2013-03-20T04:58:32-05:002013-03-20T04:58:32-05:00https://perlbuzz.com/2013/03/20/debunking_the_five_weekends_every_823_years_myth_with_perl<p>Have you seen this bit of bunk posted recently? It goes like this:</p>
<blockquote>
<p>This year, July has five Fridays, five Saturdays and five Sundays. This happens once every 823 years. So: copy to your status and money will arrive within 4 days… based on Chinese Feng Shui.</p>
</blockquote>
<p>Of course it’s crap, and of course
<a href="http://www.snopes.com/inboxer/trivia/fivedays.asp">Snopes.com does a fine job of debunking it</a>
But what if you want more evidence? Perl to the rescue! Here’s a little
program that finds and counts all the months with five full Fri-Sun
weekends in the next 823 years.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#!/usr/bin/perl
# Debunking the five weekends myth
# http://www.snopes.com/inboxer/trivia/fivedays.asp
use strict;
use warnings;
use feature 'say';
use DateTime;
my $start = 2013;
my $end = $start + 823 - 1;
say "Months with five full weekends between $start and $end";
my $nmonths = 0;
for my $year ( $start .. $end ) {
for my $month ( 1..12 ) {
my $eom = DateTime-&gt;last_day_of_month(
year =&gt; $year, month =&gt; $month );
if ( $eom-&gt;day == 31 &amp;&amp; $eom-&gt;day_of_week == 7 ) {
say $eom-&gt;month_name, ' ', $year;
++$nmonths;
}
}
}
say "There will be $nmonths months with five full weekends in the 823 years between $start and $end.";
$ ./five-weekends
Months with five full weekends between 2013 and 2835
March 2013
August 2014
May 2015
...
October 2832
July 2833
December 2834
There will be 823 months with five full weekends in the
823 years between 2013 and 2835.
</code></pre></div></div>Andy LesterHave you seen this bit of bunk posted recently? It goes like this:Where can someone find Perl modules to contribute to?2012-12-10T03:25:06-06:002012-12-10T03:25:06-06:00https://perlbuzz.com/2012/12/10/where_can_someone_find_some_perl_modules_to_help_with<p>
Bradley Andersen (his gmail username is bradley.d.andersen) wrote asking
</p>
<blockquote>
i am looking to try and contribute to perl and foss in general. do you know of a list of perl modules that are currently marked as 'maintainer died' that i might hijack, or, some list of module maintainers who may be looking for help? i am interested also in learning python and doing roughly the same kind of thing in that domain.
</blockquote>
<p>
Here are my suggestions, and I trust others will add suggestions in the comments:
</p>
<ul>
<li>First, pick a module that you actually use. The best module to work on is one that you already use. Check its bug queue, which will probably be at <a href="http://rt.cpan.org">rt.cpan.org</a>, but not necessarily.</li>
<li>Take a look at the <a href="http://search.cpan.org/~adoptme/">ADOPTME</a> user on CPAN. This is where orphaned packages go to live.</li>
<li>Check out <a href="http://24pullrequests.com/">24pullrequests.com</a>, a holiday-themed site of open source projects looking for help.</li>
<li><a href="https://openhatch.org/">OpenHatch</a> says it is "dedicated to matching prospective free software contributors with communities, tools, and education."</li>
</ul>
<p>
Remember, not all contributions to projects need to be in the form of written code. There are plenty of ways to help out that work on the module tests and infrastructure, or don't require any coding at all. For some ideas, see my article <a href="http://blog.smartbear.com/software-quality/bid/167051/14-Ways-to-Contribute-to-Open-Source-without-Being-a-Programming-Genius-or-a-Rock-Star">14 Ways To Contribute to Open Source Without Being a Rock Star"</a>.
</p>Andy LesterBradley Andersen (his gmail username is bradley.d.andersen) wrote asking i am looking to try and contribute to perl and foss in general. do you know of a list of perl modules that are currently marked as 'maintainer died' that i might hijack, or, some list of module maintainers who may be looking for help? i am interested also in learning python and doing roughly the same kind of thing in that domain. Here are my suggestions, and I trust others will add suggestions in the comments: First, pick a module that you actually use. The best module to work on is one that you already use. Check its bug queue, which will probably be at rt.cpan.org, but not necessarily. Take a look at the ADOPTME user on CPAN. This is where orphaned packages go to live. Check out 24pullrequests.com, a holiday-themed site of open source projects looking for help. OpenHatch says it is "dedicated to matching prospective free software contributors with communities, tools, and education." Remember, not all contributions to projects need to be in the form of written code. There are plenty of ways to help out that work on the module tests and infrastructure, or don't require any coding at all. For some ideas, see my article 14 Ways To Contribute to Open Source Without Being a Rock Star".