Month: June 2017

Avoid the vagueness of dies_ok() in Test::Exception

June 28, 2017 CPAN, Tools No comments , , ,

It’s good to check that your code handles error conditions correctly, but dies_ok() in Test::Exception is too blunt an instrument to do it.

Consider this code that checks that the func() subroutine dies if not passed an argument.

#!/var/perl/bin/perl

use warnings;
use strict;

use Test::More tests => 4;
use Test::Exception;

sub func {
    die 'Must pass arg' unless defined $_[0];
}

# Test for failures if the arg is not passed.
dies_ok(   sub { func() }, '#1: Dies without an argument' );
throws_ok( sub { func() }, qr/Must pass arg/, '#2: Throws without an argument' );
lives_ok(  sub { func(42) }, '#3: Lives with an argument' );

# Oops, we made a typo in our function name, but this dies_ok() still passes.
dies_ok(   sub { func_where_the_name_is_incorrect() }, '#4: Func dies without an argument' );

In case #4, the call to func_where_the_name_is_incorrect() indeed dies, but for the wrong reason. It dies because the function doesn’t exist. If we had used throws_ok instead of dies_ok like so:

throws_ok( sub { func_where_the_name_is_incorrect() }, qr/Must pass arg/, '#4: Func dies without an argument' );

then the test would have failed because the exception was incorrect:

#   Failed test '#4: Func dies without an argument'
#   at foo.t line 19.
# expecting: Regexp ((?^:Must pass arg))
# found: Undefined subroutine &main::func_where_the_name_is_incorrect called at foo.t line 19.

Why do I post this? I found an example of this in some code I was working with, where the test had been passing for the wrong reason for the past six years. Take the time to be specific in what you check for.

Dueling code wizardry is one of the things I love most about Perl

June 27, 2017 Community, CPAN 3 comments , , ,

At least week’s Perl Conference, Damian Conway talked about some new magical awesomeness he created, as he so frequently does. It’s Test::Expr, and it makes it easier to write tests:

# Write this ...                 ... instead of this.
ok $got eq $expected;            is        $got, $expected;
ok $got ne $unexpected;          isnt      $got, $unexpected;
ok $got == $expected;            is_deeply $got, $expected;
ok $got ~~ $expected;            unlike    $got, $pattern;
ok $got =~ $pattern;             like      $got, $pattern;
ok $got !~ $pattern;             unlike    $got, $pattern;
ok $obj->isa($classname);        is_ok     $got, $classname;
ok $obj->can($methodname);       can_ok    $obj, $methodname;

It also improves the diagnostics by showing the expression that failed.

#   Failed test '$got eq $expected'
#   at t/synopsis.t line 13.
#   because:
#          $got --> "1.0"
#     $expected --> 1

Chad Granum, the maintainer of much of Perl’s testing infrastructure took that last part as a challenge and overnight created his own magic in response: Test2::Plugin::SourceDiag.

use Test2::V0;
use Test2::Plugin::SourceDiag;

ok(0, "fail");

done_testing;

Produces the output:

not ok 1 - fail
Failure source code:
# ------------
# 4: ok(0, "fail");
# ------------
# Failed test 'fail'
# at test.pl line 4.

instead of:

not ok 1 - fail

#   Failed test 'fail'
#   at foo.t line 4.

This kind of dueling wizardry is one of the things that I love so much about Perl and its community.

Watch Chad’s lightning talk:

Improve your test logs with simple distro diagnostics

June 11, 2017 CPAN No comments ,

Automated module testing systems are becoming more and more common.  In addition to our long-serving CPAN Testers service, Perl authors can have their modules tested by Travis for Linux and Appveyor for Windows.  CPAN Testers tests each distribution uploaded to PAUSE, whereas Travis and Appveyor keep an eye on your GitHub account (or other services) and try testing after each push to the home repo.

Something that I’ve found helps out with diagnosing problems is by having a diagnostic dump of modules in the the first test.  I’ll have a test like t/00-modules.t, like this one from ack:


#!perl -T

use warnings;
use strict;
use Test::More tests => 1;

use App::Ack;   # For the VERSION
use File::Next;
use Test::Harness;
use Getopt::Long;
use Pod::Usage;
use File::Spec;

my @modules = qw(
    File::Next
    File::Spec
    Getopt::Long
    Pod::Usage
    Test::Harness
    Test::More
);

pass( 'All external modules loaded' );

diag( "Testing ack version $App::Ack::VERSION under Perl $], $^X" );
for my $module ( @modules ) {
    no strict 'refs';
    my $ver = ${$module . '::VERSION'};
    diag( "Using $module $ver" );
}

Then, when the user or automated tester runs make test, the first thing out tells me exactly what we’re working with.

[19:15:52] t/00-load.t .................. 1/23 # Testing ack version 2.999_01 under Perl 5.026000, /home/andy/perl5/perlbrew/perls/perl-5.26.0/bin/perl
# Using File::Next 1.16
# Using File::Spec 3.67
# Using Getopt::Long 2.49
# Using Pod::Usage 1.69
# Using Test::Harness 3.38
# Using Test::More 1.302073

This is also very useful for when users have test failures and submit their logs to a bug tracker. It’s especially important in this case to show the Getopt::Long version because ack has had problems in the past with some changes in API in the past.

Perl::Critic 1.128 fixes bugs and works with Perl 5.26

June 11, 2017 CPAN No comments

I’ve just released a new official release of Perl::Critic, the static code analysis tool for Perl. It uses the new version of the PPI Perl-parsing module, and it works with the new Perl 5.26, which does not include . in @INC by default.

If you’ve never used Perl::Critic to analyze your code base for potential bugs and stylistic improvements, mostly based on Damian Conway’s Perl Best Practices, try it out.

Here’s the changelog:

    [Bug Fixes]
    * PPI misparsing a module caused an incorrect "Must end with a
      recognizable true value."  This is fixed by upgrading to PPI
      1.224. (GH #696, GH #607)
    * A test would fail under the upcoming Perl 5.26 that omits the current
      directory from @INC.  Thanks, Kent Fredric.
    * Fixed an invalid test in the RequireBarewordsIncludes test.  Thanks,
      Christian Walde. (GH #751)
    * If an element contained blank lines then the source "%r" displayed
      for a violation was wrong. Thanks, Sawyer X. (GH #702, #734)

    [Dependencies]
    Perl::Critic now requires PPI 1.224.  PPI is the underlying Perl parser
    on which Perl::Critic is built, and 1.224 introduces many parsing fixes
    such as:
    * Fixes for dot-in-@INC.
    * Parse left side of => as bareword even if it looks like a keyword or op.
    * $::x now works.
    * Higher accuracy when deciding whether certain characters are operators or
      variable type casts (*&% etc.).
    * Subroutine attributes parsed correctly.

    [Performance Enhancements]
    * Sped up BuiltinFunctions::ProhibitUselessTopic ~7%.  Thanks, James
      Raspass. (GH #656)

    [Documentation]
    * Fixed incorrect explanation of capture variables in
      ProhibitCaptureWithoutTest.  Thanks, Felipe Gasper.
    * Fixed incorrect links. Thanks, Glenn Fowler.
    * Fixed incorrect example for returning a sorted list.  Thanks, @daviding58.
    * Fixed invalid POD.  Thanks, Jakub Wilk. (GH #735)
    * Updated docs on ProhibitYadaOperator.  Thanks, Stuart A Johnston. (GH #662)
    * Removed all the references to the old mailing list and code repository
      at tigris.org.  (GH #757)