Tools

Git is my hero

September 24, 2008 Tools No comments

By Selena Deckelmann

Last night, an incredible number of people showed up to Code-N-Splode to hear about Git.

I was not at all prepared for that number of people, but grateful that a couple experts were on hand to help – Sarah Sharp, Audrey Eschright and Michael Schwern (and others) all contributed to the discussion. Thanks for all the great questions! I’m looking forward to Sarah’s advanced tutorial next month.

Selena Deckelmann leads the Portland PostgreSQL Users Group and regularly contributes to Code-N-Splode, a group dedicated to getting more women involved in open source. She likes Perl. She’ll be running a code sprint and giving a lightning talk on having more fun with open source at PostgreSQL Conference West, on October 17-19, 2008.

Use Getopt::Long even if you don’t think you need to

May 25, 2008 Code craft, Tools 3 comments

Thread over on perlmonks talks about Tom Christiansen’s assertion that you should use it, by default, even when you only have one command-line argument to parse:

What seems to happen is that at first we just want to add–oh say for example JUST ONE, SINGLE LITTLE -v flag. Well, that’s so easy enough to hand-hack, that of course we do so… But just like any other piece of software, these things all seem to have a way of overgrowing their original expectations… Getopt::Long is just *wonderful*, up–I believe–to any job you can come up with for it. Too often its absence means that I’ve in the long run made more work for myself–or others–by not having used it originally. [Emphasis mine — Andy]

I can’t agree more. I don’t care if you use
Getopt::Long or
Getopt::Declare or
Getopt::Lucid or any of the other
variants out there. You know know know that you’re going to add more arguments down the road, so why not start out right?

Yes, it can be tricky to get through all of its magic if you’re unfamiliar with it, but it’s pretty obvious when you see enough examples. Take a look at prove or ack for examples. mech-dump is pretty decent as an example as well:

GetOptions(
'user=s'        => $user,
'password=s'    => $pass,
forms           => sub { push( @actions, &dump_forms ); },
links           => sub { push( @actions, &dump_links ); },
images          => sub { push( @actions, &dump_images ); },
all             => sub { push( @actions, &dump_forms, &dump_links, &dump_images ); },
absolute        => $absolute,
'agent=s'       => $agent,
'agent-alias=s' => $agent_alias,
help            => sub { pod2usage(1); },
) or pod2usage(2);

Where the value in the hashref is a variable reference, the value gets stored in there. Where it’s a sub, that sub gets executed with the arguments passed in. That’s the basics, and you don’t have to worry about anything else. Your user can pass –abs instead of –absolute if it’s unambiguous. You can have mandatory flags, as in agent=s, where –agent must take a string. On and on, it’s probably got the functionality you need.

One crucial reminder: You must check the return code of GetOptions. Otherwise, your program will carry on. If someone gives your program an invalid argument on the command-line, then you know that the program cannot possibly be running in the way the user intended. Your program must stop immediately.

Not checking the return of GetOptions is as bad as not checking the return of open. In fact, I think I smell a new Perl Critic policy….

What commands do you run?

April 16, 2008 Tools 2 comments

People have been posting in their blogs about what command they run, based on their shell histories. The command that I’ve seen looks like this:

history|awk '{a[$2]++} END{for(i in a){ 
printf "%5dt%s n",a[i],i}}'|sort -rn|head

That works, of course, but who wants to use awk and the shell? I pulled out the old Data::Hash::Totals module I wrote a while back, along with Perl’s built-in awk simulation:

$ history | perl -MData::Hash::Totals -ane'$x{$F[1]}++;' 
-e'END{print as_table(%x, comma => 1)}' | head
207 vim
143 svn
125 make
90 ack
77 cd
45 sdvx
34 ssq
31 ls
25 ./login-fixup
19 tail
alester:~ : cat `which sdvx`
#!/bin/sh
svn diff -x -w $* | view -

and ssq is just an alias for svn status -q.

Don’t use Plucene for real work

March 25, 2008 Tools 2 comments

Just in case anyone’s using Plucene as a search engine, please don’t. It’s terribly slow. Take a look at KinoSearch instead. Plucene is written in pure Perl, mostly as a proof of concept. It is literally orders of magnitude slower than KinoSearch, which uses C for the hard bits. Here’s a page of benchmarks to make the point.

ack 1.78 is out

March 24, 2008 Tools 1 comment

After three months of lots of development work and intermediate releases, I’ve released ack 1.78. There are tons of new features and lots of compatibility fixes for Windows. ack is a replacement for grep that is geared to working with trees of code.

Highlights in this release include:

  • Files specified on the command line are always searched, even if they don’t match a known filetype
  • Ability to ignore directories
  • Pager support
  • More flexible grouping options
  • Many more languages recognized and existing ones improved, including CFMX, Actionscript, assembly, Tcl, Lisp, Smalltalk
  • Ability to define your own languages based on filetype

ack may well change the way you work on the command-line with source code. Try it out and let me know what you think. You can install it by installing App::Ack from CPAN, or downloading the standalone version to your ~/bin directory.

Use seq or jot to do repetitive numbering

March 8, 2008 Tools No comments

I just now had to clean up some tables in a PostgreSQL database. The prior DBA thought that it would be good to split up tables into lx1, lx2, lx3 up to lx20. After I combined all the tables together, I needed to drop the originals. I could have written a Perl program to generate a series of drop table lx1; commands to feed into the psql command-line client, but instead I used the seq tool:

$ seq -f'drop table lx%g;' 1 20
drop table lx1;
drop table lx2;
...
drop table lx20;

If you don’t have seq on your system, as on Mac OS X, you probably have jot, as in:

jot -w'drop table lx%g;' 20 1

Then again, if you just have to do it in Perl:

perl -le'print qq{drop table lx$_;} for 1..20'

but I like to use other tools than the Swiss Army Chainsaw sometimes.

Devel::NYTProf is the hot new profiler in town

March 5, 2008 Tools No comments

Adam Kaplan has released a cool new profiling tool, Devel::NYTProf. It’s apparently taken the formatting beauty of Devel::Cover, the code test coverage tool, and used it to identify hotspots in your code. The results are beautiful: Overview and module-level detail. There’s also a report writer that provides CSV output and lets you create your own output in your own format.

I ran into a divide-by-zero error that I patched in my local copy, but other than that the results seem good. I look forward to whatever other improvements come. I’m also glad it’s been released with the support of the New York Times.

Leopard cheat sheet

February 14, 2008 Tools No comments

Here’s a handy little cheat sheet for Mac OS X Leopard from O’Reilly. I didn’t realize there were so many Finder shortcuts! Command-T puts something in the sidebar! Command-Shift-G lets you type in a folder name! Cool!

How to: Find all modules used in a tree

February 7, 2008 Regexes, Tools 1 comment

On the perl-qa list tonight, we were discussing how best to find all the modules used in a source tree. To do the job right, you’d have to run the code and then look at the %INC:: hash, which shows the paths of all modules loaded. The low-tech and usually-good-enough solutions we came up with use ack:

$ ack -h '^uses+(w+(?:::w+)*).*' --output=$1 | sort -u

Thanks to Andy Armstrong for coming up with a better regex than mine, which assumed that the use statement would necessarily end with a semicolon.

Getting lines from a website with GET and ack

January 31, 2008 Tools No comments

Oh no, hourly smoke test failures in my inbox today! Looks like I put some bad HTML on work’s website’s home page, and every hour one of the automated tests, via Test::HTML::Lint, told me once an hour that

# HTML::Lint errors for http://devserver.example.com/
#  (72:53) <a> at (61:53) is never closed

Well, phooey, it’s a PHP-driven website, so I can’t open index.php and check lines 72 and 61. I can use GET, installed with LWP, to fetch the website and save the source:

$ GET http://devserver.example.com/ > foo.html
$ vim foo.html +61

but since I’m a Perl programmer, I want to be as lazy as possible by using the tools at my disposal. In this case, it’s ack, and ack has the --line option to display ranges of lines instead of results of a regex. (Thanks to Torsten Blix for implementing this!)

$ GET http://devserver.example.com/ | ack --lines=61-72

So much nicer that way!
and look at it in an editor, but how much easier to not have to do that.