Recently in Tools Category

People have been posting in their blogs about what command they run, based on their shell histories. The command that I've seen looks like this:

history|awk '{a[$2]++} END{for(i in a){ \
printf "%5d\t%s \n",a[i],i}}'|sort -rn|head

That works, of course, but who wants to use awk and the shell? I pulled out the old Data::Hash::Totals module I wrote a while back, along with Perl's built-in awk simulation:

$ history | perl -MData::Hash::Totals -ane'$x{$F[1]}++;' \
-e'END{print as_table(\%x, comma => 1)}' | head
207 vim
143 svn
125 make
 90 ack
 77 cd
 45 sdvx
 34 ssq
 31 ls
 25 ./login-fixup
 19 tail

alester:~ : cat `which sdvx`
#!/bin/sh

svn diff -x -w $* | view -

Just in case anyone's using Plucene as a search engine, please don't. It's terribly slow. Take a look at KinoSearch instead. Plucene is written in pure Perl, mostly as a proof of concept. It is literally orders of magnitude slower than KinoSearch, which uses C for the hard bits. Here's a page of benchmarks to make the point.

After three months of lots of development work and intermediate releases, I've released ack 1.78. There are tons of new features and lots of compatibility fixes for Windows. ack is a replacement for grep that is geared to working with trees of code.

Highlights in this release include:

• Files specified on the command line are always searched, even if they don't match a known filetype
• Ability to ignore directories
• Pager support
• More flexible grouping options
• Many more languages recognized and existing ones improved, including CFMX, Actionscript, assembly, Tcl, Lisp, Smalltalk
• Ability to define your own languages based on filetype

ack may well change the way you work on the command-line with source code. Try it out and let me know what you think. You can install it by installing App::Ack from CPAN, or downloading the standalone version to your ~/bin directory.

I just now had to clean up some tables in a PostgreSQL database. The prior DBA thought that it would be good to split up tables into lx1, lx2, lx3 up to lx20. After I combined all the tables together, I needed to drop the originals. I could have written a Perl program to generate a series of drop table lx1; commands to feed into the psql command-line client, but instead I used the seq tool:

$ seq -f'drop table lx%g;' 1 20
drop table lx1;
drop table lx2;
...
drop table lx20;

If you don't have seq on your system, as on Mac OS X, you probably have jot, as in:

jot -w'drop table lx%g;' 20 1

Then again, if you just have to do it in Perl:

perl -le'print qq{drop table lx$_;} for 1..20'

but I like to use other tools than the Swiss Army Chainsaw sometimes.

Adam Kaplan has released a cool new profiling tool, Devel::NYTProf. It's apparently taken the formatting beauty of Devel::Cover, the code test coverage tool, and used it to identify hotspots in your code. The results are beautiful: Overview and module-level detail. There's also a report writer that provides CSV output and lets you create your own output in your own format.

I ran into a divide-by-zero error that I patched in my local copy, but other than that the results seem good. I look forward to whatever other improvements come. I'm also glad it's been released with the support of the New York Times.

Marcel Grünauer posted to his Twitter account that he freed up 3 GB by removing non-English localization resource files (*.lproj) with the following command:

find / \
    \! -name 'English.lproj' \
    \! -name 'en.lproj' \
    -name '*.lproj' -type d \
    -exec rm -rf -- {} \; -prune

If you want to make sure you're deleting the right files before you delete them, see the list of the .ljprojf files first:

find / \
    \! -name 'English.lproj' \
    \! -name 'en.lproj' \
    -name '*.lproj' -type d -print -prune

Here's a handy little cheat sheet for Mac OS X Leopard from O'Reilly. I didn't realize there were so many Finder shortcuts! Command-T puts something in the sidebar! Command-Shift-G lets you type in a folder name! Cool!

On the perl-qa list tonight, we were discussing how best to find all the modules used in a source tree. To do the job right, you'd have to run the code and then look at the %INC:: hash, which shows the paths of all modules loaded. The low-tech and usually-good-enough solutions we came up with use ack:

$ ack -h '^use\s+(\w+(?:::\w+)*).*' --output=\$1 | sort -u

Thanks to Andy Armstrong for coming up with a better regex than mine, which assumed that the use statement would necessarily end with a semicolon.

Oh no, hourly smoke test failures in my inbox today! Looks like I put some bad HTML on work's website's home page, and every hour one of the automated tests, via Test::HTML::Lint, told me once an hour that

# HTML::Lint errors for http://devserver.example.com/
#  (72:53) <a> at (61:53) is never closed

Well, phooey, it's a PHP-driven website, so I can't open index.php and check lines 72 and 61. I can use GET, installed with LWP, to fetch the website and save the source:

$ GET http://devserver.example.com/ > foo.html
$ vim foo.html +61

but since I'm a Perl programmer, I want to be as lazy as possible by using the tools at my disposal. In this case, it's ack, and ack has the --line option to display ranges of lines instead of results of a regex. (Thanks to Torsten Blix for implementing this!)

$ GET http://devserver.example.com/ | ack --lines=61-72

So much nicer that way! and look at it in an editor, but how much easier to not have to do that.

There is no project so small, so trivial, that it is not worth you putting it into a Subversion repository. If it's worth your time to work on it, it's worth saving. Putting it in Subversion is a matter of a few statements, and you don't have to do any big fancy-shmancy server setup.

Let's assume you're working on Linux/Unix, and you have svn installed, which is pretty standard these days. Say you're working on a game called bongo, and you've just been keeping it in ~/bongo. Do this:

# Create the Subversion repo
$ mkdir /svn

# Create the bongo repo
$ svnadmin create /svn/bongo

# Import bongo into its project
$ cd ~/bongo
$ svn import file:///svn/bongo -m'First import of bongo into Subversion'

# Move the original bongo directory out of the way,
# in case something goes wrong
$ mv ~/bongo ~/bongo-original

# Check out bongo from Subversion
svn co file:///svn/bongo

At this point, you'll have a checked-out version of bongo in ~/bongo, and you can make commits against it.

Ricardo Signes points out that Git makes it even easier.

# Go to the bongo directory
$ cd ~/bongo

# Import bongo
$ git init

With Git, everything is put in your ~/.git directory, and you don't have to check out anything from the project.

Whatever route you choose, version control is so simple these days there's just no excuse not to do it. Your programming life will never be the same.