Month: March 2008

rt.cpan.org source released

March 15, 2008 Community, CPAN No comments

Jesse Vincent announced that Best Practical has released the source for rt.cpan.org. All the general magic that Jesse &amp. co. use to make the de facto bug tracking system for the CPAN is now available from BestPractical’s Subversion site.

Jesse says “If you’ve been hankering for a new feature in rt.cpan.org, now’s the time to start sending patches. After 3 good patches, we’ll grant you a commit bit to the rt.cpan.org extensions.” Kudos to Jesse for opening things up and helping spread the workload of maintaining this crucial tool.

Sloppy and lazy database handling: PHP’s shortcomings, part #47

March 14, 2008 Uncategorized 10 comments

Here’s yet another reason why PHP is deficient when it comes to doing Real Work with Real Databases.
We’re currently working on a PostgreSQL-based database with six million book records. We want to walk
the book table and slurp up the rows as we go. The code looks like this.

$sql = 'select field1, field2, ... field8 from book';
$set = pg_query( $sql );
while ( $row = pg_fetch_row( $set ) ) {
// Do some processing
}
pg_free_result( $set );

Nothing crazy, right? I’m just walking the table, one row at a time. However,
the PHP PostgreSQL functions slurp up
all six million rows, even though my calling code doesn’t need to see all the rows at once.
This eats up tons of RAM and makes us worry about memory allocation for the PHP process.
too bad for us users! We can’t even get to the first pg_fetch_row() call because the pg_query slurps up all the rows immediately.

Now, there IS functionality where you can retrieve a given row, like:

$row = pg_fetch_row( $set, $rowno );

In this case, it’s necessary to iterate to that given $rowno, but even then you wouldn’t have to get to the end of the dataset.

If that’s not lazy enough coding on the back end, check out
this little nugget from the docs for the pg_free_result
function:

pg_free_result() frees the memory and data associated with the specified PostgreSQL query result resource.

This function need only be called if memory consumption during script execution is a problem. Otherwise, all result memory will be automatically freed when the script ends.

The PHP docs encourage sloppy coding. Rather than promoting good, safe programming behaviors, they’re saying “Eh, no big deal.” It’s like the PHP team knows that
PHP won’t be used on large datasets, and it becomes a self-fulfilling prophecy.

PHP needs to be left to the ghetto of guestbooks and the odd bulletin board system. Leave the real work to the real languages.

USENIX opens conference proceedings

March 13, 2008 Conferences No comments

USENIX, the Advanced Computing Systems Organization, has announced:

USENIX is pleased to announce open public access to all its conference proceedings.

This significant decision will allow universal access to some of the most important technical research in advanced computing. In making this move USENIX is setting the standard for open access to information, an essential part of its mission.

USENIX could not achieve such goals without the support and dedication of its membership. We urge you to encourage others to join USENIX. Membership helps us present over 20 influential conferences each year and offer open access to the technical information presented there.

The proceedings are available at
http://www.usenix.org/publications/library/proceedings/

LINSWAN: An acronym we can steal from Ruby

March 10, 2008 Community 2 comments

Pat Eyler’s recent blog post introduced me to a term I hadn’t seen from the Ruby community. MINSWAN stands for “Matz Is Nice, So We Are Nice.”

What a marvelous idea! I’d like to steal it as LINSWAN, for “Larry Is Nice, So We Are Nice,” although I suspect that people might get confused and think of Pittsburgh Steelers Hall of Famer Lynn Swann.

Use seq or jot to do repetitive numbering

March 8, 2008 Tools No comments

I just now had to clean up some tables in a PostgreSQL database. The prior DBA thought that it would be good to split up tables into lx1, lx2, lx3 up to lx20. After I combined all the tables together, I needed to drop the originals. I could have written a Perl program to generate a series of drop table lx1; commands to feed into the psql command-line client, but instead I used the seq tool:

$ seq -f'drop table lx%g;' 1 20
drop table lx1;
drop table lx2;
...
drop table lx20;

If you don’t have seq on your system, as on Mac OS X, you probably have jot, as in:

jot -w'drop table lx%g;' 20 1

Then again, if you just have to do it in Perl:

perl -le'print qq{drop table lx$_;} for 1..20'

but I like to use other tools than the Swiss Army Chainsaw sometimes.

More companies openly supporting Perl projects

March 5, 2008 Business, Community No comments

More companies are showing their support for open source projects, and I couldn’t be happier about it.

Those of you following Ovid’s blog on use.perl.org, or reading his code improvements in the perl-qa mailing list, should give thanks to the BBC for supporting his Perl work. It’s not all philanthropic, of course, since the BBC wants good tools for themselves, but I love that they’re letting Ovid hitch his stories to the BBC wagon. That helps give Perl some credence in the eyes of open source skeptics.

Now, as you readers of Mechanix know, Devel::NYTProf is the hot new profiler in town. Not only is the New York Times allowing code to be released, it turns out there’s a blog, open.blogs.nytimes.com, where Adam Kaplan announced the module. I love that a company that’s not (exactly) in the software business is blogging about their open source software work. Let’s hope it’s a light in the darkness that others will lend their illumination to as well.

Devel::NYTProf is the hot new profiler in town

March 5, 2008 Tools No comments

Adam Kaplan has released a cool new profiling tool, Devel::NYTProf. It’s apparently taken the formatting beauty of Devel::Cover, the code test coverage tool, and used it to identify hotspots in your code. The results are beautiful: Overview and module-level detail. There’s also a report writer that provides CSV output and lets you create your own output in your own format.

I ran into a divide-by-zero error that I patched in my local copy, but other than that the results seem good. I look forward to whatever other improvements come. I’m also glad it’s been released with the support of the New York Times.

The worst way to shorten names

March 5, 2008 Code craft 4 comments

Dropping vowels to shorten names is a terrible practice. Quick, someone give me an idea what $hdnchgdsp means, an Actual Variable from some Bad Code I’m working on today.

It’s not just variables names, either. Filenames often need to be shortened, but dropping vowels is not the way to do it. You’re left with unpronounceable names that are annoying to type.

The key to effective abbreviation is not removal of letters from the middle of the words, but from the end. Sometimes, it doesn’t make sense to shorten a word at all, like “post”. If you have a file that is supposed to “post audit transactions”, call it “post-aud-trans” or “post-aud-trx”, not “pst_adt_trns”.

Perl Foundation needs new members

March 5, 2008 Perl Foundation 2 comments

The Perl Foundation needs new blood. Jim Brandt writes:

Have you ever wanted to get involved in The Perl Foundation, but didn’t know how? Well, now’s your chance. I’m pleased to announce open self-nominations for the following TPF roles:

You can follow the links above to read descriptions of each of the positions. If you think you’re a good fit for one or more of them, send me an email at cbrandt at perlfoundation dot org. I’ll then invite you to a dedicated wiki we have set up just for the election.

Once you join the wiki, you’ll set up a page to post all of your experience and answer the questions provided in each section above. The wiki is private, but you’ll be able to see the other candidate pages, and they’ll see yours.

The deadline to get all of your information in is midnight next Tuesday, March 11. Our committees elect their members, so the Conferences Committee will be voting on the CC chair and the Steering Committee will vote on the chair and PR positions. After we have a chance to look over everyone’s information, we vote and select our newest members.

You only have a week, so don’t wait too long. I look forward to hearing from you.

Karen Pauley is stepping up to run for Steering Committee chair, so how about you? Maybe that’s a spot you’d like to work on, or maybe public relations is more up your alley. This is your chance to help lead TPF lead Perl and Perl development.

Astute followers of TPF will note that the PR spot is open, a spot that I once held. Yes, I am no longer doing PR for TPF. I’ve done that job for a while, and now I’m moving on to do other things, not least of which is this little news called perlbuzz.com.

Lessons in workshop organization from Frozen Perl 2008

March 3, 2008 Community, Conferences No comments

Dave Rolsky has written up a recap of Frozen Perl from the organizers’ point of view. Some interesting tidbits:

  • 2/3rds of FP2008 expenses were paid by sponsors, keeping attendee cost low
  • Some sponsors are interested in the tax deduction; some see it as a marketing expense
  • With very low ticket prices, each additional attendee ends up being a net loss
  • A potential solution to the lightning talk machine dance could be a KVM.
  • Income $7,190 – Expenses $6,213.43 = $976.57 overage

The workshop came together beautifully. I was very impressed with what Dave & Co. pulled together.