• Why create?

    why the lucky stiff encapsulates so much in this tweet:

    when you don't create things, you become defined by your tastes rather than ability. your tastes only narrow & exclude people. so create.

    So much to think about. So much explained on a few short sentences.

    It also meshes beautifully with Habit 1.

    I'm going to be rolling that around in my head for a while.

  • Why do designers fail to adopt Perl?

    A Perlbuzz guest editorial, by Dave Everitt

    As the "the glue/duct tape/chainsaw that holds the web together", Perl is all but invisible to most web users. It is still one of the best-established languages of choice, yet remains behind the scenes and out of the limelight. This "brand invisibility," coupled with the factors outlined below, has contributed to the erroneous perception that Perl is outdated or unfashionable in the glaring light of other more energetically-promoted solutions. Despite the continuing dominance of LAMP, what do most web designers think the "P" stands for? Not Perl.

    Cultural perception and perceived popularity don't help anyone to make a logical evaluation; this is an attempt to put the record straight.

    "Design? That's for designers!"

    Many Perl monks (and other alpha-geek programmers) hold design in some kind of contempt. Here are some comments by Perl coders:

    • Don't trust designers with code:
      "...now you can pass "helloworld.tmpl.html" to your web designer to modify it in her/his favorite HTML editor. (Just tell her/him not to touch the HTML tags starting with "<TMPL_")" -- HTML::Template Tutorial
    • Design is trivial:
      "If you find yourself lacking as a web designer or generally couldn't care less, you can give your templates to a web designer who can spice them up. As long as they do not mangle the special tags or change form variable names, your code will still work." - HTML::Template Tutorial

    This kind of wilful separation from design is a big put-off factor to newbie web programmers, and is endemic throughout the Perl community: "designers can't come in here until they're willing to learn serious code". In other words: "the aspirant must show humility". Understandable perhaps; and, once you start trying, the community is as helpful and friendly as can be. Just not overly welcoming to "designers"; in fact, some Perl websites appear almost deliberately or even proudly ugly. Perl templating solutions like HTML::Template or Template Toolkit get ignored by most web designers in favour of comfy, close-to-hand solutions like PHP, ASP.NET, whatever Dreamweaver offers, or younger and prettier models like Ruby on Rails or Python's Django.

    Here's a more rational approach to the "design vs. code" separation:

    "...the best Perl programmers are rarely the best HTML designers, and the best HTML designers are rarely the best Perl programmers. It is for this reason that the separation of these two elements is arguably the most beneficial design decision you can make when devising an application architecture." -- Jesse Erlbaum, Using CGI::Application

    This remains true, except that increasing numbers of people and teams are becoming good enough at coding and design to handle both passably well, They're choosing to invest the time necessary to learn both, as the Ruby on Rails explosion demonstrated. The Perl community needs to respond with a makeover and, maybe, some key "we're cool" websites like Catalyst's website to help rescue Perl's dated image, even if many Perl programmers don't care about it.

    Most designers want to be included in the "our site is cool, therefore we're cool crowd," not shoved in next to an ancient HTML website in ANSI colours.

    Where are my web pages? How do I know what does what?"

    Programmers know that effective templating separates logic from content from style. However, designers familiar with web pages and visual appeal are often used to working with actual files, unlike programmers who can hold everything in their heads, and in lines of code that don't look at all like what they're going to produce.

    Instead of these cosy actual files (the static web pages), most templating means that a site's links call a script that generates the pages. In other words, your website becomes a web application program, with all the power and flexibility. The literal, clunky file-for-every-page website disappears in the process, and this is just where many designers get a bit lost in starting the journey.

    Both the original DHTML and current Ajax are effectively still attempts to turn web pages into dynamic web applications. What isn't obvious to the designer is that these technologies have been around for ages. They don't see that secure and reliable interaction with the server, solid yet flexible programming in Perl can drive a zippy Web 2.0 site as easily as Ruby on Rails.

    Designers who want to start programming sometimes fail to make the conceptual leap from web pages as static files, to web applications consisting of components and screens. Failing to make this leap means they fall instead between logic and content by defaulting to PHP, Dreamweaver templates, Cold Fusion, etc. Perhaps this failure to make the complete leap happens because designers feel uncomfortable without seeing the actual "pages" they're working with, or because the dominant web design software makes it easy to adopt its own default solutions.

    Where are the "Perl tools" for Dreamweaver? All the publiclity and books read "Dreamweaver CS3 with ASP, ColdFusion, and PHP." If Eric Meyer can improve CSS support in Dreamweaver, could the Perl community do the same towards integrating Perl tools? A million design houses across the world might then at least see a menu item containing the word "Perl" in their favourite software.

    "Perl is impossible to read!"

    Perl itself can be no harder to learn or read than the PHP to which many designers default. Learning Perl is far from dead, now in its 5th edition, and is regarded as one of the best books for anyone learning programming in general.

    However, Perl programmers often delight in the potential economy of the language, and their concise code can appear obscure and impenetrable. I've even heard good programmers, fluent in other languages, say "Perl is horrible" or "unreadable". Yet to a beginner, Java (to take one example) is far more "horrible", verbose and complicated -- just compare "hello world" in both languages.

    Actually, beyond that initial step, how easy code is to understand depends on who does the coding. Precisely because there's more than one way to do it, Perl can be written as readably as any other language. Most times this isn't obvious to a beginner because Perl coders don't need to keep it obvious when they can use powerful shortcuts or write one-liners. With this in mind, my Perl monk friend replied to a draft of this article with a "line noise" of caution:

    I wouldn't recommend it as a first language. It's a second or third language. That's when it becomes powerful. I wouldn't want to put people off programming by making them look at @s=map{s/_/!/gi}grep{!~/^#/}@p.

    "Perl/CGI is slow (and gets slower with increasing server load)"

    This isn't true with FastCGI FastCGI or mod_perl. But try telling a designer who may only just be discovering the unforgiving nature of the command-line that these are an easy install. Or persuading a popular shared hosting provider (where many designers are hosting) to supply them on their list of default options.

    To complicate the issue, there's also a groundswell of opinion against the complexity of frameworks in general, with sites prototyped in Rails being redeveloped with other tools. Then there's the shared hosting debate.

    Isn't there a gap opening up here for tried-and-tested but nicely packaged Perl solutions for web development? Can't we fit between a designer's static pages and a full-blown framework? But just try finding a good-looking Perl templating tutorial that shows off the kind of tidy, well-crafted and semantic XHTML and cutting-edge CSS that the best designers produce, or a one-click Perl template install with a good-looking promotional website.

    Rails has demonstrated that speed isn't initially the issue. Ease, buzz, style and ubiquitous market presence are. Perl doesn't have to remain the dowdy elder sister -- given the same kind of push, it can go to the ball too.

    "So what's a poor designer to do?"

    If you want to develop your skills, ask "will I be using a programming language at least 2-3 times a week to actually do something; that is, as much as you currently use (X)HTML/CSS?" If not, you'll almost certainly forget what programming you learn and might be better off leaving it to "real programmers".

    However, if you do want to develop those skills, a good route is to experiment with a templating system that encourages the separation of logic and presentation, like HTML::Template or Template Toolkit. Work through the tutorial websites and just get something working. All you need to know for starters is that HTML::Template fills HTML templates dynamically with data according to the logic in your Perl, or that Template Toolkit generates static pages from your Perl.

    What you, the designer, must learn first

    You do run Apache locally on your machine, don't you? Okay. Now get CPAN and use it to install Perl modules for your platform. Be mindful of case-folding if you're on Mac OS X Yes, I know there should be a guide to all this, but like all learning curves in big territories, there's no authoritative manual. To make it easier, perhaps there could be.

    To make an intelligent comparison, you might want to try Ruby on Rails. The latest OS X already has RoR bundled. If you've done all this and are still committed to PHP, keep the logic apart from the design or tears of despair will follow. Steer clear of the convolutions that PHP allows, and note the following well:

    PHP has full support for classes, attributes and methods, and supports object-orientation, but the key problem is that no one seems to use it! There is a vast body of sample PHP code on the web, but nearly all of it is absolutely dreadful. HUGE long procedures, no structure, little code refactoring, and intermixed code and HTML that is impossible to read and must be impossible to maintain.
    -- Tim Roberts, Python Wiki

    Anyone who has been there will echo "yes, it is impossible to maintain," but that's not the language's fault. It might not all be squeaky-clean, but there isn't a vast body of "absolutely dreadful" Perl on the web. As a designer looking to settle on a language, that's a very valuable pointer. If you've ever had to plough through lines of "impossible to read" and "impossible to maintain" PHP, you'll understand that although PHP can be written as well as any other language, it often isn't.

    If ways can't be found to improve the overall quality of PHP authorship, well-styled bridges just need to be built for designers serious about programming to visit the Perl community. When they get there, they'll need to feel at home.

    Dave Everitt is a print-turned-web designer/developer who internalised HTML and CSS in the last century and steadily developing Perl skills in this one. His work includes the roles of web designer, educator and new media academic.

  • The first year of Perlbuzz

    Perlbuzz is a year old today.

    I just paid my first renewal for perlbuzz.com. It took about three weeks from when I first registered the domain name one night while brainstorming with Skud about a new newsy sort of site. The first news story: Further reports from YAPC::Europe.

    Now, Skud is off doing other things, most recently her wiki at geekfeminism.org. Perlbuzz's output has slowed some as I try to finish up my book "Job Hunting For Geeks" (not that that's the title we'll use) for Pragmatic Bookshelf.

    What hasn't changed at all is my appreciation for every one of you who reads Perlbuzz, or subscribes to the RSS feeds. Keep reading, and I'll keep writing.


  • Working to bring Perl to the Google App Engine

    Brad Fitzpatrick has announced that he is working on bringing Perl to the Google App Engine.

    I'm happy to announce that the Google App Engine team has given me permission to talk about a 20% project inside Google to to add Perl support to App Engine. To be clear: I'm not a member of the App Engine team and the App Engine team is not promising to add Perl support. They're just saying that I (along with other Perl hackers here at Google) are now allowed to work on this 20% project of ours out in the open where other Perl hackers can help us out, should you be so inclined.

    The architectural changes to make this happen are not insignificant. Check out Brad's post to find out how you can help.

  • Perl is an act of love

    The always-great Clay Shirky talks about the power of communities, and uses Perl and comp.lang.perl.misc as examples. "Perl is a Shinto shrine. Perl is an act of love." he says, and I can't disagree.

  • Devel::NYTProf will knock your socks off

    When Adam Kaplan first released Devel::NYTProf, I loved it. It stole the code grid feel from Devel::Cover, and it worked well.

    Now, Mr. DBI, Tim Bunce, has done some amazing work and released Devel::NYTProf 2.0. Follow the link and check out the screenshots.

    As cool as everything looks, and as helpful as the color-coding is, the big advance in 2.0 is the clickability:

    On lines that define a subroutine NYTProf now adds ‘comments’ giving the total number of times the sub was called, the inclusive time spent in that sub, and the average. Then it adds a break-down of the same details for every location that called the subroutine.

    For anyone concerned with finding bottlenecks in Perl code, Devel::NYTProf is clearly the gold standard.

  • Crazy ideas for book indexing

    I'm looking through Apache access logs, trying to remember what HTTP code 301 is. I have O'Reilly's _HTTP: The Definitive Guide_. I look in the index, and I see that 301 is aggregated with bq. 300-399 status codes, 61-64, 502 Why not spell out all the codes? bq. 301 status code, 61, 502 302 status code, 61, 502 303 status code, 62, 502 For that matter, why not have the index include the answer I want? bq. 301 status code: Moved Permanently, 61, 502 I could have the answer right there, without having to go any further. Further, the book has more than 600 pages, so I could even be able to look on page 301 and find out. It could have a footnote on the bottom of page 301 saying "301: Moved Permanently. Turn to pages 61-64, 502 for more information." Alas, I think that Chris Shifflet's purple HTTP book is half the pages, so that wouldn't do there.
  • Perl Foundation wants to give you money to work on Perl projects

    The Perl Foundation is calling for grant proposals for Perl-related projects. This can be a great way to get funding a project you're working on, or would like to see worked on. TPF has funded Strawberry Perl, Perl::Critic, pVoice and dozens of other projects in the past. Maybe yours can be the next.

  • YAPC::NA 2008 photo wrap-up

    I posted a bunch of YAPC::NA 2008 photos on my Flickr account, but Flickr limits me to 200 photos visible on a free account, so I trimmed out 34 of them, and someone (NOT MENTIONING ANY NAMES, SCROTTIE) squawked when I pruned the picture that he happened to be. So, I dumped all 113 over in my Picasa account in a photoset. These are practically all the pictures that I took during YAPC, other than some entirely blurry and/or black ones.

    If you've posted pictures from YAPC::NA 2008 somewhere, please mention it here in the comments.

  • Josh McAdams preaches in the press

    Hooray for Josh McAdams, new PR guy for the Perl Foundation, getting some serious space in this Infoworld article "Scripting languages spark new programming era". It's your standard trend-watching high-level gloss for watchers from the sidelines, apparently programming managers who only know of Java (an important target market, to be sure).

    It's unfortunate that Andi Gutmans of PHP has to wave the "I'm full of crap" banner with...

    PHP has been around a long time, and some advocates of newer languages say it's past its prime. PHP adherents disagree. Perl is complex and hard to maintain, Gutmans says. "Perl has pretty much disappeared when it comes to the Web."

    ... although it's interesting that it's PHP, not Perl, getting accused of being "past its prime." I also can't imagine much of anything harder to maintain than PHP slapped together by thousands of non-programmers. Of course, Josh counters with:

    But McAdams defended Perl's vitality, citing examples of major users. "I would ask him what Ticketmaster and Amazon use for their back ends," McAdams says. "[Perl] has a very large user base in Web apps but also has a strong presence in the financial industry."

    A fine article, Josh, and I thank you for taking on this important role in TPF.

  • YAPC::NA 2008 recap: The six-year-old version

    Quinn woke me up from my much-needed sleep this morning. "Dad, I missed you! How was your meetings?"

    "The meetings were good, and I got to see a lot of my friends. There was Pete, of course, and Josh and Heather from Chicago, and there was Yaakov from —"

    "Dad, are you fooling? Is that his name?"

    "Yes, that's his name. Yaakov. Different people have different sounding names. He's very nice. And I saw Schwern..."

    "That kind of sounds funny too!"

    "That's his name, Michael Schwern, and he is pretty funny. We watched bad movies. And Jim came from New York and another Jim came from New York. He just had triplets."

    "I remember that!"

    "And there were people from all over the world. I had pizza last night with Adam from Australia, and José was from Portugal and I saw Karen from Ireland, and I saw Abigail and he came from the Netherlands in Europe."

    "He's a man named Abigail? Why is he named Abigail?"

    "Because that's his name. And I also saw my friend Barbie who is a man from England."

    (This proves too much) "A man named BARBIE!?!?" (laughing, thinking of her own Barbie dolls.)

    "Yup. And I saw my friend Allison and I found out she just moved to Africa. And there were a lot of people from all over the world that came to be in those three days of computer meetings. They seemed to like it. And I am very very glad to see all these people every year, but I did miss you, too, Shmoo, and I'm glad to be home."

  • Schwern on skimmable code

    Michael Schwern gave a talk on skimmable code at YAPC this morning, much of which was a refactoring of some ugly parts of WWW::Mechanize. How embarrassing!

    Do take a look at his slides. Lots of goodness in there. One idea that had never occurred to me: "End-of-scope comments violate the DRY principle, you're repeating code."

  • Scrabble cheating with Perl one-liners

    I'm in the middle of a game of Scrabulous with Christoper Humphries on Facebook, and I get "tolkien" handed to me in my tray. Good letters, and I ought to be able to make a bingo out of them. Alas, the best I could get to play on the board was "knot", but what else could I have made? Perl to the rescue!

    All I need to do is match across the contents of /usr/share/dict/words in a Perl one-liner. The -n flag means "loop over the input file, but don't print $_". My little program goes in -e, and it looks like this:

    $ perl -lne'print if /t/ && /o/ && /l/ && /k/ && /i/ &&
    /e/ && /n/' /usr/share/dict/words

    Lots of good words, but they're awfully long. Let's limit it to seven-letter bingos. We have to use the -l flag to drop the linefeed from the input lines, so the length call is accurate.

    $ perl -lne'print if /t/ && /o/ && /l/ && /k/ && /i/ &&
    /e/ && /n/ && length($_)==7' /usr/share/dict/words

    Shoot, nothing there. Let's try eight.

    perl -lne'print if /t/ && /o/ && /l/ && /k/ && /i/ &&
    /e/ && /n/ && length($_)==8' /usr/share/dict/words

    "knotlike"! That would have been beautiful. Oh well. :-(

  • Perl Foundation funds five grants for summer 2008

    Alberto Simões, grant coordinator for the Perl Foundation, announces:

    It took a long time, but for this round we had a big number of excellent proposals. Unfortunately TPF is unable to fund all the proposed grants as they exceed the funds available for Grants. Thus, TPF GC ranked proposals accordingly with its relevance to the community, and the first few were funded.

    These are the funded proposals:

    As a side note, the last proposal was funded using Perl 6 funds, from Ian Hague donation, and thus, outside the GC budget.

    Follow the proposals that, although accepted, will not be funded:

    Unfortunately, there was a few proposals that did not convince the committee, and were rejected. Without any particular order:

  • Technical debt: Seek first to understand

    Grand Rapids Perl Monger Matt Heusser is organizing a workshop on technical debt in August. A recent blog posting from Matt gives some great talking points about technical debt, including one angle I hadn't thought of before, probably because I haven't taken any programming classes in over 20 years:

    Our educational system, for the most part, is built on one-off projects. Students build a program that inventories their CD collection. It doesn't work well - it even has some bugs - but it is good enough to demo and get a B+.... [They] can get hacked out in a weekend with a lot of pizza and caffeine....[S]tudents never have to live with the pain of maintaining the pile of mud they write. Thus, our first exposure to programming actively rewards us for tech debt.

    There's a lack of training, perhaps, but is programming leadership at fault as well? The old management saw of "What gets measured gets done" holds here. If your performance review is based on getting things done (for some value of "done") by a deadline, you're going to go aim for that. Heck, you can turn to daytime television sage Dr. Phil for this one: People do what works.

    I'm dealing with that right now. I've got a code base where the previous programmer's #1 goal was apparently to complete the task as quickly as possible. I've got PHP code like this:

    $rslt=pg_exec($conn,"drop table keys1x");
    $rslt=pg_exec($conn,"drop table keys2x");
    $rslt=pg_exec($conn,"drop table keys3x");
    $rslt=pg_exec($conn,"drop table keys13x");
    $rslt=pg_exec($conn,"drop table keys14x");
    # The tables are then recreated with 14 cut-n-paste
    # calls of identical CREATE TABLE DDL statements.

    Stupid, right? But was it? If your goal is to get a change made as quickly as possible, with no regard to other design factors, then cut & paste of code makes perfect sense. This guy was acclaimed for his ability to turn around program changes quickly, so at least someone was happy, even if that someone wasn't the programmer who followed. As Dr. Phil would say, he did what works.

    The causes of technical debt can be varied, and multiple. The programmer might not know better. It might be what worked best for him in a given situation. It could have been the only way out of an otherwise-impossible situation. Whatever the reason, it's best to know why things are the way they are before you try to pay down the debt.

  • New cross-language test automation lists, feeds and classes announced

    Gábor Szabó writes:

    I have set up a newsletter called Test Automation Tips where I am going to send out various ideas I have on the subject. As someone who has been practicing it for several years and teaching it since 2003 (see QA Test Automation using Perl) I might have a few interesting things to say. Especially as mostly it is stuff I learned from the folks on the Perl-QA list. I hope it will be an interesting read. The tips will come from several languages, most probably Perl, Python, PHP, Ruby and JavaScript but I might give examples in Java and maybe even in .NET.

    In addition I have set up the Test Automation Discussion list, a public mailing list where people from the various languages and projects can come together and discuss their experiences. For example on the PHP-QAT mailing list there is currently a discussion on the addition of something called XFAIL which is very similar to the TODO tests in TAP. I am sure people from the Perl world could give their opinion there, but the same subject could be discussed on a cross language mailing list as well.

    For the RSS and Atom junkies I have setup a Planet called Test Automation Planet. It is aggregating blogs and is trying to filter on subjects related to testing. I'd be really glad to get more suggestions on what to add. Earlier I hardly read blogs but this already brought me several interesting entries.

    Lastly, I am going to teach my QA Test Automation using Perl course on 19-20, June, right after YAPC::NA in Chicago. The syllabus is here. The registration is on the same page as the other master classes organized by brian d foy: Master Classes after YAPC::NA::2008

    Gábor Szabó - born in Hungary and living in Israel - has been using Perl since 1995 and teaching it both in Israel and overseas since 2000 via his company, Perl Training Israel. He is the organizer of the Perl mongers in Israel and he has been organizing the YAPC and OSDC conferences in Israel since 2003.

  • Use Getopt::Long even if you don't think you need to

    Thread over on perlmonks talks about Tom Christiansen's assertion that you should use it, by default, even when you only have one command-line argument to parse:

    What seems to happen is that at first we just want to add--oh say for example JUST ONE, SINGLE LITTLE -v flag. Well, that's so easy enough to hand-hack, that of course we do so... But just like any other piece of software, these things all seem to have a way of overgrowing their original expectations... Getopt::Long is just *wonderful*, up--I believe--to any job you can come up with for it. Too often its absence means that I've in the long run made more work for myself--or others--by not having used it originally. [Emphasis mine -- Andy]

    I can't agree more. I don't care if you use Getopt::Long or Getopt::Declare or Getopt::Lucid or any of the other variants out there. You know know know that you're going to add more arguments down the road, so why not start out right?

    Yes, it can be tricky to get through all of its magic if you're unfamiliar with it, but it's pretty obvious when you see enough examples. Take a look at prove or ack for examples. mech-dump is pretty decent as an example as well:

    'user=s'        => $user,
    'password=s'    => $pass,
    forms           => sub { push( @actions, &dump_forms ); },
    links           => sub { push( @actions, &dump_links ); },
    images          => sub { push( @actions, &dump_images ); },
    all             => sub { push( @actions, &dump_forms, &dump_links, &dump_images ); },
    absolute        => $absolute,
    'agent=s'       => $agent,
    'agent-alias=s' => $agent_alias,
    help            => sub { pod2usage(1); },
    ) or pod2usage(2);

    Where the value in the hashref is a variable reference, the value gets stored in there. Where it's a sub, that sub gets executed with the arguments passed in. That's the basics, and you don't have to worry about anything else. Your user can pass --abs instead of --absolute if it's unambiguous. You can have mandatory flags, as in agent=s, where --agent must take a string. On and on, it's probably got the functionality you need.

    One crucial reminder: You must check the return code of GetOptions. Otherwise, your program will carry on. If someone gives your program an invalid argument on the command-line, then you know that the program cannot possibly be running in the way the user intended. Your program must stop immediately.

    Not checking the return of GetOptions is as bad as not checking the return of open. In fact, I think I smell a new Perl Critic policy....

  • My schedule for YAPC::NA 2008

    At work, we're planning the talks we want to go see at YAPC::NA in June. Here's the stuff that I wanna go see:

    • Monday
    • Tuesday
      • 08:15: Social Perl (Facebook apps): Writing Facebook apps is going to be my new stretch project I think. I don't know what I want to do, but FB as app platform is clearly going some place.
      • 08:40: Fey::ORM: Mmmm, object mapping. I hate ORMs, and Dave Rolsky and I have talked about why they suck, so maybe his new one sucks less.
      • 09:05: Perl 5 VM: Every conference needs a session that hurts your head!
      • 10:05: Moose: All the cool kids are all about the Moose!
      • 10:55: Rakudo: As someone who contributes to Parrot and Rakudo, I really don't know much about its guts.
      • 13:00: Catalyst workshop: Catalyst seems ready to be the Perl Rails.
      • 14:50: pQuery and jQuery: I don't know jQuery at all, and JavaScript frameworks are beyond me. Let's hope this is a good introduction.
      • 16:05: Smolderbot: We use the smolderbot, but we don't yet have it running PHP tests. :-(
    • Wednesday

    I'm also planning on being at the Parrot hackathon and the Sox game.

    RJBS and hdp have posted their itineraries and commentary as well. If you post yours somewhere, please leave a comment here.

  • Oslo QA hackathon wrapup

    In April, a dozen or so folks interested in Perl and testing got together in Oslo, Norway for a hackathon. This video gives a recap of what they accomplished.

    Oslo QA Hackathon, Closing Remarks from Stonehenge on Vimeo.

    Nice way to recap the proceedings!

  • Help the Perl 5 wiki get to 1,000 entries

    Aaron Trevena wants to get the Perl 5 wiki up to 1,000 pages, and it's almost at 900.

    The Most Wanted Pages is a great place to start. Pick a page that some other page has linked to, and create it. It's that simple.