Perlsphere

Contrary to the strawman arguments set up and knocked down so easily by people who believe that breaking the DarkPAN is a sin almost as bad as apparent hypocrisy on the Internet, no one wants to change Perl 5 willy-nilly. Yet many people believe that a few carefully-planned backwards-incompatible changes will allow Perl 5 to evolve into a much more powerful, much cleaner, much more understandable, and even much simpler language suitable not only for maintaining legacy projects but also for creating amazing new projects.

I intended my deconstruction of fear surrounding Perl core language changes to continue a long-standing debate. If you read Planet Perl Iron Man or Planet Perl, you've read other people arguing over the desirability of supporting the DarkPAN.

If your code is part of the DarkPAN (and it likely isn't, as you're likely a part of the Perl community if you're reading this), you may bristle at the suggestion that some people believe that retaining core language backwards compatibility to support your code is one of the worst ideas in Perl history. (If you haven't heard that suggestion, let me repeat it: the idea that the Perl community should support hidden and proprietary code of people who aren't part of the community, code that the Perl community can't see let alone test, and code that may or may not even exist is sufficiently ridiculous on its face that it mocks itself.)

Now then.

My previous post suggested that the real problem the Perl community is trying to solve is one of risk management. If it doesn't make sense to push all of the costs to manage the risk that an upgrade might change the way your DarkPAN code behaves to the Perl community (and it doesn't make sense), who bears that cost?

You do.

Don't panic, yet. It's simple to address that risk. I didn't say it's easy. I said it's simple.

Your Responsibility as a DarkPAN Maintainer

As a DarkPAN maintainer, you need to answer several questions about your code.

• Is this software worth maintaining?
• Is this software under active maintenance?
• Does this software need community support for the Perl core and any external dependencies?
• Will this software work unchanged with a newer version of Perl?
• If not, why not?
• What changes are necessary to make this software work with a newer version of Perl?

Software outside of active maintenance can stay untouched. Don't worry about it. (Of course, if you have to migrate to a new machine or a new architecture or other dependencies, it's under active maintenance at least for the sake of this argument.)

The other questions are more important. You need to be able to verify that your code will perform as you expect if you change one or more dependency.

Likely you already need to know this.

You need a comprehensive test suite which can run on any likely candidate machine. You need a test environment where you can run your software's test suite against an updated dependency -- a new CPAN distribution, for example, or a new major release of Perl. You need unambiguous results and comprehensible diagnostics.

In short, you need a sane development environment, just like any serious project worth your time already has.

(You should also have a ten minute build, but if you can verify that your software does what you expect in an hour or less and report the results to the Perl community, you're well on your way.)

If you're really good, you can even test your DarkPAN code against bleadperl or maintenance snapshots just to help p5p understand your expectations (and to understand potential changes to the language which may require you to revise your expectations, if not your code).

Anyone serious about software development should have a comprehensive and useful test suite anyway. It's simple. It's not always easy; the Perl community needs to make CPAN distribution installation and versioning easier and cleaner. That debate is happening. The solution isn't obvious yet, but a process of iteration and refinement will produce it.

Yet any DarkPAN code of such low importance that its maintainers can't produce a test suite to verify that it works isn't worth p5p's time. Forget that code. It's a dead end. Let's concentrate instead on refining a language for serious software developers to write new code worth maintaining.

Posted by chromatic on darkpan, maintenance, modern perl, perl, perl programming, software development at Sat, 04 Jul 2009 00:56:30 -0000 | Permalink

Organizing a conference is hard, let's go shopping! For the first time I'm officially helping, not just for the Italian Perl Workshop this year, but possibly for YAPC::EU::2010 too. I've been working with the perl.it guys on the proposal to host the European Perl Conference 2010 in Pisa.

We submitted the bid on Monday, and it's just been announced that the teams competing are: Pisa (us) and Kiev in Ukraine. Wow! YEF have actually published both our bids on that link, which is fantastic for transparency.

It also means that we can read their bid... and it's a good one! Looks like we've got some competition. Of course our Pisa bid is excellent too - in any case, the next couple of weeks till we find out who won are going to be a nailbiting time!

Posted by osfameron on perl, .it, .ua, yapceu at Fri, 03 Jul 2009 19:56:33 +0100 | Permalink

If this is on the CPAN, please let me know. I can't find it. If not, we probably need it at work

The W3C duration standard, derived from the ISO 8601 duration standard, allows one to specify durations in a format like PnYnMnDTnHnMnS (e.g.: "P1Y2MT2H", a duration of one year, two months and two hours). I can't find a proper parser for it and it turns out to have some tricky bits. Basically, I want to do this:

my $duration = DateTime::Duration::W3C->new_from_w3c("P1Y2MT2H");

Which would be more or less equivalent to:

my $duration = DateTime::Duration->new(
    years  => 1,
    months => 2,
    hours  => 2,
);

The only real difference would be (I think) adding string overloading to allow one to print the duration and have it come out in the proper format.

I can write this and try to handle the special cases I find documented (I'll be damned if I'm going to pay 130 Swiss Francs for the damned ISO 8601 standard), but I'd love to know if it's already out there.

Posted by Ovid on journal at Fri, 03 Jul 2009 13:45:44 -0000 | Permalink

For a little while now I've been working on a WebGUI plugin for Padre (the Perl Application Development and Refactoring Environment). Progress has been pretty slow, partly due to lack of time but also because often the things I want to do are at the edge of what Padre currently supports (or just that there's no existing examples of how to do something easily).

I've released the code onto CPAN, and the current dev version is available via github, but since it's all a bit hacky and alpha at the moment I thought it worthwhile doing a little tour of what currently works (on my dev box at least).

Installation

Installation is the same as any other Padre plugin, either install via cpan

cpan install Padre::Plugin::WebGUI

or from git:

git clone git://github.com/pdonelan/Padre-Plugin-WebGUI.git

perl Makefile.PL

make && make test && make install

Since this is a WebGUI plugin, you need to first install WebGUI, and also WGDev.

Once you've done that, you will see the WebGUI plugin listed in the Padre Plugin Manager:

Click the enable button, and you'll see a WebGUI item appear in the Plugin menu.

Controlling WRE Services

The first thing you can do with the plugin is control your WRE services (mysql, modperl, modproxy and spectre).

Click on: WebGUI > WRE Services > Modperl > Restart and you'll see a restart status message appear in the Padre output panel.

Currently this is just a simple wrapper around the wreservice.pl script that ships with WebGUI. A nice enhancement would be to turn the menu item into a services dashboard with lights telling you which services are currently running, and buttons to start/stop/restart.

Online Resources

Another item in the menu called Online Resources gives you access to a bunch of handy WebGUI links that open in your browser – including one that takes you straight to a web-based IRC client connected to #webgui.

WebGUI Log

If you click the "Logview" checkbox on the WebGUI menu, a WebGUI Log panel will open. This panel will start monitoring all of your WRE log files (webgui.log, modperl.log, modproxy.log, etc..) and display changes in real-time. WebGUI uses Log4perl as its logging engine, but it uses the sitename as the Log4perl category, which means that you can't easily e.g. turn on DEBUG output and filter out everything except the module you're working on (unless you use my Log4perl::Filter::CallerMatch module). Once we add in the ability to easily customise what files are being logged and apply filters to what gets displayed, this will make the above scenario a breeze.

The logview panel uses a long-running Padre task and File::Tail::select to continuous monitor a bunch of files in realtime. The main thread writes commands to a temp file to send commands to the monitoring thread (the most important command being "exit" which tells the thread to break out of its monitoring loop!). Later I plan on adding support for other commands such as "add/remove this file from the list of files you're watching", "add/remove filter", etc.. Since this seems like a generic sort of feature I'm hoping to bundle  Padre::Plugin::WebGUI::Logview and Padre::Plugin::WebGUI::Task::Logview as a stand-alone module for other Padre plugin developers to use (and improve).

Asset Tree

And finally, the coolect feature the WebGUI plugin has is the Asset Tree.

If you turn on the Asset Tree from the plugin menu, a new panel appears with a "Connect" button on it. Currently this is hardwired to the default WebGUI dev site at dev.localhost.localdomain but later I plan on adding a Site Manager to allow you to define sites here.

Double-clicking on the Connect button connects to the site, and pulls down the asset tree:

This is the same Asset Tree you see when you log in to your WebGUI site and turn admin mode on. As you can see from the screenshot, right-clicking on a node brings up a context menu, from which you can do asset-specific actions. The plugin already gives you access to all of the WGDev commands from the main plugin menu (with a dialog box asking you what arguments you want to pass to the command), but it becomes a lot more useful once you hook these WGDev commands to the Asset Tree context menu. Then instead of having to remember what url or assetId your asset has, you can just click through the tree to find the node you want, right-click and off you go.

For example, WGDev has a package command that lets you package up node and all of its children into a tarball that can be imported into other sites, used in upgrade scripts, etc.. By integrating this command into the context menu, you can have Padre prompt you you for the folder that you want to save the WebGUI package in (via your native Save File dialog box).

Similarly, WGDev has an edit command that lets you edit an asset (for example a template file) via your favourite editor and then have the changes pushed back into the database after you save and close the file. By integrating this command into Padre, you could leave the file open, and push the changes up every time you hit save(allowing for more convenient continuous editing).

Posted by Patrick on Perl, WebGUI at Fri, 03 Jul 2009 08:42:06 -0000 | Permalink

Jeff Atwood is the best tech blogger in the world, in my opinion. And his most recent post contains my nominee for the quote of the year:

Open source software only comes in one edition: awesome.

Posted by ChrisDolan on journal at Fri, 03 Jul 2009 02:24:36 -0000 | Permalink

Posted by jquelin@gmail.com (Jérôme Quelin) on moose, games, pandemic, tk, dzill, perl, poe at Thu, 02 Jul 2009 17:03:00 -0000 | Permalink

A recent thread on the Mason users list reminded me of the problems I had grokking Catalyst when I first looked at it. Raymond Wan wrote "I'm skimming over the MVC part and as my system doesn't use an SQL database, I'm wondering if Catalyst is overkill??"

Those of us who know Catalyst know that this question is based on some wrong assumptions, but it's easy to see why Raymond has those assumptions. Look at the Catalyst tutorial. In the very first chapter, it's already talking about databases and DBIx::Class.

It's easy to look at this and assume that Catalyst is somehow tightly bound to DBIx::Class or SQL databases.

The problem is that the tutorial docs really need to serve two different audiences, though both audiences are Catalyst newbies.

On the one hand, you have people with relatively little web app experience. Presumably, they know some Perl, and they do a web search for "perl web application framework". Eventually, they'll get to Catalyst and start reading about it. For those people, being given a set of standards and a full walk through of Model and View (in addition to Controller) is very valuable. It gives them all the tools they need to get started on simple web apps without having to make too many choices.

The other audience is people who have some real web app development experience with something more primitive than Catalyst. These could be people coming from a Mason-based site without any real controllers, or people who used something like Apache::Template with mod_perl, or maybe they wrote their own controllers "by hand" using the mod_perl API.

Many of those folks will already have some experience with models. To refer back to Raymond, presumably his system already has some sort of model code, it just doesn't use a SQL DBMS. Those people will look at the existing Catalyst tutorial and get confused. Isn't Catalyst flexible? Then why does it look like RoR, with all my tool choices made for me?

It took me a while to realize that Catalyst, at its core, is smaller than you might think, and you can use just the pieces you like.

The very core of Catalyst is its dispatching system. Given a URI, it selects a piece of code to run. Its dispatcher is very powerful (Chained methods are great!), and with plugins like Catalyst::Action::REST, it's even better.

Along with the dispatching system, Catalyst also provides an abstraction over the typical web app request/response cycle. The Request makes it easy to look at incoming query arguments, POST data, file uploads, and headers. The Response lets you set headers and return output to the client, whether that be HTML or a downloadable file.

Catalyst also includes engines (think "environment adapters") for a number of common web application environments, including vanilla CGI, mod_perl, FastCGI, and more. These engines make sure that the Request/Response API works exactly the same in any environment where Catalyst can be deployed.

This is a huge win, since you can write your app without worrying about the deployment environment. If you're writing an app for public distribution, it gives the installing users a choice of how to deploy.

These core pieces are really the only parts of Catalyst you have to use when you use Catalyst. If you don't want dispatch and a request/response API, you don't want Catalyst.

Catalyst (really Catalyst-Devel) also includes a fantastic single-process development server. This server can be started straight from an application's checkout directory with one command. Even better, this dev server can be told to monitor all relevant files and restart itself when any of them change. Note that this is a proper restart, which avoids all the myriad problems that afflict Apache::Reload and its ilk, which attempt to reload modules in the same Perl interpreter.

Just these things - controllers, a request/response abstraction, deployment agnosticism, and a great dev environment - are enough to make Catalyst a great choice. Ignore everything else it does and you'll still have improved your development process and improved your code.

Catalyst also does some other things ...

It has a component system which has allowed people to release a whole host of useful plugins. If you look on CPAN, you'll find things like sessions, powerful authentication, dumb authentication, I18N, and much more. If a plugin does what you need, it'll save you a lot of development time.

Note that the old "Catalyst::Plugin" system is in the process of being deprecated, but the concept of pluggable components is still core to what Catalyst is. All that's changed is the way pluggability works.

Catalyst lets you have multiple views. While many apps will just output HTML via a templating system, this flexibility is great for RESTful apps that may want to output XML, JSON, and still fall back to HTML for browsers (see my REST-ForBrowsers for some help with that).

Catalyst also has configuration file handling built-in. Personally, I avoid it, because it only works within the context of the whole "Catalyst environment". That means it's awkward at best to load the configuration outside of the web environment. I always make sure that application wide setting are available for things like cron jobs. This falls into a category of Catalyst features which are not best practices, but are probably useful for people writing their first web app.

Catalyst gives you hooks for models. Again, this is something I never use, but it's another "useful for web app n00bs" feature.

There's probably many other things I've left out. The point is that Catalyst provides a very powerful set of core web app tools, and that core is actually small.

Relative to a "my way or the highway" type framework (RoR and Jifty, I'm looking at you), it's easy to port an existing application to Catalyst. In fact, using Catalyst::Controller::WrapCGI, you can wrap an existing CGI application with Catalyst, and then convert slowly over to "native" controllers.

And most importantly, you can to Catalyst without touching your model at all! Since many applications have the bulk of their code in the models (at least, they do if you're doing it right), this is a huge win.

Next step is to turn some of this rambling into doc patches. I think a section of the Catalyst tutorial aimed at folks coming from an "old school" web app background would be great, and would really help people like Raymond (and would've helped me a few years back).

Posted by Dave Rolsky on Programming at Thu, 02 Jul 2009 16:54:32 -0000 | Permalink

Inheritance, as most of us know, is rather problematic. Specifically, it's been around since 1967 and for the past 42 years, people have been arguing about how to do it right. Some OO languages don't have inheritance (Self, Javascript). Some languages don't have multiple inheritance (C#, Ruby). Some languages have multiple inheritance with many safeguards (Eiffel) or none (Perl). Some languages just do strange things with inheritance (BETA). Even very experienced OO developers focusing on a single language will argue vociferously about whether or not a design has been done correctly (do you really give a damn about strict equivalence in overridden methods?)

The problem is ultimately that classes fulfill conflicting needs. As an agent of responsibility, a class needs to do everything a class needs to do. That sounds like a stupid tautology, but what it means is that as systems grow, classes grow. Thus, classes tend to get larger. Unfortunately, as agents of code reuse -- via inheritance or delegation -- classes should be smaller. How many times have you seen (or written) code which inherits from something because it needs one or two methods but you're pulling in a lot of extra behavior which you don't need?

This responsibility/reuse tension which leads to classes wanting to be both larger and smaller at the same time is much of the reason why inheritance has proven so problematic. Interfaces in Java and C#, along with mixins in Ruby and other languages were an attempt to decouple the responsibility and reuse needs of classes, but they've all had their problems. Roles seem to handle the decoupling of responsilibity and reuse quite nicely, but some of you may have noticed some of the disagreements that I and others have had about the proper use of roles. Why is that?

Ultimately, the disagreement boiled down to the fact that -- like inheritance -- roles serve more than one master. Roles provide behavior and roles provide an interface. Are we faced with another 40 years of arguing because we've again tried to shoehorn too much into one thing? I don't think this need be the case as, unlike classes, the different things the roles provide aren't necessarily antagonistic, but it would be nice if the different parties sat down and tried to come up with a strategy (syntax?) for implementing roles which cleanly addresses these disparate needs. Otherwise, there really will another 40 years of arguing.

Posted by Ovid on journal at Thu, 02 Jul 2009 13:47:33 -0000 | Permalink

It is probably not that interesting for most of the people but I have my excuses. What is more interesting - at least to me - is why others are involved.

After all there were already more than 40 people who made some contribution to Padre.

So I asked them to start writing down why are they involved. Some of them have already added themselves to the developers page and we started a separate page for translators though that one is new and has no been filled yet.

So why do I write Padre?

There is a short explanation on that page but let me bore you with a longer one here.

I have been teaching Perl 5 for almost 10 years now. Both beginner level and advanced courses. In the beginner courses the majority of the people use Windows with about 20-30 percent using Linux. Most of the Windows users use Notepad++ or a similar editor. Some of them can configure their editor to run the perl script right from the editor, others don't even know how to enable syntax highlighting for Perl 5. Some of the people on Windows don't know what the command line is but even those who know how to open the command prompt are either afraid of it or just dislike it.

Mind you these are not stupid people or anything like that. Some of them have many years of hardware design behind them. They are just used to some kind of IDEs.

Others might use Linux/Unix but many not by choice. In many cases the company they are working for gives them telnet access to an oldish Unix machine and tell them to code in Perl. They hardly know vi, they don't know how to configure syntax highlighting and they don't know how to install any other editor.

I have been using vim for many years and I can testify that it is a superb editor. Emacs is similary strong but I have not used it since university. The problem with both of them is that they are totally different from the editors most people are used to and their learning curve is long and steep. It takes several months or even years to become familiar with them. Most of the people don't want to invest that energy and I certainly don't have the time for that in a 4 days long Perl 5 course.

So I usually tell them to use whatever they are familiar with. I think I never had a student who picked Eclipse or Emacs. There were a few using vi/vim but most of the people used Notepad++ or a similar Windows based editor.

I don't think my students are too different from the average people learning and using Perl. Most of them will never write a full blown web application. Heck most of them won't have a need for object oriented coding as they write only 100-200 line long scripts in Perl.

They never get really familiar with Perl and they will always have to deal with strange code written by others. With all the other tasks they are required to do in their primary language or tool they will keep wondering what is $_ when they see it and will be surprised when they don't.

They will wonder what is the relationship between $_ and $_[0].

So my first objective was to provide a simple Notepad++ like editor with strong support for beginners or people with relatively little experience with Perl.

For example I'd like to make sure that Padre users will be able to highlight $_, press F1 and get an explanation. In case they highlight $_ from a $_[0] expression Padre should be clever enough to point this out and give an explanation on @_ and @_[0].

I knew it is a big task though I did not think it is soo big. In any case I knew I cannot handle this alone, especially as I am not such a good programmer. So from the very beginning I tried to make it interesting, accessible to experienced Perl 5 developers and in turn they made it fun and hackable.

Unfortunatelly most of the experienced Perl developers are also hard-core vim or emacs users and it is nearly impossible to move them away from their editor. Luckily there were a few who got interested by the idea of having a lot of control over their editor and that made the difference between a failing one man project to a project with a lot of potential.

Then at one point came the idea to integrate Parrot and Rakudo and try to provide a platform for Perl 6 development as well. With the awesome work of Ahmad M. Zawawi Padre has a lot to offer to anyone who wants to write Perl 6 code. With some more work we'll be able to integrate Parrot even better and then Padre will be able to handle any language running on top of Parrot.

While there are still lots of problems with Padre and installation can still be difficult it already more or less has the same power as Notepad++ on both Windows and Linux and probably on Mac too and it has some extra features that even vim people started to steal.

Posted at Thu, 02 Jul 2009 02:24:31 -0700 | Permalink

adam@svn:~/svn.ali.as/db$ ls -l
total 30884
-rw-r--r-- 1 adam adam 9558294 Jul  2 03:45 cpandb.gz
-rw-r--r-- 1 adam adam 8538979 Jul  2 03:45 cpandb.bz2
-rw-r--r-- 1 adam adam 5960155 Jul  2 03:45 cpandb.lz
-rw-r--r-- 1 adam adam 3014480 Jun 30 06:46 cpanmeta.gz
-rw-r--r-- 1 adam adam 2658756 Jun 30 06:46 cpanmeta.bz2
-rw-r--r-- 1 adam adam 1825600 Jun 30 06:46 cpanmeta.lz

Posted by Alias on journal at Thu, 02 Jul 2009 06:18:57 -0000 | Permalink

The good people at GitHub have implemented a JSONP api for Gists, so I have updated my graceful degarding script accordingly (see previous post).

Now the code no longer needs to hijack document.write and has better load time characteristics as well. If you use traditional <script> tags then each gist will block the loading of the page at that point, causing jittery page renderings, and preventing the rest of the content from appearing. This script styles the <pre> tags when the DOM becomes ready, and only fetches the syntax highlighted HTML after the onload event, when all other page elements have finished loading.

The second new bit is the Perl scripts I've added to streamline working with these degradable Gists.

The first script is very simple, given a Gist ID it prints out a <pre> tag suitable for inclusion into your HTML document. For instance make_pre_tag.pl 115368 outputs:

<pre id="fake-gist-115368" class="fake-gist">use Moose;
has fun =&gt; ( isa =&gt; &quot;Constant&quot; );</pre>

The second script, update_gists.pl is much more interesting. It uses XML::LibXML to find all the pre tags with the class fake-gist in a document. If these tags have no ID then App::Nopaste is used to create new Gists. Otherwise AnyEvent::HTTP is used to concurrently fetch the updated versions from http://gist.github.com/.

I start by editing a plain XHTML document in my editor, adding code in the <pre> tags, and then I post the code blocks to github all at once. I use CDATA sections to escape the source code in the XHTML document.

<pre lang="perl" class="fake-gist"><![CDATA[
$object->method(
    foo => $bar, 
);
]]></pre>

The script re-escapes that using HTML entities so that it is valid HTML (which doesn't support CDATA). This way the <pre> tags render fine under quirks mode, allowing jQuery to append HTML strings to the document.

Posted by nothingmuch on jquery, github, xml, perl, gist at Thu, 02 Jul 2009 02:08:00 -0000 | Permalink

If you read Planet Perl Iron Man (and you should) or listen to the discussions of the corehackers project, you may have seen more discussion about Perl 5's DarkPAN problem.

One of the big tensions in the Perl 5 world is between progress and stability. (I use both terms with the same sense of distaste I hear the terms "pro-choice" and "pro-life". Then again, I sympathize with linguistic prescriptivism, if only to clarify motives and intent.)

"Perl 5 must change," some people cry. "There's no good reason Perl shouldn't enable strictures and warnings by default for all new programs!"

"Perl 5 cannot change," retort others. "There's too much existing code to change Perl's behavior!"

I find the latter argument ridiculous such that withering mockery is the only good response. That's rarely useful, however.

When people say "Perl 5 cannot change its default behavior!", I believe they have in mind several other points. Some of them are good points. Yet until the Perl community as a whole can address those points directly, we'll remain at an impasse. (The word "impasse" overstates things; to a man, the active Perl 5 pumpkings appear to hew strongly to the "Change is painful and bad and wrong" philosophy, even going as far to say that frequent releases are undesirable hassles because stat calls are not cheap.)

Translation to English of Various Meanings of "Stability Über Alles"

With that in mind, here are several possible meanings of "You can't change default behavior!

• Distributors may upgrade Perl 5 in their installations and may have to upgrade packages which depend on Perl 5 to work with the new version. This is true. This is what distributors do. This is what distributors do with all of their dependencies. This is why distributors exist. This is also only a problem if no optional mechanism of disabling new features exists -- and such a feature needs to exist.
• Changing Perl 5's default behavior may render existing tutorials and examples obsolete. Good. Many existing examples of Perl 5 code are horrible. A steadfast refusal to run unmaintainable code may even encourage the creation of better tutorials and the publication of better examples.

• Existing code -- left untouched for a decade -- may suddenly break.

  I don't understand this point.

  I ran into Perl 4 code the other day. Somehow the last sixteen years of Perl 5 releases have not yet managed to erase all perl4 binaries -- as well as the Perl 4 source code -- from the world's hard drives and tape drives and USB drives. Why should anyone believe that Perl 5.10.0 will not be available when Perl 5.10.1 comes out? Ditto Perl 5.12.0.

  Sometimes this argument has nuance to it. We must use a version of Perl supported by a vendor to whom we offer supplications of fresh fruits, wines, native crafts, and large checks. In other words, you're paying for the privilege of not upgrading. Good for you. Go bug the organization you're paying. That's why you're paying them.

  Sometimes this argument indicates that the arguer has no business working with computers in a professional setting. We don't know what software we're running and we won't know what will break if we upgrade and we don't know how to fix it if it does. If that's you, write your stakeholders a letter suggesting that they try to avoid upgrading, ever. Then find another line of work, perhaps something involving no technology more complex than one rock stacked atop another.

  If you can't test your software against newer dependencies, identify any potential problems, and work with upstream to resolve those issues before you perform an upgrade -- or if you're unwilling to do so -- then you are dangerously incompetent. That kind of incompetence is not the Perl community's responsibility.

  Don't short your stock, either -- that smacks of insider trading.

• Frequent, experimental, zig-zag changes to Perl 5 syntax and semantics will be confusing! Yeah. That's why no one's suggested doing them. Suggesting that Perl 5 could use real function signatures or strictures-on-by-default is very different from throwing every potential combination of hash-and-array-sort function into one big global namespace.

  The desire to add missing features and the desire for more frequent releases by no means implies a lack of foresight or holistic design, nor a lack of comprehensive testing, nor thoughtful refinement of an idea and implementation to the point where it's obviously right.

• Changes mean bugs, and we can't have bugs! There are already bugs. There are already regressions -- including a performance regression that would have affected only a few people if a stable 5.10.1 had come out in early 2008.

  The only way to avoid bugs altogether is to avoid writing software. The best you can do is make them unlikely, catch them early, and fix them quickly (remembering that unreleased software may as well not exist to your users).

• It's irresponsible to break someone else's code, especially if you can't see it. It's insane to support invisible code that may not even exist.
• There are so many competing implementations of this idea on the CPAN, it's obvious there's no one right way to do things! There are so many competing implementations of this idea on the CPAN because there's no obvious good, default, built-in way to do things.
• My code has to run on several different major versions of Perl; I can't take advantage of these new features. You have a change management problem. Not me.
• I can't show you a test case, but this change breaks my code! There's an invisible sign on the road by my house that gives me the right to charge a $5 toll. Pay up.
• This is the way it's always been. How's that working out?

A More Serious Take on Stability

Change doesn't have to be painful. Change doesn't have to be chaotic. It's possible to meet many of the real underlying goals with technical means.

The problem isn't technical, however. It's social. It's fear.

This is the fear of risk -- the risk that unknown problems lurk in seemingly minor changes. This is also the fear of the risk that the cost of mitigating this risk is too high. This is especially the fear of the risk that changing Perl 5 will appear so expensive that people will stop using it.

With all of that mockery out of the way, perhaps the Perl community can have a sober assesment of risk, bereft of fears and stupid technical blatherings that serve only to obscure the real question:

Whose needs do the features and policies and strategies and goals and visions of Perl 5 development serve?

Change for the sake of change itself is useless. Stability for the sake of stability is equally useless. No one wants complete stability or complete chaos. (Even if you think you want complete stability, you don't; when you find a bug or a typo or a confusing section of documentation, you've found a place where perfect stability gets in your way.)

(You'd almost think the Perl 5 community didn't have a few experienced project managers, methodologists, risk managers, and software developers, if not several thousand people who know how to create, maintain, sustain, and release free software projects.)

There must be a middle ground. There must be a way to identify real needs, prioritize useful changes, and deliver those changes to stakeholders in an efficient and effective fashion. I reject the false dilemmas which state that we have to make a choice between relentless, plodding conservatism and Psychotic Hyperactive Purposless-esque change...

... especially when the alternative is to suggest that every file containing modern Perl code start with a wall of boilerplate:

use 5.010;

use strict;
use warnings;

use utf8;
use Carp;
use Want;
use CLASS;
use SUPER;
use autodie;
use autobox;
use signatures;

use mro 'c3';

use IO::Handle;
use File::stat;
use autobox::Core;
use UNIVERSAL::ref;

I hate to channel John Stuart Mill, but if Perl 5 stays like that for long, it won't be a language suitable for novices to write new programs. It'll be merely a great language to maintain code written in the late '90s not yet replaced with something with slightly saner defaults.

That would be a pity.

Posted by chromatic on advocacy, modern perl, perl, perl 5i, perl programming at Wed, 01 Jul 2009 23:31:55 -0000 | Permalink

At the Perl QA hackathon, Andreas Koenig, the maintainer of PAUSE, demonstrated a way to decrease the lag between new distribution uploads to PAUSE and syndication to CPAN mirrors.  (See File::Rsync::Mirror::Recent on CPAN for details).   Running minicpan against a one of the new, fast updating mirrors results a very fresh minicpan, but, until now, only for regular releases, not for alpha development releases.

Last year, I wrote CPAN::Mini::Devel to include development releases in a minicpan, but it depends on an obscure, large, infrequently updated index.¹ Now, with smaller, fast-updating index files available for rsync, I released CPAN::Mini::Devel::Recent to take advantage of them for minicpan as well.

This is all still experimental, but if you do Perl programming and want bleeding-edge development releases within minutes of their upload to PAUSE, then find a fast mirror, try out CPAN::Mini::Devel::Recent and let me know how it works for you.  After installation, you can configure it to be used automatically by adding “class: CPAN::Mini::Devel::Recent” to your .minicpanrc file.

1. indices/find-ls.gz

Posted by david on cpan, perl at Wed, 01 Jul 2009 22:03:34 -0000 | Permalink

The code is starting to take a more stable form. I’ve prepended an underscore to all private/internal subroutines and documented the rest. Perl authors wishing to use grok’s functionality will now have an easier time doing so.

I’ve also added various author tests to the distribution to help keep the code in shape (or at least remind me when it’s not), notably Test::Pod, Test::Pod::Coverage, and Test::Perl::Critic.

Since my last blog post, grok has gained a few features. It can print the name of the target file (like perldoc -l), print an index of known documentation files, output xhtml, and detect whether the target file has Pod 5 or Pod 6 in it. It’s also got some Win32 fixes and more informative error messages.

Pretty soon I will start bringing in some more docs to bundle with it. Tutorials and such. I will also most likely implement function documentation lookup (like perldoc -f) in grok.

Posted by Hinrik Örn Sigurðsson on english, gsoc, perl, perl6 at Wed, 01 Jul 2009 19:33:57 -0000 | Permalink

I'm going to try to do a series of posts about learning Perl internals. I am still a beginner, I have trouble remembering the many macros, or keeping everything in my head all at once, so hopefully I will be able to make some sense of this stuff in a way that is accessible to other beginners.

While I'm definitely diving right in to the deep end, I think PL_runops is a good place to start as any, there's not a lot you need to learn to see how it works. I don't think Perl has a shallow end.

PL_runops is a variable in the interpreter containing a function pointer, which in most cases will be Perl_runops_standard.

Perl_runops_standard is the function that executes opcodes in a loop. Here's how it's defined:

int
Perl_runops_standard(pTHX)
{
    dVAR;
    while ((PL_op = CALL_FPTR(PL_op->op_ppaddr)(aTHX))) {
        PERL_ASYNC_CHECK();
    }

    TAINT_NOT;
    return 0;
}

Perl makes extensive use of macros, which can sometimes be confusing, but in this instance it's not too daunting. This loop will essentially keep calling PL_op->op_ppaddr, assigning the result to PL_op. As long as a valid value is returned it will keep executing code.

Just to get it out of the way, PERL_ASYNC_CHEK is a macro that checks to see if any signals were delivered to the process, and invokes the handlers in %SIG if necessary.

So what's PL_op? It's essentially Perl's instruction pointer, it refers to the opcode currently being executed. When Perl compiles source code it produces an optree. This is probably one of the more complicated structure in Perl internals, but right now we're only concerned with one small part, the op_ppaddr field of a single node in the tree. The op_ppaddr field contains a pointer to the function that implements the op.

PP stands for push/pop, which means that it's a function that operates in the context of the Perl stack, pushing and popping items as necessary to do its work, and returns the next opcode to execute.

pp.h defines a PP macro, which sets up a signature for a function that returns an op pointer. Let's have a look at two simple PP functions. First is pp_const:

PP(pp_const)
{
    dVAR;
    dSP;
    XPUSHs(cSVOP_sv);
    RETURN;
}

This is an implementation of the const op, which pushes the value of a literal constant to the stack. The cSVOP_sv macro is used to get the actual SV (scalar value structure) of the constant's value from the optree. SVOP is an OP structure that contains an SV value. The c stands for "current".

Let's rewrite the body of the macro using some temporary values:

/* the current opcode is op_const, so the op structure is an SVOP
 * Instead of using the long chain of macros we'll put it in a variable */
SVOP *svop = (SVOP *)PL_op; 

/* and then we can simply use the op_sv field, which
 * contains a pointer to a scalar value structure */
SV *sv = svop->op_sv;

This SV is then pushed onto the stack using the XPUSHs macro. The X in XPUSHs means that the stack will be extended if necessary, and the s denotes SV. To read the documentation of these macros, refer to perlapi.

The next thing that executes is the RETURN macro. This macro is defined in pp.h, along with a few others:

#define RETURN          return (PUTBACK, NORMAL)

#define PUTBACK         PL_stack_sp = sp

/* normal means no special control flow */
#define NORMAL          PL_op->op_next

/* we'll use this one later: */
#define RETURNOP(o)     return (PUTBACK, o)

Recall that there are no lists in C. The comma operator in C executes its left side, then its right, and returns that value (unfortunately we have that in Perl, too). The RETURN macro therefore desugars to something like:

PL_stack_sp = sp;
return(PL_op->op_next);

The XPUSHs macro manipulated the local copy of the pointer to the stack, sp as it was adding our SV. This change is not immediately written to the actual stack pointer, PL_stack_sp . The PUTBACK macro sets the "real" stack to the version we've manipulated in the body of our opcode.

Then the opcode simply returns PL_op->op_next. The op_next field in the op contains a pointer to the next op that should be executed. In this case if the code being executed was:

my $x = 42;

then the const op compiled to handle the 42 literal would have pushed an SV containing the integer 42 onto the stack, and the op_next in this case is the assignment operator, which will actually use the value.

So, to recap, when PL_op contains a pointer to this const op, PL_op->op_ppaddr will contain a pointer to pp_const. PL_runops will call that function, which in turn will push ((SVOP *)PL_op)->op_sv onto the stack, update PL_stack_sp, and return PL_op->op_next.

At this point runops_standard will assign that value to PL_op, and then invoke the op_ppaddr of the next opcode (the assignment op).

So far so good?

To spice things up a bit, here's the implementation of logical or (||):

PP(pp_or)
{
    dVAR; dSP;
    if (SvTRUE(TOPs))
        RETURN;
    else {
        if (PL_op->op_type == OP_OR)
            --SP;
        RETURNOP(cLOGOP->op_other);
    }
}

The or op is of a different type than the const op. Instead of SVOP it's a LOGOP, and it doesn't have an op_sv but instead it has an op_other which contains a pointer to a different branch in the optree.

When pp_or is executed it will look at the value at the top of the stack using the TOPs macro, and check if it evaluates to a true value using the SvTRUE macro.

If that's the case it short circuits to op_next using the RETURN macro, but if it's false it needs to evaluate its right argument.

--SP is used to throw away the argument (so that $a || $b doesn't end up returning both $a and $b).

Then the RETURNOP macro is used to call PUTBACK, and to return PL_op's op_other. RETURN is essentialy the same as RETURNOP(NORMAL). op_other contains a pointer to the op implementing the right branch of the ||, whereas op_next is the op that will use the value of the || expression.

This is one of the most basic parts of the Perl 5 virtual machine. It's a stack based machine that roughly follows the threaded code model for its intermediate code structures.

The data types mostly revolve around the SV data structure, and moving pointers to SVs from op to op using the stack.

For me the hardest part to learn in Perl is definitely the rich set of macros, which are almost a language in their own right.

If you search for runops on the CPAN you will find a number of interesting modules that assign to PL_runops at compile time, overriding the way opcodes are dispatched.

For more information on Perl internals, the best place to start is perlguts, and the wonderful perlguts illustrated.

Posted by nothingmuch on internals, c, perl at Wed, 01 Jul 2009 19:17:00 -0000 | Permalink

On the back my my improved and high-coverage CPAN::Mini::Visit and Archive::* fixes, I've finally managed to build a complete-coverage version of CPANDB.

CPANDB is a merged and cleaned up schema that combines the CPAN index, the "CPAN Uploads" database (for PAUSE upload dates), and both class and distribution level dependency information held in META.yml files (replacing the CPANTS dependency graph).

To take a look at it, you can grab a copy of the SQLite database directly from the following URL.

http://svn.ali.as/db/cpandb.gz.

The data sources used to generate it are not perfectly time-synced yet, so I expect to see a few minor flaws for another release or two. But compared to everything else available (from pretty much everybody) this should be a significant improvement.

As well as clearing up the last tiny data quality issues, I'm also yet to merge in the rt.cpan.org database (which is almost ready) and the CPAN Ratings database (which is a text file I really don't want to have to parse).

But don't let this stop you trying it out now (I've appended the schema to the bottom of this post so you can get a clearer idea of what's in there).

As usual, feedback is welcome.

CREATE TABLE author (
        author TEXT NOT NULL PRIMARY KEY,
        name TEXT NOT NULL
);

CREATE TABLE distribution (
        distribution TEXT NOT NULL PRIMARY KEY,
        version TEXT NULL,
        author TEXT NOT NULL,
        release TEXT NOT NULL,
        uploaded TEXT NOT NULL,
        FOREIGN KEY ( author ) REFERENCES author ( author )
);

CREATE TABLE module (
        module TEXT NOT NULL PRIMARY KEY,
        version TEXT NULL,
        distribution TEXT NOT NULL,
        FOREIGN KEY ( distribution ) REFERENCES distribution ( distribution )
);

CREATE TABLE requires (
        distribution TEXT NOT NULL,
        module TEXT NOT NULL,
        version TEXT NULL,
        phase TEXT NOT NULL,
        PRIMARY KEY ( distribution, module, phase ),
        FOREIGN KEY ( distribution ) REFERENCES distribution ( distribution ),
        FOREIGN KEY ( module ) REFERENCES module ( module )
);

CREATE TABLE dependency (
        distribution TEXT NOT NULL,
        dependency TEXT NOT NULL,
        phase TEXT NOT NULL,
        PRIMARY KEY ( distribution, dependency, phase ),
        FOREIGN KEY ( distribution ) REFERENCES distribition ( distribution ),
        FOREIGN KEY ( dependency ) REFERENCES distribution ( distribution )
);

Posted by Alias on journal at Wed, 01 Jul 2009 17:17:06 -0000 | Permalink

A few times in the past, I wanted to find perl 5 CPAN distributions that only had a Module-Build-based Build.PL file and not a fallback Makefile.PL file. Yesterday, after some trial and error, I was able to formulate Yahoo Search query, to do just that.

Here it is - the shortened URL with a hyperlink to the full one:

http://xrl.us/bezbkx - Yahoo Search to find Build.PL only distributions

It works by looking for specific phrases in the /dist pages, looking for "Build.PL" and specifically excluding "Makefile.PL". I hereby place this URL and whatever associated techniques under CC0 / Public Domain. Enjoy!

Posted by shlomif@iglu.org.il (Shlomi Fish) on search, tip, tech, building, tech tip, extutils-makemaker, build.pl, perl, build, module-build, yahoo, cpan at Wed, 01 Jul 2009 15:27:35 -0000 | Permalink

I'd like to have a system that can hold all the artifacts of a test execution and that also knows a lot about TAP. Smolder is a good start as it knows about TAP but there are tons of other things I'd like to have in it.

Some of them are hard to add, others are harder.

Anyway, earlier I wrote about Reporting Test Results. Now let me write about my wish-list:

1) I'd like Smolder to collect all the artifacts of a test run. Look at the data Buildbot collects or what is collected by the PostgreSQL test farm! Each run has several stages and the actual test run is only one of them. In a Test Reporting system I'd like to have access to all the output and error files of all the stages.

2) The tests might create some additional log files, I'd like to be able to include those too.

3) One major issue we have is test counting. TAP encourages you to have a plan - the number of test units you are going to execute in a test run - so that it can check the number of actually executed test to that number. Keeping the plan up to date is annoying, people don't like it so many use no plan. Some TAP producers don't even let you plan ahead (e.g. the ones I saw in PHP) but I think generally those that are more xUnit like won't have a test plan.

In larger settings this plan does not even help as it does not provide any tools to avoid reducing the test plan accidentally along with a shortcut in the test script: A developer can comment out several tests temporarily and adjust the plan for the time while he is trying to fix a bug forgetting to revert these changes after he is done.

Smolder could help a bit here: On the first result report the administrator of the project should look at the reports and mark them as having the correct number of tests. Then if Smolder notices a test report with a different number of test for the same test file it should raise an alert. The project administrator can examine the change and decided if that is the new correct number of tests setting the new expected number to this or if it was changed by mistake.

4) History of unit tests. If we would like to be able to provide a history based on the actual test units (calls to an ok() function) then we need to have some unique identifier for each test as we cannot assume that the test number or the test name will remain the same. After all people sometimes insert new tests or remove old test at the beginning of the test scripts and the names can change.

Smolder could also help here too. It could track the test numbers and test names. If it sees a change it can alert the project administrator who can then use some GUI to connect the old test units to the new ones.

Both involves some manual work but I think large organizations will be ready to invest that work in order to get the benefit of greater control and better reports.

What would you like to have in your test reporting system?

Posted at Wed, 01 Jul 2009 03:24:40 -0700 | Permalink

Autodie 2.00 released
This weekend the long awaited autodie 2.00 for Perl was released to the CPAN, which was almost immediately replaced by 2.02, which fixes some oopsed tests and which adds a couple more features to give us a really sweet experience. This blog entry assumes you're using 2.02.

Observant viewers will notice that the major version number has changed. I've taken the great leap from 1.999 to 2.00. Clearly, something is different, and you might be wondering what.

Well, autodie 2.0 now supports a hinting interface for user-defined subroutines. Put simply, if you have a user-defined subroutine that does something funny to signify failure, you can now tell autodie about that. Once it knows, it can Do The Right Thing when checking your subroutine. You can even put the hints into the same file as those subs, and if someone is using autodie 2.00, it will find the hints and use them.

This may not sound very exciting, but it is. It means that a lot of really ugly error-checking code, both on the CPAN and the DarkPAN, can go away. Lexically. Still not convinced this will change your life? Let's look a little more closely; trust me, you'll like it.

Let's pretend you're working on a piece of legacy code. For some reason, the people who wrote this code decided the best way to signal errors is by returning the list (undef, "Error message"). I don't know why, but I've seen this anti-pattern emerge independently in three 100k+ line projects I've been involved in.

sub some_sub {
    if ( not batteries_full() ) {
        return ( undef, "insufficient energy" );
    }

    if ( not coin_inserted() ) {
        return ( undef, "insufficient credit" );
    }

    my @results = some_calculation();

    return @results;
}

If you want to check to see if some_sub() returns an error, you need to capture its return values, look at the first one to see if it's undefined, and if it's not, use the second one as your error. At least, that's what you're supposed to do.

What actually happens is most developers decide that's way too hard, and don't bother checking for errors. Then one day, the batteries on your doomsday-asteroid-destroying-satellite go flat, nobody notices, and through an ironic twist of fate you're left as the last known human survivor, and there are zombie hordes and walking killer plants outside.

So, how can autodie help us? Well, before version 2.00, it couldn't. But now, with autodie::hints, it can! We can give autodie hints about how the return values are checked. They look like this:

use autodie::hints;

autodie::hints->set_hints_for(
    'Some::Package::some_sub' => {
        scalar => sub { 1 },
        list   => sub { @_ == 2 and not defined $_[0] },
    },
);

Our hints here are simple subroutines. If they return true, our subroutine has failed. If they return false, it's executed successfully. Notice that our scalar hint always returns true. That's because we consider any call of our subroutine in scalar context to be a mistake. It's returning a list of values, and you should be checking that list.

Once we've set our hints, we can then use autodie to automatically check if we're successful:

use Some::Module qw(some_sub);

sub target_asteroid {

    use autodie qw( ! some_sub );

    # autodie has lexical scope, so only calls to some_sub inside
    # the target_asteroid subroutine are affected.

    my @results = some_sub();     # Succeeds or dies
}

sub target_ufo {
    my @results = some_sub();

    # autodie is out of lexical scope, so we have to manually
    # process @results here.
}

If you're wondering what that exclamation mark means, it means "insist on hints", and is a new piece of syntax with autodie 2.00. If for any reason autodie can't find the hints for some_sub, our code won't compile. That's a very good thing, and avoids us having a false sense of security if we use autodie on an unhinted sub.

However the error messages from autodie aren't really that useful. They're going to be things like "Can't some_sub() at space_defense.pl line 53". There's a noticable lack of explanation as to why some_sub() failed.

Luckily, since the way early versions of autodie, we've been able to register message handlers. And with the new features in autodie 2.02, we can produce very rich messages. Let's see how!

use autodie::exception;

autodie::exception->register(
    'Some::Module::some_sub' => sub {
        my ($error) = @_;

        if ($error->context eq "scalar") {
             return "some_sub() can't be called in a scalar context";
        }

        # $error->return gives a list of everything our failed sub
        # returned.  We know this particular sub puts the error
        # message the second argument (index 1).

        my $error_msg = $error->return->[1];

        return "some_sub() failed: $error_msg";
    }
);

Now, whenever some_sub() fails, it'll print a genuinely useful message, like "some_sub() failed: Insufficient energy at space_defense.pl line 53". Yes, autodie automatically adds the file and line number for you. Nice!

But wait, there's more! We don't want to see this sort of code floating around in your programs. You may be dealing with other people's modules that you can't modify, so we can't hide all this configuration in there. So, we can write our own pragma that contains all this info. Here's the full code for a theoretical my::autodie pragma, and is the exact same code used by the t/blog_hints.t file in autodie's test suite.

package my::autodie;
use strict;
use warnings;

use base qw(autodie);
use autodie::exception;
use autodie::hints;

autodie::hints->set_hints_for(
    'Some::Module::some_sub' => {
        scalar => sub { 1 },
        list   => sub { @_ == 2 and not defined $_[0] }
    },
);

autodie::exception->register(
    'Some::Module::some_sub' => sub {
        my ($E) = @_;

        if ($E->context eq "scalar") {
            return "some_sub() can't be called in scalar context";
        }

        my $error = $E->return->[1];

        return "some_sub() failed: $error";
    }
);

1;

It works exactly the same as regular autodie, except it also knows how to handle some_sub(), and display good looking error messages. Here's how we'd use it:

use Some::Module qw(some_sub);
use my::autodie qw( ! some_sub );

my @results = some_sub();  # Succeeds or dies with a useful error!

There's a lot more you can do with autodie, and if you want to learn more, I'd suggest coming to my talk at OSCON or YAPC::EU, where I'll be covering all this and more, with a distinctive Star Trek twist. ;)

Posted: 1st July 2009.

Tags: autodie hints perl

Bookmark:

Posted at Wed, 01 Jul 2009 00:00:00 -0000 | Permalink

The latest developer release of Test::More allows subtests. Subtests are great in that they solve a lot of problems in advanced Perl testing, but they have required a change in Test::Builder. Previously you could do stuff like this:

package Test::StringReverse;

use base 'Test::Builder::Module';
our @EXPORT = qw(is_reversed);

my $BUILDER = Test::Builder->new;

sub is_reversed ($$;$) {
    my ( $have, $want, $name ) = @_;

    my $passed = $want eq scalar reverse $name;

    $BUILDER->ok($passed, $name);
    $BUILDER->diag(<<"    END_DIAG") if not $passed;
    have: $have
    want: $want
    END_DIAG

    return $passed;
}

1;

And you've have a simple (untested ;) test for whether or not strings are reversed.

The reason that worked is that Test::Builder->new used to return a singleton. This is no longer true. If someone uses your test library in a subtest, the above code would break. Instead, you want to do this:

sub is_reversed ($$;$) {
    my ( $have, $want, $name ) = @_;

    my $passed  = $want eq scalar reverse $name;
    my $builder = __PACKAGE__->builder;

    $builder->ok($passed, $name);
    $builder->diag(<<"    END_DIAG") if not $passed;
    have: $have
    want: $want
    END_DIAG

    return $passed;
}

It's a minor change, it's completely backwards-compatible and it supports subtests. There's a work-around being planned, but it's not out there yet.

Posted by Ovid on journal at Tue, 30 Jun 2009 14:16:11 -0000 | Permalink

An imaginary conversation synthesized from past discussions and the responses I wish I made.

• Customer: We need read-only access to your database.
• Ovid: No.
• Customer: Please?
• Ovid: No.
• Customer: But I need ad-hoc queries.
• Ovid: Your ad-hoc cartesian join returning 12 billion rows was real fun.
• Customer: I promise I won't do it again.
• Ovid: That's what you said about the ad-hoc cartesian join returning 10 billion rows.
• Customer: But this time I mean it.
• Ovid: So do I. We guarantee backwards-compatibility in our API, not our database. If we move a field from one table to another, your queries will break.
• Customer: Then tell us when you do that.
• Ovid: We did that with another team and had to keep delaying releases while they updated their system.
• Customer: Then you can provide views to maintain backwards-compatibility.
• Ovid: We do that already. "View" as in "Model-View-Controller". It's part of our REST API; you should check it out.
• Customer: But your REST API doesn't provide all the information I need.
• Ovid: It provides more than the information you need because much of it represents knowledge not stored in the database. If you need more information, let's see what we can do to add this to the API.
• Customer: Why are you being so difficult?
• Ovid: Because your temporary convenience is not more important than my long-term pain.

Don't let external customers read directly from your database. Just don't. The usual justification is the need to support ad-hoc queries. Get a few samples and try to figure out a general mechanism to support their actual business needs. If you let them read from your database, they will become dependent on this and beg you to hold off database changes or complain if you don't. As your project grows larger, the pain grows more severe. They will have the best of intentions, but good intentions mean nothing when you need to coordinate your internals with people who should know better than to violate encapsulation.

As a side note, ad-hoc queries, even if not causing performance issues, could potentially be dangerous if the people making them aren't really thinking them through. The problem is two-fold. One, they might not be really paying attention to their core business needs (this is subtle and hard to explain, but common). The other problem is that they might very well be making a query that your API already supports, but because they don't rely as much on your API, they don't know it.

Posted by Ovid on journal at Tue, 30 Jun 2009 11:48:19 -0000 | Permalink

After my little research into the Eclipse foundation (Comparing the Eclipse Foundation with The Perl Foundation and EPO) I was told to take a look at the success of Ubuntu. While I know there is a huge difference as Ubuntu is an operating system while Perl is just a language but I got interested and here are the results.

Ubuntu does not have a foundation. It has a strong and very enthusiastic community and it is backed by Canonical, the company of Mark Shuttleworth.

That means the Perl community probably cannot learn much from the way Ubuntu and Canonical are connected but we can still look at the larger picture of how Ubuntu succeeded so far?

First I'll look at the business model of Canonical and later on the elements that helped Ubuntu become one of the leading GNU/Linux distributions in only a few years.

The Ubuntu Business model

So if we look closer I think we can safely say that Ubuntu is a product Canonical is sponsoring in order to make money somehow. I would not say by selling added value service as I am not exactly sure what is their business model. I noticed a few areas where Canonical might be making money:

• Selling wearables and accessories.
• Providing support contracts at nice sums.
• Providing Training and Certification.
• Distribution mechanism for software.

Selling T-shirts can bring in nice revenue to a small company but for Canonical it is more part of the marketing mix than the revenue mix.

Providing support can be an interesting business model, IBM makes a lot of money on it. I guess Canonical does too. This will just increase as Ubuntu invades the corporate infrastructure.

Training and Certification can be a lucrative business if executed well and I trust them to do it well. If they can convince companies that they need Ubuntu Certified Engineers to run Ubuntu smoothly then will see a huge number of people attending their classes and taking their certificate exams.

Still the biggest business opportunity I think is the one that is not even listed in the Canonical Store but only under Services. That is, they offer to be a distribution channel for ISVs.

In the Microsoft world people are used to download software and click-click-click install it. In the GNU/Linux world almost all the important software is packaged and distributed by the Linux distributor. It is especially true for Debian/Ubuntu but I think it is increasingly so with Fedora and Mandriva as well. That means the most common installation mode in these versions of Linux is via a mechanism similar to the "Add/Remove Software" on Windows.

So if you have a software which you'd like to distribute to Linux users, one of the best ways is to make sure each one of the Linux distributors will package your software. Otherwise you'll need to build .deb and .rpm files for each distribution or a stand-alone executable which is frowned upon by most of the Linux system administrators.

If your software is not open source you don't have much chance that any of the Linux distributors will add that package to their repository making it almost impossible to penetrate the Linux Desktop market.

If Canonical manages to get a large chunk of the market share in the Linux desktop market it effectively can open or close the distribution channel of ISVs making them pay well for the right to use Ubuntu as their distribution platform.

So what will happen to Ubuntu in 5 years?

I don't know.

If you look at the background of Mark Shuttleworth, he built Thawte and more or less at the peak of the dot.com bubble he sold it to VeriSign the only competitor in the digital certificates market. So he is a clever guy who won't be afraid to sell Canonical to the market leader with a near monopoly in the desktop business or to the competitor with the most money making a few billions of dollars. The sky isn't the limit

The Perl aspect

Perl is not an operating system and it is not the open source arm of a company. It cannot be sold but it can provide nice revenue streams to individuals or companies offering service with Perl. There might be other revenue sources as well but lets look at the 4 revenue streams that Canonical has:

• Selling wearables - on Perl conferences some stuff is auctioned which generates some revenue to organize the next conference. This could be further expanded but the income is very limited.
• Providing support: ActiveState does it as do a number of other small companies mostly in the local markets or in specialized fields (e.g. Shadowcat providing web development services using Catalyst or Summersault providing web development services using Titanium.) This could be exploited further providing similar services in other fields as well, not only web application development.
• Offering Perl training provides nice income to several people in the Perl community including myself. This does not seem to turn any of us rich but provides us with a reasonable income stream to support ourselves. Certification is a sour point in the Perl community. Some people say that they don't want to have any certification systems for Perl others see it as the Holy Grail to make Perl more accepted by corporations. In any case this could be a revenue stream but I don't think it will happen any time soon.
• Unlike an Operating system Perl does not control any distribution channel but it could be used in a related way. Actually it could be used similar to how the JVM provides a platform of application development. I am sure one day we will have a binary package of Perl with tons of CPAN modules already installed. Something like Strawberry Perl but with many more modules that can be installed on all major OS-es. That thing could be used as a development and distribution platform for all kinds of software.

  I am not sure this can directly generate revenue but it will help enlarge the Perl user base and it could provide more opportunities for development, support and training services for more people in the Perl community.

I'll write about the success of Ubuntu in a separate post. for the meantime you can read the Ubuntu Story between 2004 and 2006. They don't keep their history up to date either.

Posted at Tue, 30 Jun 2009 02:58:29 -0700 | Permalink

My favorite project of the year may be Schwern's perl5i. His goal is to fix as much of Perl 5 in one swoop. Unlike my Modern::Perl, he's not limiting himself to core modules and pragmata. Anything on the CPAN that fixes a deficiency or problem in the language is fair game.

That goal may remind you of the goals of modern Perl.

That means autobox and autobox::Core and signatures and Devel::Declare.

The i stands for imaginary in the same way that the square root of negative one is an imaginary number. (Yes, clever Reddit and OSNews kiddies. It can also stand for "irrational", thus Perl 5 is irrational. Your scathing wit withers discussion. Have a lollipop.)

This is important for several reasons.

First, it's a great, grand experiment to discover what Perl 5 could be right now. It's a great language and you can do amazing things with it, but it has its flaws. What if someone could correct them? Perl 5.10.1 won't fix them. Perl 5.12 probably won't. The DarkPAN is too big and scary to change Perl 5's defaults all even by the time of Perl 5.14 -- but if you choose to use perl5i, presumably you know what you're doing. What would the language look like and how would it work then?

Perhaps some features will make their way into the core.

Second, it's a great, grand opportunity to make sure that all of these new pragmata and features and syntaxes work together nicely. It's much easier to makechanges now -- to discover incompatibilities and subtle infelicities -- when they all get used together. Better yet, the perl5i hackers can file bugs and work with individual distribution maintainers to seek out consensus and compatibility.

Third, it's a great, grand educational tool to demonstrate corners of CPAN that deserve far more popularity than they have. For example, today I suggested the use of the Want module, which fixes a poorly named core keyword and extends it to greater utility. This module may even help the included autobox::Core handle reference contexts appropriately.

Right now it's not stable; it's experimental. It's worth downloading and testing in your own code, however. Write software. See what happens. File bugs. Suggest new features. See what's compatible and what isn't. Explore. Explain. Enjoy.

To help with development, please visit perl5i on GitHub or visit #perl5i on irc.perl.org.

Posted by chromatic on modern perl, perl, perl 5, perl programming, perl5i at Tue, 30 Jun 2009 00:17:59 -0000 | Permalink

I mean, they're not one-liners. They're full programs. They just do something really really simple.

This one blows away all remotes (left over from, say, GitHub) and adds all the remotes for users on our git.example.com git-on-ssh hosting box.

#!/usr/bin/perl
use strict;
use warnings;

{
  package Git::Our::Remotes;
  use base 'App::Cmd::Simple';

  use autodie qw(:default :system);
  use Cwd;
  use Getopt::Long::Descriptive;
  use Term::ReadKey;

  my $HOST = q{git.example.com};

  sub opt_spec {
    return (
      [ 'dry-run', "be like quality department; don't actually do anything" ],
    );
  }

  sub _runcmd {
    my ($self, $opt, $cmd) = @_;

    if ($opt->{dry_run}) {
      print "running: $cmd\n";
    } else {
      system($cmd);
    }
  }

  sub validate_args {
    my ($self, $opt, $args) = @_;

    my $user = $ENV{USER};

    unless (defined $args->[0]) {
      my $cwd = getcwd;
      $cwd =~ s{.*?/?([^/]+)\z}{$1};
      $args->[0] = $cwd;
    }

    my $repo = $args->[0];

    $self->usage_error("too many args given") if @$args > 1;
    $self->usage_error("do not run as root")  if $user eq 'root';
    $self->usage_error("illegal repo name")   if $repo !~ /\A[-_a-z0-9]+\z/;
    $self->usage_error("you should run this in a repo") if ! -d '.git';
  }

  sub run {
    my ($self, $opt, $args) = @_;

    my $repo = $args->[0];

    # should be using a library!
    print "press any key to reset all remotes for repo '$repo'";
    Term::ReadKey::ReadMode 'cbreak';
    Term::ReadKey::ReadKey(0);
    Term::ReadKey::ReadMode 'normal';
    print "\n";

    my $user    = $ENV{USER};
    my @others  = grep { $_ ne $user }split / /, (getgrnam('staff'))[3];

    my @remotes = `git remote`;
    chomp @remotes;

    $self->_runcmd($opt, "git remote rm $_") for @remotes;

    $self->_runcmd($opt, "git remote add deploy git\@$HOST:$repo.git");
    $self->_runcmd($opt, "git remote add origin $user\@$HOST:git/$repo.git");

    for my $o (@others) {
      $self->_runcmd($opt, "git remote add $o $user\@$HOST:~$o/git/$repo.git");
    }

    print "remotes created!\n";
  }
}

Git::Our::Remotes->import;
Git::Our::Remotes->run;

...and that's it. Anyone can run git ourremotes to fix remotes setup. If the user hasn't set up his own hosted ssh remote, he can just run git hubclone --bare-only using the script I blogged about on Saturday. Done!

Posted by rjbs at Mon, 29 Jun 2009 16:15:37 -0500 | Permalink

- Rewrote MojoX::Renderer, it is not backward compatible and templates need to be renamed in the following 3 part format "index.html.tt"!
- Added exception support to MojoX::Dispatcher::Routes, this change is not backward compatible and "dispatch" calls now return exception objects for errors and false otherwise.
- Changed routes syntax, this change is not backward compatible and you need to change all your existing routes.
"/:foo" becomes "/(foo)"
"/^foo" becomes "/((foo))"
"/*foo" becomes "/(((foo)))"
- Added full HTTP 1.1 pipelining support to all Mojo layers.
- Added layout support to MojoX::Renderer.
- Made render call optional.
- Added format support to MojoX::Routes.
- Added Mojo::Loader::Exception.
- Added wildcard symbol support to MojoX::Routes and rewrote many routes internals.
- Added Makefile.PL generator.
- Added HttpOnly support to Mojo::Cookie. (burak)
- Support more CGI implementations.
- Added support for namespaces only dispatching in MojoX::Dispatcher::Routes.
- Added encoding support to Mojo::Template and made "utf8" the default.
- Added HEAD support to Mojo::Server::Daemon. (acajou)
- Added new relaxed placeholder to MojoX::Routes::Pattern.
- Added Mojo::Template::Exception.
- Added HEAD support to the Mojo::Transaction state machine and related modules. (acajou)
- Added safe_post option to Mojo::Pipeline. (acajou)
- Made chained => 1 the default in Mojo::Base.
- Fixed compiler bug in Mojo::Template that prevented more advanced control structures, you might have to add additional semicolons to some of your templates.
- Fixed Mojo::Date to not crash on invalid dates. (vti)
- Fixed chunked support in Mojo::Server::Daemon and Mojo::Client.
- Fixed tokenizer in MojoX::Routes::Pattern to support "0" values. (Anatoly Sharifulin)
- Fixed parsing of "0" in Mojo::Path. (charsbar)
- Fix server tests on win32. (charsbar)
- Fixed leading whitespace problem in the request parser. (acajou)
- Fixed broken pipe problem in Mojo::Server::CGI. (vti)
- Added more diagnostics options to Mojo::HelloWorld. (uwe)
- Fixed empty cookie parsing. (vti)
- Fixed a case where child processes migth hang in Mojo::Server::Daemon::Prefork. (gbarr)
- Fixed a bug in MojoX::Dispatcher::Routes where the renderer would be called with an empty stack. (melo)
- Fixed a escaping problem in Mojo::Parameters. (vti)
- Updated Mojo::URL to be more template friendly.
- Improved Solaris compatibility.

Posted by Sebastian on Perl at Mon, 29 Jun 2009 19:39:04 +0200 | Permalink

[AutoPrereq]

Posted by jquelin@gmail.com (Jérôme Quelin) on plugin, dzill, dist, perl at Mon, 29 Jun 2009 16:45:00 -0000 | Permalink

I've found that one of the best ways to try new testing ideas is to run them against our test suite at work. We've over 30,000 tests at this point, with several test harnesses integrating together, along with two fundamentally different test systems. This means that when I throw something at this test suite, I often shake out bugs. My new Test::Aggregate::Nested combined with our test suite managed to find a rather serious issue with the new subtest function in Test::More. The following, for example, fails:

    use Test::More tests => 2;
    use Test::XML;

    ok 1;
    subtest 'FAIL!' => sub {
        plan tests => 1;
        is_xml '<foo/>', '<foo/>', 'Singleton fail';
    };
    __END__
    xml.t ..
    1..2
    ok 1
        1..1
    Cannot run test (Singleton fail) with active children at /home/ovid/pips_dev/work/Pips3/branches/rights_modeling/deps/lib/perl5/Test/XML. pm line 57.
        # Child (FAIL!) exited without calling finalize()

There's nothing wrong with the code as it's written, but Test::XML knows that the Test::Builder object is a singleton, so there's a false optimization in the code. Near the top of the package, you see this line, defined outside of all functions:

    my $Test = Test::Builder->new;

If every Test::XML function simply had that line in the function rather than trying to share this across all test functions, the subtest code would work fine. Instead, the author knew he had a singleton and there's no point in re-instantiating, is there?

To be fair, I've done the same thing before (see Test::JSON), even though I knew it might be a bad idea. Heck, lots and lots of testing libraries have this issue. Now we need to figure out how to deal with this problem or else subtests aren't going to play well with a lot of code. Damn.

Posted by Ovid on journal at Mon, 29 Jun 2009 14:00:00 -0000 | Permalink

I had a lot of fun at YAPC::NA this year. I met some forest friends that didn't make it last year, such as ilmari, Nick Perez, and Dylan Hardison. I also laughed until it hurt. A few times.

I was a hermit on Sunday and Monday nights because I had not finished my slides. That was very unfortunate, so I will have future talks ready well in advance. Jitting slides is so not my style.

Moose Course

The eight hour Moose course went very well. I was a passable Rolsky-substitute. The feedback I recieved was (nearly) unanimously good, so I'm thrilled that people enjoyed it. There were some minor complaints, so if I ever teach a course again I'll be sure to keep them in mind. This year, YAPC::NA is running surveys on talks and courses, so I expect I'll learn more ways that the class could be improved. If you attended the class, please do not worry about offending Dave or me if you disliked something.

I was under-prepared for the class, but the course material was very good. I got the impression that the exercises were extremely helpful. In particular, having a good test suite (powered by the metaobject protocol!) was very useful. The exercises were also useful for me because they let me sit down. Speaking on your feet for so many hours is tough.

Like the first version of anything, the course material contained some typos, inconsistencies, and bugs. It was pleasant to fix these and push them back to the source repository while people worked on the exercises. For some reason, we could not get public (i.e. noncommitter) access to the repository working, perhaps due to DNS issues. It would have been useful to have people pull my changes to fix the exercises. Instead, I settled for announcing what the problems were and how to fix them.

Context

Early in the course I had a vitriolic rant about Perl's notion of context. On the whole I think context is a nice solution to a certain class of problems, but it definitely has some edge cases where it really bites programmers. For example, propagating context is very fiddly. You must propagate context if you wrap a function to do work after it is called without affecting its return value.

sub method {
    my $self = shift;

    # call SUPER::method in the right context.
    # not handling void context is a BUG!
    my @ret;
    if (wantarray) {
        @ret = $self->SUPER::method(@_);
    } elsif (defined wantarray) {
        $ret[0] = $self->SUPER::method(@_);
    } else {
        $self->SUPER::method(@_);
    }

    $self->called_method($self->called_method + 1);

    # return what the caller asked for.
    # void context doesn't need to be explicitly handled here
    # .. or does it? maybe!
    return @ret if wantarray;
    return $ret[0];
}

I don't recall mentioning in the class that Moose does all of this for us in the after method modifier:

after method => sub {
    my $self = shift;
    $self->called_method($self->called_method + 1);
};

The other painful detail about context is in returning a list in scalar context. Most people know that an array in scalar context evaluates to the number of elements:

sub users {
    my $self = shift;
    my @users = $self->objects('users');
    return @users;
}

my $users = $game->users; # 51

However, a lot of people don't know that there is an irritating amount of subtlety in evaluating a list in scalar context:

sub administrators {
    my $self = shift;
    return ('Sartak', 'autarch');
}

my $admins = $game->administrators; # autarch

Instead of receiving the count of administrators, we get the last element of the list. The problem is actually the comma. Due to C's strong influence on Perl 5, the comma is not just a list separator. In scalar context it acts as a sequencing operator. We're evaluating the string 'Sartak', throwing it away, then evaluating the string 'autarch'. There is no list at all here!

Binary "," is the comma operator. In scalar context it evaluates its left argument, throws that value away, then evaluates its right argument and returns that value. This is just like C's comma operator.

– perldoc perlop

Almost everywhere else in the language, arrays and lists are interchangeable. I wish this inconsistency could be excised. Thanks to do BLOCK, we do not need C's comma operator at all.

Extending Moose

On Tuesday morning I presented Extending Moose for Applications (keynote source). There were maybe 20 attendees. I blame the 8AM start time. Oh well.

I'm told that at one point someone walked out saying the talk got "too abstract". It certainly is a very abstract topic, but I hope the examples showed that there is value to be had in using the ideas I presented.

Some good questions were asked, and some people did get it. I conveyed the point I wanted to make, which is that the metaobject protocol uses concepts you already know well.

I encourage you to read the slides. There are notes which is basically what I spoke when I was presenting, so they should be pretty readable. Please let me know how I did. I've received very little feedback on this talk.

TAEB

One of my good friends Jesse Luehrs (doy) gave a lightning talk about our NetHack bot, TAEB.

The line "The code I'm working on isn't really a NetHack bot; it's more of a framework for writing NetHack bots" got quite a few laughs. It's completely true, too. The topic of the talk was writing a new AI for this bot framework.

The bitrotted web interface seemed to please people. It's too bad they saw how it was when I first wrote it. Since I took those screenshots, I improved it quite a bit, making it use NetHack's colors and providing an AJAXy interface. Oh well, I should really get that working again.

This lightning talk was a hit. Several people cheered! Great job, Jesse.

Others' Talks

My favorite talk, by far, was Brock Wilcox's CGI::Inspect talk. As I said during the talk, when I wrote Carp::REPL, I was actually aiming for exactly what CGI::Inspect is. And I meant it! About two years ago, it was mentioned to me how when there's an error in your Django application, it opens an AJAXy REPL for you. I figured out how to do some of that - REPL on error - but I never actually hooked it up to Jifty. Now I don't have to. Bravo, Brock!

Another talk I enjoyed was Ricardo Signes's Git is Easy talk. I knew most of the material presented, but since seeing that talk I have had a lot more confidence in using git. In particular, remotes now make sense to me. I also now use GitX to visualize where other developers are relative to me.

Scott Walter's Perl in Vegas talk was fascinating. He drew an interesting parallel between the incredible strictures of Vegas slot machines versus America's completely opaque and mostly unregulated voting machines. One strange requirement of the slot machines is that code must be compiled. Though Perl is an interpreted language, it provides a perlcc utility to "compile" the code.

Hans Dieter Pearcey's Dist::Zilla - Automating quality since 2008 reminded me of its excellent plugin-centric design:

This slide directly caused me to submit a talk about API Design to YAPC::Asia 2009.

Finally, miyagawa's Build a desktop application with Perl, HTTP::Engine, SQLite and jQuery talk was exciting. I've been working toward using his perl-app-builder for TAEB since he blogged about it. The talk showed how we can write standalone, web-based, GUI apps, in Perl, without inflicting CPAN on our users. Because Perl has been a web language ever since the web has had languages, the tools for generating HTML, serving pages, etc. are very good. The traditional GUI tools are mediocre at best. They are why my applications have been strictly web- and curses-based.

Thanks!

Thank you Robert Blackwell, Casey West, Tom Moertel, Dan Wright, and anyone else who helped! This YAPC was fantastic.

If you have never been to YAPC, you should seriously consider going to the next one!

Posted by Shawn M Moore on moose, yapc, cgi-inspect, perl, TAEB, dist-zilla at Mon, 29 Jun 2009 03:13:00 -0000 | Permalink

Moritz Lenz mentions in his excellent Perlgeek.de blog that there is not enough pictures being used alongside posts in the Perl blogosphere.

Moritz is absolutely correct so I’ll make immediate amends by reproducing the core of my last post about how Moose Roles implemented singleton methods with some helpful diagrams.

So first here is the initial Animal class code again:

use MooseX::Declare;

class Animal {
    method walk { "unknown" }
    method wild { "wild?... I'm bliming livid!" }
}

my $baz = Animal->new;

Here is a diagram which shows the object ($baz) & each class with its methods:

From this we can see that:

• $baz is the instantiated object of the Animal class.
• The Animal class contains all the methods we defined (along with “new” & “meta”) and its superclass is Moose::Object.

NB. Moose::Object is the ultimate superclass which all Moose classes relate back to. In Moose::Object we can see all the base methods.

Next is the DoesHuman role and it being applied to the $baz object:

role DoesHuman {
    method walk { "when not drunk!" }
    method legs { 2 }
}

DoesHuman->meta->apply( $baz );

And finally this is how the class relationship now looks for $baz object after the above role was applied:

The anonymous class was created when the DoesHuman role was applied to $baz object and it now stands proudly at the front of the inheritance chain for $baz object.

Thus we have added singleton method(s) to an object by using Moose roles.

This is a clear example of a picture being definitely worth a thousand words

Posted by draegtun on Programming, moose, MooseX::Declare, perl, roles, singleton methods at Sun, 28 Jun 2009 19:24:46 -0000 | Permalink

I thought about this on and off for some time but have not done anything with it yet. Maybe someone else with better ideas can do something like this.

I was sondering if it would be interesting to create a web application that on one hand would allow people using Perl to get some personalized services on the other hand it would allow the ongoing collection of some statistical data about the Perl community and about the usage of Perl.

The services that would be offered could include alert when one of the modules used by the person gets a new version. It could include a Google-alert like service when the modules are mentioned in one of the perl related sources.

It could allow the notification about Perl related events in their city/country/continent.

It could allow people to mark their favorite modules. It could allow to follow other people whose judgement they trust for modules which in turn would provide a better way to search for modules on CPAN for everyone.

It would allow an on-going survey similar to the one that was a year or two ago but that would allow people to change the data they provide and allow the site maintainers to provide information regarding the changes as well.

Posted at Sun, 28 Jun 2009 05:34:05 -0700 | Permalink

Limbic Region recently wrote a post on his use Perl journal titled "Improving search.cpan.org (Advanced Search)". The post concludes by saying that "Feel free to add your own [ideas] - I know someone who may actually be motivated to implement some of them as an alternate to search.cpan.org is reading :-)". Since I talked with L~R about CPANHQ, a seacrch.capn.org alternative in question, that I have been contributing to, I can assume meant me.

Before I reply to L~R's points, let me introduce CPANHQ. As documented in the first and only wiki page it has so far (which I wrote two days ago), "CPANHQ aims to be a community-driven, meta-data-enhanced alternative to such sites as http://search.cpan.org/ and http://kobesearch.cpan.org/. Currently, the functionality is very basic, but we have great ambitions."

I learned about CPANHQ by overhearing a conversation on Freenode's #perl channel. I joined the effort, and made several enhancements to the Catalyst and DBIx-Class-based code, which was a useful learning experience. The functionality is still very basic, and there's a lot of work to do, so any help would be appreciated. Just fork the github repository ( or my repository, which I tend to keep somewhat more up-to-date ), and play with the code.

Now back to L~R's suggestions. At first I was a bit sceptical of them and they seemed like an overkill, and not really useful. When I'm using search.cpan.org to search for a distribution, either I know the name of the module that I'm looking for and just perform a search to be directed to the location of the exact page, or I usually find the order of the search sufficient for my needs.

However, on a second reading, it seems like what Limbic suggests (like being able to sort results by Kwalitee, creation date, number of downloads, last update date, rating, etc., providing a code search, or recommendations) actually has some merit and should be considered. One thing I'm sure about is that the default CPAN search should remain simple, while only giving an option for an advances search using a different form, where it might be possible to formulate a more complex query using a specialised language. Otherwise, if we complicate the existing search, it could prove confusing to many users.

L~R's suggestions remind me of my CPAN Module-Rank thought experiment, that aimed to define an automatically calculated metric for the relevance of CPAN modules.

L~R later referred me to a post to Perlmonks.org titled "Advanced CPAN Search?" where someone asks a similar question.

To sum up, please consider monitoring or even contributing to CPANHQ if you want to improve the future of finding stuff and interacting with CPAN. It seems that some people have issue with the current CPAN search state-of-the-art, and would like to see it improved. We'd be happy to hear what you think.

Posted by shlomif@iglu.org.il (Shlomi Fish) on search, limbic region, dbix-class, cpanhq, catalyst, region, kobesearch, perl, search.cpan.org, limbic, github, cpan at Sun, 28 Jun 2009 11:58:28 -0000 | Permalink

Boy oh boy, does nested TAP make Test::Aggregate much cleaner. It's not uploaded yet -- still documenting and working out corner cases for the new Test::Aggregate::Nexted -- but here's what aggregated test output used to look like. Remember, this is five separate test programs. Can you tell where each starts and ends?

Test-Aggregate  $ prove -lv t/pre_post.t
t/pre_post.t .. # ******** running tests for aggtests/check_plan.t ********

ok 1 - aggtests/check_plan.t ***** 1
ok 2 - aggtests/check_plan.t ***** 2
ok 3 # skip checking plan (aggtests/check_plan.t ***** 3)
ok 4 - env variables should not hang around
ok 5 - aggtests/check_plan.t ***** 4
ok 6 - findbin is reinitialized for every test
ok 7 # skip Testing skip all
#     ok - aggtests/check_plan.t (1 out of 5)
# ******** running tests for aggtests/findbin.t ********
#     ok - aggtests/findbin.t (2 out of 5)
# ******** running tests for aggtests/skip_all.t ********
#     ok - aggtests/skip_all.t (3 out of 5)
# ******** running tests for aggtests/slow_load.t ********
ok 8 - slow loading module loaded
ok 9 - env variables should not hang around
ok 10 - subs work!
ok 11 - Startup should be called once
ok 12 - ... as should shutdown
ok 13 - Setup should be called once for each test program
ok 14 - ... as should teardown
1..14
#     ok - aggtests/slow_load.t (4 out of 5)
# ******** running tests for aggtests/subs.t ********
#     ok - aggtests/subs.t (5 out of 5)
ok
All tests successful.
Files=1, Tests=14,  1 wallclock secs ( 0.02 usr  0.01 sys +  0.11 cusr  0.01 csys =  0.15 CPU)
Result: PASS

Now let's run that with Test::Aggregate::Nested (requires the development version of Test::More):

Test-Aggregate  $ prove -lv t/pre_post_nested.t
t/pre_post_nested.t ..
1..5
    1..5
        1..0 # SKIP Testing skip all
    ok 1 # skip Testing skip all
        1..1
        ok 1 - findbin is reinitialized for every test
    ok 2 - aggtests/findbin.t
        1..1
        ok 1 - subs work!
    ok 3 - aggtests/subs.t
        1..2
        ok 1 - slow loading module loaded
        ok 2 - env variables should not hang around
    ok 4 - aggtests/slow_load.t
        1..5
        ok 1 - aggtests/check_plan.t ***** 1
        ok 2 - aggtests/check_plan.t ***** 2
        ok 3 # skip checking plan (aggtests/check_plan.t ***** 3)
        ok 4 - env variables should not hang around
        ok 5 - aggtests/check_plan.t ***** 4
    ok 5 - aggtests/check_plan.t
ok 1 - nested tests
ok 2 - Startup should be called once
ok 3 - ... as should shutdown
ok 4 - Setup should be called once for each test program
ok 5 - ... as should teardown
ok
All tests successful.
Files=1, Tests=5,  2 wallclock secs ( 0.03 usr  0.01 sys +  0.11 cusr  0.02 csys =  0.17 CPU)
Result: PASS

Much, much nicer. As an added bonus, plans can now be cleanly asserted. I hope to have it on github soon, and later on the CPAN.

Posted by Ovid on journal at Sun, 28 Jun 2009 09:48:55 -0000 | Permalink

Posted by Mark on vim, perl at Sun, 28 Jun 2009 05:54:00 -0000 | Permalink

Posted by Jonathan Leto on gsoc, osbridge, gsoc2009, osb09 at Sat, 27 Jun 2009 21:25:21 -0000 | Permalink

...not really. I still really like using GitHub, and (obviously) they do much more than I could churn out in a weekend, let alone an hour. Also, I used Perl.

That said, we've recently stopped using them for work code. Things just didn't work out. I hope we can migrate back to them someday, but for now we needed to make a change.

We're using gitosis, now. I've used it before, and it's a really fantastic little tool. As with most good tools, it does one job and does it very well. That means that it doesn't get in the way of the workflow you want to use.

That also means it doesn't go out of its way to make any particular workflow easy. That's fine!

I had originally thought we'd be fine having users share their repos from their home directories, as everyone has read access to every other user's homedir. This ended up being tedious and stupid, for a number of reasons, including that each user puts his code in a different subdirectory under home. (Each choice other than my own just seems so unnatural!)

When we were using GitHub, we had a program that did something like this:

• ensure that master-user/repository existed
• fork it to you/repository if needed
• clone you/repository to cwd

I updated this program to work on our gitosis setup with uniform per-user repository URLs. After all, a GitHub fork is just a clone with a tiny smidge of added metadata, near as I can tell.

The master user in our scenario is gitosis@git.example.com, and every user also has a login on that box. Here's the program:

#!/usr/bin/perl
use strict;
use warnings;

{
  package Git::Hubclone;
  use base 'App::Cmd::Simple';

  use autodie qw(:default :system);
  use Getopt::Long::Descriptive;
  use Sys::Hostname::Long;

  my $HOST = q{git.example.com};

  sub usage_desc { '%c %o <repo>' }
  sub opt_spec {
    return (
      [ 'bare-only', "do not create a local clone, only hub copy" ],
    );
  }

  sub validate_args {
    my ($self, $opt, $args) = @_;

    my $user = $ENV{USER};
    my $repo = $args->[0];

    $self->usage_error("no repo name given") unless $repo;
    $self->usage_error("do not run as root") if $user eq 'root';
    $self->usage_error("illegal repo name")  if $repo !~ /\A[-_a-z0-9]+\z/;

    if (! $opt->{bare_only}) {
      $self->usage_error("already in a git repository!")       if -d '.git';
      $self->usage_error("directory './$repo' already exists") if -d $repo;
    }
  }

  sub run {
    my ($self, $opt, $args) = @_;

    my $user = $ENV{USER};
    my $repo = $args->[0];

    if (hostname_long eq $HOST) {
      require File::HomeDir;
      require File::Path;
      require File::Spec;

      my $git_dir  = File::Spec->catfile( File::HomeDir->my_home, 'git' );
      my $repo_dir = File::Spec->catfile( $git_dir, "$repo.git" );

      unless (-d $repo_dir) {
        File::Path::mkpath($git_dir);
        system qq{git clone --bare gitosis\@$HOST:$repo.git $repo_dir};
      }
    } else {
      system "ssh $HOST git hubclone --bare-only $repo";
    }

    unless ($opt->{bare_only}) {
      my $clone_cmd = qq{git clone $user\@$HOST:git/$repo.git};
      system $clone_cmd;
    }
  }
}

Git::Hubclone->import;
Git::Hubclone->run;

I should really fix up App::Cmd::Simple to not require that import for package-in-program applications. Still, I'm pretty happy with how easy this was! Special thanks go out to Paul Fenwick for autodie, which saved me having to write loads of error checking.

Posted by rjbs at Sat, 27 Jun 2009 15:08:39 -0500 | Permalink

First off, my slides. Whenever I give a presentation, someone will ask me whether the slides will go online. Sometimes they do, and sometimes they don't. This year, I gave three presentations: one on Git, one on some new email libraries, and one on Rx. The email slides will probably go online soon. I'm not sure the other two will.

It's great to get feedback on my slides, and of course praise is nice too, and I'd like everyone who can to be able to benefit from whatever explanation I can provide, but sometimes slides just don't make sense as a PDF online. Very often, the slides are merely visual aids for the spoken content. Without narration, the slides are often less than useful, or even counter-productive.

I would like to get my Git slides online, but it will require that I first record the audio and sync it with presentation. That will require a good bit of work, and I'm not sure when I'll make the time to do it. (After all, presenting for YAPC earns me free admission and interesting questions. Presenting for SlideShare mostly just gets me notices that someone else has posted spam to the comments.)

So, on to the YAPC recap.

I arrived Saturday with plans to get some beer at the Hofbrauhaus, but it fell through. Instead, we went to Claddagh. I was irritated that I had to order four beers before they actually had any left, and that it was Smithwick's instead of something local. Still, the service was great and the food was really good. The price was right, too!

Sunday I mostly holed up in my room and worked on slides. In the evening, Dieter and I headed out to the arrival dinner. As usual, it was underwhelming, but it was still nice to see people and have a beer. (Strangely enough, the only local beer I saw at the beer-o-centric Sharp Edge was Hop Devil, which is all I drink at home.)

During the conference proper, I kept busy polishing slides until just before each presentation. I also managed to see a good number of talks, many of which were very good. Josh ben Jore had a good talk on debugging, Jim Brandt spoke about Workflow.pm, Jonathan Swartz spoke about CHI, and there were lots of others. It was also fun to see Dieter's talk on Dist::Zilla, as it was the first time anyone else gave a talk on something I wrote. (The next day, Chris N. included a lot of App::Cmd in a talk, too.)

The conference dinner at Heinz Field was really good, and speakers got a VIP tour of the stadium, including the locker rooms, control rooms, and so on. I got lots of photos and called my brother to make him jealous. He's a big Steelers fan.

It was also great to see so many people I hadn't seen in years. Sungo, especially, had been MIA since Toronto. Lots of people have changed jobs, which was fun to hear about, or lost jobs, which was not.

It was great to see Dieter again and rehash long-dormant in-jokes. (At some point someone asked, "So... I guess you guys know each other?") Sadly, we did not produce our discussed Perl Filk Concert lightning talk.

Finally, on Wednesday evening, David Golden was in town and stopped by CMU. We had some pizza and beer, and we worked on the Metabase a bit. Only a few code changes were made, but the whole thing was documented some more and is now on the CPAN. Hopefully this will help get momentum back up.

That might be all I have to say about YAPC. It was loads of fun, and I'm looking forward to next year -- but I'm definitely going to enjoy having a few months with no slides to think about, too.

Posted by rjbs at Sat, 27 Jun 2009 13:41:55 -0500 | Permalink

A while ago Greg McCarroll gave an interesting talk at the London Perl Workshop on the topic of "JFDI". His example lead to the creation of WWW::LiveDepartureBoards.

I later used this as an example topic for a Perl talk to my LUG: Achieving the Impossible with Perl. At the same time I also supplied Greg with a patch.

Since then Network Rail have started to provide a SOAP interface to their running time data, and there is now a new interface Net::NationalRail::LiveDepartureBoards. This has the advantage that it's not a screen scrape, instead it's a SOAP monster...

What I want is a trivial interface, a LiveDepartureBoards::Simple... I suppose now is the time to have a look at them both in detail and see if I can abstract over them to create a drop-dead simple interface that uses either as a back end.

Posted by Adam Trickett (ajt) at Sat, 27 Jun 2009 11:59:00 +0100 | Permalink

Perl 6 is hurtling toward completion. The specification is nearly complete, and Rakudo now passes 68% of the specification tests. Applications like November are being written in Perl 6. Perl 6 is no vaporware, and the day when Perl 6 is ready for widespread use is coming quickly.

Perl 6 has been in development since 2000, and in that time many people may have forgotten about the plans we've had for Perl 6. There may be those who have never even heard about the plans for Perl 6. Those of us who live in the Perl world are aware of the great changes afoot, but there are plenty of people who are not.

I think that the time is right to help make those people aware of Perl 6, and to remind them constantly of what's coming. My proposed technique is simple and it takes advantage of the key elements of time and repetition to help remind everyone about Perl 6.

We need to stop referring to Perl 5 as "Perl" and start calling it "Perl 5."

Specifying the "5" in "Perl 5" calls attention to the fact that there is more than one Perl. It makes the listener or reader who is unaware of Perl 6 wonder why the 5 is specified. For the reader who knows about Perl 6, hearing "Perl 5" reminds her that Perl 6 also exists.

I don't think it will be too tough. All I ask is that, at the very least, when writing about Perl 5 in your blogs or mailing lists that you specify the version you're talking about. It doesn't even need to be every instance. I'm guessing we'll find that repeatedly saying "Perl 5" in a long message will get tedious both for writer and reader.

I think the way to look at it is that "Perl 5" is the formal name for the language, and later references can refer to it as "Perl," almost like a nickname. Just that first reminder of "Perl 5" will be enough to help lodge in the reader's brain.

With enough time & repetition, it will get to be habit in our minds. With enough time & repetition, the computing world will be reminded of Perl 6 coming soon.

Posted by Andy Lester on Perl 5, Perl 6 at Fri, 26 Jun 2009 21:09:00 -0000 | Permalink

Running a standard Perl module's test suite is very easy using prove. If the module needs compilation, just run make before hand and pass the -b flag to prove, otherwise use -l to add lib to @INC.

If you pass --state=save then information about the last run will be saved in a .prove file in the current directory. This data can be used later to run only failed tests, or to reorder tests.

My testing workflow was greatly improved using this feature. I use two shell aliases to run prove:

alias ts="prove --state=slow,save -j3 -r"
alias tf="prove --state=failed,save -v -r"

I start by running ts -l t (or ts -b t if this module requires make to run before testing). The -r flag tells it search for .t files recursively in t. The -j option enables 3 concurrent jobs, and the --state flag orders tests from slowest to fastest, and causes the .prove file to be updated at the end of the run.

When I get failing tests I switch to using tf -l or tf -b. Since this has --state=failed it only runs tests that failed in the previous run. The -v causes the TAP output to be printed (with coloring).

I fix my tests or code one by one, and keep running tf. When tf has nothing left to run everything is passing, so I switch back to ts to run the full test suite again.

These two simple aliases let me run the test suite much more easily and effectively than make test, so a great many thanks to Ovid and Andy Armstrong [Update: and Andy Lester] for writing and maintaining these tools.

Posted by nothingmuch on testing, perl at Fri, 26 Jun 2009 18:35:00 -0000 | Permalink

Whether you're talking about git, Mercurial, SVK or some other distributed version control system, it's very important to realize that one of its strongest benefits is "distributed". Many people don't care about this, but it can tremendously boost productivity when combined as a part of overall development strategy.

Recently I got so fed up with our dev box that I install Ubuntu on my work computer so I could work locally. This was because our dev box was routinely running with loads around 20, and on one occasion hit a load of 57. Seems the admins thought it would be a good idea to let a bunch of other teams develop on that server but not tell us about it. I just couldn't work like that. Instead of using my computer as a dumb workstation with Windows, I now develop locally and am (usually) not dependent on our dev box being available.

That's also the reason why I have MySQL server installed locally, rather than using our test database. Sometimes that test database goes down, so I don't want to depend on that, either. And the admins thought it would be a good idea to let a bunch of other teams develop on that server but not tell us about it.

In fact, sometimes I break down. With a recent back injury, I was working from home for a week. It was slow, frustrating and painful because I wasn't really set up to work from home and this caused me to lose even more productivity. Had I been able to work from my laptop, there would have been no performance hit. Of course, even without a back injury, we had thousands of employees unable to work full hours because of a Tube strike.

Which brings me back to version control. More than once I've been unable to check in code because the subversion server has been down (never mind when the repository simply freaks out and tells me I can't commit). With something like git, I could do all of my work locally and if I can't push to the central repository, oh well.

So yeah, you know all this, but seriously, do the math. Let's be really generous and assume that our dev box, subversion server, mysql server and ability to be physically present at work all run at 99% availability. This means that your chance being "available" to work is .99^4, or about 96%. If the average availability of all of those things seems around 95%, then your "availability" drops to about 81%. Before I started working so hard to "disconnect" from the network at work, my availability was often lower than 81%. It was awful.

Now imagine sitting in a job interview and telling them that you're only going to work 81% of the time. You're not going to get the job. But the reality is, many corporate environments enforce 81% availability. This is insane, but common. And of course, we also know that those four items aren't the only things which can disrupt your productivity, but those are the four which most impact mine, so that's why I chose them.

When you're setting up a dev environment for programmers and you want it to be as productive as possible, don't force them to depend on things they can't control, such as network availability. Of course, as an employee, it's always fun when the network goes down and you see everyone wandering around with a coffee cup in hand, having been electronically muzzled. What on earth could convince people that this is a sane development environment?

Posted by Ovid on journal at Fri, 26 Jun 2009 15:27:37 -0000 | Permalink

Posted by John Napiorkowski on perl, dbix-class, dbic at Fri, 26 Jun 2009 13:19:10 -0000 | Permalink

Some things have kept me very busy lately and I’m a bit behind on my GSoC schedule. I’m starting to catch up now, though.

First of all, as of version 0.05, you can now easily install grok from the command line on most operating systems like so:

$ cpanp -i App::Grok.

It can handle Pod 5 files now (via Pod::Text) as well (which most of the Perl 6 Synopses are written in). The ANSI-colored output looks a bit different than the Pod 6 output because Pod::Text is very conservative in its use of colors. I’ll probably make them more similar later to keep it consistent.

When called interactively, grok now uses your system’s pager to view the output by default.

It will now treat an argument as a Pod 6 file to read if said argument doesn’t match a known documentation target. As for the targets, the only known ones so far are the synopses (which are bundled with grok for now), so you can do things like:

$ grok s02
$ grok s32-rules

Posted by Hinrik Örn Sigurðsson on english, gsoc, perl, perl6 at Fri, 26 Jun 2009 05:54:33 -0000 | Permalink

Explaining Metabase

At YAPC::NA, Ricardo and I finally released some early alpha Metabase modules to CPAN. Metabase is a project we started in 2008 at the Oslo QA Hackathon to provide a new transport and storage infrastructure for CPAN Testers and other CPAN distribution metadata. We’ve been working out of github until now, but think it’s time to encourage others in the Perl programming community to take a look.

One of the challenges we’ve had is just explaining Metabase to people. Here’s what I’ve come up with: Metabase is a database framework and web API to store and search opinions from anyone about anything. In the terminology of Metabase, Users store Facts about Resources. Facts are really opinions or claims; there is no inherent truth in them. Resources are logical identifiers, generally in the form of a URI, since the things we’re most interested in describing are online.

Since CPAN Testers is our first use case, I’ll use that to illustrate the concept. Currently, CPAN Testers email test reports about distributions on CPAN. In the Metabase world, each CPAN tester is a User. The Resource is a CPAN distribution file – a ‘distfile’ URI that describes a distribution tarball on CPAN such as cpan://distfile/DAGOLDEN/Capture-Tiny-0.06.tar.gz. The Fact is the test report. Today that’s just the text of the email message, but in the future it will be structured data. ¹

It’s easy to see how this can be generalized to an entire ecosystem of facts and opinions about CPAN distributions, such as CPANTS, ratings, annotations, tags, etc. Today, every CPAN website has its own database and its own API so mashups and remixes require building an interface to each. Metabase offers the possibility of a unified way for all these tools to store, share and cross-reference information about CPAN.

Design considerations

Metabase has some similarities to other, general ontology frameworks, but we decided to focus our efforts around some pragmatic tradeoffs using CPAN Testers to guide the design. We think these decisions generalize well to other uses, but we’re trying to follow a YAGNI strategy and only implement what CPAN Testers will need before worrying about more general cases.

(1) High fact volume

There are currently about 18,000 “latest” distributions on CPAN, and the entire history of CPAN is a bit over 100,000 distributions. By contrast, there are over 4 million CPAN Testers reports already and the total is growing by about 250,000 per month. Reports are usually consumed as a batch process to create summary statistics on the CPAN Testers website and relatively few reports are ever reviewed in detail. Therefore, the Metabase design to date prioritizes Fact submission over search and retrieval.

(2) Minimalist Perl for client libraries

Many CPAN Testers want to run smoke tests using a Perl installation that has as few extra modules installed as possible. Therefore, any Metabase components used by clients such as Metabase::Fact or Metabase::Client::Simple need minimal non-core dependencies. On the other hand, anything that will run on the server side can take full advantage of Modern Perl tools like Moose, DBIC and Catalyst.

(3) No barriers to contribution

Today, new CPAN Testers don’t have to register or get authorized to start contributing, they just start sending emails to a mailing list that collects reports. For Metabase, users generate a user profile fact and submit it as a credential with their facts. Metabase manages user identities within the Metabase itself – adding new users automatically – allowing new contributors to join without any external pre-authorization hurdles that might discourage participation.²

(4) Open to extension and evolution

We want to make CPAN Testers tools available for corporations or perl5-porters or anyone with customized testing needs. We want people to come up with new facts and new resources. That means defining interfaces well and leaving implementation details open to change. Metabase allows lots of choices about things like underlying data storage technologies or permissible fact types.

Metabase specifies data storage capabilities, but the actual database storage is pluggable, from flat files to relational databases to cloud services. Metabase defines how Fact classes can provide validation and custom index data on top of a very simple data model. Moreover, Metabase is a fairly dumb repository; any intelligent analysis must be done by third parties extracting and transforming data.

Where do we go from here?

I’ve successfully submitted test reports to a local Metabase, but for Metabase to become what we need for CPAN Testers (and more), there are still quite a number of things to do:

• Improve documentation
• Test and refine Metabase components
• Design and implement search capabilities
• Establish one or more ‘development server’ Metabases for testing robustness and throughput
• Migrate the 4 million CPAN Testers reports from NNTP to Metabase
• Migrate CPAN Testers downstream analytics to use Metabase as a source
• Have some CPAN Testers pilot submitting reports to Metabase
• Develop and release new CPAN Testers clients that can submit structured Metabase reports

If you are interested in following or participating in Metabase, there is a (still low-volume) mailing list you can subscribe to and several of the key developers can be found on #toolchain on irc.perl.org.

1. Technically, a CPAN Testers report will be a collection of more granular facts like prerequisites, test output, environment and so on.
2. Verification or authorization is optional

Posted by david on cpan, cpan-testers, metabase, perl at Fri, 26 Jun 2009 01:05:05 -0000 | Permalink

I like creating software.

In truth, I like solving problems. I like making annoying problems disappear, especially before people encounter them. I enjoy the realization that a little bit of work on on my part will make the world a little bit cleaner, a little bit simpler, and a little bit nicer.

Sometimes I have to write code to do that.

I have no desire to be a pumpking, however.

What Your Pumpking Does

People like Nicholas Clark and Rafael Garcia-Suarez also love to write code. They love to clean up ugly problems and replace it with elegant solutions. They love efficient code, and tidy code, and bug-free, stable code.

They're a special class of person called a pumpking. In the Perl world, the pumpking has ultimate responsibility for leading and releasing stable versions of Perl. Think of them as Larry's lieutenants.

If you love writing code and solving problems, that sounds great. You won't catch me volunteering, however, because a pumpking must also:

• Manage bugs, including triaging them and fixing the ugly ones no one else wants to fix.
• Manage patches, rejecting them when appropriate, revising them when necessary, and applying them and watching for black smoke when unfortunate. They must also do this in a timely fashion so as not to drive away volunteers.
• Manage volunteers, helping direct their energy and enthusiasm in productive ways while responding quickly enough to take advantage of their available time but not blocking the project on promises they can't quite fulfill.
• Write documentation.
• Manage test reports, identifying when and where a commit caused problems and which commit might fix them.
• Understand various portability concerns, not just of Perl code but of core code, across operating systems of expanding lineages and hardware platforms but compiler and library support.
• Uphold the implicit support and deprecation policy, even in the face of often competing goals for features and bugfixes.
• Communicate the direction of the project through development to stabilization and release and support even to handoff to another pumpking.
• Understand the long term maintainability concerns of proposed features, existing code, and development policies.
• Liase with the maintainers of core modules, especially when a release is imminent and full release stability and behavior is important.
• Keep Perl perlish.

Sometimes they get to write code.

Nothing but Praise

I've written critically about the existing Perl 5 development process here, and I expect to do so in the future. I want to be very clear about one fact, however.

I have nothing but respect for anyone who voluntarily does the job of a pumpking. It is a difficult, thankless position. It requires someone with an amazing attention to detail and a superhuman ability to manage a lot of fiddly, underappreciated work.

Past, present, and future pumpkings deserve our praise and support and thanks.

I'm critical of the process by which we develop Perl. I have no intention to criticize the people who develop Perl, and I offer my most sincere apologies and appreciation to everyone who's felt criticized or attacked or slighted.

Now how do we make the job of a pumpking easier?

Posted by chromatic on modern perl, perl 5, perl programming, software development, support at Thu, 25 Jun 2009 18:33:34 -0000 | Permalink

has foo => ( is=>'ro', ... );

event frobnize => sub { say $_[0]->foo };

Posted by jquelin@gmail.com (Jérôme Quelin) on moose, padre, perl, poe at Thu, 25 Jun 2009 16:54:00 -0000 | Permalink

Reposted from a blog entry by Mark Stosberg

Today Melody was announced as a fork of the Perl-based Movable Type platform. I helped the Melody project as it prepared to launch, in part advising on how to best to relate to the Perl community.  One of the stated interests of Melody is to refactor the project to use CGI::Application, which I maintain. Tim Appnel has already spelled out a vision of what a "CPANization" of Movable Type might look like, and I've looked in depth at what the initial steps towards using CGI::Application could be.

My own vision for Melody is a code base that's very focused on publishing and content management, with all the infrastructure outsourced to CPAN modules that are well-written, well-documented, and well-tested.  The collaboration between Melody and CPAN would be a two-way code flow. While there are more CPAN modules that Melody could make use of, there are number of pieces of Melody which should be packaged as independent modules on their own and released to CPAN.

One example is the great "dirification" that already exists in Movable Type. This is the functionality that turns any given string of words into a reasonable representation in URLs. It seems like an easy problem on the surface, but Movable Type has a sophisticated solution that takes into account what it means to do this well across many different languages. I also couldn't find any existing CPAN module which already takes on this problem space, so I started to extract this out of Movable Type myself and published a draft of String::Dirify. For that initial release, I ripped out all the fancy multi-language support, and there is still more significant work to be done to untangle this layer from from Movable Type. (If you want to pick up that project and work on it, there's also some discussion of testing String::Dirify).

While Movable Type already had an open source release, I expect Melody to have  a more adventurous evolution, and I look forward to it becoming a shining star in the Perl community, not just for the exterior functionality, but also because internals have an opportunity to become an example of best practices.

Mark Stosberg has been using programming Perl for the web for over a decade. He is the Principal Developer at Summersault and maintains several CPAN modules including Titanium and Data::FormValidator.

Posted by Andy Lester on CPAN, Community, Perl 5 at Thu, 25 Jun 2009 14:54:33 -0000 | Permalink

Without fanfare, v0.37 of Padre has been released.

The list of changes based on the Changes file:

• "Introduce Temporary Variable" refactoring tool (SMUELLER)
• Added a friendly icon on the toolbar to toggle comments (GARU)
• Crazy Win32::API AllowSetForegroundWindow hack to allow the Single Instance Server to correctly foreground itself (ADAMK)
• Added Padre::Search search and replace API (ADAMK)
• Switching to last edited file is now Ctrl-Shift-p (SZABGAB)
• Be compatible with older version of File::Path (RYAN52)
• Links to Mibbit were replaced by links to our irc.html (SZABGAB)
• Merged the code of Padre::Plugin::Encode into Padre (SZABGAB)
• Update directory only when switching to new editor (SZABGAB)

Soon it should arrive to a CPAN mirror near you.

Enjoy and thanks to all the people who put effort in Padre!

Posted at Thu, 25 Jun 2009 03:31:17 -0700 | Permalink

Because I've been distracted writing my new CPAN dependency graph generator (to repair the main flaw in the CPAN Top 100 website) I haven't been paying the website a whole lot of attention in the last week or so.

I just run the website data generator once or twice a week, and other than that, I'm focusing on creating the next iteration of the support software.

After updating the data file today, imagine my surprise when a new module absolutely roared ahead of the previous #1 highest score, posting a new record of over 500,000 FAILure points.

It would appear that the June 18th release of DBD::mysql was something of a disaster, and the high number of modules that depend on MySQL means that the impact on end users is very high.

It's good to see that when something like this happens, it is highlighted on the list very quickly and very obviously.

However, in retrospect I'm somewhat disappointed that it took this long to highlight it (because I don't run the updater often enough). What if it just took a while because it took a few weeks to gradually climb up to the top of the list, fighting against high-dependency modules with one or two rare failures, that aren't growing so fast.

I'm wonder what might be done to spot these dramatic failures faster, within a few days of their release, rather than having to wait a week or so.

Even starting to do something like that will probably require a serious improvement to the update date of the data that feeds the website, and changes to teach the analyser about the concept of time.

Food for thought, certainly.

The question now of course, is how high a really big OMGIBROKECPAN event might score. Something in the vicinity of several million failure points certainly seems possible, especially if the number of CPAN Testers installs grows.

Posted by Alias on journal at Thu, 25 Jun 2009 05:19:46 -0000 | Permalink

On #padre the other night, a discussion broke out on the subject of collaborative editing (ala MoonEdit, SubEthaEdit, etc). A number of people had never even heard of it before.

Collaborative Editing, as it is typically implemented, involves "connecting" your editor to someone else's editor. Instead of opening the files for local editing, your typing events and such are transmitted to the original host in real-time.

The effect is that two or more people can edit the same document in real-time, everyone with their own individual syntax highlighting, key bindings, preferences and plugins active.

After establishing that all the standalone libraries for doing collab (like libgobby) weren't going to be appropriate, we started pondering how we might implement it ourself, and what features OTHER than pure collaborative editing might be handy.

Some of the sorts of things you could probably do quite easily is to "steal" a copy of an open unsaved file from someone else's Padre, open a file for remote viewing only (think live remote code reviews), or allow remote users to take actions for code that can only be run on one physical system (due to exotic hardware etc).

Then there's the whole social aspect to explore. Putting your Padre into "Conference Mode" might enable all sorts of curious effects (like automatically notifying you if someone else on the local is editing a CPAN distribution that you have a checkout of as well) and let you do different things compared to collaboration with someone that is truly physically remote. Perhaps it might allow other conference attendees to hijack your editor so they can run module test suites on platforms they don't have... the possibilities for sharing development effort in a more intimate environment are quite extensive.

There's all sorts of different things that you might do, and all sorts of different methodologies you could apply to make them happen. Multicast, broadcast, UDP, things like Bonjour.

And just as useful for Padre is that doing collaborative editing would require significant improvements in the cleanliness of Padre's internals. Before we could make half this stuff work, Padre would need to have a much more robust and sophisticated understanding of the ways in which concepts align and mix together across the Main Window, Editor Panels, Documents and Projects.

Having permanently running clients and servers would drive improvements in Padre's Task API and background processing capabilities, and will require improvements in threading and IPC.

Until this notional day in the future when we actually know how to implement all of this in Padre safely and sanely, I've created a new Padre::Plugin::Swarm plugin to serve as a vessel for all the insane test cases.

"Padre Swarm" is intended to be a container for experimental work and shiny demonstrations. It will intentionally ignore stuff like robustness, scalability and security.

Swarm gives people a chance for people to show off the kinds of shiny toys that might be possible when you don't have to care that implementing it means allowing arbitrary remote code execution by the entire world, and that it only works on multicast-capable networks, and that in the process the Single Instance server doesn't work any more and Padre crashes if you try to refresh the Outline panel.

As usual, if you find this interesting or exciting, I strongly encourage you to drop in at #padre and have a chat. If you like what is happening, we can get you commit and everything else from there.

(As an aside, Padre is now sitting at around 20 active contributors. We now only need another 50% increase in contributors to catch and exceed the number of active contributors on Emacs) :)

Posted by Alias on journal at Thu, 25 Jun 2009 00:58:31 -0000 | Permalink

Note: Pictures of my trip to the Middle East are now available.
The events in this entry happened on 12-13th June

Saudi Arabian Adventures - Day 1
My first true day in Saudi Arabia was one of rest. After 30+ hours of travel, being able to sleep in a bed was truly lovely. I woke up fairly late in the day, caught up on e-mail, and then decided to go exploring. There was a food hall just around the corner from the hotel, and while I gawked at the sign displaying opening times, (I was concerned I had missed lunch), and worried about the wording that said the hall was for Saudi Aramco staff only, one of the staff members waved me in with a smile.

What I discovered is that the food hall has both very good and very affordable food. Cans of drink, bottles of water, and cups of coffee were almost universally 1 riyal (about $0.30 AUD). Some basic meals were available for about 4-5 riyals (under $2.00 AUD), with the best value for money undoubtedly being the "budget meal", which provided soup, vegetables, rice/potatoes, a generous main meal serving, and a dessert, all for 11.5 riyals (about $4 AUD).

I got myself a salad (which were excellent), some soup, and some coffee, and felt very thankful that I had changed myself some 300 riyals (about $100 AUD) while in Dubai, since I hadn't spotted any way of doing money exchange at the Dammam airport.

The rest of my travels included a walk around the camp, the most interesting part of which was a fenced area that seemed to include a library and a bowling alley. Here also a large sign proclaimed the facilities were for Aramco employees only, and passes will be checked on entry. I was getting the feeling that having an employee pass would be a good idea.

Saudi Arabian Adventures - Day 2
Today was my first day of teaching. Fuad, my contact at Aramco, had arranged to meet me at 7am at the hotel, and I had noted from my trip to the food hall yesterday that it opened for breakfast at 5am. Collectively, these were hints that Saudis were morning people. I set my alarm for 5am, which in my jet-lagged state would feel like waking up at noon in Australia.

Going to the food hall for breakfast once again had me feel like I was in America; the most popular food that people ordered were waffles! Of course, I didn't fly half-way around the world just to have waffles for breakfast, so I ordered a generous bowl of ful medames... and waffles.

Of the two breakfast foods, ful was the clear winner. It tasted good, and unlike the waffles, it had actual nutritional content. I finished breakfast with plenty of time to meet Fuad back at the hotel at 7am.

Fuad chuckled at my attempts at an Arabic greeting, and asked in perfect English "where are you learning this stuff, a phrase book?". I think that means my Arabic has some ways to go. ;)

Fuad asked about my flight here, explained that the Aramco camp was generally divided into zones, the general business hours (7:30am-3:30pm), and helped me get set up with my lab. I was also glad to hear he would be attending my first course.

At this point, I should explain a little about what I was teaching. My class consisted primarily of people with job titles that started with "Geo-", and who regularly had to deal with data that was in formats different to what they would prefer. Unsurprisingly, they'd like to use Perl to help solve these problems. That's an increasingly common story with the courses I teach.

What made this interesting is that rather than having a good four or five days (which is what I'd prefer), I was teaching two classes, a dozen students in each, with only 2.5 days per class. To top that off, I was in a different country, and I had no idea how comprehensible my Australian accent would be. I'd put together a plan that covered the basics (syntax, variables, and control structures) a tour of the CPAN, and as much regexp content as I could squeeze in.

When we were doing introductions, it became clear that my class had a good sense of my humour. My favourite introduction was from one student who said "my name is very hard to remember... it's Mohammed".

The first day of teaching went well, and from the bountiful, well-thought out questions, it was clear that I was being understood.

Posted: 25th June 2009.

Tags: adventure aramco dhahran perl saudi travel

Bookmark:

Posted at Thu, 25 Jun 2009 00:00:00 -0000 | Permalink

So YAPC::NA is over, or at least the conference proper is (there are still a few days of hacking to look forward to). It was a pretty bad YAPC for me, I was sick the whole time and didn't really see any talks and barely socialized. I do feel better now so hopefully I can still salvage some fun.

I am also a little disappointed in my own talk as well. I was hoping to be a bit more practical, but every time I tried to write slides with that in mind I got lost in endless digressions. In the end I think it was a little too shallow and ranty, not exactly what I had hoped for. It did seem to be rather well received so at least I'm happy about that.

I also don't think I presented well, but given that I feeling very bad I guess I can't complain. By the end I was having tunnel vision and my ears felt really pressurized. Someone said I sounded like I was bored with my talk, so I guess it didn't go as bad as I felt ;-)

Anyway, here's hoping for a productive hackathon, and a quick recovery.

Posted by nothingmuch on yapc, perl at Wed, 24 Jun 2009 20:17:00 -0000 | Permalink

These links are collected from the Perlbuzz Twitter feed. If you have suggestions for news bits, please mail me at andy@perlbuzz.com.

• How Perl 6 just sells itself
• Converting an A4 PDF to Letter with Margins
• Read-optimize your source code
• Optimizing life for reduced frustration
• Request for assistance: The most important toolchain bug
• Stealing from Padre for Vim
• Allison Randal in Linux Magazine: "Why Parrot is Important"
• Anyone want to adopt Getopt::Auto?
• Rakudo Perl 6 dev release #18: Pittsburgh
• DBIx::Class vs Class::DBI vs Rose::DB::Object vs Fey::ORM
• 3D graphics with Perl 6
• A gentle introduction to Moose
• "Radical Career Success in a Down Economy": A free webcast on July 1st
• Gray coding at the pediatrician's office
• What I got out of the Parrot Workshop
• Rakudo gets mainstream press attention in SD Times
• Movable Type fork is an opportunity to harness CPAN
• YAPC::NA 2010 will be in Columbus, Ohio
• Scaling out is frictionless when you use open source software

Posted by Andy Lester at Wed, 24 Jun 2009 17:11:51 -0000 | Permalink

Schwern has released Test::Simple 0.89_01. Nested TAP is now available. In case you don't recall (or haven't heard about this), you might write a subtest like this:

  use Test::More tests => 3;

  pass("First test");

  subtest 'An example subtest' => sub {
      plan tests => 2;

      pass("This is a subtest");
      pass("So is this");
  };

  pass("Third test");

And get the following nested TAP:

  1..3
  ok 1 - First test
      1..2
      ok 1 - This is a subtest
      ok 2 - So is this
  ok 2 - An example subtest
  ok 3 - Third test

(Adrian, I believe you have a branch of Test::Class which uses this? :)

Posted by Ovid on journal at Wed, 24 Jun 2009 07:50:18 -0000 | Permalink

While I contemplate on this new pale white design of use.perl.org, hoping it's either temporary or half of what it's meant to be, I wanted to share an argument I had yesterday at $work.

A co-worker presented me a problem:

34 sub scan_array {
35     my ( $item, $arrayref ) = @_;
36
37     # does the $item and $_ equal
38     if ( any { lc($item) eq lc($_) } @{$arrayref} )   ||
39     # or does $item is part of $_ but not equal to it
40        ( any { $_ =~ /$item/i } @{$arrayref} )           {
41         return 1;
42     }
43     elsif  {
44       return 1;
45     }
46
47     return 0;
48 }
syntax error at tests_include.pl line 38, near ")   ||"
Global symbol "$arrayref" requires explicit package name at tests_include.pl line 40.
syntax error at tests_include.pl line 40, near "} )

"why can't it find the arrayref ?!"

Can you spot the error? I have. The if() is missing the encompassing brackets. What actually happened is that perl is thinking that the if() statement ended with the first closing bracket, and then could not interpret the || after. Instead of running if (CONDITION || CONDITION), it ran IF (CONDITION) || CONDITION and according to the syntax of Perl, this shouldn't work, and it didn't.

After explaining to my co-worker (who is indeed a talented programmer) the simple error, he claimed that Perl is stupid because this is the way it works. Now, I should note that I don't really care when people rag on Perl since they're plainly missing something very awesome. I merely explain and move on, albeit often I ignore when I sense a large concentration of ignorance or "showoffness". However, this annoyed me. Since I have a deep interest in optimal writing, programming patterns and coherent coding styles, I found this particularly annoying, as it is simply a matter of incorrect syntax usage, clearly not a language problem.

I stated simply that in almost every language, you have brackets around the if(), and when he proclaimed that in Pascal (of all languages, Pascal? Really?) it is not like that, I simply reaffirmed that Pascal decided not to have that, good for Pascal. However, Pascal just uses the next then to understand when the if ends. That's all. Perl decided to be more pro-active about this and allow more readable code (readable? Perl? YES!). That's also why Perl has mandatory code brackets even on a single if(). Perl however, does allow you to remove both the code brackets and if condition brackets on postfix if()s.

This argument (as many other stupid geek arguments) ensued and enrolled two other developers who both suggested that what I say makes the most sense but that even if you could write it without the condition brackets, it wouldn't be as understandable, and that's an additional minus on its own.

If you want to complain about a language, try to pick better things to complain about. I could give you half a dozen things I don't like about Perl (I think...) which are waaaaay better to complain about besides the fact that "it makes me write an additional pair of brackets around the if, what the fuck?!" <-- exact quote OMG.

Posted by xsawyerx on journal at Wed, 24 Jun 2009 07:44:26 -0000 | Permalink

A patched release is on the way to the CPAN, and you are strongly urged to upgrade.

Posted by david | Permalink

I'd like to express my warm thanks to the lovely people at Yahoo and in particular to their bot-herders. Until quite recently, their web-crawling bots had most irritatingly obeyed robot exclusion rules in the robots.txt file that I have on CPANdeps. But in the last couple of weeks they've got rid of that niggling little exclusion so now they're indexing all of the CPAN's dependencies through my site! And for the benefit of their important customers, they're doing it nice and quickly - a request every few seconds instead of the pedestrian once every few minutes that gentler bots use.

Unfortunately, because generating a dependency tree takes more time than they were allowing between requests, they were filling up my process table, and all my memory, and eating all the CPU, and the only way to get back into the machine was by power-cycling it. So it is with the deepest of regrets that I have had to exclude them.

Cunts.

[update] For fuck's sake, they're doing it again from a different netblock!

Posted by david | Permalink

Posted by david | Permalink

So what have I done while ill?

On Friday I became old (see previous post), and went to the Byzantium exhibition at the Royal Academy. It was good. You should go.

Saturday was the London Perl Workshop. My talk on closures went down well, and people seemed to understand what I was talking about. Hurrah! I decided that rather than hang around nattering and going to a few talks, I'd rather hide under my duvet for the rest of the day.

I mostly hid on Sunday too, and spent most of the day asleep. In a brief moment of productivity, I got my laptop and my phone to talk to each other using magic interwebnet bluetooth stuff. I'd tried previously without success, but that was with the previous release of OS X. With version X.5 it seems to Just Work, so no Evil Hacks were necessary.

The cold means that I can't taste a damned thing, not even bacon. So now I know what it's like to be Jewish. Being Jewish sucks.

And today, I am still coughing up occasional lumps of lung and making odd bubbling noises in my chest, although my nasal demons seem to be Snotting less than they were, so hopefully I'll be back to normal tomorrow.

Posted by david | Permalink

So, the solution.

=over

#include shared/parameters

=back

and some Magic that does the cpp-stylee substitution at make dist time. Note the 'dist' section in my call to WriteMakefile.

This is, of course, crying out to be made less horribly hacky, but it works for now, so I'm happy.

My original idea was to write some crazy shit that would do the #include at install-time, when the user was installing my code. But that has the disadvantage that tools like search.cpan wouldn't show it properly, as they simply look at the files in the distribution. So this does the #includes at the last moment just before I package up the code and upload to the PAUSE. You lovely people get the right documentation in all the right places, I only have to maintain it in one place so it stays in sync, and (in the interests of Laziness) I don't have to remember to run any extra scripts before releasing, make dist just Does The Right Thing.

Posted by david | Permalink

Philippe Bruhat's talk on Net::Proxy was great - you can tell it's great because I came away with ideas for at least four things that I need to write. First up will be a plugin for it to allow the user to specify minimum and maximum permitted data rates for proxied connections. This will permit bandwidth limits for maximum permitted rates, but will also help to defeat IDSes doing traffic analysis if you specify a minimum permitted data rate.

This will protect (eg) ssh sessions from being identified based on their very bursty traffic pattern, by "filling in the blanks" with junk data.

In the evening, the CPAN-testers BOF was productive.

Posted by david | Permalink

Posted by david | Permalink

Posted by david | Permalink

Posted by david | Permalink

Posted by david | Permalink

The conference dinner at a Heuriger on the outskirts of Vienna was great. The orga-punks had hired a small fleet of buses to get us there and back, and one of the sponsors laid on a great buffet. The local wine was pretty damned fine too, and then the evening de-generated into Schnapps, with toasts to Her Majesty, to her splendid navy, and to The Village People.

It wasn't all debauchery in the evening though - on the bus, I had a very useful chat with Philippe about Net::Proxy, and re-designing it to make it easier to create new connectors for it.

Posted by david | Permalink

At the traditional end-of-YAPC auction, I avoided spending my usual stupid amounts of money on stupid things, which was nice. Obviously the hundred quid I put in to buying the hair style of next year's organisers wasn't stupid. Oh no. Definitely not.

An orange mohican will suit Domm beautifully.

Posted by david | Permalink

There were lots of other good lightning talks. Dmitri Karasik's regexes for doing OCR, Juerd Waalboer's Unicode::Semantics, and Renée Bäcker's Win32::GuiTest were especially noteworthy.

Richard Foley's brief intro to the perl debugger was also useful. Unfortunately Hakim Cassimally's talk was about debugging web applications, which I'd not noticed on the schedule, so I didn't stay for that.

And finally, Mark Fowler's grumble about why perl sucks (and what to do about it) had a few interesting little things in it. I am having vaguely sick ideas about mixing some of that up with an MJD-stylee parser.

At the auction I paid €250 to have the Danish organisers of next year's YAPC::Europe wear the Swedish flag on their foreheads. This, I should point out, was Greg's idea. I would never be so evil on my own.

Posted by david | Permalink

It asked me to ask you all to keep the volume down.

Posted by david | Permalink

The first two legs of that are second class, cos first wasn't available on Eurostar (being a Friday evening it's one of the commuter Eurostars and gets booked up months and months in advance) and was way too spendy on the sleeper to Munich. Upgrading to first class from Munich to Vienna is cheap, so I have.

Coming back it's first class all the way cos upgrading was nearly free ...

Don't even think about trying to book online or over the phone, or at the Eurostar ticket office at Waterloo. Your best bet is to go to the Rail Europe shop on Picadilly, opposite the Royal Academy and next to Fortnums.

Posted by david | Permalink

It strips away all the useless rubbish that normally surrounds Wikipedia pages, as well as things like the editing functions which are also hard to use on portable devices. Internally, it's implemented using perl, LWP, and mod_perl, and is hosted by Keyweb.de.

Posted by david | Permalink

As expected from the latest trend in the Perl blogosphere this post will be about Roles. And Moose! And Roles in Moose! And Moose in Roles! And Roles outside Moose…

Seriously now, this is a post about a completely non-Moosey and non-Roley script I wrote to filter the use.perl.org master journals' feed. What this script does is fetch the Atom feed of all the use.perl.org journals' posts, and filters out the entries of the use.perl.org authors that the invoker specified.

Here is out to use it. First of all: svn checkout it (or otherwise fetch it using HTTP). Then you can simply use:

perl filter-use-perl-journals.pl -o everything.atom

Then you can serve everything.atom with a web-server to read it using your web feeds' aggregator.

To simply create a non-filtered copy of the feed. Now let's say you want to get rid of posts from my journal (because it sucks). In that case, say:

perl filter-use-perl-journals.pl -o non-shlomif.atom \
    --blacklist="Shlomi Fish"

Now, let's say you also want to get rid of Ovid's posts. I have no idea why you'd want to do that, because his journal is great, but it's just for the sake of the example. In that case, do:

perl filter-use-perl-journals.pl -o non-shlomif-and-ovid.atom \
    --blacklist="Shlomi Fish" --blacklist="Ovid"

Finally, there's the --rand flag which is useful in case you're running the script with cron. What it does is wait for a period of time of random length, before fetching the feed, so the HTTP server would not be overloaded at regular periods. This requires a working /dev/urandom for the time being.

The script is made available under the MIT/X11 Licence so its use is completely unencumbered. I wrote this script today, because I have a personal use for it, but other people may find it useful too. It requires a recent version of Perl (5.8.x should be enough I think) and XML-Feed.

Posted by shlomif@iglu.org.il (Shlomi Fish) | Permalink