The Copenhagen Perl 6 Hackathon and the Open Source Days conference is over.

I am left filled with magnificent impressions and at the same time suffering from a mild case of information overload.

Due to a collision with the Dutch Perl Workshop the Copenhagen Perl 6 Hackathon was scheduled to start Saturday and continue Sunday, where the Open Source Days conference started friday. In addition we had arranged for the invited Perl 6 people to have some time on their own to discuss and hack face to face, something that have shown beneficial previously for other dispersed project groups.

We started the Saturday with a line of Perl 6 presentations – there was a good turn up to the talks, also by people who I have not seen at Copenhagen Perl Mongers meetings.

I have received little, but very positive feedback on the event. One of the more interesting pieces of feedback was this tweet (http://twitter.com/Jippi/status/10071258756), which really pin-pointed that we are doing the right thing and having fun at the same time.

Perl 6 is different and I think that it is important to emphasize the difference, not just different from Perl 5, but different in many aspects and from other languages. I can only say that I think that Perl 6 will be a very modern language and as Perl 5 programmer I will need to rethink the ways I am writing Perl.

I overheard a discussion over some Perl 6 construct and one of the comments was that it was not a very Perl 6-ish way of doing things – Perl 5 is not Perl 6. Personally I am very keen on getting to use Perl 6, but I have slowly started to understand that it will be significant change. Perl 6 will address many short-comings in Perl 5 naturally, the primary language designer is the same person, but in general Perl 6 is a new language and it will address short-comings in many languages. Yes it will also have it’s own, but then we then we just address that in Perl 7 – implemented in Perl 6 magnificent isn’t it…

My experience with this kind of events is that it takes some time before the actual result show. I hope that we will be able to attract some new people to the Copenhagen Perl Mongers – and I hope that the Perl 6 community have benefitted from the opportunity to present Perl 6 at a large open source non-perl conference.

The aftermath of the hackathon is hopefully going to be more positive than the aftermath of the Open Source Days conference. One of the exhibitors made a small happening at the conference involving body painted girls. The IT business in Denmark has for a long time been working hard on attracting women to a business, that is always short on clever people of both genders. Based on the blog entries and comments I have seen the conclusion must be that the happening was a bad idea – a really bad idea and I can only concur.

Implementation wise I hope we have been able to contribute over the weekend and that we have earned the Copenhagen release – I am working on my first larger Perl 6 application and it is fun. I am constantly refactoring the code to be more Perl 6 and training my brain to start using Perl 6 patterns.

Thanks to all the invited Perl 6 people, attendees and the coord of Open Source Days for giving us the opportunity to show of Perl 6.

jonasbn, logicLAB

Posted by jonasbn on Uncategorized, copenhagen, cph.pm, hackathon, opensource, opensourcedays, perl5, perl6 at Tue, 09 Mar 2010 20:24:52 -0000 | Permalink

Posted by jquelin@gmail.com (Jérôme Quelin) on moose, curses, tests, perl at Tue, 09 Mar 2010 16:57:00 -0000 | Permalink

These links are collected from the Perlbuzz Twitter feed. If you have suggestions for news bits, please mail me at andy@perlbuzz.com.

• What's going on with San Francisco Perl Mongers? (blogs.perl.org)
• Contributing is easy: A first-timer talks about adding to CPAN (blogs.perl.org)
• Help keep the world safe from SQL injection: (perlbuzz.com)
• How to learn how to get help in Perl (use.perl.org)
• The Perl Foundation wants your input on grant proposals for 2010 (news.perlfoundation.org)
• Why Ruby is prettier and Padre changes the Perl community (use.perl.org)
• Manage your Perl modules with git (effectiveperlprogramming.com)
• Call for papers for YAPC::NA 2010 in Columbus. Theme: Modern Perl 5 (news.perlfoundation.org)
• Famous Perl One-Liners Explained part 5: Text conversion and substitution (catonmat.net)
• The Italians like ack, too (cattlegrid.info)
• Bash completion for perldoc (blogs.perl.org)
• cpanminus, the new CPAN shell superstar (marcus.nordaaker.com)
• RT @chromatic_x Kudos to *many* active #perl 5 porters for the successful 5.11 release series: (ur1.ca)
• Testing a DB-intensive app (blog.urth.org)
• More news on the TPF wiki overhaul (use.perl.org)
• Do you still program in Perl? (reddit.com)
• Benchmarking is for losers, Profiling rulez! (blog.urth.org)
• Damian Conway's been posting a series of columns on vim: (blogs.perl.org)
• Testing and looging with Dist::Zilla (rjbs.manxome.org)
• Safe.pm 2.25 fixes security hole (blogs.perl.org)
• Serving production web requests with the Catalyst development server (blog.afoolishmanifesto.com)

Posted by Andy Lester on CPAN, Code craft, Perl 5 at Tue, 09 Mar 2010 16:46:42 -0000 | Permalink

OK, you know how it works. Everybody who's really comfortable with testing often offers the following advice:

• Use Devel::Cover
• Don't strive for 100% coverage

This advice is absolutely correct, but misleading. It leaves you with an obvious question: what coverage do I want? This is where the danger lies.

While hacking on my Veure project, I find myself often digging into code that I don't understand terribly well. This also means I don't understand my interface. As a result, I often implement something I think will work, hit my browser and view the results. Which are wrong. Again. So I tinker some more. Again.

Lather, rinse, repeat. The benefit of this strategy is that I tend to add new functionality much faster. The drawback is that once something is working, it's easy to be careless with testing. And here's what I've found:

If it's not covered, it's broken. Every. Single. Time.

I admit that anecdotes are not data and maybe I'm just a bad programmer, but my code is getting complex enough that there are parts of it which look perfectly fine, but are nonetheless wrong. I recently discovered a URL hack exploit because when I was trying to increase my code coverage, I focused on a few modules with coverage around 98% or so and tried to get them to 100%. Three modules in a row with just a couple of untested lines had bugs in those lines.

You don't want to test that things already tested work (don't test getters/setters in Moose unless you're hacking on Moose directly), but you had better make sure that your code is setting them appropriately.

So don't strive for 100%, but be very, very careful about the seductive thought of "I have 95% coverage". If your project is sufficiently complex, that last 5% will be a minefield.

Posted by Ovid on tests at Tue, 09 Mar 2010 14:00:08 -0000 | Permalink

Earlier this year Renée Bäcker from $foo magazine interviewed Richard Dice and me to find out our thoughts on TPF in 2009 and our hopes going forward. An English version [pdf] of the interview is available online.

Posted by Karen on Perl Foundation at Tue, 09 Mar 2010 08:51:24 -0000 | Permalink

Posted by Jonathan Leto on gsoc, parrot, perl, gsoc2010 at Tue, 09 Mar 2010 06:47:41 -0000 | Permalink

I feel like I'm making enough progress, right now, to make a grant update every day. I don't know how long this will last, but I'm glad that it's going so well, so far.

I got a lot of preliminary improvements done to logging, mostly described in yeserday's post. They scrunched the default output way down to fit on the screen and be readable. Now that I had logging working, it seemed like time to bite the bullet and really get to work on testing.

I made some improvements to App::Cmd::Tester, trying to capture the exit code of applications that call exit. I'm not entirely happy with how it works -- it interferes with Test::Builder sometimes, it seems. Still, it's better than nothing. I made a few other changes, too, making the App::Cmd::Builder easier to extend and providing more information back in the results. For example, you can now say:

my $results    = test_app(AppClass => \@my_argv);
my $app_object = $results->app;

This is really useful when the app object is a Dist::Zilla::App, because you can do things like:

my $app_object = $results->app;
my $zilla      = $app_object->zilla;
my $build_dir  = $zilla->built_in;

Obviously, this makes testing Dist::Zilla a lot easier.

I've also written a subclass of Dist::Zilla just for testing, which will copy the source material to a temporary directory and then build to another part of that temporary directory. Then the programmer can inspect the files written to disk, the state of the Dist::Zilla object, and more. I also hope to write a simple way to ask, "Did anything alter the source tree?" (That will often constitute a bug, but not always.)

Unfortunately, I've exposed one bug and one (recent) design flaw. The bug is something I've basically known to be there all along, and hasn't mattered at all for normal use. In short, you can't build a dist unless the dist root (the place with the dist.ini file) is your current working directory. If you try, things break, because too many things work with relative paths and never properly make them absolute. This, I hope, will not be very hard to fix. Fortunately, I can work around it with judicious use of File::chdir for now.

The more annoying problem is that the logger design is a problem. I still like Log::Dispatchouli and plan to keep using it as the default logger, but the way the existing logging framework works is just too leaky. It acts like the logger can be replaced, but it clearly can't, and even doing things like logging everything to a test object isn't easy enough. I have a few possible solutions to this, ranging from the brutal-but-easy to the complicated-but-maybe-more-useful.

I think I'll start with the latter, assuming that I won't actually need the added complexity. At any rate, I think it's extremely likely that before the end of the week I will have a nice simple way to test Dist::Zilla's behavior in automated tests without copying and pasting a lot of ugly code. This is really exciting, because it means I can start writing tests for Dist::Zilla. It's also really scary, because I'm pretty sure that writing tests for Dist::Zilla is going to be a big drag. That's okay, though, because it's what I'm getting paid to do.

Finally, I thought I'd note one other really exciting thing. Dist::Zilla has a lot of dependencies, mostly because I've tried to factor out any code that can be useful elsewhere, but also because I've tried to use existing tools when they exist. Now that I'm trying to get these improvements made to Dist::Zilla, I'm finding that a lot of them need to go in external libraries. That means that this grant to improve Dist::Zilla is also going to improve App::Cmd, Log::Dispatchouli, Pod::Weaver, and probably a bunch of other libraries. That makes me pretty happy about the decisions I've made so far.

Posted by rjbs at Mon, 08 Mar 2010 10:02:32 -0500 | Permalink

I am happy to announce that the Haifa Perl mongers are going to have a meeting on the 15th March in the offices of Qualcomm in Matam, Haifa organized by Shmuel Fomberg. On the agenda is Erez Schatz How to Talk to Newbies and Yaron Meiry (aka Sawyer) Moose - A postmodern metaclass-based object system for Perl 5 The meeting will start at 18:30.

For more details please see the announcement and/or contact Shmuel.

The regular Rehovot Perl Monger meeting is going to take place on 16th March in the Weizmann Institute. Yaron Meiry (aka Sawyer) is going to give a talk about Moose - A postmodern metaclass-based object system for Perl 5

For more details please see the web site of the Rehovot Perl Mongers.

Posted at Mon, 08 Mar 2010 13:39:33 -0000 | Permalink

Lately I've been learning ExtJS (which I might write about separately sometime) to try and create a UI for a small application I'm writing for a friend.

The application uses Dancer and KiokuDB. They're both easy to work with and I spent little time working on the backend. The problem lies in the frontend, where I need some UI. I've chosen ExtJS because it has built-in widgets.

I've played with tutorials, general documentation and (tried to) read the not-as-friendly-as-I-thought API. I was able to create a grid, a form and even a viewport. However, it kept feeling like it's stuck. Not stuck as in "this is going slow", but stuck as in "not moving at all".

I don't know how to measure the progress of UI development. Usually, with backend development (which is mostly what I do), I can see the project moving (even if slowly), but here I don't. It reminds me of the stage in your code when you're still guessing yourself on the way you planned out your program, on your algorithms, on your assumptions and supposed requirements. You keep going back to the drawing board, unsure of your project.

Last night I sat once again to try and sketch things out. I decided this time to sketch the usability of the UI instead of the appearance. The difference for me means I think of what would happen instead of how it would look like. I started sketching arrows and write small comments (double click, single click, tab). Things got better, I finally felt like I know my goals, at least the ones in the beginning.

I sat down, wrote a viewport, added a tab panel, added a grid on the first panel and was even able to open and move to a new tab on double click of a record. Pretty cool. At first I was excited about this, but then dejected. Was this progress? How would I know if I made progress just now, or got stuck on something irrelevant?

I assume working on the parts that I already know by heart how to do would be a waste, which is the backend. I should focus on the frontend, that is clear. But should I focus on the panels? On the tabs? On fetching the data? Should I be picking at the grid? Should I be picking at the functionality? What about the layout? Should I be learning how to do everything in ExtJS first? Maybe strengthen a bit on the Javascript side?

I'm feeling quite confused, something that although happens often in life, doesn't occur to me often on the technical side. There are so many things to pick from and I don't know which one is correct. I reckon that is why I keep feeling like I'm making no progress. Maybe I should sketch down all the possible components and then try to assemble a puzzle out of them and then write the functionality. I don't know.

Does anyone have any advice on this?

Posted by Sawyer X on Design, Rant, api, extjs, javascript, rant at Mon, 08 Mar 2010 12:58:14 -0000 | Permalink

Last week we went live with a revised EDI process we have with one customer. The original process has been running since I started my job - how to implement it was an interview question - but it's been running on a Windows box. Our IT department wanted to decommission the box so I took the opportunity to port the application from Windows to Linux.

It's still a Perl application but the new one has better logging and configuration. It's been live for a few days and so far it's working perfectly well. It'll be a real shame when it's finally replaced with a vast SAP PI middleware framework, but in the mean time an awful lot of money has flowed through that simple Perl application!

Posted by Adam Trickett (ajt) at Mon, 08 Mar 2010 10:18:00 -0000 | Permalink

Major release of Dancer finally out. You can read about it on Alexis' blog.

Basically this version provides a lot of improvements to Dancer:

• Alexis Sukrieh worked hard on refactoring and optimizing.


• David Precious did a lot of awesome work on documentation: we now have a cookbook and a deployment manual for various situations you might encounter. He also added a session backend with clean recursive dumps.


• I added the route caching, which shows a significant speed improvement!


• We have a mailing list


• We have a new main domain (perldancer.org) and it's the first project to use a perlprojects.net subdomain: dancer.perlprojects.net.

I want to thank Sukriah and David for all their work and being great guys to work with.

Bust a move!

Posted by Sawyer X on Code, Perl, Dancer at Mon, 08 Mar 2010 09:32:42 -0000 | Permalink

Posted by The Perl Hacker Painter at Mon, 08 Mar 2010 05:28:00 -0000 | Permalink

It’s never great to post an “end-February” report a week into March, but that’s how things are going lately. I’ve been busy with family and work obligations that have meant less CT2.0 hacking. I’m sorry this is coming late, but I hope I will give anyone interested a sense of where things stand.

I should note that the original deadline for finishing CT2.0 was March 1 and clearly we’re not there yet. I’ve discussed the situation with Robert and Ask at the Perl NOC, and they’ve been willing to extend the deadline for cutting off CT1.0 for a while longer. Thank you, Robert and Ask, for your understanding!

Progress in the last couple weeks:

• I did some alpha testing of the CT2.0 Metabase hosted on Amazon. Based on that, I revised yet again (sigh) the user registration/credentials approach to minimize the hassle for old and new registrants.
• Florian Ragwitz wrote a Catalyst app to help distribute new credentials files to legacy CT1.0 users. I haven’t deployed it yet (since I now need to regenerate all the credentials), but greatly appreciate his quick turnaround.
• I implemented the Metabase search capabilities that Barbie will need to update the CPAN Testers statistics database. This will be based on the excellent SQL::Abstract approach to WHERE clause construction.
• I wrote several helper modules to simplify configuration of a CPAN Testers metabase in preparation for deployment. The first of these has already been uploaded to CPAN: Net::Amazon::Config
• I finalized the “version 0″ API for the Metabase web service and revised the interface between the Catalyst app and the Metabase to reflect the latest changes to the library.

In the last several weeks, members the #catalyst and #moose IRC channels were very, very helpful and patiently answered my many stupid questions. Thank you in particular to confound, perigrin, rafl, rjbs, stevan and t0m.

Coming up:

• Deploy all my new code onto the server for “beta” testing
• Release all the code to CPAN that people will need to configure their clients for beta testing
• Regenerate user profiles and deploy rafl’s app to distribute them to legacy users
• Whip the server into production shape (e.g. proper boot scripts to auto-start the CT2.0 apps on restart)
• Get back to work on legacy report migration

It’s at the point now where I suspect the “hard thinking” part is pretty much done and it’s a lot of grotty but straightforward tasks to go. Hopefully, beta testing won’t reveal any major issues and the end of March update will be focused on planning on orderly transition from CT1.0 to CT2.0.

Posted by dagolden on cpan-testers, metabase, perl-programming, ironman at Mon, 08 Mar 2010 04:59:04 -0000 | Permalink

Latest Web::Simple master (git://git.shadowcat.co.uk/catagits/Web-Simple.git) is now using Plack, the perl web server directly for CGI and FastCGI and I hope do do another release shortly with notes on deploying as both on shared hosts (more specifically Dreamhost but with an invitation to bitch if they don’t work on other budget hosts).

mst,  ”Oh Subdispatch, Oh Subdispatch“

Every time someone mentions a cool new web framework like Web::Simple, Dancer or Tatsumaki, my immediate reaction is to start thinking up cool little niche web apps I could build with it. You know, the kind you imagine yourself whipping up during your lunch break, that will probably never get more than 10 curious users but just might turn into the next big thing if only you got it up and running on a public server. And that’s when the second thought immediately arrives: where am I going to run this thing? Do I really think the idea has enough legs to justify paying for a virtual server plan? Can I be bothered going through the pain of figuring out how to deploy it on a shared hosting provider? And normally that’s the point where I sigh wistfully and go back to reading my RSS feeds.

The thing is, with the advent of things like local::lib it’s getting a lot easier to deploy Perl web apps on shared hosting. And with most web frameworks adopting Plack/PSGI, the work required to deploy Perl web apps on budget hosts is converging into a similar sequence of steps.

So this time during my lunch break I started envisaging a centralised, SEO-friendly information source (hello perlsharedhosting.com) that cobbles together all of the currently available information into an easy to digest form to make it ridiculously easy to choose a shared hosting provider, deploy your web app and troubleshoot common problems.

A site that works like this:

• The front page contains a list of Perl-friendly hosting providers, with a meta-score based on how many features they support (+) and how many unresolved issues they have (-), and maybe user review scores thrown into the mix too
• Each hosting provider has a page of its own, listing in full which features are supported (used to compute the meta-score). Users can post comments on each of these pages, to leave testimonials (“I am currently running a Web::Simple site that gets x hits per hour on this host”) and note unresolved issues (“module X::Y fails to install on this host”). As issues get solved these become tips.
• Each technology/feature/technique also gets a page of its own, with links to the official man pages, shared-hosting specific notes and again user comments
  • dependency related: local::lib, cpan, cpanm, ..
  • environment related: locating and running perl, paths, daemons, viewing error output, ..
  • web deployment related: CGI, FastCGI, ..
  • framework / web app related: Web::Simple, Dancer, MojoMojo, Catalyst, Tatsumaki, ..

I haven’t decided yet what engine would fit best.. MojoMojo is a front-runner, and in fact the Catalyst Friendly Hosting page runs on MojoMojo does a large chunk of what I’ve described above.

The site would be very “cookbook” oriented, since we’re specifically targeting people who can’t be bothered learning the ins and outs of 10 different technologies (not to mention figuring out how to get them to play nicely together) for the sake of deploying toy web applications on cheap shared hosting. Just the essentials: this is what you need, this is how you achieve it. And if you want to learn more, go here.

With that in place, we’d end up with a single, visible, place to research and document the complications of running Perl web apps on shared hosting. And if the site became popular, hosting providers might even take notice and start offering more Perl-friendly shared environments. We could have live demo pages running on different hosts, possibly even donated by hosting providers if they see it as a way of showcasing their Perl-friendliness (mojomojo.dreamhost.perlsharedhosting.com, dancer.resellerzoom.perlsharedhosting.com, etc..).

I got as far as registering the domain name; I was planning on getting a simple MojoMojo prototype up and running, but I got side-tracked researching how to deploy it on my shared hosting account…

Posted by Patrick on Comp, Perl at Mon, 08 Mar 2010 01:23:45 -0000 | Permalink

In Item 110, “Compile and install your own perls”, we showed you how to compile and install several versions of perl so that they don’t conflict with each other and you can use them simultaneously. Since they don’t install their programs, they are left in their $prefix/bin directories. With several perls, each of which has their own modules directories, using tools such as cpan and perldoc can get confusing. Which version of those tools are you using and which perl are they trying to use?

When you compile and install you perl, it adjusts the shebang line of the tools to point to the right perl. If your prefix is /usr/local/perls/perl-5.10.1, the you have a bunch of Perl tools in /usr/local/perl/perl-5.10.1/bin. They each have their right shebang:

% cd /usr/local/perls/perl-5.10.1/bin
% head -1 *
==> c2ph <==
#!/usr/local/perls/perl-5.10.1/bin/perl

==> config_data <==
#!/usr/local/perls/perl-5.10.1/bin/perl 

==> corelist <==
#!/usr/local/perls/perl-5.10.1/bin/perl

==> cpan <==
#!/usr/local/perls/perl-5.10.1/bin/perl

==> cpan2dist <==
#!/usr/local/perls/perl-5.10.1/bin/perl

==> cpanp <==
#!/usr/local/perls/perl-5.10.1/bin/perl

==> cpanp-run-perl <==
#!/usr/local/perls/perl-5.10.1/bin/perl

...and so on and so on...

Anywhere you copy those programs, they'll find the right version of Perl because it's built into the source. Where to put them though? You don't want a zillion paths in your environment, and you don't want to type the long paths every time you want to use a tool.

Everyone is probably going to have their preferred version of Perl. This project wants to use Perl 5.10 but another wants to use Perl 5.8. Still another project works on Perl 5.8.8, but its coders wonder if they can upgrade to Perl 5.8.9 for some minor bug fixes.

The easiest thing to maintain might be to make symlinks in your personal bin/ directory:

% ln -s /usr/local/perls/perl-5.10.1/bin/perl5.10.1 ~/bin/perl5.10.1
% ln -s /usr/local/perls/perl-5.10.1/bin/cpan ~/bin/cpan5.10.1
% ln -s /usr/local/perls/perl-5.10.1/bin/perldoc ~/bin/perldoc5.10.1

Now, when you want to install a module for your Perl 5.10.1 installation, you use the right cpan:

% cpan5.10.1 LWP::Simple Net::Twitter

When you want to read the docs for Perl 5.10.1, you use the right perldoc:

% perldoc5.10.1 perlsyn

If you want to make one version your default perl, you can make the right links for that:

% ln -s /usr/local/perls/perl-5.8.9/bin/perl5.8.9 ~/bin/perl
% ln -s /usr/local/perls/perl-5.8.9/bin/cpan ~/bin/cpan
% ln -s /usr/local/perls/perl-5.8.9/bin/perldoc ~/bin/perldoc

You don't have to live with that though. Although I'll leave it as an exercise for the reader, you can create a script to change the default version by switching the links for ~/bin/perl and so on to the new default. You don't have to do that with just perl though. With Furlani modules you can do it with your entire environment (but that's another story).

Now, all of that linking is quite tedious if you need to do it for several perls, so you might want to adapt the script that I use. I install perls as /usr/local/perls/perl-5.minor.point and usually want to make the links in ~/bin.

#!perl

use 5.010;

use strict;
use warnings;

use File::Basename;
use File::Spec::Functions;

my $perls_directory = catfile(
	$ARGV[0] // '/usr/local/perls',
	'perl*'
	);
die "$perls_directory does not exist!\n"
	unless -d dirname $perls_directory;

my $links_directory = $ARGV[1] // catfile( $ENV{HOME}, 'bin' ); #/
die "$links_directory does not exist!\n" unless -d $links_directory;

foreach my $directory ( glob( $perls_directory ) )
	{
	say "Processing $directory...";

	unless( -e catfile( $directory, 'bin' ) )
		{
		say "\tNo bin/ directory. Skipping!";
		next;
		}

	my @perls = glob( catfile( $directory, qw( bin perl5* ) ) );	

	my( $perl_version ) = $perls[0] =~ m/(5\.\d+\.\d+)\z/;
	say "\tperl version is $perl_version";

	foreach my $bin ( glob( catfile( $directory, 'bin', '*' ) ) )
		{
		say "\tFound $bin";
		my $basename = basename( $bin );

		my $link_basename = do {
			if( $basename =~ m/5\.\d+\.\d+\z/) { $basename }
			else                               { "$basename$perl_version" }
			};

		my $link = catfile( $links_directory, $link_basename );
		next if -e $link;
		say "\t\tlinking $bin => $link";
		symlink $bin => $link or
			warn "\t\tCould not create symlink [$!]: $bin => $link!";
		}
	}

Posted by brian d foy on Uncategorized at Sun, 07 Mar 2010 18:06:04 -0000 | Permalink

In case you were wondering what Damian been's doing lately (and don't you dare say "Damian Who?" -- unless your name is Damian), he's been writing a series of articles for IBM Developer Works about the Vim editor.

• Scripting the Vim editor, Part 1: Variables, values, and expressions
• Scripting the Vim editor, Part 2: User-defined functions
• Scripting the Vim editor, Part 3: Built-in lists
• Scripting the Vim editor, Part 4: Dictionaries
• Scripting the Vim editor, Part 5: Event-driven scripting and automation

On an unrelated note, one of my brothers is a psychiatric nurse. He often has to deal with patients in the Conway Ward. Coincidence?

Posted by Ovid on vim at Sun, 07 Mar 2010 16:55:41 -0000 | Permalink

I've been continuing to work on adding improved logging capabilities to Dist::Zilla. They're not going to be terribly complex, but I'm trying to make sure that they're easy to use, and that they lend themselves to sane default output and useful debugging output. Yesterday, I realized that these lines were really annoying:

[DZ] Dist::Zilla::PluginBundle::RJBS/Dist::Zilla::PluginBundle::Git/Dist::Zilla::Plugin::Git::Push initialized

I went through and fixed all the bundles I could to produce simpler (but still useful) names and then used each plugin's logger for logging initialization. The result is now more like this:

[@RJBS/@Git/Push] online, Dist::Zilla::Plugin::Git::Push v1.234567

That's also been demoted to debug output, so there's much less noise by default. Releasing Dist::Zilla this morning looked like this:

~/code/cs/Dist-Zilla$ dzil release
reading configuration using Dist::Zilla::Config::Finder
[DZ] beginning to build Dist-Zilla
[DZ] guessing dist's main_module is lib/Dist/Zilla.pm
[DZ] extracting distribution abstract from lib/Dist/Zilla.pm
[@RJBS/Classic/ExtraTests] rewriting release test xt/release/pod-coverage.t
[@RJBS/Classic/ExtraTests] rewriting release test xt/release/pod-syntax.t
[DZ] writing Dist-Zilla in Dist-Zilla-1.100660
[DZ] writing archive to Dist-Zilla-1.100660.tar.gz
[@RJBS/@Git/Check] branch master is in a clean state
[@RJBS/Classic/UploadToCPAN] registering upload with PAUSE web server
[@RJBS/Classic/UploadToCPAN] POSTing upload for Dist-Zilla-1.100660.tar.gz
[@RJBS/Classic/UploadToCPAN] PAUSE add message sent ok [200]

Not bad!

I also made some improvements to Pod::Weaver and the PodWeaver plugin so that it can log its work as it goes. This will produce output like:

[@RJBS/PodWeaver] [Name] added name/abstract section to lib/Dist/Zilla.pm

I'm starting to wonder if Dist::Zilla isn't the project where I'll end up wanting more fine-grained control than "normal logging" or "debugging logging," but I think things will go as this issue always has: I'll put it off until everything else is done and then realize I don't care about it.

Posted by rjbs at Sun, 07 Mar 2010 09:22:12 -0500 | Permalink

1. A 32-bit 5.10.1.2.


2. A 32-bit 5.11.6.0.


3. A 64-bit 5.11.6.0.


4. A 32-bit 5.10.1.2 with the Professional additions may or may not happen.

Posted on strawberry perl at Sun, 07 Mar 2010 07:21:28 -0000 | Permalink

CeBIT was way more successful than I expected. Thanks to Renee Baecker, the main organizer and the other people who were at the stand during the week we made a lot of contacts with individual developers and companies using Perl. We also talked to quite a number of people who have heard about Perl but never tried it and some people who never heard about it.

It was both very hard to be there standing 8-9 hours and almost constantly talking to people every day but it was also a lot of fun.

Originally we were requested to always have two people at the stand but it turned out that there were times when even 7 of the Perl::Staff members could not handle the number of visitors at the same time. We were lucky as our stand was part of the Open Source Lounge of 15 projects slightly out of the main stream of people. On one hand that meant many people walking along the main alleys missed it but on the other hand those who came by had more space and time to talk to us.

It was very heart warming to see people coming to the booth with no or even negative view on Perl and leaving with a feeling that they saw some really cool things they want to try now.

So what were we showing?

To people with now real Perl knowledge or people who used Perl 10 years ago we showed some modern Perl code such as Moose, MooseX::Declare and some related stuff. We showed them Padre so they can see you can write large desktop applications in Perl. We were also showing Catalyst, especially to PHP programmers but also to others who were web developers.

On one of the days there were two Foswiki developers and they showed their wiki to the people interested in that. I think OTRS got a bit less attention on our stand but they had their own booth very they made a lot of business contacts.

The strategy was simple. Some of us were standing in the alley and as people walked by we gave them a flyer and then started to talk to them. Obviously there were many who were not interested, we just let them go. I was the only presenter who did not speak German my strategy was to first ask them if they speak English. With many people it was clear that they don't know English well enough for a conversation or just prefer German. That gave me a chance to direct them closer to the booth to talk to "one of our other representative, who speaks German". So in fact I was acting a lot as the "catch man". I think it worked out quite well. When they were OK speaking English or even preferred that language then I kept the conversation going on. I asked what are they using Perl for, or if they did not know or have not used Perl I asked what other languages are they using and what kind of things are they using. That usually gave me an opportunity to ask if they would like to see some modern technologies in Perl. Most of them were interested so we stepped to one of the computers we set up (we had 2 sometimes 3 computers on our booth, which was high enough so we did not have to bend over to view the screens or type). There I showed them Padre, some code in Padre. Sometimes I showed a few pages from the Moose documentation and some small examples in Catalyst.

There were several people who were interested in Perl 6. For them I opened the web-page of my Perl 6 training slides and showed a few pages just to get them impressed by some of the nice features of the language.

Others, as I saw started by asking if the visitor "Knew Perl?" and from that point some kind of a conversation evolved in German that I could not follow anyway but that will be described by the other presenters. (Renee and Sewi have already written. All the reflections will be linked from the TPF wiki page related to Perl on CeBIT)

Some Improvements

There were a lot of things we could improve in our presentation. First of all none of us knew all the things we wanted to show and we did not have ready made pages to show them. So I'd like to have several slide-shows that we can load on all the computers we are using for presentation and that each one of us can use to present various technologies. After all I don't need to be an expert in Catalyst in order to give a short introduction and show a few examples, assuming they were already prepared and I went over the slides with someone who is an expert.

Same with the Perl 6 examples. Others kept referring people to me to show Perl 6 examples but we could have put together a few slides with interesting examples that we can flip though pointing out the nice features. Within the slides we could even have instructions - show the Rakudo web site now, or in other cases to show the web page of the CPAN Testers.

This of course needs more preparation and even some time before the event starts to show the slides to each other but it pays off as it will make the presentation smoother.

Even that does not mean there won't be questions that need some more thinking on how to answer. There can also be cases when the knowledge of the presenter ends and there are still questions but then we can find the other presenter who has deeper knowledge in that subject or refer the person to the community channels where can get more information. After all, a large part of what we would like to achieve is that our visitors will start learning more about these subjects and start to participate in our communities.

Pictures

Some pictures from the Perl stand on CeBIT.

Posted at Sun, 07 Mar 2010 07:10:09 -0000 | Permalink

Module::Install exists because installing CPAN distributions is not always perfectly easy.

Unfortunately, it didn't help—at least, not entirely. According to the completely unscientific process by which I install CPAN distributions, Module::Install accounts for a greater amount of pain than it should, at least according to its frequency of use. (Again, this is completely unscientific. I could guess that half of the CPAN client sessions which encounter Module::Install require me to fix things manually, but it's probably closer to 20%. It's more memorable because of my severe dislike for M::I prompting to install dependencies during configuration time.)

M::I addresses a real bootstrapping problem. I want to be able to use libraries during configuration, building, testing, and installation. I don't know which versions of those libraries you have available. Bundling known-good versions of those libraries with the distribution itself solves part of that problem...

... except when it doesn't. If I were to use M::I, I would have to re-release all of my distributions for every new release of every bundled library, at least if they contain important bug fixes for the various platforms about which I care. The cheap perfume of static linking leaves its musk heavy in the air.

It's easy to fall into the trap of a false dilemma. "You fool!" you prepare to comment below. "It's either that or the chaos of trying to make do with whatever version of those dependencies users may or may not have installed on their systems!" You're right; those are two possibilities. They're not the only two possibilities.

Part of the real problem is that bootstrapping during configuration is much too late. By the time you're running the configuration system, you're already running the configuration system. If your version of the configuration system is too old or too new, you have a problem. Bail out? Revert? Upgrade? There's no good heuristic for determining this. (The CPAN itself has an opinion. That's part of the problem.)

M::I hackers do deserve credit for helping to develop the META.yml standard. (I think M::I is the wrong approach, but I intend no slight toward its users, advocates, and developers. Invention requires the courage to get things wrong sometimes, even as it requires the courage to abandon false leads.) The META.yml specification is a big step in the right direction. If most CPAN modules have static requirements and follow a standard set of conventions, there's little or no configuration necessary. A sufficiently smart CPAN client can perform the appropriate configuration without running code from the distribution itself.

You can't avoid that in all cases; distributions with XS components, for example, need to probe system information. Good luck writing a sufficiently smart CPAN client and getting the community to agree on specific standards that let you find OpenGL headers in a cross-platform fashion, for example. Yet if 80% of CPAN distributions can get by with static, upload-time configuration, a lot of complexity of installation can go away.

Yes, that would make Module::Build and ExtUtils::MakeMaker unnecessary for (probably) most CPAN distributions, at least at the point of configuration, building, and installation. (I'm a recent fan of Dist::Zilla for automating away tedium on behalf of distribution maintainers; there's less need for Module::Install in such a world. If I never write another Build.PL again, so much the better.)

That helps, but the real problem with CPAN installations is that the CPAN itself is merely an uploading, indexing, and mirroring system. Projects such as META.yml attempt to add (and extract) meaning from the system, but they cannot work around one fundamental design feature of the CPAN. That limitation is the source of most woes for end users.

Clever readers (or experienced CPAN users) have already identified this limitation. I'll reveal it in the next installment.

Posted by chromatic on CPAN, modern perl, perl, Perl 5 at Sat, 06 Mar 2010 20:11:00 -0000 | Permalink

As the first talk in series of talks on form validation, Fred Moyer will present an overview of Data::FormValidator. Real world code examples will be presented, and you'll see how you can use Data::FormValidator to implement form validation for legacy codebases as well as new code. Data::FormValidator is a loosely coupled, highly flexible, and easy to use form validation module written by Mark Stosberg.

This meeting will take place on Tuesday, March 23rd at 7pm at Six Apart World Headquarters.

Fred Moyer's CPAN page:
http://search.cpan.org/~phred/

Data::FormValidator on CPAN:
http://search.cpan.org/dist/Data-FormValidator/

Announcement posted via App::PM::Announce

RSVP at Meetup - http://www.meetup.com/San-Francisco-Perl-Mongers/calendar/12793946/

Posted by Phred on journal at Fri, 05 Mar 2010 19:01:17 -0000 | Permalink

I'm in the metro of Paris, testing Typepad :

• new beta account I just subscribed to
• new mobile version of the blog
• Iphone App
• automatic Twitter posting
• per category feeds

Typepad is Based on Movable Type, the best blog system ouuta there. It's Perl based. It rocks.

Posted by dams on Perl at Fri, 05 Mar 2010 11:33:39 -0000 | Permalink

In Pre-commit hooks and breaking the build, I wrote a little wrapper around 'svn' to ensure that I could locally alter how it functions. However, I hard-coded the name to the actual executable. I should actually be doing the equivalent of $self->next::method to find the next executable in my path. So I wrote the following bash function to give this a try.

function next() {
  curr_exec=$1
  shift
  if [ -z $curr_exec ]; then
    echo "next_exec()" requires an argument
    exit 1
  fi
  if ! type $curr_exec >/dev/null 2>&1; then
    echo "next_exec()" argument must be an executable
    exit 2
  fi
  curr_path=$(echo $curr_exec | sed 's|/\w*$||')
  executable=$(basename $curr_exec)

  found=0
  for path in `echo $PATH | sed  's/:/\n/g'`; do
    if [ "$path" = "$curr_path" ]; then
      found=1
    elif [ $found -eq 1 ]; then
      if [ -x "${path}/$executable" ]; then
        next_exec="${path}/$executable"
        break
      fi
    fi
  done

  if [ -z $next_exec ]; then
    echo Could not find a next_exec for $curr_exec
    exit 3
  else
    $next_exec "$@"
  fi
}

This needs some work, but basically, it figures out your path and your executable name. Then it searches through the path to find the next executable with that name. This, I think, makes it safer to write override scripts for various executables and redispatch to them.

So with my svn wrapper, I can call my wrapper or call the thing it wraps.

svn diff
next svn diff

next seems like an awfully common word, though. Will this break anything?

Posted by Ovid on bash at Fri, 05 Mar 2010 10:48:51 -0000 | Permalink

Based on an explanation from Aristotle which has worked for me, here's one way you can make blogs.perl.org work a bit better for you. I've approved these and they seem to be OK.

As you may know, anonymous commenters are getting "Text was entered wrong" errors. This is because of a currently undiagnosed error with ReCaptcha. To work around this:

• Log in
• Click "POST"
• On the next page, select "Comment" from the preferences menu
• At the top, where it says "Immediately approve comments from", make sure that "Trusted commenters only" is selected
• Further down where it says "E-mail Notification", select "On".
• Near the bottom where it says "CAPTCHA Provider", select "None"
• Now select "Registration" from the "Preferences" menu.
• For "Authentication Methods", make sure "Anonymous Comments" is checked.

Whenever someone posts to your blog, you should be emailed notification. If they're "not trusted", you'll have to approve their posts. When you do, you'll have the option to make them a trusted commenter and not have to approve their posts again.

Yeah, it's a little more work to do, but setting it up is a one-shot deal. As you mark more people as trusted, you'll have to approve fewer posts.

As an added bonus, under "Preferences -> Entry", you can select your default format (text formatting). I've just switched mine to Markdown. You can go to "Preferences -> Comment" to allow the same for comments.

Posted by Ovid on community at Fri, 05 Mar 2010 07:42:10 -0000 | Permalink

With Miyagawa's cpanminus (or as I like to think of it, cpantiny, but hey it's his module) now officially the New Shiny of the moment, the same old arguments are resurfacing about less is more, and worse is better.

And the great wheel of opinion circles again.

Rather than bother commenting on cpanminus itself (I suspect many could guess my opinions) I thought I would add some caveats to the praise, that hopefully can help you identify and engineer your own equivalent successes.

1. These "subset" modules are truly successful only when you make the accuracy trade offs worth it.

Thus, the installation must be effortless, the user interface must be super-simple, zero-conf is an absolute must, and the module must succeed in all those niches where the "real" application is too hard, too big, or too clumsy.

If your accuracy is going to suck, NOTHING else is allowed to suck.

2. As I hint about with "real" these subset modules work best when they are an alternative solution, not the only solution.

If you are inaccurate and the only solution, you are an annoying limitation.

If you are inaccurate and an alternate solution, you are handy because you create a kind of user-pays situation. Simple people with simple use cases get a simple solution. Hardcore people with hardcore needs get a hardcore solution.

So if you want to make a small solution, it has to be a subset of something larger that works better. It can't be the only solution.

3. 99% is a just a marketing number, but the real number does matter.

Why? Take a look at the Heavy 100 index (http://ali.as/top100/") and you'll notice that many major and popular CPAN modules (like, say, Catalyst) have more than 100 dependencies.

With a 99% success rate (using stupidly naive "statistics") every single module on that Top 100 list will fail to install. In a large system with lots of recursive dependencies, it doesn't take much for your install count to grow to 20 or 50 or 100 dependencies.

So you need to be sure about WHICH subset you want to support, so you can be sure what percentage you REALLY need.

For example, despite all the work to support it, there is only one single module currently using Bzip2. Would it be worth it to remove bzip2 support entirely from the CPAN and help that author to convert? Probably.

The biggest benefits of these alternate solutions are often not only to do what they do. It's to demonstrate what is possible, pushing the competitors to do it as well.

Posted by Alias on journal at Fri, 05 Mar 2010 07:29:44 -0000 | Permalink

I am pleased to announce that David Mitchell's grant application has been successful. I would like to thank all the community members who took time to provide feed-back on this proposal and Booking.com whose generous contribution to TPF has allowed us to fund this grant.

Posted by Karen on Grants, Perl 5 at Fri, 05 Mar 2010 06:31:25 -0000 | Permalink

I proposed a grant for Dist::Zilla improvements to The Perl Foundation, and a notice of its approval was posted yesterday. I've gotten to work.

A number of the things I proposed to do have been on my mind for quite a while, but I haven't been able to commit to working on them. I had some of them already sketched out in branches that were unready to merge. Last night, I rebased and merged them, resolved a few conflicts, and tried to cut a development release. Unfortunately, I foolishly cut a production release.

Fortunately, nothing seems to have been horribly broken. A few missing prerequisites were exposed here and there, but those have been fixed. Some other, more serious problem were fixed tonight, and I cut an actual dev release. You can see most of my "to do" list in the git repository, if you're not content with the summary in the grant proposal.

So far, I've been checking out upstream test coverage, adding a proper logging facility, and changing the way prerequisites are registered. The logging changes are mostly to facilitate better testing so I can start writing tests for the entirety of Dist::Zilla as it stands now, before I get to any more refactoring or extension. The upstream coverage is mostly an issue because if we test lots of Dist::Zilla, but not the code written to make it go, that's a big coverage gap. I'd like to make sure that all the code (that I wrote) to make Dist::Zilla work is adequately tested. I want to be able to rely on this code, after all!

The prereq stuff has just been on my mind and in the margin notes of my recent CPAN work, especially as it related to fixing some bugs in the excellent AutoPrereq module. It may introduce a few bugs in the short term, but those should clear up. It will also make it much easier to report prerequisites, including build-time or configure-time prereqs. In another release or two, the AutoPrereq plugin should be able to report libraries found in ./t as being test requirements, separate from runtime requirements, for example.

When these tasks are complete, I'll be moving on to testing Dist::Zilla itself. That means writing some tools to make that easier, and probably a few changes to how Dist::Zilla works here and there, to improve diagnostics and configuration.

For now, I'm happy that I'm going to be able to spend some time on this. It's sorely needed improvement.

Posted by rjbs at Thu, 04 Mar 2010 23:18:09 -0500 | Permalink

Alan Haggai's work on the Archive::Zip library is complete. Numerous onerous bugs have been corrected, and the library can now handle far more kinds of data. A developer release should reach the CPAN soon, hopefully to be followed in short order by a new complete release of the library.

Posted by Ricardo Signes on grant-2008q3-archive-zip, grants at Thu, 04 Mar 2010 22:33:06 -0000 | Permalink

Been very busy with work lately so it nearly slipped completely past me that I had a large spike in traffic to this blog last week with over 1200+ unique hits on one day.

The culprit: Reddit (Programming). My post Anyone for Perl 6 metaprogramming? had been posted there .

My first blog entry to make reddit *blush*

Glad it created enough stir, comments & even points (29 in total… 59 up & 30 down!). Unfortunately some of the commenters didn’t follow link on my page to previous post and so didn’t understand the full context (only 17 of the 1200+ reddit surge actually did click the said link).

Apart from quick scans to see if anything interesting is on Perl Reddit, I don’t go to Reddit anymore. I much prefer Hacker News. In fact its been so long that I couldn’t remember what my password was so I gave up even trying to leave a comment on the Reddit post

/I3az/

Update: Gosh I also nearly missed that it had been posted to Hacker News only few days later as well!

Update 2: Things go from weird to bizarre. You can now buy a T-shirt with my post title

Posted by draegtun on Programming, perl, reddit at Thu, 04 Mar 2010 18:58:08 -0000 | Permalink

I'm excited about this for a few reasons:

• I just started using Search::GIN a while ago and it's a lot of fun. It's also an amazing example of how to write code correctly, using roles, abstraction and introspection. Reading the source is illuminating.


• Search::GIN 0.04 has a few fixes that existed for a while in the Github repo but were not uploaded to CPAN.


• There is now some docs on writing queries to help beginners. Hopefully I'll get around to documenting the extractors and write up some usage examples.

A major thanks goes to Stevan Little which helped me understand KiokuDB and Search::GIN and Yuval Kogman for writing these great tools. :)

Posted by Sawyer X on Code, Perl, CPAN, Search::GIN at Thu, 04 Mar 2010 15:28:24 -0000 | Permalink

One thing I love about git it how you have control over your local environment. Need a pre-commit hook? Just add the damned thing. Need it for Subversion and you're not the admin of the subversion server? Sucks to be you. Fortunately, having a reasonable command line lets you work around this. And I needed to because I broke the build. Twice. In a week. Buying the donuts for this would be slightly less annoying if it weren't for the fact that I get to "enjoy" low-calorie yoghurt bars instead. (Though I'm now below 82 kilos for the first time in a couple of decades).

So no more breaking the build. Right. On my previous team, if I needed a module, I could usually add it. On this team, it's a bit painful, so I just use what I need locally and then break the build when I forget and commit use Test::Most 'die';. So I wrote a little bash scrip to make svn less brain-dead.

#!/bin/bash

if [ "$1" = commit ]; then
    command="ack 'Test::Most|Data::Dumper::Simple' t/ lib/"
    lines=`$command | wc -l`
    if [ $lines > 0 ]; then
      echo $command
      echo `$command`
      echo You suck!
      exit 1
    fi
fi

if [ "$1" = log -o "$1" = annotate ]; then
  /usr/bin/svn "$@" | less
elif [ "$1" = diff ]; then
  /usr/bin/svn --diff-cmd colordiff -x "-u" "$@" | less -R
else
  /usr/bin/svn "$@"
fi

Basically, if I try to commit and I've left offending modules in the code, it tells me I suck and fails. As an added bonus, I realized I no longer need to pipe common commands to less and I get nicely coloured diff output.

This is dangerous, though. I could easily add "run perltidy on my code prior to commit". This sounds great until you realize you've just tidied a Makefile. Oops/ Still, my life is a touch easier now. And I hope that my future "breaking the build" will be in new and creative ways.

My bash isn't very good, so critiques welcome. And don't forget to write tools like this for yourself. We're programmers. Little itches become big problems when we don't scratch them. Make them go away.

And if you're wondering why I'm still using svn, it's because git-svn is a dream when working with our branches, but several git gurus have given up trying to find out why merging back into trunk on our project fails so badly. So I don't do that and still rely on svn for that tiny task.

Posted by Ovid on git, svn at Thu, 04 Mar 2010 13:32:52 -0000 | Permalink

We have received a Hague grant request, submitted by Solomon Foster.

Before the Board votes on this proposal we would like to get feedback and endorsements from the Perl community. Please leave feedback in the comments or send email with your comments to karen at perlfoundation.org.

Project Title:

Numeric and Real Support

Synopsis:

This grant aims to refactor Rakudo's current numeric classes to support the
Perl 6 specification's new Numeric and Real roles, in the process making it
easier to create new numeric classes.

Benefits to Perl 6 Development:

In late 2009, in response to users' and developers' experiences with Rakudo,
the Perl 6 specification was enhanced with the creation of the Numeric and
Real roles. This grant's purpose would be to implement those roles and
refactor Rakudo's core numeric type implementations to work with them,
including Int, Rat, Num, and Complex. Special care will be taken to simplify
adding new classes that fill the Numeric and Real roles, as there are more in
the specification that will need to be implemented at some point. In addition,
the relevant portions of the test suite would be revamped. Straightening this
all out before Rakudo * will present the newcomers to Perl 6 with a clean,
consistent numerical interface instead of the current obsolete hodgepodge.

Deliverables:

D1. Implement Numeric. [S32::Numeric: "Numeric"]
D2. Implement Real. [S32::Numeric: "Real"]
D3. Refactor Int, Rat, and Num to be Real. [S32::Numeric: "Int", "Rat", and
"Num", also S02::Bits and Pieces]
D4. Refactor Complex to be Numeric. [S32::Numeric: "Complex"]
D5. Refactor the trig functions to fit in this scheme. [S32::Numeric:
"Trigonometric functions"]
D6. Add Numeric and Real specific tests to the spectests.
D7. Enhance our current numeric tests to better work in the new system.

Project Details:

D1 and D2 require implementing a the Real and Numeric roles, and possibly
modifying the Spec a bit in the process. Probably the biggest issue here is
working out to what extent Real should implement default versions of its
methods.

D3, D4, and D5 involve refactoring the existing methods for these classes to
fit into the new role-based scheme. Special attention will be given to
facilitating the addition of new numeric types which fill these roles.

D6 would include simple tests -- do the classes belong to their appropriate
roles? -- as well as more complex tests -- can users readily create their own
Numeric class?

D7 would reorganize the existing numeric tests to better take advantage of the
new system, as well as forking role-based versions of those tests.

Project Schedule:

This is envisioned as a two-month project. The first three weeks would be spent
establishing the basic framework and getting the Numeric methods in place for
Int, Rat, Num, and Complex. The second three weeks would be spent working on
the Real methods for those classes. The last two weeks would be set aside for
trig.

I am able to begin work on this project at any time.

Report Schedule:

Blog posts every other week on a new Perl 6 programming blog on Wordpress,
also to be submitted to Perl Planet Six.

Public Repository:

All code, documentation and other relevant files that relate to Rakudo will be
checked into the Rakudo repository. All contributions to the specification and
specification tests will be checked into the Pugs repository.

Grant Deliverables ownership/copyright and License Information:

All work on produced as a result of this grant will be licensed under the
Artistic License Version 2.0. I already have signed the relevant CLA for The
Perl Foundation.

Bio:

After flirting briefly with Pugs back in its very early days, I started
regular experimentation with Perl 6 via Rakudo in late 2008. In mid-2009 I
started contributing, first to the spectests and then in August to Rakudo
itself. Since then I have been the third most active Rakudo committer,
including extensive work on Rat, Complex, and trig support.

I have an MS in mathematics and a BS with honors in mathematics and computer
science, both from the University of Michigan. I have twelve years' experience
as a professional C++ library developer.

Country of Residence:

United States

Nationality:

United States

Amount Requested:

$3000

Okay to publish proposal?:

Yes

Suggestions for Grant Manager:

Jonathan Worthington

Posted by Karen on Grants, grants, Perl 6 at Thu, 04 Mar 2010 08:40:05 -0000 | Permalink

I'm just aghast, but it's true. On April 11, 2006, the US Patent Office granted a patent for linked lists. It's mind-boggling.

The only thing conceivably unique? It's triply-linked, thus allowing multiple traversal orders. I'm confused. How is this different from a directed graph?

Posted by Ovid on patents, rights at Thu, 04 Mar 2010 07:22:01 -0000 | Permalink

From the YAPC::NA 2010 website: "YAPC::NA 2010 is actively searching for sponsorship, if you or your company would interested in being a sponsor please take a look at our Executive Summary or Sponsorship Prospectus."

Posted by Josh McAdams on Conferences, conferences, YAPC, yapc conferences, yapc-na, yapc2010, YAPC::NA at Thu, 04 Mar 2010 05:18:53 -0000 | Permalink

Posted by Michele Beltrame on conferenza, ironman, italia, italiano, perl, pisa, yapc, yapceu, yapceu2010 at Wed, 03 Mar 2010 22:38:53 -0000 | Permalink

This was the oldest grant running. Ingy dot net was working intermittently on it presenting some of the results that were promised in the proposal but not them all. TPF and Ingy agreed to close the grant with a half payment for the (half) work done.

Ingy wrote a report on the work done. It can be read at https://www.socialtext.net/yaml/index.cgi?requiem_for_a_grant

Posted by Alberto Simões on Grants, grants at Wed, 03 Mar 2010 21:34:42 -0000 | Permalink

First, here's the tl;dr summary ... Benchmarking is for losers, Profiling rulez!

I've noticed a couple blog entries in the Planet Perl Iron Man feed discussing which way of stripping whitespace from both ends of a string is fastest.

Both of these entries discuss examples of benchmarking. Programmers love benchmarks. After all, it's a great chance to whip out one's performance-penis and compare sizes, trying to come up with the fastest algorithm.

Unfortunately, this is pointless posturing. Who cares that one version of a strip-whitespace operation is three times faster than another? The important question is whether the speed matters.

Until you answer that question, all the benchmarking in the world won't help you, and that brings us to profiling.

Profiling is a lot harder than benchmarking, which may be why people talk about it less often. Profiling doesn't compare multiple versions of the same operation, it tells us where the slowest parts of our code base are.

In order to make profiling useful, we need to write code that simulates typical end user use of the code we're profiling. Then we run that code under a profiler, and we know what's worth optimizing.

Once we know that, then we can start speeding up our code. At this point, benchmarking might be handy. If, for example, on some crazy bizarro world, our program spent a lot of its runtime trimming whitespace from strings, we could benchmark different approaches, and use the fastest.

Of course, in the real world, this will never be the slowest thing your program is doing. In most cases, the slowest parts of the program are usually the parts with IO, such as reading files, talking to a DBMS, or making network calls. If this isn't the case, we may be operating on a lot of data in memory with some sort of non-trivial algorithm, and that's the slowest part.

Either way, without profiling, benchmarking is just a pointless distraction.

Of course, I'd be remiss if I didn't point out that Perl has an absolutely fantastic profiler available these days, Devel::NYTProf. It actually works (no segfaults!), and produces fantastically useful reports, so use it.

Posted by Dave Rolsky on Programming at Wed, 03 Mar 2010 20:59:47 -0000 | Permalink

For this quarter TPF had 7 grant proposals, from which 5 were voted for rejection, one for acceptance, and the final one will be kept for new evaluation in the next round.

The five rejected grants are:
* Creating a ctypes-Like Interface for Perl to External Subroutines by Shlomi Fish
* Enhancing CPANHQ by Shlomi Fish
* perl core memory improvements by Jim Cromie
* Enhancing Perl 6 Pattern Matching by Morris M. Siegel
* CPAN Reviews by Alexandr Ciornii

The grant kept for new evaluation in the next round:
* Perl Compiler by Reini urban

And finally, congratulations to Ricardo Signes for his accepted grant:
* Improve Dist::Zilla by Ricardo Signes

These results took in consideration the proposal texts and the work relevance for the Perl community, the proposal authors curriculum and interaction with the Perl community, and the comments and reactions from the Perl community on this blog.

Posted by Alberto Simões on Grants, GP2010Q1, grants at Wed, 03 Mar 2010 20:58:31 -0000 | Permalink

I really have no energy to write much. The first day of CeBIT was a bit low on visitors but even that went quite well. I met with several people I wanted to talk. Especially interesting were the chats with the people from the Eclipse Foundation.

Today, on the second day, was a lot busier. Many people were walking around and many people came to the Perl stand as well. There are always 3-4 Perl::Staff people at the stand and usually they are busy talking to visitors.

Some PHP and Ruby developers came and we could show them interesting things that can be done with Perl, Moose, Catalyst and DBIx::Class. Many people also liked Padre, you know the Perl IDE

There were many people who said that the main problem they are facing is the lack of Perl developers and some of them told us that they are actually considering switching language because of this. It seems to them that it is easy to find programmers to many other languages, something that was the case with Perl as well a few year ago but now there don't seem to be any Perl programmers. I wonder what could be the reason of that? Where are does developers gone? Have they stopped using Perl and have they remove Perl from their CV? Are there many more jobs now filled by those developers without new developers learning Perl to fill the newly created jobs? Maybe the expectations from a Perl developer are now higher than 5-10 years ago so the same people who could counted as Perl developers 5 years ago aren't considered as such any more?

Anyway we have a lot more work to do in the coming days. I'll try to add some pictures to the next report.

Renee made some nice pictures of Perl on CeBIT

Posted at Wed, 03 Mar 2010 20:26:18 -0000 | Permalink

I've just released KiokuDB version 0.37, which introduces class versioning.

This feature is disabled by default to avoid introducing errors to existing schemas^[1]. To try it out pass check_class_versions => 1 to connect:

KiokuDB->connect(
    dsn => ...,
    check_class_versions => 1,
);

To use this feature, whenever you make an incompatible change to a class, also change the $VERSION. When KiokuDB tries to load an object that has been stored before the change was made, the version mismatch is detected (versions are only compared as strings, there is no meaning to the values).

Without any configuration this mismatch will result in an error at load time, but the KiokuDB::Role::Upgrade::Handlers::Table role allows you to declaratively add upgrade handlers to your classes:

package Foo;
use Moose;

with qw(KiokuDB::Role::Upgrade::Handlers::Table);

use constant kiokudb_upgrade_handlers_table => {

    # we can mark versions as being equivalent in terms of their
    # data. 0.01 to 0.02 may have introduced an incompatible API
    # change, but the stored data should be compatible
    "0.01" => "0.02",

    # on the other hand, after 0.02 there may have been an
    # incompatible data change, so we need to convert
    "0.02" => sub {
        my ( $self, %args ) = @_;

        return $args{entry}->derive(
            class_version => our $VERSION, # up to date version
            data => ..., # converted entry data
        );
    },
};

For more details see the documentation, especially KiokuDB::TypeMap::Entry::MOP.

^[1] In the future this might be enabled by default, but when data without any version information is found in the database it is assumed to be up to date.

Posted by nothingmuch on modules, kiokudb, perl at Wed, 03 Mar 2010 18:15:00 -0000 | Permalink

in a typical graphical application, menu entries are often also available in toolbars or other widgets. and sometimes, you want to enable or disable a given action, and this means having to update all those entries and widgets everywhere this action is allowed / forbidden.

this is not fun.

therefore, i wrote tk::action, a module to help managing actions in a tk gui: just create a new object, associate some widgets and bindings with add_widget() / add_binding() and then de/activate the whole action at once with enable() or disable().

Posted by jquelin@gmail.com (Jérôme Quelin) on tk, perl at Wed, 03 Mar 2010 12:22:00 -0000 | Permalink

Jonathan Worthington has successfully completed his Hague Grant for Rakudo Signature Improvements. This is his final report:

Introduction

A while back, I filed an application for a grant to work on a wide range of improvements to Rakudo's signature, capture and binding implementation, in order to improve performance, bring us in line with the specification and implement a range of features that we did not have support for.

I have now completed the grant tasks, and am filing this report to discuss the work that was done under it. I have missed the originally planned schedule for the grant to be completed. This was in a large part due to a lot of Rakudo development since October taking place in a branch, ng, which set out to refactor and improve many aspects of Rakudo and get us in good shape for the Rakudo * release in a couple of months time. It made far more sense to implement some aspects of this grant in the branch rather than in master, and the branch also needed my attention in other key areas. The branch has recently become master, and since then I've been able to complete the final pieces of the grant and bring it to completion.

Deliverables Status

I proposed six deliverables, and have completed all of them.

D1 called for an extensive refactoring of signature internals and parameter binding. Rakudo now has a single custom signature binder, designed to handle its binding needs in accordance with the Perl 6 specification. This is a far better situation than before, where we used the Parrot binder and then layered extra binding semantics atop of it, meaning we had to make two passes through the parameters.

Having our own custom binder that takes into account the needs of Perl 6 has made it possible to do all of the other items in this grant in a clean and performant way. It has also enabled other bits of progress outside the scope of the grant; for example, I was able to get a first cut of coercion (the "as" trait) of parameters in signatures implemented in little over half an hour a few days ago.

We also have compile-time objects to represent signatures and parameters in the compiler. This has allowed much cleaner code in the actions that build the AST. In the future, this approach will also be highly useful for building extra static checking and an optimizer.

D2 required implementing support for binding named arguments to positional parameters. I considered this in the initial re-design of the binder, so it just worked "out of the box" once I switched Rakudo over to using it.

The target of D3 - unpacking of arguments - was made much easier by having a custom binder. The binder was designed to be re-entrant, and therefore unpacking arguments was implemented by specifying how various data structures could coerce themselves into a capture, and then making a recursive call with the capture that was produced.

For D4, I spent some time re-working capture support overall, and as well as being able to write a capture into a signature - and to unpack it with a sub-signature - we can also interpolate a capture into the arguments for a call. Before this didn't really work - it would lose the named parameters, for example.

Implementing D5 - support for multiple return values - first meant working with Larry to figure out the correct semantics. It turned out they were easier to get in place in the refactored "ng" branch - which has now become the master branch. I also implemented the support for binding the return values against a signature; this means all the power of unpacking arguments is available for returns too. It's all routed through the same underlying binder implementation.

Finally, for D6 I first put a proposal into the specification for how signature introspection should look, then wrote an implementation and reviewed the test coverage.

Dissemination

I have written some blog posts (1, 2, 3) about the work that I have done under this grant. I've also been active on the #perl6 IRC channel and, thanks to the evalbot, been able to give some demonstrations of the improvements there too. I also continue to attend Perl conferences and give talks about Perl 6.

Conclusions

This grant has, like my previous two, brought another major piece of Rakudo essentially "up to spec" and provided a good, stable base for future development. Our coverage of S06 is vastly improved, along with performance. Overall this grant has helped us meet critical targets on the path to the Rakudo * release and, beyond that, towards Rakudo becoming a complete Perl 6 implementation.

Posted by Karen on Grants, grants hague rakudo at Wed, 03 Mar 2010 10:30:36 -0000 | Permalink

Not thinking that any Perl books would be available for Kindle, I searched anyway. I can get at least Learning Perl and Mastering Perl from the Kindle store. I'm thinking about this because I'd like to see what I can do with an iPad Kindle app.

I'm on a trip with severe weight requirements for luggage, so I've been trying the Kindle app for the iPhone. It seemed weird at first, but after a couple chapters of the first book, I don't mind it anymore. The tech books are a bit different. If you're used to the beautiful and understated typography that is one of O'Reilly's hallmarks, you're going to think the Kindle is a bit odd. I think I have it slightly nicer on the iPhone, but it's still different.

The version I have uses multiple font faces, so you can get the monospaced font for inline code and code sections. The iPhone code wrapping is odd, and the paragraph spacing is set up for novels: that is, there's no extra space between the start of a code section and a body section. I don't think I'd want to read a tech book for the first time this way, but it's a nice reference.

Now I just need to figure out how to map something I see in the Kindle app to the physical page number so I can respond to errata (which are reported by page numbers instead of chapter and section). I guess I could develop that index myself, and I've often thought of doing that, but I'm hoping it comes as a feature to Kindle.

I should note that the books aren't any cheaper from the Kindle store. The Kindle is not about saving money: it's about convenience. If you want to save money, you can get the very cheap PDF versions of books from O'Reilly. Those should look just like the physical book, although they are probably harder to read on the iPhone.

Posted by brian d foy on books, kindle, perl at Wed, 03 Mar 2010 07:54:28 -0000 | Permalink

David Golden praised Tatsuhiko Miyagawa's excellent new cpanminus CPAN client in The power of not being all things to all people. You should consider using cpanminus.

Don't overlook something else insightful that David wrote:

It's a lot of work to be all things to all people and I keep wondering whether making things simpler and better for 99% of people would be a better choice.

The only reliable way I've ever seen to "make the easy things easy and the hard things possible" is to make the easy things the default without preventing customization of the hard things. That's a design principle for languages, APIs, and tools.

Figure out what's most common (though not necessarily what people think they want, but what they need). Optimize for that. Consider what they might need and don't prevent it.

That's not easy, and what people need will change over time, but if you want to solve problems well, you have to solve the right problems.

Posted by chromatic on CPAN, language design, modern perl, perl 5, Perl 6, TIMTOWTDI at Wed, 03 Mar 2010 00:41:02 -0000 | Permalink

The Perl community seems abuzz about the cool new cpanminus client by Tatsuhiko Miyagawa. After about two weeks of development, this is a reasonably functional CPAN client with just a fraction of the overhead, complexity and verbosity of the CPAN and CPANPLUS clients that come with the Perl core.

It’s a remarkable achievement, not only technically, but in the reaction it has sparked. As one of the (sometimes reluctant) maintainers of CPAN and CPANPLUS, I’ve realized that I both love and hate cpanminus.

• I love that Miyagawa has done so much with so little and in such a short span of time
• I hate that fanboy-types flocked to it and trashed the older clients without noting cpanminus’ limitations
• I love that Perl toolchain maintainers have rallied around Miyagawa and contributed their wisdom to make cpanminus better instead of rejecting it
• I hate that one of Perl’s great strengths (CPAN) has legacy clients that are so unwieldy, hated and difficult to maintain

Miyagawa graciously acknowledges standing on the shoulders of giants. Still, I can’t shake the nagging thought that cpanminus should never have been necessary in the first place.

What I’ve come to realize is that cpanminus is another example of the power of not being all things to all people. Miyagawa doesn’t promise that it works for all of CPAN or that it works everywhere that Perl does. He doesn’t have to. Making it work for 99% of CPAN for 99% of people is more than good enough.

I’ve been co-maintaining various parts of the Perl toolchain for a while now. It’s a frustrating challenge needing to make thing work everywhere, for everything, and trying as hard as possible not to break backwards compatibility. Plus, I don’t even get to use CPAN to make life easier. I don’t get to use handy tools like Moose or DateTime or Regexp::Common or SQLite or anything in the Config::* namespace or even basic tools like Archive::Zip. Nearly everything is done by hand.

Things have to work with just core Perl on a diverse set of platforms and with an incredibly limited set of assumptions. For example, the Perl core still doesn’t come with an HTTP client, so CPAN has to rely on FTP or command line programs to bootstrap LWP. (This is something I personally plan to tackle during the Perl 5.13 development series later this year.)

I think this is an ongoing challenge for core Perl development in general. It’s a lot of work to be all things to all people and I keep wondering whether making things simpler and better for 99% of people would be a better choice. (Anyone else for use strict by default? I hope that finally comes to pass in Perl 5.14.) chromatic writes about this topic often in his Modern Perl blog and I usually tend to agree with the points he makes. (October February 2009 had a particularly good series of posts.)

In the meantime, I look at cpanminus with greed and envy. Miyagawa++

Posted by dagolden on Uncategorized, cpan, perl-programming, toolchain, ironman at Tue, 02 Mar 2010 23:35:03 -0000 | Permalink

I was just in a room with three competent, professional Perl developers, all of whom agreed that you can’t strip whitespace from both ends of a string in a single regexp.

The common knowledge is:

$str =~ s/^\s+//;

$str =~ s/\s+$//;

I fiddled around and found this:

$str =~ s/^\s*(.*?)\s*$/$1/;

I think it’s right.  Is it better?  Not sure.   Is it faster?  Slower?  Clearer?

Speed I can test with benchmark… and using two is faster.

% perl try.pl
Rate single multi
single 444444/s — -74%
multi 1724138/s 288% –

So, nevermind.  Use two regexps.

Is one clearer?  Not sure.  But it’s slower!

Posted by Laufeyjarson on Perl at Tue, 02 Mar 2010 01:19:28 -0000 | Permalink

I've never liked ExtUtils::MakeMaker. I've liked Module::Build from the beginning. I've never, ever liked Module::Install, even though Ingy sat in my living room and hacked on what would eventually become M::I way back several years ago.

I don't believe people who use Module::Install should be shot on sight, but I do believe that Module::Install has set the usability of the CPAN back by several years.

Ingy did identify a real problem: there's too much code strewn about the configuration and build systems of the CPAN and not enough code shared. When he found himself writing something complicated to configure, compile, and install a new distribution, he'd crib code from someone like Tim Bunce or Graham Barr or Nick Ing-Simmons. In other words, to make the CPAN—perhaps the world's largest repository of redistributable and sharable library code—work, he had to copy and paste code.

M::I did get that right; turning the configuration and build system into reusable, redistributable libraries also available from the CPAN helped reduce the amount of boilerplate code and the amount of copy and paste code in configuration systems. The people behind M::I have also helped push for better configuration of CPAN clients and better tracking of dependencies and versions and types of dependencies (optional, compilation, bundling, testing, et cetera). The CPAN ecosystem is better off for that work, even though M::I itself isn't the answer.

One of M::I's biggest failings, of course, is that it prolongs the lifespan of EU::MM. Unfortunately, I think Ingy missed the big problem when he saw copy and paste code. To do anything reasonably complex with EU::MM, you have to be able to write Perl 5 code which generates (and modifies with regular expressions) cross-platform Makefiles which themselves call into Perl 5 code because Perl 5 has a sane baseline of well-understood and reliable behavior across platforms in a way that Makefiles and shells do not.

If that's not sufficient horror, consider that the way to customize EU::MM behavior the last time I looked at it (Two notes here. First, its current maintainer refuses to add new features or change existing features because it's so awful to maintain. Second, I wrote tests for some of those behaviors, so I've read and understood the code.) you write a custom superclass called MY from which EU::MM inherits to change the behavior of the various steps of generating cross-platform Makefiles which may or may not invoke Perl 5 to perform shell functions.

I am not making this up and I did not make any typos. EU:MM inherits from your custom class.

Keeping EU::MM viable long past the point where Module::Build did everything n a saner way is but a little crime. (There's a reason almost no one recommends the use of h2xs to make skeletons for new modules anymore; most new modules aren't mere wrappers around C libraries. The need for a compilation step with a pure-Perl distribution seems more than a little bit superfluous.)

Module::Install reinvoking your current CPAN client recursively to install dependencies when your current CPAN client already has a perfectly good way to install dependencies is a slightly larger crime. (I understand the justification; what if someone is trying to install a distribution from a tarball manually without a CPAN client, but is there a Principle of Least Possible Differentiation at work with that design choice?)

I don't particularly care that using M::I as a distribution maintainer means that you have to keep track of every new release of M::I which could fix bugs or make upgrading impossible and release a new version of every one of your distributions with the new M::I because of its autobundling problems, because if you get your kicks that way, more power to you. (Don't expect me to jump up and down for joy at upgrading all of your distributions on my machines, though.)

I never particularly cared for the FUD about Module::Build from some M::I discussions, especially the nonsense about "Module::Build doesn't support --install so it spews files all over the place!" (as if EU::MM ever worked properly there) or "Module::Build doesn't support uninstalling!" (as if anyone ever used that from the CPAN client to know that it worked with any system.)

I appreciate that Module::Install provided a much nicer interface than EU::MM did, and that that interface worked transparently to hide the nasty details of EU::MM. Those are true benefits, and I don't blame anyone for choosing M::I for that reason.

Even so, Module::Install's greatest crime is that it's been a distraction from identifying and fixing the real problems of the CPAN... but that's another post.

Posted by chromatic on cpan, modern perl, perl 5, perl programming at Mon, 01 Mar 2010 23:33:39 -0000 | Permalink

Posted by Dave Cross on Training, april 2010, training. london at Mon, 01 Mar 2010 21:50:34 -0000 | Permalink

Lately, I've been writing some PDL (Perl Data Language) code and noticed that I've been doing the following:

my $temp_pdl = $pdl->…->final_expr();
my $result_pdl = $temp_pdl->where($temp_pdl < 0);

I've been thinking if I could somehow write it like that:

my $result_pdl = $pdl->…->final_expr()->with(sub { $_->where($_ > 0)});

Which seems cleaner and more elegant. I went on IRC asking about it and someone suggested rindolf: (sub { local $_ = shift; $_ * $_ * $_})-gt;( $obj->... ); (up to an episolon, but naturally this reverses the order and messes with the flow of the code). Eventually Matt S. Trout (mst) proposed the following solutions:

sub UNIVERSAL::with
{
    local $_ = shift;
    my $sub = shift;
    return $sub->($_,@_);
}

my $_with = sub {
    local $_ = shift;
    my $sub = shift;
    return $sub->($_,@_);
};

(I had figured something like that was possible previously, but I was beating around the bush.)

The $_with approach is covered in one of Matt's blog posts titled "Madness with Methods".

I wondered if it will work with autobox and mst said that I don't know and I hope not since the UNIVERSAL approach is evil.. But it does, out-of-the-box:

#!/usr/bin/perl

use strict;
use warnings;

use autobox;

sub UNIVERSAL::with
{
    local $_ = shift;
    my $sub = shift;
    return $sub->($_,@_);
}

my $_with = sub {
    local $_ = shift;
    my $sub = shift;
    return $sub->($_,@_);
};

print +(10+1)->with(sub { $_ * $_; }), "\n";
print +(10+1)->$_with(sub { $_ * $_; }), "\n";

This prints 121 twice.

I now wonder what to do with this knowledge. I have some aspirations of releasing it to the cpan as a mini-module. Still, it's really cool.

Finally, I should note that it seems like I'm going to lose my current Planet Perl Iron Man karma due to inadequate blogging. Oh well.

Posted by Shlomi Fish ( shlomif@iglu.org.il ) on perl, fun at Mon, 01 Mar 2010 17:22:55 -0000 | Permalink

Many years ago, when I was... roughly 15 (I think), I met Theodor Ts'o (one of the first hardcore Linux Kernel hackers, since version 0.90, I believe) at a Linux event IBM organized in Israel. I should note that he is a very nice person.

After the event, we got to talk a bit. We talked about our favorite games. Mine was "avoiding segfaults". This was back when I was programming in C.

Today I wrote the following regex:
qr/^([\w|\.]+)\s+(?(\d+)\s+)?(\w+)\s+(?:(\d+)\s+)?([\w|\d|\.]+)$/;

Then got a Segmentation fault

Can you spot the error?
Here's a hint: it is missing a colon (:).

This is perl, v5.10.0 built for i486-linux-gnu-thread-multi

Posted by Sawyer X on Code, Perl, The Stupids, stupid, tdd at Mon, 01 Mar 2010 13:04:42 -0000 | Permalink

This is just a short promotional post.

If you are interested in the Perl/PostGIS http://geo2gov.com.au/ application we took second prize with during the Mashup Australia competition (or the subject of open government data in general) my "Nerds for Democracy" partner in crime Jeffery Candiloro will be giving a talk as part the Ignite Sydney speaking event on Tuesday night Sydney time.

You should be able to watch a live stream of the talk at the Ignite video website here.

http://igniteshow.com/

In other Nerds for Democracy news, work is now underway on our entry for the NSW apps4nsw competition. It's looking pretty awesome, in particular because it will feature a set of government data we uncovered by accident, and which we think the public has pretty much never seen before.

Posted by Alias on journal at Mon, 01 Mar 2010 02:05:42 -0000 | Permalink

A patched release is on the way to the CPAN, and you are strongly urged to upgrade.

Posted by david | Permalink

I'd like to express my warm thanks to the lovely people at Yahoo and in particular to their bot-herders. Until quite recently, their web-crawling bots had most irritatingly obeyed robot exclusion rules in the robots.txt file that I have on CPANdeps. But in the last couple of weeks they've got rid of that niggling little exclusion so now they're indexing all of the CPAN's dependencies through my site! And for the benefit of their important customers, they're doing it nice and quickly - a request every few seconds instead of the pedestrian once every few minutes that gentler bots use.

Unfortunately, because generating a dependency tree takes more time than they were allowing between requests, they were filling up my process table, and all my memory, and eating all the CPU, and the only way to get back into the machine was by power-cycling it. So it is with the deepest of regrets that I have had to exclude them.

Cunts.

[update] For fuck's sake, they're doing it again from a different netblock!

Posted by david | Permalink

Posted by david | Permalink

Did you ever wonder how BEGIN, CHECK, END and so on are called in Perl? Well, they didn't have a good name, until recently.

The Perl 6 spec listed them under closure traits, which is unwieldy, and not really exact either. Now they are called phasers, because they tell you which execution phase the block or statement runs in.

There are so many possible puns that I'll refrain from writing any.

Posted by Moritz Lenz | Permalink

How hard is it to build a good search engine? Very hard. So far I thought that only one company has managed to build a search engine that's not only decent, but good.

Sadly, they seem to have overdone it. Today I searched for tagged dfa. I was looking for a technique used in regex engines. On the front page three out of ten results actually dealt with the subjects, the other uses of dfa meant dog friendly area, department of foreign affairs or other unrelated things.

That's neither bad nor unexpected. But I wanted more specific results, so I decided against using the abbreviation, and searched for the full form: tagged deterministic finite automaton. You'd think that would give better results, no?

No. It gave worse. On the first result page only one of the hits actually dealt with the DFAs I was looking for. Actually the first hit contained none of my search terms. None. It just contained a phrase, which is also sometimes abbreviated dfa.

WTF? Google seemed to have internally converted my query into an ambiguous, abbreviated form, and then used that to find matches, without filtering. So it attempted to be very smart, and came out very stupid.

I doubt that any Google engineer is ever going to read this rant. But if one is: Please, Google, keep it stupid, stupid.

I'm fine with getting automatic suggestions on how to improve my search query; but please don't automatically "improve" it for me. I want to find what I search for. I'm not interested in dog friendly areas.

Posted by Moritz Lenz | Permalink

(This blog post will probably only interest Perl 6 hackers, since it talks only about infrastructure.)

One of the central pieces of Perl 6 infrastructure is the Pugs svn repository. It holds not only the Pugs source code, but also lots of other Perl 6 projects:

• Specification
• Test suite
• STD.pm (the standard grammar)
• SMOP
• mildew and mildew-js
• sprixel
• vill
• mp6, kp6, perlito
• elf
• various websites (perl6.org, pugscode.org, perlcabal.org/syn/)
• a host of scripts related to keep things running (rebuild the HTML version of the synopsis; smartlink checking; cronjobs for updating websites etc.)
• various documentation efforts
• An unknown number of projects more or less related to Perl 6

It's huge, but at the same time it's very practical: anybody who is interested can get write access very easily, create a new subfolder for a new project, or can fix a typo in someone else's README file without asking for commit access first.

The pugs repo is also viral: Anybody with commit access and invite new committers. Despite what you might think of it: it actually works in practice, so far I haven't seen a single case of abuse.

The pugs repository is hosted on feather, a server kindly provided by Juerd and his company. It contains three virtualized servers, feather{1,2,3}. feather2 is used for "sensitive" data (for example an IRC bot that has API keys for various github accounts, and the perl6.org website). Only a handful of "trusted" users have an account there. feather3 is used for low security stuff like evalbots which might go astray.

feather1 holds all the rest. That means the pugs repository, commitbit (the software we use for handing out commit bits and resetting svn passwords), various websites, and a whole bunch of Perl 6 developers and users have a shell account there, for trying out Perl 6 and hosting their screen + irssi sessions there.

I was about to write feather1 is maintained by a bunch of volunteers, but that would be a lie. It is "maintained" on an as-needed basis by whoever has time and feels half-way competent. It is an "interesting" mixture of Debian unstable and experimental. And it's becoming unmaintainable.

It seems clear what to do: set up a fourth virtual machine, set up a replacement for feather1, and en passant migrate some things (like websites and the pugs repo) to feather2.

But wait! There is an issue. There's always an issue. Setting up a new machine and migrating services takes time. Lots of time. And nobody wants to do it right now. Quite understandably.

Take the pugs repository for example: you might think it's easy enough to copy /var/svn/... recursively on the new host and set up Apache... except that you need authentication. And authentication is coupled to commitbit. And commitbit runs on Jifty. And if you want to install Jifty, you install half of CPAN. Does anybody know if commitbit is still maintained? and if it installs cleanly on a Debian stable?

So I thought about some changes to the infrastructure to make it easier to maintain. For example we handle commit bits differently for git projects on github: an IRC bot knows the API keys of the project owners, and can add committers to the projects. That's nice, and the IRC frontend is much leaner than the Jifty-based web frontend. But we lose virality. For security reasons the bot has to keep a whitelist of IRC users who are allowed to invite commiters. Since IRC nick names and (git|svn) user names don't always match, such a list has to be maintained manually.

The second question is: should we take the chance and move the pugs repo to git? I prefer git over svn by far, but there are also costs involved. For example Rakudo checks out a copy of the test suite via svn. The test suite is only a subdirectory of the pugs repo, so unless we split the pugs repo, we'd have to find a way to check out only a part of a git repository. I have no idea if that's possible.

Either way, I'd like to hear your thoughts, and learn from your experience:

• Do you know any "viral" git or svn hosting solution (ie where every committer can invite more committers)? (don't say commitbit - it's a PITA to maintain. Something more lightweight would be appreciated)
• If you use the pugs repository now: do you prefer svn or git? how strongly?

Posted by Moritz Lenz | Permalink

Most programming languages handle failures with either of two paradigms: failing routines return special values, or they throw exceptions.

Either way has its severe problems: in languages like C it can be very simple to forget to catch such a return value, and very tedious to propagate them to the caller; on the other hand throwing exceptions often clutters the code with way too many try blocks, and it's generally unfriendly if you try to automatically parallelize expressions.

So Perl 6 offers a middle ground: soft or unthrown exceptions. If a routine calls fail("message"), a new Failure object is created and returned from the current routine. That object behaves as an undefined value, which stores the message, file and line information of the fail() location, a backtrace and so on.

When you ask such an object whether it's true or false, or defined or undefined, you'll get a correct answer, and the exception is marked as handled. However if you try to use it as an ordinary value, it turns into an (ordinary) fatal exception. So both of these work:

# Variant 1: no exception thrown

my $handle = open('nonexistingfile');
if $handle {
    .say for $handle.lines;
} else {
    # do something else
}


# Variant 2

my $handle = open('nonexistingfile');

# throws a fatal exception while calling $handle.lines
.say for $handle.lines;

Now if you do some automatically parallelized operations, a single failure doesn't have to abort the whole operation, and neither is information lost

# divide @a1 by @a2 element-wise, a division by zero might occur:
@a1 »/« @a2;

The API for accessing the Failure objects isn't very mature yet, but the concept stands. See S04/Exceptions for the gory details, as they stand today.

Posted by Moritz Lenz | Permalink

And to give a pretty web front-end to it for other people to read code without having to check it out of the repository, I use cgit, which I mostly chose because it's a dead simple CGI and not a huge fancy application.

One problem with cgit is that by default it doesn't do code highlighting. But it has the ability to run blobs of code through any filter you care to name before displaying them, so to get something nice like this all you need to do is write a highlighter and add a single line to your cgitrc:

source-filter=/web/www.cantrell.org.uk/cgit/highlighter

My highlighter program is this:

   1 #!/usr/local/bin/perl
   2 
   3 use warnings;
   4 use strict;
   5 
   6 my $file = shift;
   7 
   8 if($file =~ /\.(p[ml]|t)$/i) {
   9     system "/usr/local/bin/perltidy -html -st -ntoc -npod -pre -nss -nnn"
  10 } else {
  11     system "cat -n";
  12 }

Posted by david | Permalink

Update: now 48.3% even shinier - see on the right

Posted by david | Permalink

I've blogged about the Perl 6 Advent Calendar 2009, and want to follow up that it's been a huge success so far.

We're about half-way through, and you might be interested in our number of visits per day:

On 6th and 7th of December we had about 18k visitors in sum, courtesy of slashdot and Tim O'Reilly on Twitter.

As a result we got a lot of new people in our #perl6 IRC channel, asking for help on how to get Rakudo working, how to code some specific things in Perl 6, or asking about general design decisions.

Also mj41 reported that he has collected more than 50% more Perl 6/parrot blog posts than in the previous year.

We've also acquired perl6.org for our uses this year, and it has been promoted enough to be the third hit on a google search for Perl 6.

All in all I'm rather happy with the marketing state of Perl 6 in 2009, and hope to see similar efforts for 2010. With the upcoming releases of Rakudo Star and a Perl 6 book I'm pretty sure we'll do well, and I hope for more slashdot coverage :-).

Posted by Moritz Lenz | Permalink

So what have I done while ill?

On Friday I became old (see previous post), and went to the Byzantium exhibition at the Royal Academy. It was good. You should go.

Saturday was the London Perl Workshop. My talk on closures went down well, and people seemed to understand what I was talking about. Hurrah! I decided that rather than hang around nattering and going to a few talks, I'd rather hide under my duvet for the rest of the day.

I mostly hid on Sunday too, and spent most of the day asleep. In a brief moment of productivity, I got my laptop and my phone to talk to each other using magic interwebnet bluetooth stuff. I'd tried previously without success, but that was with the previous release of OS X. With version X.5 it seems to Just Work, so no Evil Hacks were necessary.

The cold means that I can't taste a damned thing, not even bacon. So now I know what it's like to be Jewish. Being Jewish sucks.

And today, I am still coughing up occasional lumps of lung and making odd bubbling noises in my chest, although my nasal demons seem to be Snotting less than they were, so hopefully I'll be back to normal tomorrow.

Posted by david | Permalink

It works if you have a Unixish toolchain. I need to wait for the CPAN-testers reports to see if I managed not to break anything on Windows. Unfortunately, even though the lovely Mr. Alias has worked hard to make Windows machines available to developers, I found it to be just too hard to use. Even downloading my code to the Windows machine was hard, as Windows seemed to think it knew better and shouldn't download the file I told it to download. Then once I had downloaded it, Windows decided to hide it somewhere that I couldn't get to using the command line. So I gave up.

I might try again once there are some decent tools on the machines: wget, tar, and gzip at minimum, as given those I can quickly bootstrap anything else. Software development isn't just about having compilers available.

Posted by david | Permalink

So, the solution.

=over

#include shared/parameters

=back

and some Magic that does the cpp-stylee substitution at make dist time. Note the 'dist' section in my call to WriteMakefile.

This is, of course, crying out to be made less horribly hacky, but it works for now, so I'm happy.

My original idea was to write some crazy shit that would do the #include at install-time, when the user was installing my code. But that has the disadvantage that tools like search.cpan wouldn't show it properly, as they simply look at the files in the distribution. So this does the #includes at the last moment just before I package up the code and upload to the PAUSE. You lovely people get the right documentation in all the right places, I only have to maintain it in one place so it stays in sync, and (in the interests of Laziness) I don't have to remember to run any extra scripts before releasing, make dist just Does The Right Thing.

Posted by david | Permalink

Philippe Bruhat's talk on Net::Proxy was great - you can tell it's great because I came away with ideas for at least four things that I need to write. First up will be a plugin for it to allow the user to specify minimum and maximum permitted data rates for proxied connections. This will permit bandwidth limits for maximum permitted rates, but will also help to defeat IDSes doing traffic analysis if you specify a minimum permitted data rate.

This will protect (eg) ssh sessions from being identified based on their very bursty traffic pattern, by "filling in the blanks" with junk data.

In the evening, the CPAN-testers BOF was productive.

Posted by david | Permalink

Posted by david | Permalink

Posted by david | Permalink

These tidings posts seem to become rather sparse, but I hope you get some news by reading the Planet Six feed aggregator anyway.

Specification

• Larry lifted up the dual nature of Ranges. They mostly serve as an interval now, for smart iteration the series operator has been pimped up. You can now write for 1, 3 ... *+2, 9 { .say } to print all the odd numbers between 1 and 9. (r28344, r28348, r28351).
• Rational and Complex types now have their own literals (r28173).
• Stubbed classes are now documented (r28196, r28197).
• The new S08 documents Parcels and Captures.
• The numeric types have been cleaned up a lot (r28502 and later commits up to r28597).
• New and improved signature introspection (r28664, r28665).

Compilers

Rakudo

As opposed to two months ago, Rakudo now

• supports the Rat type
• supports overloading of many built-in operators
• has contextual variables
• has a faster and much better signature binder
• supports all kind of trigonometric functions, including on complex numbers
• implements sophisticated signature introspection

Patrick Michaud is also working on a new tool named npq-rx, which combines a self-hosting NQP compiler and a new regex engine, which already supports proto regexes, NQP code assertions and closures, and is generally much faster and better than PGE.

Sprixel

Mathew Wilson aka diakopter started sprixel, a Perl 6 to Javascript compiler.

Mildew

Mildew now has an experimental javascript emitter.

Other matters

perl6.org is redesigned again, this time spanning multiple pages, thus allowing much more stuff to be linked there.

Four Perl 6 and Rakudo hackers announced that they are writing a Perl 6 book, the print release of which shall coincide with the release of Rakudo Star.

Posted by Moritz Lenz | Permalink

I think it's not too bold to call the Perl 6 Advent Calendar a full success: over 40k page views, more than 150 non-spam comments, and lots of new faces and nicknames in #perl6 - it's been a very pleasant surprise.

I asked myself why it became such a success, and came up with this list of things:

Limited in Scope

24 days are over rather quickly, so everyone who contributes knows when the deed is done

It appeals to very different contributors

Some people like the shiny new regex and grammar features, others are interested in the object model, operator overloading or complex numbers. Since we had no fixed topics, everybody could choose a topic that played to their strengths.

Each task is small and well limited

Writing a good post about a topic you know takes about half an hour up to an hour, maybe two if you need to do some research on the topic. Either way it's a relatively small task (compared to "write a regex engine" or "redesign the build system of $compiler" or so).

Schedule and polite nagging

We had a schedule for each day, and people could add their name to a free slot. When the day approached, the other authors started to ask politely how it was progressing, making sure people did not forget their posts. Also authors felt the subtle pressure to finish their posts on time.

Lots of positive feedback

We had lots of feedback, most of it quite positive: blog comments, visitors in our IRC channel, being featured on slashdot. That was very encouraging.

Also other authors would preview and proof-read the posts before they were published, pointing out falsities and gems.

We had a driving force

PerlJam and colomon took care. The decided to make the advent calendar happen, set up a blog, discussed things and so on - they made it happen.

All these factors encouraged contributions. I don't think all of them are necessary for a successful, lively projects, but they certainly help.

What other factors do you know that encourage contribution in open-source projects? What could we have done even better?

Posted by Moritz Lenz | Permalink

Posted by david | Permalink

If you have an array @a and want to access the first element, in Perl 5 you write that as $a[0], in Perl 6 you write it as @a[0]. We call the former mutable sigil, and the latter immutable sigil.

You might think that's a small change, but the implications are rather deep, and we've had quite a few discussions about it in #perl6. In particular people often ask if it's possible to backport the Perl 6 behavior to Perl 5. The answer is "not easily".

In Perl 5 context propagates inwards, which means that in a statement like

... = func()

The compiler wants to know at compile time which context func() is in. If it doesn't, it complains:

2$ perl -ce '(rand() < 0.5 ? $a : @a) = func()'
Assignment to both a list and a scalar at -e line 1, at EOF
-e had compilation errors.

This also means that, in Perl 5, array slices and scalar array accesses have to be syntactically distinguished:

my @a;
$a{other_func()} = ...; # scalar context
@a{other_func()} = ...; # list context

So you can't just make sigils in Perl 5 immutable without also rewriting the whole context handling rules.

In Perl 6 that's not a problem at all, because functions don't know the context they're in, in fact can't know because of multi dispatch.

Instead functions return objects that behave appropriately in various contexts, and the context is determined at run time.

After getting used to it the immutable sigils are quite nice, and less complicated when references are involved. Anybody who objects without having tried it for at least three weeks, and is spoiled by Perl 5, will be shot.

Posted by Moritz Lenz | Permalink

Posted by david | Permalink

The conference dinner at a Heuriger on the outskirts of Vienna was great. The orga-punks had hired a small fleet of buses to get us there and back, and one of the sponsors laid on a great buffet. The local wine was pretty damned fine too, and then the evening de-generated into Schnapps, with toasts to Her Majesty, to her splendid navy, and to The Village People.

It wasn't all debauchery in the evening though - on the bus, I had a very useful chat with Philippe about Net::Proxy, and re-designing it to make it easier to create new connectors for it.

Posted by david | Permalink

At the traditional end-of-YAPC auction, I avoided spending my usual stupid amounts of money on stupid things, which was nice. Obviously the hundred quid I put in to buying the hair style of next year's organisers wasn't stupid. Oh no. Definitely not.

An orange mohican will suit Domm beautifully.

Posted by david | Permalink

We want a Perl 6 book. We want it badly enough to write it ourselves. So that's what we're doing: writing one.

We, that is Patrick Michaud (architect of the Rakudo Perl compiler), Jonathan Worthington (prolific contributor to both Rakudo and Parrot), Carl Mäsak (frenetic Rakudo user, and our number one bug finder) and Moritz Lenz (keeper of the Perl 6 test suite, and Perl 6 user and blogger). We are also open to contribution from others - already Jonathan Scott Duff has written an initial preface for us.

We don't have a name yet for our book. We want to cover the basics of Perl 6, enough to get your feet wet, and enough to make you want to use it. We want it to be based on useful examples. It is not going to be the definitive book, that task we leave to Larry Wall and Damian Conway.

Our vision is to present primarily the subset of Perl 6 that Rakudo understands, and have printed copies available by the time Rakudo Star is released, that is April or May 2010. chromatic and Allison Randal have kindly offered to published it via Onyx Neon Press.

Until then, monthly releases will be published under a Creative Commons license (noncommercial, attribution, share-alike).

Currently we have four chapters under construction, and the intention of writing the more introductory chapters later, when we know what we need to introduce for the later chapters. So far we have

• Multi dispatch
• Classes and Object
• Regexes
• Grammars

You can download the preliminary PDF version of the book here.

Interested? Check out the git repository, and join us in irc://freenode.net#perl6book.

Posted by Moritz Lenz | Permalink

There were lots of other good lightning talks. Dmitri Karasik's regexes for doing OCR, Juerd Waalboer's Unicode::Semantics, and Renée Bäcker's Win32::GuiTest were especially noteworthy.

Richard Foley's brief intro to the perl debugger was also useful. Unfortunately Hakim Cassimally's talk was about debugging web applications, which I'd not noticed on the schedule, so I didn't stay for that.

And finally, Mark Fowler's grumble about why perl sucks (and what to do about it) had a few interesting little things in it. I am having vaguely sick ideas about mixing some of that up with an MJD-stylee parser.

At the auction I paid €250 to have the Danish organisers of next year's YAPC::Europe wear the Swedish flag on their foreheads. This, I should point out, was Greg's idea. I would never be so evil on my own.

Posted by david | Permalink

It asked me to ask you all to keep the volume down.

Posted by david | Permalink

The first two legs of that are second class, cos first wasn't available on Eurostar (being a Friday evening it's one of the commuter Eurostars and gets booked up months and months in advance) and was way too spendy on the sleeper to Munich. Upgrading to first class from Munich to Vienna is cheap, so I have.

Coming back it's first class all the way cos upgrading was nearly free ...

Don't even think about trying to book online or over the phone, or at the Eurostar ticket office at Waterloo. Your best bet is to go to the Rail Europe shop on Picadilly, opposite the Royal Academy and next to Fortnums.

Posted by david | Permalink

In the great tradition of Perl Advent Calendars, colomon started and announced the 2009 Perl 6 Advent Calendar, with a post about Perl 6 each day.

After the first post many #perl6 regulars volunteered to contribute a post, so 20 of the 24 days are already allocated.

I'm looking forward to many nice posts, most of which will probably highlight a small Perl 6 feature.

Posted by Moritz Lenz | Permalink

In Perl 5 there is the undef value. Uninitialized variables contain undef, as well as non-existing hash values, reading from unopened or exhausted file handles and so on.

In Perl 6 the situation is a bit more complicated: variables can have a type constraint, and are initialized with the corresponding type object:

my Int $x;
say Int.WHAT();     # Int()

These type objects are also undefined, but in Perl 6 that doesn't mean they are a magical value named undef, but that they respond with False to the defined() subroutine and method.

In fact there is no undef anymore. Instead there are various values that can take its place:

Mu is the type object of the root type of the object hierarchy (or put differently, every object in Perl 6 conforms to Mu). It's the most general undefined value you can think of.

Nil is a "magic" value: in item (scalar) context it evaluates to Mu, in list context it evaluates to the empty list. It's the nothing to see here, move along value.

Each type has a type object; if you want to return a string, but can't decide which, just return a Str.

Other interesting undefined values are Exception (which usually contain a message and a back trace), Failure (unthrown exceptions), Whatever is a generic placeholder that can stand for "all", "infinitely many", "many" or as a placeholder for a real value.

Posted by Moritz Lenz | Permalink

A few days ago masak blogged about the social aspect of the is Perl 6 really Perl? question.

He presumes that the answer is yes, but doesn't tell us why. I'll try to give some reasons.

Perl 6 started as the successor to Perl 5

Perl 6 started off as the successor to Perl 5, at a Perl 5 meeting, by the Perl crowd. It was a plan to escape both the backwards compatibility trap (which meant that broken things couldn't be fixed without many people yelling), and the lack of momentum in the community.

Perl 6 embraces the Perl philosophy

What makes Perl Perl? In my opinion it's not the sigils on variables that make Perl Perl, or that writing a regex only need two characters and so on. It's mostly the philosophy that makes the difference.

There are some underlying principles like TIMTOWTDI, context sensitivity, convenience over consistency, making simple things easy and hard things possible, often used constructs short and less frequent things longer, and so on.

Perl 6 is founded on all those philosophies and ideals, and also shares some technical principles. For example sigils on variables (oh, I mentioned them already ...), easy access to powerful regexes, that fact that operations coerce their arguments (instead of the type of arguments determining the operation like in javascript, where a + can either mean addition or string concatenation).

So if you agree with my definition of what makes Perl Perl, Perl 6 is also Perl. If not, please tell me what's the essence of Perl!

Posted by Moritz Lenz | Permalink

<meta>From my useless musings series.</meta>

As a programmer you have to have confidence in your skills, to some extent, and at the same time you have to constantly doubt them. Weird, eh?

Confidence

You need some level of confidence to do anything efficiently. Planning ahead requires confidence that you can achieve the steps on your way.

As a programmer you also need some confidence with the language, libraries and other tools you're using.

If you program for money, you also have to assess what kind of programs you can write, and where you might have problems.

Doubt

In the process of programming you make a lot of assumptions, some of the explicit, some of them implicit. If you want to write a good program, it's essential that you are aware of as many assumptions as possible.

When you find a bug in your program, you have to challenge previous assumptions, and that's where doubt comes in. You not only suspect, but you know that at least one of the assumptions was false (or maybe just a bit too specific), and you know that you did something wrong.

Sometimes programmers make really stupid mistakes which are rather tricky to track down. That's when you have to question your own sanity.

One example (that luckily doesn't happen all that often to me) is when I edit my program, and nothing seems to change. Nothing at all. Depending on the setup it might be some cache, but something it is even more devious - for example I didn't notice that the console where I edit and the console where I test are on different hosts - and thus the edits actually have no effect at all.

After having done such a thing once or twice I adopted the habit of just adding a die('BOOM'); instruction to my code, to verify that the part I'm looking at is actually run.

These are moments when I question my own sanity, thinking "how could I have possibly done such a stupid thing?". Doubt.

The same phenomena applies when doing scientific research: since you usually do things that nobody has done before (or at nobody has published about it yet), you can't know the results beforehand -- if you could, your research would be rather boring. So you have no external reference for verification, only your intuition and discussion with peers.

Posted by Moritz Lenz | Permalink

When you learn a programming language, you not only learn about the syntax, semantics and core libraries, but also the coding style and common idioms.

Idioms are common usage patterns; learning and reusing them means you have to spend less time thinking on common things, and have more time working out the algorithms you deal with.

That's different if you learn Perl 6 - it's a largely unexplored field, and while there are loads of nice features, you might still feel a bit lost. At least I do. That's because I often think "There's got to be a much easier way to achieve $this, but it often takes time to find that easier solution - because nobody developed an idiom for it.

In those cases it helps to ask on the #perl6 IRC channel; many smart people read and write there, and are rather good in coming up with simpler solutions.

For example see masak's ROT13 implementation in Perl 6. In the right column you can see later revisions, and how they gradually improve, steady up to a one-liner.

I also made some simplifications to JSON::Tiny, which basically shows that when I wrote these reduction methods first I used Perl 6 baby talk language.

The nice things about exploring the Perl 6 wonderland of unexplored idioms is that it really pushes your ego if you find a nice simplification, and that you have something to blog about for the Planet Perl Iron man ;-)

Posted by Moritz Lenz | Permalink

It strips away all the useless rubbish that normally surrounds Wikipedia pages, as well as things like the editing functions which are also hard to use on portable devices. Internally, it's implemented using perl, LWP, and mod_perl, and is hosted by Keyweb.de.

Posted by david | Permalink

Until now, it's used a SQLite database of test results that I downloaded every day and then mangled a bit to do things like add some necessary indices, figure out which reports are from dev versions of perl, and so on. That worked really well back in the summer of 2007, when there were only half a million reports in the database. I started worrying a bit at the beginning of 2009 when we hit 3 million, but the update happened overnight so I didn't care. But now that we've got over 6 million reports, the update would take anywhere between 8 and 14 hours. Not only is that not sustainable given the current growth rate, it also hurts the other users on that machine, because almost all of that time is spent waiting for disk I/O - which means that they're also waiting for the disk. On top of that, when you have big databases, a SQLite CGI ain't a great idea because indices have to be fetched from disk every time, so reads pound the disk too. Doubleplusungood!

Fun fact: SQLite is great for prototyping, but it doesn't scale :-)

So now it uses MySQL. Having a database daemon running all the time means that there's now some caching, so reads are quicker. In addition, given that I can't just simply fiddle with the structure of the database that I download to produce what I want, and instead have to import the data into MySQL, it now only imports new records, so the daily update takes only a few seconds.

I also re-jigged the structure of how it caches test results. Instead of being all in one directory with hundreds of thousands of files, they're split into a hierarchy. This probably won't have any significant effect on normal operations, but it will certainly make it faster for me to navigate around and see what's going on when people submit bug reports!

Posted by david | Permalink

Much has happened in the Perl 6 land in 2009. Here is my humble attempt to summarize some of it; If you find something that I missed, feel free to contact me, I'll try to add it.

Specification

The year started with lots of improvements to S19. In January we also learned that *-1 constructs a closure, which means that Perl 6 has semi-automatic currying features built into most operators.

Lists, Captures and Parcels

We've seen a lot of talk about slices, lists, captures and parcels. The heart of the discussions is always how interpolation and non-interpolation of lists can be made both flexible and intuitive. For example: should 1, 2, 3 Z 'a', 'b', 'c' return a single, flat list? or instead a list of lists? How can a function which receives the result decide for itself what it want to receive? How does that mix with multi-dimensional arrays?

I haven't followed these discussions very closely, and so I'm hard pressed to give a good summary; however it seems that in the end an agreement was reached: each parenthesis constructs a Parcel, short for Parenthesis cell. A Parcel can behave context sensitively: A single-item Parcel degrades to its contents; as a signature list it is converted to a Capture object; code object also return parcels.

It remains to be seen how multi-dimensional slices (with the @@ sigil) evolve, and if we can't find anything suitable to replace them.

Built-in Routines

S29, the list of built-in functions and methods, finally got some long awaited attention in 2009, starting with Carl Mäsak's S29 Laundry List, and later carried on by Timothy Nelson, who split S29 into a set of documents summarized as S32.

In December it was decreed that most built-in methods have a candidate in a new class Cool, (Convenient OO Loopbacks), of which all value types and container types in Perl 6 inherit. That way maximal DWIMyness can be retained, while keeping user defined types clean of the more than hundred methods defined in Cool.

It is rather perlish to have a distinct name for each operation, and make it coerce its arguments. A few exceptions exist in Perl 5 (like reverse, which is list reverse in list context, and string reverse in string context); in Perl 6, most of these exceptions have been removed: reverse now only reverses lists, strings are reverted with flip, hashes with invert.

At the Nordic Perl Workshop, Larry decided that the prefix:<=> operator had to go, and replaced it with the .get and .lines methods.

Operators

The Cross Meta Operator is now Xop instead of XopX; in analogy the R meta operator reverses the argument list, so $a R- $b is the same as $b - $a.

Ranges served two purposes: one for denoting ranges in the sense that the mathematicians use them, and for generating lists according to simple schemes. These two functions have been separated: ranges are still constructed with two dots, but the :by adverb is gone; more intricate, lazy list generation can be achieved with the new series operator:

.say for 1, 1.1, 1.2 ... 5;
.say for 1 ... *+0.1, 5;

Numbers

The above actually works, and doesn't suffer from floating-point arithmetics, because 0.1 isn't stored as a floating-point number, but rather as a fractional number of type Rat.

Other languages decided against that approach, because some very simple loops quickly produce rather large numerators and denominators, degrading performance of the integer operations. Perl 6 instead has a limit in denominator size, and falls back to floating-point operations when that limit is crossed.

Implementations

Rakudo

A lot of work has been done in Rakudo; in fact it's hard to remember how it used to be in January 2009; Most features were implemented by Patrick Michaud and Jonathan Worthington, but we had a lot of other contributors too.

In January, Rakudo left the Parrot repository and since then lives on github as a git repository. It now relies on an installed parrot.

Rakudo implements many new features and lifts old limitations:

• Many built-in routines are now written in Perl 6
• eval() and classes now have access to outer lexical variables
• Much improved Unicode support, both in IO and regular expression
• punning of roles when .new is called
• Typed arrays and hashes, parametric roles
• Routine return types are now enforced
• Error messages now contain backtraces with filenames and line numbers
• Multi dispatch is now implemented with a custom dispatcher and signature binder, bringing much improvements over the dispatch and binding semantics that parrot supports.
• User-defined operators now possible, and automatically generate some of their associated meta-operators.
• Contextual variables
• User-defined traits are now possible; some of the built-in traits are now written in pure Perl 6.
• Rational numbers are now implemented, and support for Complex numbers has been much improved.
• routine signatures can now be introspected properly.

SMOP and Mildew

SMOP and Mildew have seen a major refactoring, connected to the changed semantics of slices, captures and parcels, and to the way method invocations are stored.

Paweł Murias implemented multi dispatch as a Summer of Code project. Mildew now supports an impressive set of features, but since it is not very user oriented, I know of no projects that actually use mildew as a platform.

Other implementations

Elf development seems to have stalled. Pugs mostly sleeps, too, though Audrey updated it to work with the latest Haskell compilers. (It doesn't live in the Pugs repository anymore though, and is distributed by cabal, the Haskell package manager).

New in the field are Sprixel, a Perl 6 to Javascript compiler, and vill, an experimental LLVM backend to STD.pm+viv.

Test Suite

The test suite continued to grow; most tests have now been moved to t/spec/, the official Perl 6 test suite. Most tests in the other remaining files are either rather dubious, or rely on behaviour that's not officially specified (or are specific to an implementation).

Many new tests have been contributed by two new faces: Solomon Foster contributed a large number of tests for trigonometric functions on the various number types, and rational and complex numbers. Kyle Hasselbacher provided us with many regression tests for Rakudo which are also useful to other implementations.

Documentation

Bemoaning the fact that Perl 6 has nearly no user-level documentation, Carl Mäsak started u4x, User-Level Documentation for X-Mas. Hinrik Örn Sigurðsson chimed in, and started to write grok, a tool for retrieving and showing documentation, sponsored by the Google Summer of Code project.

Patrick Michaud, Jonathan Worthington, Carl Mäsak, Jonathan Scott Duff and Moritz Lenz started to work on a Perl 6 book, with a few chapters already being written.

Websites

In an attempt to provide an up-to-date link list, Moritz registered perl6-projects.org and collected links. Later Susanne "Su-Shee" Schmitt contributed a nice design, and Daniel Wright made the domain perl6.org available to us.

So we now have a community driven, central Perl 6 site at perl6.org.

Leo Lapworth redesigned perl.org, and also the old Perl 6 development page, and updated it a bit.

Blogs

As an attempt to improve the visibility of the Perl community, Matt S. Trout issued the Ironman Perl Blogging Challenge. So far it's a huge success, and quite a few hackers blog about Perl 6 there. Also the blog roll of the Planetsix Blog Aggregator continued to grow, some excellent new blogs were added in 2009.

Carl Mäsak blogged at least once per day in Novemeber, same procedure as least year :-)

IRC

The #perl6 IRC channel has been very pleasant and active in 2009, with three times the activity of 2008.

The Future

For April 2010 the Rakudo developers have planned a big release called Rakudo *, not feature complete but still useful and usable. Around the same time the new Perl 6 book will be released.

The specification is still evolving, and has some areas that are in need of implementation before they can evolve more; among them are macros, concurrency and IO.

Update: improved floating point example as per comment from Matthias.

Posted by Moritz Lenz | Permalink