Perl Foundation News: Embeding Perl into C++ Applications - Update Report

Leon just sent an update on his grant:

Past month I've been to YAPC::EU where I spoke about part of my project. I also used it to discuss some of the Windows related issues I ran into with others and they have helped me getting clear what's the problem. I haven't quite fixed it yet though. I also worked on refactoring and expanding the documentation, and fixed some bugreports (though one still open). I wanted to be finished with both points by September 1st, but due to some unexpected school obligations I haven't been able to do so. I've set myself a new deadline for October 1st.

logicLAB.org: CPAN Major release: Bundle::JONASBN renamed and refactored to Task::BeLike::JONASBN

The idea behind Bundles on CPAN is a very good. The implementation however relies somewhat on magic in the client. Adam Kennedy’s Task concept, solves this.

After reading the documentation and doing some considering I decided to deprecate Bundle::JONASBN and then redo the same distribution as Task::BeLike::JONASBN.

The project is in the same repository and Bundle::JONASBN was refactored to support this and the project site was renamed also. Many of the pointers are still using the internally used project key BJONASBN.

The Task concept is fairly easy and being a supporter of KISS, I am thinking about using this concept for other things currently on the drawing board.

Kudos should go to Adam Kennedy for the Task contribution to CPAN, it is simply beautiful in all it’s simplicity.

Task::BeLike::JONASBN has been released to CPAN as version 1.00 making it a major release on the same code base.

Task::BeLike::JONASBN and Bundle::JONASBN will live in parallel for an amount of time, but Bundle::JONASBN will be removed from CPAN eventually and all future development will be centered around Task::BeLike::JONASBN.

Modern Perl Books, a Modern Perl Blog: What's Going Right in Perl

Eighteen months after I started this site, the Modern Perl book is almost out, as is Using Perl 6. Perl 5.10.1, Perl 5.12.0, Perl 5.12.1, and (very nearly) Perl 5.12.2 are out, with Perl 5.14 coming next spring (not month, as I mistakenly typed). Spring means, of course, April or May 2011. (Sorry, southern hemisphere; you should elect a volunteer release manager.)

Rakudo Star has had two impressive releases, which brings Perl 6 to ever more people. The Parrot VM had its 1.0 and 2.0 and 2.3 and 2.6 releases, and if you want an order of magnitude performance improvement in Rakudo Perl 6, the Parrot 3.0 series will deliver some impressive gains through the Lorito reorganization.

I've spent a lot of time critiquing the Perl language and community and even more time critiquing perceptions of Perl from within and without. I'm glad to review what's going right. It's easy to make a list of great new features of Perl and the community, but I can limit it to my favorites, in terms of immediate and longer-term significance.

By no means are these the only important developments in Perl in the past 18 months; please by all means let us all know what you find most important in your own journals and talks and presentations.

  • The revitalization of Perl 5 core development has amazed me. Regular monthly releases have become boring, as they should be. A rotating series of release managers helps avoid the burnout that's claimed every Perl 5 pumpking up to this point, and it helps to make the process of releasing a new stable version of Perl 5 easier. That's why you can have confidence upgrading to Perl 5.12.2 when it comes out, and why in a year Perl 5.10.1 will look old.
  • I've wanted Plack for Perl for ages; I've long thought Python's WSGI is a good example of the "There should be one obvious way to do it philosophy" (it works much better for interfaces to well-defined problem domains than language features). The rapid adoption of Plack for so many web frameworks and libraries within Perl—as well as the number of backends supported by Plack—has solved many of the deployment problems of Perl web applications. It also allows greater collaboration on middleware, such as debugging and profiling tools.
  • Ancillary tools such as perlbrew and cpanminus have demonstrated that very simple interfaces devoted to solving the most common problems can improve the user experience immensely. I've known how to maintain my own user- and app-specific Perl 5 installations for years, but I've never wanted to maintain the morass of symlinks necessary to do so. Now I don't have to. Similarly, cpanminus lets me install CPAN modules often in the time it takes the official CPAN client to download the indexes.
  • Regular releases of the first Perl 6 distribution (Rakudo Star) demonstrate the power and consistency and disruptive potential of Perl 6 to even more people. Every month brings new features and improvements. Every bug report and new module written and benchmark help the development community make it an even better platform for new projects.
  • Schwern made the CPAN forkable through gitPAN, and the world is better for it. Public distributed version control for Perl 5 improves the experience of submitting changes, and public distributed version control for CPAN distributions has helped me submit and publish more changes too. Noticing a typo in documentation on search.cpan.org has become almost an enjoyable experience, if I can find the appropriate repository on Github, fork it, make my changes, and submit a pull request within five minutes. Often I can.

I look forward to several other projects in the world of Perl, such as Ryan Jendoubi's Ctypes for Perl 5 and the ongoing attempts in the Perl 5 core to rein in a sane set of functions for extensions to use. I've heard that various help forums have become more helpful and less abusive (especially #perl on irc.perl.org). I've even noticed a shift in how community members talk about marketing, especially as the discussion has changed from "Marketing? That's for those sick Java fans!" to "Hey, look at all of the cool stuff we're doing with Perl!"

We can and should still make improvements, but if the past year and a half is any guide, we can safely shift our cautious optimism for the present and future of Perl to regular optimism.

logicLAB.org: CPAN Feature Release Business::DK::CVR 0.06 – OOP follow-up

The CPAN module Business::DK::CPR has followed it’s cousin Business::DK::CVR and has been updated with an OOP implementation (see: the earlier announcement).

Apart from the class addition, the release only addresses minor issues, like a failing test case reported by CPAN-testers. POD has been updated and the Build file has been adjusted.

The Changes file does not contain all the changes, it is lacking a serious update so please refer to the project change log for details.

Source code is available from: http://logicLAB.jira.com/svn/BDKCPR/

Project homepage: http://logiclab.jira.com/browse/BDKCPR/

Wiki: http://logiclab.jira.com/wiki/display/BDKCPR/Home

Feedback and patches more than welcome,

Perl Foundation News: Hague Grant Acceptance: Meta-model Improvements and Natively Typed Attributes

I am pleased to announce that Jonathan Worthington's Hague Grant for Meta-model Improvements and Natively Typed Attributes has been accepted. Patrick Michaud has agreed to be the grant manager for this project.

I would like to thank everyone who provided feedback on this proposal.

Yet Another Perl Game Hacker (YAPGH): SDL 2.512 released - Layers Support added

The new release of SDL adds Layer support, which means that rendering surfaces by layers is a lot easier. SDL 2.512 has several other changes too. An experimental physics interface and minor fixes. Have a look at the CHANGELOG.


Here is an implementation of the new Layer stuff. Solitaire.



Ovid: Testing that darned filehandle

It wasn't fun finding out that some legacy code creates a bunch of filehandles on the fly to do logging.

package Some::Client;
use Some::Logging::Module;

# later ...
print MAGICLOGHANDLE $some_data;

That MAGICLOGHANDLE is dynamically generated and shoved into my namespace (along with a ton of other handles, one per log). So I needed to test what was being printed to it. Fortunately, Perl's dynamism makes this really, really easy.

Here's the basics:

use Test::Class::Most parent => 'My::Test::Class';

use Some::Client;
use IO::Scalar;

sub logging : Tests {
    my $test = shift

    my $SH = IO::Scalar->new(\my $data);
    no warnings 'redefine';
    local *Some::Client::MAGICLOGHANDLE = $SH;

    # run a bunch of code
    like $data, qr/whateever should be in the log/,
        '... I love diddling namespaces;
}

Yup. It's just that easy to hijack a filehandle for my own nefarious purposes.

A Curious Programmer: Perl Compiler Speed – It’s Fast!

A Curious Programmer

As a developer who is not responsible for many infrastructure modules, I shouldn’t really waste so much of time micro-benchmarking. Profiling my code after the fact should be sufficient if even that is necessary.

But it seems I’ve got into a bad habit.


I’m not the only person that thinks that MooseX loading all those pre-requisites is what is taking a lot of time. Dave Rolsky says in the latest Moose blog post:

What’s a bit sad, however, is that this only appears to save 6-10% of the compilation time in real usage. I’m not sure why that is, but I think the issue is that the perl core’s compilation time may be a big factor. As I said, loading that one Markdent modules loads a lot of modules (203 individual .pm files), and the raw overhead of simply reading all those files and compiling them may be a significant chunk of the compilation time.

That gets me thinking about the speed of the perl compilation step. Although using eval doesn’t have the same disk IO as use, it still exercises the compiler.

Loop Unrolling with Eval

Does anyone remember back in the day when we used to unroll loops? For example, if we had an instruction sequence that was do_something, add, jne and do_something was relatively short, you could save some time unrolling that to do_something, add, do_something, add, jne – you would only have to execute approximately half as many jne instructions across a loop lifetime.

That is something I never worry about in perl. Unless I have to come up with contrived examples for my blog.

sub eval_loop
{
    my $count = 0;
    my $s = '';
    for (my $i = 0; $i < 100_000; ++$i) {
        $s .= '$count += ' . $i . ";\n";
    }
    eval $s;
    $count;
}

sub vanilla_loop
{
    my $count = 0;
    for (my $i = 0; $i < 100_000; ++$i) {
        $count += $i;
    }
    $count;
}

my $code = '';
for (my $i = 0; $i < 100_000; ++$i) {
    $code .= '$count += ' . $i . ";\n";
}

$code = '
sub totally_unrolled {
    my $count = 0;' . "\n" . $code . '
    $count;
};';

eval $code;

eval_loop compiles the unrolled loop every time the subroutine is called whereas totally_unrolled is compiled prior to benchmarking.

I also wanted some partially unrolled loops for comparison sake. Obviously these examples are not suitable for production use. For example, consider what would happen if unrolled_loop_4 needed to execute a number of times that was not a multiple of 4.

sub unrolled_loop_2
{
    my $count = 0;
    for (my $i = 0; $i < 100_000; ++$i) {
        $count += $i;
        $count += ++$i;
    }
    $count;
}

sub unrolled_loop_4
{
    my $count = 0;
    for (my $i = 0; $i < 100_000; ++$i) {
        $count += $i;
        $count += ++$i;
        $count += ++$i;
        $count += ++$i;
    }
    $count;
}

Benchmarking

The full benchmarking code is as follows. I test the result of each function to ensure I’m getting the same answer.

use Benchmark qw(:hireswallclock);

use strict;
use warnings;

sub vanilla_loop
{
    my $count = 0;
    for (my $i = 0; $i < 100_000; ++$i) {
        $count += $i;
    }
    $count;
}

sub unrolled_loop_2
{
    my $count = 0;
    for (my $i = 0; $i < 100_000; ++$i) {
        $count += $i;
        $count += ++$i;
    }
    $count;
}

sub unrolled_loop_4
{
    my $count = 0;
    for (my $i = 0; $i < 100_000; ++$i) {
        $count += $i;
        $count += ++$i;
        $count += ++$i;
        $count += ++$i;
    }
    $count;
}

sub eval_loop
{
    my $count = 0;
    my $s = '';
    for (my $i = 0; $i < 100_000; ++$i) {
        $s .= '$count += ' . $i . ";\n";
    }
    eval $s;
    $count;
}

my $code = '';
for (my $i = 0; $i < 100_000; ++$i) {
    $code .= '$count += ' . $i . ";\n";
}

$code = '
sub totally_unrolled {
    my $count = 0;' . "\n" . $code . '
    $count;
};';

eval $code;

print vanilla_loop(), "\n";
print unrolled_loop_2(), "\n";
print unrolled_loop_4(), "\n";
print eval_loop(), "\n";
print totally_unrolled(), "\n";

Benchmark::cmpthese(-1, {
    'vanilla' => \&vanilla_loop,
    'unrolled-2' => \&unrolled_loop_2,
    'unrolled-4' => \&unrolled_loop_4,
    'eval' => \&eval_loop,
    'unrolled-max' => \&totally_unrolled,
});

And the results are:

$ perl unrolling.pl
4999950000
4999950000
4999950000
4999950000
4999950000
               Rate        eval  unrolled-2  unrolled-4     vanilla unrolled-max
eval         3.25/s          --        -93%        -95%        -95%         -96%
unrolled-2   48.1/s       1381%          --        -25%        -26%         -45%
unrolled-4   63.9/s       1865%         33%          --         -2%         -27%
vanilla      65.1/s       1902%         35%          2%          --         -26%
unrolled-max 87.7/s       2598%         82%         37%         35%           --

Conclusions

I am actually surprised that eval is as fast as it is. It runs less than 20 times slower than the vanilla loop, yet it has to construct a string by appending to it 100,000 times and run the compiler on the result. Or to put it another way, it compiles and runs 100,000 lines of code in under a third of a second. Good job perl compiler guys! Or did you put something in there for patholgically ridiculous examples like this?

And I was right never to worry about unrolling my loops. unrolled-2 is quite significantly slower than vanilla. As I can’t think of any good reason for that, it makes me worried that I’ve done something wrong.

Even fully unrolling the loop, which is quite ridiculous gives me a relatively tiny 35% speed increase.



Filed under: Perl Tagged: benchmarking

dagolden: How to script git with perl and Git::Wrapper

If you work with git, eventually you will want to script something that git doesn't do already and can't be accomplished with an alias. If you use Perl, the Git::Wrapper module makes scripting git easy.

The big advantage of Git::Wrapper is that it wraps the binary version of git, which keeps your perl and git independent. Upgrade perl and not git? Add Git::Wrapper and you're done. Upgrade git while keeping your existing perl? No problem. If you get your git from an OS distribution package, but roll your own perl, then Git::Wrapper is for you!

Git::Wrapper just makes it really easy to pass git commands and get back useful data. It really shines on "git log" commands, as it parses each commit message into Git::Wrapper::Log objects with accessors for id, author, date, and message. For everything else, it's not much more than a wrapper around qx(), but it returns arrays of lines and throws exceptions when things go wrong, which just saves a little time and code.

Here is a short example that I whipped up recently to automate the process of adding a github repository remote for someone when they send me a pull request. (It also uses my Getopt::Lucid module, but as the name suggests, it should be clear what that does.)

#!/usr/bin/env perl
use 5.010;
use strict;
use warnings;
use autodie;
use Git::Wrapper;
use Getopt::Lucid qw/:all/;

# USAGE: github-remote [-r REPO] [NAME]
#     NAME -- a github user name (or defaults to 'dagolden')
#     REPO -- a github repository path (or is parsed out of my private origin repo)

my $opts = Getopt::Lucid->getopt([
  Param('repo|r'),
]);

my $who = shift @ARGV;

# Get a wrapper for the current directory
my $git = Git::Wrapper->new(".");

# Locate the origin repository URL
my ($origin) = grep { /origin/ } $git->remote("-v");
die "Couldn't determine origin\n" unless $origin;
$origin =~ s/^origin\s+//;
$origin =~ s/\s+\(.*$//;

# Parse out the repository path
my $repo = $opts->get_repo;
unless ( $repo ) {
  # me@git.example.com:my-repo-name.git
  ($repo) = $origin =~ m/^[^:]+:(.+)$/;
}

# Set up someone else's github repo
if ( $who ) {
  $git->remote("add", $who, "git://github.com/${who}/${repo}");
}
# Or set up my own github mirror
else {
  $git->remote("add", "github", "git\@github.com:dagolden/${repo}");
}

That's it. I admit the program is a little crufty because of how I manage my repositories with a private origin that I mirror to a github repository, but there's almost no cruft around interactions with git. Consider this:

  $git->remote("add", $who, "git://github.com/${who}/${repo}");

If the command fails (if there's already $who as a remote name), an exception will be thrown. That means I don't have to write any error handling myself, which is perfect for a simple automation program like this one.

transfixed but not dead!: given/when – the Perl switch statement

transfixed but not dead! » perl

Its been a while since my last post; summer holidays have inflicted upon me some big distractions! So to ease myself back in gently I’ll I’ve pulled out a (nearly finished but unposted) article from last year (in true Blue Peter tradition!)

This post was inspired by this nice & straightforward blog article: How A Ruby Case Statement Works And What You Can Do With It. I’ve simply converted the Ruby code to Perl and added some so called insightful notes :)

Now what Ruby calls case/when statements is more commonly known in computer parlance has the switch statement.

Perl itself has had such a switch statement for quite a while now:

use Switch;

switch ($grade) {
    case "A"   { say "Well done!" }
    case "B"   { say "Try harder!" }
    case "C"   { say "You need help!!!" }
    else         { print "You just making it up" }
}

However DON’T USE IT!!! Its a source filter and is being deprecated.

Instead use given/when which is the Perl 6 switch statement that was introduced at perl 5.10.

use strict;
use warnings;
use feature qw(switch say);

print 'Enter your grade: ';
chomp( my $grade = <> );

given ($grade) {
    when ('A') { say 'Well done!'       }
    when ('B') { say 'Try harder!'      }
    when ('C') { say 'You need help!!!' }
    default { say 'You are just making it up!' }
}

And at perl 5.12 you could also use when has a statement modifiers:

use 5.012;
use warnings;

print 'Enter your grade: ';
chomp( my $grade = <> );

given ($grade) {
    say 'Well done!'        when 'A';
    say 'Try harder!'       when 'B';
    say 'You need help!!!'  when 'C';
    default { say 'You are just making it up!' }
}

And even more enhancements are coming with perl 5.14 because given now returns the last evaluated expression:

use 5.013;  # 5.014
use warnings;

print 'Enter your grade: ';
chomp( my $grade = <> );

say do {
    given ($grade) {
        'Well done!'        when 'A';
        'Try harder!'       when 'B';
        'You need help!!!'  when 'C';
        default { 'You are just making it up!' }
    }
};

Which can lead to some very pleasant looking dry code.

But there is more to given/when than just single value conditionals. You can also check multi-values and ranges:

use 5.012;
use warnings;

print 'Enter your grade: ';
chomp( my $grade = <> );

given ($grade) {
    say 'You pretty smart'  when ['A', 'B'];
    say 'You pretty dumb!!' when ['C', 'D'];
    default { say "You can't even use a computer" }
}

And also pattern match using a regex:

use 5.012;
use warnings;

print 'Enter some text: ';
chomp( my $some_string = <> );

given ($some_string) {
    say 'String has numbers'  when /\d/;
    say 'String has letters'  when /[a-zA-Z]/;
    default { say "String has no numbers or letters" }
}

In fact you can use anything that Smart Matching can resolve.

And because smart matching is used under the hood by given/when then you can customise it by overloading the ~~ smart match operator:

use 5.012;
use warnings;

{
    package Vehicle;
    use Moose;
    use overload '~~' => '_same_wheels', fallback => 1;

    has number_of_wheels => (is => 'rw', isa => 'Int');

    sub _same_wheels {
        my ($self, $another_vehicle) = @_;
        $self->number_of_wheels == $another_vehicle->number_of_wheels;
    }
}

my $four_wheeler =  Vehicle->new( number_of_wheels =>  4 );
my $two_wheeler  =  Vehicle->new( number_of_wheels =>  2 );
my $robin_reliant = Vehicle->new( number_of_wheels =>  3 );

print 'Enter number of wheel for vehicles: '; chomp( my $wheels = <> );
my $vehicle = Vehicle->new( number_of_wheels => $wheels );

given ($vehicle) {
    say 'Vehicle has the same number of wheels as a two-wheeler!'
        when $two_wheeler;

    say 'Vehicle has the same number of wheels as a four-wheeler!'
        when $four_wheeler;

    say 'Gosh... is that Del Trotter!!'
        when $robin_reliant;

    default {
        say   "Don't know of a vehicle with that wheel arrangement!"
    }
}

Hopefully that was a nice lightweight introduction to given/when. And I haven’t even touched upon break or continue :)

/I3az/


Modern Perl Books, a Modern Perl Blog: Backporting Features

One current discussion on p5p is the namespace of fixed versions of reftype(), refaddr(), and blessed(), recently added to bleadperl.

These functions are part of Scalar::Util right now. Unfortunately, they return undef for non references. This leads to code like:

my $ref_type = reftype( $maybe_ref );
do_something_with_ref( $ref_type ) if defined $ref_type;

(If you don't follow all of the reasons why you have to do that, that's good; you haven't imagined all of the odd and strange ways people might name their classes in various parts of the Perl 5 internals and DarkPAN modules. Safety dictates finding these corner cases.)

Anyhow.

Making these fixed functions available in the core without having to load Scalar::Util offers some advantages but also some disadvantages. It's the "You can't add new keywords to Perl 5" problem. What if someone else defined functions of that name?

Worse, what if someone loads Scalar::Util and imports its functions and expects those semantics instead of the new semantics?

(Apparently it's impossible to detect the declaration or importation of symbols with conflicting names so as to produce warnings or exceptions in this case, though I don't see why. If we were in the habit of declaring the version of Perl 5 our code uses, we wouldn't have these problems.)

The discussion inevitably circled back to the ineffable question of "If this new feature is going in the core, how do people use it in previous releases of Perl 5?" That is, "You've added a new feature to Perl 5 which will be in Perl 5.14 released next year. Is there any way to backport that feature to Perl 5.12? How about Perl 5.10? You know, RHEL 3 is still around. Why do you hate Perl .5.8?"

At some point the weight of ensuring that code written for a future version of Perl 5 can run correctly (if you cram in enough flying buttresses of CPAN shims) on code written for obsolete versions of Perl 5 will pull down the entire edifice. Allowing code written for previous versions of Perl 5 to run on modern versions unmodified (or with a simple use 5.10; at the top—I know a great combination of find and sed which can perform this kind of textual manipulation {and if you're on Windows, use PPT to get portable versions of these tools}) is often good and useful.

Going the other way... well, I don't understand it. If it's possible and someone wants to do it, fine. Why should it block improving the next release of Perl 5 though?

Ovid: Prime Factors in Perl 6

There's really nothing extraordinary about this and certainly nothing specific to Perl 6, but I really liked how it felt writing code to find prime factors. I do note, however, that the habit of using a dash instead of an underscore in identifiers does slow down the Perl 6 to Perl 5 translation (since I'm doing that manually). Hopefully we won't need to keep doing that translation in the future.

sub prime-factors(Int $number is copy) {
    # don't try to factor prime numbers
    return { $number => 1 } if is-prime($number);

    my %factors;
    for @PRIMES -> $prime-number {
        last if $prime-number ** 2 > $number;
        while $number % $prime-number == 0 {
            %factors{$prime-number} //= 0;
            %factors{$prime-number}++;
            $number /= $prime-number;
        }
    }
    %factors{$number}++ if $number != 1;  # we have a prime left over
    return %factors;
}

for 17, 53, 90, 94, 200, 289, 62710561 -> $number {
    say "Prime factors of $number are: ", prime-factors($number).perl;
}

The is-prime sub merely relies on a list of the first 1000 prime numbers (kept in the @PRIMES array). It was much faster that way. If you desperately must see it, it's in the extended part of the blog entry.

It takes almost 7 seconds to run on my MacBook Pro, but .007 under Perl 5. I think I'll keep this one around for a while to see Rakudo benchmark improvements. As usual, code improvements welcome.

And the boring bit:

my @PRIMES = <2 3 5 7 11 13 17 19 23 29 31 37 41 43 47 53
59 61 67 71 73 79 83 89 97 101 103 107 109 113 127 131 137 139 149 151 157 163
167 173 179 181 191 193 197 199 211 223 227 229 233 239 241 251 257 263 269
271 277 281 283 293 307 311 313 317 331 337 347 349 353 359 367 373 379 383
389 397 401 409 419 421 431 433 439 443 449 457 461 463 467 479 487 491 499
503 509 521 523 541 547 557 563 569 571 577 587 593 599 601 607 613 617 619
631 641 643 647 653 659 661 673 677 683 691 701 709 719 727 733 739 743 751
757 761 769 773 787 797 809 811 821 823 827 829 839 853 857 859 863 877 881
883 887 907 911 919 929 937 941 947 953 967 971 977 983 991 997 1009 1013 1019
1021 1031 1033 1039 1049 1051 1061 1063 1069 1087 1091 1093 1097 1103 1109
1117 1123 1129 1151 1153 1163 1171 1181 1187 1193 1201 1213 1217 1223 1229
1231 1237 1249 1259 1277 1279 1283 1289 1291 1297 1301 1303 1307 1319 1321
1327 1361 1367 1373 1381 1399 1409 1423 1427 1429 1433 1439 1447 1451 1453
1459 1471 1481 1483 1487 1489 1493 1499 1511 1523 1531 1543 1549 1553 1559
1567 1571 1579 1583 1597 1601 1607 1609 1613 1619 1621 1627 1637 1657 1663
1667 1669 1693 1697 1699 1709 1721 1723 1733 1741 1747 1753 1759 1777 1783
1787 1789 1801 1811 1823 1831 1847 1861 1867 1871 1873 1877 1879 1889 1901
1907 1913 1931 1933 1949 1951 1973 1979 1987 1993 1997 1999 2003 2011 2017
2027 2029 2039 2053 2063 2069 2081 2083 2087 2089 2099 2111 2113 2129 2131
2137 2141 2143 2153 2161 2179 2203 2207 2213 2221 2237 2239 2243 2251 2267
2269 2273 2281 2287 2293 2297 2309 2311 2333 2339 2341 2347 2351 2357 2371
2377 2381 2383 2389 2393 2399 2411 2417 2423 2437 2441 2447 2459 2467 2473
2477 2503 2521 2531 2539 2543 2549 2551 2557 2579 2591 2593 2609 2617 2621
2633 2647 2657 2659 2663 2671 2677 2683 2687 2689 2693 2699 2707 2711 2713
2719 2729 2731 2741 2749 2753 2767 2777 2789 2791 2797 2801 2803 2819 2833
2837 2843 2851 2857 2861 2879 2887 2897 2903 2909 2917 2927 2939 2953 2957
2963 2969 2971 2999 3001 3011 3019 3023 3037 3041 3049 3061 3067 3079 3083
3089 3109 3119 3121 3137 3163 3167 3169 3181 3187 3191 3203 3209 3217 3221
3229 3251 3253 3257 3259 3271 3299 3301 3307 3313 3319 3323 3329 3331 3343
3347 3359 3361 3371 3373 3389 3391 3407 3413 3433 3449 3457 3461 3463 3467
3469 3491 3499 3511 3517 3527 3529 3533 3539 3541 3547 3557 3559 3571 3581
3583 3593 3607 3613 3617 3623 3631 3637 3643 3659 3671 3673 3677 3691 3697
3701 3709 3719 3727 3733 3739 3761 3767 3769 3779 3793 3797 3803 3821 3823
3833 3847 3851 3853 3863 3877 3881 3889 3907 3911 3917 3919 3923 3929 3931
3943 3947 3967 3989 4001 4003 4007 4013 4019 4021 4027 4049 4051 4057 4073
4079 4091 4093 4099 4111 4127 4129 4133 4139 4153 4157 4159 4177 4201 4211
4217 4219 4229 4231 4241 4243 4253 4259 4261 4271 4273 4283 4289 4297 4327
4337 4339 4349 4357 4363 4373 4391 4397 4409 4421 4423 4441 4447 4451 4457
4463 4481 4483 4493 4507 4513 4517 4519 4523 4547 4549 4561 4567 4583 4591
4597 4603 4621 4637 4639 4643 4649 4651 4657 4663 4673 4679 4691 4703 4721
4723 4729 4733 4751 4759 4783 4787 4789 4793 4799 4801 4813 4817 4831 4861
4871 4877 4889 4903 4909 4919 4931 4933 4937 4943 4951 4957 4967 4969 4973
4987 4993 4999 5003 5009 5011 5021 5023 5039 5051 5059 5077 5081 5087 5099
5101 5107 5113 5119 5147 5153 5167 5171 5179 5189 5197 5209 5227 5231 5233
5237 5261 5273 5279 5281 5297 5303 5309 5323 5333 5347 5351 5381 5387 5393
5399 5407 5413 5417 5419 5431 5437 5441 5443 5449 5471 5477 5479 5483 5501
5503 5507 5519 5521 5527 5531 5557 5563 5569 5573 5581 5591 5623 5639 5641
5647 5651 5653 5657 5659 5669 5683 5689 5693 5701 5711 5717 5737 5741 5743
5749 5779 5783 5791 5801 5807 5813 5821 5827 5839 5843 5849 5851 5857 5861
5867 5869 5879 5881 5897 5903 5923 5927 5939 5953 5981 5987 6007 6011 6029
6037 6043 6047 6053 6067 6073 6079 6089 6091 6101 6113 6121 6131 6133 6143
6151 6163 6173 6197 6199 6203 6211 6217 6221 6229 6247 6257 6263 6269 6271
6277 6287 6299 6301 6311 6317 6323 6329 6337 6343 6353 6359 6361 6367 6373
6379 6389 6397 6421 6427 6449 6451 6469 6473 6481 6491 6521 6529 6547 6551
6553 6563 6569 6571 6577 6581 6599 6607 6619 6637 6653 6659 6661 6673 6679
6689 6691 6701 6703 6709 6719 6733 6737 6761 6763 6779 6781 6791 6793 6803
6823 6827 6829 6833 6841 6857 6863 6869 6871 6883 6899 6907 6911 6917 6947
6949 6959 6961 6967 6971 6977 6983 6991 6997 7001 7013 7019 7027 7039 7043
7057 7069 7079 7103 7109 7121 7127 7129 7151 7159 7177 7187 7193 7207 7211
7213 7219 7229 7237 7243 7247 7253 7283 7297 7307 7309 7321 7331 7333 7349
7351 7369 7393 7411 7417 7433 7451 7457 7459 7477 7481 7487 7489 7499 7507
7517 7523 7529 7537 7541 7547 7549 7559 7561 7573 7577 7583 7589 7591 7603
7607 7621 7639 7643 7649 7669 7673 7681 7687 7691 7699 7703 7717 7723 7727
7741 7753 7757 7759 7789 7793 7817 7823 7829 7841 7853 7867 7873 7877 7879
7883 7901 7907 7919>;
my %IS-PRIME = map {; $_ => 1 }, @PRIMES;

sub is-prime(Int $number) {
    return False  if $number < 2; # special case
    die "$number is too large" if $number > @PRIMES[*-1]**2;
    return %IS-PRIME{$number};
}

See, I told you that bit was boring :)

The Effective Perler: Use when() as a statement modifier

Perl 5.10 introduced the given-when statement, and Perl 5.12 refines it slightly by letting you use the when as a statement modifier.

A statement modifier puts the conditional expression at the end of the statement (see perlsyn). You’ve probably already used many of these:

print "It's too darned hot!\n" if $city eq 'Baltimore';
print "I'm this far!\n" unless $Quiet;
print "An element is $_.\n" foreach @array;
print "Found cat at @{[pos]}\n" while /cat/g;
do { $count++; $i_am_bored = int rand 2 } until $i_am_bored;

In Perl 5.10, you have to put the when first, put parentheses around the conditional expression, and follow it with a block:

use 5.012;

my %microchips = (
	  'Mimi'   => 123,
	  'Buster' => undef,
	  'Roscoe' => 345,
	);
my @array = ( 5 .. 7 );

foreach ( 1 .. 10, qw(Mimi Buster) ) {
	when( @array )      { say "$_: in array" }
	when( %microchips ) { say "$_: in hash"  }
	}

Perl 5.12 allows you to use the when as a statement modifier instead, which means that you can put the when after the expression that you want to run:

use 5.012;

...;

foreach ( 1 .. 10, qw(Mimi Buster) ) {
	say "$_: in array" when( @array );
	say "$_: in hash"  when( %microchips );
	}

Just like you can with the other statement modifiers, you can omit the parentheses around the condition:

use 5.012;

...;

foreach ( 1 .. 10, qw(Mimi Buster) ) {
	say "$_: in array" when @array;
	say "$_: in hash"  when %microchips;
	}

As a statement modifier, the expression before the when has an implicit break when you use it with given or an implicit next when you use it with foreach. That is, once a when condition evaluates to true, Perl doesn’t continue with the rest of the block.

The foreach case actually looks like this:

use 5.012;

...;

foreach ( 1 .. 10, qw(Mimi Buster) ) {
	do { say "$_: in array"; next } when @array;
	do { say "$_: in hash";  next } when %microchips;
	}

And the given case actually looks like:

use 5.012;

...;

given ( $cat ) {
	do { say "$_: in array"; break } when @array;
	do { say "$_: in hash";  break } when %microchips;
	}

You’re still supposed to use when inside a topicalizer (e.g. foreach or given). You might be tempted to use it more liberally so you can take advantage of its implicit smart matching anywhere that you like. A curious and perhaps unintended parsing makes it a runtime error instead of a compilation error:

use 5.012;

my %microchips = (
	  'Mimi'   => 123,
	  'Buster' => undef,
	  'Roscoe' => 345,
	);

while( <STDIN> ) {
	chomp;
	# this works (without a warning) as long as the value in $_
	# is not a hash key
	say "Found cat with id [$microchips{$_}]" when %microchips;
	}

The while isn’t a topicalizer, even though in this particular idiom it sets $_ for you. You get the output along with a warning:

$ perl5.12.1 no-topicalizer.pl
Buster
Found cat with id []
Can't use when() outside a topicalizer at test line 11, <STDIN> line 1.

However, you don’t get a fatal error when the condition is false, which is probably another bug (filed as RT #77510). There’s no fatal error, no warning message, and the output shows that Perl kept going:

Ella
Mimi
Found cat with id 123
Can't use when() outside a topicalizer at test line 11, <STDIN> line 2.

Just because it it does just what you think it should do then dies telling you it doesn’t do that, don’t think you should do it. You could use eval to catch the fatal error, but that’s a kludge that will only work until the Perl developers fix the problem. Besides, if you are going to go that far, it’s just as easy to make the smart match explicit with an if statement modifier (and that is even backward compatible with 5.10):

use 5.010;

my %microchips = (
	  'Mimi'   => 123,
	  'Buster' => undef,
	  'Roscoe' => 345,
	);

while( <STDIN> ) {
	chomp;
	say "Found cat with id $microchips{$_}" if $_ ~~ %microchips;
	}

The only question now is how long you will be able to use this before your Perl::Critic policies update to forbid it, whether inside or outside of a topicalizer. Use it while you can: time is running out.

Post to Twitter Post to Delicious Post to Digg Post to Facebook Post to Reddit

Dereferenced.com: Perl Module Release: Image-Size 3.230

Version: 3.230

Released: Sunday August 29, 2010, 04:00:00 PM -0700

Changes:

  • imgsize
  • lib/Image/Size.pm

perlcritic clean-ups from new rules.

  • lib/Image/Size.pm
  • t/Test_emf_small.emf (added)
  • t/all.t

RT #59995: Added support for Windows Enhanced Metafile Format (EMF).

  • t/00_load.t (deleted)
  • t/01_pod.t (deleted)
  • t/02_pod_coverage.t (deleted)
  • t/03_meta.t (deleted)
  • t/04_minimumversion.t (deleted)
  • t/05_critic.t (deleted)
  • xt/00_load.t (added)
  • xt/01_pod.t (added)
  • xt/02_pod_coverage.t (added)
  • xt/03_meta.t (added)
  • xt/04_minimumversion.t (added)
  • xt/05_critic.t (added)

Move the author/distro-sanity tests to an “xt” directory.

Damien Learns Perl: New Android blog!

Hi all!
Just a quick word to tell you about my new blog venture on Android Application programming. DamienLearnsPerl is not dead, just transfixed ;)
Please check out "Do Androids Dream of Google Apps" for the fun!



Perl Foundation News: New mailing address for TPF.

The Perl Foundation has a new mailing address! We are working to get contact information on all of our sites updated. Please update your records:

Yet Another Society d/b/a The Perl Foundation
340 S LEMON AVE #6055
WALNUT, CA 91789
UNITED STATES

schwern's Journal: A month of Test::Builder2

schwern's Journal

I've had a grant open for Test::Builder2 for, oh god over two years now. Since I started it, Perl 6 has had a release! I think its the second oldest running dev grant.

I've cleared the decks of other responsibilities and can dedicate September to, if not finishing, then at least releasing something people can poke at. First alpha release was supposed to be "two weeks after the start" ha ha ha! oh god. The design has evolved and simplified greatly in the intervening two years, but its time to get something the hell out the door. At least a Test::Builder2 Star if you will.

There's critical components missing. There's no diagnostics, YAML or otherwise. The issues with nested asserts are still congealing. Plans are not enforced. Result objects are in the middle of being remodeled... again. But Test::Builder is using what parts of Test::Builder2 are usable. Multiple output formats and streams work. Asserts can be nested in the common, simple cases without having to fiddle with $Level. And you can hook into various events.

Step one is I'm going to seal up what's there, write docs where they're missing, and release something.

A release before October or the grant dies.

The Effective Perler: Perl 5.14 Items coming soon

The Perl 5 Porters are currently working on Perl 5.13, the development track that will end up as Perl 5.14. By reading the perldelta513* documentation, you can get a peek at what will mostly likely be in the next maintenance version of Perl. You need to read each of the perldelta5* in a development series because they only document the changes in to the previous development release, not the cumulative changes since the last major release.

Some things you’ll find include:

  • A non-destructive substitution with the /r flag: s///r
  • New ways to specify binary, octal, and hexadecimal numbers.
  • given returns its last evaluated expression (like if-elsif already does).
  • srand returns its seed so you can reuse it.

Post to Twitter Post to Delicious Post to Digg Post to Facebook Post to Reddit

John Napiorkowski’s Perl Development Blog: Recent CPAN activity

In no particular order:

This week released two version of Test::DBIx::Class.  These were both minor changes intended to try to reduce the number of test failures reported.  To that end I am now forcing installation of a more recent version of Data::Dumper.  Although I am not using that myself, I do use Test::Differences which relies on Data::Dumper, but since Test::Differences doesn't currently specify a version of Data::Dumper it is possible that if you have an older, buggy Data::Dumper you'd get meaningless test failures.  I submitted a bug to Test::Differences; hopefully we can fix the problem at the true site of its cause, but for now my updates should make it easier to install Test::DBIx::Class.

I Also made some minor changes related to making sure the code runs on a modern version of Moose.  So if you have trouble, please let me know :)

Also released was a significant update to Catalyst::Controller::ActionRole, which leverages some updates in modern versions of Catalyst so that it now properly matches the functionality in Catalyst::Controller.  Based on this change we were able to release an update to CatalystX::Declare which uses Catalyst::Controller::ActionRole as its base controller.  Doing this we can now support parameterization in action roles much more cleanly.  Example to follow after I play with this a bit more :)

Given the new Catalyst::Controller::ActionRole I was finally able to release Catalyst::ActionRole::BuildDBICResult  which is an action role intended to reduce effort in mapping incoming arguments to rows in your DBIx::Class managed database.  Lastly, I wanted to learn more about Plack, so did a quick debug panel: Plack::Middleware::Debug::W3CValidate

All in all  it was a reasonable productive week for me CPAN wise.   Next up will hopefully be more work on the Plack Catalyst branch.

Thanks!

Read and post comments | Send to a friend

Perlbuzz: Perlbuzz news roundup for 2010-08-27

These links are collected from the Perlbuzz Twitter feed. If you have suggestions for news bits, please mail me at andy@perlbuzz.com.

brian d foy: The Foursquare gaming code challenge

Mayank Lahiri writes a Foursquare checkin tool in nine Perl statements, and Alex Ford says he has a two line solution, where line means statement, and statement. Part of the reason their solutions look so ugly is that they want to do it with the Standard Library. I was slightly interested in this because someone had asked Randal if he was checking in with Perl (he is not).

Larry Wall once said that he could program anything in one-line of Perl given a sufficiently long line. That is, in a language where lines aren't significant, line count is stupid. If you want to count lines, Python is probably your language.

Faking a Foursquare checking is not straightforward with LWP, which actually makes this problem a little more interesting. The Foursquare API uses either Basic or OAuth authorization, but it doesn't think about web browsers. It figures that your application doesn't need a prompt (i.e. a 403 response) for an authorization header, so there is no authorization realm. This means that LWP's credentials method is useless. However, I can use the add_handler method to fix up the request to force the Authorization header:

use LWP::UserAgent;

my $ua = LWP::UserAgent->new;

# The Foursquare API doesn't have a 403 response, so it has
# no realm. This means I have to force the authorization
# header.
$ua->add_handler( 
    request_prepare => sub { 
        my( $request, $ua, $h ) = @_;
        $request->authorization_basic( @ENV{ qw(FOURSQUARE_USER FOURSQUARE_PASS) } );
        },
    );

$ua->post( 
    'http://api.foursquare.com/v1/checkin', 
    Content => {
        vid     => $ARGV[0] || 8212918,  # The Perl Review
        private => 0,
        geolat  => $ARGV[1] ||  42.01974,
        geolong => $ARGV[2] || -87.67446,
        }
    );

My script isn't exactly the same as theirs, though. I'll let cron handle the other details.

brian d foy: I'm going to Tokyo in December

I'm going to be in Tokyo from December 5 to 10. I realized that if I could make it to somewhere in Asia before the end of the year, I will have been on every continent in 2010. Not only that, I hear the Shibuya Perl mongers not only have the largest YAPC, but also have to turn people away just because its so popular (tickets are on sale now). I want to see what's going on there, even if I can't show up in the right month.

So far I have no plans, although Karen Pauley might find something for me to do. So far she and Marty are the only people I know in Tokyo, but I'm hoping to change that. I'll come to social meetings, give a talk, give out copies of my latest book, or whatever.

Maybe we should even have some sort of exchange program to connect the Asian and American/European Perl programs.

brian d foy: YAPC::EU 2010 videos are on YouTube

The YAPC::EU 2010 team has uploaded to YouTube videos of the conference in the yapceu2010 channel. There's even one of me presenting the White Camel Awards to cog and to Barbie. I don't think I'm terribly interesting, but I like their comments as they accept their awards.

Paul Fenwick couldn't be there, but maybe someone can convince him to do a video acceptance speech. He's dressed as a Star Fleet officer, a pirate, and a mad scientist. What should he be in the video?

Clinton Gormley: ElasticSearch::AnyEvent pre-release available on github

I've just pushed ElasticSearch::AnyEvent - this brings async requests to the Perl ElasticSearch client. It is still lacking proper documentation and any tests, which will soon follow.

It is available on the anyevent branch on github

This is my first foray into async programming in Perl, so I'd appreciate feedback on the API and code.

Briefly, it can be used as follows:

use ElasticSearch::AnyEvent();
 my $es = ElasticSearch::AnyEvent->new(servers=>'127.0.0.1:9200');

 # Blocking

     my $cv = $es->current_server;
     print $cv->recv;

     # or

     print $es->current_server->recv

 # Callback

     my $cv = $es->current_server;
     $es->cb(sub {
         my $result = shift || die $@;
         ....
     });

 # Context

     my $cv = $es->current_server;
     undef $cv;                      # cancels

     $es->refresh_servers;           # fire-and-forget
     start_event_loop();

     {
        my $cv1 = $es->foo;
        my $cv2 = $es->foo;
        $cv2->cb(...);
     }
     # $cv1 is cancelled
     # $cv2 is backgrounded / fire-and-forget

 # Multitask:

     $es->multi_task(
         action      => 'index',
         pull_queue  => sub {
             # returns a list of HASHes to be passed to $es->$action(...)
             # can return (eg) 1000 at a time
         },
         on_success  => sub {                # optional
             my ($args,$result) = @_;
             ...
         },
         on_error    => sub {                # optional
             my ($error,$args,$queue) = @_;
             ...
         },
     )->recv || die "Error";

The Cattle Grid: RHPS at YAPC::Europe 2010

YAPC::Europe 2010 is well over, and now... we have the video of our (as organizers) interpretation of some parts of the Rocky Horror Picture Show. If you were in Pisa and already saw it, you'll likely won't like to watch it again. On the other hand, if you weren't there, you might want to take a look: http://www.perl.it/blog/archives/000675.html It's at your risk. And... well... we did it because we auctioned it in Lisbon the year before....

ldami: plat_forms 2010/2011 : call for Perl teams

A new edition of the Plat_Forms programming contest is going to take place in end 2010 or beginning of 2011 : see http://www.plat-forms.org/2010/plat-forms-in-2010 .

The first edition took place in 2007 and our Geneva team on Catalyst was winner of the Perl track (http://use.perl.org/~dami/journal/33559 ). Unfortunately we will not be able to participate again this time; but I wish that another Catalyst team will be present. Reports from that contest got significant coverage in specialised press , and can play an important role in propagating a positive image of Perl. See for example the August 2010 issue of the prestigious "Communications of ACM" (http://cacm.acm.org/magazines/2010/8/96637-platforms-is-there-one-best-web-development-technology/abstract , full text available at http://page.mi.fu-berlin.de/prechelt/Biblio/platforms07-cacm-2010.pdf ); the conclusion reads :

Is Perl outdated? No. In contrast, the small size of the solutions and their good modifiability suggest that Perl may be a particularly strong platform with respect to maintainability.

So, Perl programmers, please build up teams to participate in the 2010/2011 edition!

Perl Hacks: Net::Songkick

Sometimes it's good to just take a new idea and hack on it for a couple of hours to see what happens. That's what I've done this evening.

I've been using Songkick for a while. Songkick is a web site that tracks users' attendance at gigs. I've been tracking the gigs I've been going to as well as trying to fill in over thirty years of old gigs.

I've known for some time that Songkick has an API, but until yesterday the API has been for invited partners only. But yesterday on Get Satisfaction they announced that it was now public.

The public API doesn't do that much yet. There are only four documented calls. But it's already useful and I'm sure it will gain more functionality quickly.

And this evening I've spent some time writing a very simple Net::Songkick module. This is really just a proof of concept, but I've got big plans for improving it. And I can already write a useful program like this:

#!/usr/bin/perl

use strict;
use warnings;

use XML::LibXML;
use Net::Songkick;

my $user = shift || 'davorg';
my $sk = Net::Songkick->new({
  api_key => $ENV{SONGKICK_API_KEY}
});

my $xml = $sk->get_upcoming_events({
  user => $user,
});

my $xp = XML::LibXML->new->parse_string($xml);

foreach ($xp->findnodes('//event/@displayName')) {
  print $_->to_literal, "\n";
}

If you have any interest in gigs then I highly recommend Songkick to you. And if you like attending gigs and hacking Perl (although not necessarily at the same time) then why not give Net::Songkick a try. All you'll need is an API key from Songkick.

There's a Github repository too; if you feel like hacking on it...

Shlomi Fish: Perl Debugger Tip: A Session Startup File

Here's a small Perl debugger tip: in order to have a file whose commands will be executed at the start of the debugging session (for example in order to get to a certain point in the code) - a session startup file, similar to gdb's --command=cmds.gdb flag - one can do the following:

First of all write a file with the debugger commands you want (let's call it cmds.perldb) and then when inside the perl debugger say:

source cmds.perldb

This will execute all the commands.

I noticed that after a while the perl debugger stores it inside the history (assuming you're using Term-ReadLine-Gnu), so you can recall it with s and then pressing the history-search-forward and history-search-backward that were set up (Mandriva assigns them to Page Up and Page Down).

Enjoy and I'm sorry for having neglected this blog for a long while!

Sawyer X: Help newbies, your project and the community - README

If you want newbies to help you with a project - in fixing code, writing tests or even documentation - my course students will be given a course completion project to help out an open source project. That project can be yours!

Background:
I've been very inspired by DJB (Daniel J. Bernstein) and his course on UNIX Security Holes. The highlight to me was how his students interacted with the outside world - something I personally never did while studying programming (or anything else, for that matter) at school. I saw two of my brothers go through their degree (computer science, biology), helped with their homework and even attended some classes. They too never interacted with the outside world! I decided I want this changed in my course.

The project:
There are about seven lessons left to the semester in which the students will have to do their course completion project. I've decided that project will be on working with the community. It will entail either forming a project (which will be part of the "extra-credit" group) or join an existing project and helping with anything. This covers: code refactoring, reporting bugs, fixing bugs, documentation and tests. Any of those, for any project, is acceptable. They will also be advised to open a blog (either in blogs.perl.org, dreamwidth, use.perl.org or even blogger) and report their progress through the month/month and a bit. I'll try and run an aggregator for these blogs.

This will help them achieve better - practical - understanding of Perl, programming and cooperation. It will help them learn more and improve their skills. It will help start their resumes and proven practical experience. It will let them interact with the outside world and contribute something that will be of use to others. Overall, important and exciting, in my opinion.

Where you come in:
Instead of having the students roam around CPAN for modules they think might be good to work with, I'd like to showcase them existing projects to work on. Projects that are in progress (and not abandoned), useful (and not personal projects no one will ever be able to use except the author) and interesting.

If you're in a project community, if you have your own project (be it an application or even a small module) and you would like help on it, let me know so I could showcase your project in class - talk about it and try to interest people in it. These students are mostly beginners but with guidance, they could help out in applying patches, in writing tests, in documenting your code and hopefully with fixing a bug or two. This is your chance to get more people involved in your project, your project's community and the Perl community itself.

I need:
At minimum, a few lines on what to tell students (I'm also willing to simply read prepared text) about your project: what it is, why is it cool, why do you think they would enjoy working on it, how could they help out?

You could write to me either in a comment to this blog entry or you can email me at xsawyerx AT cpan DOTTT org.

If you want to prepare a short slide, a short video or a picture or two - I'm sure it will only help the students get interested in your project.

Let's get this show on the road!

A Curious Programmer: Spurious Benchmark.pm Warnings

A Curious Programmer

My most recent post has a few comments, but because I’ve been on holiday I didn’t reply to them in a timely manner. Sorry about that folks, I did read all of them.

And time has moved on so if I reply to those comments in the comments section, I’m guessing the original commenters won’t read the response. Having said that, they might not read this post either, but I think that is less likely.

Therefore this is a post to respond to the commenters, and to keep up with the IronMan schedule even though I’m not sure I’m being measured.

First of all, Roman, thanks for the comment. The figures you provide are interesting and yours is a second complaint about opaque error messages. That would bother me as well – I get enough of them from emacs macros and I have at least got tools (well, a tool – macroexpand) to debug those.

@Max – Hi. I’m not sure how you got the impression that I was interested in object creation time. We have primarily been talking about Moose start-up overhead here. I think I’m measuring the right thing.

@Anonymous – when you say "Run each benchmark item for 5 seconds or more." I think you missed this bit of the post:

Benchmark: timing 200 iterations of 1just-moose, 2all-includes, 3nocreate, 4create...
  1just-moose: 100.196 wallclock secs ( 0.05 usr +  0.25 sys =  0.30 CPU) @ 673.40/s (n=200)
2all-includes: 279.252 wallclock secs ( 0.08 usr +  0.34 sys =  0.42 CPU) @ 475.06/s (n=200)
    3nocreate: 318.256 wallclock secs ( 0.06 usr +  0.28 sys =  0.34 CPU) @ 581.40/s (n=200)
      4create: 320.979 wallclock secs ( 0.06 usr +  0.27 sys =  0.33 CPU) @ 611.62/s (n=200)

Not one of my benchmarks runs for less than a minute and a half. Most of them are around the five minute mark. And I have deadlines!

So where is this message coming from? I had a look in Benchmark.pm.

    print "            (warning: too few iterations for a reliable count)\n"
        if     $n < $Min_Count # 4
            || ($t->real < 1 && $n < 1000)
            || $t->cpu_a < $Min_CPU; # 0.4

My iterations is 200 so $Min_Count is not the problem. $t->real is at least 100 so that leaves $Min_CPU. Hmmm… yeah, I would have had to double the number of iterations to be sure. Look at 4create – 321 wallclock seconds of which 0.3 is CPU. No worries, I’m good – the warnings were spurious as I originally thought.


Filed under: Perl Tagged: benchmarking, moose

dagolden: Are you ready for CPAN Testers 2.0?

If you haven't seen the official notice, the CPAN Testers project is migrating from using email to send reports (CPAN Testers 1.0) to using web-based submission (CPAN Testers 2.0). As of September 1, 2010, the old email list will be shut down and only web submissions will be accepted.

I've just updated the "quick-start" documentation on the CPAN Testers wiki: Quick HowTo for CPAN Testers 2.0

It's not perfect, but it's a lot better than it used to be. If you are still submitting reports by email, please read and follow the instructions before the end of August. If you have trouble, please ask for help on the cpan-testers-discuss mailing list or on #cpantesters-discuss on irc.perl.org.

Curtis Jewell's Perl stuff: Strawberry Perl updates.

There is some stress going on in my personal life, but Strawberry Perl is rolling forward.

Two major things are happening:

1) Making the perl-building process itself pluggable. This is 99% done, and development versions with this code, and the plugins required to use them, will be released over the next few days.

2) Allowing the user to choose how much of an "on top of Strawberry" distribution is to be installed.

I'll be gone over the weekend, (Friday to Sunday) but should be back online and working Monday or Tuesday. If 5.12.2 is out by then, I'll release a beta version of it then.

I've also checking why a "Strawberry-process" 5.13.4 won't work ( http://rt.perl.org/rt3/Ticket/Display.html?id=77416 ), and checking how well a branch of ExtUtils::ParseXS will work for Strawberry (answer: we're working on it.)

Steffen Mueller: Tiny vim convenience hack

Since I maintain a lot of code that wasn't originally written by me, I am in the unfortunate situation that I get to edit code with many different coding and indentation styles on a regular basis. Reformatting usually isn't a good idea since not only may the original author object, but the revision control history is more important than code and indentation style.

I'm a vi(m) user. Modifying the settings to suit whatever insane tab-compression-indentation scheme some crazy emacs user decided on for individual documents is severely annoying. My editor should do this for me. If I was using Padre for my daily work, it would do this for me automatically. Today, finally, I sat down and wrangled vi until it would auto-detect the indentation style of the current document. More precisely, Text::FindIndent does (version 0.09 was just released and is required). Install the module, include this in your .vimrc on one line:


map <F5> <Esc> :perl use Text::FindIndent;VIM::DoCommand($_) for Text::FindIndent->to_vim_commands(join "\n", $curbuf->Get(1..$curbuf->Count()));<CR>

Now, open up somebody else's source code that uses some crazy indentation scheme. Hit F5. Your vi settings have now been marvelously modified to produce the same broken indentation as whatever you opened for editing.

Perlgeek.de : Programming Languages Are Not Zero Sum

From Wikipedia, the free encyclopedia:

In game theory and economic theory, zero-sum describes a situation in which a participant's gain or loss is exactly balanced by the losses or gains of the other participant(s). If the total gains of the participants are added up, and the total losses are subtracted, they will sum to zero.

Being advocate, implementor, tester and co-designer of a new programming language, I often hear objections along the lines of you are killing $other_programming_language, combined with a mixture of fear and resentment. People are afraid that having a new player on the market will decrease market share of their own, favorite programming language.

While I can understand these thinking patterns, there is no reason for concern. The market for programming languages is not a zero-sum situation. While I don't have hard data, I have the impression that the programming job sector is growing, and the US government expects it to grow further, too.

Certainly the growth of world population sets a rapidly increasing baseline, and even if we assume a constant percentage of all people related to programming in some way, the total number of programmers rises, and will continue for quite some time.

(I'm pointing to some resources about programming jobs, and I fully realize that it's not the same as number of overall programmers; but it's easier to get data for jobs, and I do think that the general trend statements are true for both).

So as long as the total number of programmers increases, a decrease in relative market share doesn't automatically mean a loss. In fact the job trends show an increase for "scripting" languages, and while Ruby is certainly the winner in terms of growth, Python, Perl and PHP win too!

Non-job data shows for example a noisy but steady growth of uploads to the Comprehensive Perl Archive Network (CPAN) -- data from a programming language that is often perceived as a loser of ruby's and python's success.

A recent Linux distribution trend analysis fell into the same trap: it shows relative numbers of search terms, and talks about a decline for all distributions except Ubuntu. Again I don't have hard numbers (the mirror infrastructure of most Linux distributions makes it nearly impossible to get accurate download counts), but I haven't seen any evidence that total usage numbers of any of the Linux distributions actually decreased.

Dave's Free Press: Bryar security hole

Someone on IRC reported a bug in Bryar. Namely that a Naughty Person can exploit the feature that notifies you of blog-spam by email to execute arbitrary code on your machine, as the user you run Bryar under.

A patched release is on the way to the CPAN, and you are strongly urged to upgrade.

Dave's Free Press: Thanks, Yahoo!

[originally posted on Apr 3 2008]

I'd like to express my warm thanks to the lovely people at Yahoo and in particular to their bot-herders. Until quite recently, their web-crawling bots had most irritatingly obeyed robot exclusion rules in the robots.txt file that I have on CPANdeps. But in the last couple of weeks they've got rid of that niggling little exclusion so now they're indexing all of the CPAN's dependencies through my site! And for the benefit of their important customers, they're doing it nice and quickly - a request every few seconds instead of the pedestrian once every few minutes that gentler bots use.

Unfortunately, because generating a dependency tree takes more time than they were allowing between requests, they were filling up my process table, and all my memory, and eating all the CPU, and the only way to get back into the machine was by power-cycling it. So it is with the deepest of regrets that I have had to exclude them.

Cunts.

[update] For fuck's sake, they're doing it again from a different netblock!

Perlgeek.de : What is the "Cool" class in Perl 6?

In Perl, subroutine and operator names determine what happens, usually not the type of the arguments. Instead the arguments are coerced to a type on which the operation makes sense:

say uc 34;      # coerces 34 to a string, and upper-cases it
say 1 + "2";    # converts "2" to a number before adding

To make things more extensible, the uc function re-dispatches to the uc method on its argument. So for the example above to work, we need an uc function in Int. And in Array, so that @a.uc works. And so on.

The original approach was to stuff all these methods into Any, the base class of the object hierarchy. Which kinda worked, but also meant that all user-defined classes ended up having some few hundreds methods to start with. Not good.

These days, the type Cool fills this niche: most built-in types (all that are meant to be used in that polymorphic way) inherit from Cool, so the uc method is actually defined in class Cool, coerces to string, and then re-dispatches to the internal logic that actually does the upper-casing.

The name either stands for Convenient object oriented loopback, or just expresses that that most built-ins are cool with an argument of that type.

If users want to write a type that can be used like a built-in type now just inherit from Cool, and define coercion methods to other built-in types. If the types don't inherit from Cool, they are more light-weight, and less magic. There's more than one way to do it.

Cool is a class (and not a role), because classes are mutable; so if you want to inject behavior into nearly all built-in types, augmenting Cool is an option (though usually considered evil, and should not be done lightly).

Dave's Free Press: Module pre-requisites analyser

As a service to module authors, here is a tool to show a module's pre-requisites and the test results from the CPAN testers. So before you rely on something working as a pre-requisite for your code, have a look to see how reliable it and its dependencies are.

Dave's Free Press: Thankyou, Anonymous Benefactor!

I got home this evening to find an Unexpected Parcel waiting for me, full of books. I have no idea who it's from, but I'm guessing that it's from someone who finds <a href=http://deps.cpantesters.org/>CPANdeps</a> useful. Thank you, Anonymous Benefactor! Your generosity is much appreciated!

Dave's Free Press: cgit syntax highlighting

For the last few months I've been using git for my version control system. It's better than CVS because it can handle offline commits. So if I'm using my laptop on a train, I can still use version control without having to have a notwork connection.

And to give a pretty web front-end to it for other people to read code without having to check it out of the repository, I use cgit, which I mostly chose because it's a dead simple CGI and not a huge fancy application.

One problem with cgit is that by default it doesn't do code highlighting. But it has the ability to run blobs of code through any filter you care to name before displaying them, so to get something nice like this all you need to do is write a highlighter and add a single line to your cgitrc:

source-filter=/web/www.cantrell.org.uk/cgit/highlighter

My highlighter program is this:

   1 #!/usr/local/bin/perl
   2 
   3 use warnings;
   4 use strict;
   5 
   6 my $file = shift;
   7 
   8 if($file =~ /\.(p[ml]|t)$/i) {
   9     system "/usr/local/bin/perltidy -html -st -ntoc -npod -pre -nss -nnn"
  10 } else {
  11     system "cat -n";
  12 }

Dave's Free Press: Graphing tool

I made a shiny thing! It can plot arbitrary functions of the form x=f(y) or y=f(x). Under the skin, it just massages its arguments and passes them through to Gnuplot. Here's the source code.

Update: now 48.3% even shinier - see on the right

Dave's Free Press: Ill

I am ill. I've been ill since Thursday, with a cold. You're meant to be able to cure a cold with [insert old wives tale remedy here] in 5 days, or if you don't, it'll clear itself up in just under a week. So hopefully today is the last day.

So what have I done while ill?

On Friday I became old (see previous post), and went to the Byzantium exhibition at the Royal Academy. It was good. You should go.

Saturday was the London Perl Workshop. My talk on closures went down well, and people seemed to understand what I was talking about. Hurrah! I decided that rather than hang around nattering and going to a few talks, I'd rather hide under my duvet for the rest of the day.

I mostly hid on Sunday too, and spent most of the day asleep. In a brief moment of productivity, I got my laptop and my phone to talk to each other using magic interwebnet bluetooth stuff. I'd tried previously without success, but that was with the previous release of OS X. With version X.5 it seems to Just Work, so no Evil Hacks were necessary.

The cold means that I can't taste a damned thing, not even bacon. So now I know what it's like to be Jewish. Being Jewish sucks.

And today, I am still coughing up occasional lumps of lung and making odd bubbling noises in my chest, although my nasal demons seem to be Snotting less than they were, so hopefully I'll be back to normal tomorrow.

Dave's Free Press: Devel::CheckLib can now check libraries' contents

Devel::CheckLib has grown a new feature. As well as checking that libraries and headers exist, it can now also check that particular functions exist in a library and check their return values. This will be particularly useful if you need to check that a particular version of a library is available.

It works if you have a Unixish toolchain. I need to wait for the CPAN-testers reports to see if I managed not to break anything on Windows. Unfortunately, even though the lovely Mr. Alias has worked hard to make Windows machines available to developers, I found it to be just too hard to use. Even downloading my code to the Windows machine was hard, as Windows seemed to think it knew better and shouldn't download the file I told it to download. Then once I had downloaded it, Windows decided to hide it somewhere that I couldn't get to using the command line. So I gave up.

I might try again once there are some decent tools on the machines: wget, tar, and gzip at minimum, as given those I can quickly bootstrap anything else. Software development isn't just about having compilers available.

Dave's Free Press: POD includes

One of my CPAN distributions is CPAN-FindDependencies. It contains a module CPAN::FindDependencies, and a simple script that wraps around it so you can view dependencies easily from the command line. That script, naturally, has a man page. However, that manpage basically says "if you want to know what arguments this program takes, see the CPAN::FindDependencies docs". This is Bad from a usability point of view, good from a not-duplicating-stuff point of view, and good from a laziness point of view. Which means that it's Bad.

So, the solution.

=over

#include shared/parameters

=back

and some Magic that does the cpp-stylee substitution at make dist time. Note the 'dist' section in my call to WriteMakefile.

This is, of course, crying out to be made less horribly hacky, but it works for now, so I'm happy.

My original idea was to write some crazy shit that would do the #include at install-time, when the user was installing my code. But that has the disadvantage that tools like search.cpan wouldn't show it properly, as they simply look at the files in the distribution. So this does the #includes at the last moment just before I package up the code and upload to the PAUSE. You lovely people get the right documentation in all the right places, I only have to maintain it in one place so it stays in sync, and (in the interests of Laziness) I don't have to remember to run any extra scripts before releasing, make dist just Does The Right Thing.

Perlgeek.de : Perl 6 Questions on Perlmonks

Since the Rakudo Star release, there has been a noticeable increase in Perl 6 questions on perlmonks - a good sign, because it means people are using it.

I've assembled this list by looking through the 100 newest nodes in Seekers of Perl Wisdom, which makes it 6% of the questions asked, or on average 1 per day (used to be around 1 per week).

Most of the questions are related to environmental issues (building/installing stuff), or beginner's questions related to syntax.

It's good to see the questions flowing in, and I hope that we'll soon see more questions where I can show off cool Perl 6 features in the answers :-).

Dave's Free Press: YAPC::Europe 2007 report: day 1

As is becoming normal, I used the times between talks to bugfix some of my modules - this time Tie::STDOUT and Data::Transactional. The former was failing on perl 5.6, the latter on 5.9.5. The former was a bug in perl (you can't localise tied filehandles and expect the tieing to go away in 5.6, so it now declares a dependency on 5.8), the latter was a bug in my code.

Philippe Bruhat's talk on Net::Proxy was great - you can tell it's great because I came away with ideas for at least four things that I need to write. First up will be a plugin for it to allow the user to specify minimum and maximum permitted data rates for proxied connections. This will permit bandwidth limits for maximum permitted rates, but will also help to defeat IDSes doing traffic analysis if you specify a minimum permitted data rate.

This will protect (eg) ssh sessions from being identified based on their very bursty traffic pattern, by "filling in the blanks" with junk data.

In the evening, the CPAN-testers BOF was productive.

Dave's Free Press: XML::Tiny released

I have released my XML::Tiny module. The parser at its core is less than twenty lines of code. Pretty easy to follow code too, I think, and that also includes error handling. One of my aims in writing it was to keep memory usage and code to the absolute minimum, so it doesn't handle all of XML. The documentation says that it supports "a useful subset of XML". Personally, I think it supports the useful subset. It's certainly enough to parse the data I get back from Amazon when I use their web services, and to parse an RSS feed.

Dave's Free Press: Palm Treo call db module

To make up for a disappointing gap in Palm's software for the Treo smartphone, I wrote a <a href=http://www.cantrell.org.uk/david/tech/treo/call-dumper/>small perl script</a> to parse the database that stores my call history. I then re-wrote it as <a href=http://search.cpan.org/search?query=Palm%3A%3ATreoPhoneCallDB>a re-useable module</a> which also figgers out whether the call was incoming or outgoing.

Dave's Free Press: Number::Phone release

There's a new release, <a href=http://www.cantrell.org.uk/david/tech/perl-modules/Number-Phone-1.58.tar.gz>version 1.58</a>, of Number::Phone, my set of perl modules for picking information out of phone numbers. Changes from the previous release are that Mayotte, Reunion and Comoros can't decide which country is which, and there's the usual updates to the database of UK numbers, mostly to support the <a href=http://www.ofcom.org.uk/media/news/2007/02/nr_20070213b>new 03 numbers</a>.

Dave's Free Press: CPANdeps

<a href=http://cpandeps.cantrell.org.uk/>CPANdeps</a> now lets you filter test results by perl version number, and also knows what modules were in core in which versions of perl. Hurrah!

Dave's Free Press: YAPC::Europe 2007 report: day 2

A day of not many talks, but lots of cool stuff. Damian was his usual crazy self, and MJD's talk on building parsers was really good. Although I probably won't use those techniques at work as functional programming seems to scare people.

The conference dinner at a Heuriger on the outskirts of Vienna was great. The orga-punks had hired a small fleet of buses to get us there and back, and one of the sponsors laid on a great buffet. The local wine was pretty damned fine too, and then the evening de-generated into Schnapps, with toasts to Her Majesty, to her splendid navy, and to The Village People.

It wasn't all debauchery in the evening though - on the bus, I had a very useful chat with Philippe about Net::Proxy, and re-designing it to make it easier to create new connectors for it.

Dave's Free Press: YAPC::Europe 2006 report: day 3

There were quite a few interesting talks in the morning, especially Ivor's one on packaging perl applications. Oh, and mine about rsnapshot, of course, in which people laughed at the right places and I judged the length of it just right, finishing with a couple of minutes left for questions.

At the traditional end-of-YAPC auction, I avoided spending my usual stupid amounts of money on stupid things, which was nice. Obviously the hundred quid I put in to buying the hair style of next year's organisers wasn't stupid. Oh no. Definitely not.

An orange mohican will suit Domm beautifully.

Dave's Free Press: YAPC::Europe 2007 report: day 3

My Lightning Talk on cpandeps went down really well, although as José pointed out, I need to fix it to take account of File::Copy being broken. I also need to talk to Domm after the conference is over to see if I can get dependency information from CPANTS as well as from META.yml files.

There were lots of other good lightning talks. Dmitri Karasik's regexes for doing OCR, Juerd Waalboer's Unicode::Semantics, and Renée Bäcker's Win32::GuiTest were especially noteworthy.

Richard Foley's brief intro to the perl debugger was also useful. Unfortunately Hakim Cassimally's talk was about debugging web applications, which I'd not noticed on the schedule, so I didn't stay for that.

And finally, Mark Fowler's grumble about why perl sucks (and what to do about it) had a few interesting little things in it. I am having vaguely sick ideas about mixing some of that up with an MJD-stylee parser.

At the auction I paid €250 to have the Danish organisers of next year's YAPC::Europe wear the Swedish flag on their foreheads. This, I should point out, was Greg's idea. I would never be so evil on my own.

Dave's Free Press: Perl isn't dieing

Perl isn't dieing, but it tells me that it wishes it was. Last night it went out on the piss with Python and Ruby (PHP was the designated driver) and it did rather too many cocktails. It isn't quite sure what happened, but it woke up in the gutter in a puddle of its own fluids and its head hurts a lot.

It asked me to ask you all to keep the volume down.

Dave's Free Press: YAPC::Europe 2007 travel plans

I'm going to Vienna by train for YAPC::Europe. If you want to join me you'll need to book in advance, and probably quite some way in advance as some of these trains apparently get fully booked.

arrdepdate
Waterloo1740Fri 24 Aug
Paris Nord2117
Paris Est2245
Munich08590928Sat 25 Aug
Vienna1335

The first two legs of that are second class, cos first wasn't available on Eurostar (being a Friday evening it's one of the commuter Eurostars and gets booked up months and months in advance) and was way too spendy on the sleeper to Munich. Upgrading to first class from Munich to Vienna is cheap, so I have.

Coming back it's first class all the way cos upgrading was nearly free ...

arrdepdate
Vienna0930Fri 31 Aug
Zurich1820
Zurich1402Sun 2 Sep
Paris Est1834
Paris Nord2013
Waterloo2159

Don't even think about trying to book online or over the phone, or at the Eurostar ticket office at Waterloo. Your best bet is to go to the Rail Europe shop on Picadilly, opposite the Royal Academy and next to Fortnums.

Perlgeek.de : Pascal's Triangle in Perl 6

Today on IRC, Larry Wall showed this piece of Perl 6 code, which he wrote for Rosetta Code:

sub pascal { [1], -> @p { [0, @p Z+ @p, 0] } ... * };
say pascal[^10].perl
# output (reformatted for easy readbility):
# ([1],
#  [1, 1],
#  [1, 2, 1],
#  [1, 3, 3, 1],
#  [1, 4, 6, 4, 1],
#  [1, 5, 10, 10, 5, 1],
#  [1, 6, 15, 20, 15, 6, 1],
#  [1, 7, 21, 35, 35, 21, 7, 1],
#  [1, 8, 28, 56, 70, 56, 28, 8, 1],
#  [1, 9, 36, 84, 126, 126, 84, 36, 9, 1])

That's Pascal's triangle, generated in one line of Perl 6.

The ... is the series operator, which generates lists by feeding the previous value(s) (here always one array) to the generating block on its left, until it reaches the goal on the right (in this case "whatever", which means it returns a lazy, infinite list).

So for example if the previous item was the array [1, 2, 1], the code block evaluates 0, 1, 2, 1 Z+ 1, 2, 1, 0.

Z is the zip operator, Z+ is pairwise addition (ie adding the pairs that the zip operator produced). In our example that leads to 0+1, 1+2, 2+1, 1+0 or 1, 3, 3, 1.

It takes a while to get used to the meta operators and the series operator, but once you've understood them, you can do pretty neat things with them.

Perlgeek.de : What is "Modern Perl"?

These days you often hear term Modern Perl, as something new(ish), and much improved over the old ways.

But what is it exactly? Well, there's no proper definition, but here is what that term means to me:

It's a set of tools, ideas and attitudes that help you to write better Perl programs, and allows you to have more fun while writing them.

Here are some aspects of Modern Perl

  • Testing: Most modern Perl modules have extensive test suites, that make development sane and robust
  • Some built-ins now come with safer forms: the three-argument form of open() allows you to open files safely with arbitrary characters in them, without any extra precautions. Lexical file handles make things safer and easier too.
  • use strict; use warnings;
  • Proper OO: with Perl 6 and with Moose in Perl 5, we have good object systems, that require less low-level fiddling than the standard Perl 5 object system
  • Following Best Practices
  • (For open source projects) Liberally handing out commit privileges. The source is stored in a version control system anyway, so low-quality changes or vandalism can simply be reverted (but that doesn't happen often in practice).
  • Caring about marketing: do tell people that you built something cool and useful
  • Small handy modules such as List::Util and Try::Tiny
  • Development tools such as Devel::Cover and Devel::NYTProf
  • (update) perlbrew and local::lib to help maintain your own perl installation and locally installed modules.

All of these techniques help to write scalable Perl programs by making proper encapsulation much easier, or by avoiding common errors, identifying performance bottlenecks etc.

Update: after watching some discussions about this post in various media, I should add a few more tools that I forgot about earlier:

Perlgeek.de : This Week's Contribution to Perl 6 Week 8: Implement $*ARGFILES for Rakudo

For this week's contribution to Perl 6 we ask you to implement the $*ARGFILES special variable (and underlying object) for Rakudo.

(Introduction to this series of challenges)

Background

In Perl 5, there is a "magic" way to iterate over input: while (my $line = <>) { ... }. This reads from standard input if no command line arguments were provided. If there are command line arguments, they are taken as file names, and <> iterates over the contents of these files.

In Perl 6 this magic is performed with the $*ARGFILES special variable. Please implement it!

To do that, you have to understand how file reading works in Perl 6. Once you have a file handle, there are two ways to read lines: either by calling $handle.get, which returns just one line, or by calling $handle.lines, which returns a (lazy) list of all the remaining lines.

You can obtain the command line arguments from @*ARGS, open a file with my $handle = open $filename; for reading; And finally you can use lines('filename') to read all lines from a file.

What you can do

Implement a backend class for $*ARGFILES. You can do that in normal Perl 6 code, no need to change the actual compiler. Once that's done, we will plug it into Rakudo. Your code might look like this:

class IO::ArgFiles {
    has @!filenames;
    method get() { ... }
    method lines() { ... }
    method filename() {
        # return the current filename, or '-' if standard input
        ...
    }
}
my $*ARGFILES = IO::ArgFiles.new(filenames => @*ARGS);
.say for $*ARGFILES.lines();

(Of course this is only a rough skeleton, you might need to change some details, or add some things).

Submission

Please submit your source code to the perl6-compiler@perl.org mailing list (and put moritz@faui2k3.org on CC, because the mailing list sometimes lack quite a bit).

Update:: there have been two submissions, and a mixture of both has been applied.

Perlgeek.de : Common Perl 6 data processing idioms

NAME

"Perl 5 to 6" Lesson 27 - Common Perl 6 data processing idioms

SYNOPSIS

  # create a hash from a list of keys and values:
  # solution 1: slices
  my %hash; %hash{@keys} = @values;
  # solution 2: meta operators
  my %hash = @keys Z=> @values;

  # create a hash from an array, with
  # true value for each array item:
  my %exists = @keys Z=> 1 xx *;

  # limit a value to a given range, here 0..10.
  my $x = -2;
  say 0 max $x min 10;

  # for debugging: dump the contents of a variable,
  # including its name, to STDERR
  note :$x.perl;

  # sort case-insensitively
  say @list.sort: *.lc;

  # mandatory attributes
  class Something {
      has $.required = die "Attribute 'required' is mandatory";
  }
  Something.new(required => 2); # no error
  Something.new()               # BOOM

DESCRIPTION

Learning the specification of a language is not enough to be productive with it. Rather you need to know how to solve specific problems. Common usage patterns, called idioms, helps you not having to re-invent the wheel every time you're faced with a problem.

So here a some common Perl 6 idioms, dealing with data structures.

Hashes

  # create a hash from a list of keys and values:
  # solution 1: slices
  my %hash; %hash{@keys} = @values;
  # solution 2: meta operators
  my %hash = @keys Z=> @values;

The first solution is the same you'd use in Perl 5: assignment to a slice. The second solution uses the zip operator Z, which joins to list like a zip fastener: 1, 2, 3 Z 10, 20, 30 is 1, 10, 2, 20, 3, 30. The Z=> is a meta operator, which combines zip with => (the Pair construction operator). So 1, 2, 3 Z=> 10, 20, 30 evaluates to 1 => 10, 2 => 20, 3 => 30. Assignment to a hash variable turns that into a Hash.

For existence checks, the values in a hash often doesn't matter, as long as they all evaluate to True in boolean context. In that case, a nice way to initialize the hash from a given array or list of keys is

  my %exists = @keys Z=> 1 xx *;

which uses a lazy, infinite list of 1s on the right-hand side, and relies on the fact that Z ends when the shorter list is exhausted.

Numbers

Sometimes you want to get a number from somewhere, but clip it into a predefined range (for example so that it can act as an array index).

In Perl 5 you often end up with things like $a = $b > $upper ? $upper : $b, and another conditional for the lower limit. With the max and min infix operators, that simplifies considerably to

  my $in-range = $lower max $x min $upper;

because $lower max $x returns the larger of the two numbers, and thus clipping to the lower end of the range.

Since min and max are infix operators, you can also clip infix:

 $x max= 0;
 $x min= 10;

Debugging

Perl 5 has Data::Dumper, Perl 6 objects have the .perl method. Both generate code that reproduces the original data structure as faithfully as possible.

:$var generates a Pair ("colonpair"), using the variable name as key (but with sigil stripped). So it's the same as var => $var. note() writes to the standard error stream, appending a newline. So note :$var.perl is quick way of obtaining the value of a variable for debugging; purposes, along with its name.

Sorting

Like in Perl 5, the sort built-in can take a function that compares two values, and then sorts according to that comparison. Unlike Perl 5, it's a bit smarter, and automatically does a transformation for you if the function takes only one argument.

In general, if you want to compare by a transformed value, in Perl 5 you can do:

    # WARNING: Perl 5 code ahead
    my @sorted = sort { transform($a) cmp transform($b) } @values;

    # or the so-called Schwartzian Transform:
    my @sorted = map { $_->[1] }
                 sort { $a->[0] cmp $b->[0] }
                 map { [transform($_), $_] }
                 @values

The former solution requires repetitive typing of the transformation, and executes it for each comparison. The second solution avoids that by storing the transformed value along with the original value, but it's quite a bit of code to write.

Perl 6 automates the second solution (and a bit more efficient than the naiive Schwartzian transform, by avoiding an array for each value) when the transformation function has arity one, ie accepts one argument only:

    my @sorted = sort &transform, @values;

Mandatory Attributes

The typical way to enforce the presence of an attribute is to check its presence in the constructor - or in all constructors, if there are many.

That works in Perl 6 too, but it's easier and safer to require the presence at the level of each attribute:

    has $.attr = die "'attr' is mandatory";

This exploits the default value mechanism. When a value is supplied, the code for generating the default value is never executed, and the die never triggers. If any constructor fails to set it, an exception is thrown.

MOTIVATION

N/A

Perlgeek.de : Gabor: Keep going

After reading this blog post, I just want to say: keep going.

Gabor does really awesome things for Perl (and especially for Perl 6, which I tend to notice more): beginner's tutorials, screencasts, training courses, writes an IDE and so on. If his primary interest was his own success, his priorities would be quite different.

I hope that Gabor doesn't pay too much attention to the hostilities from parts of the Perl community, and I wish him and us all the best for his current project.

Perlgeek.de : Want to write shiny SVG graphics with Perl 6? Port Scruffy!

First let my apologize for waiting so long to come up with a new "weekly" Perl 6 challenge - I'm running out of ideas, and the one I have left needs more time to prepare.

Instead I want to motivate you to help porting the ruby scruffy charting library to Perl 6. It can generate shiny SVG graphics (they are currently broken on their main website, hence the waybackmachine link).

There's already an initial version Perl 6 port called "tufte", but it's not running yet. It needs your help. If you know a little ruby, and want to learn some more Perl 6, join #perl6, ask for a commit bit, and translate some ruby code into Perl 6. And in the end you'll be rewarded with nice SVG charts :-).

Perlgeek.de : The State of Regex Modifiers in Rakudo

During the last one and a half month, I've been working on making regex modifiers easily available in Rakudo.

The regex compiler itself has to support only a few of the adverbs that can be applied to regexes; those include :ignorecase, :sigspace, :ignoremark and :continue/:pos. NQP-rx, the regex engine that Rakudo uses under the hood, supports those (except :ignoremark), so previously you could write

if 'ABC' ~~ /:i abc/ {
    say "case insensitive match";
}

But not

if 'ABC' ~~ rx:i/abc/ {
    say "case insensitive match";
}

nor m:i/abc/, for that matter.

I've patched Rakudo to actually recognize those adverbs outside of the regex, and also for s/// substitutions.

Another category of adverbs are those that apply to regex calls, not to the compilation of a regex. Among those are :global/:g, :overlap/:ov, :nth($n), :x. I've implemented those for substitutions, but implementing them for m// turns out to be quite a bit harder.

The reason is the return value: each regex match returns a Match object, which can store positional and named parts. S05 says that regex matches with multiple results should return a single match object, with all results as positional parts. It can be distinguished from a normal match object by evaluating it in slice context... which Rakudo doesn't support yet.

Now the subst method and thus s/// are implemented by calling .match(:global, ...), and without slice context, it can't distinguish between multiple matches, and a single match with subcaptures. And so my changes to the global match broke the substitution, and I see no easy way to fix it.

Anyway, here are a few examples of what works today:

$_ = 'ab12fg34';
s:g/\d/X/;
.say; # output: abXXfgXX


$_ = 'Hello, World';
# :ii is the same as :samecase
s:ii/world/perl/;
.say; # output: Hello, Perl

$_ = 'I did not know that that work together';
s:2nd/that/they/;
.say; # output: I did not know that they work together

Perlgeek.de : This Week's Contribution to Perl 6 Week 9: Implement Hash.pick for Rakudo

For this week's contribution to Perl 6 we ask you to implement the Hash.pick method (which does a weighted random selection) for Rakudo.

(Introduction to this series of challenges)

Background

In Perl 6 the List class has a method called pick, which randomly selects one item from a list. It has a few more options too:

<a b c>.pick;       # pick one random element
<a b c>.pick(2);    # pick two distinct, random elements
<a b c>.pick(2, :replace); # pick two random elements, it's ok
                           # if they are the same
<a b c>.pick(*);    # return a random permutation of the elements
<a b c>.pick(*, :replace); # infinite, random stream of elements

This is already implemented through several multi methods in Rakudo.

Now the specification describes such a method for hashes too (actually it talks about Bags, but Rakudo doesn't have Bags yet. Pretend it says "Hash" instead). It assumes that each value in the hash is numeric, and that the value is a weight that determines the probability of picking one value. For example

{a => 1, b => 2}.pick;  # returns 'a' with probability 1/3
                        # and 'b' with probability 2/3

{a => 1, b => 2}.pick(*);  # <a b b> with probability 1/3
                           # <b a b> with probability 1/3
                           # <b b a> with probability 1/3
{a => 1, b => 0.5}.pick(*) # dies, because the weights aren't all integers
{a => 1, b => 0.5}.pick(*, :replace)  # ok 

What you can do

Implement Hash.pick. It's ok if your patch doesn't cover all cases. It would be nice if it supported non-integer weights.

Hint: this could be done by storing a list of accumulated weights, and a list of keys.

{a => 1, b => 2.5, c => 1}

# could translate to 
my @keys = ('a', 'b', 'c');
my @accumulated_weights = (1, 3.5, 4.5);

# now pick a random number between 0 and 4.5,
# find the next-highest index in @accumulated_weights
# with a binary search, and then use that to obtain the key.

Of course other schemes are fine too.

Second hint: because it takes quite some time to recompile Rakudo, it is probably easier to implement the actual logic in a function in a normal source file first, and only later move it into src/core/Hash.pm.

Submission

Please submit your source code to the perl6-compiler@perl.org mailing list (and put moritz@faui2k3.org on CC, because the mailing list sometimes lack quite a bit).

Update: there's one submission on the perl6-compiler mailing list already, which looks pretty good.

Perlgeek.de : Fixing Rakudo Memory Leaks

Rakudo has been leaking memory for a few month. The other day, after some nagging, Will Coleda identified a memory leak, and Tyler Curtis fixed it.

Now we can again make long-running processes with Rakudo. For example for my talk at YAPC::EU I plotted a resonance curve. For that I needed to start a new Rakudo process for every data point because it would leak so badly that it died after processing a few data points. Now I recalculated a whole curve in one process, with memory usage not exceeding 200MB of virtual mem.

resonance curve

I also had some fun recalculating a mandelbrot fractal in a size that would previously make Rakudo segfault or consume too much memory.

Mandelbrot fractal, rendered by Rakudo

(Rendered with colomon's mandelbrot code)).

Dave's Free Press: Wikipedia handheld proxy

I got irritated at how hard it was to use Wikipedia on my Treo. There's so much rubbish splattered around their pages that it Just Doesn't Work on such a small screen. Given that no alternatives seemed to be available - at least, Google couldn't find any - I decided to write my own Wikipedia handheld proxy.

It strips away all the useless rubbish that normally surrounds Wikipedia pages, as well as things like the editing functions which are also hard to use on portable devices. Internally, it's implemented using perl, LWP, and mod_perl, and is hosted by Keyweb.de.

Dave's Free Press: CPANdeps upgrade

While you won't notice any changes, there have been biiiig upgrades at CPANdeps. Here's the diff.

Until now, it's used a SQLite database of test results that I downloaded every day and then mangled a bit to do things like add some necessary indices, figure out which reports are from dev versions of perl, and so on. That worked really well back in the summer of 2007, when there were only half a million reports in the database. I started worrying a bit at the beginning of 2009 when we hit 3 million, but the update happened overnight so I didn't care. But now that we've got over 6 million reports, the update would take anywhere between 8 and 14 hours. Not only is that not sustainable given the current growth rate, it also hurts the other users on that machine, because almost all of that time is spent waiting for disk I/O - which means that they're also waiting for the disk. On top of that, when you have big databases, a SQLite CGI ain't a great idea because indices have to be fetched from disk every time, so reads pound the disk too. Doubleplusungood!

Fun fact: SQLite is great for prototyping, but it doesn't scale :-)

So now it uses MySQL. Having a database daemon running all the time means that there's now some caching, so reads are quicker. In addition, given that I can't just simply fiddle with the structure of the database that I download to produce what I want, and instead have to import the data into MySQL, it now only imports new records, so the daily update takes only a few seconds.

I also re-jigged the structure of how it caches test results. Instead of being all in one directory with hundreds of thousands of files, they're split into a hierarchy. This probably won't have any significant effect on normal operations, but it will certainly make it faster for me to navigate around and see what's going on when people submit bug reports!

Perlgeek.de : Notes from the YAPC::EU 2010 Rakudo hackathon

At YAPC::EU 2010 we had a long discussion about Perl 6, Rakudo and related matters. Here are some (very incomplete) notes of the ongoing discussions and results.

Attendees

Patrick Michaud, Jonathan Worthington, Carl Mäsak, Moritz Lenz, Gabor Szabo, and a fluctuation of other Perl 6 hackers.

Speed

What can we do to improve Rakudo's performance?

jnthn's grant proposal for a low-level meta object protocol

See http://news.perlfoundation.org/2010/07/hague-grant-application-meta-m.html. Will probably bring the biggest speed improvement of all options we have under our control

Rakudo built-in optimizations

Most Rakudo built-ins are written for correctness first, and without a good feeling for what constructs are fast and what aren't. A thorough review (and preferably profiling) could bring decent speed improvements, as the case of int ranges showed.

Garbage collector

Parrot's GC is... suboptimal. To be gentle.

Optimization framework

We will try to convince people that Tyler Curtis' optimization framework for PAST and POST should be shipped with parrot (probably compile PIRs in ext/, just like NQP-rx does it now). Using that, we can do constant folding

Moving stuff to compile time

Number parsing needs to be moved to compile time.

What do we need to keep hacking?

Brought up by Gabor

Money

We do much volunteer work, but when we get funding, we can devote more time to hacking

Travel/Conferences

We'd like to get together a few times (2? 3? 4?) a year, in real life.

Funding and organization would be very welcome

Short-time funding

It would be nice to have a way to have funding available much more quickly than through the usual grant process, which tends to be longish.

Rakudo Star feedback

Good: It worked. It did what we wanted it to.

Bad:

  • It lacked a module installer (It shipped proto, but didn't install it).

  • Compilation takes too much memory. pmichaud will try a hack to split the setting, which would solve that problem.

  • There was some discussion about the roles + outer scopes bugs, which was way over my head. It seems to be related to the fact that parrot has two outer chains for nested blocks: one at compile time, one at runtime. Since role methods are flattened into classes, there compile time outer block is actually different than where it runs, and that screws up ... forget it, somebody else must describe it.

  • Lack of modules - doesn't seem to bee a big problem

  • Lack of features: not a big problem.

    Biggest complaints: missing perl6doc. Missing non-blocking IO, binary file support.

  • Prefix paths with spaces are not supported :(

    jnthn: "I actually tried to write a C program that binary patches the perl6 executable to allow spaces in path names. It almost worked."

  • We will try to advocate compilation to PBC, not PIR - once that's supported.

Proto/Pls

Proto needs to be end-of-life'd.

It confuses people that there are two different project lists, and the lists diverge.

We would like to decentralize the module list somehow. Still open how.

People don't release Perl 6 modules, because there's no need so far, and it's tedious to add the version name in each .pm/.pm6 file. We might need to come up with a clever idea for that.

Backend diversity

Additionally to the parrot backend, we want to run Perl 6 code on other virtual machines.

jnthn will work on a .NET/CLR port. He wants to prototype the new low-level class composition code in .NET anyway, which will provide the basic foundations for running NQP.

pmichaud wants to explore javascript on V8 as a possible backend. "I managed PIR, I'll certainly manage javascript" :-)

  • Huge time sink, but still worth doing it

  • Apache runtime library might be worth looking into

  • risks: stalled refactors are dangerous (see: PHP 6, cardinal (the ruby-on-parrot compiler))

    We want to avoid fragmentation into many subprojects

  • We want to increase the number of possible contributors to rakudo by enabling non-parrot people to contribute.

  • Code for different backends will be maintained as directories in Rakudo and NQP, not as branches.

  • pir:: things will be hidden behind an nqp:: abstraction layer

Attracting contributors

Moritz wants to continue with the "weekly" challenges, but runs out of ideas. Add ideas to http://svn.pugscode.org/pugs/misc/helpnow/README.

We will try to apply patches faster, thus encouraging people who already did the first step.

Documentation

  • in p5 pod for now, so that people can contribute easily

  • masak and szabgab expressed interest in working on pod6 tools

Perlgeek.de : My first YAPC - YAPC::EU 2010 in Pisa

This week I attended my first international Perl conference, the YAPC::EU 2010 in Pisa, Italy. I very much enjoyed it.

I arrived a few days earlier, and spent the time visiting Pisa, talking with old and new friends, and did some collaborative hacking. I especially enjoyed meeting people with whom I had had only contact via Internet so far, and found all of them to be very nice in meat space.

On Tuesday, the day before the conference started officially, a group of Perl 6 hackers met and discussed topics around Perl 6 and Rakudo. I recall talking with Patrick Michaud, Jonathan Worthington, Carl Mäsak, Paweł Murias, Gabor Szabo, smash (I can't spell his full name correctly from memory, sorry for that), and we have a very productive discussion (about 5 hours or so). Notes from the discussion will be published later.

On Wednesday the actual conference started, and there were plenty of very interesting talks, and very amusing lightning talks. I generally like the humor that is widespread in the Perl community.

On Thursday I continued to attend nice and informative talks, and also gave a talk on my own. It was about physical modelling with Perl 6, and in general the feedback was very positive, and somebody even commented that while he didn't understand everything I wrote, it reminded him that it was important to learn Perl 6 now. Win \o/. (There was also some criticism, but from somebody who apparently hasn't read the abstract; the "write-only" meme seems to apply to perl bloggers, not code). You can find the slides to my talk here.

Firday was the day of my departure too - sadly I had to leave after the first talk, and missed the rest of the day, including the closing keynote by mst (I did attend a talk of his) and the traditional auction.

I especially enjoyed...

  • meeting Patrick, Larry, Gloria, Aaron, Gabor and many others for the first time in real-life
  • a thorough, high-bandwith discussion with Larry, Jonathan, Patrick and Carl about p6 spec questions, and the Perl 6 discussions mentioned earlier
  • a lightning talk imagine you're in a data center with no connection to the outside, and you accidentally executed chmod -x chmod. What would you do?
  • good Italian Pizzas (although I had to wait 50 minutes for one of them)
  • A talk by Tim Bunce, where he demonstrated database access in Perl 6 both with libraries that do native calls, and through the Blizkost project using the Perl 5 DBI/DBD::SQLite modules
  • the general relaxed attitude in the Perl community, where people help each other, and don't seem to be easily offended (at least in meat space :)
  • being surrounded by many other geeks
  • realizing that several Perl hackers brought their family to the conference

It was an overwhelming experience, and I look forward to my next YAPC!

Perlgeek.de : Currying

NAME

"Perl 5 to 6" Lesson 28 - Currying

SYNOPSIS

  use v6;
  
  my &f := &substr.assuming('Hello, World');
  say f(0, 2);                # He
  say f(3, 2);                # lo
  say f(7);                   # World
  
  say <a b c>.map: * x 2;     # aabbcc
  say <a b c>.map: *.uc;      # ABC
  for ^10 {
      print <R G B>.[$_ % *]; # RGBRGBRGBR
  }

DESCRIPTION

Currying or partial application is the process of generating a function from another function or method by providing only some of the arguments. This is useful for saving typing, and when you want to pass a callback to another function.

Suppose you want a function that lets you extract substrings from "Hello, World" easily. The classical way of doing that is writing your own function:

  sub f(*@a) {
      substr('Hello, World', |@a)
  }

Currying with assuming

Perl 6 provides a method assuming on code objects, which applies the arguments passed to it to the invocant, and returns the partially applied function.

  my &f := &substr.assuming('Hello, World');

Now f(1, 2) is the same as substr('Hello, World', 1, 2).

assuming also works on operators, because operators are just subroutines with weird names. To get a subroutine that adds 2 to whatever number gets passed to it, you could write

  my &add_two := &infix:<+>.assuming(2);

But that's tedious to write, so there's another option.

Currying with the Whatever-Star

  my &add_two := * + 2;
  say add_two(4);         # 6

The asterisk, called Whatever, is a placeholder for an argument, so the whole expression returns a closure. Multiple Whatevers are allowed in a single expression, and create a closure that expects more arguments, by replacing each term * by a formal parameter. So * * 5 + * is equivalent to -> $a, $b { $a * 5 + $b }.

  my $c = * * 5 + *;
  say $c(10, 2);                # 52

Note that the second * is an infix operator, not a term, so it is not subject to Whatever-currying.

The process of lifting an expression with Whatever stars into a closure is driven by syntax, and done at compile time. This means that

  my $star = *;
  my $code = $star + 2

does not construct a closure, but instead dies with a message like

  Can't take numeric value for object of type Whatever

Whatever currying is more versatile than .assuming, because it allows to curry something else than the first argument very easily:

  say  ~(1, 3).map: 'hi' x *    # hi hihihi

This curries the second argument of the string repetition operator infix x, so it returns a closure that, when called with a numeric argument, produces the string hi as often as that argument specifies.

The invocant of a method call can also be Whatever star, so

  say <a b c>.map: *.uc;      # ABC

involves a closure that calls the uc method on its argument.

MOTIVATION

Perl 5 could be used for functional programming, which has been demonstrated in Mark Jason Dominus' book Higher Order Perl.

Perl 6 strives to make it even easier, and thus provides tools to make typical constructs in functional programming easily available. Currying and easy construction of closures is a key to functional programming, and makes it very easy to write transformation for your data, for example together with map or grep.

SEE ALSO

http://perlcabal.org/syn/S02.html#Built-In_Data_Types

http://hop.perl.plover.com/

http://en.wikipedia.org/wiki/Currying

Header image by Tambako the Jaguar. Some rights reserved.