Perlsphere

Perl Data Language (PDL) 2.4.10 Release

nobody@example.com (Joel Berger) — 2012-02-06T02:56:34Z

Since the Perl Data Language (PDL) does not have a large presence in the Perl Blogosphere, I have the honor of reposting Pumpking Chris Marshall’s announcement of PDL 2.4.10.

For those of you who don’t know, PDL gives standard Perl the ability to compactly store and speedily manipulate the large N-dimensional data arrays which are the bread and butter of scientific computing. For more information on PDL please visit its website at http://pdl.perl.org.

Chris’ release message is reposted below, the full text can be seen on the mailing list archive.

PDL-2.4.10 released

The PDL Development Team is pleased to announce the PDL-2.4.10 release of the Perl Data Language and the first PDF release of the PDL Book.

PDL-2.4.10 is the latest point release with more functionality, portability, and robustness than ever before, including:

POSIX threads support for all platforms
Auto parallelization of PDL threadloops
Support for PDLs larger than 2GiB
PDL Book draft release (PDF format)
Much, much, more…

As always, the source distribution will be available at a CPAN mirror near you within a couple of days. Our sf.net site has the source distribution and the PDL Book:

http://sourceforge.net/projects/pdl/files/PDL/2.4.10/PDL-Book-20120205.pdf/download

Windows binary PPM are available in the usual site, see “Get PDL” in the sidebar at http://pdl.perl.org (the PDL website for links, documentation and info for all things PDL).

The SciPDL-2.4.10 release for MacOS X systems will be announced when it is available.

Enjoy and Happy PDL-ing!

Chris Marshall for the PDL Development Team

Updates to Sys::HostIP

nobody@example.com (Sawyer X) — 2012-02-05T09:41:09Z

Sys::HostIP simply parses ifconfig/ipconfig (supports GNU/Linux/BSD/Windows) and gives you the interfaces and IPs found.

I stumbled upon this module when I was looking for a way to clean get all the ips of every interface on machines at $work. I needed something that parses ifconfig cleanly, because I didn't want to do it myself. I found a few and Sys::HostIP was by far the easiest, simplest, fastest and the one I decided to use. I noticed a few bugs open and contacted the author who gave me co-ownership. I since updated it and cleaned most of the ticket pool.

I've recently stumbled another situation, in which I want to find the IPs by order. Thinking I could just tweak the module to allow fetching it in order, I found that the logics is a bit of a mind boggle. I retained much of the original code and it supported both functional programming (as in functions) and object oriented, and I wanted to keep that.

I started by mapping on my board at $work all of the paths that methods and functions take. This was a total mess and I decided it's time to pick one or the other. I've picked object oriented in order to save user configurations, allow results caching, reserve status and have reusability.

This means that the next release of Sys::HostIP will warn when the functional interface will be used, and the release after that (version 2.00) will have it removed completely. I intend to have that release in at least two weeks from today, perhaps even a month.

Another important change is that Taint mode actually works now. Previous the regex parsing used to get the binary directory (for setting the PATH environment) was wrong, and I removed it in favor of the core module File::Basename.

I also removed the need for Test::TinyMocker and Test::Exception. This makes the module have zero non-core depdendencies. Hey, why not?

Feedback is welcome.

Thanks goes to Joel Berger, Yanick Champoux and Gabor Szabo for spreading the idea of promoting our distribution, and actually doing it. :)

The 'Perl Tutorial' Tutorial

nobody@example.com (Joel Berger) — 2012-02-05T00:50:32Z

The web is crowded with tutorials about Perl. Perl has improved over the years, bringing new features, safer constructs and clearer syntax. However the old tutorials still are read and learned by far too many new Perlers. This article is to help you be able to select your information source critically.

Some things to look for:

The name of the language is Perl

Yes that’s Perl not PERL. The community is very adamant about this, so if the tutorial says PERL you know that it is written by someone outside the Perl community.

The current Perl version is 5.14, the year is 2012

Many tutorials that were great for Perl 4, or even 5.6 are now woefully out of date. If there is a date on the tutorial older than about 2007 or Perl version 5.10, it will be missing many of the new features and may use older, more dangerous syntax.

Recommends using ‘strict’ and ‘warnings’

While the strict and warnings pragmas are not required, and there are still a few people who prefer not to use them, the vast majority of Perlers do. They help prevent typos and gotchas that are very easy for even the most experienced Perlers to accidently fall into.

A good tutorial site should at least mention that these represent a good safety net, but may skip including them on each example to save space. An even better site will include them in all longer code snippets.

Finally, you may see snippets which use a -w on the top line, this is a giveaway that the tutorial was probably good, but is now a little old (see above).

Use of ‘my’ variables

Along with use strict comes the requirement that variables are declared with some scope. This means that the first time a variable is introduced, it should come with a declaration of my, our, local or state.

my $phrase = 'Hello World';
our @names = ('Jim', 'Bob');

If this is not the case, I would move on to another tutorial. One exception is if the code is written as a “one-liner”, which will look like

perl -e 'some code here'

in which these declarations are not often used. New Perlers may want to learn some basics before attempting one-liners however.

Modern use of the ‘open’ command

The open command has seen some big changes over the years, and while the old use will still work, the modern syntax is much better. Since Perl is often used to work on text files, the open command is in almost every tutorial. As such, the treatment of this one command can serve as a useful litmus test as to the age and “goodness” of a tutorial.

A huge red-flag of an old tutorial looks like this

open HANDLE, '<filename';

rather that a modern

open my $handle, '<', 'filename' or die "Cannot open filename: $!";

This will take some explanation. First modern Perl lets one assign the filehandle to a variable (i.e. $handle, which may need the my declarator if it has not been declared previously), which is highly recommended.

Separating the open type, < or > (for read and write, respectively) from the file name is now recommended as well. This is called “the 3-arg open” and should be used whenever possible.

Checking to make sure that open succeeded is also recommended. The snippet above does this with the or die ... conditional. Another simple way to accomplish this task is to use autodie near the top of the script. Would you want your code to work on a file that didn’t open correctly? I doubt it.

A good tutorial should do, or at least mention, all of the previous concerns. If they lax on checking success on opening after mentioning it, this is probably acceptable (from a purely space-saving perspective). If not, move on.

Preach code reuse

Perl is good at munging text, however, for all but simple cases, you might want to use standard modules when appropriate. A new user should probably know how to parse out a basic line of comma-separated text, and a tutorial can show you how to do it manually. That said, a very good tutorial will mention that this is very fragile and that you should, in practice use the Text::CSV module. Same goes for HTML, XML, JSON and other formats.

Finally while code-reuse is a good idea, it can only as good as the code that is being reused. The CGI.pm module helped put Perl on the map, but its days are numbered. A tutorial that spends too much time on it is probably too old to trust.

In Conclusion

While none of these guidelines is absolute, I think you the reader will find that most tutorials will usually fall into very good or very bad/old. Hopefully this helps you find those good Perl tutorials, and steer clear of the others.

Gabor Szabo, himself the author of a new series of Perl tutorials, reminds me that there is a project aimed at screening and recommending tutorials. You might want to start your search for a modern tutorial there: Perl Tutorial Hub.

P.S.

To my fellow Perlers, you are encouraged to add to this list in the comments, as well as suggest some good tutorials. This post was made partially to be linked to posters on Stack Overflow and others when bad code from old tutorials shows up (you know what I mean).

Mexican Corn

nobody@example.com (PerlSquirrel) — 2012-02-05T00:02:00Z

This may not have much to do with Perl, but it is this Squirrel's favorite food!

Mexican Corn

Recipe can be found here.

Debugging Perl scripts

nobody@example.com (Gabor Szabo) — 2012-02-04T22:05:06Z

When I studied computer sciences in the university, we learned a lot on how to write programs, but as far as I remember no one told us about debugging. We heard about the nice world of creating new things, but no one told us that most of the time we'll have to spend trying to understand other people's code. It turns out that while we mostly cherish writing the program, we spend a lot more try trying to understand what we (or others) wrote, and why does it misbehave, than the time we spent writing it in the first time.

For the full article visit Debugging Perl scripts

2012Q1 Grant Proposals

nobody@example.com (Alberto Simões) — 2012-02-04T13:45:55Z

The Perl Foundation Grants Committee is proud to present three grant proposals for community discussion. As in previous rounds, please comment and clarify any issue about the grant in each grant post and not in this post.

The submitted grants are:

Alien::Base - Base classes for Alien:: modules by Joel Berger
EPublisher Website by Boris Däppen
Easy installation of non-CPAN dependencies by Alexandr Ciornii

The Grants Committee expects to vote and have a result for Carnival!

2012Q1 Grant Proposal: Alien::Base - Base classes for Alien:: modules

nobody@example.com (Alberto Simões) — 2012-02-04T13:42:38Z

Name:: Joel Berger
Email:: [hidden email]
Amount Requested:: $500

Synopsis

Artur Bergman's documentation for the Alien namespace specifically says that there is no framework for Alien:: modules, however, in writing the Alien::GSL manpage I find that most of the code I am writing has nothing to do with the GSL library and could as easily apply to many external libraries. I intend for Alien::Base to comprise some base classes for other Alien:: modules, including Alien::GSL.

Benefits to the Perl Community

Perl is great, CPAN is great, CPAN.pm and cpanminus are great, but if a module depends on an external library that is not installed, none of these can help. The Alien namespace has attempted to meet this demand by providing installers for external libraries needed by CPAN modules.

Sadly, comparitively few libraries are provided. In my opinion this is because writing the Alien module itself is too hard. With an Alien::Base system in place, perhaps more libraries could be provided and Perl would have an even larger base of available code to build from.

Deliverables

A CPAN distribution named Alien::Base, comprised of at least the classes:

Alien::Base - used as the base class for an Alien module (e.g. Alien::OtherLibrary), which is called by the dependent module (e.g. Something::Other::XS).
Alien::Base::ModuleBuild - a the Module::Build manpage subclass which handles the actions of fetching, building and installing the library.
Tests
Documentation

Project Details

Alien::Base::ModuleBuild will be able to

Query an FTP or HTTP server for a list of files
Fetch a file (archive) from either type of server, based on a required or else newest version
Extract the archive to a temporary directory
Perform a series of system calls to configure, build and install the library (to a the File::ShareDir manpage type folder)
Extract some information about the library and store it in a the Module::Build::ConfigData manpage storage module

Alien::Base will be able to

Provide the information about the library's local installation
Make the library available to a dependent module and/or its builder

The aim will be to allow much, if not all, of the subclass (i.e. Alien::OtherLibrary) specifics to be set with parameters/properties just like the Module::Build manpage installations, rather than by needing to write extra methods to do any of the above-listed tasks. I envision that the Build.PL for Alien::GSL could look like this (properties like alien_ are from Alien::Base):

 use strict;
 use warnings;

 use Alien::Base::ModuleBuild;

 my $builder = Alien::Base::ModuleBuild->new(
   module_name => 'Alien::GSL',
   dist_abstract => 'Easy installation of the GNU Scientific Library',
   license => 'perl',
   configure_requires => {
     'Alien::Base' => 0,
   },
   requires => {
     'perl' => '5.8.1',
     'Alien::Base' => 0,
   },
   dist_author => 'Joel A. Berger <joel.a.berger@gmail.com>',
   alien_name => 'gsl',
   alien_source_ftp => { 
     server  => 'ftp.gnu.org',
     folder  => '/gnu/gsl',
     pattern => qr/^gsl-([\d\.])+\.tar\.gz$/,
   },
 );
 $builder->create_build_script;

Initially I intend to focus on GNU libraries as they are fairly consistent in their build process. This should allow Alien:: authors to wrap this large and important set of libraries, employing only minimal configuration. I intend to use this for the Alien::GSL manpage and perhaps something like Alien::NCurses (i.e. a library I am not as familiar with) if I feel so inclined.

Hopefully the process will be sufficiently configurable so as to allow other non-GNU libraries to be wrapped easily. Certainly something involved like the Alien::SDL manpage isn't a prime candidate for this framework, but there are many smaller libraries which might benefit from being wrapped in this manner.

After initial releases, I hope that the community will help to make Alien::Base even more broadly applicable to more libraries.

Inch-stones

Each of the bullets under Project Details will serve as inch-stones.

Project Schedule

I have already begun working on Alien::Base; see it at https://github.com/jberger/Alien-Base. Development shouldn't take too long; I imagine only a few total weeks work should do the trick. Unfortunately my time cannot be given completely to this task since I am also currently working on my thesis (see Bio). Therefore, I propose that my project be 3-4 months (calendar) duration. Hopefully I will not need all that time. Also I have been trying to include tests as I code (countering a usual fault of mine) which is taking longer. This grant will help me to focus on this project for a shorter burst of development to completion, rather than working on it here and there as able.

Completeness Criteria

Distribution released to CPAN, complete with a decent test suite and documentation. Further convert the Alien::GSL manpage to the Alien::Base framework, giving an in-the-wild example for others to follow.

Bio

I am a Physics Ph.D. candidate at the University of Illinois at Chicago. I am writing a large simulation, a major part of my thesis, which uses Perl and my the Math::GSLx::ODEIV2 manpage module. This in turn requires version 1.15 of the GNU Scientific Library (GSL). I want my committee to be able to try out the simulation on their own; I have worked hard to make sure that it has a nice API, but certainly I cannot install it for each of them on their computers. Unfortunately, very few Linux distros come with this newest version of GSL (although this is always improving) and Windows users certainly will not have it.

My goal is to be able to have my committee members install Perl (Strawberry presumably on Windows) and my simulation module, which depends on the Math::GSLx::ODEIV2 manpage and therefore the Alien::GSL manpage (which would now depend on the Alien::Base manpage). From their perspective, I want it to be a totally transparent installation.

Since I have already made this effort, I would like it if my work on Alien::GSL could be used to make the task of writing other Alien:: modules easier for other authors.

2012Q1 Grant Proposal: EPublisher Website

nobody@example.com (Alberto Simões) — 2012-02-04T13:36:35Z

Boris Däppen

Email:

[hidden email]

Amount Requested:

800$

Synopsis

Develop and deploy a website which offers the service of rendering different content into eBooks. There will be several input and output formats supported by the website. An explicit input option will be CPAN documentation. POD in general will be an input option. Output options will be eBook formats. The open EPUB but also Amazons MOBI format will be implemented.

The framework used by the website will offer a plugin architecture, so that other input or output formats can be developed and published on CPAN.

Benefits to the Perl Community

Perl developers will benefit from a webservice, which allows them to receive CPAN documentation or other POD content as eBooks. They will also profit from several deliverables which will be published as Perl code in CPAN.

Deliverables

Develop and publish a module (e.g. EBook::MOBI) which offers a simple Perl interface to create eBooks in the MOBI format (which is used by Amazons Kindle).
Maintain and debug the framework EPhublisher (published by RENEEB on CPAN), including writing plugins for the architecture, which allows simple transformation between different document formats.
Develop and run a website, which offers a simple user-interface, to receive different content a an eBook.

Project Details

CPAN module for the MOBI format

The first deliverable will be a module (e.g. EBook::MOBI) released to CPAN. This module will allow Perl developers to write MOBI eBooks, using an object orjented interface in Perl. The Module will borrow some code from the project MobiPerl (which is not located on CPAN, it can be found here: https://dev.mobileread.com/trac/mobiperl). MobiPerl is GPL3 and the developer is fine with it being used (See forum thread: http://www.mobileread.com/forums/showthread.php.

Maintaining Framework

The framework EPublisher from Renée Bäcker will be used for this purpose. The author supports me in debugging the module and in writing input and output plugins for it.

Developing Website

The website will be developed using Mojolicious. It will have a simple user interface (e.g. similar to https://metacpan.org/).

Offering the Webservice

The website will be offered by http://perl-services.de/, so that e.g. Perl programmers can use it as a free service.

Inch-stones

I will work about 10 hours per week on this project, so it will take much longer than a week for sure.

The first deliverable will be the module for creating MOBI formats. This module will be available on CPAN at least in february 2012.

Writing plugins and some debuging work on the publishing framework will probably find a useful result in march 2012.

The development of the website will take some time, because I have to work myself into Mojolicious. I hope the code will be ready for use at end of march.

So my inch-stones are:

Release CPAN module (something like EBook::MOBI)
Bugfixing of publishing framework from Renée Bäcker.
Writing plugins (input: CPAN, POD / output: EPUB, MOBI (PDF)).
Coding the website.
Taking the webservice online.

Project Schedule

I can start immediately. The webservice should be ready for users in May 2012. Perl developers can profit from some deliverables (e.g. EBook::MOBI) already in February 2012.

Completeness Criteria

CPAN module for writing eBooks in the MOBI format.
Plugins (input: CPAN, POD / output: EPUB, MOBI) for Renées framework.
Website online.

Bio

I successfully finished a bachelor level degree in IT at an university of applied science in Zurich, Switzerland in 2006. I learned my first Perl at a IT-service company, located in Zurich, after my studies. Since 2010 I am studying a masters in philosophy of technology in Darmstadt, Germany. In my spare time I am programming Perl for a small Perl-company (perl-services.de)

I enjoy programming Perl since several years, and the fun continues until today. I also have excellent support in this project from Renée, so progress is assured.

2012Q1 Grant Proposal: Easy installation of non-CPAN dependencies

nobody@example.com (Alberto Simões) — 2012-02-04T13:31:28Z

Name:: Alexandr Ciornii (CHORNY)
Email:: [hidden email]
Amount Requested:: $2000

Synopsis

Many CPAN modules depend on non-CPAN libraries (or programs). Installation of such module requires that library be installed before installing CPAN module. For beginners it is more confusing as library may be installed, but its headers may not. I'd like to automate this step and install corresponding Linux package as part of module install.

Currently main module of this project has codename CPAN::non, it will be renamed later after discussion in module-authors mailing list.

Benefits to the Perl Community

For any user installing CPAN modules, as it will become much simpler. It is especially important when installing many modules or in case of inexpirienced user.
Module authors, because module installation will become easier and they would not need to periodically explain that users need to install non-CPAN library.
CPAN/OS porters, because dependencies will be listed more clearly, in future it may be also automated.
Package tools like Shipwright or carton can use CPAN::non to install OS packages.

Deliverables (to CPAN)

New version of Linux::Distribution. XML::Parser will automatically install expat (of course if user agrees). Module for yum interaction. CPAN::non and Module::Install plugin will be released.

Project Details

CPAN modules that want to get rid of this problem usually bundle libraries either directly or as Alien:: CPAN distributions. This creates several problems:

More work for authors and OS package maintainers
Duplication
Necessity to update library if serious problem is found
Necessity to replicate additional patches for every OS

But it is easy to install such dependency with OS tools. CPAN module will have configuration file that will specify build and runtime dependencies. CPAN::non will be bundled with module, or installed using configure_requires in META.yml/json. In future OS vendors may include it into their versions of Perl.

CPAN::non will detect version of Linux, check if dependency and its headers are installed, find out which name has dependency and headers on this OS from configuration file, install it.

For yum there is no module CPAN, I plan to write module for it. Also there is no general module that would be dist-neutral, so I will write it too.

Module::Install plugin will be convenient as Module::Install allows automating bundling of run-time dependencies.

XML::Parser will be used as a first test for CPAN::non as I have comaint rights for it.

Inch-stones

Improve tests and release new version of Linux::Distribution.
Test Linux::APT. Write/submit patches if necessary.
Install CentOS and Fedora. Write a module for yum interaction
Write and test an OS-independent module for software package installation (apt and yum)
CPAN::non - prototype
CPAN::non - write code
CPAN::non - test
CPAN::non - documentation
Use it in XML::Parser for expat
Module::Install plugin
Blog about it, help CPAN authors use it, improve main module if necessary

Project Schedule

The project will take 2-3 months. I will start right after grant approval.

Completeness Criteria

New versions of Linux::Distribution and XML::Parser.
CPAN::non, Module::Install plugin and module for yum released.

Future

This project is a working solution, but after it will be adopted, more interesting things may be implemented, like support for FreeBSD, even Windows, support in tools for OS package generation, etc.

Bio

I maintain many CPAN modules and fixed bugs in many more. Participated in such projects as Strawberry Perl, Padre, ExtUtils::MakeMaker and others. Contributed to Module::Install. My CPAN account: http://search.cpan.org/~chorny/ and Ohloh: https://www.ohloh.net/accounts/chorny/positions.

For 9 years I was a lead programmer of credit card payment system InterCardPro. Currently work for kariera.md.

Perforce auth-check trigger for Unix passwd file

nobody@example.com (Laufeyjarson) — 2012-02-04T04:56:04Z

I just installed Perforce (what a hassle!) and discovered that it won’t authenticate to the local Unix accounts out of the box. The nice folks at Perforce let you call an external trigger, and provide a complex sample that will handle talking to Active Directory, but don’t have one do handle ordinary Unix passwd access.

Totally annoying. A simple Perl script fixed it. Don’t know why they didn’t put it in their samples, though.

#!/usr/bin/perl

use strict;
use warnings;

=head1 NAME p4_check_passwd

Called with the user name on the command line and the password on standard
input, this program returns a success or failure depending on that password's
correctness.

Uses getpwnam to do so, and must run as root on in shadow group to access
/etc/shadow and password data.

This should be configured in p4 triggers as an auth-check trigger:

Triggers:
unixpw auth-check auth "/home/perforce/bin/p4_check_passwd.pl %user%"

This uses the crypt system call, which should handle any variants of encryption
in your password file. It also uses the getpwnam() function to access
the password data, so should be able to access data from NIS or other
configured password system.

=cut

# Perforce wants all messages on stdout. Guarantee that.
open(STDERR, ">&STDOUT") or die "Can't redirect to stdout\n";

# Get the user name
my $username = shift;
die "Need user name on command line\n" unless $username;

# Get the password
my $password = <STDIN>;
chomp $password;

# Get username and password from the system
my ($name, $passwd) = getpwnam($username);

die "Can't log in as $username\n" unless ($name and $username eq $name);
die "Can't access encrypted passwords\n" if(not defined $passwd or $passwd eq ''
or $passwd eq 'x');

# Check password and return correct exit code or die.
if(crypt($password, $passwd) eq $passwd) {
	exit 0; # Silent, successful exit worked!
} else {
	die "Can't log in as $username\n";
}

Works for me, anyway.

A Real Developer Challenge

nobody@example.com (Michael G Schwern) — 2012-02-03T23:21:00Z

Spotify is having a coding challenge to find "top-notch talent to join our NYC team". The challenge is to solve the most algorithmic puzzles in four hours... alone. "You may not cooperate with anyone, and you may not publish any discussion of solutions." What sort of developer will win this competition? Someone who is quick, dirty, has a mathematical mindset and lucky enough to write something that happens to work for the test data set. The "rockstar". Is this somebody you want on your team? Would you want to maintain their code?

Last year while on contract, the company in question was passing around their coding problem they used to test new hires. It was pretty typical stuff: give the data going in, the data they want out, and write a little program to do the transform. They even supplied most of the program, including a test; the prospective hire just needed to write one sort subroutine which could deal with "Low", "Medium" and "High" as well as numbers.

Predictably, this halted all coding in the office for a solid half day while everyone figured out the most clever way to sort the data. My opus was to observe that the input data was already sorted, so I redefined the shuffle() routine. The best one was from a co-worker who observed that "Low", "Medium" and "High" sort correctly by their last letter in reverse order. It was fun for us, but it wasn't very useful.

This is a pretty typical coding problem used to judge potential hires, and it sucks. All it tells you is the candidate is not completely incompetent. Why do we keep using them? They're easy. They're easy to think up, easy to judge and easy to administer. They're also the sort of clean, algorithmic problems a stereotypical programmer loves to solve. Do they have anything to do with detecting a good developer? No. Can we fix it? Yes!

Before this can be fixed, first we have to work out what a project wants in a developer. What do developers do all day? We can work this out by reversing every contrived element of our typical programming contest: clean, well defined inputs; clean, well defined expected output; a clear description of the algorithm; pre-existing template code; pre-existing acceptance tests. When's the last time you were handed a problem like that in the real world?

Instead, we get poorly defined inputs, sample input that has anything to do with reality being a luxury, inputs riddled with mistakes. Expected behavior and output are vaguely defined. As a developer we're presented with a clean sheet, no template, no tests, just a blinking cursor and a blank page.

How does one solve the contrived example? First the specification, inputs and outputs are carefully examined. If there's any ambiguity it's discussed up front with the person who probably wrote the problem and understands it perfectly. Then the candidate goes off and writes some code until it passes the tests. There is little or no interaction with people, everything is handed on a silver platter in unambiguous terms.

How does one solve the real example? First you have to find somebody who understands the problem, usually not a programmer, and discuss the problem. Then you drag some samples out of them, converting them into a format you can actually use. The user probably doesn't know what they really want, so the behavior/output will be ill defined. Pressed, the user will make up something that you know will be wrong as soon as you show it to them. Armed with this "information" you hammer it into some sort of algorithm and now need to write some code.

But you don't just hammer out a script. You need to do it in a way that matches the team's coding style. You'll probably want to make the meat of it reusable, so you need to write it as a library not just a one-off script. It'll have to deal with the inevitable bad input, which means good error handling and recovery. Other people will have to use it, which means good documentation. It needs tests written in a way which works with whatever integration server the team is using. And, of course, it should all be checked into version control with well defined and logged commits.

And then, when you've written all that and got it working, you take it back to the user and they tell you it's not what they wanted. Or they show you some new input that doesn't match what they originally said. If you're good... and lucky, your code is robust enough to handle it. If you're not... back to coding with you! Repeat until dead.

In that light, a good developer is one who works well with others, but also can make a multitude of small, detailed decisions about a problem they know very little about. They need to take vague requirements and work with a user to turn them into something a computer can do repeatedly. A good developer looks beyond the immediate requirements and thinks ahead ensuring the code is flexible enough to withstand future change. A good developer writes code not for themselves, but for everyone else on the team.

So... how do you test all that? And in less than an hour? Turns out it's pretty easy with some simple modifications to the classic example. You keep the same basic algorithmic problem, but you give it to the candidate the way a user would. It can be as simple as this:

Could you sort this data please? [Excel document attached]

You can be a bit more clever, feeding the candidate what seems like enough information to do the work but is full of subtle ambiguities, tempting them to take the easy route and just get to coding, weeding out those with the undesirable tendency to program without questioning what they're doing and why, but that's the crux of it.

Now sit back and see what the candidate does with that. Respond to their questions, but remain in the persona of a user. What you're looking for here is how they push back. What sort of questions to do they ask to gather and clarify the requirements? What sort of assumptions do they make? How well do they bridge the communications gap between customer and programmer?

Given a blank page to code on, what do they do with that? Do they write tests and documentation? Do they write a quick script or a library? Do they use version control? Do they ask about your code standards? Do they use pre-existing libraries? Do they write to the letter of the requirements or leave some flexibility? And again, what are their communications like through this process?

Once they've submitted their first solution, change the requirements on them subtly by offering a second set of inputs. These will be the same but subtly different. Maybe throw in some Unicode or deliberately malformatted lines. Maybe change the sort criteria. Does their code fail gracefully? Messily? Silently? Do they notice the changes? What shape does the discussion over the change take? Are they indignant about the changing requirements, do they take them in stride? How much work does it take them to adapt their code to the new requirements?

This test will take more time than a traditional puzzle test, but it can be administered easily enough over email and doesn't involve more than a few minutes attention at any given time. A dud developer can sink your team and cause far more damage than good, so it's worth the little bit of extra effort to find out if they can actually do what developers do, and not just solve clever puzzles.

True + True == 2

nobody@example.com (Michael G Schwern) — 2012-02-03T22:47:28Z

I've seen lots of new Perl programmers confuse 1 and true before, usually something like this:

do_something if boolean_function() == 1;

This is not only redundant, but it's also brittle and redundant. In Perl, it's very easy for a boolean function to return something other than one.

sub has_stuff {
    ...
    return scalar @stuff;
}

This code I found today in Module::Build::Base takes the cake.

if ( $self->check_prereq + $self->check_autofeatures != 2) {

(Yes, I fixed it)

UPDATE: Whoops, I broke it! As rsimões pointed out in the comments, the original does not short circuit and my change does. That was not a bug, both check routines should always run (a consequence of the checks having side effects). I had to think about it a bit, and the simplest thing I came up with is:

if( grep { !$_ } $self->check_prereq, $self->check_autofeatures ) {

Which goes to show, and I should have known this myself, don't assume the author is an idiot. There's usually a reason why they did what they did. Find it before touching the code.

A Practical Use for Macros in Perl

nobody@example.com (chromatic) — 2012-02-03T20:46:43Z

People occasionally ask for practical examples of macros when I lament the lack of macros in Perl 5. While I'm usually pleased at the degree to which Perl lets me design code to get and stay out of my way, sometimes its abstractions just aren't quite enough enough to remove all of the duplication available.

(I've been refactoring one of our business projects in preparation for another round of deployment in the next couple of weeks. We could launch without these improvements, but administrative work took almost two weeks longer than the afternoon I'd planned for it, so I decided it was worth my time to reduce technical friction so that further improvements are easier. More users means more work, so why not accelerate that work while I have the chance? I have another longer technical post to write to praise the use of Moose roles for a plugin system and to show off the stupidly-great task launcher, but that's for later.)

I found myself writing two code couplets that were similar enough they triggered my "Hey, refactor away this duplication!" alert. It's extra sensitive, because I know I'll have a few more couplets like this in the very near future:

while (my $stock = $stock_rs->next)
{
    my $pe_update = $self->analyze_pe( $stock );
    $stock_txn->add( $pe_update ) if $pe_update;

    my $cash_yield_update = $self->analyze_cash_yield( $stock );
    $analysis_txn->add( $cash_yield_update ) if $cash_yield_update;
}

The *_txn variables contain objects representing deferred and scoped SQL updates. I'll talk about that at YAPC::NA 2012 in When Wrong is Better.

The general pattern is this: for every stock in the appropriate resultset, call a method in this plugin. The method will return nothing if it fails (or has nothing to do) or it will return data to be added to the appropriate transaction. I have at least two types of transactions available here at the moment, and may have more later: one transaction updates stock data and the other updates analysis data.

I have several options. I could rework the data model so that this stage always only updates one transaction, in which the loop body could instead look like:

{
    for my $method (qw( analyze_pe analyze_cash_yield ))
    {
        next unless my $result = $self->$method( $stock );
        $txn->add( $result );
    }
}

This technique of hoisting the variants into an ad hoc data structure and using existing looping techniques works well sometimes. (I use it in other parts of the system.) It's relatively easy to expand, even though it moves interesting information ("I'm calling the analyze_pe method!") to a place where tools have more trouble finding it. (I search for >analyze_pe when I want to find method calls.) You may have used something similar to define several parametric methods at BEGIN time. It's the same type of pattern, and while Perl 5 provides most of the tools necessary to allow this, it doesn't natively express this pattern well.

I could also change the transaction object's add() method to do nothing when it receives an empty list of arguments. I like that in some ways, but I don't like it in others. I've come down on the side of keeping its invariant (it always takes only one scalar as an object) pure for now. If I change it to take a list of updates, that might be the right time to reconsider this.

What I notice in the code as it stands right now is that the individual variables $pe_update and $cash_yield_update are synthetic variables. They only exist to support the code as written; they're not necessary for the algorithm. If I were to modify this code but only this code, I'd really rather write:

{
    ADD_TXN_WITH( $self, analyze_pe,         $stock, $stock_txn    );
    ADD_TXN_WITH( $self, analyze_cash_yield, $stock, $analysis_txn );
}

... though that syntax doesn't thrill me either. The clearest possibility I see right now is:

{
    $stock_txn->add(    SKIP unless $self->analyze_pe( $stock )         );
    $analysis_txn->add( SKIP unless $self->analyze_cash_yield( $stock ) );
}

... where SKIP does some magic to move to the next statement, not the next loop iteration. (I have some ideas how to write XS to make this work, but that creepy yak needs a shave and some mouthwash.)

The second best option right now is adding a function or method as indirection to encapsulate the synthetic code. I'd rather avoid synthetic code, but at least it reduces the possibility of copy and paste bugs.

For now, with only two steps in this analysis, I'm leaving it as it is. Two repetitions of something this similar set off my refactoring alarm, but I resist the urge for refactorings this small until I see three instances of near-duplicate code.

Perl devroom @ FOSDEM2012

nobody@example.com (claudio) — 2012-02-03T18:29:05Z

Just a short reminder of the Perl talks at FOSDEM2012.

The Perl dev-room will be held this Sunday February 5th, from 9 to 17h on room AW1.121. We have a wide range of talks. Some talks target Perl programmers with subjects ranging from a beginner to an advanced level. Other talks don’t focus on the language itself, but rather on projects that use Perl as a building stone.

So please, drop by if you are at FOSDEM…

Event	Speaker	Room	When
Room: AW1.121
Sunday 2012-02-05
Welcome to the Perl devroom	Claudio Ramirez	AW1.121	09:00-09:05
Moose Primer	Nicholas Perez	AW1.121	09:05-09:25
Advanced Moose Techniques	Nicholas Perez	AW1.121	09:35-09:55
Perlude: a taste of Haskell in Perl	Marc Chantreux	AW1.121	10:05-10:45
Perlito	Flávio Glock	AW1.121	11:05-11:45
The LemonLDAP::NG Project	Clément Oudot	AW1.121	11:55-12:15
LedgerSMB: Open source accounting running on Perl	Erik Huelsmann	AW1.121	12:25-12:45
Modern PerlCommerce	Stefan Hornburg	AW1.121	13:25-14:05
Rapid real-world testing using git-deploy	Ævar Arnfjörð Bjarmason	AW1.121	14:15-14:35
POSIX::1003	Mark Overmeer	AW1.121	15:00-15:40
The FusionInventory Project	Guillaume Rousse	AW1.121	15:50-16:10
Using Moose objects with Memcached	Marius Olsthoorn	AW1.121	16:20-16:40

Filed under: Uncategorized Tagged: dev-room, fosdem, FOSDEM2012, Perl

Perl on the command line

nobody@example.com (Gabor Szabo) — 2012-02-03T16:33:43Z

While most of the tutorial deals with scripts saved in a file we'll also see a couple of examples for one-liners. Even if you are using Padre or some other IDE that would let you run your script from the editor itself it is very important to familiarize yourself with the command line (or shell) and be able to use perl from there.

For the full article visit Perl on the command line

Perl in Plat_Forms 2012 Web Development Contest

nobody@example.com (chromatic) — 2012-02-03T01:33:35Z

Thanks to Lars Dɪᴇᴄᴋᴏᴡ for coordinating entries into the Plat_Forms 2012 web development contest. This annual contest pits some of the best developers of frameworks and languages against each other to solve real problems. Perl developers have participated for several years. Teams using Catalyst, Dancer, and Mojolicious have all demonstrated their platform strengths and weaknesses.

No Perl team has won the contest—yet. With the rise of modern Perl 5, the extra energy from competition between all of the great platforms, and (of course) the power of the CPAN, we all expect great things. Consider entering and showing what we can do.

TODO February 2012

nobody@example.com (Gabor Szabo) — 2012-02-01T11:33:07Z

In January I started to write annual and monthly reports and TODO lists. On one hand, I hope that this will help me track my progress and on the other hand it lets you, my reader, take a look at my plans and participate in the journey.

For the full article visit TODO February 2012

On CPAN Namespaces

nobody@example.com (Joel Berger) — 2012-02-01T03:32:52Z

After the public buyers-remorse that I (almost) had on the name of my latest module Tie::Select, I got to thinking about the filling of CPAN namespaces.

CPAN is awesome; many have said that it is Perl’s killer app. It has tons of modules to do many many things. The problem is that some of the obvious names are gone. Certain namespaces have implications. For Tie::Select I thought of IO::Select, which exists. I thought of going underneath an existing module’s namespace, for example IO::Handle::Select, but this implies, at least to me, that it is either a part of, or derivative of IO::Handle. Its not, and in fact it isn’t even OO as that module is.

I have settled on the current name for now, but I am not totally happy with it. I got to thinking about other naming problems I’ve had. (And note I am not complaining about any of this, just ruminating):

Duke Leto’s SWIGing of the GSL library effectively took up the entire Math::GSL namespace. My Math::GSLx::ODEIV2 needed that little x to “clear things up”
One might expect that my MooseX::Types::NumUnit uses MooseX::Types, but it doesn’t. Perhaps it should, or perhaps it should be in another namespace. Then again, this is the descriptive one.
It seems like there are more that I am forgetting

Anyway, it begs the question, as CPAN continues to mature, what happens as the namespaces fill up? What happens to old dead modules Tie::CSV_File which aren’t really worth fixing, but will decrease the searching signal-to-noise?

Gasp! What happens when ACME:: fills up!

I know that some people’s answer will be to adopt the old modules. But what if I don’t plan to keep the API or the implementation. What if the functionality is different but the names should be similar (my Tie::Select vs IO::Select).

I know I’m rambling a bit now and I can’t say that I know what I want, but I know what I don’t want. I don’t want new CPAN modules to have to start picking odd names just to have their own namespaces. I’m sure many of us have see Miyagawa’s module’s funny names and their reasoning.

But there just aren’t enough descriptive names.

Why I Run Tests on Install

nobody@example.com (chromatic) — 2012-01-31T18:41:08Z

Jonathan Swartz makes a polemic statement:

cpanm and perlbrew should not run tests by default.

His points are reasonable, but his complaints are mostly about side effects and not the real problem. (I should clarify: the real problem I encounter.)

If running tests slow down installs, speed up the tests. (Do you want to get the wrong answer faster? Easy: it's 42. No need for a quantum computer to do the calculation in constant time. This algorithm is O(0).)

If running tests exposes the fragility of the dependency chain, improve the dependency chain.

If dependency test failures prevent the installation of downstream clients... this is a weakness of the CPAN toolchain. A well-written test suite for a downstream client should reveal whether bugs or other sources of test failures in a dependency affect the correctness of the client.

Note the assumptions in that sentence.

Anyone who's experienced the flash of enlightenment that comes from working with well tested code and who's shared that new zeal with co-workers has undoubtedly heard the hoary old truism that testing cannot prove the complete absence of bugs. It's no less true for its age, though it's also true that good testing only improves our confidence in the correctness and efficacy of our code.

For me, a 95% certainty that my code works and continues to work for the things to which I've tested it is more than sufficient. I focus on testing the things I'm most likely to get wrong and the things which need to keep working correctly. (I don't care much about pixel-perfect placement, but I do care that a book's index uses the right escapes for its data and markup.)

Without tests running on the machines themselves in the environments themselves where I expect my code to run, I don't have that confidence.

Put another way, I'm either not smart enough or far too lazy to want to attempt to debug code without good tests. That's why I write tests, and that's why I run them obsessively. That's good for me as a developer, and you're getting the unvarnished developer perspective.

I also care about the perspective of mere users. (Without users, we're amusing ourselves, and I can think of better ways to amuse myself than by writing software no one uses.).

Yes, an excellent test suite can help a user help a developer debug a problem. Many (most?) CPAN authors have had the wonderful experience of receiving a bug report with a failing test case. Sometimes this even includes a code patch.

Not all users are developers of that sort, nor should they be.

The CPAN ecosystem has improved greatly at automated testing and dependency tracking, but we can improve further. What if we could identify the severity of test failures? (We have TODO and SKIP, but they don't convey semantic meaning.) What if we could identify buggy or fragile tests? (My current favorite is XML::Feed tests versus DateTime::Format::Atom because it catches me far too often, it doesn't affect the operation of the code, and it's a stupid fix that's lingered for a few months.) What if the failures are transient (Mechanize relying on your ISP not ruining DNS lookups for you) or specific to your environment (a test suite written without parallelism in mind).

As Jonathan rightly implies, how do you expect an end-user to understand or care about or debug those things?

I'm still reluctant to agree that disabling tests for end-user installations is the right solution. I want to know about failures in the wild wider world. I want that confidence, but I can't bring myself to trade away that confidence for the sake of a little more speed of installation.

Yet his point about lingering points of fragility in the ecosystem are true and important, even if the proposed solution of skipping tests isn't right. Fortunately, improving dependency management and tracking and use and testing can help solve both issues: perhaps to the point where we can run only those tests users most care about and identify and report material failures in dependencies.

Rakudo Star Jan 2012 Released

nobody@example.com (Pradeep Pant) — 2012-01-31T15:25:49Z

Rakudo Perl, a Perl 6 compiler for the Parrot virtual machine is released .
The tarball for the January 2012 release is available from http://github.com/rakudo/star/downloads.

See this link for official announcement.

Filed under: Perl, Programming, Server side (Perl 5, Perl 6) Tagged: linkedin, perl6, Rakudo

A localizable interface to `select`

nobody@example.com (Joel Berger) — 2012-01-31T11:46:09Z

Have you ever forgotten to restore your original STDOUT (or other handle) when you want to select another one for a brief time? Does the usage

my $stdin = select *STDERR;
print "To STDERR";
select $stdin;

seem clunky?

I have always loved the interface provided by File::chdir. It gives you the $CWD variable. When you assign to it you change the working directory. Even better, if you do that with the local directive, you dynamically scope the change!

It struck me, while I was working on the test suite for another project, that I wished that I could have that same kind of interface to select. And why not, they both need changing and restoring of a global property? So I gave it a try, and lo and behold it was simple!

So here it goes, I present Tie::Select, which provides the $SELECT variable. Use it like this and I think you will wonder why you ever did it the other way!

 use Tie::Select;

 {
   local $SELECT = *STDERR;
   print "This goes to STDERR";
 }

 print "This goes to STDOUT";

The right way to install CPAN modules

nobody@example.com (Gabor Szabo) — 2012-01-31T11:16:02Z

There are several ways to install CPAN modules and about everyone advocates something else. You can install the modules supplied by your vendor (e.g. using aptitude on Debian). If the module is not available from the vendor you can prepare a package (in the case of Debian using dh-make-perl) or you can install the modules using a CPAN client. In that case you can do it to the standard system directories or using local::lib. You could also build your own perl with or without perlbrew and then you can install everything with a CPAN client.

For the full article visit The right way to install CPAN modules

Perl 5 Grant Application: Travel to QA Hackathon

nobody@example.com (Karen) — 2012-01-31T10:18:05Z

We have received the following Perl 5 grant application from Ricardo Signes.

Before the Board votes on this proposal we would like to have a period of community consultation. Please leave feedback in the comments or if you prefer send email with your comments to karen at perlfoundation.org.

Name

Ricardo Signes

Project Title

Perl QA Hackathon 2012
Amount Requested: $900

Synopsis

This grant will be used to pay for travel for Ricardo Signes to and from the Perl QA Hackathon held in Paris in Q1 2012.

Benefits to Perl 5

I have attended three of the four Perl QA Hackathons (Oslo, Birmingham, Amsterdam) and have, at each of them, been able to contribute several solid work days of very productive work to the infrastructure behind the CPAN and related tools. Specifically, I was one of the chief implementors of the new CPAN Testers platform (Metabase) and built the Fake CPAN system for testing CPAN tools, and several reusable software libraries that are used to power both Metabase and Fake CPAN.

In 2012, I hope to continue some work performed with David Golden in 2011, refactoring PAUSE and adding more regression tests to it. PAUSE is the system which processes contributor uploads to the CPAN, manages CPAN contributor identity, and builds the CPAN indexes used by CPAN clients to locate libraries for installation.

In previous years, I also spent a significant amount of time working with other attendees on their contributions, and plan to do the same this year. This is one of the several reasons that attendance in person is incomparably superior to "virtual attendance."

Deliverable Elements

The QA Hackathon does not have a set agenda, so promising specific work product from it up front seems unwise. I have detailed, above, the sort of work that I am almost certain to do, however. Further, I will provide a public, written report of my activities at the Hackathon.

Any software that I produce will be released under the Perl 5 standard license terms.

Applicant Biography

I have been building software in Perl professionally for about twelve years. I am a frequent contributor of original software to the CPAN and a frequent contributor to, or maintainer of, other popular CPAN libraries. I am also a contributor to the core Perl 5 project, and its current project lead.

I have been the recipient of TPF grants twice before, both of which were completed promptly and successfully.

CPAN Release Business::DK::Phonenumber 0.04

nobody@example.com (jonasbn) — 2012-01-31T08:16:42Z

My friend, colleague and fellow Copenhagen Perl Monger Asbjørn Thegler reported a bug with Business::DK::Phonenumber (BDKPHN-10).

Looking at the code we found out that parts of it made no sense. Asbjørn provided me with a good example demonstrating the bug, so this was quickly translated to a test (BDKPHN-10) and the non-sense code and the issue could be addressed.

The release has been uploaded to PAUSE and should be available on CPAN.

Thanks to Asbjørn for reporting the bug.

jonasbn, Copenhagen

PS. this post was cross-posted from the project blog on Business::DK::Phonenumber

Stop running tests on install!

nobody@example.com (Jonathan Swartz) — 2012-01-31T06:51:13Z

chromatic shows users how to run tests faster on cpanm and perlbrew installs. This strikes me as well-meaning advice that misses a much more basic point:

cpanm and perlbrew should not run tests by default.

This may sound heretical. Perl has always had a strong testing culture, and end-user testing may have once played a valuable role in testing a distribution under many systems. But we now have a CPAN Testers network which will run tests on countless systems and Perl versions, and report failures back to the author promptly and automatically. Distributions can be sent through the Testers’ gauntlet before ever being officially released. In this environment, it’s hard to see much additional value in ad hoc end-user testing.

As Dave Rolsky points out, we install most other software without running tests and rarely give it a thought.

None of this would matter if end-user testing was free. But it is not.

The costs of end-user testing

Slower installs. On my system, a fresh install of Moose and its dependencies takes three times longer with tests (2 minutes versus 41 seconds). A fresh install of Catalyst and its dependencies takes nearly four times longer with tests (9.5 minutes versus 2.5 minutes).

How many new Perl users find CPAN installs much slower than they need to be? How many would choose a 3-4 times speedup if they knew it was an option? It’s like having a turbo button and leaving it unpressed by default.

False positives. The more tests CPAN authors write, the more likely an occasional false-positive failure sneaks through (as in “Failed 1/1746 tests.”) In most such cases the module will still work for the user’s purposes. But the default behavior is to prevent the module from being installed at all. If your module depends on other modules, then any failure up the dependency chain likewise prevents your module from being installed, even if the failure has no bearing on your module’s efficacy.

How many new Perl users have unnecessarily failed to install a module like Moose or Catalyst because of an obscure, temporary failure deep in the dependency chain?

Fear of dependencies and code reuse. Slower installs and false positives are the main reasons why people complain about distributions having “too many dependencies”. (If the dependencies installed quickly and reliably, as they do with –notest and apt-get and yum, would anyone complain or even notice?) These complaints in turn encourage module authors to reduce or eliminate their dependencies, thus reinventing where they could be reusing.

It’s the wrong default for new users

Some veteran Perl folks may like tests to run on every install. That’s fine. But I suspect new Perl users just want things to install quickly and reliably, and in any event don’t have the experience to evaluate or take action on a test failure (especially one in an obscure dependency). For these users running tests is simply the wrong default.

I turn on –notest on each system I administer and preach it enthusiastically to every new Perl user I encounter. But I wish I didn’t have to mention it at all.

SQL Change Management Sans Duplication

nobody@example.com (David E. Wheeler) — 2012-01-30T16:00:00Z

In the previous episode in this series, I had one issue with regard to SQL change management that I wanted to resolve: duplication of code between deploy and revert scripts sucks. Worse still is the duplication of code to change just one line of a procedure. Here’s how I propose to eliminate the dupes.

Perlbuzz news roundup for 2012-01-30

nobody@example.com (Andy Lester) — 2012-01-30T15:05:03Z

These links are collected from the Perlbuzz Twitter feed. If you have suggestions for news bits, please mail me at andy@perlbuzz.com.

Open source is improv, so say "Yes and" (petdance.com)
craigslist donates $100,000 to the Perl Foundation (news.perlfoundation.org)
Text::AsciiTeX renders TeX formulas as ASCII art (blogs.perl.org)
The most important pig in the office (with nod to @chipsalz, too) (petdance.com)
An hour-long video of Damian Conway talking Perl and physics (blip.tv)
Speed up Perlbrew with parallel tests (modernperlbooks.com)
Visualizing the Perl 5 support policy (dagolden.com)
MooseX::Role::Loggable adds logging goodness to your objects (blogs.perl.org)
Rakudo Star 2012.01 released (rakudo.org)
What's next in Rakudo (6guts.wordpress.com)
I'll be presenting "29 ways to get started in open source today" at YAPC:NA (act.yapcna.org)
DC-Baltimore Perl Workshop is April 14th. $25 for two tracks of talks. (dcbpw.org)

Fixing Perl5 Core Bugs: Report for Month 22

nobody@example.com (Karen) — 2012-01-30T08:46:42Z

Dave Mitchell writes:

As per my grant conditions, here is a report for the December period.

Spent it continuing my overhaul of the re_eval mechanism. This month I finished off fixing literal code blocks within patterns; that is to say, in all places where a code block appears literally within a quoted-regex context in perl source, that code block will only be compiled
once, at the same time as the surrounding perl code, and then passed round as necessary rather than being recompiled.

The main things left to do are to fix non-literal code blocks, e.g.

$code = '(?{...})';
use re 'eval';
/abc$code/;

and make the code blocks be invoked in such a way that things like die,
return and caller() will work.

Over the last month I have averaged 8 hours per week.

As of 2011/12/31: since the beginning of the grant:

94.9 weeks
1052.4 total hours
11.1 average hours per week

There are now 248 hours left on the grant.

Report for period 2011/12/01 to 2011/12/31 inclusive

Summary

Effort (HH::MM):

0:00 diagnosing bugs
37:00 fixing bugs
0:00 reviewing other people's bug fixes
0:00 reviewing ticket histories
0:00 review the ticket queue (triage)
-----
37:00 Total

Numbers of tickets closed:

0 tickets closed that have been worked on
0 tickets closed related to bugs that have been fixed
0 tickets closed that were reviewed but not worked on (triage)
-----
0 Total

Short Detail

37:00 [perl #34161] METABUG - (?{...}) and (??{...}) regexp issues

Call of the callback: Inversion of Control or just coolness?

nobody@example.com (Sawyer X) — 2012-01-30T08:32:25Z

Suppose I have a Runner which holds a Prober object. I need the Runner to call the Prober and update stuff according to the result. Pretty simple, right?

my $prober = Prober->new(...); my $result = $prober->check; if ( $result eq 'whatiwant' ) { $self->update($stuff); }

However, I'm running in asynchronous mode, which means I need the prober and update to run somewhere in the loop, and I can't just sit around and wait for a reply. I gotz otha dingz to do, mon! So, I have two options:

- I call the Prober and sends it the Runner object ($self to you). Once the Prober is done, it will call the correct update method in the Runner object object.
- The Runner calls the Prober and sends it a callback containing updating code.

I asked around at $work and had been suggested to use the first method. Instead, I actually went for the second. (let this be a lesson to you: always listen to suggestions, but make up your mind youself!)

What I didn't like about the first method is that I'm sending the entire Runner object to a very small intermediate object. Felt kind of dirty. But, the more I thought about it, I noticed there was something else I didn't like about it. I'm basically putting a lot of Runner object into the Prober. That's not right...

By sending a callback to the Prober instead of the entire Runner object, I gain the following:
- The Prober doesn't need to know about any object other than itself. It doesn't know what you send it (other than a callback), not does it care.
- If I need to change the updating code, I only need to change what I send the Prober. I never need to change the Prober itself, it remains exactly the same.
- I can send a lot of other stuff as well, not just updating code.

In my case I could even set this callback on the initialization of the Prober object. This is how it would look:
Prober->new( ... on_change => sub { $self->log("Updating..."); $self->update($stuff); # suppose this is curried method from $self->updater }, );

How cool is that? Very cool!

It would be much cooler though if I could present this situation to you in an interesting way, such as how Tom Wyant does it. Damn, I'm jealous! I'll try though!

Birthday pizzas!

nobody@example.com (C.J. Adams-Collier) — 2012-01-29T01:05:32Z

Happy birthday, Scarlet, Teresa, Hannah and Tyra!

What Google Thinks of Me

nobody@example.com (Andrew) — 2012-01-28T06:36:54Z

35-44? I still get ID’d for beer!

Also, I am obviously a geek (but I don’t like Java…).

Speed up Perlbrew with Test Parallelism

nobody@example.com (chromatic) — 2012-01-27T21:20:46Z

Steven Haryanto's Perl First World Problems #1 reminded me of something I've taken for granted lately.

You may have read my Controlling Test Parallelism with prove and Parallelism and Test Suites. I still have Test::Harness parallelism enabled by default on most of the machines where I install my own Perls. While I haven't yet filed tickets and tried to write patches for modules which need a little help to run tests in parallel, I've only found a few lately that need work. That's nice—having a module install through cpanm in five seconds is a lot better than ten seconds or more. (I like cpanm because it's fast and quiet, and part of its speed comes from not printing to the console.)

I like instant feedback.

Like Steven, I noticed quite a while that installing a custom Perl through perlbrew takes a while, but then I remembered that a lot of work went into the Perl 5 test suite to make tests run in parallel. (We did something similar with Parrot several years ago, and it changed the way I work forever.)

To run core tests in parallel, set the environment variable TEST_JOBS=n, where n depends on your computer. I use a value of 9 on a quad-core machine; in practice, that tends to keep the CPU busy while not blocking anything too long on IO. You can set it globally in your shell's configuration file or create an alias or wrapper for perlbrew.

As most of the time spent compiling and installing Perl 5 through perlbrew goes to running the test suite, this has saved me a measurable amount of time.

YAPC Achievements

nobody@example.com (brian d foy) — 2012-01-27T18:46:39Z

Want to make YAPC a game? Want elite points for doing things at YAPC? Want to be the person to give out the points?

JT Smith was at the Windy City Perl Mongers last night giving his The Game Crafter talk as an exemplar of the talks he wants at YAPC in Madison. It's more about his application of Perl to one of his businesses than just explaining another module or framework.

After the talk when JT and I were headed back uptown, we started talking about achievements for YAPC. You do something and you get some sort of recognition. That might be for giving a talk, showing up at the keynote, donating money, talking to Schwern, posting a picture of YAPC on Twitter, and all sorts of other things.

The trick however, is to find the capacity to implement all the ideas. Someone must already have implemented something close.

Yanick already has perl achievements (although it's not on CPAN, wtf Yanick? :)

QR codes are one of the big things at YAPC this year. I wonder if someone could combine the two. You access the URL, which identifies your scanning device somehow, and it activates the right achievement.

It would really be nice to integrate this into Act so you can have trans-YAPC achievements and so on, but hey, baby steps.

This is actually something I would like to do, but JT has me working on another big project for YAPC, and this year is all about focus.

Updating p5-* packages in pkgsrc

nobody@example.com (rhaen) — 2012-01-27T08:50:21Z

I am updating Perl packages in pkgsrc right now. pkgsrc is the package management system from the NetBSD project and it is great. It works on different platforms and different operating systems such as Linux, Solaris and even on FreeBSD.

The Perl packages really need some love there – some are horribly outdated and broken. There seem to be Perl packages in pkgsrc which are older than the versions which are shipped with the Perl core distribution. It’s alot of work.

Here what I will do in the next days – let’s see how far I can get:

update the packages (at least the most)
clean up the licenses
check the dependencies inside the Makefile.PL (script)

As I’ve mentioned on a talk – it’s not only fun to maintain Perl packages for a package distribution. But it seems to be fair – compared to someone who follows the KDE packages :)

Ah folks – here is something which I don’t like:

snowflake:p5-Archive-Tar rhaen$ cvs ci ../../doc/CHANGES-2012 
cvs commit: Up-to-date check failed for `../../doc/CHANGES-2012'
cvs [commit aborted]: correct above errors first!

Can i haz working version control system?

VCS-Enabled SQL Change Management

nobody@example.com (David E. Wheeler) — 2012-01-27T07:32:00Z

In my previous post, I outlined the basics of a configuration-file and dependency-tracking SQL deployment architecture. But I left it off wanting to eliminate the need for such a file and still have it all work. This post outlines just how to do that by relying on VCS history to determine what changes need to be deployed or reverted.

Devel::CheckLib can now check libraries' contents

nobody@example.com

How fast is Rakudo's "nom" branch?

nobody@example.com (Moritz Lenz)

Nearly one year ago, the Rakudo Perl 6 developers proudly released the first Rakudo Star, a distribution aimed at showing the world what Perl 6 can look like, and in turn get feedback from more early adaptors.

And feedback we got. While the overall response was very positive, people had one main concern: it was too slow. That didn't come as a surprise, considering that we had focused on features first. Now it was time to change that, and work on massive performance improvements.

That is easier said than done. One of the reasons is that Rakudo is tightly coupled to the parrot virtual machine, but there is a lot of mismatch between the two. For example parrot provides multi dispatch built-in, but not quite with the semantics that Perl 6 needs. Same for parameter binding, objects and a number of other areas.

In the following year, parrot got a new, faster garbage collector, and Jonathan Worthington came up with a cache for type checks at routine call time.

This sped up this simple mandelbrot fractal generator at size 201x201 from 18 minutes to 16 minutes 14 seconds. Actually the speedup was better than that, but we paid a performance penalty for new features, bug fixes and parrot performance regressions.

But it was clear that more substantial improvements where needed. One of the most promising candidates for speedups is a complete redesign of the object model, resulting in the "nom" (new object model) branch of Rakudo. Additionally to providing much more well suited OO primitives than parrot can offer right now, it also allows to share more information between compile time and run time, making a lot of optimizations possible.

Yesterday I sped up some operations on Complex numbers, and implemented a built-in that was missing to run the mandelbrot script. And today I timed it: 3 Minutes. From originally 18 Minutes.

Now that's a speedup by more than a factor of 5. I'm not sure if it will extend to other operators, but it sure is encouraging.

And that's without the optimizations that will now be possible, for example inlining operators. So after a literally slow start, Rakudo Perl 6 has a bright and fast future ahead. And it's already here, just not evenly distributed.

Second Grant Report: Structured Error Messages

nobody@example.com (Moritz Lenz)

Progress on my grant for error message is slower than expected, as expected :-). Life and work, though rewarding, take their toll. Also many of my original ideas turned out to be not very good, and got shot down by @Larry (with good reason).

However I did get around to lay out some specification for exceptions in S32::Exception (rendered) - some basic roles, and the relationships between exception, Failure (lazy/unthrown exceptions) and backtraces.

More importantly I think I'm at a point where I could start to actually implement some of that stuff, and use that experience to update the specification.

Palm Treo call db module

nobody@example.com

The Three-Fold Function of the Smart Match Operator

nobody@example.com (Moritz Lenz)

In Perl 5, if you want to match a regex against a particular string, you write $string =~ $regex.

In the design process of Perl 6, people have realized that you cannot only match against regexes, but lots of other things can act as patterns too: types (checking type conformance), numbers, strings, junctions (composites of values), subroutine signatures and so on. So smart matching was born, and it's now written as $topic ~~ $pattern. Being a general comparison mechanism is the first function of the smart match operator.

But behold, there were problems. One of them was the perceived need for special syntactic forms on the right hand side of the smart match operator to cover some cases. Those were limited and hard to implement. There was also the fact that now we had two different ways to invoke regexes: smart matching, and direct invocation as m/.../, which matches against the topic variable $_. That wasn't really a problem as such, but it was an indicator of design smell.

And that's where the second function of the smart match operator originated: topicalization. Previously, $a ~~ $b mostly turned into a method call, $b.ACCEPTS($a). The new idea was to set the topic variable to $a in a small scope, which allowed many special cases to go away. It also nicely unified with given $topic { when $matcher { ... } }, which was already specified as being a topicalizer.

In the new model, MATCH ~~ PAT becomes something like do { $_ = MATCH; PAT.ACCEPTS($_) } -- which means that if MATCH accesses $_, it automatically does what the user wants.

Awesomeness reigned, and it worked out great.

Until the compiler writers actually started to implement a few more cases of regex matching. The first thing we noticed was that if $str ~~ $regex { ... } behaved quite unexpectedly. What happend was that $_ got set to $str, the match was conducted and returned a Match object. And then called $match.ACCEPTS($str), which failed. A quick hack around that was to modify Match.ACCEPTS to always return the invocant (ie the Match on which it was called), but of course that was only a stop gap solution.

The reason it doesn't work for other, more involved cases of regex invocations is that they don't fit into the "does $a match $b?" schema. Two examples:

# :g for "global", all matches
my @matches = $str ~~ m:g/pattern/; 

if $str ~~ s/pattern/substitution/ { ... }

People expect those to work. But global matching of a regex isn't a simple conformance check, and that is reflected in the return value: a list. So should we special-cases smart-matching against a list, just because we can't get global matching to work in smart-matching otherwise? (People have also proposed to return a kind of aggregate Match object instead of a list; that comes with the problem that Match objects aren't lazy, but lists are. You could "solve" that with a LazyMatch type; watch the pattern of workarounds unfold...)

A substitution is also not a simple matching operation. In Perl 5, a s/// returns the number of successful substitutions. In Perl 6, that wouldn't work with the current setup of the smart match operator, where it would then smart-match the string against the returned number of matches.

So to summarize, the smart match operator has three functions: comparing values to patterns, topicalization, and conducting regex matches.

These three functions are distinct enough to start to interact in weird ways, which limits the flexibility in choice of return values from regex matches and substitutions.

I don't know what the best way forward is. Maybe it is to reintroduce a dedicated operator for regex matching, which seems to be the main feature with which topicalization interacts badly. Maybe there are other good ideas out there. If so, I'd love to hear about them.

An offer for software developers: free IRC logging

nobody@example.com (Moritz Lenz)

The Perl 6 developers communicate a lot through IRC. Some of the conversation is still valuable later on, so we have public IRC logs.

The software powering these logs was written especially for #perl6, but works fine for other channels too. Among the other users are TestML, CDK (Chemistry Development Kit), darcs, mojo, Padre, the Perl IDE, Parrot and Rosetta Code.

If you are also developing software, and would like public logs for your channel (either on freenode or irc.perl.org; other servers might be added on demand), feel free to contact me (moritz on freenode, or per email: moritz at faui2k3.org)

Features include: linking to individual lines, permanent URLs and volatile URLs for the current day, automatic linking of URLs and readable color schemes.

A current limitation is that you can't have two channels with the same name from different networks, in case of conflict "first come, first served" holds.

Graphing tool

nobody@example.com

XML::Tiny released

nobody@example.com

Perl 6 in 2011 - A Retrospection

nobody@example.com (Moritz Lenz)

The change of year is a good occasion to look back. Here I want to reflect on the development of Perl 6, its compilers and ecosystem.

At the start of the year, masak's Perl 6 Coding Contest continued from 2010, concluding in the announcement of the winner. I must admit that I still haven't read all the books I won :-)

Specification

2011 was a rather quiet year in terms of spec changes; they were a mixture of responses to compiler writer and user feedback, and some simplifications and cleanups.

Positional parameters used to be allowed to be called by name; this feature is now gone. That both makes the signature binder simpler, and removes accidental dependencies on names that weren't meant to be public. Read the full justification for more background.

A small change that illustrates the cleanup of old, p5-inherited features was the change that made &eval stop catching exceptions. There is really no good reason for it to catch them, except Perl 5 legacy.

say now uses a different stringification than print. The reasoning is that print is aimed at computer-readable output, whereas say is often used for debugging. As an example, undefined values stringify to the empty string (and produce a warning), whereas say calls the .gist method on the object to be said, which produces the type name on undefined values.

An area that has been greatly solidified due to implementation progress is Plain Old Documentation or Pod. Tadeusz Sośnierz' Google Summer of Code project ironed out many wrinkles and inconsistencies, and changed my perception of this part of the spec from "speculative" to "under development".

Rakudo

Rakudo underwent a huge refactoring this year; it is now bootstrapped by a new compiler called "nqp", and uses a new object model (nom).

It allows us to gain speed and memory advantages from gradual typing; for example the mandelbrot fractral generator used to take 18 minutes to run on a machine of mine, and now takes less than 40 seconds. Speedups in other areas are not as big, but there is still much room for improvement in the optimizer.

With the nom branch came support for different object representations. It makes it possible to store object attributes in simple C-like structs, which in turn makes it much easier and more convenient to interoperate with C libraries.

Tadeusz' work on Pod gave Rakudo support for converting Pod to plain text and HTML, and attach documentation objects to routines and other objects.

Rakudo now also has lazy lists, much better role handling, typed exceptions for a few errors, the -n and -p command line options, support for big integers, NFA-based support for proto regexes and improvements to many built-in functions, methods and operators.

Niecza

It is hard to accurately summarize the development of Niecza in a few sentences; instead of listing the many, many new features I should give an impression on how it feels and felt for the user.

At the start of 2011, programming in niecza was a real adventure. Running some random piece of Perl 6 code that worked with Rakudo rarely worked, most of the time it hit a missing built-in, feature or bug.

Now it often just works, and usually much faster than in Rakudo. There are still some missing features, but Stefan O'Rear and his fellow contributors work tirelessly on catching up to Rakudo, and it some areas Niecza is clearly ahead (for example Unicode support in regexes, and longest-token matching).

Since Niecza is implemented on top of the Common Language Runtime (CLR) (which means .NET or mono), it makes it easy to use existing CLR-based libraries. Examples include an interactive fractal generator and a small Tetris game in Perl 6.

Perlito

Perlito aims to be a minimal compiler with multiple backends, which can be used for embedding and experimenting with Perl 6. It had several releases in 2011, and has interesting features like a Javascript backend.

Ecosystem

The presence of two usable compilers (and in the case of Rakudo, two viable but very different branches) has led to many questions about the different compilers. The new Perl 6 Compiler Feature matrix tries to answer the questions about the state of the implemented features in the compilers.

With Panda we now have a module installer that actually works with Rakudo. It still has some lengths to go in terms of stability and feature completeness, but it is fun to work with.

The new Perl 6 Modules page gives an overview of existing Perl 6 modules; we hope to evolve it into a real CPAN equivalent.

Community

This year we had another Perl 6 Advent Calendar, with much positive feedback both from the Perl 6 community and the wider programming community.

We were also happy to welcome several new prolific contributors to the Perl 6 compilers and modules. The atmosphere in the community still feels relaxed, friendly and productive -- I quite enjoy it.

The year ends like it started: with a Perl 6 Coding Contest. This is a good opportunity to dive into Perl 6, provide feedback to compiler writers, and most of all have fun.

YAPC::Europe 2007 report: day 2

nobody@example.com

Perl 6 Compiler Feature Matrix

nobody@example.com (Moritz Lenz)

We now have a nice table that tells you which Perl 6 compiler implements what..

Such a thing was long overdue. When the topic came up in the past, people have suggested mostly automated solutions that compared test coverage of compiles to generate such a table. Nothing came out of it, it would have been a rather large endeavor. Now Eevee blogged about the lack of some easy overview that tells you what is implemented in Rakudo., and I thought it was time to tackle the problem.

Instead of some advanced automated system, we now have a simple text file, and a short perl script that converts it to a HTML page.

I'd like to thanks Will Coleda, Patrick Michaud and Stefan O'Rear for their contributions, and encourage everybody to keep the data up to date.

Does Your Bug Tracker Do This For You?

nobody@example.com (Moritz Lenz)

While working on a fast moving Open Source project, we sometimes get some platform specific bug reports. And even if nobody explicitly works on the problem, sometimes they go away due to general improvements of robustness.

So sometimes an intrepid volunteer goes through the list of old bugs, and asks the reporter if these platform specific bugs still persist. Quite often, we don't get a reply. Then the reports aren't very useful to us, and should be closed. Which requires a second pass through the bug queue. With more than 600 open bugs this quite a bit of work.

So I'd like to have an option to automatically close a bug within a given time span (say, four weeks or two months) if no feedback occurs. Our bug tracker (or at least our installation) doesn't have such a button.

Does your favorite bug tracker have it? Would you also sometimes like such a button?

YAPC::Europe 2007 travel plans

nobody@example.com

Wikipedia handheld proxy

nobody@example.com

Bryar security hole

nobody@example.com

Fourth Grant Report: Structured Error Messages

nobody@example.com (Moritz Lenz)

Progress on my grant for error message is slow but steady. Since my last report, I've done the following things:

Merged the nom-exceptions Rakudo branch, so now you can reliably throw Perl 6 objects as exceptions.
Implemented several error classes and roles in Rakudo
Started to throw typed errors both from runtime libraries and from inside the compiler
Hacked the default exception printer Rakudo to be much more flexible, for example you can now write exception classes that supress backtraces from the standard handler.
Wrote tests for typed run time and compile time errors, and at the same time developed a test function that makes it easy to write such tests.

It's time for a quick review of how far I am along the various deliverables in the original grant proposal.

D1: Specification. I think the hard work here is done already, what remains to do is finding good default and how to manipulate them (for example, how to generally switch on/off printing of backtraces?).
D2: Error catalogue, tests: I've not worked on this one too much. The error classes and roles so far mostly served to exercise the implementation; going through the existing errors from the various compilers and formalizing them will be quite a bit of work, but only moderately complicated.
D3: Implementation, documentation. Like D1, the hard part is mostly done. We can now throw errors from within the compiler actions and from the setting, next up will be the grammar. Then all places where errors are thrown need to be changed to use the new error classes. Again that'll be much work, but easy to do. Documentation is still missing.

All in all I feel I'm well on the way, and most complex decisions have been made.

For a more user oriented view of the new exception system I'd like to point you to my Perl 6 advent calendar post on exceptions.

Why Rakudo needs NQP

nobody@example.com (Moritz Lenz)

Rakudo, a popular Perl 6 compiler, is built on top of a smaller compiler called "NQP", short for Not Quite Perl.

Reading through a recent ramble by chromatic, I felt like he said "Rakudo needs NQP to be able to ditch Parrot, once NQP runs on a different platform" (NQP is the "another layer", which sits between Rakudo and Parrot, mentioned in the next-to-final paragraph).

I'm sure chromatic knows that VM independence is the least important reason for having NQP at all, but the casual reader might not, so let me explain the real importance of NQP for Rakudo here.

The short version is just a single word: bootstrapping.

The longer version is that large parts of Rakudo are written in Perl 6 itself (or a subset thereof), and something is needed to break the circularity.

In particular the base of the compiler is written in a subset of Perl 6, and NQP compiles those parts to bytecode, which can then compile the rest of the compiler.

This is not just because we have a fancy for Perl 6, and thus want to write as much of the code in Perl 6, but there are solid technical reasons for writing the compiler in Perl 6.

In Perl 6, the boundary between run time and compile time is blurred, as well as the boundary between the compiler, the run time library and user-space code. For example you alter the grammar with which your source code is parsed, by injecting your own grammar rules.

"Your own grammar rules" above refers to user-space code, while the grammar that is being altered is part of the compiler. If we had written the compiler in something else than Perl 6 (for example Java), it would be horribly difficult to inject user-space Perl 6 code into compiled code from a different language.

And the code not only needs to be injected, but the data passed back and forth between the compiler and the user space need to be Perl 6 objects, so all important data structures in the compiler need to be Perl 6 based anyway.

And it's not just for grammar modifications: At its heart, Perl 6 is an object oriented language. When the compiler sees a class definition, it translates them to a series of method calls on the metaobject, which again needs to be a Perl 6 object, otherwise it wouldn't be easily usable and extensible from the user space.

Now you might think that grammar modifications and changes to the Metaobject are pretty obscure features, and you could get along just fine with an incomplete Perl 6 compiler that neglected those two areas. But even then you'd have lots of interactions between run time and compile time. For example consider a numeric literal like 42. Obviously that needs to be constructed of type Int. What's less obvious is that it needs to be constructed to be of type Int at compile time already, because Perl 6 code can run interleaved with the compilation. So the compiler needs to be able to handle Perl 6 objects in all their generality, which is a huge pain if the compiler is not written in Perl 6.

Rakudo has cheated on that front in the past, and consequently has had lots of bugs and limitations due to non-Perl 6 objects leaking out at unexpected ends. If you ever got a "Null PMC Access" from Rakudo, you know what I mean.

The lesson we learned was that you need a Perl 6 compiler to implement a Perl 6 compiler, even if that first Perl 6 compiler can handle only a rather limited subset of Perl 6.

And there are also quite some benefits to this approach. For example NQP's new regex engine is implemented as a role in NQP. It is mixed into an NQP class which allows us to build Rakudo, but it is also mixed in a Perl 6 class, which allows the generation of Perl 6-level Match objects without any need to create NQP-level match objects first, and then wrap them in Perl 6 Match objects.

That's what NQP does for us. It allows us to actually write a Perl 6 compiler.

Thoughts on masak's Perl 6 Coding Contest

nobody@example.com (Moritz Lenz)

Masak's Perl 6 Coding Contest (short p6cc) is now in its final stage - the public commentary of solutions. So far masak has commented on p1, p2, and p3. I enjoyed the reviews and explanations so far, and look forward to more (except to the p4 review, because I botched the solution to this one).

The reviews made a good read, and here are only a few very minor points that I find worth mentioning. No criticism intended (neither to the author nor the reviewer).

Matthias' p1 solution uses side effects in subroutine named to-string. I would avoid that in "production" code (for whatever I might mean with that word...), since it's not what I would expect from the name. Instead of a counter, the array indexes could be used to identify which matrix to stringify.
Masak's review of fox' p1 solution contains the question I wonder why the @items array deserved a plural but the @matrix only a singular.... I'd say that's because @matrix holds one matrix, but @items holds many items. Speaking of which, I don't like the name @matrix - it describe a structure rather than the contents. The structure becomes pretty obvious through the access, but what is in the matrix?
After viewing my own p1 submission without syntax hilighting, I wonder why I chose so many double blank lines. My vim color scheme uses a dark blue for comments, which means that comment blocks need more visual distance from code, in my personal opinion. But when sharing code, I shouldn't make layout decisions based on non-shared syntax hilighting.
colomon's p2 submission uses complex numbers for coordinates. I considered that myself, and there's nothing wrong with it. Just strange that he then reinvented subtraction of two complex numbers in sub lines-intersect. Maybe he didn't want to come up with names for the intermediate results, $slope and $axes-intercept might have been viable ones.
Matthias could have simplified comb: /<&number>/ to just comb: &number -- a regex is just a callable, and when it's explicit in the current scope, you can just use a hard reference to it.
colomon's p3 solution could, as far as I can tell, replace all regexes with tokens - less backtracking, fewer surprises.
Reviewing my own p3 solution, masak asked I wonder what stopped moritz from, rather than doing $_ && .including on line 39, doing .?including instead. Either he considered that too cute, or he didn't consider it. I did consider it, but I decided against it, because the two things are subtly different. Mine only calls the method if the topic is true, whereas .?including always tries the call - even on a type object, where accessing an attribute leads to a fatal error. I think in my code that case doesn't show up, but it did appear during debugging in earlier versions.

Thankyou, Anonymous Benefactor!

nobody@example.com

Number::Phone release

nobody@example.com

Ill

nobody@example.com

CPANdeps upgrade

nobody@example.com

Third Grant Report: Structured Error Messages

nobody@example.com (Moritz Lenz)

Progress on my grant for error message is slower than expected, as expected :-). Yes, you've read that sentence before.

In the past months, general hacking on the nom branch of Rakudo was just too much fun -- and partially a prerequisite for the exceptions work.

I did manage to redo the backtraces that are generated from error messages.

Backtraces are now generated mostly in Perl 6 code, making them much more hackable. There's a Backtrace class, which is a list of Backtrace::Frame objects, each knowing the code object associated with it, as well as line number and file. (This is both specced and works in Rakudo)

Routines can have the is hidden_from_backtrace trait, which makes them not show up in the default backtrace stringification (one can still request a .full string representation). This is useful for routines which are internally used to generate exceptions, like die().

Rakudo also has a --ll-exceptions command line option which provides PIR-level backtraces, in the rare case the Perl 6 level backtraces hide too much information.

I've also started the nom-exceptions branch in Rakudo, which aims at lifting current limitations in Rakudo's exception handling. Currently die() and friends generate a parrot exception, and then there's a routine that fills the error variable $!. This routine generates a new Exception object, and sticks the parrot exception into it.

This practice means that if you create a subclass of Exception, instantiate it and throw it, you still only get an Exception in the error handler, not an object of the subclass. Since the actual exception type is very important for the ongoing work, that has to change. The branch mentioned earlier allows one to generate a Perl 6 exception, and pass that on as the payload of the parrot exception, which is then unwrapped when filling $!.

As a proof of concept this works, but it suffers from not being robust enough -- as it is, we could accidentally unwrap the payload of a CONTROL exception, placing meaningless junk into $!. So this needs a bit more work, which I plan to do this week (or next, if it proves to be more difficult than anticipated).

As always, your feedback is very welcome.

CPANdeps

nobody@example.com

Module pre-requisites analyser

nobody@example.com

Perl isn't dieing

nobody@example.com

YAPC::Europe 2007 report: day 3

nobody@example.com

Introducing my new project: Quelology organizes books

nobody@example.com (Moritz Lenz)

For about half a year I've been working on a website called quelology, which collects book series and translations.

It is intended to answer questions of the form: I've now read "Harry Potter and the Order of the Phoenix", which is the next book in that series? or What's the name of the French translation of that book?

The website and data mining behind it are written in Perl, and it is based on book meta data by isfdb, amazon and worldcat.

I'm working on importing data from more sources, next up will be the Swedish National Library.

After completing the data mining stage, I'll add an interfaces that allows the visitor to edit the book, series and translations data, so that users can extend the data body.

First Grant Report: Structured Error Messages

nobody@example.com (Moritz Lenz)

My Hague Grant proposal for designing, implementing and testing structured error messages for Perl 6 has been acceepted, and I've started my work on it in my copious free time.

Before the grant started I've unified the error messages of several compilers to use "Cannot" instead of a wild mixture of "Cannot", "Can not" and "Can't".

In the past week I created a repository for the initial work on the error message spec, and added a list of existing error messages across different compilers, and some notes regarding the upcoming spec.

So far I've outlined some thoughts about separation of concerns, classification of the error messages, testing error messages for certain properties, and calling syntax for die() and fail().

Any constructive feedback on it is very welcome.

Thanks go to Ian Hague and The Perl Foundation for supporting my work financially.

Perl 6 notes from February 2011

nobody@example.com (Moritz Lenz)

Lately real life has prevented me from blogging, so here are just a few random notes from the Perl 6 developers:

The Perl bug tracker now has tags testneeded and testcommitted, which can mark tests that need or have tests in the spectest suite. Since the URLs for querying these tags are unwieldy and non-obvious, I've created some aliases: http://rakudo.de/testneeded and http://rakudo.de/testcommitted.

Development of the new nqp and rakudo-on-the-new-object-model is progressing nicely. I had some fun porting some PIR code to NQP, and writing some new code. Most interesting to read is the source of the new meta model, much of which is written in a subset of Perl 6 (so quite readable, if you happen to know Perl 6. For example you can see how the method resolution order for multiple inheritance is calculated.

There is a parrot branch that adds a generation garbage collector to parrot. Its release is planned for shortly after the 3.1.0 release due tomorrow. Initial benchmarks show that Rakudo is between 25% and 30% faster on that parrot, as measured by a spectest run. I very much look forward to having that in the parrot main line.

Writing code for niecza is quite a nice experience. It still has a big startup cost, but then runs much faster than rakudo (at least it feels that way). There are still lots of features missing (for example non-integer number literals), but feature requests are usually implement quite quickly.

Travelling in time: the CP2000AN

nobody@example.com

YAPC::Europe 2007 report: day 1

nobody@example.com

Thanks, Yahoo!

nobody@example.com

POD includes

nobody@example.com

cgit syntax highlighting

nobody@example.com

CPAN Testers' CPAN author FAQ

nobody@example.com

Why is my /tmp/ directory suddenly only 1MB big?

nobody@example.com (Moritz Lenz)

Today I got a really weird error on my Debian "Squeeze" Linux box -- a processes tried to write a temp file, and it complained that there was No space left on device.

The weird thing is, just yesterday my root parition was full, and I had made about 7GB free space in it.

I checked, there was still plenty of room today. But behold:

$ df -h /tmp/
Filesystem            Size  Used Avail Use% Mounted on
overflow              1.0M  632K  392K  62% /tmp

So, suddenly my /tmp/ directory was a ram disc with just 1MB of space. And it didn't show up in /etc/fstab, so I had no idea what cause it.

After googling a bit around, I found the likely reason: as a protection against low disc space, some daemon automatically "shadows" the current /tmp/ dir with a ram disc if the the root partition runs out of disc space. Sadly there's no automatic reversion of that process once enough disc space is free again.

To remove the mount, you can say (as root)

umount -l /tmp/

And to permanently disable this feature, use

echo 'MINTMPKB=0' > /etc/default/mountoverflowtmp

YAPC::Europe 2006 report: day 3

nobody@example.com