The Copenhagen Perl 6 Hackathon and the Open Source Days conference is over.
I am left filled with magnificent impressions and at the same time suffering from a mild case of information overload.
Due to a collision with the Dutch Perl Workshop the Copenhagen Perl 6 Hackathon was scheduled to start Saturday and continue Sunday, where the Open Source Days conference started friday. In addition we had arranged for the invited Perl 6 people to have some time on their own to discuss and hack face to face, something that have shown beneficial previously for other dispersed project groups.
We started the Saturday with a line of Perl 6 presentations – there was a good turn up to the talks, also by people who I have not seen at Copenhagen Perl Mongers meetings.
I have received little, but very positive feedback on the event. One of the more interesting pieces of feedback was this tweet (http://twitter.com/Jippi/status/10071258756), which really pin-pointed that we are doing the right thing and having fun at the same time.
Perl 6 is different and I think that it is important to emphasize the difference, not just different from Perl 5, but different in many aspects and from other languages. I can only say that I think that Perl 6 will be a very modern language and as Perl 5 programmer I will need to rethink the ways I am writing Perl.
I overheard a discussion over some Perl 6 construct and one of the comments was that it was not a very Perl 6-ish way of doing things – Perl 5 is not Perl 6. Personally I am very keen on getting to use Perl 6, but I have slowly started to understand that it will be significant change. Perl 6 will address many short-comings in Perl 5 naturally, the primary language designer is the same person, but in general Perl 6 is a new language and it will address short-comings in many languages. Yes it will also have it’s own, but then we then we just address that in Perl 7 – implemented in Perl 6 magnificent isn’t it…
My experience with this kind of events is that it takes some time before the actual result show. I hope that we will be able to attract some new people to the Copenhagen Perl Mongers – and I hope that the Perl 6 community have benefitted from the opportunity to present Perl 6 at a large open source non-perl conference.
The aftermath of the hackathon is hopefully going to be more positive than the aftermath of the Open Source Days conference. One of the exhibitors made a small happening at the conference involving body painted girls. The IT business in Denmark has for a long time been working hard on attracting women to a business, that is always short on clever people of both genders. Based on the blog entries and comments I have seen the conclusion must be that the happening was a bad idea – a really bad idea and I can only concur.
Implementation wise I hope we have been able to contribute over the weekend and that we have earned the Copenhagen release – I am working on my first larger Perl 6 application and it is fun. I am constantly refactoring the code to be more Perl 6 and training my brain to start using Perl 6 patterns.
Thanks to all the invited Perl 6 people, attendees and the coord of Open Source Days for giving us the opportunity to show of Perl 6.
jonasbn, logicLAB
OK, you know how it works. Everybody who's really comfortable with testing often offers the following advice:
This advice is absolutely correct, but misleading. It leaves you with an obvious question: what coverage do I want? This is where the danger lies.
While hacking on my Veure project, I find myself often digging into code that I don't understand terribly well. This also means I don't understand my interface. As a result, I often implement something I think will work, hit my browser and view the results. Which are wrong. Again. So I tinker some more. Again.
Lather, rinse, repeat. The benefit of this strategy is that I tend to add new functionality much faster. The drawback is that once something is working, it's easy to be careless with testing. And here's what I've found:
If it's not covered, it's broken. Every. Single. Time.
I admit that anecdotes are not data and maybe I'm just a bad programmer, but my code is getting complex enough that there are parts of it which look perfectly fine, but are nonetheless wrong. I recently discovered a URL hack exploit because when I was trying to increase my code coverage, I focused on a few modules with coverage around 98% or so and tried to get them to 100%. Three modules in a row with just a couple of untested lines had bugs in those lines.
You don't want to test that things already tested work (don't test getters/setters in Moose unless you're hacking on Moose directly), but you had better make sure that your code is setting them appropriately.
So don't strive for 100%, but be very, very careful about the seductive thought of "I have 95% coverage". If your project is sufficiently complex, that last 5% will be a minefield.
Earlier this year Renée Bäcker from $foo magazine interviewed Richard Dice and me to find out our thoughts on TPF in 2009 and our hopes going forward. An English version [pdf] of the interview is available online.
I feel like I'm making enough progress, right now, to make a grant update every day. I don't know how long this will last, but I'm glad that it's going so well, so far.
I got a lot of preliminary improvements done to logging, mostly described in yeserday's post. They scrunched the default output way down to fit on the screen and be readable. Now that I had logging working, it seemed like time to bite the bullet and really get to work on testing.
I made some improvements to
App::Cmd::Tester, trying to
capture the exit code of applications that call exit. I'm not entirely happy
with how it works -- it interferes with Test::Builder sometimes, it seems.
Still, it's better than nothing. I made a few other changes, too, making the
App::Cmd::Builder easier to extend and providing more information back in the
results. For example, you can now say:
my $results = test_app(AppClass => \@my_argv);
my $app_object = $results->app;
This is really useful when the app object is a Dist::Zilla::App, because you can do things like:
my $app_object = $results->app;
my $zilla = $app_object->zilla;
my $build_dir = $zilla->built_in;
Obviously, this makes testing Dist::Zilla a lot easier.
I've also written a subclass of Dist::Zilla just for testing, which will copy the source material to a temporary directory and then build to another part of that temporary directory. Then the programmer can inspect the files written to disk, the state of the Dist::Zilla object, and more. I also hope to write a simple way to ask, "Did anything alter the source tree?" (That will often constitute a bug, but not always.)
Unfortunately, I've exposed one bug and one (recent) design flaw. The bug is
something I've basically known to be there all along, and hasn't mattered
at all for normal use. In short, you can't build a dist unless the dist root
(the place with the dist.ini file) is your current working directory. If you
try, things break, because too many things work with relative paths and never
properly make them absolute. This, I hope, will not be very hard to fix.
Fortunately, I can work around it with judicious use of
File::chdir for now.
The more annoying problem is that the logger design is a problem. I still like Log::Dispatchouli and plan to keep using it as the default logger, but the way the existing logging framework works is just too leaky. It acts like the logger can be replaced, but it clearly can't, and even doing things like logging everything to a test object isn't easy enough. I have a few possible solutions to this, ranging from the brutal-but-easy to the complicated-but-maybe-more-useful.
I think I'll start with the latter, assuming that I won't actually need the added complexity. At any rate, I think it's extremely likely that before the end of the week I will have a nice simple way to test Dist::Zilla's behavior in automated tests without copying and pasting a lot of ugly code. This is really exciting, because it means I can start writing tests for Dist::Zilla. It's also really scary, because I'm pretty sure that writing tests for Dist::Zilla is going to be a big drag. That's okay, though, because it's what I'm getting paid to do.
Finally, I thought I'd note one other really exciting thing. Dist::Zilla has a lot of dependencies, mostly because I've tried to factor out any code that can be useful elsewhere, but also because I've tried to use existing tools when they exist. Now that I'm trying to get these improvements made to Dist::Zilla, I'm finding that a lot of them need to go in external libraries. That means that this grant to improve Dist::Zilla is also going to improve App::Cmd, Log::Dispatchouli, Pod::Weaver, and probably a bunch of other libraries. That makes me pretty happy about the decisions I've made so far.
I am happy to announce that the Haifa Perl mongers are going to have a meeting on the 15th March in the offices of Qualcomm in Matam, Haifa organized by Shmuel Fomberg. On the agenda is Erez Schatz How to Talk to Newbies and Yaron Meiry (aka Sawyer) Moose - A postmodern metaclass-based object system for Perl 5 The meeting will start at 18:30.
For more details please see the announcement and/or contact Shmuel.
The regular Rehovot Perl Monger meeting is going to take place on 16th March in the Weizmann Institute. Yaron Meiry (aka Sawyer) is going to give a talk about Moose - A postmodern metaclass-based object system for Perl 5
For more details please see the web site of the Rehovot Perl Mongers.
Lately I've been learning ExtJS (which I might write about separately sometime) to try and create a UI for a small application I'm writing for a friend.
The application uses Dancer and KiokuDB. They're both easy to work with and I spent little time working on the backend. The problem lies in the frontend, where I need some UI. I've chosen ExtJS because it has built-in widgets.
I've played with tutorials, general documentation and (tried to) read the not-as-friendly-as-I-thought API. I was able to create a grid, a form and even a viewport. However, it kept feeling like it's stuck. Not stuck as in "this is going slow", but stuck as in "not moving at all".
I don't know how to measure the progress of UI development. Usually, with backend development (which is mostly what I do), I can see the project moving (even if slowly), but here I don't. It reminds me of the stage in your code when you're still guessing yourself on the way you planned out your program, on your algorithms, on your assumptions and supposed requirements. You keep going back to the drawing board, unsure of your project.
Last night I sat once again to try and sketch things out. I decided this time to sketch the usability of the UI instead of the appearance. The difference for me means I think of what would happen instead of how it would look like. I started sketching arrows and write small comments (double click, single click, tab). Things got better, I finally felt like I know my goals, at least the ones in the beginning.
I sat down, wrote a viewport, added a tab panel, added a grid on the first panel and was even able to open and move to a new tab on double click of a record. Pretty cool. At first I was excited about this, but then dejected. Was this progress? How would I know if I made progress just now, or got stuck on something irrelevant?
I assume working on the parts that I already know by heart how to do would be a waste, which is the backend. I should focus on the frontend, that is clear. But should I focus on the panels? On the tabs? On fetching the data? Should I be picking at the grid? Should I be picking at the functionality? What about the layout? Should I be learning how to do everything in ExtJS first? Maybe strengthen a bit on the Javascript side?
I'm feeling quite confused, something that although happens often in life, doesn't occur to me often on the technical side. There are so many things to pick from and I don't know which one is correct. I reckon that is why I keep feeling like I'm making no progress. Maybe I should sketch down all the possible components and then try to assemble a puzzle out of them and then write the functionality. I don't know.
Does anyone have any advice on this?
Major release of Dancer finally out. You can read about it on Alexis' blog.
Basically this version provides a lot of improvements to Dancer:
I want to thank Sukriah and David for all their work and being great guys to work with.
Bust a move!
It’s never great to post an “end-February” report a week into March, but that’s how things are going lately. I’ve been busy with family and work obligations that have meant less CT2.0 hacking. I’m sorry this is coming late, but I hope I will give anyone interested a sense of where things stand.
I should note that the original deadline for finishing CT2.0 was March 1 and clearly we’re not there yet. I’ve discussed the situation with Robert and Ask at the Perl NOC, and they’ve been willing to extend the deadline for cutting off CT1.0 for a while longer. Thank you, Robert and Ask, for your understanding!
Progress in the last couple weeks:
In the last several weeks, members the #catalyst and #moose IRC channels were very, very helpful and patiently answered my many stupid questions. Thank you in particular to confound, perigrin, rafl, rjbs, stevan and t0m.
Coming up:
It’s at the point now where I suspect the “hard thinking” part is pretty much done and it’s a lot of grotty but straightforward tasks to go. Hopefully, beta testing won’t reveal any major issues and the end of March update will be focused on planning on orderly transition from CT1.0 to CT2.0.
Latest Web::Simple master (git://git.shadowcat.co.uk/catagits/Web-Simple.git) is now using Plack, the perl web server directly for CGI and FastCGI and I hope do do another release shortly with notes on deploying as both on shared hosts (more specifically Dreamhost but with an invitation to bitch if they don’t work on other budget hosts).
Every time someone mentions a cool new web framework like Web::Simple, Dancer or Tatsumaki, my immediate reaction is to start thinking up cool little niche web apps I could build with it. You know, the kind you imagine yourself whipping up during your lunch break, that will probably never get more than 10 curious users but just might turn into the next big thing if only you got it up and running on a public server. And that’s when the second thought immediately arrives: where am I going to run this thing? Do I really think the idea has enough legs to justify paying for a virtual server plan? Can I be bothered going through the pain of figuring out how to deploy it on a shared hosting provider? And normally that’s the point where I sigh wistfully and go back to reading my RSS feeds.
The thing is, with the advent of things like local::lib it’s getting a lot easier to deploy Perl web apps on shared hosting. And with most web frameworks adopting Plack/PSGI, the work required to deploy Perl web apps on budget hosts is converging into a similar sequence of steps.
So this time during my lunch break I started envisaging a centralised, SEO-friendly information source (hello perlsharedhosting.com) that cobbles together all of the currently available information into an easy to digest form to make it ridiculously easy to choose a shared hosting provider, deploy your web app and troubleshoot common problems.
A site that works like this:
I haven’t decided yet what engine would fit best.. MojoMojo is a front-runner, and in fact the Catalyst Friendly Hosting page runs on MojoMojo does a large chunk of what I’ve described above.
The site would be very “cookbook” oriented, since we’re specifically targeting people who can’t be bothered learning the ins and outs of 10 different technologies (not to mention figuring out how to get them to play nicely together) for the sake of deploying toy web applications on cheap shared hosting. Just the essentials: this is what you need, this is how you achieve it. And if you want to learn more, go here.
With that in place, we’d end up with a single, visible, place to research and document the complications of running Perl web apps on shared hosting. And if the site became popular, hosting providers might even take notice and start offering more Perl-friendly shared environments. We could have live demo pages running on different hosts, possibly even donated by hosting providers if they see it as a way of showcasing their Perl-friendliness (mojomojo.dreamhost.perlsharedhosting.com, dancer.resellerzoom.perlsharedhosting.com, etc..).
I got as far as registering the domain name; I was planning on getting a simple MojoMojo prototype up and running, but I got side-tracked researching how to deploy it on my shared hosting account…
In Item 110, “Compile and install your own perls”, we showed you how to compile and install several versions of perl so that they don’t conflict with each other and you can use them simultaneously. Since they don’t install their programs, they are left in their $prefix/bin directories. With several perls, each of which has their own modules directories, using tools such as cpan and perldoc can get confusing. Which version of those tools are you using and which perl are they trying to use?
When you compile and install you perl, it adjusts the shebang line of the tools to point to the right perl. If your prefix is /usr/local/perls/perl-5.10.1, the you have a bunch of Perl tools in /usr/local/perl/perl-5.10.1/bin. They each have their right shebang:
% cd /usr/local/perls/perl-5.10.1/bin % head -1 * ==> c2ph <== #!/usr/local/perls/perl-5.10.1/bin/perl ==> config_data <== #!/usr/local/perls/perl-5.10.1/bin/perl ==> corelist <== #!/usr/local/perls/perl-5.10.1/bin/perl ==> cpan <== #!/usr/local/perls/perl-5.10.1/bin/perl ==> cpan2dist <== #!/usr/local/perls/perl-5.10.1/bin/perl ==> cpanp <== #!/usr/local/perls/perl-5.10.1/bin/perl ==> cpanp-run-perl <== #!/usr/local/perls/perl-5.10.1/bin/perl ...and so on and so on...
Anywhere you copy those programs, they'll find the right version of Perl because it's built into the source. Where to put them though? You don't want a zillion paths in your environment, and you don't want to type the long paths every time you want to use a tool.
Everyone is probably going to have their preferred version of Perl. This project wants to use Perl 5.10 but another wants to use Perl 5.8. Still another project works on Perl 5.8.8, but its coders wonder if they can upgrade to Perl 5.8.9 for some minor bug fixes.
The easiest thing to maintain might be to make symlinks in your personal bin/ directory:
% ln -s /usr/local/perls/perl-5.10.1/bin/perl5.10.1 ~/bin/perl5.10.1 % ln -s /usr/local/perls/perl-5.10.1/bin/cpan ~/bin/cpan5.10.1 % ln -s /usr/local/perls/perl-5.10.1/bin/perldoc ~/bin/perldoc5.10.1
Now, when you want to install a module for your Perl 5.10.1 installation, you use the right cpan:
% cpan5.10.1 LWP::Simple Net::Twitter
When you want to read the docs for Perl 5.10.1, you use the right perldoc:
% perldoc5.10.1 perlsyn
If you want to make one version your default perl, you can make the right links for that:
% ln -s /usr/local/perls/perl-5.8.9/bin/perl5.8.9 ~/bin/perl % ln -s /usr/local/perls/perl-5.8.9/bin/cpan ~/bin/cpan % ln -s /usr/local/perls/perl-5.8.9/bin/perldoc ~/bin/perldoc
You don't have to live with that though. Although I'll leave it as an exercise for the reader, you can create a script to change the default version by switching the links for ~/bin/perl and so on to the new default. You don't have to do that with just perl though. With Furlani modules you can do it with your entire environment (but that's another story).
Now, all of that linking is quite tedious if you need to do it for several perls, so you might want to adapt the script that I use. I install perls as /usr/local/perls/perl-5.minor.point and usually want to make the links in ~/bin.
#!perl
use 5.010;
use strict;
use warnings;
use File::Basename;
use File::Spec::Functions;
my $perls_directory = catfile(
$ARGV[0] // '/usr/local/perls',
'perl*'
);
die "$perls_directory does not exist!\n"
unless -d dirname $perls_directory;
my $links_directory = $ARGV[1] // catfile( $ENV{HOME}, 'bin' ); #/
die "$links_directory does not exist!\n" unless -d $links_directory;
foreach my $directory ( glob( $perls_directory ) )
{
say "Processing $directory...";
unless( -e catfile( $directory, 'bin' ) )
{
say "\tNo bin/ directory. Skipping!";
next;
}
my @perls = glob( catfile( $directory, qw( bin perl5* ) ) );
my( $perl_version ) = $perls[0] =~ m/(5\.\d+\.\d+)\z/;
say "\tperl version is $perl_version";
foreach my $bin ( glob( catfile( $directory, 'bin', '*' ) ) )
{
say "\tFound $bin";
my $basename = basename( $bin );
my $link_basename = do {
if( $basename =~ m/5\.\d+\.\d+\z/) { $basename }
else { "$basename$perl_version" }
};
my $link = catfile( $links_directory, $link_basename );
next if -e $link;
say "\t\tlinking $bin => $link";
symlink $bin => $link or
warn "\t\tCould not create symlink [$!]: $bin => $link!";
}
}
In case you were wondering what Damian been's doing lately (and don't you dare say "Damian Who?" -- unless your name is Damian), he's been writing a series of articles for IBM Developer Works about the Vim editor.
On an unrelated note, one of my brothers is a psychiatric nurse. He often has to deal with patients in the Conway Ward. Coincidence?
I've been continuing to work on adding improved logging capabilities to Dist::Zilla. They're not going to be terribly complex, but I'm trying to make sure that they're easy to use, and that they lend themselves to sane default output and useful debugging output. Yesterday, I realized that these lines were really annoying:
[DZ] Dist::Zilla::PluginBundle::RJBS/Dist::Zilla::PluginBundle::Git/Dist::Zilla::Plugin::Git::Push initialized
I went through and fixed all the bundles I could to produce simpler (but still useful) names and then used each plugin's logger for logging initialization. The result is now more like this:
[@RJBS/@Git/Push] online, Dist::Zilla::Plugin::Git::Push v1.234567
That's also been demoted to debug output, so there's much less noise by default. Releasing Dist::Zilla this morning looked like this:
~/code/cs/Dist-Zilla$ dzil release
reading configuration using Dist::Zilla::Config::Finder
[DZ] beginning to build Dist-Zilla
[DZ] guessing dist's main_module is lib/Dist/Zilla.pm
[DZ] extracting distribution abstract from lib/Dist/Zilla.pm
[@RJBS/Classic/ExtraTests] rewriting release test xt/release/pod-coverage.t
[@RJBS/Classic/ExtraTests] rewriting release test xt/release/pod-syntax.t
[DZ] writing Dist-Zilla in Dist-Zilla-1.100660
[DZ] writing archive to Dist-Zilla-1.100660.tar.gz
[@RJBS/@Git/Check] branch master is in a clean state
[@RJBS/Classic/UploadToCPAN] registering upload with PAUSE web server
[@RJBS/Classic/UploadToCPAN] POSTing upload for Dist-Zilla-1.100660.tar.gz
[@RJBS/Classic/UploadToCPAN] PAUSE add message sent ok [200]
Not bad!
I also made some improvements to Pod::Weaver and the PodWeaver plugin so that it can log its work as it goes. This will produce output like:
[@RJBS/PodWeaver] [Name] added name/abstract section to lib/Dist/Zilla.pm
I'm starting to wonder if Dist::Zilla isn't the project where I'll end up wanting more fine-grained control than "normal logging" or "debugging logging," but I think things will go as this issue always has: I'll put it off until everything else is done and then realize I don't care about it.
CeBIT was way more successful than I expected. Thanks to Renee Baecker, the main organizer and the other people who were at the stand during the week we made a lot of contacts with individual developers and companies using Perl. We also talked to quite a number of people who have heard about Perl but never tried it and some people who never heard about it.
It was both very hard to be there standing 8-9 hours and almost constantly talking to people every day but it was also a lot of fun.
Originally we were requested to always have two people at the stand but it turned out that there were times when even 7 of the Perl::Staff members could not handle the number of visitors at the same time. We were lucky as our stand was part of the Open Source Lounge of 15 projects slightly out of the main stream of people. On one hand that meant many people walking along the main alleys missed it but on the other hand those who came by had more space and time to talk to us.
It was very heart warming to see people coming to the booth with no or even negative view on Perl and leaving with a feeling that they saw some really cool things they want to try now.
To people with now real Perl knowledge or people who used Perl 10 years ago we showed some modern Perl code such as Moose, MooseX::Declare and some related stuff. We showed them Padre so they can see you can write large desktop applications in Perl. We were also showing Catalyst, especially to PHP programmers but also to others who were web developers.
On one of the days there were two Foswiki developers and they showed their wiki to the people interested in that. I think OTRS got a bit less attention on our stand but they had their own booth very they made a lot of business contacts.
The strategy was simple. Some of us were standing in the alley and as people walked by we gave them a flyer and then started to talk to them. Obviously there were many who were not interested, we just let them go. I was the only presenter who did not speak German my strategy was to first ask them if they speak English. With many people it was clear that they don't know English well enough for a conversation or just prefer German. That gave me a chance to direct them closer to the booth to talk to "one of our other representative, who speaks German". So in fact I was acting a lot as the "catch man". I think it worked out quite well. When they were OK speaking English or even preferred that language then I kept the conversation going on. I asked what are they using Perl for, or if they did not know or have not used Perl I asked what other languages are they using and what kind of things are they using. That usually gave me an opportunity to ask if they would like to see some modern technologies in Perl. Most of them were interested so we stepped to one of the computers we set up (we had 2 sometimes 3 computers on our booth, which was high enough so we did not have to bend over to view the screens or type). There I showed them Padre, some code in Padre. Sometimes I showed a few pages from the Moose documentation and some small examples in Catalyst.
There were several people who were interested in Perl 6. For them I opened the web-page of my Perl 6 training slides and showed a few pages just to get them impressed by some of the nice features of the language.
Others, as I saw started by asking if the visitor "Knew Perl?" and from that point some kind of a conversation evolved in German that I could not follow anyway but that will be described by the other presenters. (Renee and Sewi have already written. All the reflections will be linked from the TPF wiki page related to Perl on CeBIT)
There were a lot of things we could improve in our presentation. First of all none of us knew all the things we wanted to show and we did not have ready made pages to show them. So I'd like to have several slide-shows that we can load on all the computers we are using for presentation and that each one of us can use to present various technologies. After all I don't need to be an expert in Catalyst in order to give a short introduction and show a few examples, assuming they were already prepared and I went over the slides with someone who is an expert.
Same with the Perl 6 examples. Others kept referring people to me to show Perl 6 examples but we could have put together a few slides with interesting examples that we can flip though pointing out the nice features. Within the slides we could even have instructions - show the Rakudo web site now, or in other cases to show the web page of the CPAN Testers.
This of course needs more preparation and even some time before the event starts to show the slides to each other but it pays off as it will make the presentation smoother.
Even that does not mean there won't be questions that need some more thinking on how to answer. There can also be cases when the knowledge of the presenter ends and there are still questions but then we can find the other presenter who has deeper knowledge in that subject or refer the person to the community channels where can get more information. After all, a large part of what we would like to achieve is that our visitors will start learning more about these subjects and start to participate in our communities.
I'm in the metro of Paris, testing Typepad :
Typepad is Based on Movable Type, the best blog system ouuta there. It's Perl based. It rocks.
In Pre-commit hooks and breaking the build, I wrote a little wrapper around 'svn' to ensure that I could locally alter how it functions. However, I hard-coded the name to the actual executable. I should actually be doing the equivalent of $self->next::method to find the next executable in my path. So I wrote the following bash function to give this a try.
function next() {
curr_exec=$1
shift
if [ -z $curr_exec ]; then
echo "next_exec()" requires an argument
exit 1
fi
if ! type $curr_exec >/dev/null 2>&1; then
echo "next_exec()" argument must be an executable
exit 2
fi
curr_path=$(echo $curr_exec | sed 's|/\w*$||')
executable=$(basename $curr_exec)
found=0
for path in `echo $PATH | sed 's/:/\n/g'`; do
if [ "$path" = "$curr_path" ]; then
found=1
elif [ $found -eq 1 ]; then
if [ -x "${path}/$executable" ]; then
next_exec="${path}/$executable"
break
fi
fi
done
if [ -z $next_exec ]; then
echo Could not find a next_exec for $curr_exec
exit 3
else
$next_exec "$@"
fi
}
This needs some work, but basically, it figures out your path and your executable name. Then it searches through the path to find the next executable with that name. This, I think, makes it safer to write override scripts for various executables and redispatch to them.
So with my svn wrapper, I can call my wrapper or call the thing it wraps.
svn diff
next svn diff
next seems like an awfully common word, though. Will this break anything?
Based on an explanation from Aristotle which has worked for me, here's one way you can make blogs.perl.org work a bit better for you. I've approved these and they seem to be OK.
As you may know, anonymous commenters are getting "Text was entered wrong" errors. This is because of a currently undiagnosed error with ReCaptcha. To work around this:
Whenever someone posts to your blog, you should be emailed notification. If they're "not trusted", you'll have to approve their posts. When you do, you'll have the option to make them a trusted commenter and not have to approve their posts again.
Yeah, it's a little more work to do, but setting it up is a one-shot deal. As you mark more people as trusted, you'll have to approve fewer posts.
As an added bonus, under "Preferences -> Entry", you can select your default format (text formatting). I've just switched mine to Markdown. You can go to "Preferences -> Comment" to allow the same for comments.
With Miyagawa's cpanminus (or as I like to think of it, cpantiny, but hey it's his module) now officially the New Shiny of the moment, the same old arguments are resurfacing about less is more, and worse is better.
And the great wheel of opinion circles again.
Rather than bother commenting on cpanminus itself (I suspect many could guess my opinions) I thought I would add some caveats to the praise, that hopefully can help you identify and engineer your own equivalent successes.
1. These "subset" modules are truly successful only when you make the accuracy trade offs worth it.
Thus, the installation must be effortless, the user interface must be super-simple, zero-conf is an absolute must, and the module must succeed in all those niches where the "real" application is too hard, too big, or too clumsy.
If your accuracy is going to suck, NOTHING else is allowed to suck.
2. As I hint about with "real" these subset modules work best when they are an alternative solution, not the only solution.
If you are inaccurate and the only solution, you are an annoying limitation.
If you are inaccurate and an alternate solution, you are handy because you create a kind of user-pays situation. Simple people with simple use cases get a simple solution. Hardcore people with hardcore needs get a hardcore solution.
So if you want to make a small solution, it has to be a subset of something larger that works better. It can't be the only solution.
3. 99% is a just a marketing number, but the real number does matter.
Why? Take a look at the Heavy 100 index (http://ali.as/top100/") and you'll notice that many major and popular CPAN modules (like, say, Catalyst) have more than 100 dependencies.
With a 99% success rate (using stupidly naive "statistics") every single module on that Top 100 list will fail to install. In a large system with lots of recursive dependencies, it doesn't take much for your install count to grow to 20 or 50 or 100 dependencies.
So you need to be sure about WHICH subset you want to support, so you can be sure what percentage you REALLY need.
For example, despite all the work to support it, there is only one single module currently using Bzip2. Would it be worth it to remove bzip2 support entirely from the CPAN and help that author to convert? Probably.
The biggest benefits of these alternate solutions are often not only to do what they do. It's to demonstrate what is possible, pushing the competitors to do it as well.
I am pleased to announce that David Mitchell's grant application has been successful. I would like to thank all the community members who took time to provide feed-back on this proposal and Booking.com whose generous contribution to TPF has allowed us to fund this grant.
I proposed a grant for Dist::Zilla improvements to The Perl Foundation, and a notice of its approval was posted yesterday. I've gotten to work.
A number of the things I proposed to do have been on my mind for quite a while, but I haven't been able to commit to working on them. I had some of them already sketched out in branches that were unready to merge. Last night, I rebased and merged them, resolved a few conflicts, and tried to cut a development release. Unfortunately, I foolishly cut a production release.
Fortunately, nothing seems to have been horribly broken. A few missing prerequisites were exposed here and there, but those have been fixed. Some other, more serious problem were fixed tonight, and I cut an actual dev release. You can see most of my "to do" list in the git repository, if you're not content with the summary in the grant proposal.
So far, I've been checking out upstream test coverage, adding a proper logging facility, and changing the way prerequisites are registered. The logging changes are mostly to facilitate better testing so I can start writing tests for the entirety of Dist::Zilla as it stands now, before I get to any more refactoring or extension. The upstream coverage is mostly an issue because if we test lots of Dist::Zilla, but not the code written to make it go, that's a big coverage gap. I'd like to make sure that all the code (that I wrote) to make Dist::Zilla work is adequately tested. I want to be able to rely on this code, after all!
The prereq stuff has just been on my mind and in the margin notes of my recent
CPAN work, especially as it related to fixing some bugs in the excellent
AutoPrereq module. It may introduce a few bugs in the short term, but those
should clear up. It will also make it much easier to report prerequisites,
including build-time or configure-time prereqs. In another release or two, the
AutoPrereq plugin should be able to report libraries found in ./t as being
test requirements, separate from runtime requirements, for example.
When these tasks are complete, I'll be moving on to testing Dist::Zilla itself. That means writing some tools to make that easier, and probably a few changes to how Dist::Zilla works here and there, to improve diagnostics and configuration.
For now, I'm happy that I'm going to be able to spend some time on this. It's sorely needed improvement.
Alan Haggai's work on the Archive::Zip library is complete. Numerous onerous bugs have been corrected, and the library can now handle far more kinds of data. A developer release should reach the CPAN soon, hopefully to be followed in short order by a new complete release of the library.
I'm excited about this for a few reasons:
A major thanks goes to Stevan Little which helped me understand KiokuDB and Search::GIN and Yuval Kogman for writing these great tools. :)
One thing I love about git it how you have control over your local environment. Need a pre-commit hook? Just add the damned thing. Need it for Subversion and you're not the admin of the subversion server? Sucks to be you. Fortunately, having a reasonable command line lets you work around this. And I needed to because I broke the build. Twice. In a week. Buying the donuts for this would be slightly less annoying if it weren't for the fact that I get to "enjoy" low-calorie yoghurt bars instead. (Though I'm now below 82 kilos for the first time in a couple of decades).
So no more breaking the build. Right. On my previous team, if I needed a module, I could usually add it. On this team, it's a bit painful, so I just use what I need locally and then break the build when I forget and commit use Test::Most 'die';. So I wrote a little bash scrip to make svn less brain-dead.
#!/bin/bash
if [ "$1" = commit ]; then
command="ack 'Test::Most|Data::Dumper::Simple' t/ lib/"
lines=`$command | wc -l`
if [ $lines > 0 ]; then
echo $command
echo `$command`
echo You suck!
exit 1
fi
fi
if [ "$1" = log -o "$1" = annotate ]; then
/usr/bin/svn "$@" | less
elif [ "$1" = diff ]; then
/usr/bin/svn --diff-cmd colordiff -x "-u" "$@" | less -R
else
/usr/bin/svn "$@"
fi
Basically, if I try to commit and I've left offending modules in the code, it tells me I suck and fails. As an added bonus, I realized I no longer need to pipe common commands to less and I get nicely coloured diff output.
This is dangerous, though. I could easily add "run perltidy on my code prior to commit". This sounds great until you realize you've just tidied a Makefile. Oops/ Still, my life is a touch easier now. And I hope that my future "breaking the build" will be in new and creative ways.
My bash isn't very good, so critiques welcome. And don't forget to write tools like this for yourself. We're programmers. Little itches become big problems when we don't scratch them. Make them go away.
And if you're wondering why I'm still using svn, it's because git-svn is a dream when working with our branches, but several git gurus have given up trying to find out why merging back into trunk on our project fails so badly. So I don't do that and still rely on svn for that tiny task.
I'm just aghast, but it's true. On April 11, 2006, the US Patent Office granted a patent for linked lists. It's mind-boggling.
The only thing conceivably unique? It's triply-linked, thus allowing multiple traversal orders. I'm confused. How is this different from a directed graph?
From the YAPC::NA 2010 website: "YAPC::NA 2010 is actively searching for sponsorship, if you or your company would interested in being a sponsor please take a look at our Executive Summary or Sponsorship Prospectus."
This was the oldest grant running. Ingy dot net was working intermittently on it presenting some of the results that were promised in the proposal but not them all. TPF and Ingy agreed to close the grant with a half payment for the (half) work done.
Ingy wrote a report on the work done. It can be read at https://www.socialtext.net/yaml/index.cgi?requiem_for_a_grant
First, here's the tl;dr summary ... Benchmarking is for losers, Profiling rulez!
I've noticed a couple blog entries in the Planet Perl Iron Man feed discussing which way of stripping whitespace from both ends of a string is fastest.
Both of these entries discuss examples of benchmarking. Programmers love benchmarks. After all, it's a great chance to whip out one's performance-penis and compare sizes, trying to come up with the fastest algorithm.
Unfortunately, this is pointless posturing. Who cares that one version of a strip-whitespace operation is three times faster than another? The important question is whether the speed matters.
Until you answer that question, all the benchmarking in the world won't help you, and that brings us to profiling.
Profiling is a lot harder than benchmarking, which may be why people talk about it less often. Profiling doesn't compare multiple versions of the same operation, it tells us where the slowest parts of our code base are.
In order to make profiling useful, we need to write code that simulates typical end user use of the code we're profiling. Then we run that code under a profiler, and we know what's worth optimizing.
Once we know that, then we can start speeding up our code. At this point, benchmarking might be handy. If, for example, on some crazy bizarro world, our program spent a lot of its runtime trimming whitespace from strings, we could benchmark different approaches, and use the fastest.
Of course, in the real world, this will never be the slowest thing your program is doing. In most cases, the slowest parts of the program are usually the parts with IO, such as reading files, talking to a DBMS, or making network calls. If this isn't the case, we may be operating on a lot of data in memory with some sort of non-trivial algorithm, and that's the slowest part.
Either way, without profiling, benchmarking is just a pointless distraction.
Of course, I'd be remiss if I didn't point out that Perl has an absolutely fantastic profiler available these days, Devel::NYTProf. It actually works (no segfaults!), and produces fantastically useful reports, so use it.
For this quarter TPF had 7 grant proposals, from which 5 were voted for rejection, one for acceptance, and the final one will be kept for new evaluation in the next round.
The five rejected grants are:
* Creating a ctypes-Like Interface for Perl to External Subroutines by Shlomi Fish
* Enhancing CPANHQ by Shlomi Fish
* perl core memory improvements by Jim Cromie
* Enhancing Perl 6 Pattern Matching by Morris M. Siegel
* CPAN Reviews by Alexandr Ciornii
The grant kept for new evaluation in the next round:
* Perl Compiler by Reini urban
And finally, congratulations to Ricardo Signes for his accepted grant:
* Improve Dist::Zilla by Ricardo Signes
These results took in consideration the proposal texts and the work relevance for the Perl community, the proposal authors curriculum and interaction with the Perl community, and the comments and reactions from the Perl community on this blog.
I really have no energy to write much. The first day of CeBIT was a bit low on visitors but even that went quite well. I met with several people I wanted to talk. Especially interesting were the chats with the people from the Eclipse Foundation.
Today, on the second day, was a lot busier. Many people were walking around and many people came to the Perl stand as well. There are always 3-4 Perl::Staff people at the stand and usually they are busy talking to visitors.
Some PHP and Ruby developers came and we could show them interesting things that can be done with Perl, Moose, Catalyst and DBIx::Class. Many people also liked Padre, you know the Perl IDE
There were many people who said that the main problem they are facing is the lack of Perl developers and some of them told us that they are actually considering switching language because of this. It seems to them that it is easy to find programmers to many other languages, something that was the case with Perl as well a few year ago but now there don't seem to be any Perl programmers. I wonder what could be the reason of that? Where are does developers gone? Have they stopped using Perl and have they remove Perl from their CV? Are there many more jobs now filled by those developers without new developers learning Perl to fill the newly created jobs? Maybe the expectations from a Perl developer are now higher than 5-10 years ago so the same people who could counted as Perl developers 5 years ago aren't considered as such any more?
Anyway we have a lot more work to do in the coming days. I'll try to add some pictures to the next report.
Renee made some nice pictures of Perl on CeBIT
Jonathan Worthington has successfully completed his Hague Grant for Rakudo Signature Improvements. This is his final report:
Introduction
A while back, I filed an application for a grant to work on a wide range of improvements to Rakudo's signature, capture and binding implementation, in order to improve performance, bring us in line with the specification and implement a range of features that we did not have support for.
I have now completed the grant tasks, and am filing this report to discuss the work that was done under it. I have missed the originally planned schedule for the grant to be completed. This was in a large part due to a lot of Rakudo development since October taking place in a branch, ng, which set out to refactor and improve many aspects of Rakudo and get us in good shape for the Rakudo * release in a couple of months time. It made far more sense to implement some aspects of this grant in the branch rather than in master, and the branch also needed my attention in other key areas. The branch has recently become master, and since then I've been able to complete the final pieces of the grant and bring it to completion.
Deliverables Status
I proposed six deliverables, and have completed all of them.
D1 called for an extensive refactoring of signature internals and parameter binding. Rakudo now has a single custom signature binder, designed to handle its binding needs in accordance with the Perl 6 specification. This is a far better situation than before, where we used the Parrot binder and then layered extra binding semantics atop of it, meaning we had to make two passes through the parameters.
Having our own custom binder that takes into account the needs of Perl 6 has made it possible to do all of the other items in this grant in a clean and performant way. It has also enabled other bits of progress outside the scope of the grant; for example, I was able to get a first cut of coercion (the "as" trait) of parameters in signatures implemented in little over half an hour a few days ago.
We also have compile-time objects to represent signatures and parameters in the compiler. This has allowed much cleaner code in the actions that build the AST. In the future, this approach will also be highly useful for building extra static checking and an optimizer.
D2 required implementing support for binding named arguments to positional parameters. I considered this in the initial re-design of the binder, so it just worked "out of the box" once I switched Rakudo over to using it.
The target of D3 - unpacking of arguments - was made much easier by having a custom binder. The binder was designed to be re-entrant, and therefore unpacking arguments was implemented by specifying how various data structures could coerce themselves into a capture, and then making a recursive call with the capture that was produced.
For D4, I spent some time re-working capture support overall, and as well as being able to write a capture into a signature - and to unpack it with a sub-signature - we can also interpolate a capture into the arguments for a call. Before this didn't really work - it would lose the named parameters, for example.
Implementing D5 - support for multiple return values - first meant working with Larry to figure out the correct semantics. It turned out they were easier to get in place in the refactored "ng" branch - which has now become the master branch. I also implemented the support for binding the return values against a signature; this means all the power of unpacking arguments is available for returns too. It's all routed through the same underlying binder implementation.
Finally, for D6 I first put a proposal into the specification for how signature introspection should look, then wrote an implementation and reviewed the test coverage.
Dissemination
I have written some blog posts (1, 2, 3) about the work that I have done under this grant. I've also been active on the #perl6 IRC channel and, thanks to the evalbot, been able to give some demonstrations of the improvements there too. I also continue to attend Perl conferences and give talks about Perl 6.
Conclusions
This grant has, like my previous two, brought another major piece of Rakudo essentially "up to spec" and provided a good, stable base for future development. Our coverage of S06 is vastly improved, along with performance. Overall this grant has helped us meet critical targets on the path to the Rakudo * release and, beyond that, towards Rakudo becoming a complete Perl 6 implementation.
Not thinking that any Perl books would be available for Kindle, I searched anyway. I can get at least Learning Perl and Mastering Perl from the Kindle store. I'm thinking about this because I'd like to see what I can do with an iPad Kindle app.
I'm on a trip with severe weight requirements for luggage, so I've been trying the Kindle app for the iPhone. It seemed weird at first, but after a couple chapters of the first book, I don't mind it anymore. The tech books are a bit different. If you're used to the beautiful and understated typography that is one of O'Reilly's hallmarks, you're going to think the Kindle is a bit odd. I think I have it slightly nicer on the iPhone, but it's still different.
The version I have uses multiple font faces, so you can get the monospaced font for inline code and code sections. The iPhone code wrapping is odd, and the paragraph spacing is set up for novels: that is, there's no extra space between the start of a code section and a body section. I don't think I'd want to read a tech book for the first time this way, but it's a nice reference.
Now I just need to figure out how to map something I see in the Kindle app to the physical page number so I can respond to errata (which are reported by page numbers instead of chapter and section). I guess I could develop that index myself, and I've often thought of doing that, but I'm hoping it comes as a feature to Kindle.
I should note that the books aren't any cheaper from the Kindle store. The Kindle is not about saving money: it's about convenience. If you want to save money, you can get the very cheap PDF versions of books from O'Reilly. Those should look just like the physical book, although they are probably harder to read on the iPhone.
David Golden praised Tatsuhiko Miyagawa's excellent new cpanminus CPAN client in The power of not being all things to all people. You should consider using cpanminus.
Don't overlook something else insightful that David wrote:
It's a lot of work to be all things to all people and I keep wondering whether making things simpler and better for 99% of people would be a better choice.
The only reliable way I've ever seen to "make the easy things easy and the hard things possible" is to make the easy things the default without preventing customization of the hard things. That's a design principle for languages, APIs, and tools.
Figure out what's most common (though not necessarily what people think they want, but what they need). Optimize for that. Consider what they might need and don't prevent it.
That's not easy, and what people need will change over time, but if you want to solve problems well, you have to solve the right problems.
The Perl community seems abuzz about the cool new cpanminus client by Tatsuhiko Miyagawa. After about two weeks of development, this is a reasonably functional CPAN client with just a fraction of the overhead, complexity and verbosity of the CPAN and CPANPLUS clients that come with the Perl core.
It’s a remarkable achievement, not only technically, but in the reaction it has sparked. As one of the (sometimes reluctant) maintainers of CPAN and CPANPLUS, I’ve realized that I both love and hate cpanminus.
Miyagawa graciously acknowledges standing on the shoulders of giants. Still, I can’t shake the nagging thought that cpanminus should never have been necessary in the first place.
What I’ve come to realize is that cpanminus is another example of the power of not being all things to all people. Miyagawa doesn’t promise that it works for all of CPAN or that it works everywhere that Perl does. He doesn’t have to. Making it work for 99% of CPAN for 99% of people is more than good enough.
I’ve been co-maintaining various parts of the Perl toolchain for a while now. It’s a frustrating challenge needing to make thing work everywhere, for everything, and trying as hard as possible not to break backwards compatibility. Plus, I don’t even get to use CPAN to make life easier. I don’t get to use handy tools like Moose or DateTime or Regexp::Common or SQLite or anything in the Config::* namespace or even basic tools like Archive::Zip. Nearly everything is done by hand.
Things have to work with just core Perl on a diverse set of platforms and with an incredibly limited set of assumptions. For example, the Perl core still doesn’t come with an HTTP client, so CPAN has to rely on FTP or command line programs to bootstrap LWP. (This is something I personally plan to tackle during the Perl 5.13 development series later this year.)
I think this is an ongoing challenge for core Perl development in general. It’s a lot of work to be all things to all people and I keep wondering whether making things simpler and better for 99% of people would be a better choice. (Anyone else for use strict by default? I hope that finally comes to pass in Perl 5.14.) chromatic writes about this topic often in his Modern Perl blog and I usually tend to agree with the points he makes. (October February 2009 had a particularly good series of posts.)
In the meantime, I look at cpanminus with greed and envy. Miyagawa++
I was just in a room with three competent, professional Perl developers, all of whom agreed that you can’t strip whitespace from both ends of a string in a single regexp.
The common knowledge is:
$str =~ s/^\s+//;
$str =~ s/\s+$//;
I fiddled around and found this:
$str =~ s/^\s*(.*?)\s*$/$1/;
I think it’s right. Is it better? Not sure. Is it faster? Slower? Clearer?
Speed I can test with benchmark… and using two is faster.
% perl try.pl
Rate single multi
single 444444/s — -74%
multi 1724138/s 288% –
So, nevermind. Use two regexps.
Is one clearer? Not sure. But it’s slower!
Lately, I've been writing some PDL (Perl Data Language) code and noticed that I've been doing the following:
my $temp_pdl = $pdl->…->final_expr(); my $result_pdl = $temp_pdl->where($temp_pdl < 0);
I've been thinking if I could somehow write it like that:
my $result_pdl = $pdl->…->final_expr()->with(sub { $_->where($_ > 0)});
Which seems cleaner and more elegant. I went on IRC asking about it and someone suggested rindolf: (sub { local $_ = shift; $_ * $_ * $_})-gt;( $obj->... ); (up to an episolon, but naturally this reverses the order and messes with the flow of the code). Eventually Matt S. Trout (mst) proposed the following solutions:
sub UNIVERSAL::with
{
local $_ = shift;
my $sub = shift;
return $sub->($_,@_);
}
my $_with = sub {
local $_ = shift;
my $sub = shift;
return $sub->($_,@_);
};
(I had figured something like that was possible previously, but I was beating around the bush.)
The $_with approach is covered in one of Matt's blog posts titled "Madness with Methods".
I wondered if it will work with
autobox and mst said that
I don't know and I hope not since the UNIVERSAL approach is evil.
. But
it does, out-of-the-box:
#!/usr/bin/perl
use strict;
use warnings;
use autobox;
sub UNIVERSAL::with
{
local $_ = shift;
my $sub = shift;
return $sub->($_,@_);
}
my $_with = sub {
local $_ = shift;
my $sub = shift;
return $sub->($_,@_);
};
print +(10+1)->with(sub { $_ * $_; }), "\n";
print +(10+1)->$_with(sub { $_ * $_; }), "\n";
This prints 121 twice.
I now wonder what to do with this knowledge. I have some aspirations of releasing it to the cpan as a mini-module. Still, it's really cool.
Finally, I should note that it seems like I'm going to lose my current Planet Perl Iron Man karma due to inadequate blogging. Oh well.
Many years ago, when I was... roughly 15 (I think), I met Theodor Ts'o (one of the first hardcore Linux Kernel hackers, since version 0.90, I believe) at a Linux event IBM organized in Israel. I should note that he is a very nice person.
After the event, we got to talk a bit. We talked about our favorite games. Mine was "avoiding segfaults". This was back when I was programming in C.
Today I wrote the following regex:
qr/^([\w|\.]+)\s+(?(\d+)\s+)?(\w+)\s+(?:(\d+)\s+)?([\w|\d|\.]+)$/;
Then got a Segmentation fault
Can you spot the error?
Here's a hint: it is missing a colon (:).
This is perl, v5.10.0 built for i486-linux-gnu-thread-multi
This is just a short promotional post.
If you are interested in the Perl/PostGIS http://geo2gov.com.au/ application we took second prize with during the Mashup Australia competition (or the subject of open government data in general) my "Nerds for Democracy" partner in crime Jeffery Candiloro will be giving a talk as part the Ignite Sydney speaking event on Tuesday night Sydney time.
You should be able to watch a live stream of the talk at the Ignite video website here.
In other Nerds for Democracy news, work is now underway on our entry for the NSW apps4nsw competition. It's looking pretty awesome, in particular because it will feature a set of government data we uncovered by accident, and which we think the public has pretty much never seen before.
Did you ever wonder how BEGIN, CHECK,
END and so on are called in Perl? Well, they didn't have a good
name, until recently.
The Perl 6 spec listed them under closure traits, which is unwieldy, and not really exact either. Now they are called phasers, because they tell you which execution phase the block or statement runs in.
There are so many possible puns that I'll refrain from writing any.
How hard is it to build a good search engine? Very hard. So far I thought that only one company has managed to build a search engine that's not only decent, but good.
Sadly, they seem to have overdone it. Today I searched for tagged dfa. I was looking for a technique used in regex engines. On the front page three out of ten results actually dealt with the subjects, the other uses of dfa meant dog friendly area, department of foreign affairs or other unrelated things.
That's neither bad nor unexpected. But I wanted more specific results, so I decided against using the abbreviation, and searched for the full form: tagged deterministic finite automaton. You'd think that would give better results, no?
No. It gave worse. On the first result page only one of the hits actually dealt with the DFAs I was looking for. Actually the first hit contained none of my search terms. None. It just contained a phrase, which is also sometimes abbreviated dfa.
WTF? Google seemed to have internally converted my query into an ambiguous, abbreviated form, and then used that to find matches, without filtering. So it attempted to be very smart, and came out very stupid.
I doubt that any Google engineer is ever going to read this rant. But if one is: Please, Google, keep it stupid, stupid.
I'm fine with getting automatic suggestions on how to improve my search query; but please don't automatically "improve" it for me. I want to find what I search for. I'm not interested in dog friendly areas.
(This blog post will probably only interest Perl 6 hackers, since it talks only about infrastructure.)
One of the central pieces of Perl 6 infrastructure is the Pugs svn repository. It holds not only the Pugs source code, but also lots of other Perl 6 projects:
It's huge, but at the same time it's very practical: anybody who is interested can get write access very easily, create a new subfolder for a new project, or can fix a typo in someone else's README file without asking for commit access first.
The pugs repo is also viral: Anybody with commit access and invite new committers. Despite what you might think of it: it actually works in practice, so far I haven't seen a single case of abuse.
The pugs repository is hosted on feather, a server kindly provided by Juerd and his company. It contains three virtualized servers, feather{1,2,3}. feather2 is used for "sensitive" data (for example an IRC bot that has API keys for various github accounts, and the perl6.org website). Only a handful of "trusted" users have an account there. feather3 is used for low security stuff like evalbots which might go astray.
feather1 holds all the rest. That means the pugs repository, commitbit (the software we use for handing out commit bits and resetting svn passwords), various websites, and a whole bunch of Perl 6 developers and users have a shell account there, for trying out Perl 6 and hosting their screen + irssi sessions there.
I was about to write feather1 is maintained by a bunch of volunteers, but that would be a lie. It is "maintained" on an as-needed basis by whoever has time and feels half-way competent. It is an "interesting" mixture of Debian unstable and experimental. And it's becoming unmaintainable.
It seems clear what to do: set up a fourth virtual machine, set up a replacement for feather1, and en passant migrate some things (like websites and the pugs repo) to feather2.
But wait! There is an issue. There's always an issue. Setting up a new machine and migrating services takes time. Lots of time. And nobody wants to do it right now. Quite understandably.
Take the pugs repository for example: you might think it's easy enough to
copy /var/svn/... recursively on the new host and set up
Apache... except that you need authentication. And authentication is coupled
to commitbit. And commitbit runs on Jifty. And if you want to install Jifty,
you install half of CPAN. Does anybody know if commitbit is still maintained?
and if it installs cleanly on a Debian stable?
So I thought about some changes to the infrastructure to make it easier to maintain. For example we handle commit bits differently for git projects on github: an IRC bot knows the API keys of the project owners, and can add committers to the projects. That's nice, and the IRC frontend is much leaner than the Jifty-based web frontend. But we lose virality. For security reasons the bot has to keep a whitelist of IRC users who are allowed to invite commiters. Since IRC nick names and (git|svn) user names don't always match, such a list has to be maintained manually.
The second question is: should we take the chance and move the pugs repo to git? I prefer git over svn by far, but there are also costs involved. For example Rakudo checks out a copy of the test suite via svn. The test suite is only a subdirectory of the pugs repo, so unless we split the pugs repo, we'd have to find a way to check out only a part of a git repository. I have no idea if that's possible.
Either way, I'd like to hear your thoughts, and learn from your experience:
Most programming languages handle failures with either of two paradigms: failing routines return special values, or they throw exceptions.
Either way has its severe problems: in languages like C it can be very
simple to forget to catch such a return value, and very tedious to propagate
them to the caller; on the other hand throwing exceptions often clutters the
code with way too many try blocks, and it's generally unfriendly
if you try to automatically parallelize expressions.
So Perl 6 offers a middle ground: soft or unthrown
exceptions. If a routine calls fail("message"), a new
Failure object is created and returned from the current
routine. That object behaves as an undefined value, which stores the message,
file and line information of the fail() location, a backtrace and so on.
When you ask such an object whether it's true or false, or defined or undefined, you'll get a correct answer, and the exception is marked as handled. However if you try to use it as an ordinary value, it turns into an (ordinary) fatal exception. So both of these work:
# Variant 1: no exception thrown my $handle = open('nonexistingfile'); if $handle { .say for $handle.lines; } else { # do something else } # Variant 2 my $handle = open('nonexistingfile'); # throws a fatal exception while calling $handle.lines .say for $handle.lines;
Now if you do some automatically parallelized operations, a single failure doesn't have to abort the whole operation, and neither is information lost
# divide @a1 by @a2 element-wise, a division by zero might occur: @a1 »/« @a2;
The API for accessing the Failure objects isn't very mature
yet, but the concept stands. See S04/Exceptions for the
gory details, as they stand today.
I've blogged about the Perl 6 Advent Calendar 2009, and want to follow up that it's been a huge success so far.
We're about half-way through, and you might be interested in our number of visits per day:
On 6th and 7th of December we had about 18k visitors in sum, courtesy of slashdot and Tim O'Reilly on Twitter.
As a result we got a lot of new people in our #perl6 IRC channel, asking for help on how to get Rakudo working, how to code some specific things in Perl 6, or asking about general design decisions.
Also mj41 reported that he has collected more than 50% more Perl 6/parrot blog posts than in the previous year.
We've also acquired
perl6.org for our uses this year, and it has been promoted enough to be
the third hit on a google search for Perl 6.
All in all I'm rather happy with the marketing state of Perl 6 in 2009, and hope to see similar efforts for 2010. With the upcoming releases of Rakudo Star and a Perl 6 book I'm pretty sure we'll do well, and I hope for more slashdot coverage :-).
These tidings posts seem to become rather sparse, but I hope you get some news by reading the Planet Six feed aggregator anyway.
for 1, 3 ... *+2, 9 { .say } to print all
the odd numbers between 1 and 9. (r28344,
r28348,
r28351).Rational and Complex types now have their own literals
(r28173).As opposed to two months ago, Rakudo now
Patrick Michaud is also working on a new tool named npq-rx, which combines a self-hosting NQP compiler and a new regex engine, which already supports proto regexes, NQP code assertions and closures, and is generally much faster and better than PGE.
Mathew Wilson aka diakopter started sprixel, a Perl 6 to Javascript compiler.
Mildew now has an experimental javascript emitter.
perl6.org is redesigned again, this time spanning multiple pages, thus allowing much more stuff to be linked there.
Four Perl 6 and Rakudo hackers announced that they are writing a Perl 6 book, the print release of which shall coincide with the release of Rakudo Star.
I think it's not too bold to call the Perl 6 Advent Calendar a full success: over 40k page views, more than 150 non-spam comments, and lots of new faces and nicknames in #perl6 - it's been a very pleasant surprise.
I asked myself why it became such a success, and came up with this list of things:
24 days are over rather quickly, so everyone who contributes knows when the deed is done
Some people like the shiny new regex and grammar features, others are interested in the object model, operator overloading or complex numbers. Since we had no fixed topics, everybody could choose a topic that played to their strengths.
Writing a good post about a topic you know takes about half an hour up to an hour, maybe two if you need to do some research on the topic. Either way it's a relatively small task (compared to "write a regex engine" or "redesign the build system of $compiler" or so).
We had a schedule for each day, and people could add their name to a free slot. When the day approached, the other authors started to ask politely how it was progressing, making sure people did not forget their posts. Also authors felt the subtle pressure to finish their posts on time.
We had lots of feedback, most of it quite positive: blog comments, visitors in our IRC channel, being featured on slashdot. That was very encouraging.
Also other authors would preview and proof-read the posts before they were published, pointing out falsities and gems.
PerlJam and colomon took care. The decided to make the advent calendar happen, set up a blog, discussed things and so on - they made it happen.
All these factors encouraged contributions. I don't think all of them are necessary for a successful, lively projects, but they certainly help.
What other factors do you know that encourage contribution in open-source projects? What could we have done even better?
We want a Perl 6 book. We want it badly enough to write it ourselves. So that's what we're doing: writing one.
We, that is Patrick Michaud (architect of the Rakudo Perl compiler), Jonathan Worthington (prolific contributor to both Rakudo and Parrot), Carl Mäsak (frenetic Rakudo user, and our number one bug finder) and Moritz Lenz (keeper of the Perl 6 test suite, and Perl 6 user and blogger). We are also open to contribution from others - already Jonathan Scott Duff has written an initial preface for us.
We don't have a name yet for our book. We want to cover the basics of Perl 6, enough to get your feet wet, and enough to make you want to use it. We want it to be based on useful examples. It is not going to be the definitive book, that task we leave to Larry Wall and Damian Conway.
Our vision is to present primarily the subset of Perl 6 that Rakudo understands, and have printed copies available by the time Rakudo Star is released, that is April or May 2010. chromatic and Allison Randal have kindly offered to published it via Onyx Neon Press.
Until then, monthly releases will be published under a Creative Commons license (noncommercial, attribution, share-alike).
Currently we have four chapters under construction, and the intention of writing the more introductory chapters later, when we know what we need to introduce for the later chapters. So far we have
You can download the preliminary PDF version of the book here.
Interested? Check out
the git repository, and
join us in irc://freenode.net#perl6book.
In the great tradition of Perl Advent Calendars, colomon started and announced the 2009 Perl 6 Advent Calendar, with a post about Perl 6 each day.
After the first post many #perl6 regulars volunteered to contribute a post, so 20 of the 24 days are already allocated.
I'm looking forward to many nice posts, most of which will probably highlight a small Perl 6 feature.
In Perl 5 there is the undef value. Uninitialized variables
contain undef, as well as non-existing hash values, reading from
unopened or exhausted file handles and so on.
In Perl 6 the situation is a bit more complicated: variables can have a type constraint, and are initialized with the corresponding type object:
my Int $x; say Int.WHAT(); # Int()
These type objects are also undefined, but in Perl 6 that doesn't
mean they are a magical value named undef, but that they respond
with False to the defined() subroutine and
method.
In fact there is no undef anymore. Instead there are various
values that can take its place:
Mu is the type object of the root type of the object hierarchy
(or put differently, every object in Perl 6 conforms to Mu). It's
the most general undefined value you can think of.
Nil is a "magic" value: in item (scalar) context it evaluates to
Mu, in list context it evaluates to the empty list. It's the
nothing to see here, move along value.
Each type has a type object; if you want to return a string, but can't
decide which, just return a Str.
Other interesting undefined values are Exception (which
usually contain a message and a back trace), Failure (unthrown
exceptions), Whatever is a generic placeholder that can stand for
"all", "infinitely many", "many" or as a placeholder for a real value.
A few days ago masak blogged about the social aspect of the is Perl 6 really Perl? question.
He presumes that the answer is yes, but doesn't tell us why. I'll try to give some reasons.
Perl 6 started off as the successor to Perl 5, at a Perl 5 meeting, by the Perl crowd. It was a plan to escape both the backwards compatibility trap (which meant that broken things couldn't be fixed without many people yelling), and the lack of momentum in the community.
What makes Perl Perl? In my opinion it's not the sigils on variables that make Perl Perl, or that writing a regex only need two characters and so on. It's mostly the philosophy that makes the difference.
There are some underlying principles like TIMTOWTDI, context sensitivity, convenience over consistency, making simple things easy and hard things possible, often used constructs short and less frequent things longer, and so on.
Perl 6 is founded on all those philosophies and ideals, and also shares
some technical principles. For example sigils on variables (oh, I mentioned
them already ...), easy access to powerful regexes, that fact that operations
coerce their arguments (instead of the type of arguments determining the
operation like in javascript, where a + can either mean addition
or string concatenation).
So if you agree with my definition of what makes Perl Perl, Perl 6 is also Perl. If not, please tell me what's the essence of Perl!
<meta>From my useless musings series.</meta>
As a programmer you have to have confidence in your skills, to some extent, and at the same time you have to constantly doubt them. Weird, eh?
You need some level of confidence to do anything efficiently. Planning ahead requires confidence that you can achieve the steps on your way.
As a programmer you also need some confidence with the language, libraries and other tools you're using.
If you program for money, you also have to assess what kind of programs you can write, and where you might have problems.
In the process of programming you make a lot of assumptions, some of the explicit, some of them implicit. If you want to write a good program, it's essential that you are aware of as many assumptions as possible.
When you find a bug in your program, you have to challenge previous assumptions, and that's where doubt comes in. You not only suspect, but you know that at least one of the assumptions was false (or maybe just a bit too specific), and you know that you did something wrong.
Sometimes programmers make really stupid mistakes which are rather tricky to track down. That's when you have to question your own sanity.
One example (that luckily doesn't happen all that often to me) is when I edit my program, and nothing seems to change. Nothing at all. Depending on the setup it might be some cache, but something it is even more devious - for example I didn't notice that the console where I edit and the console where I test are on different hosts - and thus the edits actually have no effect at all.
After having done such a thing once or twice I adopted the habit of just
adding a die('BOOM'); instruction to my code, to verify that
the part I'm looking at is actually run.
These are moments when I question my own sanity, thinking "how could I have possibly done such a stupid thing?". Doubt.
The same phenomena applies when doing scientific research: since you usually do things that nobody has done before (or at nobody has published about it yet), you can't know the results beforehand -- if you could, your research would be rather boring. So you have no external reference for verification, only your intuition and discussion with peers.
When you learn a programming language, you not only learn about the syntax, semantics and core libraries, but also the coding style and common idioms.
Idioms are common usage patterns; learning and reusing them means you have to spend less time thinking on common things, and have more time working out the algorithms you deal with.
That's different if you learn Perl 6 - it's a largely unexplored field, and while there are loads of nice features, you might still feel a bit lost. At least I do. That's because I often think "There's got to be a much easier way to achieve $this, but it often takes time to find that easier solution - because nobody developed an idiom for it.
In those cases it helps to ask on the #perl6 IRC channel; many smart people read and write there, and are rather good in coming up with simpler solutions.
For example see masak's ROT13 implementation in Perl 6. In the right column you can see later revisions, and how they gradually improve, steady up to a one-liner.
I also made some simplifications to JSON::Tiny, which basically shows that when I wrote these reduction methods first I used Perl 6 baby talk language.
The nice things about exploring the Perl 6 wonderland of unexplored idioms is that it really pushes your ego if you find a nice simplification, and that you have something to blog about for the Planet Perl Iron man ;-)