Every time I store an API token in a plaintext file or an environment variable, it creates a lingering annoyance that follows me around whenever I go. Every year or two, another one of these lands on the pile. I am finally working on purging them all. I’m doing it with the 1Password CLI, and so far so good.

op

1Password’s CLI is op, which lets you do many, many different things. I was only concerned with two: It lets you read single fields from the vault and it lets you read entire items. For example, take this login:

You can see there are a bunch of fields, like username and password and website. You can fetch all of them or just one. It’s a little weird, but it’s much easier to get a locator for one field than for the whole item. If you click the “Copy Secret Reference” option, you’ll get something like this on your clipboard:

"op://rjbs/Pobox/password"

You can pass that URL to op read and it will print out the value of the field. Here, that’s the password. Getting one field at a time can be useful if you only need to retrieve a password or TOTP secret or API token. Often, though, you’ll want to get the whole login at once. It would mean you could just store the item’s id rather than a cleartext username and a reference to the password field. Or worse, a reference to the password field and another one to the the TOTP field. Also, since each field needs to be retrieved separately with op read, it means more external processes and more possibility of weird errors.

The op item get command can fetch an entire item with all its fields. It can spit the whole item out as JSON. Here’s a limited subset of such a document:

{
  "fields": [
    {
      "id": "password",
      "type": "CONCEALED",
      "purpose": "PASSWORD",
      "label": "password",
      "value": "eatmorescrapple",
      "reference": "op://rjbs/Pobox/password",
      "password_details": {
        "strength": "DELICIOUS"
      }
    }
  ]
}

Unfortunately, 1Password doesn’t make it trivial to get the argument you need to pass op item get, but it’s not really hard. You can use “Copy Private Link”, which will get you a URL something like this (line breaks introduced by me):

https://start.1password.com/open/i?a=XB4AE5Q2ESODUTKETZB3BQGCM4
    &v=flk3x357inyiw22qpoiubhsgin
    &i=7wdr3xyzzym2xgorp4zx22zq3h
    &h=example.1password.com

The i= parameter is the item’s id. You can use that as the argument to op item get. Alternatively, given the URL like op://rjbs/Pobox/password you can extract the vault name (“rjbs”) and the item name (“Pobox”) and pass those as separate parameters that will be used to search for the item.

But why do either? You can just use Password::OnePassword::OPCLI!

Password::OnePassword::OPCLI

Here are two tiny examples of its use:

my $one_pw = Password::OnePassword::OPCLI->new;

# Get the string found in one field in your 1Password storage:
my $string = $one_pw->get_field("op://rjbs/Pobox/password");

# Get the complete document for an item, as a hashref:
my $pw_item = $one_pw->get_item("7wdr3xyzzym2xgorp4zx22zq3h");

Hopefully by now you can imagine what this is all doing. get_item returns the data structure that you’d get from op item get. You can look at its fields entry and find what you need. It does have one other trick worth mentioning. Because it’s a bit annoying to get the unique identifier for an item id, you can pass one of those op:// URLs, dropping off the field name, like this:

# Get the complete document for an item, as a hashref:
my $pw_item = $one_pw->get_item("op://rjbs/Pobox");

I’m currently imagining a world where I stick those URLs in place of API tokens and make my software smart enough to know that if it’s given an API token that string starts with op://, it should treat it as a 1Password reference. I haven’t implemented everything I need for that, but I did write something to use this with Dist::Zilla

Dist::Zilla and 1Password

The first thing I wanted to use all this for was my PAUSE password. Unfortunately for me, this was sort of complicated. Or, if not complicated, just tedious. I made a few false starts, but I’m just going to describe the one that I’m running with.

Dist::Zilla is the tool I use (and wrote) for making releases of my CPAN distributions. It’s usually configured with an INI file, like this one:

name    = Test-BinaryData
author  = Ricardo Signes <cpan@semiotic.systems>
license = Perl_5
copyright_holder = Ricardo Signes
copyright_year   = 2010

[@RJBS]
perl-window = long-term

Each section (the things in [...]) is a plugin of some sort. If the name starts with an @ it’s a bundle of plugins instead. But there’s another less commonly seen sigil for plugins: %. A percent sign means that the thing being loaded isn’t a plugin but a stash, which holds data for other plugins to use. These will more often be in ~/.dzil/config.ini than in each project.

The UploadToCPAN plugin, which actually uploads tarballs to the CPAN, looks in a few places for your credentials:

• the %PAUSE stash (or another stash of your choosing)
• ~/.pause, where CPAN::Uploader usually puts these credentials
• user input when prompted

The %PAUSE stash was slightly overspecified in the code. It had to be a bit of configuration with the username and passwords given as text. What I did was relax that so that any stash implementing the (long-existing) Login role could be used. Then I wrote a new implementation of that role, Dist::Zilla::Stash::OnePasswordLogin. In that version of the stash, you only need to provide an item locator, and it will look up the username and password just in time. So, I have something like this in my global config now:

[%OnePasswordLogin / %PAUSE]
item = op://rjbs/PAUSE

Who cares if somebody steals this URL? They can’t read the credential unless I authenticate with 1Password at the time of reading. Putting other login credentials into your configuration for other plugins is similarly safe. Now, when I run dzil release, at the end I’m prompted to touch the fingerprint scanner to finish releasing. Not only is it more secure, but it feels very slightly like I’m in some kind of futuristic hacker movie.

What more could I want from my life as a computer programmer?

Years ago, I found an iOS app called World Uncovered. I used it for a while, then forgot about it, then started using it again. It’s pretty cool, and I keep telling people about it, so I thought I’d write a post about it.

It’s like this: you let it track your movements the same way that a fitness app like RunKeeper would, and instead of telling you how many steps you’re getting in, it tells you where you’ve ever walked, ever. Keeping in mind that I sometimes have it turned off, and forgot about it for years, check out my map for Philadelphia:

Sometimes I joke with my coworkers that I have never been to West Philadelphia. It’s not true, but… I mean, you can see my usual stomping grounds. You can also see the little excursions up toward Bethlehem, off to Camden, and down… well, I don’t know what I was doing down in South Philly, but that little diagonal below the Italian Market is definitely representative of a few trips to Milk Jawn.

Like I said, I’d forgotten all about the app until February, when I was in Vienna for a few days. For some reason I was flipping through my phone’s screens and saw it and thought, “I should log this trip!” It was great fun, and it made me realize all the trips I had failed to log for the past few years: Norway, Brussels, England, Australia, and others. I like to see new places, and these little marked-up maps are a fun memento. Here’s Vienna:

A few months later, I was in Lisbon, which you’ll know if you’ve been keeping up with my posts here. Lisbon was fun because I took a train off to the west coast, so I got a map with two hubs of activity with a long straight train ride between them:

Lisbon was also where I finally got a “landmark” achievement. The app has its own (somewhat eccentric, I think) collection of landmarks, and by visiting them you can tick it off your list. If you visit enough, you get an achievement. Is this a good way to plan your travel? No. Is it fun if it happens anyway? Well, for me it was. I got Belém Tower.

Weirdly, there are no World Uncovered landmarks in Philly or in Melbourne. Still, I’ll keep an eye out for landmarks on future trips. I also wouldn’t mind getting a few of the “passport” achievements for visiting new countries. That’ll take some time, though!

As for the app itself, it feels sort of dated. It hasn’t had an updated in three or four years, and the UI is kind of clunky. The backup feature constantly needs to be reauthenticated with Dropbox, and unless you knew to turn on GPX trip mapping from day one, most of your backup will be in an encrypted zip file. So, I live in modest fear of losing all this data someday, and might look at some better way to do this. (Maybe log all my GPX data in some other app and then import it here later? I don’t know.)

I did have some email interaction with the developer recently, who told me that the app isn’t dead, just done. I can respect that, given all the software of mine that I feel is just done. I’ll keep enjoying it and not worry about its future.

Oh, and as for how I’m enjoying it, I should talk about one more thing: shorties.

Philadelphia is a grid. I like this, because it makes the city much easier for me to navigate and think about. The weird layout of Boston was sort of charming, but also kind of a total pain. I like the grid. In Center City, the grid runs from 30th Street at the west to Front Street (which is basically 1st Street) at the east. Right in the middle, Broad Street (which is basically 14th) divides the city in half. The other main north/south streets are numbered 2-29. Then the major east/west streets run from the Schuylkill River at the west to the Delaware River at the east. They’re mostly named after trees, and there are few enough that it’s pretty easy to name them all.

The thing is, there are lots of other streets in the grid. For example, between 11th and 12th is Marvine Street, running north/south. It’s not always there, though, just sometimes. It runs from Catharine north to Bainbridge, but then stops. It shows up again way further north at Race, running just one block to Vine, and then showing up again later. These streets have been dubbed “shorties” by my excellent colleague Lacey. They’re great, with lots of character and not much traffic. A hundred years ago, each length of shorty might have its own name, but they were rationalized in the 20th century. The street halfway between 8th and 9th is Darien. If it’s between 8th and 9th, but west of center, it’s Schell. If it’s east of center, it’s Mildred. Some blocks have one, two, or all three between them. (Marvine is a funny one. Between Lombard and Walnut, it’s Quince instead. I bet there’s history.)

Anyway, I would like to walk down every shorty in center city. This is not hard, it’s just going to take a lot of time, and a lot of consultation with the map. World Uncovered makes that easy. I think my next plan for tackling these is to start picking one block between home and work and on the way in, hit all of its shorties. Then another one on the way home. That will only get me about a quarter of the city, tops, but it’s a start.

The real treat is when I stop at a street that’s really rich in shorties, especially interior second-order ones, like this gem:

Honestly, look at that tiny length of Irving Street, which often connects two numbered streets east to west. Here, it’s a tiny little alley inside the block, only reachable by another shorty. What a city!

One of the things I wrote at the first PTS (back when it was called the Perl QA Hackathon) was Module::Faker. I wrote about it back then (way back in 2008), and again eleven years later. It’s a library that, given a description of a (pretend) CPAN distribution, produces that distribution as an actual file on disk with all the files the dist should have.

Every year or two I’ve made it a bit more useful as a testing tool, mostly for PAUSE. Here’s a pretty simple sample of how those tests use it:

$pause->upload_author_fake(PERSON => 'Not-Very-Meta-1.234.tar.gz', {
  omitted_files => [ qw( META.yml META.json ) ],
});

This writes out Not-Very-Meta-1.234.tar.gz with a Makefile.PL, a manifest, and other stuff. The package and version (and a magic true value) also appear in lib/Not/Very/Meta.pm. Normally, you’d also get metafiles, but here we’ve told Module::Faker to omit them, so we can test what happens without them. When we were talking about testing the new PAUSE server in Lisbon, we knew we’d have to upload distributions and see if they got indexed. Here, we wouldn’t want to just make the same test distribution over and over, but to quickly get new ones that wouldn’t conflict with the old ones.

This sounded like a job for Module::Faker and a random number generator, so I hot glued the two things together. Before I get into explaining what I did, I should note that this work wasn’t very important, and we really only barely used it, because we didn’t really need that much testing. On the other hand, it was fun. I had fun writing it and seeing what it would generate, and I have plans to have more fun with it. After a long day of carefully reviewing cron job logs, this work was a nice goofy thing to do before dinner.

Data::Fake::CPAN

Data::Fake is a really cool library written by David Golden. It’s really simple, but that simplicity makes it powerful. The ideas are like this:

1. it’s useful to have a function that, when called, returns random data
2. to configure that generator, it’s useful to have a function that returns the kind of function discussed in #1
3. these kinds of functions are useful to compose

So, for example, here’s some sample code from the library’s documentation:

my $hero_generator = fake_hash(
    {
        name      => fake_name(),
        battlecry => fake_sentences(1),
        birthday  => fake_past_datetime("%Y-%m-%d"),
        friends   => fake_array( fake_int(2,4), fake_name() ),
        gender    => fake_pick(qw/Male Female Other/),
    }
);

Each of those fake... subroutine calls returns another subroutine. So, in the end you have $hero_generator as a code reference that, when called, will return a reference to a five-key hash. Each value in the hash will be the result of calling the generators given as values in the fake_hash call.

It takes a little while to get used to working with the code generators this way, but once you do, it comes very easy to snap together generators of random data structures. (When you’re done here, why not check out David Golden’s talk about using higher-order functions to create Data::Fake?) Helpfully, as you can see above, Data::Fake comes with generators for a bunch of data types.

What I did was write a Data::Fake plugin, Data::Fake::CPAN, that provides generators for version strings, package names, CPAN author identities, license types, prereq structures and, putting those all together, entire CPAN distributions. So, this code works:

use Data::Fake qw(CPAN);

my $dist = fake_cpan_distribution()->();

my $archive = $dist->make_archive({ dir => '.' });

When run, this writes out an archive file to disk. For example, I just got this:

$ ./nonsense/module-blaster
Produced archive as ./Variation-20010919.556.tar.gz (cpan author: MDUNN)
- Variation
- Variation::Colorless
- Variation::Conventional
- Variation::Dizzy

There are a few different kinds of date formats that it might pick. This time, it picked YYYYMMDD.xxx. That username, MDUNN, is short for Mayson Dunn. I found out by extracting the archive and reading the metadata. Here’s a sample of the prereqs:

{
  "prereqs" : {
    "build" : {
       "requires" : {
          "Impression::Studio" : "19721210.298"
       }
    },
    "runtime" : {
       "conflicts" : {
          "Writer::Cigarette" : "19830107.752"
       },
       "recommends" : {
          "Error::Membership" : "v5.16.17",
          "Marriage" : "v1.19.6"
       },
       "requires" : {
          "Alcohol" : "v12.16.0",
          "Competition::Economics" : "v19.1.7",
          "People" : "20100228.011",
          "Republic" : "20040805.896",
          "Transportation::Discussion" : "6.069"
       }
    }
  }
}

You’ll see that when I generated this, I ran ./nonsense/module-blaster. That program is in the Module-Faker repo, for your enjoyment. I hope to play with it more in the future, changing the magic true values, maybe adding real code, and just more variation — but probably centered around things that will have real impact on how PAUSE indexes things.

Probably very few people have much use for Module::Faker, let alone Data::Fake::CPAN. I get that! But Data::Fake is pretty great, and pretty useful for lots of testing. Also, generating fun, sort of plausible data makes testing more enjoyable. I don’t know why, but I always like watching my test suite fail more when it’s spitting out fun made-up names at the same time. Try it yourself!

Almost exactly a year since the last Perl Toolchain Summit, it was time for the next one, this time in Lisbon. Last year, I wrote:

In 2019, I wasn’t sure whether I would go. This time, I was sure that I would. It had been too long since I saw everyone, and there were some useful discussions to be had. I think that overall the summit was a success, and I’m happy with the outcomes. We left with a few loose threads, but I’m feeling hopeful that they can, mostly, get tied up.

Months later, I did not feel hopeful. They were left dangling, and I felt like some of the best work I did was not getting any value. I was grouchy about it, and figured I was done. Then, though, I started thinking that there was one last project I’d like doing for PAUSE: Upgrading the server. It’s the thing I said I wanted to do last year, but barely even started. This year, I said that if we could get buy-in to do it, I’d go. Since I’m writing this blog post, you know I went, and I’m going to tell you about it.

PAUSE Bootstrap

Last year, Matthew and I wanted to make it possible to quickly spin up a working PAUSE environment, so we could replace the long-suffering “pause2” server. We were excited by the idea of starting from work that Kenichi Ishigaki had done to create a Docker container running a test instance. We only ended up doing a little work on that, partly because we thought we’d be starting from scratch and didn’t know enough Docker to be useful.

This year, we decided it’d be our whole mission. We also said that we were not going to start with Docker. Docker made sense, it was probably a great way to do it, but Matthew and I still aren’t Docker users. We wanted results, and we felt the way to get them was to stick to what we know: automated installation and configuration of an actual VM. We pitched this plan to Robert Spier, one of the operators of the Perl NOC and he was on board. I leaned on him pretty hard to actually come to Lisbon and help, and he agreed. (He also said that a sufficiently straightforward installer would be a good starting point for turning things into Docker containers later, which was reassuring.)

At Fastmail, where Matthew and I work, we can take every other Friday for experimental or out-of-band work, and we decided we’d get started early. If the installer was done by the time we arrived, we’d be in a great position to actually ship. This was a great choice. Matthew and I, with help from another Fastmail colleague, Marcus, wrote a program. It started off life as unpause, but is now in the repo as bootstrap/mkpause. You can read the PAUSE Bootstrap README if you want to skip to “how do I use this?”.

The idea is that there’s a program to run on a fresh Debian 12 box. That installs all the needed apt packages, configures services, sets up Let’s Encrypt, creates unix users, builds a new perl, installs systemd services, and gets everything running. There’s another program that can create that fresh Debian 12 box for you, using the DigitalOcean API. (PAUSE doesn’t run in DigitalOcean, but Fastmail has an account that made it easy to use for development.)

I think Matthew and I worked well together on this. We found different rabbit holes interesting. He fixed hard problems I was (barely) content to suffer with. (There was some interesting nonsense with the state of apt locking and journald behavior shortly after VM “cloud init”.) I slogged through testing exactly whether each cron job ran correctly and got a pre-built perl environment ready for quick download, to avoid running plenv and cpanm during install.

Before we even arrived, we could go from zero to a fully running private PAUSE server in about two and a half minutes! Quick builds meant we could iterate much faster. We also had a script to import all of PAUSE’s data from the live PAUSE. It took about ten minutes to run, but we had it down to one minute by day two.

When we arrived, I took my todo and threw it up on the wall in the form of a sticky note kanban board.

We spent day one re-testing cron jobs, improving import speed, and (especially) asking Andreas König all kinds of questions about things we’d skipped out of confusion. More on those below, but without Andreas, we could easily have broken or ignored critical bits of the system.

By the end of day two, we were confident that we could deploy the next day. I’d hoped we could deploy on day two, but there were just too many bits that were not quite ready. Robert had spent a bunch of time running the installer on the VM where he intended to run the new production PAUSE service, known at the event as “pause3”. There were networking things to tweak, and especially storage volume management. This required the rejiggering of a bunch of paths, exposing fun bugs or incorrect assumptions.

The first thing we did on day three was start reviewing our list of pre-deploy acceptance tests. Did everything on the list work? We thought so. We took down pause2 for maintenance at 10:00, resynchronized everything, watched a lot of logs, and did some uploads. We got some other attendees to upload things to pause3. Everything looked good, so we cut pause.perl.org over to pause3. It worked! We were done! Sort of.

We had some more snags to work through, but it was just the usual nonsense. A service was logging to the wrong place. The new MySQL was stricter about data validation than the old one. An accidental button-push took down networking on the VM. Everything got worked out in the end. I’ll include some “weird stuff that happened” below, but the short version is: it went really smoothly, for this kind of work.

On day four, we got to work on fit and finish. We cleaned up logging noise, we applied some small merge requests that we’d piled up while trying to ship. We improved the installer to move more configuration into files, instead of being inlined in the installer. Also, we prepared pull requests to delete about 20,000 lines of totally unused files. This is huge. When trying to learn how an older codebase works, it can be really useful to just grep the code for likely variable names or known subroutines. When tens of thousands of lines in the code base are unused, part of the job becomes separating live code out from dead code, instead of solving a problem.

We also overhauled a lot of documentation. It was exciting to replace the long and bit-rotted “how to install a private PAUSE” with something that basically said “run this program”. It doesn’t just say that, though, and now it’s accurate and written from the last successful execution of the process. You can read how to install PAUSE yourself.

Matthew, Robert, and I celebrated a successful PTS by heading off to Belém Tower to see the sights and eat pastéis.

I should make clear, finally, that the PAUSE team was five people. Andreas König and Kenichi Ishigaki were largely working on other goals not listed here. It was great to have them there for help on our work, but they got other things done, not documented in this post!

Here’s our kanban board from the end of day four:

Specifics of Note

run_mirrors.sh

This was one of the two mirroring-related scripts we had to look into. It was bananas. Turns out that PAUSE had a list of users who ran their own FTP servers. It would, four times a day, connect to those servers and retrieve files from them directly into the users’ home directories on PAUSE.

Beyond the general bananas-ness of this, the underlying non-PAUSE program in /usr/bin/mirror no longer runs, as it uses $*, eliminated back in v5.30. Rather than fix it and keep something barely used and super weird around, we eliminated this feature. (I say “barely used”, but I found no evidence it was used at all.)

make-mirror-yaml.pl

The other mirror program! This one updated the YAML file that exposes the CPAN mirror list. Years ago, the mirror list was eliminated, and a single name now points to a CDN. Still, we were diligently updating the mirror list every hour. No longer.

rrrsync

You can rsync from CPAN, but it’s even better to use rrr. With rrr, the PAUSE server is meant to maintain a few lists of “files that changed in a given time window”. Other machines can then synchronize only files that have changed since they last checked, with occasionally full-scan reindexes.

We got this working pretty quickly, but it seemed to break at the last minute. What had happened? We couldn’t tell, everything looked great, and there were no errors. Eventually, I found myself using strace against perl. It turned out that during our reorganization of the filesystem, we’d moved where the locks live. We put in a symlink for the old name, and that’s what rrr was using… but it didn’t follow symlinks when locking. Once we updated the configuration to use the canonical name and not the link, everything worked.

Matthew said, “You solved a problem with strace!” I said, “I know!” Then we high fived and got back to work.

I was never happy with the symlinks we introduced during the filesystem reorganization, but I was happy when I eliminated the last one during day four cleanup!

the root partition filled up

We did all this work to keep the data partition capable of growth, and then / filled up. Ugh.

It turned out it was logs. This wasn’t too much of a surprise, but it was annoying. It was especially annoying because we decided early on that we’d just accept using journald for all our logging, and that should’ve kept us from going over quota.

It turned out that on the VM, something had installed a service I’d never heard of. Its job was to notice when something wanted to use the named syslog socket, and then start rsyslogd. Once that happened, we were double-logging a ton of stuff, and there was no log rotation configured. We killed it off.

We did other tuning to make sure we’d keep enough logs without running out of space, but this was the interesting part.

Future Plans

We have some. If nothing else, I’m dying to see my pull request 405 merged. (It’s the thing I wrote last year.) I have a bunch of half-done work that will be easier to finish after that. But the problem was: would this wait another year?

We finished our day — just before heading off to Belém — by talking about development between now and then. I said, “Look, I feel really demotivated and uninterested if I can’t continuously ship and review real improvements.” Andreas said, “I don’t want to see things change out from under me without understanding what happened.”

The five of us agreed to create a private PAUSE operations mailing list where we’d announce (or propose) changes and problems. We all joined, along with Neil Bowers, who is an important part of the PAUSE team but couldn’t attend Lisbon. With that, we felt good about keeping improvements flowing through the year. Robert has been shipping fixes to log noise. I’ve got a significant improvement to email handling in the wings. It’s looking like an exciting year ahead for PAUSE! (That said, it’s still PAUSE. Don’t expect miracles, okay?)

Thanks to our sponsors and organizers

The Perl Toolchain Summit is one of the most important events in the year for Perl. A lot of key projects have folks get together to get things done. Some of them are working all year, and use this time for deep dives or big lifts. Others (like PAUSE) are often pretty quiet throughout the year, and use this time to do everything they need to do for the year.

Those of us doing stuff need a place to work, and we need to a way to get there and sleep, and we’re also pretty keen on having a nice meal or two together. Our sponsors and organizers make that possible. Our sponsors provide much-needed money to the organizers, and the organizers turn that money into concrete things like “meeting rooms” and “plane tickets”.

I offer my sincere thanks to our organizers: Laurent Boivin, Philippe Bruhat, and Breno de Oliveira, and also to our sponsors. This year, the organizers have divided sponsors into those who handed over cash and those who provided in-kind donation, like people’s time or paying attendee’s airfare and hotel bills directly. All these organizations and people are helping to keep Perl’s toolchain operational and improving. Here’s the breakdown:

Monetary sponsors: Booking.com, The Perl and Raku Foundation, Deriv, cPanel, Inc Japan Perl Association, Perl-Services, Simplelists Ltd, Ctrl O Ltd, Findus Internet-OPAC, Harald Joerg, Steven Schubiger.

In kind sponsors: Fastmail, Grant Street Group, Deft, Procura, Healex GmbH, SUSE, Zoopla.

Breno especially should get called out for organizing this from five thousand miles away. You never could’ve guessed, and it ran exceptionally smoothly. Also, it meant I got to see Lisbon, which was a terrific city that I probably would not have visited any time soon otherwise. Thanks, Breno!