On Friday, May 10, I presented “Not So Extreme Programming” for Large Scale Scrum (LeSS) in NYC & Global.

The abstract:

Where “Agile” sounds pleasant and inclusive, “Extreme Programming”… doesn’t. But it’s a differentiator: teams practicing XP are seen to move with uncommon agility. The name has other problems, too. For one, XP is about much more than programming. For another, when compared with other ways software still gets developed, XP is much less extreme. This talk — for anyone involved with Agile in any role, at any scale — will take you through where Extreme Programming came from, where it’s going, what it requires, why it remains as relevant as ever, and how to take advantage.


Posted May 10, 2024 at 12:00:00 PM EDT Tags:

notqmail logo

My early imaginings of a collaborative Open Source successor to qmail, let me assure you, did not include going nearly four years between releases. Well, at least it hasn’t been more than four. notqmail 1.09 is here:

For decades, due to each administrator needing to patch in their particular missing bits of functionality, the qmail source code itself has effectively been a public API. Some future release of notqmail will include everything most everyone needs. On that day, we’ll freely make desirable code changes without worrying about breaking people’s patches. On that day, notqmail will have become a relatively normal software project operating under relatively normal constraints.

This is not that day. notqmail remains a uniquely challenging legacy-code rehabilitation project, and 1.09 is merely a solid, long-overdue release that includes the work of a couple dozen new contributors.

Since this release took too long, our next development cycle will be

  1. Time-bounded
  2. Focused on process improvements

In legacy code, every time we can turn a vicious cycle virtuous, it’s a big win. By making the code easier and safer to change, we’ll have more fun; by having more fun, we’ll make more progress; by making more progress, we’ll get more feedback; by getting more feedback, we’ll have more fun; and so on.

Have fun with notqmail 1.09! Let us know how the upgrade goes for you. (I’ll be updating the pkgsrc package soon.) And if getting involved is your kind of thing, please feel welcome to join us.

Posted May 7, 2024 at 10:44:01 AM EDT Tags:

This might turn into a collection. Let’s see.


Joining a team.


To earn trust and reciprocity.


Demonstrate good will and good skill.


Observe what’s regularly tedious, bothersome, or frustrating. Do not choose something you think they should want; this technique is to show that you’re listening, that their needs matter to you, and that you’re willing and able to help those needs get met. (Examples of what you might see or hear: make unreliable tests reliable, speed up slow tests, automate a build step.) Then go off and fix it — or if it’s too big, a tractable part of it — as promptly as possible.

By the usual criteria, this is a great gift. They want it, they’d never get it for themselves, you made it for them, and it marks a special occasion: the beginning of a fruitful collaboration.


Posted April 17, 2024 at 03:35:12 PM EDT Tags:

To everyone who helped me with the search, thank you. Today marks the start of my first regular job in seven years.

In my first job as a software developer, by a string of lucky occurrences, I happened to land someplace doing Extreme Programming. I hadn’t been looking for it; as with all the other ways people make software together, I hadn’t heard of XP and had no basis for comparison. But I remember a feeling of excitement rather larger than that of getting paid to be a programmer (which, to be clear, was not inconsiderable). And I remember that in my early Google searches for the experience reports of other people doing XP, the weblog that most stood out was James Shore’s. The way he was doing it and writing about it helped me learn to do it myself.

In my first job as a software development coach, I began to suspect that I’d need to interpret the Manifesto’s “doing it and helping others do it” to mean something more specific to me: alternating between helping and doing. I worried that after too long being free of responsibility for delivery outcomes I’d stop believing my own advice. I guessed that to refresh my contextual understanding of what helps which people deliver which results under which conditions, I’d periodically need to return to working as a teammate in a team. I supposed that after another round of being accountable to teammates and customers I’d be freshly motivated to share what I’d learned. If this pendulum could keep swinging, I could perhaps sustain a consulting career.

That prediction held true. When my employment with Pillar (an XP consultancy) ended in March 2017 I was about to become a parent, with the expectation that my life was about to change in ways I could not possibly predict. That held true, too. Being an independent mostly-remote consultant proved particularly adaptive for our family life when the pandemic struck. I prioritized client opportunities according to two criteria: their compatibility with our needs at home and their impetus for my career pendulum.

I enjoyed a fantastic mix of clients — the problems they were solving, the aspects they wanted help with, the cultures they brought with them — but in the course of interviewing for this job I realized it’s been more than seven years since I personally lived the day-to-day accumulated complexity of seeing through slow changes to the cultural norms in and around a software team. I suddenly felt a hole in my heart where that feedback wasn’t. As a consultant, every now and then I’d hear (years later) of some case where I helped make a dent. But mostly I’d never know one way or the other. It’s uncomfortable to make an effort with no expectation of finding out what good it did or didn’t do. It’s a very particular kind of uncomfortable, for a devotee of Extreme Programming, to make a habit of it.

So I’m keen to take a break from “just visiting” and move in someplace. I’m eager to confound the pendulum by inhabiting a role that combines responsibility for outcomes with responsibility for techniques. I’m pleased to share that today’s my first day as a Staff Engineer XP Coach at OpenSesame, reporting to none other than Jim Shore. And I’m looking forward — as I’m doing it and helping others do it — to sharing, as always, what I learn.

Posted April 9, 2024 at 11:10:04 AM EDT Tags:

A few weeks ago, Apple released new versions of Xcode and Command Line Tools. Not thinking too hard about how my pkgsrc developer environment often gets broken by OS or tool updates, I updated promptly. For one thing, I’m kinda used to it. For another, it doesn’t usually break. For a third thing, managing dependencies — anything not my code that can break my code — means responsibility for dealing with the inevitable trouble, and therefore the sooner I find it the better. (More on my approach to life with dependencies.)

A vendor-provided toolchain is a significant dependency. So I accepted the Command Line Tools update, and it commandeered my spare time for two weeks as I hurried carefully to repair one of the world’s biggest continuous-integration cauldrons on one of its most popular platforms. When I ran my usual pkg_rolling-replace -suv to rebuild anything outdated, it did not go well at all.

This article uses “we” because the continued smooth operation of pkgsrc on macOS reflects the contributions of many developers on many occasions, including this one: I happened to be first on the scene, but several of us of were discussing the problems and potential workarounds and all of “my” commits were adjusted accordingly.

Did I mention that a few weeks ago we were aiming to stabilize for yesterday’s quarterly release? Suddenly, if we didn’t scramble to straighten things out for macOS users, we’d have to manage a complicated situation for a while. But if we created a mess on other platforms by moving rashly, that’d be even worse.

The usual feedback mechanism for proposed infrastructure changes is to compare full bulk builds before and after. There was no time for that.

Happily, the conclusion of the story is boring: as always, the pkgsrc 2024Q1 stable branch supports macOS and its developer tools, including the latest releases of each. (So does -current pkgsrc, of course, if that’s your thing.)

Curious what we had to do to keep it boring? Read on.

Stricter clang defaults

Upstream Clang 16 and GCC 14 have promoted several warnings to errors by default, and Apple’s Clang 15 has followed suit. (Gentoo has very helpfully documented this for packagers.) These changes are intentional and well-intentioned, pushing maintainers to ship more reliable code. But pkgsrc’s job is to build nearly 30,000 codebases we don’t maintain. And stricter compiler defaults break a lot of builds.

As you might hope, we can make the breakage go away in one place.

In pkgsrc, packages declare which programming languages are required for their build. The compiler framework then selects package-and-platform-appropriate compilers, places them preferentially in the package’s build environment, and — crucially — intercepts compiler invocations and rewrites them for a variety of purposes.

When we look into pkgsrc’s clang logic, we find prior art for this specific class of problem. In September 2020, Xcode 12 (and its associated Command Line Tools) arrived even later in our quarterly schedule and promoted -Wimplicit-function-declaration to an error. The surgical fix: on macOS only, if invoking clang reveals the new stricter default, we pass -Wno-error=implicit-function-declaration to demote the error back to a warning.

Apple Clang 15’s new strictures aren’t observable in the same way, so we adjust our workaround: if clang doesn’t complain when we try demoting the new errors back to warnings, we pass those arguments too, via the same compiler-framework mechanism.

Missing m4 and yacc

This messy regression found only in the Command Line Tools update — not in the corresponding full Xcode 15.3 (build 15E204a) update — must have been unintended.

On macOS, some of the familiar Unix tools in /usr/bin are in fact stubs. When invoked, they either execute into the corresponding installed program (living somewhere under /Library/Developer) or prompt the user to install the Command Line Tools.

This Command Line Tools update uninstalls m4 and yacc from /Library/Developer. But since the OS-provided /usr/bin/m4 and /usr/bin/yacc stubs still exist, running m4 or yacc still does something: it pops up a window prompting you to reinstall the CLT. Unfortunately, as the latest available version doesn’t provide those tools, reinstalling is a waste of time.

As you might once again hope, we can hide the problem without personally visiting 29,000+ packages.

In pkgsrc, we also have a framework to control which non-compiler tools are invoked during builds. Packages declare which tools are required for their build. The tools framework then selects package-and-platform-appropriate incarnations of the declared tools and places them preferentially in the package’s build environment.

We just got handed a few new twists to handle in the framework, is all.

First, because this clever new CLT failure mode outfoxes our usual tool-detection mechanism, we special-case m4 and yacc detection on macOS, performing an existence check for the stubs’ targets. Then the selection mechanism’s usual fallback logic can provide them some other way. This prevents the primary source of needless CLT install popups. For non-macOS platforms, no change.

Second, because some packages might not yet be declaring all their tool dependencies, we special-case m4 and yacc handling on macOS: when they’re not declared, we place them in the build environment anyway, as no-ops. If the package build happens to invoke them, nothing happens. This prevents the secondary source of needless CLT install popups, at the risk of breaking macOS builds for packages that are missing these tool declarations and have heretofore gotten lucky; in such cases, the breakage will be obvious and the fix easy. For non-macOS platforms, no change. (At leisure, we might like to broaden this approach to help find and fix all undeclared tools on all platforms.)

Third, because the flex tool expects to invoke a GNU-compatible m4, we adjust the tools framework to infer gm4 from a flex declaration so that the framework controls which m4 gets found. This more correctly expresses our intent on all platforms, and in the macOS package build environment it restores /usr/bin/flex to a working state.

Broader xcrun search

We were already relying on xcrun for a couple of things, so when our new tool-detection special cases were sometimes getting surprising results from it, that was concerning. Turns out xcrun no longer looks solely in Apple-controlled locations, but also consults the environment’s $PATH. By invoking xcrun with an empty PATH and --no-cache, we obtain controlled, predictable tool detection.


Under the constraints, we changed as little as possible, as safely as possible, as similarly as possible to previous proven changes, avoiding novel constructs or any whiff of unforeseen consequences. We could not have done nearly as safe or thorough a job without good abstractions already in place. Total lines of pkgsrc infrastructure code changed: less than 100. Now that 2024Q1 is out, we have room to refactor.

These 15.3 updates also include a brand new linker. So far it hasn’t given us any trouble. If that changes, wanna guess whether we have one place to take care of it?

Posted April 2, 2024 at 03:19:32 PM EDT Tags: