pages tagged qmailYareev's schmonz.comhttps://schmonz.com/tag/qmail/Yareev's schmonz.comikiwiki2023-11-09T01:12:08ZNovember crowdfunding updatehttps://schmonz.com/2021/12/04/november-crowdfunding-update/Amitai Schleier2023-04-22T11:55:21Z2021-12-04T10:43:48Z
<p><em>[ <a href="https://schmonz.com/crowdfunding/">About my public-facing work</a> ]</em></p>
<p>Starting now, I’m moving my monthly posts from
<a href="https://schmonz.com/link/patreon/">Patreon</a>
to my own website here.
Why?</p>
<ol>
<li>I prefer to own my data</li>
<li>There are many ways (Patreon merely one among them) in which people can fund my public-facing work</li>
</ol>
<p>My
<a href="https://latentagility.com">corporate work</a>
focuses on
<a href="https://latentagility.com/tech/">learning together, experientially</a>.
My public-facing work is similar: I’m creating learning experiences, Open Source code, and combinations thereof — at present, like so:</p>
<ul>
<li>Packaging third-party software for a cross-platform Unix package manager</li>
<li>Developing an email server</li>
<li>Fixing bugs and adding features to a (mostly) static site generator</li>
<li>Facilitating ensemble/mob programming sessions</li>
<li>Streaming solo programming sessions</li>
<li>Organizing meetups about programming and Agile</li>
</ul>
<p>For more, see
<a href="https://schmonz.com/crowdfunding/">crowdfunding</a>.
I’m grateful for your support.</p>
<h1>Experiences</h1>
<p>Held our final
<a href="https://pubmob.com/offerings/amitaischleier-legacy-open-source-fridays/">Legacy Open Source Fridays</a>
ensemble session of 2021.
Started back up with
<a href="https://www.twitch.tv/schmonzie">streaming my solo programming sessions on Twitch</a>,
mostly pkgsrc-related so far.
Improving my stream a bit each time.</p>
<p>For
<a href="https://www.meetup.com/Jersey-City-Java-User-Group-JC-JUG/">Jersey City Java</a>,
experimented with having a vendor present their product: a brief introduction to the tool, followed by
<a href="https://schmonz.com/2021/11/11/jcjug-november-2021-pejman-ghorbanzade/">programming together with Pejman Ghorbanzade</a>.
Glad we tried it.
If we do another vendor session sometime, this’ll be how.</p>
<p>Building momentum with
<a href="https://www.meetup.com/Southern-Connecticut-Agile-Meetup/">Southern Connecticut Agile</a>,
our second meetup was an extremely well liked
<a href="https://schmonz.com/2021/11/23/soctagile-november-2021-esther-derby-matthew-carlson/">conversation with Esther Derby and Matthew Carlson</a>.
We’ll skip December (too much holiday stuff), though
<a href="https://www.meetup.com/Jersey-City-Java-User-Group-JC-JUG/events/282467826/">JC-JUG’s session</a>
will be of interest.
I’m excited for our January SoCTAgile speaker.</p>
<h1>Build farm</h1>
<p><a href="https://www.virtualbox.org/wiki/Changelog-6.1#v30">VirtualBox 6.1.30</a> fixed the macOS Monterey
<a href="https://www.patreon.com/posts/57678431">troubles I encountered last month</a>.</p>
<p>Upgrading Devuan 3.1 to 4.0 was straightforward, as was updating Ubuntu aarch64 to 21.10.</p>
<p>After much reading and trying stuff,
<a href="https://schmonz.com/2021/11/22/now/">bringing up a 2007 MacBook</a>
(64-bit system, 32-bit EFI) with Lubuntu 21.10 was ultimately uneventful.
It’s no speed demon.
I doubt I’ll keep it running.
But the tricks I’ve just learned should apply to my original 2006 Mac Pro, boosted many years ago with SSD and lots of RAM and needing only an OS that can be kept current.
In the meantime, a cursory build of my usual packages turned up a build failure in libspf2.</p>
<h1>pkgsrc fixes</h1>
<ul>
<li>Doing cross-platform testing of an Ubuntu 21.10 fix for libspf2 (works nearly everywhere else, but needs more fixing on OpenBSD and Void)</li>
<li><a href="http://mail-index.netbsd.org/pkgsrc-users/2021/11/29/msg034809.html">Reviewed a fix needed in my cross-platform build environment</a>, now awaiting commit by the author</li>
<li>lighttpd: upstream patch for use-after-free</li>
<li>libhighlight: bump required API version to fix runtime errors seen on wiki.netbsd.org</li>
<li>ikiwiki: provide pkgsrc-compatible default values for configurable paths to fix runtime errors seen on wiki.netbsd.org</li>
<li>Linux with non-executable glibc (such as Ubuntu/aarch64 21.10): fall back to detecting GLIBC_VERSION another way</li>
<li>qmail and djbdns: catch up to pkgsrc’s switch from RMD160 to BLAKE2s hashes</li>
<li>gdk-pixbuf2: fix macOS build</li>
<li>ucspi-tools: fix Linux build</li>
<li>bootstrap: note that Solaris 11 works</li>
</ul>
<h1>pkgsrc updates</h1>
<ul>
<li>mob to 2.1.0</li>
<li>texttest to 4.0.8</li>
<li>p5-App-Sqitch to 1.2.0</li>
<li>py-approvaltests to 3.1.0</li>
<li>getmail to 5.16</li>
</ul>
<h1>pkgsrc additions</h1>
<ul>
<li>ucspi-udp</li>
<li>tcpexec</li>
<li>fd-proxy</li>
<li>pikchr</li>
<li>AusweisApp2 (to pkgsrc-wip for further attention)</li>
<li>dstp (also to pkgsrc-wip)</li>
</ul>
<h1>notqmail</h1>
<p>Legacy Open Source Fridays has produced a few pull requests which we’re still working through.
I made some progress on getting
<a href="https://github.com/notqmail/notqmail/pull/224">Add tests for qmail-send:job_*() functions</a>
past the Solaris autobuilds.</p>
<p>Legacy Open Source Fridays has also produced a few people with motivation to continue programming notqmail.
I had not imagined this possibility, and am gratified that it’s happened.</p>
<h1>ikiwiki</h1>
<p>My motivation for packaging pikchr was to be able to integrate it into ikiwiki.
Ikiwiki already has a
<a href="http://ikiwiki.info/plugins/graphviz/">graphviz plugin</a>
which I’ve been using to generate
<a href="https://schmonz.com/software/acceptutils/">somewhat explanatory diagrams of acceptutils</a> — but I’m not thrilled with my diagrams, pikchr appears designed to run in precisely this kind of context, and maybe I’ll like it better.
When I write the pikchr plugin for ikiwiki, it’ll be streamed
(<a href="https://www.twitch.tv/schmonzie">subscribe to my Twitch</a>).
In the meantime, you can
<a href="https://youtu.be/MxbqNHbjTBc">watch me create the pikchr package</a>.</p>
notqmail 1.08 releasedhttps://schmonz.com/2020/05/20/notqmail-108-released/Amitai Schleier2023-11-09T01:12:08Z2020-05-20T21:14:45Z
<p><a href="https://notqmail.org"><img src="https://schmonz.com/software/notqmail.png" width="188" height="188" alt="notqmail logo" title="notqmail logo" class="img" align="right" /></a></p>
<p>I’m pleased to announce notqmail 1.08, the latest update to
<a href="https://schmonz.com/2019/08/20/announcing-notqmail/">notqmail</a>.
It addresses
<a href="https://www.qualys.com/2020/05/19/cve-2005-1513/remote-code-execution-qmail.txt">security vulnerabilities in qmail 1.03 reported yesterday by Qualys</a>.
As a fork, we’ve inherited qmail’s relatively few bugs;
as an <em>active</em> qmail fork, we’ve addressed the vulnerabilities with a timely release;
and as the collaborative Open Source successor to qmail, we
<a href="https://notqmail.org/trust">show our work</a>.</p>
<p>The 1.08 release also includes many other small improvements we’ve made since 1.07, including fewer compiler warnings, less dead code, published intent to remove more presumed-dead code,
<a href="https://schmonz.com/software/remote/">my QMAILREMOTE patch</a>,
and our first few unit tests.
For more detail, see the
<a href="https://notqmail.org/releases/1.08">notqmail 1.08 release notes</a>.</p>
Programming Leadership: Collaboration and Notqmail with Amitai Schleierhttps://schmonz.com/talk/20200110-programming-leadership/Amitai Schleier2020-10-06T22:03:54Z2020-01-10T12:51:40Z
<p>A few weeks later,
<a href="https://schmonz.com/2019/08/20/announcing-notqmail/">notqmail had its first release</a>.
But in July, when
<a href="https://twitter.com/justzeros">Marcus Blankenship</a>
had me on his show, I was standing on nothing but wobbly aspirations and a semi-clever name.
Marcus and I talked about building a collaborative open-source team in a software community with a long history of… not that.
Here’s
<a href="https://marcusblankenship.com/collaboration-and-notqmail-with-amitai-schleier/">our conversation</a>.</p>
NYCBUG: What is notqmail?https://schmonz.com/2020/01/08/nycbug-what-is-notqmail/Amitai Schleier2020-02-13T00:48:55Z2020-01-08T20:40:14Z
<p>On Wednesday, January 8,
I attended the
<a href="https://www.nycbug.org">New York City BSD User Group</a>
to present
<a href="https://www.nycbug.org/index?action=view&id=10675">What is notqmail?</a>,
a perhaps not entirely surprising followup to my
<a href="https://schmonz.com/2019/03/06/nycbug-maintaining-qmail-in-2019/">March talk</a>.
At the time, I’d been trying to avoid creating yet another qmail fork.
This talk is about my failure — notqmail is alive and well — and about our success thus far.</p>
<p>Abstract:</p>
<blockquote><p>We all use email, so we all use email servers.
notqmail is software for running an
<a href="https://en.wikipedia.org/wiki/Message_transfer_agent">email server</a>.
Someday, if we do a good job, some of the many
<a href="https://arstechnica.com/information-technology/2014/02/how-to-run-your-own-e-mail-server-with-your-own-domain-part-1/">articles about how and why to run your own</a>
will recommend notqmail.</p>
<p><a href="http://notqmail.org">notqmail</a>
is a community-driven fork of
<a href="https://cr.yp.to/qmail.html">qmail</a>,
beginning where
<a href="http://netqmail.org">netqmail</a>
left off: providing stable, compatible, small releases to which existing qmail users can safely update.
notqmail also aims higher: developing an extensible, easily packaged, and increasingly useful modern mail server.</p></blockquote>
<ul>
<li><a href="https://schmonz.com/2020/01/08/nycbug-what-is-notqmail/slides/">Slides</a></li>
</ul>
<hr />
<h2>Work with me</h2>
<p>Would you personally benefit from an individualized session with an experienced, inquisitive, and empathetic conversation partner?
Maybe you’re facing a challenging situation at work, a learning opportunity in some code — or both.
Last week a new client went from “frustrated” to “energized” in the span of an hour.
Get in touch: <a href="https://latentagility.com">latentagility.com</a></p>
<p>Would your org benefit from a rare combination of technical coaching and impactful conversations?
(Take
<a href="https://latentagility.com/testimonials">other people’s word for it</a>,
not mine.)
It’s not too late to book some time with me in 2020.
Let’s talk about
<a href="https://latentagility.com">what fits for you</a>.</p>
Announcing notqmailhttps://schmonz.com/2019/08/20/announcing-notqmail/Amitai Schleier2023-11-09T01:12:08Z2019-08-20T15:13:00Z
<p><a href="https://schmonz.com/2017/03/27/automation-for-mail-hosting/">Running my own email server</a> has its challenges.
Chief among them:
<a href="https://cr.yp.to/qmail.html">my favorite mail-server software</a>
hasn’t been updated since I started using it in 1998.</p>
<table class="img align-left"><caption>The qmail logo</caption><tr><td><a href="https://cr.yp.to/qmail.html"><img src="https://schmonz.com/2019/08/20/Q.6.01.Logo.lg.jpg" width="203" height="186" alt="qmail logo" title="qmail logo" class="invertible" /></a></td></tr></table>
<p>Okay, that’s not entirely true.
While qmail hasn’t been updated by
<a href="https://cr.yp.to/djb.html">its original author</a>,
a group of respected users created
<a href="http://netqmail.org">netqmail</a>,
a series of tiny updates that were informed, conservative, and careful.
By their design, it was safe for everyone running qmail to follow netqmail, so everyone did.
But larger changes in the world of email — authentication, encryption, and ever-shifting anti-spam techniques — remained as puzzles for each qmail administrator to solve in their own way.
And netqmail hasn’t been updated since 2007.</p>
<h2>One fork per person</h2>
<p>In the interim, devotees have continued maintaining their own individual qmail forks.
Some have shared theirs publicly.
I’ve preferred the design constraints of making minimal, purpose-specific, and conflict-avoidant
<a href="https://schmonz.com/software/">add-ons and patches</a>.
Then again, these choices are motivated by the needs of
<a href="https://schmonz.com/software/pkgsrc-qmail-run/">my qmail packaging</a>,
which I suppose is itself a <em>de facto</em> fork.</p>
<p>I’ve found this solo work quite satisfying.
I’ve
<a href="https://schmonz.com/2016/07/08/how-to-learn-c-part-1/">learned more C</a>,
reduced build-time complexity, added run-time configurability, and published
<a href="https://schmonz.com/2019/01/07/2018q4-qmail-updates-in-pkgsrc/">unusually polished and featureful qmail packages for over 20 platforms</a>.
Based on these experiences, I’ve given
<a href="https://schmonz.com/2019/01/25/devopsdays-nyc-run-your-own-email-server/">dozens</a>
of
<a href="https://schmonz.com/2019/05/01/deliveragile-2019-strangle-your-legacy-code/">workshops</a>
and
<a href="https://schmonz.com/2019/03/06/nycbug-maintaining-qmail-in-2019/">talks</a>.
In seeking to simplify system administration for myself and others,
I’ve become a better programmer and
<a href="https://latentagility.com">consultant</a>.</p>
<p>Still, wouldn’t it be more satisfying if we could somehow pool our efforts?
If, long after the end of DJB’s brilliant one-man show, a handful of us could shift how we relate to this codebase — and to each other — in order to bring a collaborative Open Source effort to life?
If, with netqmail as inspiration, we could produce safe updates while also evolving qmail to meet more present-day needs?</p>
<h2>One fork per community</h2>
<table class="img align-right"><caption>My subtle artwork</caption><tr><td><a href="https://notqmail.org"><img src="https://schmonz.com/software/notqmail.png" width="188" height="188" alt="notqmail logo == qmail logo overlaid by a red circle with a slash through it" title="notqmail logo == qmail logo overlaid by a red circle with a slash through it" class="img" /></a></td></tr></table>
<p>Say hello to
<a href="https://notqmail.org">notqmail</a>.</p>
<p>Our
<a href="https://notqmail.org/releases/1.07">first release</a>
is informed, conservative, and careful — but bold.
It reflects our brand-new team’s rapid convergence on where we’re going and how we’ll get there.
In the span of a few weeks, we’ve:</p>
<ul>
<li>Started this project</li>
<li>Grown to four active developers with diverse concerns, opinions, and skills</li>
<li>Defined our big-picture <a href="https://notqmail.org/goals-and-non-goals">goals (and non-goals)</a></li>
<li>Identified milestones for
<a href="https://notqmail.org/roadmap">future releases</a></li>
<li>Agreed on standards for
<a href="https://notqmail.org/pull-requests">pull requests</a></li>
<li>Merged only the changes that absolutely had to be in the first release</li>
<li>Shipped
<a href="https://github.com/notqmail/notqmail/releases/tag/notqmail-1.07">our first release</a></li>
</ul>
<p>I say “bold” because, for all the ways we intend to hew to qmail tradition, one of our explicit goals is a significant departure.
Back in the day, qmail’s
<a href="https://cr.yp.to/softwarelaw.html">lack of license</a>,
<a href="https://web.archive.org/web/20060127000918/http://cr.yp.to/qmail/dist.html">redistribution restrictions</a>,
<a href="https://cr.yp.to/qmail/var-qmail.html">technical barriers</a>,
and
<a href="http://pyropus.ca/personal/writings/12-steps-to-qmail-list-bliss.html#step9">social norms</a>
made it hard for OS integrators to create packages, and hard for package users to get help.
netqmail 1.06
<a href="https://marc.info/?l=qmail&m=119689105301544&w=2">expressed a desire to change this</a>.
In notqmail 1.07, we’ve
<a href="https://notqmail.org/install">made packaging much easier</a>.
(I’ve already
<a href="https://github.com/NetBSD/pkgsrc/commit/e1bbfdbd03f2607dd43bf91154647b56db9e063f">updated pkgsrc from netqmail to notqmail</a>,
and some of my colleagues have prepared
<a href="https://software.opensuse.org/download.html?project=home%3Anotqmail&package=notqmail">notqmail RPM and .deb packages</a>.)
Further improvements for packagers are part of what’s slated for 1.08.</p>
<h2>What’s next</h2>
<p>Looking much further ahead, another of our <a href="https://notqmail.org/goals-and-non-goals">explicit goals</a> is “Meeting all common needs with OS-provided packages”.
We have a <a href="https://notqmail.org/roadmap">long way to go</a>.
But we couldn’t be off to a better start.</p>
<p>By our design, we believe we’ve made it safe for everyone running qmail to follow notqmail.
We hope you’ll vet our changes carefully, then update your installations to notqmail 1.07.
We hope you’ll start observing us as we continue the work.
We hope you’ll discuss freely on the
<a href="https://cr.yp.to/lists.html#qmail">qmail mailing list</a>.
We hope you’ll be a part of the qmail revival in ways that are comfortable for you.
And we hope that, in the course of time, notqmail will prove to be the community-driven Open Source successor to qmail.</p>
NYCBUG: Maintaining qmail in 2019https://schmonz.com/2019/03/06/nycbug-maintaining-qmail-in-2019/Amitai Schleier2021-11-25T09:54:33Z2019-03-07T02:59:47Z
<p>On Wednesday, March 6,
I attended
<a href="https://www.nycbug.org">New York City BSD User Group</a>
and presented
<a href="https://www.nycbug.org/index?action=view&id=10665">Maintaining qmail in 2019</a>.
This one pairs nicely with my recent DevOpsDays
<a href="https://en.wikipedia.org/wiki/Ignite%5F%28event%29">Ignite</a>
talk about why and how to
<a href="https://schmonz.com/2019/01/25/devopsdays-nyc-run-your-own-email-server/">Run Your @wn Email Server!</a>
That this particular “how” could be explained in 5 minutes is remarkable, if I may say so myself.
In this NYCBUG talk — my first since
<a href="https://schmonz.com/2014/03/05/nycbug-2014-one-weird-trick-to-simplify-package-management/">2014</a>
— I show my work.
It’s a real-world, open-source tale of methodically, incrementally reducing complexity in order to afford added functionality.</p>
<p>My abstract:</p>
<blockquote><p>qmail 1.03 was notoriously bothersome to deploy.
Twenty years later, for common use cases, I’ve finally made it pretty easy.
If you want to try it out, I’ll help!
(Don’t worry, it’s even easier to uninstall.)
Or just listen as I share the sequence of stepwise improvements from then to now — including pkgsrc packaging, new code, and testing on lots of platforms — as well as the reasons I keep finding this project worthwhile.</p></blockquote>
<ul>
<li><a href="https://schmonz.com/2019/03/06/nycbug-maintaining-qmail-in-2019/slides/">Slides</a></li>
</ul>
<p>Here’s the video:</p>
<div class="video-container">
<iframe src="https://www.youtube-nocookie.com/embed/xR776OFqqeA" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</div>
DevOpsDays NYC: Run Your @wn Email Server!https://schmonz.com/2019/01/25/devopsdays-nyc-run-your-own-email-server/Amitai Schleier2019-02-21T16:40:24Z2019-01-25T22:53:41Z
<p>In late January,
I was at
<a href="https://www.devopsdays.org/events/2019-new-york-city">DevOpsDays NYC</a>
in midtown Manhattan
to present
<a href="https://www.devopsdays.org/events/2019-new-york-city/program/amitai-schleier">Run Your @wn Email Server!</a></p>
<p>My abstract:</p>
<blockquote><p>When we’re responsible for production, it can be hard to find room to
learn. That’s why I run my own email server. It’s still “production”
— if it stays down, that’s pretty bad — but I own all the decisions,
take more risks, and have learned lots. And so can you! Come see why
and how to get started.</p>
<p>With one command, install famously secure email software. A couple
more and it’s running. A few more and it’s encrypted. Twiddle your
DNS, watch the mail start coming in, and start feeling responsible for
a production service in a way that web hosting can’t match.</p></blockquote>
<ul>
<li><a href="https://schmonz.com/2019/01/25/devopsdays-nyc-run-your-own-email-server/slides/">Slides</a></li>
</ul>
2018Q4 qmail updates in pkgsrchttps://schmonz.com/2019/01/07/2018q4-qmail-updates-in-pkgsrc/Amitai Schleier2023-06-28T04:40:46Z2019-01-07T18:19:43Z
<p>Happy 2019!
Another three months, another stable branch for
<a href="https://www.pkgsrc.org">pkgsrc, the practical cross-platform Unix package manager</a>.
I’ve shipped quite a few improvements for qmail users in our
<a href="http://mail-index.netbsd.org/pkgsrc-users/2018/12/30/msg027871.html">2018Q4 release</a>.
In three sentences:</p>
<ol>
<li><a href="https://schmonz.com/software/pkgsrc-qmail-run/">qmail-run</a>
gains TLS, SPF, IPv6, SMTP recipient checks, and many other
sensible defaults.</li>
<li>Most qmail-related packages — including the new ones used by
qmail-run — are available on most pkgsrc platforms.</li>
<li><a href="http://pkgsrc.se/pkgtools/rc.d-boot">rc.d-boot</a>
starts <code>rc.conf</code>-enabled pkgsrc services at boot time on many
platforms.</li>
</ol>
<p>In one:</p>
<blockquote><p>It’s probably easy for you to run qmail now.</p></blockquote>
<p>On this basis, at
<a href="https://schmonz.com/talk/2019-devopsdays/">my DevOpsDays NYC talk in a few weeks</a>,
I’ll be recommending that everyone try it.</p>
<h2>Try it</h2>
<script>
window.matchMedia('(prefers-color-scheme: dark)').addEventListener('change', event => {
const newColorScheme = event.matches ? 'dark' : 'light';
const oldColorScheme = event.matches ? 'light' : 'dark';
const players = document.getElementsByClassName('asciinema-player asciinema-theme-solarized-' + oldColorScheme);
while (players.length > 0) {
players[0].className = 'asciinema-player asciinema-theme-solarized-' + newColorScheme;
}
});
const initialColorScheme = (window.matchMedia && window.matchMedia('(prefers-color-scheme: dark)').matches) ? 'dark' : 'light';
</script>
<p>Here’s a demo on CentOS 7, using binary packages:</p>
<div id="centos7"></div>
<script>
AsciinemaPlayer.create('../centos7-pkgsrc-qmail.cast', document.getElementById('centos7'), {
cols: 88,
rows: 42,
autoPlay: false,
preload: false,
loop: false,
startAt: 0,
speed: 1,
theme: 'solarized-' + initialColorScheme,
poster: 'data:text/plain,pkgsrc 2019Q2: qmail on CentOS 7',
fit: false,
});
</script>
<p>The main command I ran:</p>
<pre><code>$ sudo env PKG_RCD_SCRIPTS=yes pkgin -y install qmail-run rc.d-boot
</code></pre>
<p>Here’s another demo on Debian 9, building from source packages:</p>
<div id="debian9"></div>
<script>
AsciinemaPlayer.create('../debian9-pkgsrc-qmail.cast', document.getElementById('debian9'), {
cols: 88,
rows: 42,
autoPlay: false,
preload: false,
loop: false,
startAt: 0,
speed: 1,
theme: 'solarized-' + initialColorScheme,
poster: 'data:text/plain,pkgsrc 2018Q4: qmail on Debian 9',
fit: false,
});
</script>
<p>The commands I ran:</p>
<pre><code>$ cd ...pkgsrc/mail/qmail-run && make PKG_RCD_SCRIPTS=yes install
$ cd ../../pkgtools/rc.d-boot && make PKG_RCD_SCRIPTS=yes install
</code></pre>
<p>These improvements were made possible by
<a href="https://schmonz.com/software/acceptutils/">acceptutils</a>,
my redesigned TLS and SMTP AUTH implementation that obviates the
need for several large and conflicting patches.
Further improvements are expected.</p>
<p>Here’s the full changelog for qmail as packaged in pkgsrc-2018Q4.</p>
<h2>Removed</h2>
<ul>
<li>From <code>mail/mess822</code>:
<ul>
<li>The SMTP AUTH patch.</li>
</ul>
</li>
<li>From <code>mail/qmail</code>:
<ul>
<li>The SMTP AUTH patch.</li>
<li>The <code>qmail-smtpd</code> half of the TLS patch.</li>
<li>The RCPTCHECK patch.</li>
</ul>
</li>
<li>From <code>mail/qmail-run</code>:
<ul>
<li>Dependency on spamdyke.</li>
<li>Dependency on stunnel.</li>
</ul>
</li>
<li>From <code>sysutils/checkpassword</code> and <code>sysutils/checkpassword-pam</code>:
<ul>
<li>The setuid bit.</li>
</ul>
</li>
</ul>
<h2>Updated</h2>
<ul>
<li><code>mail/qmail</code>:
<ul>
<li>To the latest
<a href="https://schmonz.com/software/tlsonlyremote/"><code>qmail-remote</code> half of the TLS patch</a>.</li>
<li>To warn if the queue filesystem is case-insensitive.</li>
</ul>
</li>
<li><code>mail/qmail-rejectutils</code>:
<ul>
<li>To let <code>qmail-rcptcheck</code> run under
<a href="http://qmail-spp.sourceforge.net/">qmail-spp</a>,
so that other RCPTCHECK programs can continue to run unmodified.</li>
<li>To deprecate <code>qmail-qfilter-ofmipd-queue</code> and
<code>qmail-qfilter-smtpd-queue</code> in favor of <code>qmail-qfilter-queue</code>.</li>
</ul>
</li>
<li><code>mail/qmail-run</code>’s defaults:
<ul>
<li>To <code>sslserver</code> (from <code>tcpserver</code>).</li>
<li>To listen on IPv6 when available.</li>
<li>To auto-enable TLS for message submission, incoming SMTP, and
POP3 (as well as remote delivery) when certs are in place.</li>
<li>To tag log entries with <code>nbqmail/send</code>, <code>nbqmail/smtpd</code>, etc.
(inspired by Postfix).</li>
<li>To find tcprules in <code>control/tcprules/*</code> (and auto-migrate from
<code>/etc/qmail/tcp.*</code>).</li>
<li>To rebuild outdated tcprules CDBs on startup.</li>
<li>To delay the SMTP greeting by 2 seconds (a simple anti-spam measure).</li>
<li>To check the <code>zen.spamhaus.org</code> RBL.</li>
<li>To check recipients using qmail’s delivery logic before accepting mail.</li>
<li>To record a <code>Received-SPF:</code> header.</li>
<li>To skip
<a href="https://en.wikipedia.org/wiki/greylisting">greylisting</a>
(if any) when SPF returns “pass”.</li>
<li>To record a <code>Received:</code> header with TLS protocol and ciphers.</li>
<li>To let users
<a href="https://schmonz.com/software/acceptutils/features/">configure their own <code>ofmipd</code> address-rewriting
rules</a>.</li>
</ul>
</li>
</ul>
<h2>Added</h2>
<ul>
<li><a href="http://pkgsrc.se/mail/greylisting-spp"><code>mail/greylisting-spp</code></a>:
<ul>
<li>For greylisting.</li>
</ul>
</li>
<li><a href="http://pkgsrc.se/mail/qmail-spp-spf"><code>mail/qmail-spp-spf</code></a>:
<ul>
<li>For SPF checks.</li>
</ul>
</li>
<li><code>pkgtools/rc.d-boot</code>:
<ul>
<li>For starting pkgsrc-provided services at boot on a variety of systems.</li>
</ul>
</li>
<li>To <code>devel/syncdir</code>:
<ul>
<li>A
<a href="https://schmonz.com/software/syncdirdlsym/"><code>dlsym()</code>-based implementation</a>,
for systems without <code>syscall()</code>.</li>
</ul>
</li>
<li>To <code>mail/qmail</code>:
<ul>
<li>The qmail-spp patch, for flexibly modifying SMTP behavior at runtime.</li>
</ul>
</li>
<li>To <code>mail/qmail-rejectutils</code>:
<ul>
<li>Manual pages.</li>
</ul>
</li>
<li>To <code>mail/qmail-run</code>:
<ul>
<li><code>greylisting-spp-wrapper</code>, for whitelisting recipient addresses or
whole domains, and optionally omitting IP address from
greylisting’s tuples.</li>
</ul>
</li>
<li>To <code>mail/qmail</code> and <code>mail/qmail-run</code>:
<ul>
<li>Cleaner uninstall, so people can feel comfortable trying qmail.</li>
</ul>
</li>
</ul>
Legacy Code Lessons: pymsgauthhttps://schmonz.com/2018/08/07/legacy-code-lessons-pymsgauth/Amitai Schleier2023-04-22T11:55:21Z2018-08-07T15:22:02Z
<h1>What’s pymsgauth?</h1>
<p>Posting to a mailing list is usually, but not always, as simple as sending an email.
I’m on a few lists where
<a href="https://jdebp.uk/FGA/djb-qsecretary.html">an automated “secretary” responds to each post, requiring me to reply</a>
before the post is allowed through.
This cuts down on both spam and ease of participation.</p>
<p>A fellow participant on these lists wrote
<a href="http://pyropus.ca/software/pymsgauth/">pymsgauth</a>
to automatically handle these mailing list confirmation notices.
With pymsgauth configured on my mail server, I can post to the list and be done.
(Opportunities for automation are one big reason
<a href="https://schmonz.com/2017/01/18/qmail-smtp-auth-tls-redux/">why I run my own mail server</a>.)</p>
<h1>Why change the code?</h1>
<p>pymsgauth was written for
<a href="https://en.wikipedia.org/wiki/History%20of%20Python">Python 1.x</a>,
was last updated by its author in 2003, and is unlikely to receive further updates.
Thanks to
<a href="http://python3porting.com/strategies.html">Python’s compatibility strategy</a>,
pymsgauth continues to run well under 2.7.
But it doesn’t run under Python 3 at all.
And
<a href="https://pythonclock.org/">2.7’s days are numbered</a>.</p>
<p>Given the option, I’d rather deal with this well before the clock runs out.
And since I recently wrote some
<a href="https://github.com/schmonz/smtp-strangler">code targeting both Python 2.7 and 3</a>,
now seemed like a good time for my brain to do the same for pymsgauth.</p>
<p>In order for it to modify the right outbound messages and respond to the right inbound ones, all my email flows through pymsgauth.
Since my email is important to me — did I mention
<a href="https://schmonz.com/2017/03/27/automation-for-mail-hosting/">going to some effort to run my own server</a>? — pymsgauth’s reliability is important to me.</p>
<p>As usual with legacy code, my goal here was to obtain the desired change in observable behavior with just enough safety, just enough changed code, and just enough learning along the way.
No more than necessary.</p>
<h1>What needed to change?</h1>
<p>Zeroth, I created a git repository and
<a href="https://github.com/schmonz/pymsgauth/tree/pymsgauth-2.1.0">tagged the last release of pymsgauth</a>.</p>
<p>First, I ran
<a href="http://python3porting.com/2to3.html">2to3</a>.
It made
<a href="https://github.com/schmonz/pymsgauth/commit/7706821c30999c448d0a494b0184ede455f933ef">many of the obvious changes</a>,
such as exception syntax, explicit calls to <code>list()</code>, and replacing <div class="highlight-python"><pre class="hl"><span class="hl kwa">if</span> mydict<span class="hl opt">.</span><span class="hl kwd">has_key</span><span class="hl opt">(</span>mykey<span class="hl opt">)</span>
</pre></div> with <div class="highlight-python"><pre class="hl"><span class="hl kwa">if</span> mykey <span class="hl kwa">in</span> mydict
</pre></div>
(It missed a few things I assumed it’d catch, as I discovered later!)</p>
<p>Then I entered the try-see-fix cycle: try the code under Python 3, see the next error, fix it.</p>
<p>The first errors came from the config file parser.
It was subclassing <code>UserDict</code>
(<a href="https://docs.python.org/2/library/userdict.html">part of Python 2’s standard library</a>)
and the <code>import UserDict</code> was failing.
I guessed this meant <code>UserDict</code> was gone in Python 3 and I’d have to learn enough about its behavior to subclass Python’s <code>dict</code> instead.
Luckily, I guessed wrong.
<code>UserDict</code> merely
<a href="https://docs.python.org/3/library/collections.html#collections.UserDict">moved</a>.
I tweaked the imports, the type instance-check syntax, and (almost all) the string method calls, and
<a href="https://github.com/schmonz/pymsgauth/commit/0ef4491296bbe78628ed3ca75447e8307fab0688">stopped getting errors from <code>ConfParser.py</code></a>.</p>
<p>The next batch of errors came from <code>pymsgauth.py</code> itself.
It was failing to <code>import rfc822</code>.
This time, unluckily, I was right:
<a href="https://docs.python.org/2/library/rfc822.html">that module</a> was gone, replaced in Python 3 by something called
<a href="https://docs.python.org/3/library/email.html#module-email"><code>email</code></a>.</p>
<h1>How to switch from <code>rfc822</code> to <code>email</code>?</h1>
<p>The interfaces were different.
Among other things, <code>rfc822</code> had a file pointer we were using to read a line at a time.
<code>email</code> doesn’t.</p>
<p>I found a
<a href="https://github.com/MarkNenadov/rfc822py3">Python 3 port of <code>rfc822</code></a>.
Maybe I could bundle it with pymsgauth, or add an external dependency.
It looked experimental, though.
Clearly better to avoid if possible.</p>
<p>So I searched for all instances of <code>rfc822.Message</code>, and all methods that were being called on them, and (under Python 2.7) wrote
<a href="https://en.wikipedia.org/wiki/Characterization%20test">tests to characterize pymsgauth’s expectations</a>.</p>
<p>Then I
<a href="https://en.wikipedia.org/wiki/Adapter%20pattern">extracted an adapter class</a>
<code>RFC822Message</code> (right there in the test file) with all the same methods, trivially delegating to <code>rfc822</code>.</p>
<p>Since <code>email</code> is already available under Python 2.7, I replaced <code>import rfc822</code> with <code>import email</code> and figured out how to
<a href="https://github.com/schmonz/pymsgauth/commit/53326de945f0a51197b1212e60d54914c0a74dd0">make the tests pass with the new delegate</a>.</p>
<p>Tests passed equally well with Python 3, modulo a few warnings (fixed).
So I
<a href="https://github.com/schmonz/pymsgauth/commit/2560a0f665fa813647f6bd3ebe1d1559558a4b40">moved my <code>RFC822Message</code> class</a>
out of the test file and into <code>pymsgauth.py</code>, where I replaced all three instances of <code>rfc822.Message</code> with <code>RFC822Message</code>.</p>
<p>Not bad.</p>
<h1>Done yet?</h1>
<p>Nope, on to the next error.
Can’t <code>import popen2</code>.
I bounced confusedly around the docs and eventually understood the one-line change to replace <code>popen2.Popen3()</code> with <code>subprocess.Popen()</code>.</p>
<p>While there, I had a false start.
Based on my recent experience developing an 8-bit-clean SMTP proxy, I thought I’d want to open streams as binary and use Python’s <code>b'this is a sequence of bytes'</code>.
But that started looking like too much learning and changing the code.
pymsgauth had always worked well enough with Unicode as it was.
I reverted to the previous behavior, in a way that worked across Python 2.7 and 3, by adding this at the top:</p>
<div class="highlight-python"><pre class="hl"><span class="hl kwa">if</span> sys<span class="hl opt">.</span>version_info<span class="hl opt">[</span><span class="hl num">0</span><span class="hl opt">] <</span> <span class="hl num">3</span><span class="hl opt">:</span>
<span class="hl kwa">import</span> codecs
sys<span class="hl opt">.</span>stdin <span class="hl opt">=</span> codecs<span class="hl opt">.</span><span class="hl kwd">getreader</span><span class="hl opt">(</span><span class="hl sng">'utf-8'</span><span class="hl opt">)(</span>sys<span class="hl opt">.</span>stdin<span class="hl opt">)</span>
</pre></div>
<p>Next: can’t <code>import sha</code>.
That one was easy.
2.7 and 3 both have <code>hashlib</code> and it has a <code>sha1()</code>.</p>
<p>Before running on my server, I manually tested 3 of the 4 programs: <code>pymsgauth-mail</code>, <code>pymsgauth-filter</code>, and <code>pymsgauth-clean</code>.
They ran under both Python 2.7 and 3, with no apparent errors, and with the observable behavior I expected.
And I sort of tested <code>pymsgauth-confirm</code>, but to be really sure, it would need to run on my real mail server and do its real work.</p>
<h1>How about now?</h1>
<p>Almost.</p>
<p>I tried announcing my new patch to one of the mailing lists that issues these confirmation notices.
The notice showed up in my inbox, for the first time in a while.
This was disconcerting, but ideal: I had failed to announce a patch that was evidently not quite working.</p>
<p>With verbose logging, I saw I’d missed converting a few static string methods to object methods.
Fixed.</p>
<p>I tried sending my announcement again and watched the logs.
<code>pymsgauth-filter</code> added the magic token to the headers, <code>pymsgauth-confirm</code> found it and auto-replied, and the only message that appeared in my inbox was the one I had sent to the mailing list.</p>
<h1>Today’s Legacy Code Lessons</h1>
<p>I could have interpreted
<a href="https://en.wikipedia.org/wiki/YAGNI">YAGNI</a>
to mean “wait until you’re having a problem”.
Maybe Python 2.7 will get a stay of execution, such that other people will have more time to consider fixing it themselves.
Or maybe, compared to the other things I need to get done, I can’t prioritize this one right now.</p>
<p>As it happens, I often have free time in the mornings, and
<a href="https://twitter.com/schmonz/status/1025075087911936000">mitigating a risk is my favorite use of slack time</a>.</p>
<p>If my goal had been zero change in observable behavior, then the way I accomplished it was terribly wasteful.
I could have changed zero code.
My goal was zero change to observable behavior <em>later</em>, despite a known source of impending change.
(Or to start making other plans well in advance, if my goal had proved prohibitively expensive.)</p>
<p>The Adapter pattern isolates dependencies.
I use it pretty eagerly whenever I
<a href="https://schmonz.com/2017/03/08/how-to-manage-dependencies/">take on a new dependency</a>.
It’s also extremely useful for minimizing the impact of replacing an existing one.</p>
<p>Developing a class directly in the test file comes from
<a href="https://cumulative-hypotheses.org/2011/08/30/tdd-as-if-you-meant-it/">TDD As If You Meant It</a>,
where we wait for application code to really need our new class before making it available in its own file.
This wasn’t TDD — I wasn’t seeking design feedback, one test at a time — but I knew what I wanted our application to need from our new class.
Flipping from file to file would have slowed me down.
So I avoided it.</p>
<p>In the end:</p>
<ul>
<li>I didn’t have to understand much of pymsgauth’s code</li>
<li>I didn’t have to change much of it, either</li>
<li>I got fairly safely and cheaply where I needed to go</li>
</ul>
<p>Here’s
<a href="https://schmonz.com/software/pymsgauthfilter/">my patch to pymsgauth</a>.</p>
<p>Until next time, I hereby declare Legacy Code Success!</p>
pkgsrcCon 2018: Maintaining qmail in 2018https://schmonz.com/2018/07/07/pkgsrccon-2018-maintaining-qmail-in-2018/Amitai Schleier2023-04-21T21:04:25Z2018-07-07T12:17:00Z
<p><a href="http://pkgsrc.org/pkgsrcCon/2018/">pkgsrcCon</a>
is an annual conference about the challenges of cross-platform open-source package management, focusing on the development and usability of
<a href="http://pkgsrc.org">pkgsrc</a>.
I’ve gone to something like half the pkgsrcCons since its inception in 2004, even though they’re always in Europe and I’m usually not.
Conveniently, this summer I’m mostly in Germany and pkgsrcCon was entirely in Berlin.
Just a train ride away.</p>
<p>I had to cut my attendance very short, so I’m glad I got to see some familiar and new-to-me faces, eager to see the remaining talks when the videos are published, and pleased to have been able to share the continuing story of my long-running and perhaps misplaced
<a href="https://schmonz.com/2017/03/27/automation-for-mail-hosting/">efforts to improve our packaging</a>
of
<a href="https://schmonz.com/software/">qmail</a>.</p>
<ul>
<li><a href="https://schmonz.com/2018/07/07/pkgsrccon-2018-maintaining-qmail-in-2018/slides/">Slides</a></li>
<li><a href="https://video.wiedi.hk/pkgsrcCon-2018/pkgsrcCon-2018-05-schmonz-qmail.mp4">Video</a></li>
</ul>