Water of Leith

This week, the Octoberth of August, was my first ever visit to Edinburgh. It was also the first of my tour stops to be scheduled. I’d met Barney Dellar a few months ago at the Mob Programming Conference. When I announced my coding tour, and he realized the best week for his team would be during the height of the Edinburgh Festival Fringe, he moved quickly and decisively to book my lodging and travel. I was grateful to know well in advance that I’d get to visit at least one company — Canon Medical Research Europe — and that as such my tour would, in fact, exist.

Monday

  • Joined a mob (2.5-minute rotations!)!
  • My worst-facilitated Gilded Rose ever: OS+keyboard mismatch, AND language (C++ version wouldn’t build in IDE beforehand, so… Java)
  • Took in Barney’s talk at C++ Edinburgh
  • Ate irresponsibly
  • Walked lots

On the way back from Barney's talk

Tuesday

Learning Hour: Gilded Rose kata

Wednesday

Thursday

Learning Hour: Intro to TDD

  • Paired with junior developer (with extremely Scottish name)
  • Test-Driven Development intro: 3 slicing options, then Roman numerals
  • River walk with tech lead concerned with how new hires learn
  • Dependency setup hell
  • Nap and dinner chez Barney, beach walk and 🍻

Friday

  • Mob 2 refactoring in smaller steps — 3 commits in an hour!
  • Use whiteboard to write observations down and stay focused
  • Learning Hour: “Strangle Your Legacy Code”
  • Tech lead’s exercise: practice code design skills
  • Review the week

Here are all #CodingTour tweets from the week.

Reflections

After my visit here, which needed to be planned well in advance, I’m also appreciating the corresponding just-in-time planning that made sense during the week. Rather than choosing all five Learning Hours up front, each morning we selected what sounded best to us that day.

Another useful contrast with my previous tour stops: this week I programmed with a total of two teams. Mostly with Barney’s team, and also twice with another that’s newer to mobbing. I found it easy and comfortable to get to spend more time with fewer people, to get to know each other better, and to accumulate more solved problems together. If I were staying longer, I’d really start to feel part of the team.

Neither team followed a strict Driver-Navigator rule, and in neither case did it seem important to ask about it or otherwise draw attention to it, because there was never much crosstalk or persistent disagreement about where to navigate. Barney’s team rotated especially smoothly and frequently (every 150 seconds!), pausing after each Mobodoro to retrospect and take a short break. I’m a fan of extremely short, on-the-spot retros whenever we notice something. Now that I’ve seen Mobodoros up close, I’ll likely adopt this practice as well.

A bunch of us got together Friday afternoon to share some of our highlights from the week. Have a look.

Posted August 17, 2018 at 12:46:40 PM EDT Tags:

On Wednesday, August 15 in Newcastle, England, I attended NEBytes and presented Strangle Your Legacy Code. As a mob, we added features to an old SMTP server without modifying its code.

Derek Graham (who kindly invited me and orchestrated my appearance) also has some notes and photos of the event.

Posted August 15, 2018 at 05:10:00 PM EDT Tags:

On Tuesday, August 14 in Edinburgh, I attended Lean Agile Edinburgh and presented Programming for Non-Programmers. As a mob, we did TDD For Non-Programmers (not a coding exercise!) followed by programming FizzBuzz in Python at Cyber-Dojo.

The event was livestreamed. Here’s the recording, skipping ahead to where the audio got good:

Posted August 14, 2018 at 06:32:13 PM EDT Tags:

What’s pymsgauth?

Posting to a mailing list is usually, but not always, as simple as sending an email. I’m on a few lists where an automated “secretary” responds to each post, requiring me to reply before the post is allowed through. This cuts down on both spam and ease of participation.

A fellow participant on these lists wrote pymsgauth to automatically handle these mailing list confirmation notices. With pymsgauth configured on my mail server, I can post to the list and be done. (Opportunities for automation are one big reason why I run my own mail server.)

Why change the code?

pymsgauth was written for Python 1.x, was last updated by its author in 2003, and is unlikely to receive further updates. Thanks to Python’s compatibility strategy, pymsgauth continues to run well under 2.7. But it doesn’t run under Python 3 at all. And 2.7’s days are numbered.

Given the option, I’d rather deal with this well before the clock runs out. And since I recently wrote some code targeting both Python 2.7 and 3, now seemed like a good time for my brain to do the same for pymsgauth.

In order for it to modify the right outbound messages and respond to the right inbound ones, all my email flows through pymsgauth. Since my email is important to me — did I mention going to some effort to run my own server? — pymsgauth’s reliability is important to me.

As usual with legacy code, my goal here was to obtain the desired change in observable behavior with just enough safety, just enough changed code, and just enough learning along the way. No more than necessary.

What needed to change?

Zeroth, I created a git repository and tagged the last release of pymsgauth.

First, I ran 2to3. It made many of the obvious changes, such as exception syntax, explicit calls to list(), and replacing

if mydict.has_key(mykey)
with
if mykey in mydict
(It missed a few things I assumed it’d catch, as I discovered later!)

Then I entered the try-see-fix cycle: try the code under Python 3, see the next error, fix it.

The first errors came from the config file parser. It was subclassing UserDict (part of Python 2’s standard library) and the import UserDict was failing. I guessed this meant UserDict was gone in Python 3 and I’d have to learn enough about its behavior to subclass Python’s dict instead. Luckily, I guessed wrong. UserDict merely moved. I tweaked the imports, the type instance-check syntax, and (almost all) the string method calls, and stopped getting errors from ConfParser.py.

The next batch of errors came from pymsgauth.py itself. It was failing to import rfc822. This time, unluckily, I was right: that module was gone, replaced in Python 3 by something called email.

How to switch from rfc822 to email?

The interfaces were different. Among other things, rfc822 had a file pointer we were using to read a line at a time. email doesn’t.

I found a Python 3 port of rfc822. Maybe I could bundle it with pymsgauth, or add an external dependency. It looked experimental, though. Clearly better to avoid if possible.

So I searched for all instances of rfc822.Message, and all methods that were being called on them, and (under Python 2.7) wrote tests to characterize pymsgauth’s expectations.

Then I extracted an adapter class RFC822Message (right there in the test file) with all the same methods, trivially delegating to rfc822.

Since email is already available under Python 2.7, I replaced import rfc822 with import email and figured out how to make the tests pass with the new delegate.

Tests passed equally well with Python 3, modulo a few warnings (fixed). So I moved my RFC822Message class out of the test file and into pymsgauth.py, where I replaced all three instances of rfc822.Message with RFC822Message.

Not bad.

Done yet?

Nope, on to the next error. Can’t import popen2. I bounced confusedly around the docs and eventually understood the one-line change to replace popen2.Popen3() with subprocess.Popen().

While there, I had a false start. Based on my recent experience developing an 8-bit-clean SMTP proxy, I thought I’d want to open streams as binary and use Python’s b'this is a sequence of bytes'. But that started looking like too much learning and changing the code. pymsgauth had always worked well enough with Unicode as it was. I reverted to the previous behavior, in a way that worked across Python 2.7 and 3, by adding this at the top:

if sys.version_info[0] < 3:
    import codecs
        sys.stdin = codecs.getreader('utf-8')(sys.stdin)

Next: can’t import sha. That one was easy. 2.7 and 3 both have hashlib and it has a sha1().

Before running on my server, I manually tested 3 of the 4 programs: pymsgauth-mail, pymsgauth-filter, and pymsgauth-clean. They ran under both Python 2.7 and 3, with no apparent errors, and with the observable behavior I expected. And I sort of tested pymsgauth-confirm, but to be really sure, it would need to run on my real mail server and do its real work.

How about now?

Almost.

I tried announcing my new patch to one of the mailing lists that issues these confirmation notices. The notice showed up in my inbox, for the first time in a while. This was disconcerting, but ideal: I had failed to announce a patch that was evidently not quite working.

With verbose logging, I saw I’d missed converting a few static string methods to object methods. Fixed.

I tried sending my announcement again and watched the logs. pymsgauth-filter added the magic token to the headers, pymsgauth-confirm found it and auto-replied, and the only message that appeared in my inbox was the one I had sent to the mailing list.

Today’s Legacy Code Lessons

I could have interpreted YAGNI to mean “wait until you’re having a problem”. Maybe Python 2.7 will get a stay of execution, such that other people will have more time to consider fixing it themselves. Or maybe, compared to the other things I need to get done, I can’t prioritize this one right now.

As it happens, I often have free time in the mornings, and mitigating a risk is my favorite use of slack time.

If my goal had been zero change in observable behavior, then the way I accomplished it was terribly wasteful. I could have changed zero code. My goal was zero change to observable behavior later, despite a known source of impending change. (Or to start making other plans well in advance, if my goal had proved prohibitively expensive.)

The Adapter pattern isolates dependencies. I use it pretty eagerly whenever I take on a new dependency. It’s also extremely useful for minimizing the impact of replacing an existing one.

Developing a class directly in the test file comes from TDD As If You Meant It, where we wait for application code to really need our new class before making it available in its own file. This wasn’t TDD — I wasn’t seeking design feedback, one test at a time — but I knew what I wanted our application to need from our new class. Flipping from file to file would have slowed me down. So I avoided it.

In the end:

  • I didn’t have to understand much of pymsgauth’s code
  • I didn’t have to change much of it, either
  • I got fairly safely and cheaply where I needed to go

Here’s my patch to pymsgauth.

Until next time, I hereby declare Legacy Code Success!

Posted August 7, 2018 at 11:22:02 AM EDT Tags:

[Update: Alex wrote a few words about our week together. They’re quite nice.]

After a week visiting BREDEX in Braunschweig, I’m struck by just how many teams I got to meet — nearly all of whom I was lucky enough to introduce to mob programming. Alex Schladebeck had prepared everyone to try it, scheduled 3 teams per day into 2-hour timeslots, and pre-selected several of the week’s Learning Hours. Pretty much all I had to do was hang out in the big meeting space, keep an eye on the schedule, and facilitate. When Learning Hour ended, some folks would take me to lunch.

First mob on Monday morning

Monday

  • 3 teams new to Mob Programming (4-minute rotations)
  • Non-programmers in each mob
  • 3 GUIs: 2 ~directly from SQL, one with more Java thinking
  • ”Care and Feeding of T-Shaped People”
  • True fact: “Thought it’d take 2 days”, nearly done in 2 hours

Tuesday

  • Python, Strangler, etc. w/Alex
  • ”Gilded Rose” (big group)
  • Team’s first mob session is also their first Test-Driven Development
  • Team builds a feature iteratively: first client UI, then message bus, then server, then database
  • More non-coders trying Driver role

Learning Hour: Gilded Rose kata

Wednesday

  • Team (new mob) brought egg timer! Idea: don’t splain me the feature, just start. I got it. My egg timer skills, tho
  • ”Intro to TDD” (Roman Numerals)
  • Team (new mob) ran into sticky design choice. Externalized our concerns as words on whiteboard

Thursday

  • Arrived late, team already mobbing (💪)
  • Got navigated in German, did OK despite tricky Denglisch identifiers
  • ”Mob Programming for Non-Programmers”
  • Flaky test failed alone, passed from suite. Binary-searched for the dependency. Got fast test working.

Friday

  • 7am gym w/Alex
  • Finished UI ding from Day 1
  • Got navigated auf Deutsch, ganz gut gemacht
  • ”Bring Your Whole Self To Work”
  • Useful feedback on “Mob Programming for Non-Programmers”
  • Fixed Jubula bug
  • Heimwärts!

Here are all #codingtour tweets from the week.

Reflections

After last week’s remarkable mix of technologies, it was relaxing in a way to have a week where some teams used Eclipse, others used IntelliJ, and the public meetup I attended was the local Java User Group. After my presentation there, several of us entered into a lively and worthwhile discussion about diversity in our field, including arguments for and against Codes of Conduct. This is the first time my habit of always asking about a CoC when I speak led to some conversation (at least one with me included). I’m happy for this result and hopeful I can get it to happen again.

Doing so much mobbing with so many teams reminded me of the importance of having a flipchart or whiteboard:

”We notice things as we go. Simply writing them down, where everyone can see, frees us to finish one idea at a time.”

Alex (Head of Testconsulting) and Zeb (developer) joined me to share some of our highlights from the week. Have a look.

Posted July 20, 2018 at 10:13:10 AM EDT Tags: