In my previous post I gave an example of the benefits of fixing a bug by first writing a test. Even if my example had actually shown such a test, bugfixing isn't the archetypal TDD scenario. Most people — including me — primarily think of Test-Driven Development as a way to develop new features.

Some context

Ikiwiki isn't only for wikis. It's a fairly general-purpose web content management system.

In ikiwiki, any collection of pages can be turned into a blog. What makes a typical blog? Posts sorted by date, newest first, with a feed. People using feed-reading software can subscribe to the feed and receive new posts automatically.

A podcast is a blog that contains media enclosures. Since it's a blog, any feedreader can display it; since it's a special kind of blog, a specialized kind of feedreader can download and manage enclosures. The best-known podcatcher is probably iTunes.

In ikiwiki, any blog post that can't be transformed into HTML — say, an MP3 — is published as is. Browsers see it as a link, feedreaders see it as an enclosure, and podcatchers can automatically download it.

Ikiwiki's support for publishing podcasts was pretty cool. I needed it to be cooler.

The feature

This website is mostly just a blog, but also a podcast. I'd really like to be publishing it with ikiwiki.

Most podcasts (mine included) accompany every episode with descriptive “show notes”. Ikiwiki could only produce blog posts that were either text or media. I wanted it to be able to produce posts with both. In order to migrate smoothly, I also wanted to reach feature parity with tru_podcast.

The design

To post a new podcast episode in Textpattern with tru_podcast, after some one-time setup, I write an article, upload an MP3, and give the article a particular kind of reference to the MP3. I find the workflow sensible and comfortable, modulo being forced to write in the browser, upload a large file out of band, and obtain Textpattern's unique identifier for the file.

Ikiwiki has a [[!meta ]] directive for defining metadata about a page. If I were to invent an enclosure parameter that let me write [[!meta enclosure="WikiLink/to/media.mp3"]] in a podcast post, ikiwiki would have enough information to produce my desired output from much simpler input.

The implementation

Compatibility

Before I went and changed anything, I needed to decide what should happen with ikiwiki's existing podcast behavior.

  • Did my new idea replace the old? No, it wasn't a strict superset.
  • Could I break the old way a little if I needed to? Preferably not, because people (including me) were using it.
  • Would I need to break the old way? Glancing at the code, I didn't think so.
  • Would I able to tell for sure whether I'd broken it? Not easily enough.

A safety net behind me

My first step, therefore, was to write automated tests to cover the existing behavior.

It's common and useful to think of these as regression tests, but when I'm adding tests for code that didn't have any — especially someone else's code — I prefer to think of them as characterization tests. I didn't know enough to decide whether the existing behavior was right or wrong, I just wanted to be sure to find out if I unintentionally changed it. With my simple_podcast() characterization tests in place, I felt safe about working in this area of the ikiwiki code.

Throw a bit more net in front, take another step

What behaviors would my new implementation demonstrate when complete? Ikiwiki would need to render pages containing the new [[!meta enclosure]] parameter…

  • In browsers: as normal, plus an appended link to the media file
  • In feeds: with the text as the entry's content and the media file as its enclosure

To figure out whether I was even on the right track, I wrote tests for a single HTML page invoking my directive-to-be, then made the smallest possible changes to the meta plugin and to page.tmpl that caused the new tests to pass. When single_page_html() turned green — and simple_podcast() didn't turn red — I gained confidence in my understanding of ikiwiki and in my ability to take it where I wanted to go. A small first step, but it felt big.

Iterate: generate internal links correctly

My expansion of the new enclosure parameter was simplistic. I extended the tests to inspect the generated link, then expanded the parameter's value as a WikiLink adhering to the LinkingRules.

Iterate: add documentative tests

I wasn't quite ready to work on feeds yet, but I suddenly remembered that a feed entry can have no more than one enclosure, and I wanted to document what would happen if a page tried to define more than one. No new code, just tests.

Iterate: render links in inlined HTML

Ikiwiki generates blogs (and podcasts) with the [[!inline ]] directive, which combines multiple input pages into one output page. I wrote tests to inline a few pages containing my new parameter and look for the generated links. The tests failed. I figured out that it was because ikiwiki uses a separate template for inlined pages. Tweaking inlinepage.tmpl made the new inlined_pages_html() tests pass. The other tests continued to pass.

Iterate: always generate absolute links

In all the popular podcast feeds I'd inspected, enclosure URLs were always fully absolute, even when they could be expressed concisely as relative. Apple's example did likewise. So I wrote failing tests expressing this requirement, and adjusted the code to make them pass.

Iterate: extract method

At this point, my code was generating the right links and displaying them in the browser. It was time to think about feeds. I could see that I was about to need to call ikiwiki's enclosure-generating code two different ways, so I extracted it to a subroutine, relying on my characterization tests to prove that I hadn't changed simple podcasts in any way.

Iterate: render enclosures in feeds

The quickest way to find out whether I could now generate fancy podcast feeds was to munge a copy of the simple_podcast() tests into fancy_podcast(). To make the new tests pass, I taught [[!inline ]] to look for the [[!meta enclosure]] parameter and pass it to the enclosure-generation subroutine I'd just extracted.

But the characterization tests had started failing! By populating <content> with the entry's text for fancy podcasts, I had unintentionally added an empty but non-null <content> to simple podcasts. Remember, the existing behavior was that a feed entry could be either text content or an enclosure, not both. Now there were both. I had changed behavior I hadn't meant to change. My characterization tests had done their job and notified me. I made the conservative choice and adjusted the code and {atom,rss}item.tmpl to preserve the existing behavior precisely.

Iterate: style enclosures like content

Ikiwiki includes a default style sheet and some themes. They needed to know how to style the new enclosure section. The changes were simple enough to test manually. (As I was writing this article, James Shore tweeted promisingly about CSS testing.)

Iterate: match RSS and Atom item metadata

Previewing fancy podcasts on my phone, I noticed that ikiwiki's RSS and Atom feeds were slightly inconsistent. The RSS version of a podcast episode wasted valuable title space displaying the author first, when there's a perfectly good author field for that; the Atom version didn't. I corrected rssitem.tmpl.

Iterate: extract test method

En route to fancy podcasts, I had made copypasta of my tests. The simple and fancy tests were different, yet similar. I extracted the common assertions and parameterized the differences. No new code, no new tests, just refactored tests.

Iterate: match RSS and Atom feed metadata

I noticed that RSS feeds were missing some fields found in Atom feeds. I filled in the blanks in rsspage.tmpl and checked the result in a feed validator.

Iterate: manually test migration to fancy podcasts

Fairly confident that the pieces were in place, I wanted to see what it would be like to fancy up a simple podcast, and to move a podcast from another system (such as Textpattern) into ikiwiki. So I did. Both worked as I expected.

The review

I felt pretty good about my code: I knew it did good new things, I didn't think it did any bad new things, and at any rate it wasn't going to be a waste of anyone's time to look over my work. So I asked for review, and Joey provided it.

Feedback

There were a handful of small tweaks I needed to make:

And there was one big one: I broke planets. A “planet” is a site whose job it is to subscribe to a bunch of individual blogs and republish all of them as one big community blog. My changes to how ikiwiki makes podcasts (which are blogs) wound up having bad side effects on how ikiwiki makes planets (which are blogs). I fixed it by consistently applying the following rule to both RSS and Atom: if the blog's name and author are different, show both.

There weren't tests to catch this, so it's a good thing Joey spotted it. I wasn't expecting to touch the planet code, but I'm happy I got a chance to improve it a bit.

Pictured

  • A simple design
  • Tiny iterations
  • A careful implementation, driven by tests
  • One mistake the tests prevented me from making
  • Another mistake no tests prevented me from making
  • Some reasons why the discipline of TDD is valuable

Not pictured

  • Many other mistakes — in both design and implementation — the tests prevented me from making
  • The speed of the test-driven development cycle
  • The knowledge that, like simple podcasts, fancy podcasts won't get broken later by mistake

Significance

My code has been merged upstream and will be in the next ikiwiki release. My tests were good enough to shape and prove out my idea, but some months later, I no longer fully understand them. Test code is just code, after all, subject to the same challenges of clearly expressing intent — and if tests' intent isn't clear now, they're not good tests now.

When I'm ready for my next round of ikiwiki podcast enhancements, I'll start by refactoring my test code until it's meaningful to me again. Then I'll know what I know, and iterate, and iterate.