Previously
This is the sixth in a series about TDD’s value to humans, following
- Keeping my job, in which I lost my manager’s trust,
- A last-minute feature, in which I regained some,
- Deadline pressure, in which I earned quite a bit more than I’d started with,
- Continual delivery, in which the circle of trust grew to include our Operations team, and
- Fail-safe, in which we were always deploying soon
Cosmic justice
TDD had been helping us build things right. By enabling our new monthly delivery cadence, it had just begun helping us make more impactful choices about what to build. So by some conservation law or other, that’s precisely when a large, non-optional project landed atop our backlog with a hard deadline that was possibly impossible.
Why us?
Our first reaction was to inquire why this project, which was not a straightforward and harmonious logical extension of our product’s capabilities, had been assigned to our team. The response, reasonably, was that the other options under consideration were less straightforward and harmonious. (Unstated, but probably also a factor, was that we had proven ourselves an effective and reliable team.)
Why not us?
Our second reaction was to quickly determine whether the deadline was obviously impossible. To deliver the project outcome required that our storage backend scale to several orders of magnitude more data, along with a probable two- or threefold increase in concurrent network access. That these were achievable was hardly given: our storage backend was SQLite, not designed for this usage pattern. If we couldn’t deliver the outcome without switching backends, then we’d know up front we couldn’t hit the deadline, and we’d be able to give the business almost all the time on the clock to pursue other options. So we spiked it. The result: it’d be mostly fine, and for several smaller bottlenecks there were effective actions we’d be able to take.
Test-driven planning
Our third reaction was to slowly and painstakingly make arrangements to find out ASAP along the way if we ever slipped into danger of missing the deadline. Our product planning and prioritizing had been haphazard and unstructured, but that hadn’t quite become our biggest problem. Now it clearly was. My teammate Nathan walked us through the reasoning:
- We have a hard deadline in eight months.
- We’ll be working on this project, and only this project, until it’s done.
- We need early agreement from stakeholders that this is what they want, because we can’t afford to waste effort.
- We need early agreement from the upstream DBA team to work to our schedule, because we can’t afford to be late.
”And that’s why we’re going to make a project backlog,” said Nathan — an ordered list of the fewest possible stories that’d get us to the project outcome, with the simplest possible estimates attached to each story, so that we could do the simplest possible math to get from the deadline to the velocity we’d need. “Then we’ll start completing stories, observe the velocity we have, and decide whether that’s okay. It might be okay. We’ll find out soon enough.”
The first time we checked, a few weeks in, our velocity was fine. A couple months later, still fine.
Stick to the plan
Meanwhile, we’d been continuing with our usual monthly production deployments, even though the expected business value most months was low, because the expected risk mitigation was high. We were regularly making larger-than-usual internal changes, and it helped everyone involved in the project to see pretty often that we still hadn’t broken anything.
Extending our identity-management system to manage this archaic database system’s not-exactly-identities, with their very different behaviors and business rules, necessitated a significant set of refactorings (afforded by our previous commitment to TDD, of course) and a new network service, for which we designed the API we needed from the DBA team and provided them a matching set of red tests. Then we mocked the API and test-drove around them. When they came back rather later asking whether they could provide a different API, we said no, with two whole legs to stand on:
- See our test results here? With the API as previously defined, our system is ready.
- See our project burndown here? With the deadline as previously defined, our system had better be ready.
Our discussion turned promptly to the red API tests we’d provided and how we could help them get to green. Which they eventually, not a moment too soon, did.
Change the plan
The third time we checked our velocity, thanks in part to the API kerfuffle, it had dropped a bunch. Not fine. So in the month leading up to the deadline, we made two adjustments:
- We de-scoped the last few weeks of stories (which everyone had agreed early on would be okay to leave out).
- We gritted our teeth and temporarily tolerated the API having been implemented in production only.
If we ever wanted to change it or the code that consumed it, we’d want a parallel version in non-prod. But DBA was having enough trouble meeting their commitment to us. We came up with a plan to safely end-to-end-test in production during weekend work windows, with the new capabilities turned off for all users except us. It took two Friday-into-Saturday all-nighters, but we got it tested well enough.
Mission accomplished
On the Friday of the deadline, we completed the release into production for all users, fulfilling the purpose of the project, with several whole minutes to spare before midnight.
After midnight and a bit of Nathan’s favorite single malt, we wrote down a backlog of all the corners that had been cut along the way and needed repairing, starting with getting our hands on a non-production instance of DBA’s new API, pronto. We sent the list to our management asking for their backing. Despite repeated pleas, nothing ever came of it — possibly because they wound up leaving the company within the year. And that’s how it came to pass that we wound up running some production code we couldn’t confidently change.
While it was disappointing that our management was unwilling to support or match our level of discipline, we’re still awfully glad we did it our way — and had they cared to notice, they might have been glad too.
Because of our commitment to TDD…
Our domain knowledge was easily shared among new team members. Because of our shared knowledge, we thought of all the right stories to include and defined them well.
Our development flow had become fairly smooth by the time this project landed on us. Because our flow was generally smooth, point estimates led us to useful predictions.
Every completed story was a shippable increment. Because we had the freedom to ship every month, we exercised that freedom to buy production-quality certainty in our progress.
Our just-enough-process plan actually worked. We made meaningful hypotheses and drew informed conclusions that enabled us to adjust the wheel, stay on schedule, and deliver an 8-month project with confidence.
Conclusion
We continued to expand the scope and impact of our disciplined way of working. Nathan had planted in our team the seed of Scrum. No standups, no iterations, no retros; nothing more than an ordered backlog with points. Just enough to solve the problem at hand.
Given how the project went, you can have exactly one guess whether that seed grew.