When solving a problem, we often take advantage of known solutions. For sufficiently small and repeatable problems, we can buy solutions at the store. Usually they don’t solve the entire problem all by themselves. Bleach doesn’t clean the toilet; I do, with the help of bleach and other tools.
In software problem-solving parlance, bleach is a “dependency”. It doesn’t have to be. I’m free to try to solve my problem using some other product. But for as long as I believe bleach is what I need — maybe because it gets me there sooner and more reliably than anything else I can think of — then I’m depending on it.
I’m also free to try to solve my problem using no products whatsoever. It might sound like an unqualified bad decision to get in there with my fingernails. But what if bleach and scrub brushes haven’t been invented yet? What if they have, but the store can’t reliably stock them? What if they can, but I can’t reliably get to the store or pay for it? Or what if cleaning toilets is my core business, and my main differentiator is artisanal hand-maintained porcelain?
Whether to add a new dependency, and if so which, are two of the many, many decisions we make every day in software development. We can have reasons for generally preferring to keep the total number small. But however small that is, it will never reach zero. For me to clean the toilet depends, at minimum, on whatever it takes to keep me alive, able, and present. And on the privilege of having a toilet.
Minimize the number
One Agile strategy for reducing dependency risk is to do the simplest thing that could possibly work. Sometimes that can mean copy-pasting a function from Stack Overflow instead of depending on a third-party library for it.
Another Agile strategy for reducing dependency risk is to maximize the amount of work not done. Sometimes we can deliberately decide that a sub-problem isn’t worth solving.
The only way to entirely avoid dependencies is to entirely avoid solving problems.
Then we’d need new jobs, which would be a problem. Sort of a circular dependency.
We need techniques for living with dependencies more effectively.
A dependency isn’t just one more line in a file. It’s
- An expected interface (adapter pattern)
- Noticing when it changes (contract tests)
—Moi
Amber is the force behind Self.conference, which I’ve spoken at and attended and which just posted its list of sessions and speakers. If you’re a human involved in the making of software, I highly recommend it, and can give you a code good for 10% off your ticket.
And also all of its transitive dependencies — dependencies can have dependencies too — as Amber Conville reminded me.
In the moment where we decide to add one more dependency, it could look like just one more line in a file specifying names and minimum versions. In the moment where we finish the feature and get it out the door, that could feel like enough.
Dependency now, dependency later
We may think we’ve chosen to depend on something as it is today. That’s true.
We may think we can cheaply remove or replace it tomorrow if we have a better idea. That might be true too.
We may even think we can cheaply remove or replace it in a few months if need be. That’s a lot less likely to be true.
Why? If we’re not careful, our expectations of it will be dispersed throughout our code. If it then changes in a way that fails to meet our expectations, we’ll have to be expensively careful to adjust to it. If our own expectations need to change, we’ll have to be expensively careful about how we change them. And if we hadn’t been careful, the way we’ll find out we suddenly need to be expensively careful might well come at an expensive time: in production.
Maximize control
In software, when we are careful, we can arrange to reduce our dependency risk:
I follow similar reasoning when I update all the dependencies on my server every week. The tool I use for this recently underwent a major upgrade. So last week I did my build on Friday, a little early. Over the weekend, I made the fewest possible changes to bring forward my existing configuration, did a build with the new tool, found a regression, did another build, went to production, and reported the fix. If it hadn’t worked, I’d still have another week to figure it out or revert without interrupting my cadence. But since it did work, my dependency on the build tool has been safely managed through some breaking changes. Next week, if I feel like it, I can see about taking advantage of some of the new features.
Adapter pattern: Define the calls we want to be able to make, then implement them by backing our interface with the dependency. Lets us adjust faster when something changes. (more on Wikipedia)
Contract tests: For each call our adapter makes to the dependency, write automated tests for the behavior we’re relying on. Tells us sooner when adjustments must be made. (more at Martin Fowler’s Bliki)
Given these techniques, another Agile risk-mitigation strategy clicks into place: If it hurts, do it more often. Figure out how to get notified when any of your dependencies have been updated. When you get a notification, before you do anything else, update your code to use the new version. If tests break, before you do anything else, fix them. If it’s not clear how to do that, before you do anything else, raise the risk with your team. By uncovering the dependency problem as early as possible, you’ve maximized your options for handling it well.
Once your tests are green, ship it as soon as you can. If a dependency problem somehow slips past your tests into production, it’ll be relatively easy to find, because you’ve narrowed the search space considerably. Roll back first, if you have to. Then test-drive a bugfix and ship it again.
The goal here, given that surprises are inevitable, is to control the influx of entropy into your system. Track updates and put out releases less frequently, and the surprises get larger, take longer to track down, and offer fewer, more expensive options for resolution. Or reduce batch size to get the opposite effects.
Sorry, I only understand toilets
No problem. To keep your delivery schedule from getting clogged, flush regularly.
Here’s how I documented my reasoning for a product I both managed and tech lead-ed:
Why release every month?
- Each release contains less total change. Why this matters:
- Code change is risky. Smaller increments of change help manage the risk.
- If a bug survives into production, finding it is easier, so fixing it is faster.
- If a feature doesn’t meet expected requirements, customers will complain earlier, so fixing it can happen earlier.
- The next release is always soon. Why this matters:
- Small features (or bugfixes) don’t have to wait long to get into customers’ hands.
- Big features can only be delivered via composable solutions, implemented one tractable piece at a time.
- Each change can be tested well because it’s small. Each change must be tested well because it’s about to ship.
- The master branch is always production-ready. We can always ship what we have right now.
- The previous release is always recent. Why this matters:
- Release deployment is risky. More frequent practice — and being able to remember what went wrong last time — helps manage the risk.
Why skip a month?
- If Operations is fully booked on other product releases and doesn’t have someone available.
- If up- or downstream systems are changing and it’s too risky to change ours at the same time.
- If a particular big feature simply can’t be decomposed into month-sized chunks of work. (This is very rare.)
When do you notify Operations of new dependencies to package?
By Thursday afternoon, the day before release day, we have a complete or near-complete list. In general, we try to declare each codebase’s dependencies in one place so that we can simply diff it against the previous release to see everything that’s new (updated counts as new). Then we list those dependencies on the release’s wiki page and notify our local Operations representative.
Occasionally a last-minute code change will add another dependency or two. Asking Operations to build one or two more last-minute packages isn’t terrible if we don’t do it often. More packages than that means the change probably isn’t a smart last-minute choice.
Why are you always upgrading to the latest available dependencies?
Because we depend on them.
Less elliptical answer: because they’re code we rely on but don’t control. Therefore we’re especially susceptible to changes in them. Therefore we minimize our exposure by staying up to date whenever possible, giving us the easiest possible rollback option when (inevitably) an unexpected problem occurs.
See also “Why release every month?”
Another dependency inversion principle
GeePaw Hill likes to say The code works for me, I don’t work for the code. If you’d like to put dependencies fully at your service — and not the other way around — I invite you to join me next month for my hands-on workshop at the Agile Alliance Technical Conference.