A characterization test that cannot fail is decoration, not a safety net.

Most messy-repo refactors fail the same way. You record golden values, they all pass, and you feel safe. Then you extract a function and ship a silent behavior change. The goldens never noticed, because they were never able to notice.

Here is a workflow that fixes that. Record behavior first. Then prove each recorded case can fail. Only then make the smallest safe change.

The rule: a golden you have not falsified is a guess

Golden values freeze what the code does today, including its bugs. That is the point. But a passing test proves nothing on its own.

A test only earns trust when you can make it fail on purpose. Without that step, your suite may be asserting on an empty result, a swallowed exception, or a stub that never runs.

So the gate is simple. Every characterization case must survive one deliberate, minimal mutation of the code under test.

Step 1: Inject the seams before you capture anything

Messy functions hide their collaborators. Time, randomness, network calls, and global writers make goldens flaky.

Do not refactor yet. Patch those seams in the harness instead. Notice the patch is temporary and lives in test code.

# golden_capture.py
import json
from app.legacy import settle_order # the messy 90-line function under test
import app.legacy as legacy

CASES = [
{"id": "empty_cart", "args": [[], "US"]},
{"id": "one_item", "args": [[{"sku": "A1", "cents": 500, "qty": 1}], "US"]},
{"id": "qty_zero", "args": [[{"sku": "A1", "cents": 500, "qty": 0}], "US"]},
{"id": "unknown_region", "args": [[{"sku": "A1", "cents": 500, "qty": 2}], "ZZ"]},
]

LEDGER =

[]def spy(name):
def wrap(*a, **kw):
LEDGER.append((name, [repr(x) for x in a], tuple(sorted(kw.items()))))
return 0
return wrap

def snapshot(case):
LEDGER.clear()
legacy.charge_card = spy("charge_card")
legacy.send_receipt = spy("send_receipt")
try:
return {"status": "returned", "value": settle_order(*case["args"])}
except Exception as exc:
return {"status": "raised", "type": type(exc).__name__, "msg": str(exc)}
finally:
pass

def full_snapshot(case):
out = snapshot(case)
out["calls"] = [{"fn": n, "args": a, "kwargs": dict(k)} for n, a, k in LEDGER]
return out

The call ledger matters more than the return value here. Extracting a writer often preserves the result and reorders the side effects.

Step 2: Capture the corpus into a file you can diff

Write goldens to disk. Commit them. A reviewable diff beats a magic assertion.

if __name__ == "__main__":
goldens = {c["id"]: full_snapshot(c) for c in CASES}
with open("goldens.json", "w") as fh:
json.dump(goldens, fh, indent=2, sort_keys=True)
print(f"recorded {len(goldens)} cases")

Run it once against the untouched file. Inspect the JSON by hand. Delete any case whose recorded behavior looks like an artifact of your harness.

Step 3: Replay the goldens as a test

Keep the replay boring. One parametrized test, exact equality, no fuzzy matching.

# tests/test_goldens.py
import json
import pytest
from golden_capture import full_snapshot, CASES

GOLDENS = json.load(open("goldens.json"))

@pytest.mark.parametrize("case", CASES, ids=lambda c: c["id"])
def test_behavior_is_frozen(case):
assert full_snapshot(case) == GOLDENS[case["id"]]

At this point every test passes. That is expected and meaningless. Step 4 is the one people skip.

Step 4: Mutation gate — prove the suite can fail

Mutate the target file in a scratch copy. If the suite still passes, that golden is untested for that branch.

# mutation_gate.py
import pathlib, re, shutil, subprocess, sys, tempfile

MUTANTS = [
(r"if qty <= 0:", "if qty < 0:"),
(r"total = 0\b", "total = 1"),
(r"return total", "return total + 1"),
]

def run_gate(target="app/legacy.py"):
src = pathlib.Path(target).read_text()
survivors =
[] for pattern, repl in MUTANTS:
mutated, hits = re.subn(pattern, repl, src)
if hits == 0:
continue
with tempfile.TemporaryDirectory() as td:
copy = pathlib.Path(td) / "repo"
shutil.copytree(".", copy, ignore=shutil.ignore_patterns(".git", ".venv", "__pycache__"))
(copy / target).write_text(mutated)
r = subprocess.run(
[sys.executable, "-m", "pytest", "tests/test_goldens.py", "-q", "-x"],
cwd=copy, capture_output=True, text=True,
)
if r.returncode == 0:
survivors.append(pattern)
return survivors

if __name__ == "__main__":
survivors = run_gate()
print("survivors:", survivors)
sys.exit(1 if survivors else 0)

A survivor means one of two things. Either no case exercises that branch, or your spy layer hides the effect. Add a case, not a comment.

Treat the counts in that output as illustrative. The real number depends on your function.

Where a free model actually helps

The expensive part is enumerating branches, not writing asserts. This is the narrow job I hand to a model: read the messy function and propose input classes I forgot.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I use MonkeyCode's free model access and free server option, as described by the operator, to draft candidate cases and a first-pass mutation list. The model's output is raw material only. Every case still has to pass capture, replay, and the mutation gate before it earns a line in the corpus. The gate is the anti-hallucination step, since a model cannot verify its own tests.

Step 5: Make the smallest safe change

The gate is green when every mutant dies. Now touch production code once.

Extract one function. Change nothing else. No renames, no formatting, no logging tweaks in the same commit.

python mutation_gate.py && pytest -q && git diff --stat

If the golden diff is empty and the gate still fails every mutant, you have a real safety net. If a golden changes, stop and explain why before continuing.

Decision table: characterize, or walk away

Situation

Characterize first?

Why

Function you touch weekly

Yes, full corpus plus gate

The investment pays back fast

One-off script, deleted next sprint

No

Goldens outlive their value

Heavy nondeterminism, no injectable seam

Patch seams first, then yes

Flaky goldens teach nothing

Function with 40+ branches, no tests

Yes, but time-box it

Gate tells you when coverage is enough

Behavior you are about to delete

No, add deletion tests after

Freezing a bug is counterproductive

Limitations and who should skip this

Goldens freeze existing bugs, not correct behavior. That is intentional, and it is also a trap if you never revisit them.

Seam patching gets fragile. If collaborators are imported deeply, the harness grows faster than the refactor.

The mutation gate needs a fast test run. On a suite that takes minutes per case, this loop becomes unusable.

Skip this approach for prototypes, generated code, or any file scheduled for replacement. Also skip it if your team will not review the golden JSON in pull requests. Unreviewed goldens turn into noise nobody trusts.

Run the capture, run the gate, then make one small change. Ship the diff you can explain line by line. If you want a place to draft those first-pass cases, the free model access and free server option in MonkeyCode are a reasonable starting point.#testing #refactoring #python #ai #software #coding #development #engineering #inclusive #community
Make Each Characterization Test Fail Once Before You Refactor

There is an uncomfortable truth about most non-tech enterprises: a lot of their software is insanely shit.

I don't mean that the engineers are shit. In many cases, the opposite is true. I've worked with plenty of smart engineers who were perfectly capable of building good software. The problem is that good software is expensive, and for a long time the economics of building it simply didn't make sense.

Think about the incentives inside a large enterprise. You have an internal system that employees have to use every day. The interface is confusing, the workflows are clunky, the codebase is a mess, and everyone knows it needs a serious refactor. Maybe there are thousands of lines of code that should be deleted. Maybe the architecture was designed ten years ago around assumptions that haven't been true for years. Maybe adding a simple feature now takes three weeks because nobody really understands how the thing works anymore.

The engineers know this. The users know this. Everyone complains about it.

And then someone proposes spending three months fixing it.

Why?

What exactly does the business get at the end of those three months?

There probably isn't a new customer. There isn't a new revenue stream. There isn't a shiny feature that can go into a quarterly presentation. Nobody in sales can put the refactor into a demo. The business has simply spent three months making something that already worked—technically speaking—less shit.

From the perspective of an executive looking at a roadmap, that's a difficult proposition to approve.

And in many enterprises, the incentives are even worse than that. The users aren't going anywhere. If you work for a bank, an airline, a government department, an insurance company, or some giant industrial company, you don't get to decide that you're going to use a competitor's internal software instead. You're using whatever system your employer gives you.

So why obsess over UX?

If your customers can leave, product quality matters enormously. If your customers are employees who have no choice, the incentive is very different.

This is one of the reasons I think a lot of terrible enterprise software actually makes sense.

Not to me, necessarily. But within the incentive structure of the organization, it can make perfect sense.

The company needs feature X by September. The team has four engineers. The existing system technically works. There is a giant backlog. Nobody has allocated time for a rewrite. The person responsible for the budget doesn't personally use the software. The person who understands the architecture is worried about hitting the deadline. And the consequences of making the code slightly worse won't show up on anyone's quarterly report.

So you ship it.

Then you ship the next thing.

Then the next thing.

And eventually you have a system that everyone hates.

The hidden cost of software everyone hates

The problem is that "the software technically works" is an incredibly low bar.

A piece of software can work perfectly well and still make thousands of people's lives slightly worse every single day.

A workflow that should take two minutes takes ten. A page that should load instantly takes thirty seconds. An employee has to enter the same information into three different systems. An error message tells them absolutely nothing. A form resets itself for no apparent reason. A button is hidden behind some bizarre legacy navigation. Nobody knows why the process works this way, but changing it feels too risky.

Each individual annoyance is tiny.

The problem is that these tiny annoyances compound.

If 5,000 employees waste ten minutes a day fighting with internal software, that's more than 200,000 hours of human time every year. And even that calculation misses something important: the psychological cost.

People don't just lose ten minutes.

They become frustrated.

They lose momentum. They have to remember workarounds. They start distrusting the systems around them. They learn that doing their job means fighting with the tools provided to them.

And eventually, they start hating the job itself.

We talk endlessly about employee engagement and morale. Companies spend fortunes on offsites, leadership programmes, wellness initiatives and motivational speakers. But there is something deeply absurd about trying to improve employee morale while forcing people to spend eight hours a day using software they hate.

Everyone knows that teams with low morale don't win championships.

We understand this intuitively in sports. Nobody looks at a dysfunctional football team and says, "It's fine, the players are still technically capable of running around."

We care about the environment in which people perform.

Software is part of that environment.

And yet, in many companies, the quality of the software employees use every day is treated as an engineering concern rather than a business concern.

The dinosaur problem

There is also a generational problem here.

A lot of people making technology decisions in large organizations didn't grow up thinking of software as a product. They grew up thinking of it as infrastructure: something the IT department provides so that the business can function.

You don't ask whether the electricity in the office has a delightful user experience.

You don't care whether the database has beautiful onboarding.

You just need it to work.

That mentality made a lot more sense when software was something employees occasionally interacted with. It makes considerably less sense when software is effectively the workplace itself.

But organizational thinking moves slowly.

Companies are enormous machines with long memories. The incentives, processes and mental models that made sense twenty years ago can survive long after the world that created them has disappeared.

And this is where I think something genuinely exciting is happening.

Coding agents are changing the economics of software quality.

The three-month refactor

Imagine that you have a horrible codebase.

For years, the engineering team has known that it should be cleaned up. Everyone has a list of things they would change if they had the time. But the time never comes.

The reason is not necessarily that management doesn't care. The reason is that the opportunity cost is enormous.

A traditional refactor might require one or two engineers to spend weeks or months understanding the existing system, designing the changes, implementing them, testing everything, fixing regressions and gradually migrating the code.

Three months is a long time.

And three months of engineering time is expensive.

Now imagine that instead of taking three months, an extremely capable coding agent can do a large part of the work in a day.

Not every refactor, obviously. Not every codebase. And certainly not without human oversight, testing and engineering judgement.

But the economic threshold has changed.

Something that previously required a three-month business case might now require an afternoon.

That is a completely different proposition.

AI makes quality economically viable

This is the part of AI-assisted programming that excites me much more than the promise of simply writing code faster.

I don't think the most interesting consequence of coding agents is that developers can produce more lines of code.

Frankly, the world already has enough lines of code.

The interesting consequence is that code quality itself can become economically viable in places where it wasn't before.

For a long time, software engineering involved an uncomfortable tradeoff. You could build the right thing, properly, but it would take longer. Or you could ship something that was good enough and move on.

There were always engineers arguing for the former, and there were always business pressures pushing toward the latter.

Sometimes the business won for perfectly rational reasons.

But what happens when the cost of doing the right thing falls by an order of magnitude?

Suddenly, the argument changes.

The refactor doesn't have to compete with the feature roadmap in the same way. The cleanup doesn't necessarily require taking a team off delivery for a quarter. The ugly piece of legacy code doesn't have to remain ugly simply because nobody can justify spending six weeks fixing it.

And perhaps most importantly, engineers don't have to spend as much time convincing people that quality matters.

This is a subtle but profound shift.

In the old world, if I wanted to spend three months improving a codebase, I had to convince someone that three months of engineering time was worth spending on something that users might not immediately notice.

I needed a business case.

I needed projections about future productivity.

I needed to explain how technical debt compounds.

I needed to convince someone to spend money today to avoid costs that might appear six months or two years from now.

That's hard.

But if the same improvement can be done in eight hours with an agent, the conversation becomes very different.

Maybe I don't need a business case.

Maybe I can just do it.

The software we already have

This is why I think the biggest impact of coding agents might not be that they allow us to build more software.

We already build an enormous amount of software.

It might be that they finally allow us to make the software we already have good.

There are millions of applications sitting inside organizations that are not fundamentally broken. They are just mediocre. They are slow, awkward, overcomplicated, badly structured and unpleasant to maintain.

And for years, that was the equilibrium.

The software was bad, but fixing it was too expensive.

So everyone learned to live with it.

Engineers learned the workarounds. Employees learned the weird workflows. Managers learned to accept the complaints. Executives learned that the system "worked."

The organization adapted itself around the limitations of its software.

That is a very strange thing when you step back and look at it.

We built computers to make people more productive, and then spent decades making people adapt their behaviour to accommodate the computers.

Coding agents could start reversing that relationship.

If an engineer can take a horrible codebase and dramatically improve it without blowing up the delivery schedule, the default assumption no longer has to be that technical debt is permanent.

Maybe the ugly code doesn't have to stay.

Maybe the terrible internal tool can actually become pleasant.

Maybe the three-month refactor becomes an afternoon.

And if that happens at scale, the consequences aren't just technical.

They are organizational.

Because when the software gets better, the people using it get better tools. When people have better tools, they waste less time. They get less frustrated. They can move faster. They spend less mental energy fighting the system and more energy doing the thing the system was supposed to help them do.

Good software isn't just a technical luxury.

It is part of the working environment.

And for the first time, we may be entering a world where making it good is cheap enough that even enterprises have less of an excuse not to.

Maybe the future of coding agents isn't that we'll write ten times as much code.

Maybe it's that we finally get to stop writing quite so much shit code.#ai #discuss #refactoring #architecture #software #coding #development #engineering #inclusive #community
Maye Enterprise Software Doesn’t Have to Suck Anymore

Characterization Tests First, Then the Smallest Safe Change

Stop refactoring. Start recording. The smallest safe change wins only if you can prove nothing else moved.

That is the whole method. You freeze the hidden inputs, snapshot the current output, then change one line. The snapshot decides whether you were safe.

This article walks through a runnable loop on a small legacy stand-in module. Swap in your real module and the steps stay the same.

1. Turn the ticket into a behavior, not a design

The ticket here is vague: unknown PLAN breaks invoice generation. Do not translate that into an architecture plan yet.

Translate it into one observable statement. "Unknown plan raises KeyError before any row is processed."

Now the statement is testable. It also tells you the bug lives in one lookup, not in the whole loop.

2. Freeze the hidden globals before you assert anything

Legacy modules read two hidden inputs constantly: the clock and the environment. Both change between your laptop and CI.

Here is the stand-in under test. It is deliberately small and deliberately dirty.

# billing/cycle.py
import datetime
import os

RATES = {"standard": 1.0, "pro": 0.8, "legacy": 1.25}

def invoice_lines(rows):
today = datetime.date.today()
plan = os.environ.get("PLAN", "standard")
rate = RATES
[plan] out =
[] for row in rows:
days = (today - row["start"]).days
if days < 0:
days = 0
amount = round(row["units"] * rate * days, 2)
out.append({"id": row["id"], "days": days, "amount": amount})
return out

Two hidden inputs sit in four lines. datetime.date.today() reads the machine clock. os.environ.get reads the process environment.

Freeze both by patching the names inside the module under test. Never patch the stdlib module globally.

# tests/test_cycle_char.py
import datetime
import json
import os
import pathlib
import types

import pytest

from billing import cycle

GOLDEN = pathlib.Path(__file__).parent / "golden" / "invoice_lines.json"

class FixedDate(datetime.date):
@classmethod
def today(cls):
return cls(2026, 3, 1)

@pytest.fixture
def frozen(monkeypatch):
monkeypatch.setenv("PLAN", "pro")
monkeypatch.setattr(cycle, "datetime", types.SimpleNamespace(date=FixedDate))

This works because the module calls datetime.date.today() by attribute access. A module that does from datetime import date needs a different patch point.

That detail matters. Note it, or you will fight a passing test that proves nothing.

3. Record a golden, then assert against it

Write the rows once, run them, and save the output. The recorder is the source of truth, not your memory of the old behavior.

ROWS = [
{"id": "a1", "start": datetime.date(2026, 2, 1), "units": 3},
{"id": "a2", "start": datetime.date(2026, 3, 4), "units": 2},
{"id": "a3", "start": datetime.date(2026, 3, 1), "units": 0},
]

def test_invoice_lines_matches_golden(frozen):
got = cycle.invoice_lines(ROWS)
if os.environ.get("RECORD_GOLDEN") == "1":
GOLDEN.parent.mkdir(parents=True, exist_ok=True)
GOLDEN.write_text(json.dumps(got, indent=2))
assert got == json.loads(GOLDEN.read_text())

Record once with RECORD_GOLDEN=1 python -m pytest tests/test_cycle_char.py -q. The first run always passes, and that is expected.

Read the recorded file by hand before you commit it. A snapshot is not a truth claim.

[
{"id": "a1", "days": 28, "amount": 67.2},
{"id": "a2", "days": 0, "amount": 0.0},
{"id": "a3", "days": 0, "amount": 0.0}
]

Pin the error path too. Empty input still hits the lookup, so the exception fires before the loop.

def test_unknown_plan_raises_keyerror(monkeypatch, frozen):
monkeypatch.setenv("PLAN", "platinum")
with pytest.raises(KeyError):
cycle.invoice_lines([])

4. Prove the harness bites before you trust it

A test that cannot fail is decoration. Break a copy of the code and confirm the suite goes red.

rsync -a --exclude .git ./ /tmp/char-mut/
sed -i 's/if days < 0:/if days < -1:/' /tmp/char-mut/billing/cycle.py
cd /tmp/char-mut && python -m pytest tests/test_cycle_char.py -q; echo "exit=$?"

Expect exit=1. The clamped future date now leaks a negative days value, and the golden catches it.

If the suite still passes, your cases do not cover the branch. Fix that before touching the real module.

5. Take the smallest change, then edit the pin on purpose

Smallest here is one expression. Replace the strict lookup with a guarded one.

- rate = RATES
[plan]+ rate = RATES.get(plan, RATES["standard"])

This changes observable behavior, so the pinned error test must change with it. Edit it deliberately, in the same commit.

-def test_unknown_plan_raises_keyerror(monkeypatch, frozen):
- monkeypatch.setenv("PLAN", "platinum")
- with pytest.raises(KeyError):
- cycle.invoice_lines([])
+def test_unknown_plan_falls_back_to_standard(monkeypatch, frozen):
+ monkeypatch.setenv("PLAN", "platinum")
+ assert cycle.invoice_lines([]) ==

[]Now the diff shows one intended behavior edit. Everything else stays green, which is the only proof you have.

Use Python 3.11 or newer here. Python 3.9 is already past its end-of-life date, so new test tooling should not target it.

6. Use a change ladder instead of judgment calls

Rank candidate changes by blast radius. Take rank 0 first, then climb one rung per commit.

Rank

Change

Lines touched

Gate to pass

0

Rename a local variable

1

goldens unchanged

1

Add a guard clause

1-3

goldens unchanged

2

Extract one pure helper

5-15

goldens unchanged, call order identical

3

Change observable behavior

1-5

exactly one pinned test edited on purpose

4

Move the module

many

old import path still pinned

5

Rewrite the module

all

stop, split into ranks 0-4

Rank 3 is the one people skip. They mix a behavior fix with a structure change and lose the ability to review either.

7. Know when to stop the loop

Set stop rules before you start. Seam hunting expands without limits otherwise.

• Stop if freezing the hidden globals takes more than about twenty minutes.
• Stop if the golden output contains wall-clock timestamps or process IDs you cannot stub.
• Stop if behavior depends on network calls, random seeds, or thread interleaving you cannot reproduce.
• Stop if you cannot run the code at all. Characterization without execution is fiction.

Each stop is a signal to shrink scope, not to lower standards.

8. Where model assistance fits, and where it does not

Drafting the edge-case rows in step 3 is the slow part. That is the part worth delegating.

I use MonkeyCode's free model access to propose candidate input rows and hidden-global guesses. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The free server option lets me run the mutation check on a repo copy without provisioning a machine of my own.

The model proposes. The recorder decides. Never let a model write the expected output from memory.

Free access and a free server option are the only two availability claims I make. I have not measured throughput or uptime, and a free tier is not an SLA.

9. Limitations and who should not use this

Characterization tests pin current behavior, including bugs. They do not prove correctness and they do not replace intent-based tests.

Exact JSON equality is brittle with floats. Compare amounts with pytest.approx when your golden is hand-edited.

Clock patching only works when the module reads time by attribute access. Import styles, C extensions, and third-party clients may refuse the patch.

Skip this method if you are writing greenfield code, deleting the module next sprint, or have no way to execute it. Skip it if your team cannot review a behavior diff honestly.

If you run this loop on a repo copy, start with the recorder and keep it in charge of the truth.#testing #python #refactoring #legacycode #software #coding #development #engineering #inclusive #community
Characterization Tests First, Then the Smallest Safe Change

Mixed None-and-raise APIs fail under model rewrites. The error taxonomy is the real product surface. Pin that taxonomy with characterization tests first, always.

Change one except clause only after the pin stays green. Happy-path unit tests will not save this refactor. Callers already branch on None, dict keys, and types.

Collapse any one of those shapes and production breaks. The defect is an unrecorded taxonomy, not missing types.

Why model rewrites miss the contract

AI coding threads keep offering full-file cleanups. Those cleanups prefer one error type. They also prefer raising over returning None.

That preference is style, not evidence. A messy dispatcher often has three outcomes. Some inputs raise ValueError for bad payloads.

Some inputs return None after a send failure. Some inputs return {"ok": False, "status": 429}. Downstream code already checks all three shapes.

Loop-style agent edits make the same cut. They unify handlers because duplication looks sloppy. Duplication here is the published contract.

Four observables to freeze

Record these four fields for every fixture. Skip prose messages on the first pass. Message strings drift across harmless edits.

1. Escaping exception type, or a sentinel if none.
2. Return shape: None, mapping keys, or other.
3. Integer status when a mapping returns.
4. Count of WARNING-or-higher log records.

Types and keys usually stay stable. Status integers also stay stable. Log counts need a named logger, not the root.

Decision table for one dispatcher

The table below is the spec. It is a labeled example. It is not a production trace.

Fixture

Escapes

Return

Status

WARN+

empty body

RuntimeError

n/a

n/a

0

invalid JSON

ValueError

n/a

n/a

0

JSON list

ValueError

n/a

n/a

0

missing id

none

None

n/a

1

send TypeError

none

None

n/a

1

send TimeoutError

none

None

n/a

1

downstream 429

none

dict

429

1

downstream 200

none

dict

200

0

Keep the send TypeError row. That row is the trap. A cleaner except often drops it.

Artifact: characterization harness

The code is a worked example. Run it locally before any extract. Do not treat it as measured field data.

# events.py — labeled example, not a live service
from __future__ import annotations

import json
import logging
from typing import Any, Callable

log = logging.getLogger("events")

SendFn = Callable[[dict], tuple[int, Any]]

def dispatch_event(raw: str | None, send: SendFn):
"""Messy contract: None, dict, or raise."""
if raw is None or raw == "":
raise RuntimeError("empty body")
try:
payload = json.loads(raw)
except Exception:
raise ValueError("bad json")
if not isinstance(payload, dict):
raise ValueError("bad json")
if "id" not in payload:
log.warning("missing id")
return None
try:
status, body = send(payload)
except Exception:
log.warning("send failed")
return None
if status >= 400:
log.warning("downstream %s", status)
return {"ok": False, "status": status, "body": body}
return {"ok": True, "status": status, "body": body}

# test_events_contract.py — worked example
import json
import logging

import pytest

from events import dispatch_event

def _records(caplog):
return [r for r in caplog.records if r.name == "events" and r.levelno >= logging.WARNING]

def _run(raw, send, caplog):
caplog.set_level(logging.WARNING, logger="events")
try:
value = dispatch_event(raw, send)
return None, value, _records(caplog)
except Exception as exc:
return type(exc), None, _records(caplog)

def test_empty_body_raises_runtime_error(caplog):
exc, value, recs = _run("", lambda p: (200, "ok"), caplog)
assert exc is RuntimeError
assert value is None
assert len(recs) == 0

@pytest.mark.parametrize("raw", ["{", "[]", "null"])
def test_bad_json_raises_value_error(raw, caplog):
exc, value, recs = _run(raw, lambda p: (200, "ok"), caplog)
assert exc is ValueError
assert value is None
assert len(recs) == 0

def test_missing_id_returns_none(caplog):
exc, value, recs = _run('{"name": "x"}', lambda p: (200, "ok"), caplog)
assert exc is None
assert value is None
assert len(recs) == 1

def test_send_type_error_returns_none(caplog):
def send(_payload):
raise TypeError("broken client")

exc, value, recs = _run('{"id": 1}', send, caplog)
assert exc is None
assert value is None
assert len(recs) == 1

def test_send_timeout_returns_none(caplog):
def send(_payload):
raise TimeoutError("late")

exc, value, recs = _run('{"id": 1}', send, caplog)
assert exc is None
assert value is None
assert len(recs) == 1

def test_downstream_429_is_dict(caplog):
exc, value, recs = _run('{"id": 1}', lambda p: (429, "slow"), caplog)
assert exc is None
assert value == {"ok": False, "status": 429, "body": "slow"}
assert len(recs) == 1

def test_downstream_200_is_dict(caplog):
exc, value, recs = _run('{"id": 1}', lambda p: (200, {"n": 1}), caplog)
assert exc is None
assert value == {"ok": True, "status": 200, "body": {"n": 1}}
assert len(recs) == 0

Run the file before editing handlers. Use a short traceback during the first pin.

python -m pytest test_events_contract.py -q --tb=short

Prove the suite catches a collapsed taxonomy

A green suite with no mutation check is weak. You must see a failed unified-error fork. Keep that fork out of the merge.

# test_events_mutation.py — must fail against a unified raise rewrite
import events

def test_unified_raise_would_break_send_type_error(monkeypatch, caplog):
def rewritten(raw, send):
payload = __import__("json").loads(raw)
status, body = send(payload) # TypeError now escapes
return {"ok": status < 400, "status": status, "body": body}

monkeypatch.setattr(events, "dispatch_event", rewritten)
with pytest.raises(TypeError):
events.dispatch_event('{"id": 1}', lambda p: (_ for _ in ()).throw(TypeError("x")))

Expect this mutation test to fail on the rewrite. Restore the messy handler after that check. The failure is the evidence, not a vibe.

Numbered workflow

1. Copy the messy module into a branch. Do not edit handlers yet.
2. Inventory callers that check is None or catch types.
3. Fill the four-column table from those callers.
4. Encode each row as one test function.
5. Add one mutation test that unifies errors.
6. Confirm the mutation test fails on purpose.
7. Restore the original handler body.
8. Apply one extract or one except edit.
9. Re-run the same characterization file.
10. Revert if any row changes shape.

Step two is not optional. Tests invented from the callee miss caller branches. Caller branches are the contract.

rg -n "dispatch_event\(" -g "*.py"
rg -n "is None|except ValueError|except RuntimeError" -g "*.py"

Record each hit as a fixture name. Missing hits become missing rows. Missing rows make unsafe extracts look safe.

The smallest safe change

Do not narrow except Exception on send first. The table says TypeError becomes None. Narrowing that clause raises TypeError instead.

That raise is a contract break. Extract the block without narrowing. Keep the same log line and None return.

def _send_or_none(payload, send):
try:
return send(payload)
except Exception:
log.warning("send failed")
return None

def dispatch_event(raw, send):
if raw is None or raw == "":
raise RuntimeError("empty body")
payload = _parse_object(raw)
if "id" not in payload:
log.warning("missing id")
return None
result = _send_or_none(payload, send)
if result is None:
return None
status, body = result
if status >= 400:
log.warning("downstream %s", status)
return {"ok": False, "status": status, "body": body}
return {"ok": True, "status": status, "body": body}

That extract is boring. Boring is the point. The taxonomy stays in the table.

A later change can narrow exceptions. Do that only with a version note. Add a row that expects TypeError to escape. Tell callers before you merge.

Parse-side except narrowing

The JSON branch is different. json.loads should raise json.JSONDecodeError. Re-raising ValueError is the published type.

Narrowing except Exception to except json.JSONDecodeError can be safe. Prove it with the invalid JSON row. Prove it with the JSON list row too.

def _parse_object(raw):
try:
payload = json.loads(raw)
except json.JSONDecodeError:
raise ValueError("bad json") from None
if not isinstance(payload, dict):
raise ValueError("bad json")
return payload

Both bad-JSON fixtures must still raise ValueError. A leaked JSONDecodeError is a taxonomy change. Do not ship that leak without a caller audit.

from None also drops __cause__. Add a cause fixture if any caller reads it. Skip that fixture when no caller inspects causes.

Where a free coding model belongs

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A model can draft the extract. It must not invent a new taxonomy. MonkeyCode provides free model access and a free server option.

Use either only after the characterization file is green. Feed the model the table and the test file. Ask for one extract, not a rewrite.

Reject a patch that removes a row. Reject a patch that changes None into a raise. This is not an agent loop.

One prompt. One diff. Same tests.

Limitations

The harness does not prove semantic equality. It pins types, keys, status, and log counts. It misses timing, retry storms, and byte identity.

Log counts break if a library logs extra warnings. Pin the logger name events. Cap propagation on that logger.

Do not pin the root logger. Root pins go red on unrelated imports. That noise hides a real taxonomy drift.

The table is only as good as the fixtures. One unlisted caller path is an untested shape. Untested shapes are how "safe" extracts land in incident channels.

Who should not use this

Do not use this flow for greenfield APIs. Design one error shape there. Do not preserve None plus raises on purpose.

Do not use this flow for security boundaries. Characterization will pin insecure behavior. Pinning is not hardening.

Do not use this flow without tests you can run offline. A model server cannot replace the table. If pytest cannot collect locally, stop.

Teams with a published OpenAPI error schema may skip dict-key rows. Use the schema as the table instead. Still pin process-local exception escapes.

HTTP schemas often omit those local raises. Omitting them is how ValueError turns into a 500. Keep the escape column anyway.

Checklist before merge

1. Characterization file is the only new proof.
2. Diff touches one handler extract or one except.
3. Mutation test still fails against a unified-error fork.
4. No fixture row changed shape.
5. None callers still exist, or a version note ships.

If a model patch violates any line, drop the patch. Rewrite pressure is not evidence. The table is.

If the characterization file is already green, one extract prompt on the free server is enough. Skip that prompt when the table still has empty cells.#python #testing #refactoring #ai #software #coding #development #engineering #inclusive #community
Freeze the Error Contract Before One except Change

Do not extract file helpers from a messy module yet. Snapshot every resolved absolute path before the first edit. Relative opens form a hidden contract across three roots.

Most failed extracts start as a path-root mismatch. The helper looks cleaner after the extract lands. The files then land in a different directory tree.

Three roots, one unpublished API

A typical messy module mixes three path roots. None of them appear in the public function signature.

os.getcwd() follows the process, not the source file. Pytest, systemd, and cron each change that value.

Path(__file__).resolve() follows the module on disk. A later package move silently retargets every relative open.

An env var such as DATA_DIR may override both. Empty, relative, and absolute values all behave differently.

The public function still accepts only a basename. Callers believe the output path stays stable. That belief does not survive a chdir.

Teaching example: three writes, three roots

The listing below is a teaching example, not production code. It writes one report beside three different roots.

# report_kit.py — messy on purpose
from __future__ import annotations

import json
import os
from pathlib import Path

HERE = Path(__file__).resolve().parent

def write_daily_report(name: str) -> dict[str, str]:
data_dir = os.environ.get("DATA_DIR", "data")
cwd_out = Path("out") / name
here_out = HERE / "out" / name
env_out = Path(data_dir) / name

payload = {"name": name, "pid": os.getpid()}
text = json.dumps(payload, sort_keys=True) + "\n"

cwd_out.parent.mkdir(parents=True, exist_ok=True)
here_out.parent.mkdir(parents=True, exist_ok=True)
env_out.parent.mkdir(parents=True, exist_ok=True)

cwd_out.write_text(text, encoding="utf-8")
here_out.write_text(text, encoding="utf-8")
env_out.write_text(text, encoding="utf-8")

return {
"cwd": str(cwd_out),
"here": str(here_out),
"env": str(env_out),
}

A naive extract wraps the three write_text calls. It often introduces Path.cwd() in one place. One root then silently absorbs the other two.

Return values still look like relative strings. Tests that assert those strings stay green. The bytes move anyway.

Artifact: a path ledger

Build a ledger before any helper extract. Record caller, raw argument, and resolved absolute path. Hash the sorted ledger. Treat that hash as a characterization oracle.

# path_ledger.py — teaching harness
from __future__ import annotations

import hashlib
import json
import os
import traceback
from pathlib import Path

LEDGER: list[dict[str, str]] =

[]def _caller() -> str:
frames = traceback.extract_stack()
for frame in reversed(frames[:-1]):
if "path_ledger.py" not in frame.filename:
return f"{frame.filename}:{frame.lineno}:{frame.name}"
return "unknown"

def record(kind: str, raw: str, resolved: Path) -> None:
LEDGER.append(
{
"kind": kind,
"caller": _caller(),
"raw": raw,
"cwd": os.getcwd(),
"resolved": str(resolved.resolve()),
}
)

def ledger_hash() -> str:
blob = json.dumps(LEDGER, sort_keys=True, indent=2)
return hashlib.sha256(blob.encode("utf-8")).hexdigest()

def dump(path: Path) -> str:
path.write_text(
json.dumps(LEDGER, indent=2, sort_keys=True) + "\n",
encoding="utf-8",
)
return ledger_hash()

Wrap writes at the test boundary only. Do not patch production code for this measurement. Run the same invocation the module already trusts.

# test_path_ledger.py — characterization, not a unit test
from __future__ import annotations

from pathlib import Path
from unittest.mock import patch

import path_ledger
import report_kit

GOLDEN = Path(__file__).parent / "goldens" / "report_kit_paths.json"
GOLDEN_HASH = Path(__file__).parent / "goldens" / "report_kit_paths.sha256"

def _traced_write_text(self: Path, *args, **kwargs):
path_ledger.record("Path.write_text", str(self), self)
return Path.write_text(self, *args, **kwargs)

def test_write_daily_report_path_ledger(tmp_path, monkeypatch):
monkeypatch.chdir(tmp_path)
monkeypatch.setenv("DATA_DIR", str(tmp_path / "env-data"))
monkeypatch.setattr(report_kit, "HERE", tmp_path / "pkg")
(tmp_path / "pkg").mkdir()

with patch.object(Path, "write_text", _traced_write_text):
report_kit.write_daily_report("daily.json")

digest = path_ledger.dump(tmp_path / "ledger.json")
if not GOLDEN.exists():
GOLDEN.parent.mkdir(parents=True, exist_ok=True)
GOLDEN.write_text(
(tmp_path / "ledger.json").read_text(encoding="utf-8"),
encoding="utf-8",
)
GOLDEN_HASH.write_text(digest + "\n", encoding="utf-8")
raise AssertionError("golden created; rerun to pin")

assert digest == GOLDEN_HASH.read_text(encoding="utf-8").strip()

Label this pin as an oracle, not as coverage. The first run writes goldens on purpose. The second run fails on any resolved-path drift.

Trace mkdir in the same harness when directories matter. A helper can reuse a folder the original code created. That reuse still retargets later writes.

Decision table for one extract

Use the table before any patch is accepted. Each row is a veto, not a preference.

Signal in the ledger

Safe extract?

Required pin

cwd-relative out/name

Not yet

chdir plus expected absolute

__file__-relative out/name

Not yet

frozen HERE

env-relative DATA_DIR/name

Not yet

empty, relative, absolute env

mixed roots in one function

No

split by root, not call shape

only basenames change

Yes

hash still matches

A model often groups the three writes together. They share write_text, so the grouping looks obvious. The ledger groups them by root instead.

Root grouping is the correct split. Call-shape grouping is the usual defect.

Numbered workflow

Follow these steps in order and skip none.

1. Record one real invocation with cwd, env, and argv stored together.
2. Trace every open, write_text, mkdir, and Path constructor you might move.
3. Resolve each raw path against the recorded cwd, storing absolutes only.
4. Hash the sorted ledger, then commit both the hash and JSON.
5. Add three extra invocations: new cwd, unset DATA_DIR, absolute DATA_DIR.
6. Refuse every extract until all four hashes stay stable across reruns.
7. Extract one root only, and keep the other two writes inline.
8. Diff the ledger JSON, not the source, before any merge.

Relative goldens will lie after a machine change. Absolute goldens survive a different checkout path. That is the entire point of the pin.

Commands for the first pin:

mkdir -p goldens
python -m pytest test_path_ledger.py -q
# first run creates goldens and fails
python -m pytest test_path_ledger.py -q
# second run must pass before any extract

Commands after a proposed extract:

python -m pytest test_path_ledger.py -q
git diff -- goldens/report_kit_paths.json
# any resolved-path line change is a rejected patch

A smoking-gun diff looks like this fragment:

- "resolved": "/tmp/pytest-of-dev/test0/out/daily.json"
+ "resolved": "/home/ci/project/out/daily.json"

The source diff can still look like a tidy helper. The ledger line is the reject signal.

Smallest safe change

The smallest change moves one root. It does not introduce a generic writer yet.

def _write_cwd_report(name: str, payload: str) -> str:
target = Path("out") / name
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(payload, encoding="utf-8")
return str(target)

Leave __file__ and DATA_DIR writes in the original function. A later extract can take the second root. Each extract must keep the ledger hash unchanged.

Do not normalize paths inside the new helper. Normalization is a behavior change, not cleanup. Record it as a new ledger when you truly need it.

Avoid .resolve() in the extracted helper. resolve() follows symlinks and can rewrite goldens. Prefer the same construction the messy module already used.

Mixed open() and Path modules

Some messy modules still call open() directly. Trace that path with a thin wrapper. Keep the same ledger schema for both styles.

import builtins
from pathlib import Path

_real_open = builtins.open

def traced_open(file, mode="r", *args, **kwargs):
raw = str(file)
path_ledger.record("open", raw, Path(file))
return _real_open(file, mode, *args, **kwargs)

Install the wrapper only inside the characterization test. Production code should keep its original open. The oracle lives in the test process.

If both open and Path.write_text appear, keep both kinds. Dropping one kind hides a root. Hidden roots are how extracts ship broken CI writes.

Where a free model fits

A free coding model can draft the one-root helper. It cannot choose the root. The ledger already chose.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode provides free model access and a free server option. Use the server to run the four pinned invocations. Feed the model the messy function, the ledger JSON, and the decision table. Reject any patch that alters a resolved field.

The model is a diff generator in this workflow. The hash remains the reviewer for every patch. Do not skip that reviewer for a cleaner diff.

Failure analysis

Hash changes but the source looks equivalent. A Path constructor started calling .resolve() early. Absolute strings then diverge on symlinks.

Hash stays stable on a laptop and fails in CI. The golden cwd was never isolated. Rerun the pin under tmp_path.

Only two of three writes appear in the ledger. mkdir created a directory the extract later reuses. Trace mkdir as a first-class event.

Env values look absolute in every golden row. The test set an absolute DATA_DIR only. Add the unset and relative cases before review.

Returned relative strings still match after the extract. Callers never saw the absolute target. Assert the ledger, not the return map.

Limitations

This ledger does not prove functional correctness of the report. File contents can still rot under this pin. Pair it with a payload hash when bytes matter.

It misses networked I/O by design. HTTP and object-store clients need a different oracle. Do not reuse this hash for those calls.

It is weak against symlink farms in deploy trees. resolve() follows links; absolute() does not. Pick one rule and keep it fixed.

Race conditions remain outside the ledger. Two processes can share one cwd. The ledger is per-process and will not serialize them.

Windows drive letters and UNC paths need extra goldens. Do not copy a POSIX hash onto Windows runners. Split those hashes by platform.

Who should not use this

Do not use this workflow on a greenfield module. Write explicit path arguments first in new code. There is nothing useful to characterize there.

Do not use it when output locations must change on purpose. Update the golden in the same commit as the move. Do not treat the hash as sacred then.

Do not use it as a substitute for backup policy. Characterization does not recover overwritten files. Keep real backups for destructive jobs.

Skip it for one-off notebooks and scratch CLIs. The process cwd is the product in those tools. A ledger mostly adds noise there.

Checklist before merge

1. Four invocations produced four hashes, and all four stayed green.
2. One root was extracted, and two roots remained inline.
3. The ledger JSON diff is empty after the helper lands.
4. The helper introduced no new .resolve() calls on write paths.
5. DATA_DIR cases include unset, relative, and absolute values.

If any box is still open, keep the helper on the branch. Ship the path pin first. The extract can wait for a green hash.

Run the ledger on a free server if you already have one. Keep the extract behind that green hash.#python #testing #refactoring #ai #software #coding #development #engineering #inclusive #community
Characterize Path Resolution Before You Move One open()

We Let AI Agents Rewrite a 92M-Message-a-Day Service in Go—Zero Incidents, by (not on Mastodon or Bluesky):

https://www.checklyhq.com/blog/agentic-rewrite-nodejs-to-go/?ref=frontenddogma.com

#aiagents #ai #refactoring #go #casestudies

Simplifying and Refactoring Introductory Calculus

https://arxiv.org/abs/1811.03459

Comments: https://news.ycombinator.com/item?id=49306196

#HackerNews #calculus #education #refactoring #math #simplification #arxiv

A Short Guide to Naming by Tim Ottinger is free with a Leanpub Reader membership! Or you can buy it for $6.50! https://leanpub.com/naming_shortguide #computer_programming #python #javascript #java #web_development #refactoring

[Перевод] Экономическая выгода рефакторинга в эпоху AI-агентов

Осваивая разработку с помощью AI-агентов, я написал веб-приложение для собственной ежедневной работы. Проект получился довольно сложным: с динамическим обновлением интерфейса и поиском, модальными окнами, автосохранением, интеграциями с внешними системами, модулями машинного обучения, текстовым анализом, фоновыми задачами и автоматическим деплоем. Объём кода составил около 150 000 строк, из которых примерно 120 000 написаны на Rust, а остальные - на TypeScript и Terraform. Весь этот код сгенерировали агенты - в основном Claude Code и частично Cursor . За редкими исключениями я почти не открывал и не читал исходные файлы. В процессе разработки я начал замечать странности. Когда в терминале мелькнула правка 4000-й строки в одном файле, я решил посмотреть на код ближе. Выяснилось, что слой доступа к данным разросся до 6000 строк. С каждой новой функцией он продолжал расти. В коде каждого запроса, чтения или записи повторялись настройка HTTP-запроса, кодирование и декодирование JSON. В итоге весь слой доступа к данным оказался в одном файле на 17 155 строк Rust.

https://habr.com/ru/articles/1065178/

#refactoring #ииагенты #ai #рефакторинг #разработка_приложений #разработка #software_engineering #software_architecture #бюджет #code_review

The Economic Benefit of Refactoring, by @mfowler:

https://martinfowler.com/articles/exploring-gen-ai/refactoring-economic-benefit.html?ref=frontenddogma.com

#refactoring #ai #economics

The Economic Benefit of Refactoring

https://martinfowler.com/articles/exploring-gen-ai/refactoring-economic-benefit.html

Comments: https://news.ycombinator.com/item?id=49111176

#HackerNews #EconomicBenefit #Refactoring #TechDebt #SoftwareDevelopment #MartinFowler

How we migrated a live routing system using #AI-assisted #refactoring https://www.datadoghq.com/blog/engineering/ai-assisted-storage-migration/

Rewriting Bun in Rust, by @jarredsumner (@bun.sh):

https://bun.com/blog/bun-in-rust?ref=frontenddogma.com

#refactoring #bun #rust