In-place mutation is the quietest part of a module's contract. Most characterization tests assert what a function returns. Almost none assert that the caller's list changed underneath. Replace rows.sort() with rows = sorted(rows) and the suite stays green. Production still reads the old list, in the old order.

This is the seam that survives a copy-versus-mutate refactor. Freeze it before you touch the body.

1. The break is aliasing, not logic

A caller passes a list. The function appends to it and returns None. The caller never reads the return value; it reads its own variable later. That is a real contract, and it lives outside your assertions.

Now a refactor returns a fresh list. Return-value tests pass. The caller keeps its stale object. Nothing raises. No error is logged.

The damage is silent because the signature did not change. So the gate must record mutations, not just outputs.

2. Build a mutation ledger

The probe below wraps a function. It deep-snapshots every argument before the call. It diffs each argument after the call and appends one JSON line per invocation.

# mutation_probe.py
import copy, functools, json

def _snapshot(value):
try:
return copy.deepcopy(value)
except Exception:
return repr(value)

def _diff(before, after, path="$"):
out =
[] if type(before) is not type(after):
return [{"path": path, "change": "type",
"before": type(before).__name__, "after": type(after).__name__}]
if isinstance(before, dict):
for key in sorted(set(before) | set(after), key=str):
if key not in before:
out.append({"path": f"{path}[{key!r}]", "change": "added"})
elif key not in after:
out.append({"path": f"{path}[{key!r}]", "change": "removed"})
else:
out.extend(_diff(before[key], after[key], f"{path}[{key!r}]"))
return out
if isinstance(before, list):
if len(before) != len(after):
out.append({"path": path, "change": "length",
"before": len(before), "after": len(after)})
for i, (b, a) in enumerate(zip(before, after)):
out.extend(_diff(b, a, f"{path}[{i}]"))
return out
if before != after:
out.append({"path": path, "change": "value", "before": before, "after": after})
return out

def freeze_mutations(fn, ledger_path):
@functools.wraps(fn)
def wrapper(*args, **kwargs):
before = {f"arg{i}": _snapshot(v) for i, v in enumerate(args)}
before.update({f"kw:{k}": _snapshot(v) for k, v in kwargs.items()})
result = fn(*args, **kwargs)
after = {f"arg{i}": _snapshot(v) for i, v in enumerate(args)}
after.update({f"kw:{k}": _snapshot(v) for k, v in kwargs.items()})
record = {"function": fn.__qualname__, "mutations": []}
for name in before:
record["mutations"].extend(_diff(before[name], after[name], name))
with open(ledger_path, "a", encoding="utf-8") as fh:
fh.write(json.dumps(record, sort_keys=True, default=repr) + "\n")
return result
return wrapper

Two notes before you run it. Snapshots cost memory, so target narrow functions. And default=repr keeps unserializable payloads from killing the run.

3. Capture against real call shapes

Tests alone give you a weak baseline. They exercise the arguments someone remembered to write.

1. List candidate functions with grep -rn "\.append(\|\.update(\|\.sort(\|\.pop(" src/.
2. Wrap the top candidates with freeze_mutations(fn, "ledger-before.jsonl").
3. Drive them with recorded payload fixtures, not synthetic ones.
4. Run the suite twice and require identical ledgers, sorted by (function, path, change).
5. Commit ledger-before.jsonl as the baseline artifact.

Step 4 matters. A ledger that differs between runs is noise, not a contract. Fix the noise before you trust the diff.

4. Turn the ledger into a gate

The gate is a set comparison. It does not care about ordering across lines. It cares about which mutation paths exist.

# mutation_gate.py
import json, sys

def keys(path):
seen = set()
with open(path, encoding="utf-8") as fh:
for line in fh:
rec = json.loads(line)
for m in rec["mutations"]:
seen.add((rec["function"], m["path"], m["change"]))
return seen

baseline, current = keys(sys.argv[1]), keys(sys.argv[2])
lost, new = baseline - current, current - baseline
for tag, rows in (("LOST", lost), ("NEW", new)):
for fn_name, path, change in sorted(rows):
print(f"{tag}\t{fn_name}\t{path}\t{change}")
sys.exit(1 if lost or new else 0)

A LOST line is the dangerous one. A mutation that used to happen no longer happens. A NEW line is a contract you just invented.

5. Make the smallest safe change

One function per commit. Never two.

1. Refactor a single function to return a new object.
2. Re-run the probe into ledger-after.jsonl.
3. Run the gate against the baseline.
4. If LOST is non-empty, revert the copy and keep in-place mutation.
5. If only NEW appears, delete it and re-check.

The revert in step 4 is not a failure. It is the gate doing its job.

6. Decide per mutation, not per module

Observed mutation

Caller relies on it?

Action

Appends to a passed-in list

Yes, verified by reader code

Keep; document the contract in a docstring

Sorts a passed-in list

Unclear

Keep; add a test that asserts the caller's list order

Mutates a config dict

No

Convert to copy; expect LOST and accept it explicitly

Mutates an argument, returns same object

Yes

Convert only if you also update every reader

Mutates across a thread boundary

Unknown

Stop; the probe cannot see the race

Run that table before you run the refactor. It converts a judgement call into a recorded decision.

7. Where a disposable box helps

Capture runs dirty the environment. They write ledgers, load fixtures, and sometimes touch a scratch database. Running them on your working checkout is friction you do not need.

MonkeyCode's free server option covers that part: one throwaway box for the capture run. Its free model access covers the drafting step, where a raw ledger becomes readable assertion names instead of JSON noise. Both are operator-supplied availability claims; check current limits yourself before planning around them.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Keep the gate in your own CI. Generated assertion names must be read against the ledger line by line. A model will happily invent a path that never mutated.

8. Limitations, and who should skip this

• List indices shift. A path like arg0[2].id moves when the list resizes. Normalize indices out before you compare across branches.
• Opaque objects leak. Deep copies of database handles, sockets, or C extensions often fail, and repr fallback hides the mutation.
• Threads and processes are invisible. The probe sees one call, one thread, one moment.
• Side effects outside arguments are out of scope. Writes, network calls, and file handles need a separate gate.
• Deep snapshot is slow. Do not wrap hot paths, and do not wrap loops that run thousands of times per request.

Skip this approach if your code already treats data as immutable. Skip it if the object graph is too large to copy. Skip it if the mutation you fear lives in another process.

The rule

Asserting return values proves half the contract. Freeze what the function did to the arguments, or the quietest break ships.#python #testing #refactoring #codequality #software #coding #development #engineering #inclusive #community
Your Refactor Copies a List. The Caller Still Holds the Old One.

進入許願式編程代碼重構的方法還是要了解的，我相信AI寫代碼都同樣會用到

https://refactoring.guru/refactoring

#Refactoring #Programming

Return values are the easy part of a refactor. Side-effect order is where messy repos break quietly.

A green suite can still merge a file that now gets written twice. It can still ship a temp file that is deleted before its reader runs. Characterization tests pin outputs. They rarely pin sequence.

This post builds a small ledger that records ordered side effects. You capture it before the refactor. You diff it after the smallest safe change.

The failure a return-value test cannot see

Here is a real shape. A nightly exporter writes a CSV, renames it into place, then shells out to a mailer.

Your test asserts the returned path and the row count. Both stay correct while the internal order flips.

# Before: write -> replace -> notify
df.to_csv(tmp)
os.replace(tmp, final)
subprocess.run(["mailer", final], check=True)

# After: notify -> write -> replace (still returns the same path)

The test passes. The mailer reads a stale file. Nobody sees it until a customer asks where the report went.

Step 1 — Name one scenario, not the whole repo

One entry point, fixed inputs, no network. Keep it boring.

# scenarios/nightly_export.py
from reports.exporter import build_nightly

def run():
build_nightly(dest="/tmp/export-check")

Do not try to ledger the whole package on day one. A scenario that touches ten modules gives you a diff you cannot read. Start where the change will land.

Step 2 — List candidates with grep, not with a hunch

grep -rnE "open\(|\.write\(|os\.replace|shutil\.(move|copy)|subprocess\.|requests\.(post|put)" reports/ \
| grep -v tests/

You want the boundaries the scenario actually reaches. Grep gives you candidates, not proof. Confirm each hit by running the scenario once and printing the files it touched.

Step 3 — Capture an ordered ledger

Sixty lines. Patch a few boundaries, record every call in arrival order, normalize volatile strings.

# effect_ledger.py — ordered, normalized side-effect capture
import builtins
import hashlib
import json
import os
import re
import subprocess
from contextlib import contextmanager
from unittest import mock

LEDGER =

[]_TMP = re.compile(r"/tmp/[A-Za-z0-9_.\-]+")
_TS = re.compile(r"\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}")

def _norm(value):
text = str(value)
return _TS.sub("<TS>", _TMP.sub("<TMP>", text))

def _record(kind, name, payload=None):
LEDGER.append({
"seq": len(LEDGER),
"kind": kind,
"name": _norm(name),
"payload": None if payload is None else _norm(payload),
})

class _WriteTracer:
"""Records a content hash when the handle closes."""

def __init__(self, handle, path):
self._h, self._path = handle, path
self._sha = hashlib.sha256()
self._closed = False

def write(self, data):
self._sha.update(data.encode() if isinstance(data, str) else data)
return self._h.write(data)

def close(self):
if not self._closed:
self._closed = True
_record("write.close", self._path, self._sha.hexdigest()[:16])
return self._h.close()

def __enter__(self):
return self

def __exit__(self, *exc):
self.close()
return False

def __getattr__(self, name):
return getattr(self._h, name)

@contextmanager
def ledgered():
LEDGER.clear()
real_open, real_replace, real_run = builtins.open, os.replace, subprocess.run

def open_hook(file, mode="r", *args, **kwargs):
handle = real_open(file, mode, *args, **kwargs)
if set(mode) & set("wax+"):
_record("write.open", file, mode)
return _WriteTracer(handle, file)
return handle

def replace_hook(src, dst, *args, **kwargs):
_record("os.replace", dst, f"from={src}")
return real_replace(src, dst, *args, **kwargs)

def run_hook(*args, **kwargs):
cmd = args[0] if args else kwargs.get("args")
_record("subprocess.run", cmd)
return real_run(*args, **kwargs)

with mock.patch("builtins.open", open_hook), \
mock.patch("os.replace", replace_hook), \
mock.patch("subprocess.run", run_hook):
yield LEDGER

def dump(path):
payload = json.dumps(LEDGER, indent=2)
with builtins.open(path, "w", encoding="utf-8") as fh:
fh.write(payload)

Note the tracer. A path name tells you a file was opened. A hash tells you it held what you expected.

Step 4 — Prove the baseline is stable before you trust it

Run the same scenario three times under different hash seeds.

for seed in 0 1 2; do
PYTHONHASHSEED=$seed python capture_ledger.py nightly_export "ledgers/base-$seed.json"
done
python diff_ledger.py ledgers/base-0.json ledgers/base-1.json
python diff_ledger.py ledgers/base-0.json ledgers/base-2.json

If those diffs are non-empty, stop. Your scenario is nondeterministic and the gate is worthless.

Sort directory listings. Freeze the clock. Pin the temp prefix. Only then record a baseline.

Step 5 — Make the smallest safe change

Extract one block into a helper. Rename one variable. Change no behavior.

python capture_ledger.py nightly_export ledgers/after.json
python diff_ledger.py ledgers/base-0.json ledgers/after.json

Exit code 1 means the effect sequence moved. Exit code 0 means this scenario saw the same effects in the same order.

That is a narrow claim. Keep it narrow and it stays trustworthy.

Step 6 — Read the diff as a decision, not an alarm

Ledger diff

Interpretation

Action

Identical sequence

Change looks behavior-preserving for this path

Commit; keep baseline

New effect, expected

Refactor introduced a deliberate boundary

Update baseline in the same commit

New effect, unexpected

Hidden coupling or double write

Revert; read the code

Missing effect

A write or notify silently dropped

Revert

Unstable baseline

Scenario has ordering or timing leaks

Fix determinism first

Commit before.json next to the harness. A baseline you cannot diff later is just a log file.

What this ledger will not catch

• Threads and asyncio. Interleaving order is not what you recorded.
• C extensions and direct syscalls below open and subprocess.
• Effects on boundaries you did not patch, such as a database driver or an SDK client.
• Paths your single scenario never executes. Coverage is still your job.

It also cannot tell you whether the new order is correct. Only whether it changed.

Who should skip this

If you already have an integration suite that exercises the real boundary, use it. This harness exists for repos with no harness at all.

Skip it for a rewrite. A ledger gate protects mechanical moves, not redesigned flows. Skip it for concurrency-heavy code, where the sequence is genuinely not fixed.

Where a free model and a free server fit

Two places, both modest.

First, boundary discovery. Paste a module's imports into a free model and ask for candidate side-effect boundaries. Verify every candidate with grep, because models miss dynamic dispatch and mixins.

Second, isolation. Run the seed loop on a disposable Linux box instead of your laptop. The free server option covers a capture run for a small repo; it is not a reason to move your whole build there.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Treat free model access and the free server option as the operator's claims. I did not benchmark throughput, quotas, or hardware here, and none of the numbers above came from them.

Start with one scenario

One entry point, one boundary list, one baseline, one extraction. Then diff.

If you try this, draft the scenario and the boundary list first. The free tier is enough for both, and the diff will tell you the truth.#python #testing #refactoring #productivity #software #coding #development #engineering #inclusive #community
Your Characterization Tests Passed. The Side Effects Changed Order.

The mark of great systems engineering: modules that are trivial to replace or delete. When you couple your orchestration logic tightly to a specific cloud vendor's proprietary SDK, you inherit all their pricing shifts and downtime.

Keep your core orchestration vendor-agnostic standard library Python.

#SoftwareArchitecture #Python #OpenSource #Refactoring

Modernizzare un applicazione PHP legacy senza riscrivere tutto

Strategie concrete per migliorare un progetto PHP datato: refactoring incrementale, test e aggiornamento graduale.

Continua a leggere su iadicola.it: https://iadicola.it/articoli/modernizzare-applicazione-php-legacy-senza-riscrivere?utm_source=mastodon&utm_medium=social&utm_content=carousel-135

#PHP #Refactoring #Freelance #Legacy

Alt...

Modernizzare un'applicazione PHP legacy senza riscrivere tutto — slide 1

Alt...

Modernizzare un'applicazione PHP legacy senza riscrivere tutto — slide 2

Alt...

Modernizzare un'applicazione PHP legacy senza riscrivere tutto — slide 3

Alt...

Modernizzare un'applicazione PHP legacy senza riscrivere tutto — slide 4

I love it when the PR has twice that many lines removed compared to the new ones.

#refactoring

CWE-477: Use of Obsolete Function

"The code uses deprecated or obsolete functions, which suggests that the code has not been actively reviewed or maintained."

I am using scanners to get deprecation warnings, so i am not unpleasently surprised one day. How do you handle that?

#security #itsec #infosec #itsecurity #tech #development #awareness #dailyCWE #deprecation #legacycode #codequality #refactoring

https://cwe.mitre.org/data/definitions/477.html

Stop letting bloated plugins slow down your application. Here is why you need a quarterly dependency audit. Read more at klytron.com.

#klytron #CodingTips #SoftwareDeveloper #CleanCode #Refactoring #DevLife

I somehow managed to execute the main scene, but the "refactoring" introduced tons of bugs.

- Enemy doesn't move
- Player slides left
- Attacking forces Player to face left

Enemy and Player share the same state logic, and I broke it.

#Godot #Gamedev #Pixelart #refactoring

The gate is simple: pin behavior on inputs you can see, then grade the refactor on inputs you cannot. A refactor that passes the visible corpus and fails the hidden one was never safe. It was fitted.

Most characterization suites rot this way. You write tests against the three files you have open. The model, or you, then patches code until those three pass. Nothing proves the other forty inputs still behave.

This post builds a fingerprint harness with a canary/holdout split. Then it sets a rule for the smallest change that can honestly ship.

1. Define the behavior surface before you touch code

A refactor preserves observable behavior. So you must name what is observable, or the word means nothing.

For a CLI-shaped tool, four surfaces cover most of it:

Surface

Captured as

Catches

Exit code

integer

swallowed exceptions, new early returns

stdout bytes

SHA-256

reordered fields, changed rounding, lost whitespace

stderr presence

boolean

new warnings, debug prints left behind

files written

path + SHA-256 list

changed output paths, dropped side files

Deliberately excluded: internal call order, log timings, object identity. Those change under every refactor. Pinning them buys you failing tests and no safety.

Write that list down first. Every later argument about "did we break it" resolves against this table, not against memory.

2. Split the corpus before you look at results

Take your input corpus. Sort the filenames. Give every third case to the holdout set. The rule is deterministic, so it is reproducible, and it is chosen before any code changes.

corpus/
case_000/input.txt
case_001/input.txt
...

Canary gets two-thirds. Holdout gets one-third. The holdout stays closed until the refactor is finished. Not after step three. After the last commit.

Why one-third and not ten percent? Because one-third of thirty cases is ten, and ten is enough to see a pattern failure. One or two holdout cases can fail by coincidence and teach you nothing.

If you have fewer than twelve cases, stop. Do not refactor. Collect inputs first.

3. Record the baseline on the unmodified tree

Run the harness on a clean checkout. Commit the JSON. This file is the contract.

#!/usr/bin/env python3
"""fingerprint.py <repo> <corpus> <out.json> - pin observable behavior."""
import hashlib, json, subprocess, sys, tempfile
from pathlib import Path

ROOT = Path(sys.argv[1]).resolve()
CORPUS = Path(sys.argv[2]).resolve()
OUT = Path(sys.argv[3]).resolve()
TIMEOUT = 30

def sha(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()

def fingerprint(case: Path) -> dict:
work = Path(tempfile.mkdtemp(prefix="fp-"))
cmd = [sys.executable, str(ROOT / "tool.py"), str(case / "input.txt")]
try:
proc = subprocess.run(cmd, capture_output=True, cwd=work, timeout=TIMEOUT)
except subprocess.TimeoutExpired:
return {"timeout": True}
files = sorted(
[p.relative_to(work).as_posix(), sha(p.read_bytes())]
for p in work.rglob("*")
if p.is_file()
)
return {
"exit": proc.returncode,
"stdout": sha(proc.stdout),
"stderr_present": bool(proc.stderr.strip()),
"files": files,
}

def split(cases: list) -> dict:
return {
"canary": [c for i, c in enumerate(cases) if i % 3 != 0],
"holdout": [c for i, c in enumerate(cases) if i % 3 == 0],
}

cases = sorted(p for p in CORPUS.iterdir() if (p / "input.txt").is_file())
result = {name: {c.name: fingerprint(c) for c in group}
for name, group in split(cases).items()}
OUT.write_text(json.dumps(result, indent=2, sort_keys=True))
print(f"canary={len(result['canary'])} holdout={len(result['holdout'])}")

The script is short on purpose. It has no dependencies, no fixtures, and no framework. You can run it in CI or on a laptop.

Run it twice on the untouched tree. If the two JSON files differ, you have nondeterminism. Fix that before any refactor, or every later failure is ambiguous.

4. Set the smallest safe change rule

The point of a gate is to constrain scope, not to bless ambition. Use the behavior surface table as a budget.

Change touches

Allowed in one commit

Evidence required

One function body, no signature

yes

canary + holdout match

One function signature, internal callers only

yes

callers updated in same commit

Module moved to a new path

no

separate commit, imports only

Output ordering or formatting

no

explicit human sign-off

Public function signature

no

not a refactor; it is a redesign

Row five is the one people argue about. A signature change is a behavior change by definition. Ship it as its own reviewed change with its own tests, not folded into a cleanup.

5. Replay, then read the two numbers separately

After the change, run the harness again. Compare per split, never in aggregate.

import json, sys

base = json.load(open(sys.argv[1]))
now = json.load(open(sys.argv[2]))

exit_code = 0
for name in ("canary", "holdout"):
drifted = [k for k, v in base[name].items() if now[name].get(k) != v]
print(f"{name}: {len(drifted)}/{len(base[name])} drifted")
for k in drifted[:5]:
print(f" - {k}")
exit_code |= bool(drifted)

sys.exit(exit_code)

Three outcomes matter.

1. Canary 0, holdout 0. The change is behavior-preserving on this corpus. Commit it.
2. Canary 0, holdout > 0. The change is wrong. The corpus was broad; the tests were narrow. Revert the commit, do not patch the harness.
3. Canary > 0. You already know what to do. Inspect the diff before touching a single assertion.

Outcome two is the one that saves you. Without a holdout split, it looks exactly like outcome one.

6. Apply the revert rule, not the negotiation rule

When holdout drifts, the instinct is to update the expected values. Resist it once.

Holdout drift cause

Action

Renamed output key

revert; rename in a separate commit

Changed float formatting

revert; formatting is a behavior change

Dropped file write

revert; a side effect disappeared

Genuine pre-existing bug

revert, then fix bug separately, then re-baseline

Corpus case is invalid input

remove the case, re-baseline on clean tree

Only the last row permits touching the baseline. And it requires re-running step three on an unmodified tree, so the contract stays honest.

Where free model access fits, and where it does not

The harness above is mechanical. The slow part is the corpus and the first draft of the fingerprint script — and that is where a free model tier is genuinely useful. I used MonkeyCode's free model access to draft the harness and to propose corpus edge cases, then read every line before running it. The operator also describes a free server option, which matters here for one reason: the replay must run on a clean machine, not your laptop with a stale virtualenv. A throwaway environment makes step three reproducible.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Use the free tooling for the disposable parts. Do not let a model decide which behavior counts as observable. That table is a design decision, and it is yours.

On availability: the operator states a free token allowance in the millions-per-account range, plus the free server option. Quotas and hardware change without notice. Verify current terms on the project page before you plan around them.

Limitations and who should skip this

• The fingerprint is byte-level. It will flag cosmetic changes you intended. That is the price of catching the ones you did not.
• Thirty cases is a floor, not a target. Real tools need cases drawn from production traffic, not from your imagination.
• Timing, memory, and concurrency are not covered. This gate says nothing about performance regressions.
• Skip this if your tool has no stable text output. A UI-heavy service needs snapshot diffs of rendered state instead.
• Skip this if the refactor changes the public contract. Then you want migration tests, not a preservation gate.

The short version

Pick four observable surfaces. Sort your corpus and hold out every third case. Baseline on a clean tree, twice. Change one thing. Replay both splits. If the holdout moves, revert.

The hidden third is the only honest reviewer you get for free. If you try the harness, keep the corpus small and the rule strict at first — and tell me which holdout case caught you.

Further reading: Martin Fowler on characterization tests, and Michael Feathers, "Working Effectively with Legacy Code", chapter 13.#refactoring #testing #python #ai #software #coding #development #engineering #inclusive #community
Hold Out a Third of Your Corpus: The Characterization Gate for a Messy Refactor

Most refactors do not break because tests are missing. They break because the tests pin the wrong surface. Model-drafted tests make this worse, not better: they are fast, plausible, and confident about internals.

My rule for messy repos is simple. Draft many candidate tests. Admit only three-gate survivors. Then make the smallest safe change.

The three gates are determinism, assertion surface, and sentinel failure. A candidate test must pass all three before it can guard a refactor. A test that cannot fail is decoration.

Below is the workflow I use, with a harness sketch you can run before trusting any thresholds.

Step 0: Record one observed baseline

Before writing a single test, capture what the code does today. Run the entrypoint once and store the raw output.

python -m yourapp.report --account 42 > baseline/report.json
sha256sum baseline/report.json

That hash is your reference point for the whole session. If the refactor changes the bytes, you will know. Not "roughly the same JSON" — the same bytes, or an explained diff.

Keep this file out of version control churn. You want diff noise to mean something.

Step 1: Draft candidates with a free model

This is where free model access helps most. Drafting candidate characterization tests is volume work, and volume is cheap. Judgment is not.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Use a prompt contract, not a vague request. The contract below forces the model onto the observable surface and demands observed values.

You are drafting characterization tests for an unchanged module.
Rules:
1. Import only the public API listed below. Never touch names starting with _.
2. Each test performs one call and asserts one observable value or exact output string.
3. Expected values must come from the observed run I pasted. Do not guess.
4. No mocks of the module under test. Patch only network and clock.
5. Name each test after the behavior, not the function: test_rounds_discount_to_cents.
Public API: calculate_invoice, render_report
Observed run: <paste baseline output>

Ask for fifteen candidates, expect eight usable ones. Reject anything that asserts on a private attribute, a log line, or an exception message you did not paste.

Run the gate script on the candidates before you read them closely. Reading first biases you toward keeping the clever ones.

Step 2: Determinism gate

A characterization test that flips on its own is worse than no test. It teaches the team to ignore red.

Run every candidate twenty times with a different PYTHONHASHSEED. Require twenty passes.

# tools/gate_candidate_tests.py (sketch, tighten thresholds for your repo)
import os, subprocess, sys
from pathlib import Path

RUNS = 20

def run_pytest(target: Path, seed: str) -> int:
env = {**os.environ, "PYTHONHASHSEED": seed}
cmd = [sys.executable, "-m", "pytest", "-q", "-p", "no:cacheprovider", str(target)]
return subprocess.run(cmd, env=env, capture_output=True).returncode

def determinism_gate(test_file: Path, runs: int = RUNS) -> bool:
failures = [n for n in range(runs) if run_pytest(test_file, str(n)) != 0]
print(f"determinism: {runs - len(failures)}/{runs} pass")
return not failures

Twenty runs is a floor, not a proof. It catches hash-order dependence, unseeded randomness, and real clock reads. It will not catch a branch that fires once a month.

Use a free server for this step if your laptop sleeps. A gate that dies at run nine and reports "unknown" is a gate you will skip next time. Any always-on box works; the point is an uninterrupted twenty-run window.

Step 3: Surface gate

Now check what the assertions actually touch. Parse the test file and flag private attribute access.

import ast

def surface_violations(test_file: Path) -> list[str]:
tree = ast.parse(test_file.read_text())
bad =
[] for node in ast.walk(tree):
if isinstance(node, ast.Attribute) and node.attr.startswith("_"):
bad.append(f"line {node.lineno}: private attribute .{node.attr}")
return bad

This is syntactic, so it misses indirect access through helpers. Treat a clean report as permission to look, not proof of safety.

Two more manual checks matter. Does the test assert on a log message? Logs are not API. Does it assert on an exception type without a pinned message? That is fine for control flow, weak for refactors.

Prefer assertions on returned data and rendered output. Those survive most internal moves.

Step 4: Sentinel gate

The most important check. A characterization test must fail when behavior changes.

Write a sentinel patch: one deliberate, tiny behavior change. Apply it in a throwaway clone. The test must go red.

--- a/pricing.py
+++ b/pricing.py
@@
- return round(subtotal * (1 - rate), 2)
+ return round(subtotal * (1 - rate), 1)

import shutil, tempfile

def sentinel_gate(repo: Path, test_file: Path, sentinel: Path) -> bool:
with tempfile.TemporaryDirectory() as tmp:
work = Path(tmp) / "repo"
shutil.copytree(repo, work, ignore=shutil.ignore_patterns(".git"))
applied = subprocess.run(["git", "apply", "--unsafe-paths", str(sentinel)],
cwd=work, capture_output=True)
if applied.returncode != 0:
print("sentinel patch did not apply; fix the sentinel first")
return False
red = run_pytest(work / test_file.relative_to(repo), "0") != 0
print(f"sentinel: {'caught the change' if red else 'DECORATION'}")
return red

One sentinel per behavior you claim to protect. A test that stays green after a rounding change protects nothing about rounding.

Step 5: Make the smallest safe change

Only survivors guard the edit. Then the refactor rule applies: one edit, one full suite run.

Green suite plus unchanged baseline hash means you moved code, not behavior. Red suite means you learned something cheap.

Keep the change smaller than your confidence. If a test goes red, do not edit the test to match the new code. That is how refactors turn into silent behavior changes nobody reviews.

Decision table

Gate result

Action

Determinism fails

Quarantine; find the nondeterministic source first

Private attribute asserted

Rewrite against returned data

Passes all but sentinel stays green

Delete the test; it protects nothing

Passes all three

Keep; this one guards the edit

Red after a legitimate fix

Remove the test; characterization is not a spec

That last row matters. Characterization tests freeze current behavior, bugs included. If the frozen behavior is a real defect, file it and delete the test.

Limitations and who should skip this

• The sentinel gate only catches changes your sentinel touches. Choose sentinels per behavior, not one global patch.
• Twenty runs cannot find rare-path flakiness. Shorten the interval or widen the run count for hot paths.
• The AST check is syntactic. Helper-mediated private access passes it.
• Model-drafted tests encode the model's reading of your output. Paste raw observations, not summaries.
• Skip this if the target module has no runnable entrypoint. Build one first; gates need something to run.
• Skip it for a module you are actively rewriting anyway. Gates pay off on code you intend to keep.

If you want to try the drafting step without spending anything, run the prompt contract against a free model, then let the three gates decide what survives. The gates are the part worth keeping.#testing #refactoring #python #ai #software #coding #development #engineering #inclusive #community
A Test That Cannot Fail Is Decoration: Gating Model-Drafted Characterization Tests

A green characterization suite is not evidence of safety. It is a claim. Verify the claim before you cut anything.

Most refactors fail in the same way. The pins pass before the change. The pins pass after the change. Then a downstream job breaks, because the pins never actually watched the behavior you moved. You need a score for the net itself.

This post covers one measurable gate: mutation-check the pins first, then make the smallest safe change. Earlier posts in this series built the pins. This one asks whether they can fail.

Two ways a pin suite lies to you

The blind pin. It asserts something stable and irrelevant. assert result is not None passes forever, including after you delete half the report.

The tautological pin. It copies the implementation into the test. It imports private helpers and asserts the same call order the source already has. It fails when you rename, not when you break.

Both produce green CI. Neither produces a safety net. Coverage numbers will not tell you which one you have.

The artifact: a pin sensitivity score

This harness applies small text-level mutations to the source, then runs the pin suite against each mutant. A good net catches most mutants. A decorative net catches none.

Save as pin_check.py. Exit code 0 means the score cleared your threshold.

#!/usr/bin/env python3
"""Measure characterization-pin sensitivity with text-level mutations.

Usage:
python pin_check.py --repo . --min-score 0.6 \
--pin-cmd "python -m pytest -q tests/pins"

Exit codes: 0 = score >= min-score, 1 = below threshold, 2 = harness error.
"""
from __future__ import annotations

import argparse
import json
import shutil
import subprocess
import sys
import tempfile
from dataclasses import dataclass, asdict
from pathlib import Path

@dataclass(frozen=True)
class Mutation:
name: str
rel_path: str
old: str
new: str

# One row per behavior you believe the pins protect.
# Text-level on purpose: cheap, reviewable, no AST tooling required.
MUTATIONS: tuple[Mutation, ...] = (
Mutation("currency_default", "billing/invoice.py",
'currency: str = "USD"', 'currency: str = "EUR"'),
Mutation("round_precision", "billing/invoice.py",
"round(total, 2)", "round(total, 1)"),
Mutation("skip_none_rows", "billing/report.py",
"if row is None:", "if not row:"),
Mutation("sort_key", "billing/report.py",
"key=lambda r: r.date", "key=lambda r: r.amount"),
Mutation("header_order", "billing/report.py",
'["id", "date", "amount"]', '["date", "id", "amount"]'),
)

IGNORE = shutil.ignore_patterns(".git", "__pycache__", "*.pyc", ".venv")

def run(cmd: list[str], cwd: Path) -> int:
proc = subprocess.run(cmd, cwd=cwd, capture_output=True, text=True)
return proc.returncode

def apply_mutation(root: Path, mut: Mutation) -> bool:
target = root / mut.rel_path
if not target.exists():
return False
src = target.read_text()
if mut.old not in src:
return False # stale pattern: refresh this row
target.write_text(src.replace(mut.old, mut.new, 1))
return True

def check(repo: Path, pin_cmd: list[str]) -> dict:
baseline_exit = run(pin_cmd, repo) # pins must be green on pristine tree
caught = applied = 0
detail: list[dict] =

[] for mut in MUTATIONS:
with tempfile.TemporaryDirectory() as tmp:
work = Path(tmp) / "repo"
shutil.copytree(repo, work, ignore=IGNORE)
if not apply_mutation(work, mut):
detail.append({"mutation": mut.name, "status": "stale-pattern"})
continue
applied += 1
hit = run(pin_cmd, work) != 0
caught += int(hit)
detail.append({"mutation": mut.name,
"status": "caught" if hit else "missed"})

score = caught / applied if applied else 0.0
return {"baseline_exit": baseline_exit, "applied": applied, "caught": caught,
"score": round(score, 2), "detail": detail}

def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--repo", default=".")
ap.add_argument("--pin-cmd", required=True)
ap.add_argument("--min-score", type=float, default=0.6)
args = ap.parse_args()

report = check(Path(args.repo).resolve(), args.pin_cmd.split())
print(json.dumps(report, indent=2))

if report["baseline_exit"] != 0:
print("harness error: pins already fail on the pristine tree", file=sys.stderr)
return 2
return 0 if report["score"] >= args.min_score else 1

if __name__ == "__main__":
raise SystemExit(main())

The mutation list is the actual deliverable. Each row is a belief you are testing. Add a row every time a reviewer asks "but does this break X?"

Step 1: Confirm the pristine tree is green

Run the pins alone first. A failing baseline invalidates the whole measurement.

python -m pytest -q tests/pins # must exit 0
python pin_check.py --repo . --min-score 0.6 \
--pin-cmd "python -m pytest -q tests/pins"

Expect baseline_exit: 0. Expect a detail array. Expect missed rows. The misses are the point.

Step 2: Read the score as a gate, not a grade

The number is a threshold check. Use it to decide whether to start, not to celebrate.

Score

Reading

Action

0.0 - 0.2

Pins are decorative

Do not refactor. Pin public outputs of the entry points first.

0.3 - 0.5

Blink spots cluster

Open the missed rows. Add one output-level pin per cluster.

0.6 - 0.8

Usable for a bounded change

Proceed with one behavior-preserving step only.

0.9 - 1.0

Possibly over-pinned

Grep for private imports. Trim pins that fail on rename.

A high score is not automatically good news. Suites that pin every internal call also score well. They just cost you every future rename.

Step 3: Probe for tautological pins

Pins that import private names rarely specify behavior. They duplicate it.

# Suspects: pins reaching into underscore-prefixed helpers.
grep -rnE "^\s*from [A-Za-z_.]+ import _" tests/pins | tee tautology-suspects.txt
wc -l < tautology-suspects.txt

Review each hit. Ask one question: does this assertion describe output, or restate the source? If it restates, rewrite it against the public boundary.

Step 4: Make the smallest safe change

One behavior-preserving step. One commit. No drive-by cleanups, no renames.

git switch -c refactor/invoice-totals
# single change here only
python -m pytest -q tests/pins # green after the change
git stash
python -m pytest -q tests/pins # green before it too
git stash pop

If the pins pass only on the changed tree, they were tailored to the change. That is a rewrite of the specification, not a verification of it.

Where free tooling fits

The mutation list is the slow part. Someone has to guess which behaviors matter. I use MonkeyCode's free model access to draft candidate rows from a module's recent diff history, then delete most of them by hand. The candidates are proposals only, and every row still needs a human to confirm it describes real behavior. Its free server option is useful when the pin suite is too heavy for a laptop and CI minutes are budgeted elsewhere.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Treat availability, limits, and model lineups as operator-supplied and changeable. Verify current terms yourself. Do not put a free tier on a release-critical path.

Limitations and who should skip this

Text-level mutations are crude. They catch only a pattern that still exists verbatim. Rename a variable and the row silently becomes stale-pattern. Refresh rows when you touch the file.

A high score does not prove correctness. It proves the net can fail on the mutations you thought of. Semantic drift outside your list stays invisible.

If you already run a real mutation framework, use it. Tools like mutmut or cosmic-ray generate far better mutants than a hand-written list. This harness is for repos where adding one more test dependency is politically harder than adding one script.

Skip this if the output has no stable entry point, or if nobody consumes the output. Skip it for throwaway scripts. The score only matters where a regression has a cost.

Start with the misses

Run the harness on your messiest module and post the missed list. The score is a gate. The misses are the map.

One behavior-preserving change, one verified net. In that order.#testing #refactoring #python #cicd #software #coding #development #engineering #inclusive #community
Test the Safety Net First: Mutation-Check Characterization Pins Before a Refactor

A late cleanup request arrived with a four-hundred-line diff that rewrote validation, deduplication, and persistence in one pass. The existing tests covered only the happy path, so a green CI run could not explain the legacy rows. The reviewer needed a smaller change that preserved every observed output before any structural boundary was allowed to move. This composite scenario shows how to freeze those outputs, then accept only the smallest safe edit.

The module under review is a batch upload normalizer that grew by successive tickets rather than by design. It reads a list of records, fills missing defaults, drops exact repeats, and writes survivors to storage. Those four jobs share one function, so a cleanup that touches all of them cannot be reviewed as one claim. The method below separates observation from edit, and it does not depend on any particular assistant.

Name the behavior you will freeze

Before changing code, write down the inputs that already matter and the outputs you are willing to treat as current truth. Include awkward rows from production samples, not only the tidy examples attached to the latest ticket. A characterization file is a pinned record of those pairs, not a statement that the behavior is desirable. If a row is known to be wrong, mark it as accepted debt instead of silently correcting it during the same change.

Three observations keep the freeze honest when the repository is already messy.

• Record the returned list and the store payload, because a matching return can still hide a second write.
• Preserve the original row order, because a sort that looks cleaner can change which duplicate survives.
• Keep skipped rows visible in the fixture notes, because a deleted branch is still a behavior change.

That list is the review contract for this module, and it should stay visible in the pull request description. It is narrower than a repository-wide characterization suite, and it is not a license to rewrite neighboring helpers. This pass does not hash a repository-wide side-effect ledger, and it does not ask you to freeze every neighboring seam. The next section turns the contract into a local command you can rerun after every proposed diff.

Pin a fixture before the extraction

The following Python example is a proposed harness, not a log from a measured production run. It uses a memory store so the test process does not touch a real database or a shared queue. The cases cover a blank SKU, a negative quantity, an exact repeat, and a lowercase SKU that the current code uppercases. Run the capture against the untouched function, then commit the fixture before you accept any cleanup.

import json
from pathlib import Path

class MemoryStore:
def __init__(self):
self.batches =

[] def write(self, rows):
self.batches.append([dict(row) for row in rows])

def normalize_upload_batch(records, store):
seen = set()
kept =
[] for record in records:
sku = (record.get('sku') or '').strip().upper()
qty = record.get('qty', 0) or 0
if not sku or qty < 0:
continue
key = (sku, int(qty))
if key in seen:
continue
seen.add(key)
kept.append({'sku': sku, 'qty': int(qty)})
store.write(kept)
return kept

CASES = [
{
'name': 'blank-sku',
'records': [{'sku': ' ', 'qty': 2}],
'debt': False,
},
{
'name': 'negative-qty',
'records': [{'sku': 'ab1', 'qty': -1}],
'debt': False,
},
{
'name': 'case-and-repeat',
'records': [
{'sku': 'ab1', 'qty': 2},
{'sku': ' AB1 ', 'qty': 2},
{'sku': 'ab1', 'qty': 3},
],
'debt': False,
},
]

def capture(cases):
report =
[] for case in cases:
store = MemoryStore()
returned = normalize_upload_batch(case['records'], store)
report.append({
'name': case['name'],
'debt': case['debt'],
'returned': returned,
'written': store.batches,
})
return report

def main():
path = Path('batch_characterization.json')
payload = json.dumps(capture(CASES), indent=2) + '\n'
path.write_text(payload, encoding='utf-8')
print('wrote', path)

if __name__ == '__main__':
main()

Compare the fixture, do not skim the diff

A second, ordinary command compares a fresh capture with the fixture you already committed to the branch. The comparison is a textual check of the JSON, not a visual skim of a large assistant diff. If you later correct a debt row, change the fixture in a separate commit and state the product reason in that commit message. Until you trust the harness, treat a non-empty fixture diff as a failed review rather than as a suggestion.

Save the harness as capture_batch.py beside the fixture so the commands below match the file you run.

python capture_batch.py
git diff -- batch_characterization.json

The sample function above is embedded so a reader can follow the article without cloning a repository. In a real repository, import the existing function instead of pasting a simplified copy into the test file. A pasted copy can drift from the production function and then bless the wrong behavior during review. Keep the fixture next to the module so a reviewer can open both without searching the whole tree.

Score the smallest safe change

After the fixture is committed, the only edit this pass allows is an extraction of the pure selection loop. The wrapper may call the new function and then perform exactly one write to the existing store. It may not add a sort, a new skip rule, a retry, or a second store call. The table below is the acceptance gate for the diff, and a single failed row is enough to reject the change.

Check

Accept this pass

Send it back

Files touched

The one module that owns the batch function

Any helper outside that file

Fixture JSON

Unchanged after a local rerun

Any added, dropped, or reordered field

Store writes

One write, matching the returned rows

An extra write, a cleared batch, or a write before selection

New branches

None

A fallback, a retry, or a temporary compatibility flag

Debt rows

Still marked debt in the fixture

Quietly corrected inside the cleanup

The extraction that the gate allows

The extracted shape that satisfies every row of the table looks like the unexecuted sketch below. It remains a proposal until you rerun the capture and the fixture diff is completely empty. Notice that the skip rules are copied, not improved, because an improvement is a behavior change and belongs in a later commit. The wrapper stays responsible for the write, so persistence bugs are not mixed into the pure move.

def select_upload_rows(records):
seen = set()
kept =
[] for record in records:
sku = (record.get('sku') or '').strip().upper()
qty = record.get('qty', 0) or 0
if not sku or qty < 0:
continue
key = (sku, int(qty))
if key in seen:
continue
seen.add(key)
kept.append({'sku': sku, 'qty': int(qty)})
return kept

def normalize_upload_batch(records, store):
kept = select_upload_rows(records)
store.write(kept)
return kept

Follow the same four steps

1. Commit the characterization fixture that you generated from the untouched function before any extraction work begins.
2. Extract only the selection loop into a pure helper, and leave the storage write inside the original wrapper.
3. Rerun the local capture after that extraction and require an empty diff on the committed fixture file.
4. Schedule debt corrections, retries, and storage changes as later commits that carry their own expected results.

That order matters more than the size or brand of the assistant that happened to propose the cleanup. A small extraction can still be wrong if you regenerate the fixture after the edit and then treat the new file as the baseline. Commit the baseline first, or the acceptance gate has nothing stable left to protect later. If step three fails, restore the extraction and inspect the case, rather than widening the diff to force a match.

Let an assistant draft, then keep the gate local

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A coding assistant can draft fixture cases or the extraction sketch after you have already named the rows you care about. For this draft, the operator supplied two availability claims: free model access and a free server option. This article does not state model names, token quotas, hardware size, duration, or benchmark scores, because those details were not provided as primary sources. Check the project documentation on the day you start, because availability terms can change and should not be copied from an older post.

The useful split should stay small even when the assistant is allowed to see the whole repository. Ask it to propose cases that match the contract, then paste those cases into the harness and run the capture yourself. Ask for the extraction only after the fixture is committed, and reject any diff that fails a row in the table. A free server can be a convenient place to run that capture when your laptop cannot match the language version. The committed fixture still belongs in the repository you review, not only on the temporary machine.

If that gate matches how you already review cleanups, read the current access notes before you try it. Then run the same fixture workflow on one module you can execute on a machine you control. Keep the product optional: the harness, the table, and the empty fixture diff are the parts a teammate can reuse without an account. A reviewer should never treat a model-written explanation as proof that the fixture file remained stable.

Limitations and who should skip this

This approach freezes current behavior, including bugs that nobody has yet scheduled for a product fix. It is a poor fit when the ticket requires a visible behavior change inside the same pull request. It is also a poor fit when you cannot obtain representative rows, because an empty fixture will bless whatever the first draft happens to emit. A memory store does not prove transaction order, locking, or partial failure, so do not use it to clear a data-migration change.

Skip the gate as your only review when the diff touches authentication, billing totals, or deletion. Those areas need explicit expected outcomes, not a snapshot of whatever the old code already did. Do not plan a team rollout around an unverified free quota or an assumed server lifetime from an old post. If today's documentation does not confirm the access terms you need, treat the assistant as optional. Keep the harness on a machine you control rather than on a shared environment you have not reviewed.

The closing claim stays modest, and a reviewer can audit it from the fixture diff alone. A four-hundred-line cleanup becomes reviewable when one characterized seam moves and the fixture file stays still. Everything else can wait for a later commit that states the behavior it intends to change. That later commit should carry its own cases, rather than borrowing the freeze you used to keep this extraction honest.#testing #refactoring #python #programming #software #coding #development #engineering #inclusive #community
Require an Empty Fixture Diff Before Extracting a Batch Loop