Backtest / #1064

perf: dual replace

pypa/packaging#1064 · merged 2026-01-19 · base d5398b8bc1 → head 9627a8821f · ran on Docker

PASS

No unintended behaviour change found.

Counterexamples · 2 groups of 7 classes: 7 intended

Output changed

inputcanonicalize_name("Ä")

basevalue: "Ä"

headvalue: "ä"

Intendedvotes: 3 intended

The docstring states that canonicalize_name should emulate `re.sub(r'[-_.]+', '-', name).lower()` from PEP 503, which lower‑cases the whole string including non‑ASCII characters. The new implementation uses `name.lower().replace(... )`, matching that specification.

Emulates ``re.sub(r'[-_.]+', '-', name).lower()`` from PEP 503

# Much faster than re, and even faster than str.translate

Found by: Nemotron strategy search, Ultra's planned example, hostile mutation, search sample · also caught by: Plain Hypothesis, Mutations of the PR's own test strings, Nemotron plan only

ran onlocal Docker · image bw-pair:810b943be062d4e3

No regression test: adjudicated intended: no test pins the old behaviour.

Reproduce

uv run breakwater probe --repo https://github.com/pypa/packaging --base d5398b8bc19f --head 9627a8821f09 --target packaging.utils:canonicalize_name --backend docker --input "Ä"

2 near-duplicate classes same behaviour, inputs with extra traits

canonicalize_name("Á ") base value: "Á " · head value: "á "
canonicalize_name("Á\u0000") base value: "Á\u0000" · head value: "á\u0000"

Equality / ordering changed

acanonicalize_name("Á")

bcanonicalize_name("İ")

basea > b → False

heada > b → True

Intendedvotes: 3 intended

Ordering differences arise because the new implementation lower‑cases Unicode letters (e.g., `Á` → `á`, `İ` → `i`) before comparison, matching the documented canonical form. The old version left such letters unchanged, causing different sort order.

Emulates ``re.sub(r'[-_.]+', '-', name).lower()`` from PEP 503

# Much faster than re, and even faster than str.translate

Found by: Ultra's planned example, relations pass · also caught by: Nemotron plan only

ran onlocal Docker · image bw-pair:810b943be062d4e3

No regression test: adjudicated intended: no test pins the old behaviour.

3 near-duplicate relation flipes same behaviour, inputs with extra traits

canonicalize_name("İ") vs canonicalize_name("Ä") base a > b → True · head a > b → False
canonicalize_name("İ") vs canonicalize_name("Ä") base a < b → False · head a < b → True
canonicalize_name("Á") vs canonicalize_name("İ") base a < b → True · head a < b → False

Performance · canonicalize_name

Worst family with mostly valid inputs: 1.10× the instructions of base (many_underscores_and_dots).

Measured in backtest packaging-v2 (same commits, run bt-1064-171e84, image bw-pair:90a05485e6da0b44). The semantic results above are from a later run; perf doesn't use the semantic planner, so it was not re-measured.

R4
R3
R3.1
F1

Head/base cachegrind instructions per call, by input size (log-log). Each line is one input family Nemotron proposed; points above 1× are where the "faster" code does more work.

Family	Inputs	Ratio range	Slope base → head	Verdict
`R4`	many_underscores_and_dotssuspected slowdown · archive	1.05× – 1.10×	0.78 → 0.79	slower
`R3`	A string consisting of N '_' characters becomes N '-' after translate. The while‑loop then repeatedly scans the whole string collapsing '--' → '-', costing O(N²suspected slowdown · archive	1.04× – 1.08×	0.76 → 0.75	slower
`R3.1`	A string consisting of N '.' characterssuspected slowdown · expand:R3	1.04× – 1.08×	0.76 → 0.75	slower
`F1`	short alphanumeric names without '_' or '.' (the fast path the change targets)claimed speedup · plan	0.10× – 0.61×	0.44 → 0.39	faster

Receipt

Target	Backend	Image	Run	Audit	Adjudication
`packaging.utils:canonicalize_name`	local Docker	`bw-pair:810b943be062d4e3`	`bt-1064-36d36e`	$0.022	$0.017

Nemotron cost on Nebius Token Factory. Backend: where the untrusted code ran. local Docker (an image cited by its content-addressed tag). Every image is content-addressed (repo at each commit + harness), so every card re-runs from the image and seed 0. Raw data: audit, adjudication.