Breakwater

Backtest / #1236

perf: simply work done in parse_tag a bit

pypa/packaging#1236 · merged 2026-06-09 · base e6cd4be753 → head 1529771b80 · ran on Docker

PASS

No unintended behaviour change found.

† Merged after Nemotron 3 Ultra was released (2026-06-04): the model cannot have seen this PR or its fixes.

Counterexamples · 1 group of 4 classes: 4 intended

P2

Exception behaviour changed

inputparse_tag(".")
baseraises ValueError
headraises InvalidTag

Packaging.tags.InvalidTag is a subclass of builtins.ValueError.

Intendedvotes: 3 intended

Same reasoning as C1: the new `InvalidTag` exception is documented as the intended error for empty tag components, replacing the generic `ValueError`.

.. versionadded:: 26.3
Raises :class:`InvalidTag` on empty tag components.

Found by: project's test strings, hostile mutation · also caught by: Plain Hypothesis, Mutations of the PR's own test strings

ran onlocal Docker · image bw-pair:d9b82063b49a1e43

No regression test: adjudicated intended: no test pins the old behaviour.

Reproduce
uv run breakwater probe --repo https://github.com/pypa/packaging --base e6cd4be75370 --head 1529771b80cf --target packaging.tags:parse_tag --backend docker --input "."
3 near-duplicate classes same behaviour, inputs with extra traits
  • parse_tag("") base raises ValueError · head raises InvalidTag
  • parse_tag(".\u0000") base raises ValueError · head raises InvalidTag
  • parse_tag("Ensure platform tags normalize spaces to underscores.") base raises ValueError · head raises InvalidTag

Performance · parse_tag

Worst family with mostly valid inputs: 1.09× the instructions of base (tiny_tag).

Measured in backtest packaging-v2 (same commits, run bt-1236-1cca09, image bw-pair:6d11476928904c1e). The semantic results above are from a later run; perf doesn't use the semantic planner, so it was not re-measured.

head slowerhead faster1×2×4×8×101001,00010,000input size (characters)F5F5: 1.09× at ~16 charsF5: 1.09× at ~16 charsF5: 1.09× at ~16 charsF5: 1.09× at ~16 charsF5: 1.09× at ~16 charsF5: 1.09× at ~16 charsF5: 1.09× at ~16 charsR1.1R1.1: 1.09× at ~13 charsR1.1: 1.84× at ~17 chars (excluded: base accepts 0% of these inputs)R1.1: 3.11× at ~41 chars (excluded: base accepts 0% of these inputs)R1.1: 5.50× at ~137 chars (excluded: base accepts 0% of these inputs)R1.1: 8.66× at ~521 chars (excluded: base accepts 0% of these inputs)R1.1: 12.00× at ~2,057 chars (excluded: base accepts 0% of these inputs)R1.1: 13.38× at ~8,201 chars (excluded: base accepts 0% of these inputs)R1R1: 1.40× at ~7 charsR1: 1.70× at ~11 charsR1: 3.04× at ~35 charsR1: 5.50× at ~131 charsR1: 8.75× at ~515 charsR1: 12.14× at ~2,051 charsR1: 13.56× at ~8,195 charsF6F6: 1.64× at ~23 charsF6: 1.86× at ~33 charsF6: 2.62× at ~93 charsF6: 3.47× at ~333 charsF6: 4.31× at ~1,293 charsF6: 5.15× at ~5,133 charsF6: 5.40× at ~20,493 charsF4F4: 1.00× at ~20 charsF4: 0.93× at ~28 charsF4: 0.86× at ~84 charsF4: 0.84× at ~324 charsF4: 0.84× at ~1,442 charsF4: 0.84× at ~6,076 charsF4: 0.85× at ~27,580 chars
  • F5
  • R1.1
  • R1
  • F6
  • F4
Head/base cachegrind instructions per call, by input size (log-log). Each line is one input family Nemotron proposed; points above 1× are where the "faster" code does more work. Hollow points are sizes where the base version rejects most of the generated inputs; they are shown but do not count toward the verdict.
FamilyInputsRatio range Slope base → headVerdict
F5tiny_tagsuspected slowdown · plan1.09× – 1.09×0.00 → 0.00slower
R1.1excess-dashes-with-suffixsuspected slowdown · expand:R1 · mostly invalid inputs (error path) · 6 size(s) excluded: mostly invalid inputs1.09× – 1.09×0.00 → 0.00slower
R1invalidsuspected slowdown · archive · mostly invalid inputs (error path)1.40× – 13.56×0.57 → 0.91complexity-regression
F6Invalid tags containing many hyphens (more than the expected two). The new implementation splits all hyphen-separated components by '.' before attempting to unpack three parts, doing extra work before the unpacking error; the old version failed fast on the unpack.suspected slowdown · plan · mostly invalid inputs (error path)1.64× – 5.40×0.75 → 0.93complexity-regression
F4Valid tags with a large number of dot-separated interpreters but only one ABI and one platform. This exercises the interpreter component split path.claimed speedup · plan0.84× – 1.00×1.00 → 0.98neutral

Receipt

TargetBackendImageRun AuditAdjudication
packaging.tags:parse_taglocal Dockerbw-pair:d9b82063b49a1e43bt-1236-831e4b$0.13$0.033

Nemotron cost on Nebius Token Factory. Backend: where the untrusted code ran. local Docker (an image cited by its content-addressed tag). Every image is content-addressed (repo at each commit + harness), so every card re-runs from the image and seed 0. Raw data: audit, adjudication.