Breakwater

Backtest / 1082@61967cca

perf: fast path for simple versions (intermediate commit 61967cca)

pypa/packaging#1082 · merged 2026-02-11 · base 59f183fe90 → head 61967cca27 · ran on Docker

BLOCK

At least one behaviour change looks like an unintended regression. A valid input family runs 1.91× the instructions of the old code.

Known regressions in this PR

Counterexamples · 4 groups of 5 classes: 3 likely bug, 2 intended

P2

Exception behaviour changed

known regression: unicode-digitsknown regression: inner-whitespace
inputVersion("0")
baseraises InvalidVersion
headreturns normally
Likely bugvotes: 3 likely-bug

The fast‑path treats any character not in `_NON_SIMPLE_VERSION_INDICATORS` as a simple numeric version. The set deliberately lists ASCII letters and punctuation (e.g. `!+-ABCDEHILOPRSTVW_abcdehiloprstvw`) but does not include full‑width digits such as `0`. Consequently `Version('0')` is accepted by the fast‑path (`int('0')` succeeds) whereas the original regex rejects it, a non‑conformant change.

_NON_SIMPLE_VERSION_INDICATORS = frozenset('!+-ABCDEHILOPRSTVW_abcdehiloprstvw')
if _NON_SIMPLE_VERSION_INDICATORS.isdisjoint(version):

Found by: Nemotron strategy search, Ultra's planned example, hostile mutation, search sample · also caught by: Nemotron plan only

ran onlocal Docker · image bw-pair:348e2d05c100a5f9

Regression test verified passes on base, fails on head
def test_1082_61967cca_p2_outcome_0():
    """Breakwater P2: base behaviour pinned (found by differential testing; label: likely-bug)."""
    import pytest
    from packaging.version import InvalidVersion, Version
    with pytest.raises(InvalidVersion) as excinfo:
        Version('0')
    assert type(excinfo.value) is InvalidVersion  # exact type: a subclass would be a behaviour change
Reproduce
uv run breakwater probe --repo https://github.com/pypa/packaging --base 59f183fe90da --head 61967cca278d --target packaging.version:Version --backend docker --input "0"
1 near-duplicate class same behaviour, inputs with extra traits
  • Version("0 ") base raises InvalidVersion · head returns normallyunicode-digitsinner-whitespace test verified (also under another group)
P2

Exception behaviour changed

known regression: inner-whitespaceknown regression: unicode-digits
inputVersion(" 1. 2.3 ")
baseraises InvalidVersion
headreturns normally
Likely bugvotes: 3 likely-bug

The fast‑path calls `version.strip()` before splitting on dots, thereby accepting leading/trailing whitespace (e.g. `' 1.2.3 '`). PEP 440 does not permit whitespace in a version string, and the original regex rejects it. The change was not mentioned in the PR description, indicating an unintended relaxation.

if _NON_SIMPLE_VERSION_INDICATORS.isdisjoint(version):
self._release = tuple(map(int, version.strip().split('.')))

Found by: hostile mutation · also caught by: Mutations of the PR's own test strings

ran onlocal Docker · image bw-pair:348e2d05c100a5f9

Regression test verified passes on base, fails on head
def test_1082_61967cca_p2_outcome_4():
    """Breakwater P2: base behaviour pinned (found by differential testing; label: likely-bug)."""
    import pytest
    from packaging.version import InvalidVersion, Version
    with pytest.raises(InvalidVersion) as excinfo:
        Version(' 1. 2.3 ')
    assert type(excinfo.value) is InvalidVersion  # exact type: a subclass would be a behaviour change
Reproduce
uv run breakwater probe --repo https://github.com/pypa/packaging --base 59f183fe90da --head 61967cca278d --target packaging.version:Version --backend docker --input " 1. 2.3 "
1 near-duplicate class same behaviour, inputs with extra traits
  • Version("0 ") base raises InvalidVersion · head returns normallyunicode-digitsinner-whitespace test verified (also under another group)
P2

Exception behaviour changed

known regression: inner-whitespaceknown regression: oversized-int-exception
inputVersion(" 10000000000000000000000000000000000000000…000000000 " (4,303 chars))
baseraises ValueError
headraises InvalidVersion

Packaging.version.InvalidVersion is a subclass of builtins.ValueError.

Intendedvotes: 2 intended, 1 needs-human

The fast‑path explicitly converts a `ValueError` from the integer parsing into `InvalidVersion` (`except ValueError: raise InvalidVersion(...)`). When the fast‑path is taken and the integer conversion fails, the new code raises `InvalidVersion` instead of letting the original `ValueError` propagate, which matches the observed change.

self._release = tuple(map(int, version.strip().split('.')))
except ValueError: raise InvalidVersion(f'Invalid version: {version!r}') from None

Found by: hostile mutation · no baseline found it

ran onlocal Docker · image bw-pair:348e2d05c100a5f9

No regression test: adjudicated intended: no test pins the old behaviour.

P2

Exception behaviour changed

known regression: unicode-digitsknown regression: oversized-int-exception
inputVersion("100000000000000000000000000000000000000000…000000.2.3" (4,304 chars))
baseraises InvalidVersion
headreturns normally
Intendedvotes: 2 intended, 1 needs-human

The fast‑path is designed to accept simple dot‑separated numeric versions without invoking the full regex. A very long all‑digit version passes the `_NON_SIMPLE_VERSION_INDICATORS.isdisjoint` test and is parsed successfully, whereas the original regex path rejects it (raising `InvalidVersion`). This is the performance optimisation the PR targets.

_NON_SIMPLE_VERSION_INDICATORS = frozenset('!+-ABCDEHILOPRSTVW_abcdehiloprstvw')
if _NON_SIMPLE_VERSION_INDICATORS.isdisjoint(version):

Found by: hostile mutation · no baseline found it

ran onlocal Docker · image bw-pair:348e2d05c100a5f9

No regression test: adjudicated intended: no test pins the old behaviour.

Regression tests

3 of 5 classes have a generated pytest that pins the old behaviour. Each passed on the base commit and failed on the head commit in the sandbox before it was offered.

Download test_1082_61967cca.py

Performance · Version

Worst family with mostly valid inputs: 1.91× the instructions of base (versions with an epoch (e.g. "1!2.3.4")).

Measured in backtest packaging-v2 (same commits, run bt-1082-61967cca-c23f4f, image bw-pair:cebb5e33c4500036). The semantic results above are from a later run; perf doesn't use the semantic planner, so it was not re-measured.

head slowerhead faster1×1.2×1.4×1.7×2×101001,00010,000input size (characters)R2R2: 1.05× at ~13 charsR2: 1.07× at ~21 charsR2: 1.14× at ~69 charsR2: 1.20× at ~261 charsR2: 1.22× at ~1,029 charsR2: 1.23× at ~4,101 charsR2: 1.23× at ~16,389 charsR3R3: 1.04× at ~9 charsR3: 1.04× at ~13 charsR3: 1.10× at ~44 charsR3: 1.15× at ~188 charsR3: 1.20× at ~921 charsR3: 1.22× at ~4,018 charsR3: 1.27× at ~19,378 charsR4R4: 1.04× at ~9 charsR4: 1.05× at ~13 charsR4: 1.10× at ~44 charsR4: 1.15× at ~188 charsR4: 1.20× at ~921 charsR4: 1.22× at ~4,018 charsR4: 1.27× at ~19,378 charsF4F4: 1.03× at ~10 charsF4: 1.04× at ~12 charsF4: 1.09× at ~24 charsF4: 1.25× at ~72 charsF4: 1.69× at ~264 charsF4: 1.91× at ~1,032 charsF4: 1.46× at ~4,104 chars
  • R2
  • R3
  • R4
  • F4
Head/base cachegrind instructions per call, by input size (log-log). Each line is one input family Nemotron proposed; points above 1× are where the "faster" code does more work.
FamilyInputsRatio range Slope base → headVerdict
R2Single repeated segment "999." n times plus prerelease c5suspected slowdown · archive1.05× – 1.23×0.79 → 0.82slower
R3Versions with many release segments and a pre‑release suffix (e.g. "1.2.3…a1"). The fast‑path scans the whole string looking for indicator characters, finds the “a”, then falls back to the regex parser – an extra O(n) pass over the input.suspected slowdown · archive1.04× – 1.27×0.74 → 0.77slower
R4Many numeric release segments followed by 'b' pre-releasesuspected slowdown · archive1.04× – 1.27×0.74 → 0.77slower
F4versions with an epoch (e.g. "1!2.3.4")suspected slowdown · plan1.03× – 1.91×0.51 → 0.60slower

Receipt

TargetBackendImageRun AuditAdjudication
packaging.version:Versionlocal Dockerbw-pair:348e2d05c100a5f9bt-1082-61967cca-8aeaff$0.047$0.050

Nemotron cost on Nebius Token Factory. Backend: where the untrusted code ran. local Docker (an image cited by its content-addressed tag). Every image is content-addressed (repo at each commit + harness), so every card re-runs from the image and seed 0. Raw data: audit, adjudication.