Breakwater

Backtest / #1082

perf: Add fast path for parsing simple versions (digits and dots only)

pypa/packaging#1082 · merged 2026-02-11 · base 59f183fe90 → head ee13ba2c1c · ran on Docker

BLOCK

At least one behaviour change looks like an unintended regression. A valid input family runs 2.85× the instructions of the old code.

Known regressions in this PR

Counterexamples · 2 classes: 1 likely bug, 1 intended

P2

Exception behaviour changed

inputVersion(["1"])
baseraises TypeError
headraises AttributeError

Neither exception type subclasses the other.

Likely bugvotes: 2 likely-bug, 1 needs-human

The fast‑path added in the PR returns early after setting a handful of private attributes (e.g., `self._epoch = 0; self._pre = None; …; self._key_cache = None; return`). The base version took the full regex path and raised a built‑in `TypeError` for the input `['1']`; the new fast path returns early and later code apparently accesses an attribute that was not initialised, causing a built‑in `AttributeError`. Since the two built‑in exception types are unrelated (neither subclasses the other), this change of exception type is an unintended regression.

if _SIMPLE_VERSION_INDICATORS.issuperset(version):
self._release = tuple(map(int, version.split('.')))

Found by: Nemotron strategy search · no baseline found it

ran onlocal Docker · image bw-pair:3f8cc97a2a8a52dc

Regression test verified passes on base, errors on head
def test_1082_p2_outcome_0():
    """Breakwater P2: base behaviour pinned (found by differential testing; label: likely-bug)."""
    import pytest
    from packaging.version import Version
    with pytest.raises(TypeError) as excinfo:
        Version(['1'])
    assert type(excinfo.value) is TypeError  # exact type: a subclass would be a behaviour change
P2

Exception behaviour changed

known regression: oversized-int-exception
inputVersion("100000000000000000000000000000000000000000…0000000000" (4,301 chars))
baseraises ValueError
headraises InvalidVersion

Packaging.version.InvalidVersion is a subclass of builtins.ValueError.

Intendedvotes: 3 intended

The PR adds a fast‑path for simple dotted‑digit versions that parses with `split('.')` and `map(int, …)`. When that fast path fails it catches `ValueError` and raises `packaging.version.InvalidVersion`, a subclass of `ValueError`. The test diff explicitly adds cases that “trigger the fast path (digits/dots only)” and expects `InvalidVersion`. This change of exception type (subclass of the original `ValueError`) is the intended improvement.

self._release = tuple(map(int, version.split('.')))
except ValueError: raise InvalidVersion(f'Invalid version: {version!r}') from None

Found by: hostile mutation · also caught by: Mutations of the PR's own test strings

ran onlocal Docker · image bw-pair:3f8cc97a2a8a52dc

No regression test: adjudicated intended: no test pins the old behaviour.

Regression tests

1 of 2 classes have a generated pytest that pins the old behaviour. Each passed on the base commit and failed on the head commit in the sandbox before it was offered.

Download test_1082.py

Performance · Version

Worst family with mostly valid inputs: 2.85× the instructions of base (Each numeric chunk increases digit count linearly while keeping total chunks fixed).

Measured in backtest packaging-v2 (same commits, run bt-1082-88702b, image bw-pair:53b50a069e664704). The semantic results above are from a later run; perf doesn't use the semantic planner, so it was not re-measured.

head slowerhead faster0.5×1×2×101001,00010,000100,000input size (characters)R3.2R3.2: 1.04× at ~10 charsR3.2: 1.07× at ~19 charsR3.2: 1.28× at ~157 charsR3.2: 1.93× at ~2,149 charsR3.2: 2.57× at ~33,157 charsR3.2: 2.85× at ~525,829 charsR3R3: 1.05× at ~13 charsR3: 1.07× at ~21 charsR3: 1.13× at ~69 charsR3: 1.19× at ~261 charsR3: 1.21× at ~1,029 charsR3: 1.22× at ~4,101 charsR3: 1.22× at ~16,389 charsR4R4: 1.04× at ~9 charsR4: 1.05× at ~13 charsR4: 1.09× at ~44 charsR4: 1.15× at ~188 charsR4: 1.20× at ~921 charsR4: 1.22× at ~4,018 charsR4: 1.27× at ~19,378 charsR1R1: 1.08× at ~9 charsR1: 1.11× at ~13 charsR1: 1.21× at ~37 charsR1: 1.37× at ~133 charsR1: 1.47× at ~517 charsR1: 1.50× at ~2,053 charsR1: 1.52× at ~8,197 charsF2F2: 1.00× at ~17 chars (excluded: base accepts 0% of these inputs)F2: 0.37× at ~9 charsF2: 0.36× at ~6 charsF2: 0.37× at ~9 charsF2: 0.43× at ~8 charsF2: 0.36× at ~7 charsF2: 0.36× at ~7 chars
  • R3.2
  • R3
  • R4
  • R1
  • F2
Head/base cachegrind instructions per call, by input size (log-log). Each line is one input family Nemotron proposed; points above 1× are where the "faster" code does more work. Hollow points are sizes where the base version rejects most of the generated inputs; they are shown but do not count toward the verdict.
FamilyInputsRatio range Slope base → headVerdict
R3.2Each numeric chunk increases digit count linearly while keeping total chunks fixedsuspected slowdown · expand:R31.04× – 2.85×0.63 → 0.74slower
R3Single repeated segment "999." n times plus prerelease c5suspected slowdown · archive1.05× – 1.22×0.79 → 0.82slower
R4Versions with many release segments and a pre‑release suffix (e.g. "1.2.3…a1"). The fast‑path scans the whole string looking for indicator characters, finds the “a”, then falls back to the regex parser – an extra O(n) pass over the input.suspected slowdown · archive1.04× – 1.27×0.74 → 0.77slower
R1Invalid version strings that nevertheless contain indicator characters (e.g. a trailing dot followed by a letter). The fast‑path still does the full scan before the regex fails.suspected slowdown · archive · mostly invalid inputs (error path)1.08× – 1.52×0.70 → 0.76slower
F2Version parsing fast path for simple numeric versionsclaimed speedup · plan · 1 size(s) excluded: mostly invalid inputs0.36× – 0.43×0.00 → 0.00faster

Receipt

TargetBackendImageRun AuditAdjudication
packaging.version:Versionlocal Dockerbw-pair:3f8cc97a2a8a52dcbt-1082-3b9756$0.080$0.025

Nemotron cost on Nebius Token Factory. Backend: where the untrusted code ran. local Docker (an image cited by its content-addressed tag). Every image is content-addressed (repo at each commit + harness), so every card re-runs from the image and seed 0. Raw data: audit, adjudication.