Breakwater

Sandboxes / #1236

perf: simply work done in parse_tag a bit

pypa/packaging#1236 · merged 2026-06-09 · base e6cd4be753 → head 1529771b80 · ran on Sandboxes

REVIEW

Behaviour changed; the evidence does not settle whether it was intended.

† Merged after Nemotron 3 Ultra was released (2026-06-04): the model cannot have seen this PR or its fixes.

Counterexamples · 1 group of 6 classes: 6 needs a human

P2

Exception behaviour changed

inputparse_tag("-")
baseraises ValueError
headraises InvalidTag

Packaging.tags.InvalidTag is a subclass of builtins.ValueError.

Needs a humanvotes: 1 likely-bug, 1 intended, 1 needs-human (no majority)

Same reasoning as C1: the refactor changes the exception type for a tag like '-' that lacks three components, raising InvalidTag instead of the original ValueError, without any documented intention.

Found by: Nemotron strategy search, Ultra's planned example, project's test strings, hostile mutation, search sample · also caught by: Plain Hypothesis, Mutations of the PR's own test strings, Nemotron plan only

ran onToken Factory Sandboxes · image 5c5765e3-5a87-48f2-8b21-497e56c1292b

Regression test verified passes on base, fails on head
def test_1236_p2_outcome_1():
    """Breakwater P2: base behaviour pinned (found by differential testing; label: needs-human)."""
    import pytest
    from packaging.tags import parse_tag
    with pytest.raises(ValueError) as excinfo:
        parse_tag('-')
    assert type(excinfo.value) is ValueError  # exact type: a subclass would be a behaviour change
Reproduce
uv run breakwater probe --repo https://github.com/pypa/packaging --base e6cd4be75370 --head 1529771b80cf --target packaging.tags:parse_tag --backend contree --input "-"
5 near-duplicate classes same behaviour, inputs with extra traits
  • parse_tag("") base raises ValueError · head raises InvalidTag test verified
  • parse_tag("- ") base raises ValueError · head raises InvalidTag test verified
  • parse_tag("\u001b-") base raises ValueError · head raises InvalidTag test verified
  • parse_tag("-ç") base raises ValueError · head raises InvalidTag test verified
  • parse_tag("- ") base raises ValueError · head raises InvalidTag test verified

Regression tests

6 of 6 classes have a generated pytest that pins the old behaviour. Each passed on the base commit and failed on the head commit in the sandbox before it was offered.

Download test_1236.py

Performance · parse_tag

Worst family with mostly valid inputs: 1.09× the instructions of base (simple tag with one interpreter, one abi and one platform).

head slowerhead faster1×1.2×input size (characters)F3.1F3.1: 1.09× at ~20 charsF3.1: 1.09× at ~20 charsF3.1: 1.09× at ~20 charsF3.1: 1.09× at ~20 charsF3.1: 1.09× at ~20 charsF3.1: 1.09× at ~20 charsF3.1: 1.09× at ~20 charsF2F2: 1.09× at ~16 charsF2: 1.09× at ~16 charsF2: 1.09× at ~16 charsF2: 1.09× at ~16 charsF2: 1.09× at ~16 charsF2: 1.09× at ~16 charsF2: 1.09× at ~16 charsF3F3: 1.09× at ~16 charsF3: 1.09× at ~16 charsF3: 1.09× at ~16 charsF3: 1.09× at ~16 charsF3: 1.09× at ~16 charsF3: 1.09× at ~16 charsF3: 1.09× at ~16 charsF2.1F2.1: 1.09× at ~18 charsF2.1: 1.09× at ~18 charsF2.1: 1.09× at ~18 charsF2.1: 1.09× at ~18 charsF2.1: 1.09× at ~18 charsF2.1: 1.09× at ~18 charsF2.1: 1.09× at ~18 chars
  • F3.1
  • F2
  • F3
  • F2.1
Head/base cachegrind instructions per call, by input size (log-log). Each line is one input family Nemotron proposed; points above 1× are where the "faster" code does more work.
FamilyInputsRatio range Slope base → headVerdict
F3.1single_hyphenclaimed speedup · expand:F31.09× – 1.09×0.00 → 0.00slower
F2simple tag with one interpreter, one abi and one platformsuspected slowdown · plan1.09× – 1.09×0.00 → 0.00slower
F3tiny_validclaimed speedup · plan1.09× – 1.09×0.00 → 0.00slower
F2.1simple tag with one interpreter, one abi and one platformsuspected slowdown · expand:F21.09× – 1.09×0.00 → 0.00slower

Receipt

TargetBackendImageRun AuditAdjudication
packaging.tags:parse_tagToken Factory Sandboxes5c5765e3-5a87-48f2-8b21-497e56c1292bbt-1236-b3de52$0.071$0.028

Nemotron cost on Nebius Token Factory. Backend: where the untrusted code ran. Token Factory Sandboxes (a microVM branched from a kept image, cited by its UUID). Every image is content-addressed (repo at each commit + harness), so every card re-runs from the image and seed 0. Raw data: audit, adjudication.