Breakwater

Sandboxes / #1108

perf: SpecifierSet use tuple instead of frozenset for _specs

pypa/packaging#1108 · merged 2026-03-07 · base 20f1523cf3 → head f59d6ee9af · ran on Sandboxes

BLOCK

At least one behaviour change looks like an unintended regression.

Known regressions in this PR

Counterexamples · 7 groups of 11 classes: 10 likely bug, 1 needs a human

P1

Output changed

known regression: dedup-str-len
inputSpecifierSet(">=1.0,>=1.0.0")
baselen: 1
headlen: 2
Likely bugvotes: 3 likely-bug

SpecifierSet now deduplicates on raw strings (dict.fromkeys) instead of on canonical Specifier equality, so two specifiers that are equal after canonicalization (e.g. '>=1.0' and '>=1.0.0') are both kept, changing len().

self._specs: tuple[Specifier, ...] = tuple(map(Specifier, dict.fromkeys(split_specifiers)))
Specifier('==1.2.3') == Specifier('== 1.2.3.0') True

Found by: Ultra's planned example, hostile mutation · also caught by: Mutations of the PR's own test strings, Nemotron plan only

ran onToken Factory Sandboxes · image 052dfee2-22cb-4d9b-b081-e27b77a69db0

Regression test verified passes on base, fails on head
def test_1108_p1_len_0():
    """Breakwater P1: base behaviour pinned (found by differential testing; label: likely-bug)."""
    from packaging.specifiers import SpecifierSet
    assert len(SpecifierSet('>=1.0,>=1.0.0')) == 1
Reproduce
uv run breakwater probe --repo https://github.com/pypa/packaging --base 20f1523cf37d --head f59d6ee9afe2 --target packaging.specifiers:SpecifierSet --backend contree --input ">=1.0,>=1.0.0"
2 near-duplicate classes same behaviour, inputs with extra traits
  • SpecifierSet(">= 1.0,>=1.0.0") base len: 1 · head len: 2dedup-str-len test verified
  • SpecifierSet(" >=1.0,>=1.0.0 ") base len: 1 · head len: 2dedup-str-len test verified
P1

Output changed

known regression: dedup-str-len
inputSpecifierSet(">=1.0,>=1.0.0")
basestr: ">=1.0"
headstr: ">=1.0,>=1.0.0"
Likely bugvotes: 3 likely-bug

__str__ joins the sorted tuple of specs; with two retained specifiers the string now contains both.

return ','.join(str(s) for s in self._sort_specs())
self._specs: tuple[Specifier, ...] = tuple(map(Specifier, dict.fromkeys(split_specifiers)))

Found by: Ultra's planned example, hostile mutation · also caught by: Mutations of the PR's own test strings, Nemotron plan only

ran onToken Factory Sandboxes · image 052dfee2-22cb-4d9b-b081-e27b77a69db0

Regression test verified passes on base, fails on head
def test_1108_p1_str_3():
    """Breakwater P1: base behaviour pinned (found by differential testing; label: likely-bug)."""
    from packaging.specifiers import SpecifierSet
    assert str(SpecifierSet('>=1.0,>=1.0.0')) == '>=1.0'
Reproduce
uv run breakwater probe --repo https://github.com/pypa/packaging --base 20f1523cf37d --head f59d6ee9afe2 --target packaging.specifiers:SpecifierSet --backend contree --input ">=1.0,>=1.0.0"
2 near-duplicate classes same behaviour, inputs with extra traits
  • SpecifierSet(">= 1.0,>=1.0.0") base str: ">=1.0" · head str: ">=1.0,>=1.0.0"dedup-str-len test verified
  • SpecifierSet(" >=1.0,>=1.0.0 ") base str: ">=1.0" · head str: ">=1.0,>=1.0.0"dedup-str-len test verified
P3

Equality / ordering changed

known regression: dedup-equality
aSpecifierSet(">=1.0,>=1.0.0")
bSpecifierSet(">=1.0.0,>=1.0.0")
basea == b → True
heada == b → False
Likely bugvotes: 3 likely-bug

Equality of SpecifierSet now depends on the tuple length after string‑level deduplication; the two sets have different lengths (2 vs 1) and therefore compare False, whereas the old frozenset canonicalized them to a single element and compared True.

if len(self._specs) != len(other._specs): return False
self._specs: tuple[Specifier, ...] = tuple(map(Specifier, dict.fromkeys(split_specifiers)))

Found by: relations pass · no baseline found it

ran onToken Factory Sandboxes · image 052dfee2-22cb-4d9b-b081-e27b77a69db0

Regression test verified passes on base, fails on head
def test_1108_p3_eq_7():
    """Breakwater P3: base behaviour pinned (found by differential testing; label: likely-bug)."""
    from packaging.specifiers import SpecifierSet
    a = SpecifierSet('>=1.0,>=1.0.0')
    b = SpecifierSet('>=1.0.0,>=1.0.0')
    assert (a == b) is True
P4

Old pickles break

known regression: old-pickles-break-comparison
inputSpecifierSet("")
basepickled object loads and compares equal
headcompare with fresh object: raises AttributeError
Likely bugvotes: 3 likely-bug

Same root cause as C2: the new '_sorted' slot is absent on objects pickled by the old implementation (including an empty SpecifierSet), leading to AttributeError when the unpickled object participates in equality checks.

__slots__ = ( '_prereleases', '_sorted', '_specs')
def __eq__(self, other: object) -> bool: ... return self._sort_specs() == other._sort_specs()

Found by: project's test strings, hostile mutation · also caught by: Plain Hypothesis, Mutations of the PR's own test strings

ran onToken Factory Sandboxes · image 052dfee2-22cb-4d9b-b081-e27b77a69db0

Regression test verified passes on base, errors on head
def test_1108_p4_base_head_eq_fresh_10():
    """Breakwater P4: base behaviour pinned (found by differential testing; label: likely-bug)."""
    import base64
    import pickle

    from packaging.specifiers import SpecifierSet
    # pickled by the base version (lazy caches warmed first)
    old = pickle.loads(base64.b64decode('gASVUAAAAAAAAACMFHBhY2thZ2luZy5zcGVjaWZpZXJzlIwMU3BlY2lmaWVyU2V0lJOUKYGUTn2UKIwMX3ByZXJlbGVhc2VzlE6MBl9zcGVjc5QokZR1hpRiLg=='))
    assert old == SpecifierSet('')
Reproduce
uv run breakwater probe --repo https://github.com/pypa/packaging --base 20f1523cf37d --head f59d6ee9afe2 --target packaging.specifiers:SpecifierSet --backend contree --input ""
P4

Old pickles break

known regression: old-pickles-break-comparison
inputSpecifierSet(">=1.0,>=1.0.0")
basepickled object loads and compares equal
headcompare with fresh object: false
Likely bugvotes: 3 likely-bug

The change adds a new slot '_sorted' and makes __hash__/__eq__ depend on it via _sort_specs. Objects pickled by the old version lack this slot; when unpickled by the new version the attribute is missing, causing an AttributeError during comparison.

__slots__ = ( '_prereleases', '_sorted', '_specs')
def __hash__(self) -> int: return hash(self._sort_specs())

Found by: Ultra's planned example, hostile mutation · also caught by: Mutations of the PR's own test strings, Nemotron plan only

ran onToken Factory Sandboxes · image 052dfee2-22cb-4d9b-b081-e27b77a69db0

Regression test verified passes on base, fails on head
def test_1108_p4_base_head_eq_fresh_9():
    """Breakwater P4: base behaviour pinned (found by differential testing; label: likely-bug)."""
    import base64
    import pickle

    from packaging.specifiers import SpecifierSet
    # pickled by the base version (lazy caches warmed first)
    old = pickle.loads(base64.b64decode('gASVDwEAAAAAAACMFHBhY2thZ2luZy5zcGVjaWZpZXJzlIwMU3BlY2lmaWVyU2V0lJOUKYGUTn2UKIwMX3ByZXJlbGVhc2VzlE6MBl9zcGVjc5QoaACMCVNwZWNpZmllcpSTlCmBlE59lChoBU6MBV9zcGVjlIwCPj2UjAMxLjCUhpSMDV9zcGVjX3ZlcnNpb26UaA2MEXBhY2thZ2luZy52ZXJzaW9ulIwHVmVyc2lvbpSTlCmBlE59lCiMBF9kZXaUTowGX2Vwb2NolEsAjApfa2V5X2NhY2hllE6MBl9sb2NhbJROjAVfcG9zdJROjARfcHJllE6MCF9yZWxlYXNllEsBSwCGlHWGlGKGlHWGlGKRlHWGlGIu'))
    assert old == SpecifierSet('>=1.0,>=1.0.0')
Reproduce
uv run breakwater probe --repo https://github.com/pypa/packaging --base 20f1523cf37d --head f59d6ee9afe2 --target packaging.specifiers:SpecifierSet --backend contree --input ">=1.0,>=1.0.0"
P3

Equality / ordering changed

aSpecifierSet(">=1.0,>=1.0.0")
bSpecifierSet(">=1.0.0,>=1.0.0")
basehash(a) == hash(b) → True
headhash(a) == hash(b) → False
Likely bugvotes: 3 likely-bug

Specifier equality is based on a canonical form that strips trailing zeros (e.g., '>=1.0' == '>=1.0.0'). The old SpecifierSet stored specifiers in a frozenset, which deduplicated equal specifiers, while the new implementation deduplicates only on the raw strings (dict.fromkeys(split_specifiers)), so two equal specifiers remain as two items. Consequently the hash of a set containing '>=1.0,>=1.0.0' differs from one containing '>=1.0.0,>=1.0' under the new code.

return self._canonical_spec == other._canonical_spec
canonicalize_version(spec_version, strip_trailing_zero=(operator != '~='))

Found by: relations pass · no baseline found it

ran onToken Factory Sandboxes · image 052dfee2-22cb-4d9b-b081-e27b77a69db0

Regression test verified passes on base, fails on head
def test_1108_p3_hash_eq_8():
    """Breakwater P3: base behaviour pinned (found by differential testing; label: likely-bug)."""
    from packaging.specifiers import SpecifierSet
    a = SpecifierSet('>=1.0,>=1.0.0')
    b = SpecifierSet('>=1.0.0,>=1.0.0')
    assert (hash(a) == hash(b)) is True
P2

Exception behaviour changed

inputSpecifierSet(">=1000000000000000000000000000000000000000….0,>=1.0.0" (4,313 chars))
baseraises AssertionError
headreturns normally
Needs a humanvotes: 3 needs-human (model abstained)

The change from frozenset to tuple does not modify the comparison logic for a single '>=' specifier, yet the observed AssertionError disappears. No documentation or test change explains this, so the cause is unclear.

Found by: hostile mutation · also caught by: Mutations of the PR's own test strings

ran onToken Factory Sandboxes · image 052dfee2-22cb-4d9b-b081-e27b77a69db0

Regression test verified passes on base, fails on head
def test_1108_p2_outcome_6():
    """Breakwater P2: base behaviour pinned (found by differential testing; label: needs-human)."""
    import pytest
    from packaging.specifiers import SpecifierSet
    with pytest.raises(AssertionError) as excinfo:
        SpecifierSet('>=10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000.0,>=1.0.0')
    assert type(excinfo.value) is AssertionError  # exact type: a subclass would be a behaviour change

Regression tests

11 of 11 classes have a generated pytest that pins the old behaviour. Each passed on the base commit and failed on the head commit in the sandbox before it was offered.

Download test_1108.py

Receipt

TargetBackendImageRun AuditAdjudication
packaging.specifiers:SpecifierSetToken Factory Sandboxes052dfee2-22cb-4d9b-b081-e27b77a69db0bt-1108-f30c47$0.098$0.070

Nemotron cost on Nebius Token Factory. Backend: where the untrusted code ran. Token Factory Sandboxes (a microVM branched from a kept image, cited by its UUID). Every image is content-addressed (repo at each commit + harness), so every card re-runs from the image and seed 0. Raw data: audit, adjudication.