Breakwater

Backtest / #1147

perf: add __slots__ to Marker

pypa/packaging#1147 · merged 2026-04-05 · base 04a7600a21 → head 76cc629825 · ran on Docker

BLOCK

At least one behaviour change looks like an unintended regression.

Known regressions in this PR

Counterexamples · 1 class: 1 likely bug

P4

Old pickles break

known regression: marker-pickle
inputMarker("\"b\" > \"a\"")
basepickled object loads and compares equal
headload: raises AttributeError
Likely bugvotes: 3 likely-bug

The PR adds `__slots__ = (('_markers',))` to `Marker` as a performance change (see PR title and added line). This change breaks backward compatibility for pickled `Marker` objects: an instance pickled by the old class (without `__slots__`) cannot be unpickled by the new class, raising `AttributeError`. The PR has no tests, docs, or changelog entry addressing pickling, so the regression is unintended.

perf: add __slots__ to Marker
__slots__ = ( '_markers', )

Found by: Ultra's planned example, project's test strings, hostile mutation · also caught by: Mutations of the PR's own test strings, Nemotron plan only

ran onlocal Docker · image bw-pair:e33c8d8e1355cf11

Regression test verified passes on base, errors on head
def test_1147_p4_base_head_load_0():
    """Breakwater P4: base behaviour pinned (found by differential testing; label: likely-bug)."""
    import base64
    import pickle
    # pickled by the base version (lazy caches warmed first)
    old = pickle.loads(base64.b64decode('gASVlgAAAAAAAACMEXBhY2thZ2luZy5tYXJrZXJzlIwGTWFya2VylJOUKYGUfZSMCF9tYXJrZXJzlF2UjBFwYWNrYWdpbmcuX3BhcnNlcpSMBVZhbHVllJOUKYGUTn2UjAV2YWx1ZZSMAWKUc4aUYmgHjAJPcJSTlCmBlE59lGgMjAE+lHOGlGJoCSmBlE59lGgMjAFhlHOGlGKHlGFzYi4='))
    assert old is not None  # loading must not raise
Reproduce
uv run breakwater probe --repo https://github.com/pypa/packaging --base 04a7600a21fe --head 76cc62982548 --target packaging.markers:Marker --backend docker --input "\"b\" > \"a\""

Regression tests

1 of 1 classes have a generated pytest that pins the old behaviour. Each passed on the base commit and failed on the head commit in the sandbox before it was offered.

Download test_1147.py

Performance · Marker

No realistic input family measured slower.

Measured in backtest packaging-v2 (same commits, run bt-1147-8b7b5a, image bw-pair:50aa962c18a1f597). The semantic results above are from a later run; perf doesn't use the semantic planner, so it was not re-measured.

head slowerhead faster1×1001,00010,000100,000input size (characters)F6F6: 1.00× at ~87 charsF6: 1.00× at ~148 charsF6: 0.98× at ~520 charsF6: 0.99× at ~2,008 charsF6: 1.00× at ~8,123 charsF6: 1.00× at ~32,730 chars (excluded: base accepts 0% of these inputs)F6: 1.00× at ~134,106 chars (excluded: base accepts 0% of these inputs)
  • F6
Head/base cachegrind instructions per call, by input size (log-log). Each line is one input family Nemotron proposed; points above 1× are where the "faster" code does more work. Hollow points are sizes where the base version rejects most of the generated inputs; they are shown but do not count toward the verdict.
FamilyInputsRatio range Slope base → headVerdict
F6Nested marker expressions (and/or groups)claimed speedup · plan · 2 size(s) excluded: mostly invalid inputs0.98× – 1.00×1.01 → 1.01neutral

Receipt

TargetBackendImageRun AuditAdjudication
packaging.markers:Markerlocal Dockerbw-pair:e33c8d8e1355cf11bt-1147-c49b2a$0.017$0.010

Nemotron cost on Nebius Token Factory. Backend: where the untrusted code ran. local Docker (an image cited by its content-addressed tag). Every image is content-addressed (repo at each commit + harness), so every card re-runs from the image and seed 0. Raw data: audit, adjudication.