Breakwater

Codeflash / patterns#12

⚡️ Speed up function sanitize_pattern by 11,547%

fabriziosalmi/patterns#12 · merged 2025-02-09 · base 1a4a2d4e42 → head fb284892f1 · ran on Sandboxes

BLOCK

At least one behaviour change looks like an unintended regression.

Counterexamples · 1 class: 1 likely bug

P2

Exception behaviour changed

inputsanitize_pattern([["pattern", "0"]])
baseraises AttributeError
headraises TypeError

Neither exception type subclasses the other.

Likely bugvotes: 3 likely-bug

The PR is presented as a pure performance refactor (caching) with no intended behavior change. The new exception type (TypeError) differs from the old (AttributeError) even though the core logic (calling re.escape) is unchanged, indicating an unintended regression.

⚡️ Speed up function `sanitize_pattern` by 11,547%
return _sanitize_pattern(pattern)

Found by: Nemotron strategy search, search sample · no baseline found it

ran onToken Factory Sandboxes · image 87c6903e-46fd-4cf5-a313-2b45b6b1248c

Regression test verified passes on base, errors on head
def test_patterns_12_p2_outcome_0():
    """Breakwater P2: base behaviour pinned (found by differential testing; label: likely-bug)."""
    import pytest
    from json2nginx import sanitize_pattern
    with pytest.raises(AttributeError) as excinfo:
        sanitize_pattern([['pattern', '0']])
    assert type(excinfo.value) is AttributeError  # exact type: a subclass would be a behaviour change

Regression tests

1 of 1 classes have a generated pytest that pins the old behaviour. Each passed on the base commit and failed on the head commit in the sandbox before it was offered.

Download test_patterns_12.py

Performance · sanitize_pattern

No realistic input family measured slower.

head slowerhead faster0.062×0.12×0.25×0.5×1×100input size (characters)F2F2: 0.07× at ~17 charsF2: 0.06× at ~17 charsF2: 0.06× at ~17 charsF2: 0.06× at ~17 charsF2: 0.06× at ~17 charsF2: 0.06× at ~17 charsF2: 0.06× at ~17 chars
  • F2
Head/base cachegrind instructions per call, by input size (log-log). Each line is one input family Nemotron proposed; points above 1× are where the "faster" code does more work.
FamilyInputsRatio range Slope base → headVerdict
F2Repeated identical pattern strings – cache hits after the first call.claimed speedup · plan0.06× – 0.07×0.00 → 0.00faster

Receipt

TargetBackendImageRun AuditAdjudication
json2nginx:sanitize_patternToken Factory Sandboxes87c6903e-46fd-4cf5-a313-2b45b6b1248cbt-patterns#12-0dc7f0$0.051$0.016

Nemotron cost on Nebius Token Factory. Backend: where the untrusted code ran. Token Factory Sandboxes (a microVM branched from a kept image, cited by its UUID). Every image is content-addressed (repo at each commit + harness), so every card re-runs from the image and seed 0. Raw data: audit, adjudication.