headroomlabs-ai/headroom · error · NotImplementedError

SmartCrusher: custom `relevance_config` / `scorer` overrides

Error message

SmartCrusher: custom `relevance_config` / `scorer` overrides are not yet supported by the Rust-backed implementation. Pass `None` to use the default HybridScorer. Tracked in RUST_DEV.md; full support lands with Stage 3c.2's relevance-crate Python bridge.

What it means

SmartCrusher was ported to a Rust backend that always uses the crate's built-in HybridScorer; the `relevance_config` and `scorer` constructor parameters remain in the signature for source compatibility but any non-None value raises NotImplementedError. The project's no-silent-fallbacks policy means a custom scorer you supplied will never be silently ignored — it fails loud instead. Full support is tracked for Stage 3c.2 (relevance-crate Python bridge).

Source

Thrown at headroom/transforms/smart_crusher.py:346

        # opaque-string CCR substitutions still emit always — they have
        # no Python equivalent and no production caller has asked for
        # them to be suppressed.
        if ccr_config is None:
            self._ccr_config = CCRConfig()
        else:
            self._ccr_config = ccr_config

        # `relevance_config` and `scorer` remain in the signature for
        # source compatibility, but the Rust port doesn't support
        # overrides yet (it always uses `HybridScorer` from the
        # relevance crate; the Python-bridged constructor surface
        # arrives in Stage 3c.2). Silently dropping a user-supplied
        # scorer would be a textbook silent fallback — if a caller
        # depends on a custom scoring function and we ignore it, the
        # compression they get back is wrong in a way they cannot see.
        # Fail loud instead. See `feedback_no_silent_fallbacks.md`.
        if relevance_config is not None or scorer is not None:
            raise NotImplementedError(
                "SmartCrusher: custom `relevance_config` / `scorer` "
                "overrides are not yet supported by the Rust-backed "
                "implementation. Pass `None` to use the default "
                "HybridScorer. Tracked in RUST_DEV.md; full support "
                "lands with Stage 3c.2's relevance-crate Python bridge."
            )

        # Lazy TOIN handle. Loaded on first compression that has items
        # to learn from. Skipping import at __init__ keeps cold-start
        # fast for environments where telemetry is disabled.
        self._toin: Any = None
        self._toin_load_failed = False

        # F2.2: per-request CompressionPolicy, set from
        # ``kwargs["compression_policy"]`` at the start of ``apply()``
        # and read by ``_record_to_toin`` to gate TOIN writes when
        # ``policy.toin_read_only`` is true (Subscription mode).
        # Defaults to ``None`` so the direct ``crush()`` / ``crush_array_json()``

View on GitHub (pinned to 322425c43b)

Solutions

Pass `None` for both parameters and use the default HybridScorer (the intended current usage)
Pin your dependency to the last Python-backed SmartCrusher release until Stage 3c.2 ships the relevance-crate Python bridge
If you must influence scoring now, check whether the supported knobs (e.g. lossless_only, compaction options) cover your use case instead of a custom scorer

Example fix

# before
crusher = SmartCrusher(relevance_config=my_cfg, scorer=my_fn)

# after
crusher = SmartCrusher()  # default HybridScorer from the relevance crate
# dependency pin if custom scoring is required:
#   pip install "headroom-ai==<last-python-backed-version>"

Defensive patterns

Strategy: type-guard

Validate before calling

def supports_custom_scorer() -> bool:
    # Stage 3c.2 (relevance-crate Python bridge) is the gate
    return False  # update when the bridge ships

if my_scorer is not None and not supports_custom_scorer():
    raise RuntimeError("custom scorer unsupported; refusing to configure SmartCrusher")

Type guard

from dataclasses import is_dataclass

def smart_crusher_args_safe(relevance_config, scorer) -> bool:
    return relevance_config is None and scorer is None

Try / catch

try:
    crusher = SmartCrusher(relevance_config=cfg, scorer=fn)
except NotImplementedError:
    crusher = SmartCrusher()  # explicit downgrade decision, logged
    log.warning("custom scorer dropped: Rust-backed SmartCrusher")

Prevention

Feature-detect at startup and refuse to run (or explicitly downgrade with a log) if custom scoring is configured
Pin the dependency version in requirements so the Rust port does not arrive as a surprise upgrade
Track RUST_DEV.md / Stage 3c.2 before reintroducing custom scorers

When it happens

Trigger: Constructing `SmartCrusher(relevance_config=cfg)` or `SmartCrusher(scorer=my_scorer)` on the Rust-backed version — anything other than leaving both as None.

Common situations: Migrating existing code from the pre-Rust Python SmartCrusher that supported custom scoring; copying older examples/docs that pass a relevance config; a dependency upgrade pulled the Rust-backed headroom version in transitively.

Related errors

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/618289974d74e38d. Report an issue: GitHub.