What causes "items_json must be JSON: {e}" in headroomlabs-ai/headroom?

Serialize before calling: pass json.dumps(items), never str/repr of a Python object.

items_json must be JSON: {e}

Error message

items_json must be JSON: {e}

What it means

Python-binding panic (not an exception): crush_array_json received an items_json string that serde_json could not parse. The Rust side deliberately panics inside the GIL-released closure because pyo3 panics cross the FFI boundary as Python RuntimeError — it treats malformed JSON as a caller contract violation.

Source

Thrown at crates/headroom-py/src/lib.rs:821

    /// runtime can reach the CCR hash directly (rather than parsing it
    /// out of the prompt marker).
    #[pyo3(signature = (items_json, query = "", bias = 1.0))]
    fn crush_array_json<'py>(
        &self,
        py: Python<'py>,
        items_json: &str,
        query: &str,
        bias: f64,
    ) -> Bound<'py, PyDict> {
        // GIL-release pattern: own the inputs, do all heavy compute
        // (JSON parse, crush, re-serialize) without the GIL, then
        // re-acquire to build the PyDict from the owned outputs.
        let items_json = items_json.to_string();
        let query = query.to_string();
        let (kept_json, ccr_hash, dropped_summary, strategy_info, compacted, compaction_kind) = py
            .detach(|| {
                let parsed: serde_json::Value = serde_json::from_str(&items_json)
                    .unwrap_or_else(|e| panic!("items_json must be JSON: {e}"));
                let items = match parsed {
                    serde_json::Value::Array(a) => a,
                    other => panic!("items_json must be a JSON array, got {}", type_name(&other)),
                };
                let result = self.inner.crush_array(&items, &query, bias);
                let kept_json = serde_json::to_string(&serde_json::Value::Array(result.items))
                    .expect("serialize kept items");
                (
                    kept_json,
                    result.ccr_hash,
                    result.dropped_summary,
                    result.strategy_info,
                    result.compacted,
                    result.compaction_kind,
                )
            });
        build_crush_array_dict(
            py,

View on GitHub (pinned to 322425c43b)

Solutions

Serialize before calling: pass json.dumps(items), never str/repr of a Python object.
Validate on the Python side with json.loads(items_json) first if the string's provenance is uncertain.
Handle empty input before the call (return early or pass '[]').
Wrap the call in try/except RuntimeError to convert the panic into a normal Python error path for defense in depth.

Example fix

# before
result = hr.crush_array_json(str(items), query, 0.5)  # str() of a list is not JSON

# after
import json
result = hr.crush_array_json(json.dumps(items), query, 0.5)

Defensive patterns

Strategy: validation

Validate before calling

# Validate before calling
import json
def safe_crush(hr, items_json, query, bias):
    if not isinstance(items_json, str) or not items_json.strip():
        raise ValueError("items_json must be a non-empty JSON string")
    json.loads(items_json)  # raises JSONDecodeError with position, better than a Rust panic
    return hr.crush_array_json(items_json, query, bias)

Type guard

def is_json_string(s: str) -> bool:
    try:
        json.loads(s)
        return True
    except (json.JSONDecodeError, TypeError):
        return False

Try / catch

try:
    result = hr.crush_array_json(json.dumps(items), query, bias)
except RuntimeError as e:  # pyo3 panic surfaces as RuntimeError
    log.error("crush failed: %s", e)
    raise

Prevention

Always json.dumps Python objects before passing — never str()/repr().
json.loads-validate strings from untrusted sources before the FFI call.
Handle empty input before calling (pass '[]').

When it happens

Trigger: Calling headroom.crush_array_json(items_json, query, bias) with a non-JSON string: Python dict/list passed without json.dumps, a truncated string, single-quoted Python-repr JSON, a JSONDecodeError-causing payload, or an empty string.

Common situations: Passing repr(items) or str(items) instead of json.dumps(items); reading JSON from a file/socket that was truncated; encoding/decoding mismatches (e.g. BOM, latin-1) making the string unparseable; empty input '' on startup.

Related errors

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/bb37700aa46c30e1. Report an issue: GitHub.