unicity-aos/aos-ce · warning

failed to deserialize compaction response payload

Error message

failed to deserialize compaction response payload: {e}

What it means

parse_hook_responses reads compaction hook reply messages and deserializes each payload as JSON. A payload that fails serde_json::from_str triggers this warning and the message is skipped; valid responses are still collected. It exists so one malformed hook reply cannot abort parsing of the whole compaction response batch.

Solutions

1. Fix the offending hook to publish replies through serde_json (or the language's JSON serializer) with the documented response envelope.
2. Log the reply topic and sender principal with the parse error to locate the faulty hook.
3. Add a producer-side test that round-trips the compaction response through serde_json before publishing.
4. If partial responses are expected, decide policy: retry failed hooks or proceed with the parsed subset.

Example fix

// before: hook publishes a bare string error
bus.publish(reply_topic, format!("hook failed: {err}"));
// after: hook publishes a JSON envelope
let reply = serde_json::json!({ "ok": false, "error": err.to_string() });
bus.publish(reply_topic, serde_json::to_string(&reply)?);

// pre-validate hook reply before publishing
serde_json::to_string(&reply).expect("hook reply must be JSON");

fn parse_reply(payload: &str) -> Option<serde_json::Value> {
    serde_json::from_str(payload).ok()
}

When it happens

Trigger: fire_before_compaction (or parsing from a poll result) receives hook replies where msg.payload is not valid JSON — a hook capsule crashed mid-write, published an empty string, or emitted a non-JSON error blob.

Common situations: A hook adapter returning raw error text instead of a JSON envelope; concurrent writers interleaving on a channel; hook implementations in other languages with buggy serializers; unit tests parse_hook_responses_nested_in_custom_data / mixed_valid_and_invalid deliberately feed bad payloads.

Understand the failure class

Background: JSON parse error: "Unexpected token" / "not valid JSON" / "failed to parse" — what JSON parsers are really complaining about — this error's family across 45 libraries.

Related errors

• failed to deserialize IPC message payload
• hook-adapter-oracle: incomplete context fan-out on
• hook-adapter-oracle: dropping mismatched context reply on
• canonical document exceeds bound
• meta-harness: malformed hook request

AI-assisted analysis of unicity-aos/aos-ce@f6f22024fb (2026-09-13). Data as JSON: /api/errors/6b839b384cb503d3. Report an issue: GitHub.

Appendix: source

Thrown at capsules/capsule-context-engine/src/lib.rs:591

    if !responses.is_empty() {
        log::info(format!(
            "Collected {} context_engine.v1.hook.before_compaction responses",
            responses.len()
        ));
    }

    merge_before_compaction_responses(&responses)
}

/// Parse hook responses from a typed `PollResult`.
fn parse_hook_responses(result: &ipc::PollResult) -> Vec<BeforeCompactionHookResponse> {
    let mut responses = Vec::new();

    for msg in &result.messages {
        let payload: serde_json::Value = match serde_json::from_str(&msg.payload) {
            Ok(v) => v,
            Err(e) => {
                log::warn(format!(
                    "failed to deserialize compaction response payload: {e}"
                ));
                continue;
            }
        };

        // Try direct payload, then nested in Custom `data` envelope.
        let maybe_response =
            serde_json::from_value::<BeforeCompactionHookResponse>(payload.clone())
                .ok()
                .filter(BeforeCompactionHookResponse::has_any_field)
                .or_else(|| {
                    payload
                        .get("data")
                        .and_then(|data| {
                            serde_json::from_value::<BeforeCompactionHookResponse>(data.clone())
                                .ok()
                        })

View on GitHub (pinned to f6f22024fb)