{"record":{"id":"ec3081a7fb16d54c","repo":"unicity-aos/aos-ce","slug":"hook-bridge-response-fan-out-on-reply-topic-lost-messages","errorCode":null,"errorMessage":"hook-bridge: response fan-out on {reply_topic} lost messages","messagePattern":"hook-bridge: response fan-out on (.+?) lost messages","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"capsules/capsule-hook-bridge/src/lib.rs","lineNumber":198,"sourceCode":"    };\n    let start = time::monotonic();\n    loop {\n        let elapsed_ms = u64::try_from((time::monotonic().saturating_sub(start)).as_millis())\n            .unwrap_or(HOOK_COLLECT_DEADLINE_MS);\n        if elapsed_ms >= HOOK_COLLECT_DEADLINE_MS {\n            break;\n        }\n        let remaining = if batch.values.is_empty() {\n            HOOK_COLLECT_DEADLINE_MS - elapsed_ms\n        } else {\n            HOOK_QUIESCENCE_MS.min(HOOK_COLLECT_DEADLINE_MS - elapsed_ms)\n        };\n        match subscription.recv(remaining) {\n            Ok(poll) if poll.messages.is_empty() => break,\n            Ok(poll) => {\n                if poll.dropped != 0 || poll.lagged != 0 {\n                    batch.complete = false;\n                    log::warn(format!(\n                        \"hook-bridge: response fan-out on {reply_topic} lost messages\"\n                    ));\n                }\n                for message in poll.messages {\n                    if message.topic != reply_topic || message.principal.verified() != principal {\n                        batch.complete = false;\n                        log::warn(format!(\n                            \"hook-bridge: dropping response with mismatched route or principal on {reply_topic}\"\n                        ));\n                        continue;\n                    }\n                    if message.payload.len() > MAX_HOOK_RESPONSE_BYTES {\n                        batch.complete = false;\n                        log::warn(format!(\n                            \"hook-bridge: dropping oversized reply on {reply_topic}\"\n                        ));\n                        continue;\n                    }","sourceCodeStart":180,"sourceCodeEnd":216,"githubUrl":"https://github.com/unicity-aos/aos-ce/blob/f6f22024fb1e8d122f28a1b4a9f75aee448ae839/capsules/capsule-hook-bridge/src/lib.rs#L180-L216","documentation":"When the hook bridge fans out a hook and collects responses from a reply-topic subscription, the poll result reports dropped and/or lagged counters. If either is nonzero, the broker dropped messages from the subscription (backpressure, slow consumer, retention expiry), so the batch cannot be considered complete; the bridge flags batch.complete = false and logs this warning.","triggerScenarios":"Raised in collect_responses (called from dispatch_hook) when subscription.recv(remaining) returns a poll whose poll.dropped != 0 or poll.lagged != 0 — i.e., the reply-topic subscription dropped or lagged over messages while gathering responses within the remaining timeout.","commonSituations":"Slow or stalled responders cause the reply subscription buffer to overflow; response bursts exceed subscription queue capacity; a consumer falls behind and the broker evicts oldest messages (lagged); reply topics configured with too-small retention/backlog limits.","solutions":["Since batch.complete is false, handle this as an incomplete result — retry the hook dispatch or surface partial results to the caller.","Increase the reply subscription's queue/backlog capacity so dropped messages don't occur under burst load.","Speed up responders or increase the recv timeout (remaining) so the consumer doesn't fall behind and lag.","Inspect broker metrics for the reply topic to identify which responder is slow and throttle or scale it."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Check poll health before trusting the batch\nif poll.dropped != 0 || poll.lagged != 0 {\n    // batch incomplete — do not consume results as authoritative\n    return Err(DispatchError::Incomplete { dropped: poll.dropped, lagged: poll.lagged });\n}","typeGuard":null,"tryCatchPattern":"// Treat incomplete batches as retryable\nmatch dispatch_hook(hook) {\n    Ok(batch) if batch.complete => use(batch),\n    Ok(batch) => schedule_retry(hook, batch),\n    Err(e) => report(e),\n}","preventionTips":["Size reply-subscription buffers for peak response bursts, not average load.","Keep responders fast and set generous recv timeouts to avoid consumer lag.","Alert on broker dropped/lagged counters for reply topics."],"tags":["rust","message-loss","backpressure","messaging"],"backgroundTag":"network-request-failed","analyzedSha":"f6f22024fb1e8d122f28a1b4a9f75aee448ae839","analyzedAt":"2026-09-13T03:04:44.565Z","contentChangedAt":"2026-09-13T03:04:44.565Z","schemaVersion":2},"datasetVersion":"2026-09-16T09:17:16.951Z"}