bytedance/deer-flow · error · HTTPException

fan-out failed for delivery {x_github_delivery!r}: {exc!r}

Error message

fan-out failed for delivery {x_github_delivery!r}: {exc!r}

What it means

Raised by the GitHub webhook receiver with status 503 when publishing the verified event onto the channel bus (fan-out to custom agents via the ChannelManager) raises any exception. 503 signals a recoverable server-side failure so GitHub will retry the delivery; the message embeds the delivery id and the causing exception for correlation with the logged traceback.

Source

Thrown at backend/app/gateway/routers/github_webhooks.py:365

            # failed so that same manual click, REST call, or recovery
            # script actually finds and recovers it. The startup-time
            # ``is_route_enabled`` check still covers fail-closed
            # *configuration* errors.
            try:
                dispatch_result = await fanout_event(
                    service.bus,
                    x_github_event,
                    x_github_delivery,
                    payload,
                    operator_default_mention_login=operator_default_mention_login,
                )
            except Exception as exc:  # noqa: BLE001 — re-raised as 503 below
                logger.exception(
                    "github_webhook: fanout failed (delivery=%s event=%s) — returning 503 (recoverable via manual/API redelivery)",
                    x_github_delivery,
                    x_github_event,
                )
                raise HTTPException(
                    status_code=503,
                    detail=f"fan-out failed for delivery {x_github_delivery!r}: {exc!r}",
                ) from exc
    else:
        logger.info(
            "github_webhook delivery=%s | unhandled event=%s action=%s repo=%s",
            x_github_delivery,
            x_github_event,
            payload.get("action"),
            (payload.get("repository") or {}).get("full_name"),
        )
        handled = False
        dispatch_result = None

    return {
        "ok": True,
        "event": x_github_event,
        "delivery": x_github_delivery,

View on GitHub (pinned to 1dd6ba1acb)

Solutions

1. Check Gateway logs for the 'github_webhook: fanout failed' exception traceback and fix the underlying bus/channel error it names.
2. Verify the channel bus service is up (same stack: make docker-logs / service health endpoints).
3. After fixing, redeliver the failed event from GitHub's webhook delivery log or via the API — 503 tells GitHub to retry automatically within its schedule.

Example fix

# before: bus down, delivery lost from view
# webhook logs: fanout failed ... 503

# after
make docker-logs SERVICE=gateway  # read traceback, e.g. stream store connection refused
make up                          # restore the bus, GitHub auto-retries (or Redeliver in webhook UI)

# preflight the dependency the fan-out needs
 curl -fsS http://gateway:8001/health || echo 'gateway unhealthy — fix before expecting webhook fan-out'

try { await postWebhook(payload); } catch (e) { if (e.status === 503) { await backoffRetry(3); } throw e; }

When it happens

Trigger: Channel bus backend down (Redis/stream store unavailable); a channel adapter raising on a malformed repo/action combination; agent-routing state missing after a partial restart; serialization errors on unusual payload shapes.

Common situations: Channel/IM infrastructure (the bus the ChannelManager consumes) not running when webhooks arrive; transient DB/stream outage; version skew during rolling deploys where the event publisher and channel service disagree on schema.

Related errors

• Webhook signature verification not configured. Set {_SECRET_
• Failed to polish input
• Configuration not available
• {label} not available
• Thread metadata store not available

AI-assisted analysis of bytedance/deer-flow@1dd6ba1acb (2026-08-14). Data as JSON: /api/errors/335d00895c33b5c1. Report an issue: GitHub.