bytedance/deer-flow · critical · SystemExit

scheduler.multi_instance=true requires run_events.backend='d

Error message

scheduler.multi_instance=true requires run_events.backend='db'. run_events.backend is '{run_events_backend}'. Set scheduler.multi_instance=false or configure run_events.backend: db.

What it means

SystemExit raised at startup when scheduler.multi_instance=true but run_events.backend is not 'db'. With multiple scheduler instances, run lifecycle events must live in the shared database; memory/JSONL stores are process-local, so peer schedulers could not observe each other's scheduled-run events and coordination guarantees would silently break.

Source

Thrown at backend/app/gateway/deps.py:98

    immediately.
    """
    try:
        workers = int(os.environ.get("GATEWAY_WORKERS", "1"))
    except (TypeError, ValueError):
        workers = 1

    scheduler = getattr(config, "scheduler", None)
    multi_instance_requested = bool(getattr(scheduler, "multi_instance", False))
    multi_instance_scheduler = bool(getattr(scheduler, "enabled", False) and multi_instance_requested)

    backend = getattr(config.database, "backend", None)
    run_events_backend = getattr(getattr(config, "run_events", None), "backend", None)
    run_ownership = getattr(config, "run_ownership", None)

    if multi_instance_requested and backend != "postgres":
        raise SystemExit(f"scheduler.multi_instance=true requires database.backend='postgres'. database.backend is '{backend}'. Set scheduler.multi_instance=false or configure Postgres.")
    if multi_instance_requested and run_events_backend != "db":
        raise SystemExit(f"scheduler.multi_instance=true requires run_events.backend='db'. run_events.backend is '{run_events_backend}'. Set scheduler.multi_instance=false or configure run_events.backend: db.")
    if multi_instance_requested and (run_ownership is None or not run_ownership.heartbeat_enabled):
        raise SystemExit("scheduler.multi_instance=true requires run_ownership.heartbeat_enabled=true so peer runs retain a valid lease. Set scheduler.multi_instance=false or enable run ownership heartbeats.")

    if workers <= 1:
        return

    if config.scheduler.enabled and not multi_instance_scheduler:
        raise SystemExit(f"GATEWAY_WORKERS={workers} cannot run with scheduler.enabled=true because each worker starts its own scheduler. Set GATEWAY_WORKERS=1, scheduler.multi_instance=true, or scheduler.enabled=false.")

    if _browser_tools_enabled_in_config(config):
        raise SystemExit(browser_multi_worker_error(workers))

    if backend != "postgres":
        raise SystemExit(f"GATEWAY_WORKERS={workers} requires database.backend='postgres', but database.backend is '{backend}'. SQLite cannot support concurrent multi-process access. Set GATEWAY_WORKERS=1 or switch to Postgres.")

    if run_events_backend != "db":
        raise SystemExit(
            f"GATEWAY_WORKERS={workers} requires run_events.backend='db', but run_events.backend is '{run_events_backend}'. "

View on GitHub (pinned to 1dd6ba1acb)

Solutions

  1. Set run_events.backend: db in config.yaml
  2. Or set scheduler.multi_instance: false
  3. Restart the Gateway and confirm startup succeeds

Example fix

# config.yaml
# before
run_events:
  backend: memory
# after
run_events:
  backend: db
Defensive patterns

Strategy: validation

Validate before calling

cfg = load_config()
mi = bool(getattr(getattr(cfg, "scheduler", None), "multi_instance", False))
re_backend = getattr(getattr(cfg, "run_events", None), "backend", None)
assert not (mi and re_backend != "db"), "multi_instance needs run_events.backend: db"

Prevention

When it happens

Trigger: config.yaml with scheduler.multi_instance: true and run_events.backend set to memory or jsonl (or left at its default), for any database backend.

Common situations: Enabling multi_instance on a config that never configured run_events; default run_events backend differing from db in the example config; environment drift between staging and production config.

Related errors


AI-assisted analysis of bytedance/deer-flow@1dd6ba1acb (2026-08-14). Data as JSON: /api/errors/17271810b64b2c23. Report an issue: GitHub.