bytedance/deer-flow · warning · QueueFull

memory update queue is full (depth {len(self._items)} >= {ma

Error message

memory update queue is full (depth {len(self._items)} >= {max_depth}); non-signal update for thread {thread_id} rejected

What it means

QueueFull from deermem's memory update queue: backpressure logic rejects NEW non-signal, non-bypass updates once queue depth reaches queue_max_depth. Same-key updates merge (never grow depth), and signal-bearing or emergency (bypass_watermark) items are always admitted because they cannot be re-fed next turn — only deferrable normal updates are shed.

Source

Thrown at backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/queue.py:188

        # Emergency (bypass) and normal updates coexist: the match key includes
        # ``bypass_watermark`` so a summarization flush (bypass=True) never
        # replaces a pending normal update for the same (thread, user, agent) --
        # replacing it would drop the normal update's un-extracted tail, which
        # the next turn may not re-feed if the user stops. Both are processed
        # independently instead.
        existing = next(
            (c for c in self._items if queue_key(c.thread_id, c.user_id, c.agent_name) == key and c.bypass_watermark == bypass_watermark),
            None,
        )
        # Backpressure: once depth reaches the cap, reject NEW non-signal normal
        # items. Same-key updates merge (do not grow depth); signal-bearing items
        # and emergency (bypass) flushes are always admitted. Signals capture
        # important memories, and the emergency path captures messages about to
        # be removed by summarization -- neither can be re-fed next turn, so
        # shedding them under load would lose data rather than merely defer it.
        max_depth = self._config.queue_max_depth
        if max_depth > 0 and not bypass_watermark and not signals and existing is None and len(self._items) >= max_depth:
            raise QueueFull(f"memory update queue is full (depth {len(self._items)} >= {max_depth}); non-signal update for thread {thread_id} rejected")

        # Merge by signal union: a signal seen on any update for this key stays.
        merged_signals = signals | (existing.signals if existing is not None else frozenset())
        context = ConversationContext(
            thread_id=thread_id,
            messages=messages,
            agent_name=agent_name,
            user_id=user_id,
            trace_id=trace_id,
            signals=merged_signals,
            bypass_watermark=bypass_watermark,
        )
        if existing is not None:
            self._items = [c for c in self._items if not (queue_key(c.thread_id, c.user_id, c.agent_name) == key and c.bypass_watermark == bypass_watermark)]
        self._items.append(context)
        return context

    def _reset_timer(self) -> None:

View on GitHub (pinned to 1dd6ba1acb)

Solutions

Catch QueueFull at the enqueue call site and drop/defer the update (it is deferrable by design — next turn re-feeds messages).
Raise queue_max_depth in the deermem config if bursts are legitimate.
Investigate why depth stays at cap: check the drain/consumer loop health (DB locks, slow writes) and fix the stall.
If the update matters, attach a signal or use the emergency path rather than retrying the normal enqueue.

Example fix

# before
queue.enqueue(ctx)  # raises QueueFull under load

# after
try:
    queue.enqueue(ctx)
except QueueFull:
    logger.warning("memory update deferred for thread %s", ctx.thread_id)
    # safe to skip: non-signal updates are re-fed next turn

Defensive patterns

Strategy: try-catch

Validate before calling

def can_enqueue(queue, bypass=False, signals=frozenset()) -> bool:
    if signals or bypass:
        return True  # always admitted
    return len(queue._items) < queue._config.queue_max_depth or queue._config.queue_max_depth <= 0

Try / catch

from deermem.core.queue import QueueFull
try:
    queue.enqueue(ctx)
except QueueFull:
    # by design: non-signal updates are deferrable — drop and log, next turn re-feeds
    logger.warning('memory update deferred (queue full) thread=%s', ctx.get('thread_id'))

Prevention

Never retry a rejected normal enqueue in a tight loop — it is shed load, not a transient error.
Monitor queue depth; sustained cap means the consumer is stalled, fix that.
Size queue_max_depth against peak turns-per-minute times worst drain latency.
Route genuinely important updates through signals or the emergency path.

When it happens

Trigger: Enqueueing a normal (no signals, no bypass) memory update while len(queue._items) >= config.queue_max_depth (>0), for a thread/user/agent key not already present in the queue.

Common situations: Memory consumer/writer slower than the agent producing updates (long-running high-traffic threads); queue_max_depth set too low for bursty traffic; background writer stalled (SQLite lock, IO hang) so depth monotonically grows.

Related errors

AI-assisted analysis of bytedance/deer-flow@1dd6ba1acb (2026-08-14). Data as JSON: /api/errors/7ef8c9714c3f8475. Report an issue: GitHub.