[request-log] ${command.kind === "raw-trace-update" ? "Retai

Error message

[request-log] ${command.kind === "raw-trace-update" ? "Retaining" : "Dropping"} ${command.kind} sequence ${command.sequence} after ${command.writeAttempts} failed write attempts: ${message.error || "request log batch failed"}

What it means

A request-log write command failed repeatedly (tracked by writeAttempts). Once attempts exceed a threshold, raw-trace-update commands are retained (retried later) while other command kinds are dropped, and this warning names the exact command kind, sequence number, attempt count, and underlying error. Log data is being lost or delayed.

Source

Thrown at packages/core/src/observability/request-log-runtime.ts:785

      this.schedulePump(true);
      return;
    }

    const command = batch.commands[0];
    if (command.writeAttempts < maxCommandWriteAttempts) {
      this.queue.unshift({ ...command, isolated: true });
    } else {
      this.outstandingBytes = Math.max(0, this.outstandingBytes - command.sizeBytes);
      this.dropped += 1;
      if (command.kind === "record") {
        this.rememberRecordAdmission(command.input.requestId, {
          accepted: false,
          degraded: false,
          reason: "writer_unavailable"
        }, 0, resolveRecordBodyCapturePolicy(command.input));
        this.scheduleRawTraceCleanup([command]);
      }
      console.warn(
        `[request-log] ${command.kind === "raw-trace-update" ? "Retaining" : "Dropping"} ` +
        `${command.kind} sequence ${command.sequence} after ` +
        `${command.writeAttempts} failed write attempts: ${message.error || "request log batch failed"}`
      );
    }
    this.schedulePump(true);
  }

  private handleWriterFailure(worker: Worker | undefined, error: Error): void {
    if (!worker || worker !== this.writerWorker) return;
    this.writerWorker = undefined;
    this.writerWorkerReady = undefined;
    void worker.terminate().catch(() => undefined);
    for (const [, batch] of [...this.inFlight].reverse()) this.queue.unshift(...batch.commands);
    this.inFlight.clear();
    rejectPending(this.writerRequests, error);
    if (this.closed) return;
    this.writerRestartCount += 1;

View on GitHub (pinned to 99f24806c6)

Solutions

Free disk space or fix permissions on the log storage path
Check message.error for the root cause (SQLITE_BUSY, EACCES, ENOSPC) and address it
If a lock conflict, stop competing processes accessing the log store or enable WAL
Restart the service after fixing storage — dropped non-trace commands cannot be recovered

Example fix

# before: log store on full disk
LOG_DB_PATH=/mnt/full-disk/logs.db

# after
LOG_DB_PATH=/var/lib/gateway/logs.db  # ensure >=10% free space

Defensive patterns

Strategy: retry

Validate before calling

const stat = await fs.promises.stat(logDir);
if (stat && (await fs.promises.statfs(logDir)).bsize * (await fs.promises.statfs(logDir)).bavail < 100 * 1024 * 1024) {
  throw new Error("<100MB free for request logs");
}

Try / catch

try { await writer.write(commands); } catch (e) { await persistToDeadLetter(commands, e); /* never drop silently */ }

Prevention

Disk space alerts on the log volume
Avoid concurrent processes locking the log store
Drain and verify writer health before high-traffic windows

When it happens

Trigger: Persistent writer failures: disk full, DB locked by another process, permission errors on the log file, or a corrupted log store; each pump retry increments writeAttempts until the retention/drop decision fires.

Common situations: Log directory on a full volume; SQLite log DB locked by a backup job; readonly filesystem in a container; crash-looping writer.

Related errors

AI-assisted analysis of musistudio/claude-code-router@99f24806c6 (2026-08-27). Data as JSON: /api/errors/7f2a719356732293. Report an issue: GitHub.