JuliusBrussee/caveman · error · MiddlewareError

deadline

deadline

Error message

deadline

What it means

MiddlewareError('deadline') is thrown inside optimize() by the remaining() helper once the configured deadlineMs budget (default 100ms) has elapsed. The runtime aborts further optimization work — including capability discovery — rather than adding latency to the model call. If strict mode is on this error propagates; otherwise it surfaces as a bypass diagnostic.

Solutions

1. Call await runtime.ready() (or the nonthrowing variant) at startup to warm the caps cache
2. Raise options.deadlineMs to a realistic budget for your network
3. Ensure the runtime endpoint is local and fast; remote endpoints should be avoided or well provisioned
4. Catch MiddlewareError with code 'deadline' and fall back to sending the unoptimized request

Example fix

// before
await runtime.ready(); // missing; first optimize() pays discovery cost
// after
const runtime = createMiddlewareRuntime({ deadlineMs: 250 });
await runtime.ready(); // warm caps cache outside the model call

if (!runtime.capsCache?.value) await runtime.ready(); // warm before model calls

try { const plan = await runtime.optimize(req); } catch (e) { if (e instanceof MiddlewareError && e.code === 'deadline') { /* send request unoptimized or retry */ } else throw e; }

When it happens

Trigger: performance.now() minus start exceeds deadlineMs during optimize(); capability discovery via this.http('capabilities') has to happen inside optimize() because the caps cache is cold and the fetch exhausts the remaining budget.

Common situations: First request after startup without a prior ready() call, so capability discovery eats the whole budget; a slow or remote runtime endpoint exceeding the tight 100ms default; calling optimize() on a stalled event loop.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

• AbortError
• awscreds: request failed
• cave_agent_tool_timeout | cave_network_error
• caveman trial proxy did not become ready on
• device login polling failed

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/5ba92e729df2f6cd. Report an issue: GitHub.

Appendix: source

Thrown at packages/sdk/typescript/src/middleware/runtime.ts:198

      attempt_id: token(context.attemptId ?? optimization?.request?.attempt_id),
    });
    this.reported = event;
    try {
      const completion = this.options.onReport?.(event);
      if (completion) void Promise.resolve(completion).catch(() => {});
    } catch { /* A reporting sink cannot change native behavior. */ }
    return event;
  }

  async optimize(options: OptimizeOptions): Promise<Optimization> {
    options.signal?.throwIfAborted();
    if (this.mode === 'off') return { status: 'off', reason: 'off', replacements: [], plan: null, request: null, cacheContinuity: 'off' };
    if (Date.now() < this.openUntil) return this.bypass('circuit_open');
    if (this.pending >= 16) return this.bypass('capacity');
    this.pending++;
    const start = performance.now();
    const deadline = this.options.deadlineMs ?? 100;
    const remaining = () => { const ms = deadline - (performance.now()-start); if (ms <= 0) throw new MiddlewareError('deadline'); return Math.ceil(ms); };
    try {
      scopeKey(options.scope);
      const caps = this.capsCache.value ?? validateCapabilities(await this.http('capabilities', undefined, remaining(), options.signal));
      this.capsCache.value = caps;
      if (options.candidates.length > 256 || options.manifest.length > 4096) return this.bypass('payload_limit');
      const ids = new Set<string>();
      const segments = [];
      for (const candidate of options.candidates) {
        if (!isToken(candidate.id) || ids.has(candidate.id) || !candidate.content.isWellFormed()) return this.bypass('unsupported_shape');
        ids.add(candidate.id);
        // Never transfer a protected or oversized leaf merely to get a skip.
        if (candidate.protected || candidate.opaque || byteLength(candidate.content) > caps.limits.segment_bytes) continue;
        segments.push({ id: candidate.id, source_id: candidate.sourceId ?? candidate.id, kind: candidate.kind ?? 'tool_result' as const,
          cache_region: candidate.cacheRegion ?? 'live_zone' as const, content: candidate.content, sha256: await sha256(candidate.content), protected: false, opaque: false });
      }
      if (segments.length === 0) return this.bypass('no_candidate', false);
      let binding: BindingWire | null = null;
      if (this.ownsBinding(options.binding, options.scope)) {

View on GitHub (pinned to 3ee70a1026)