Hmbown/CodeWhale · error · Error

Too many active Engine pet spans.

Error message

Too many active Engine pet spans.

What it means

The internal `start()` helper opens a new pet span unless 256 spans are already active. This error means the engine has hit its concurrency cap of 256 live spans and the incoming event would start span number 257 with a key that is not already active. It protects memory from runaway/unpaired span starts.

Solutions

  1. Ensure every started span gets a matching finish event so it is removed from the active map.
  2. Emit a `turn_started` event between turns — the engine clears `this.active` on it, releasing leaked spans.
  3. Audit for duplicated span keys (same id reused) that keep old spans alive; keys must be unique per open span.
  4. Back-pressure: batch or defer events when the number of concurrently open spans approaches 256.
  5. On this error, treat it as a leak signal — log `this.active.size` and the keys still open to find the unpaired starts.

Example fix

// before
for (const t of manyToolCalls) start(t); // never finished, > 256 open

// after
for (const t of manyToolCalls) {
  start(t);
  finish(t); // pair every start with a finish
}
Defensive patterns

Strategy: try-catch

Validate before calling

// no pre-call validation possible: cap depends on engine's internal active-span map
// guard at the application level by tracking how many spans you have open:
if (openSpanCount >= 250) await drainPendingFinishes(); // close spans before opening more

Try / catch

try {
  engine.observe(event, at);
} catch (err) {
  if (err instanceof Error && err.message === 'Too many active Engine pet spans.') {
    console.error('span leak: >256 open spans; emitting turn_started to reset');
    engine.observe({ event: 'turn_started' }, at); // clears active spans
    engine.observe(event, at); // retry once
    return;
  }
  throw err;
}

Prevention

When it happens

Trigger: Emitting many `*_started` events (messages, tools, turns) whose matching finish events never arrive, so spans accumulate in `this.active`; a high-volume loop that opens more than 256 distinct keyed spans before closing any.

Common situations: Unpaired start/finish pairs after an error path skipped the finish call; a `turn_started` clear() never being observed between turns; parallel workers each opening tool/message spans beyond 256 concurrently; a long-running session where spans leak one per turn.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of Hmbown/CodeWhale@433685b202 (2026-09-15). Data as JSON: /api/errors/e87e10bf1d34659c. Report an issue: GitHub.

Appendix: source

Thrown at pet/src/core/pet-engine.ts:89

  observe(value: unknown, at: number): void {
    if (!Number.isFinite(at) || at < this.lastTime || at > PET_MAX_SECONDS * 1000) throw new Error('Invalid Engine pet clock.');
    if (!value || typeof value !== 'object' || Array.isArray(value)) throw new Error('Invalid Engine pet metadata.');
    const e = value as Record<string, unknown>;
    const allowed = ['event', 'index', 'channel', 'tool_call_id', 'tool_name', 'id', 'worker_status', 'failed'];
    if (Object.keys(e).some(k => !allowed.includes(k)) || typeof e.event !== 'string'
      || Object.values(e).some(v => typeof v === 'string' && v.length > 4096)
      || e.channel !== undefined && !['text', 'reasoning'].includes(e.channel as string)
      || ['tool_call_id', 'tool_name', 'id', 'worker_status'].some(k => e[k] !== undefined && typeof e[k] !== 'string')
      || e.failed !== undefined && typeof e.failed !== 'boolean'
      || e.index !== undefined && (!Number.isSafeInteger(e.index) || (e.index as number) < 0))
      throw new Error('Invalid Engine pet metadata fields.');
    this.lastTime = at;
    this.events = this.events.filter(span => span.endTime >= at - 12_800);
    const id = (field: string) => { const s = e[field]; if (typeof s !== 'string' || !s) throw new Error(`Missing Engine ${field}.`); return s; };
    const index = () => { if (!Number.isSafeInteger(e.index)) throw new Error('Missing Engine index.'); return String(e.index); };
    const start = (key: string, name: string, category: Category, agentId?: string) => {
      if (this.active.size >= 256 && !this.active.has(key)) throw new Error('Too many active Engine pet spans.');
      this.active.set(key, this.add(name, category, at, agentId));
    };
    const finish = (key: string) => { this.pulse(key, at); this.active.delete(key); };
    switch (e.event) {
      case 'turn_started': this.active.clear(); this.waiting = undefined; break;
      case 'message_started': start(`message:${index()}`, 'assistant_message', 'communication'); this.waiting = undefined; break;
      case 'thinking_started': start(`thinking:${index()}`, 'thinking', 'reasoning'); this.waiting = undefined; break;
      case 'response_delta': {
        const reasoning = e.channel === 'reasoning';
        const key = `${reasoning ? 'thinking' : 'message'}:${index()}`;
        if (!this.active.has(key)) start(key, reasoning ? 'thinking' : 'assistant_message', reasoning ? 'reasoning' : 'communication');
        else this.pulse(key, at);
        this.waiting = undefined; break;
      }
      case 'message_complete': finish(`message:${index()}`); break;
      case 'thinking_complete': finish(`thinking:${index()}`); break;
      case 'tool_call_started': start(`tool:${id('tool_call_id')}`, id('tool_name'), toolCategory(id('tool_name'))); this.waiting = undefined; break;
      case 'tool_call_heartbeat': for (const key of this.active.keys()) if (key.startsWith('tool:')) this.pulse(key, at); break;

View on GitHub (pinned to 433685b202)