Hmbown/CodeWhale · error · Error

Too many active Engine pet spans.

Error message

Too many active Engine pet spans.

What it means

The start() helper in observe enforces a concurrency bound of 256 simultaneously active spans (tool:, thinking:, message:, agent: keys). It throws 'Too many active Engine pet spans.' when a new span key would exceed that cap and the key is not already active. This bounds memory and keeps the activity view meaningful.

Solutions

1. Ensure every started span has a matching completion event (tool_call_complete, agent_complete, *_complete)
2. Clear stale spans on turn boundaries — turn_started/turn_complete already call active.clear(); route through turn events
3. Cap actual parallelism below 256 or coalesce per-agent spans
4. On replay, feed turn_started first so prior leaked keys are cleared

Example fix

// before
for (const c of manyCalls) telemetry.observe({ event: 'tool_call_started', ... }, at); // 300 starts, no completes
// after
telemetry.observe({ event: 'turn_started' }, at);
for (const c of manyCalls.slice(0, 256)) {
  telemetry.observe({ event: 'tool_call_started', tool_call_id: c.id, tool_name: c.name }, at);
  telemetry.observe({ event: 'tool_call_complete', tool_call_id: c.id }, at);
}

if (telemetry.active.size >= 256 && !telemetry.active.has(key)) {
  throw new Error('Too many active Engine pet spans.');
}
telemetry.observe({ event: 'tool_call_started', tool_call_id: id, tool_name: name }, at);

try {
  telemetry.observe(meta, at);
} catch (err) {
  if (err.message === 'Too many active Engine pet spans.') {
    telemetry.active.clear(); // or emit synthetic completes for leaked keys, then retry
    telemetry.observe(meta, at);
  } else throw err;
}

When it happens

Trigger: More than 256 tool calls or agent spawns without matching completion events; tool_call_started storms without tool_call_complete; agent_spawned events for a fleet larger than the cap; leaked keys from missing *_complete events accumulating across a turn.

Common situations: Parallel subagent fan-out beyond 256 live workers; a dropped/lost completion event (crash, cancel path) leaving keys in active forever; a replay that spawns many keys but whose complete events arrive at timestamps already pruned/rejected.

Related errors

• Archive the legacy recording before accepting more…
• Pet Engine observation window is full.
• telemetry permission changed before arming
• Too many active Engine pet spans.
• A saved trace identifier collided. Try saving again.

AI-assisted analysis of Hmbown/CodeWhale@73e0f67d83 (2026-09-22). Data as JSON: /api/errors/d1b577bf5f5ce7ae. Report an issue: GitHub.

Appendix: source

Thrown at crates/tui/src/tui/pet_watch/pet-native.js:1749

            throw new Error('Invalid Engine pet metadata.');
        const e = value;
        const allowed = ['event', 'index', 'channel', 'tool_call_id', 'tool_name', 'id', 'worker_status', 'failed'];
        if (Object.keys(e).some(k => !allowed.includes(k)) || typeof e.event !== 'string'
            || Object.values(e).some(v => typeof v === 'string' && v.length > 4096)
            || e.channel !== undefined && !['text', 'reasoning'].includes(e.channel)
            || ['tool_call_id', 'tool_name', 'id', 'worker_status'].some(k => e[k] !== undefined && typeof e[k] !== 'string')
            || e.failed !== undefined && typeof e.failed !== 'boolean'
            || e.index !== undefined && (!Number.isSafeInteger(e.index) || e.index < 0))
            throw new Error('Invalid Engine pet metadata fields.');
        this.lastTime = at;
        this.events = this.events.filter(span => span.endTime >= at - 12_800);
        const id = (field) => { const s = e[field]; if (typeof s !== 'string' || !s)
            throw new Error(`Missing Engine ${field}.`); return s; };
        const index = () => { if (!Number.isSafeInteger(e.index))
            throw new Error('Missing Engine index.'); return String(e.index); };
        const start = (key, name, category, agentId) => {
            if (this.active.size >= 256 && !this.active.has(key))
                throw new Error('Too many active Engine pet spans.');
            this.active.set(key, this.add(name, category, at, agentId));
        };
        const finish = (key) => { this.pulse(key, at); this.active.delete(key); };
        switch (e.event) {
            case 'turn_started':
                this.active.clear();
                this.waiting = undefined;
                break;
            case 'message_started':
                start(`message:${index()}`, 'assistant_message', 'communication');
                this.waiting = undefined;
                break;
            case 'thinking_started':
                start(`thinking:${index()}`, 'thinking', 'reasoning');
                this.waiting = undefined;
                break;
            case 'response_delta': {
                const reasoning = e.channel === 'reasoning';

View on GitHub (pinned to 73e0f67d83)