{"record":{"id":"10433dafb342d8f6","repo":"mastra-ai/mastra","slug":"error-message","errorCode":null,"errorMessage":"${error.message}","messagePattern":"\\$\\{error\\.message\\}","errorType":"exception","errorClass":"ProtocolOutputError","httpStatus":null,"severity":"error","filePath":"packages/cli/src/commands/experiment/runtime.ts","lineNumber":421,"sourceCode":"    };\n    const frame = `${JSON.stringify(event)}\\n`;\n    const frameBytes = Buffer.byteLength(frame);\n    if (frameBytes > EXPERIMENT_WORKER_MAX_FRAME_BYTES) {\n      return Promise.reject(new Error('output frame exceeds maximum size'));\n    }\n    if (pendingOutputBytes + frameBytes > EXPERIMENT_WORKER_MAX_PENDING_OUTPUT_BYTES) {\n      return Promise.reject(new Error('pending protocol output exceeds maximum size'));\n    }\n    sequence += 1;\n    pendingOutputBytes += frameBytes;\n    if (type === 'heartbeat') heartbeatQueued = true;\n    const queuedWrite = writeTail.then(async () => {\n      try {\n        if (!stdout.write(frame)) {\n          try {\n            await waitForDrain();\n          } catch (error) {\n            throw new ProtocolOutputError(error instanceof Error ? error.message : String(error), true);\n          }\n        }\n      } finally {\n        pendingOutputBytes -= frameBytes;\n        if (type === 'heartbeat') heartbeatQueued = false;\n      }\n    });\n    writeTail = queuedWrite.catch(error => {\n      if (type === 'heartbeat') abortForProtocolFailure(error instanceof Error ? error.message : String(error));\n    });\n    return queuedWrite;\n  };\n  const finish = async (\n    status: 'completed' | 'completed-with-errors' | 'failed' | 'cancelled' | 'timed-out',\n    semanticEvent: ExperimentEvent,\n    exitCode: number,\n    retryable = false,\n  ) => {","sourceCodeStart":403,"sourceCodeEnd":439,"githubUrl":"https://github.com/mastra-ai/mastra/blob/75dd419e613fe9c39f846ffc500716141b74fda6/packages/cli/src/commands/experiment/runtime.ts#L403-L439","documentation":"writeEvent queues protocol frames onto a serialized write tail; if stdout.write signals backpressure, it awaits waitForDrain(). Any failure there (including the deadline backpressure error) is rethrown as a ProtocolOutputError carrying the underlying message (this error entry's message is that inner message interpolated). ProtocolOutputError marks the output stream as broken, causing the worker run to fail rather than emit corrupt or truncated protocol output.","triggerScenarios":"A queued frame write fails during waitForDrain — deadline exceeded while stdout is blocked (\"stdout backpressure exceeded the experiment deadline\"), or the stream errors while waiting — and the message is wrapped in ProtocolOutputError.","commonSituations":"Long-running experiments whose consumer stalls; stdout piped into a file on a full disk; slow CI log ingestion combined with a strict experiment deadline; large emitted payloads saturating the pipe buffer.","solutions":["Make sure the process consuming the worker's stdout drains it continuously (see 948).","Increase the experiment deadline or reduce event output size/frequency.","Inspect the inner message in the ProtocolOutputError to identify whether it was backpressure-deadline or an I/O stream error.","If caused by stream errors (e.g. EPIPE because the parent exited), keep the parent alive until the terminal frame is consumed."],"exampleFix":"// before\nconst res = await runExperimentWorker({ mastra, runExperiment }); // parent exits early -> EPIPE\n// after\nconst res = await runExperimentWorker({ mastra, runExperiment });\nawait childPromise; // keep consuming stdout until terminal frame before exiting","handlingStrategy":"try-catch","validationCode":"// pre-flight: verify the stdout destination is writable and the consumer is attached\nif (process.stdout.destroyed || (process.stdout.writableNeedDrain && process.stdout.writableLength > 1 << 20)) {\n  throw new Error('stdout consumer is not keeping up before experiment start');\n}","typeGuard":"function isProtocolOutputError(e: unknown): e is { name: 'ProtocolOutputError'; message: string; retryable: boolean } {\n  return typeof e === 'object' && e !== null && (e as any).name === 'ProtocolOutputError';\n}","tryCatchPattern":"try {\n  await runExperimentWorker({ mastra, runExperiment, build });\n} catch (e) {\n  if (isProtocolOutputError(e)) {\n    console.error(`Worker output stream failed (${e.message}); inspect the stdout consumer and deadline`);\n  } else throw e;\n}","preventionTips":["Keep the parent process reading stdout until the terminal frame arrives","Increase the deadline or shrink output for output-heavy experiments","Handle EPIPE in the parent (don't exit before the worker finishes)","Monitor writableLength/writableNeedDrain as an early backpressure signal"],"tags":["streaming","protocol","backpressure"],"backgroundTag":"protocol-output-error","analyzedSha":"75dd419e613fe9c39f846ffc500716141b74fda6","analyzedAt":"2026-08-30T00:15:31.844Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}