{"record":{"id":"023fa7a867eb62f4","repo":"paperclipai/paperclip","slug":"live-provider-execution-failed","errorCode":"live_provider_execution_failed","errorMessage":"live_provider_execution_failed","messagePattern":"live_provider_execution_failed","errorType":"error_code","errorClass":"RunnerWorkflowInfrastructureError","httpStatus":null,"severity":"error","filePath":"packages/paperclip-runner/src/eval/live-workflow-executor.ts","lineNumber":941,"sourceCode":"      unsubscribe();\n      if (budgetInterrupt !== null) await budgetInterrupt;\n      if (budgetInterruptError !== undefined) throw budgetInterruptError;\n    }\n  } catch (error) {\n    infrastructureError = error;\n  }\n\n  const snapshot = session?.snapshot();\n  if (session !== null) {\n    try {\n      await service.shutdown(session.id, \"Runner live workflow eval complete\");\n    } catch (error) {\n      infrastructureError ??= error;\n    }\n  }\n  if (infrastructureError !== undefined) {\n    await rm(runtimeRoot, { recursive: true, force: true });\n    throw new RunnerWorkflowInfrastructureError(\n      \"live_provider_execution_failed\",\n      true,\n      safeFailureMessage(infrastructureError),\n    );\n  }\n  const calls = snapshot === undefined ? [] : observedCalls(snapshot);\n  const expectedCalls = input.evalCase.assertions.requiredOperationIds ?? [];\n  const taskState =\n    snapshot === undefined\n      ? null\n      : ((\n          JSON.parse(snapshot.mockState) as {\n            tasks?: Array<{ id: string; status: string }>;\n          }\n        ).tasks?.find((task) => task.id === \"task-1\") ?? null);\n  const terminalStatuses = turns.map((turn) => turn.status);\n  const cancellationExpected = input.evalCase.id === \"cancellation-permissions\";\n  const terminalOkay = cancellationExpected","sourceCodeStart":923,"sourceCodeEnd":959,"githubUrl":"https://github.com/paperclipai/paperclip/blob/01ad8584922b5d85292b1723cae71fa0d9b07a19/packages/paperclip-runner/src/eval/live-workflow-executor.ts#L923-L959","documentation":"executeLiveRunnerWorkflow wraps a live (real provider) eval session; any error raised while creating, messaging, restoring, or shutting down the live session is captured, the runtime root is deleted, and rethrown as RunnerWorkflowInfrastructureError with code live_provider_execution_failed and retryable=true. This code marks the failure as infrastructure (provider/transport/process) rather than an assertion failure, so the failure message is the stringified underlying cause.","triggerScenarios":"Any throw inside the session lifecycle during the live run: CapabilityLiveSessionService failing to start the adapter process, sendMessage rejecting on provider API error/timeout (turnTimeoutMs exceeded), session.restore failing during the restart-recovery case, budget-interrupt errors thrown in the finally block, or service.shutdown rejecting after the run.","commonSituations":"Expired or missing provider API credentials; provider rate limits or 5xx; model name not available to the account; network egress blocked in CI; turn latency exceeding maxLatencyMs budget; adapter binary failing to launch (e.g. the trusted opencode binding errors); shutdown racing an already-dead session.","solutions":["Read the wrapped cause via safeFailureMessage in the error's message/detail to identify the underlying provider or transport failure.","Check provider credentials and environment variables required by the candidate's qualification.requiredEnvironment/profile before running the campaign.","Confirm network egress to the provider endpoint and that the requested model is enabled for the account.","Re-run: the error is flagged retryable=true, so transient provider outages and rate limits can be retried within maxAttempts.","If it reproduces, raise budget.maxLatencyMs/maxCostUsd for the candidate or run a single eval case to isolate whether startup, messaging, or shutdown fails."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Before the live run, check provider reachability and env:\nfor (const envName of candidate.qualification.requiredEnvironment) {\n  if (!process.env[envName]) throw new Error(`missing ${envName} for live provider ${candidate.provider}`);\n}\n// Optionally probe the provider endpoint health before starting the campaign.","typeGuard":"function isInfrastructureError(e: unknown): e is RunnerWorkflowInfrastructureError {\n  return e instanceof RunnerWorkflowInfrastructureError && e.code === \"live_provider_execution_failed\";\n}","tryCatchPattern":"try {\n  await executeLiveRunnerWorkflow(input);\n} catch (error) {\n  if (error instanceof RunnerWorkflowInfrastructureError && error.code === \"live_provider_execution_failed\" && error.retryable) {\n    // back off and retry within candidate.budget.maxAttempts;\n    // inspect error message for the wrapped provider/transport cause\n  } else {\n    throw error;\n  }\n}","preventionTips":["Validate all qualification.requiredEnvironment credentials before launching a live campaign.","Set realistic budget.maxLatencyMs and maxCostUsd per candidate so provider latency does not read as infrastructure failure.","Prefer warm lifecyclePolicy with an adequate idleTimeoutMs for multi-turn cases like restart-recovery.","Check provider status pages / rate limits before scheduled rotation runs; the error is retryable by design.","Run a single eval case first to isolate whether startup, messaging, or shutdown is the failing phase."],"tags":["eval","live-provider","infrastructure","retryable"],"backgroundTag":"live-provider-execution-failed","analyzedSha":"01ad8584922b5d85292b1723cae71fa0d9b07a19","analyzedAt":"2026-09-02T18:44:00.616Z","contentChangedAt":"2026-09-02T18:44:00.616Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}