JuliusBrussee/caveman · error · Error

cave_harness_transform_not_evaluated

cave_harness_transform_not_evaluated

Error message

cave_harness_transform_not_evaluated

What it means

Every transform_id selected in plan.segment_routes must appear in the request's evaluatedTransformIDs. One selected transform the request does not claim to have evaluated means execution would skip part of the locked plan, so the adapter rejects the request.

Solutions

1. Derive evaluatedTransformIDs from the same plan you pass in: [...new Set(plan.segment_routes.map(r => r.transform_id))]
2. Regenerate both transform-ID lists whenever the plan is rebuilt
3. Diff selected vs evaluated sets before run() (see defense below)

Example fix

// before
await adapter.run({ ...req, plan, evaluatedTransformIDs: ["t1"] });
// plan.segment_routes selects t1 and t2

// after
const ids = [...new Set(plan.segment_routes.map((r) => r.transform_id))];
await adapter.run({ ...req, plan, evaluatedTransformIDs: ids, appliedTransformIDs: ids });

const selected = [...new Set(plan.segment_routes.map((r) => r.transform_id))];
const evaluated = [...new Set(request.evaluatedTransformIDs)];
const missing = selected.filter((id) => !evaluated.includes(id));
if (missing.length > 0) {
  throw new Error(`transform IDs not evaluated: ${missing.join(", ")}`);
}

try {
  const result = await adapter.run(request);
} catch (error) {
  if (error instanceof Error && error.message === "cave_harness_transform_not_evaluated") {
    // A plan-selected transform is absent from evaluatedTransformIDs —
    // recompute both ID lists from the current plan and retry.
  }
  throw error;
}

When it happens

Trigger: adapter.run() where evaluatedTransformIDs omits an ID present in plan.segment_routes[].transform_id — e.g. hand-assembling the list, or reusing an old request alongside a regenerated plan that selected a new transform.

Common situations: Plan rebuilt after a source change adds a transform route while the caller's evaluated/applied lists come from a cached prior request; partially copying transform IDs; filtering IDs by mistake.

Related errors

• cave_harness_unknown_transform
• cave_harness_model_invalid
• cave_claude_terminal_missing
• cave_execution_plan_selection_mismatch
• cave_fixture_terminal_evidence_missing

AI-assisted analysis of JuliusBrussee/caveman@766dce6b13 (2026-08-18). Data as JSON: /api/errors/d897cc39365912d6. Report an issue: GitHub.

Appendix: source

Thrown at packages/agent/src/adapters.ts:676

  });
}

function validateTransformEvidence(
  evidence: Pick<HarnessRequest, "evaluatedTransformIDs" | "appliedTransformIDs" | "recoveryResolved">,
  plan: CavePlan,
): void {
  if (!Array.isArray(evidence.evaluatedTransformIDs) ||
      !Array.isArray(evidence.appliedTransformIDs) ||
      evidence.evaluatedTransformIDs.some((value) => typeof value !== "string" || value.length === 0) ||
      evidence.appliedTransformIDs.some((value) => typeof value !== "string" || value.length === 0) ||
      evidence.recoveryResolved !== true) {
    throw new Error("cave_harness_incomplete_evidence");
  }
  const selected = [...new Set(plan.segment_routes.map((route) => route.transform_id))].sort();
  const evaluated = [...new Set(evidence.evaluatedTransformIDs)].sort();
  const applied = [...new Set(evidence.appliedTransformIDs)].sort();
  if (selected.some((transformID) => !evaluated.includes(transformID))) {
    throw new Error("cave_harness_transform_not_evaluated");
  }
  if (evaluated.some((transformID) => !selected.includes(transformID)) ||
      applied.some((transformID) => !selected.includes(transformID))) {
    throw new Error("cave_harness_unknown_transform");
  }
}

function validateExecution(execution: HarnessExecution, plan: CavePlan): void {
  const [provider, ...modelParts] = plan.model.split("/");
  const model = modelParts.join("/");
  try {
    validateProviderUsage(execution, {
      expected: { provider: provider ?? "", model },
      reportedCostUsd: execution.costUsd,
      requirePriced: true,
    });
  } catch {
    throw new Error("cave_harness_incomplete_evidence");

View on GitHub (pinned to 766dce6b13)