affaan-m/ECC · error

CLI evidence evaluator must be recorded-v1

Error message

CLI evidence evaluator must be recorded-v1

What it means

RecordedEvaluator::from_evidence only accepts evidence files whose evaluator field is exactly "recorded-v1". Any other value means the evidence was produced by a different (or future) evaluator format that this CLI cannot interpret. This is a forward-compatibility guard: it fails fast instead of misreading an unknown schema.

Solutions

1. Regenerate the evidence through the current CLI's record flow so evaluator is set to "recorded-v1".
2. If the evidence is from a newer version, upgrade the CLI/harness so it accepts that format instead of editing the field.
3. If hand-writing evidence for tests, set evaluator: "recorded-v1" explicitly.
4. Verify the evidence file was not truncated or partially written (a missing evaluator field also fails this check).

Example fix

// before (hand-authored test evidence)
let evidence = RecordedEvidence { scores, health, ..Default::default() }; // evaluator empty

// after
let evidence = RecordedEvidence {
    evaluator: "recorded-v1".to_string(),
    scores,
    health,
    ..Default::default()
};

fn is_recorded_v1(evidence: &RecordedEvidence) -> bool {
    evidence.evaluator == "recorded-v1"
}

fn as_recorded_v1(evidence: RecordedEvidence) -> Option<RecordedEvidence> {
    (evidence.evaluator == "recorded-v1").then_some(evidence)
}

match RecordedEvaluator::from_evidence(evidence) {
    Ok(ev) => proceed(ev),
    Err(e) if e.to_string().contains("recorded-v1") => bail!("evidence format unsupported; regenerate or upgrade the harness"),
    Err(e) => return Err(e),
}

When it happens

Trigger: Calling from_evidence with a RecordedEvidence deserialized from a file where evaluator is missing (defaults to empty), renamed, hand-edited, or written by a newer version using "recorded-v2" or another format string.

Common situations: Upgrading the harness and trying to feed new-format evidence into an older CLI; hand-authoring evidence JSON for tests and omitting the evaluator field; a tooling change rewriting the field name (e.g. evaluator_version); mixing evidence files from different tools.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• effect lacks reference evidence
• effect evidence source duration is not bound to its receipt…
• exactly one candidate-keyed health assertion is required
• health check result does not match persisted assertion
• health evidence does not match candidate and evaluator

AI-assisted analysis of affaan-m/ECC@8321021c54 (2026-09-16). Data as JSON: /api/errors/94cb046fcf0c2c6e. Report an issue: GitHub.

Appendix: source

Thrown at ecc2/src/harness_eval.rs:376

    health_candidate: Option<String>,
    calls: Vec<(String, u64)>,
}

impl RecordedEvaluator {
    #[cfg(test)]
    pub fn new(scores: BTreeMap<(String, u64), f64>, health_ok: bool) -> Self {
        Self {
            name: "recorded-v1".into(),
            scores,
            health_ok,
            health_candidate: None,
            calls: Vec::new(),
        }
    }

    pub fn from_evidence(evidence: RecordedEvidence) -> Result<Self> {
        if evidence.evaluator != "recorded-v1" {
            bail!("CLI evidence evaluator must be recorded-v1");
        }
        let score_count = evidence.scores.values().map(BTreeMap::len).sum::<usize>();
        if score_count > 20_000 || evidence.scores.keys().any(|id| id.len() != 64) {
            bail!("recorded evidence exceeds bounded score or candidate limits");
        }
        if evidence.health.len() != 1 {
            bail!("exactly one candidate-keyed health assertion is required");
        }
        let (health_candidate, health_ok) = evidence
            .health
            .into_iter()
            .next()
            .context("candidate-keyed health evidence is required")?;
        let scores = evidence
            .scores
            .into_iter()
            .flat_map(|(id, values)| {
                values

View on GitHub (pinned to 8321021c54)