abhigyanpatwari/GitNexus · error · SandboxError

transcript_artifacts must be a list

Error message

transcript_artifacts must be a list

What it means

Raised by `_preflight_transcript_artifacts` in eval/workflow_bench/evolve.py when an evidence row's `transcript_artifacts` field exists but is not a Python list. The harness bounds every transcript reference before reading any evidence file, so the field's shape is non-negotiable.

Source

Thrown at eval/workflow_bench/evolve.py:367

        relative.is_absolute()
        or len(relative.parts) != 2
        or relative.parts[0] != "transcripts"
        or any(part in {"", ".", ".."} for part in relative.parts)
    ):
        raise SandboxError(f"unsafe results artifact path: {relative_value!r}")
    return relative.as_posix()


def _preflight_transcript_artifacts(evidence: list[dict[str, Any]]) -> list[list[Any]]:
    """Bound every transcript reference before any evidence file is read."""

    artifacts_by_row: list[list[Any]] = []
    seen_paths: set[str] = set()
    total = 0
    for artifacts_row in evidence:
        artifacts = artifacts_row.get("transcript_artifacts", [])
        if not isinstance(artifacts, list):
            raise SandboxError("transcript_artifacts must be a list")
        if len(artifacts) > MAX_TRANSCRIPT_ARTIFACTS_PER_ROW:
            raise SandboxError(
                f"transcript_artifacts exceeds the per-row session limit of {MAX_TRANSCRIPT_ARTIFACTS_PER_ROW}"
            )
        total += len(artifacts)
        if total > MAX_TRANSCRIPT_ARTIFACTS:
            raise SandboxError(f"transcript_artifacts exceeds the global evidence limit of {MAX_TRANSCRIPT_ARTIFACTS}")
        for artifact in artifacts:
            relative, _, _ = _transcript_artifact_metadata(artifact)
            normalized = _normalized_transcript_artifact_path(relative)
            if normalized in seen_paths:
                raise SandboxError(f"duplicate transcript artifact path: {normalized}")
            seen_paths.add(normalized)
        artifacts_by_row.append(artifacts)
    return artifacts_by_row


def _bound_transcript_artifact(root: Path, metadata: Any) -> str:

View on GitHub (pinned to d540b00184)

Solutions

1. Open the `--seed-results` directory's results.jsonl and find the row whose `transcript_artifacts` is not a list (grep for the field).
2. Fix the offending line so `transcript_artifacts` is a list of `{path,sha256,bytes,source}` objects, or delete the malformed row.
3. Regenerate the seed results with the current workflow_bench version if the whole file is from an older schema.

Example fix

// before: results.jsonl line has
{"task":"t1",...,"transcript_artifacts":{"sess-0":{...}}}
// after
{"task":"t1",...,"transcript_artifacts":[{...}]}

bad = [r for r in evidence if 'transcript_artifacts' in r and not isinstance(r['transcript_artifacts'], list)]
if bad:
    raise ValueError(f'non-list transcript_artifacts in rows: {[r.get('task') for r in bad]}')

def has_list_transcript_artifacts(row: dict) -> bool:
    ta = row.get('transcript_artifacts', [])
    return isinstance(ta, list)

When it happens

Trigger: Calling `proposer_evidence_entries(results_dir=..., evidence=..., ...)` (the generation-0 proposer path, active when `--seed-results` is given) where at least one row in `evidence` has `transcript_artifacts` set to a dict, string, int, or None instead of a list. `evidence` comes from `select_evidence(load_jsonl(results_dir/'results.jsonl'))`.

Common situations: A results.jsonl from an incompatible workflow_bench version, a hand-edited JSONL line, or a row written by a buggy custom runner that serialized transcript_artifacts as a dict keyed by session id.

Related errors

• unsafe results artifact path: {relative_value!r}
• transcript artifact metadata must contain only path, sha256,
• transcript_artifacts exceeds the per-row session limit of {M
• transcript_artifacts exceeds the global evidence limit of {M
• duplicate transcript artifact path: {normalized}

AI-assisted analysis of abhigyanpatwari/GitNexus@d540b00184 (2026-08-12). Data as JSON: /api/errors/a232be211a04bc07. Report an issue: GitHub.