zed-industries/zed · error

failed to generate evaluation example for commit {} at line

Error message

failed to generate evaluation example for commit {} at line {}: {}

What it means

Bailed in the single-sample path of `ep split-commit` (no `--num-samples`) when generating an evaluation example for a commit fails and `--failed` is `keep` (the default). Identical semantics to the multi-sample bail: keep aborts the run on the first failing commit, while skip/skip-no-files log and continue, and NoMatchingSplitPointError is always skipped.

Source

Thrown at crates/edit_prediction_cli/src/split_commit.rs:593

                    Ok(case) => case,
                    Err(e) => {
                        let err_msg = format!(
                            "failed to generate evaluation example for commit {} at line {}: {}",
                            annotated.commit_sha,
                            line_num + 1,
                            e
                        );
                        if e.is::<NoMatchingSplitPointError>() {
                            eprintln!("skipping: {}", err_msg);
                            continue;
                        }
                        match failed {
                            FailedHandling::Skip | FailedHandling::SkipNoFiles => {
                                eprintln!("{}", err_msg);
                                continue;
                            }
                            FailedHandling::Keep => {
                                anyhow::bail!(err_msg);
                            }
                        }
                    }
                };

                let json = if args.pretty {
                    serde_json::to_string_pretty(&case)
                } else {
                    serde_json::to_string(&case)
                }
                .context("failed to serialize evaluation case as JSON")?;

                output_lines.push(json);
            }

            processed_commits += 1;
            eprint!(
                "\rsplit-commit: processed {} commits, generated {} examples",

View on GitHub (pinned to f4178619ac)

Solutions

1. Read the inner error after the colon to see why this commit failed to convert
2. Re-run with `--failed skip` to continue past bad commits and log them
3. Remove or repair the offending commit in the input JSONL, or use a different `--split-point` kind for that data

Example fix

# before
ep split-commit input.jsonl

# after
ep split-commit --failed skip input.jsonl

fn is_expected_split_failure(err: &anyhow::Error) -> bool {
    err.is::<NoMatchingSplitPointError>()
}

Same as the multi-sample path: downcast for NoMatchingSplitPointError to skip; otherwise follow --failed (skip logs and continues, keep aborts with the inner cause).

When it happens

Trigger: Running `ep split-commit` without `--num-samples` and with `--failed keep` (default) on an input containing a commit whose evaluation-case generation fails (diff parsing, cursor extraction, split-point validation).

Common situations: First pass over a new annotated-commit dump where a few commits have malformed diffs; the default keep mode surfacing the first bad sample immediately.

Related errors

• failed to generate evaluation example for commit {} at line
• Failed to read path: {path:?}
• {} should have an extension
• --languages and/or --extensions is required (use --list to s
• unknown teacher backend `{s}`. Valid options: sonnet45, sonn

AI-assisted analysis of zed-industries/zed@f4178619ac (2026-08-20). Data as JSON: /api/errors/ebb9484e1f676495. Report an issue: GitHub.