hankcs/HanLP · error · ValueError

Prediction file {pred_file.name} does not end a sentence at

Error message

Prediction file {pred_file.name} does not end a sentence at line {idx + 1}
{p.strip()}

What it means

While aligning prediction and gold CoNLL files column-by-column, copy_cols hit a blank line (sentence boundary) in the gold file but the corresponding prediction line still has content. That means the prediction file has fewer/different sentence breaks than gold, so files cannot be zipped line-by-line.

Source

Thrown at hanlp/metrics/parsing/conllx_eval.py:59

    """Copy the first 6 columns from gold file to pred file

    Args:
      gold_file: 
      pred_file: 
      copied_pred_file: 
      keep_comments:  (Default value = True)

    Returns:

    
    """
    with open(copied_pred_file, 'w') as to_out, open(pred_file) as pred_file, open(gold_file) as gold_file:
        for idx, (p, g) in enumerate(zip(pred_file, gold_file)):
            while p.startswith('#'):
                p = next(pred_file)
            if not g.strip():
                if p.strip():
                    raise ValueError(
                        f'Prediction file {pred_file.name} does not end a sentence at line {idx + 1}\n{p.strip()}')
                to_out.write('\n')
                continue
            while g.startswith('#') or '-' in g.split('\t')[0]:
                if keep_comments or g.startswith('-'):
                    to_out.write(g)
                g = next(gold_file)
            to_out.write('\t'.join(str(x) for x in g.split('\t')[:6] + p.split('\t')[6:]))

View on GitHub (pinned to ddb1299bdd)

Solutions

Make the prediction writer emit a blank line after every sentence, including the last one (mirroring the gold file).
Verify line counts and blank-line positions match gold: compare with a quick zip check or diff on sentence boundaries.
If your decoder dropped the final newline, append it before calling evaluate.

Example fix

# before
for tok in sentence:
    f.write(tok + '\n')  # no blank line between sentences
# after
for tok in sentence:
    f.write(tok + '\n')
f.write('\n')

Defensive patterns

Strategy: validation

Validate before calling

def aligned(gold_path, pred_path):
    g = open(gold_path).read().split('\n\n')
    p = open(pred_path).read().split('\n\n')
    return len(g) == len(p) and all(not seg or seg.strip() for seg in p)

Prevention

Always write a blank line after each sentence when emitting predictions.
Write a smoke test that evaluates a 1-sentence file to catch boundary bugs early.

When it happens

Trigger: Calling evaluate()/copy_cols with a pred_file whose sentences are not aligned with gold_file: predictions missing the blank line at sentence end, extra tokens after gold's sentence end, or prediction generated per-line with a different number of newlines.

Common situations: Writing predictions without the trailing blank line between sentences; predictions produced with print per token but no print('') at sentence end; CRLF vs LF mismatches; truncation of the last sentence.

Related errors

AI-assisted analysis of hankcs/HanLP@ddb1299bdd (2026-08-27). Data as JSON: /api/errors/5b5794960dc926ca. Report an issue: GitHub.