{"record":{"id":"5b5794960dc926ca","repo":"hankcs/HanLP","slug":"prediction-file-pred-file-name-does-not-end-a-se","errorCode":null,"errorMessage":"Prediction file {pred_file.name} does not end a sentence at line {idx + 1}\n{p.strip()}","messagePattern":"Prediction file (.+?) does not end a sentence at line (.+?)\n(.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"hanlp/metrics/parsing/conllx_eval.py","lineNumber":59,"sourceCode":"    \"\"\"Copy the first 6 columns from gold file to pred file\n\n    Args:\n      gold_file: \n      pred_file: \n      copied_pred_file: \n      keep_comments:  (Default value = True)\n\n    Returns:\n\n    \n    \"\"\"\n    with open(copied_pred_file, 'w') as to_out, open(pred_file) as pred_file, open(gold_file) as gold_file:\n        for idx, (p, g) in enumerate(zip(pred_file, gold_file)):\n            while p.startswith('#'):\n                p = next(pred_file)\n            if not g.strip():\n                if p.strip():\n                    raise ValueError(\n                        f'Prediction file {pred_file.name} does not end a sentence at line {idx + 1}\\n{p.strip()}')\n                to_out.write('\\n')\n                continue\n            while g.startswith('#') or '-' in g.split('\\t')[0]:\n                if keep_comments or g.startswith('-'):\n                    to_out.write(g)\n                g = next(gold_file)\n            to_out.write('\\t'.join(str(x) for x in g.split('\\t')[:6] + p.split('\\t')[6:]))\n","sourceCodeStart":41,"sourceCodeEnd":68,"githubUrl":"https://github.com/hankcs/HanLP/blob/ddb1299bddff079e447af52ec12549c50636bfa8/hanlp/metrics/parsing/conllx_eval.py#L41-L68","documentation":"While aligning prediction and gold CoNLL files column-by-column, copy_cols hit a blank line (sentence boundary) in the gold file but the corresponding prediction line still has content. That means the prediction file has fewer/different sentence breaks than gold, so files cannot be zipped line-by-line.","triggerScenarios":"Calling evaluate()/copy_cols with a pred_file whose sentences are not aligned with gold_file: predictions missing the blank line at sentence end, extra tokens after gold's sentence end, or prediction generated per-line with a different number of newlines.","commonSituations":"Writing predictions without the trailing blank line between sentences; predictions produced with print per token but no print('') at sentence end; CRLF vs LF mismatches; truncation of the last sentence.","solutions":["Make the prediction writer emit a blank line after every sentence, including the last one (mirroring the gold file).","Verify line counts and blank-line positions match gold: compare with a quick zip check or diff on sentence boundaries.","If your decoder dropped the final newline, append it before calling evaluate."],"exampleFix":"# before\nfor tok in sentence:\n    f.write(tok + '\\n')  # no blank line between sentences\n# after\nfor tok in sentence:\n    f.write(tok + '\\n')\nf.write('\\n')","handlingStrategy":"validation","validationCode":"def aligned(gold_path, pred_path):\n    g = open(gold_path).read().split('\\n\\n')\n    p = open(pred_path).read().split('\\n\\n')\n    return len(g) == len(p) and all(not seg or seg.strip() for seg in p)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always write a blank line after each sentence when emitting predictions.","Write a smoke test that evaluates a 1-sentence file to catch boundary bugs early."],"tags":["file-format","conll","alignment","parsing"],"backgroundTag":"file-format-mismatch","analyzedSha":"ddb1299bddff079e447af52ec12549c50636bfa8","analyzedAt":"2026-08-27T03:36:54.287Z","schemaVersion":2},"datasetVersion":"2026-08-27T08:17:20.692Z"}