hankcs/HanLP · warning

Failed to parse results from smatch: {line}

Error message

Failed to parse results from smatch: {line}

What it means

format_official_scores parses the subprocess output of the smatch AMR evaluation tool; if a line's '-> P: x, R: y, F: z' triple cannot be split/parsed as floats, it warns and records NaN scores for that metric.

Source

Thrown at hanlp/metrics/amr/smatch_eval.py:82

    # Non_sense_frames -> P: 0.008, R: 0.008, F: 0.008
    # Wikification -> P: 0.000, R: 0.000, F: 0.000
    # Named Ent. -> P: 0.222, R: 0.092, F: 0.130
    # Negations -> P: 0.000, R: 0.000, F: 0.000
    # IgnoreVars -> P: 0.005, R: 0.003, F: 0.003
    # Concepts -> P: 0.075, R: 0.036, F: 0.049
    # Frames -> P: 0.007, R: 0.007, F: 0.007
    # Reentrancies -> P: 0.113, R: 0.060, F: 0.079
    # SRL -> P: 0.145, R: 0.104, F: 0.121
    scores = SmatchScores()
    for line in text.split('\n'):
        line = line.strip()
        if not line:
            continue
        name, vs = line.split(' -> ')
        try:
            p, r, f = [float(x.split(': ')[-1]) for x in vs.split(', ')]
        except ValueError:
            warnings.warn(f'Failed to parse results from smatch: {line}')
            p, r, f = float("nan"), float("nan"), float("nan")
        scores[name] = F1_(p, r, f)
    return scores


def format_fast_scores(text: str):
    # using fast smatch
    # Precision: 0.137
    # Recall: 0.108
    # Document F-score: 0.121
    scores = []
    for line in text.split('\n'):
        line = line.strip()
        if not line or ':' not in line:
            continue
        name, score = line.split(': ')
        scores.append(float(score))
    assert len(scores) == 3

View on GitHub (pinned to ddb1299bdd)

Solutions

1. Check the warned line to see the actual format and confirm your smatch version is the one HanLP expects (install via HanLP's smatch dependency)
2. Capture/inspect the raw smatch output separately to spot interleaved warnings
3. Treat returned NaN scores as the signal that this metric run is invalid and re-run

import math
def scores_valid(scores):
    return all(not math.isnan(s.f) for s in scores.values())

def scores_valid(scores: dict) -> bool:
    import math
    return all(not math.isnan(getattr(s, 'f', s)) for s in scores.values())

scores = format_official_scores(text)
if any(math.isnan(v.f) for v in scores.values()):
    logging.warning('smatch output unparseable, re-running evaluation')
    scores = format_fast_scores(text)  # alternate parser

When it happens

Trigger: Running smatch_eval / format_official_scores where the smatch binary output format differs from expected — old/new smatch versions, localized decimal separators, or interleaved log lines breaking parsing.

Common situations: A different smatch version installed on PATH emitting changed output; stderr content mixed into stdout; corrupted smatch output from a failed run.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• If no `decoder_input_ids` or `decoder_inputs_embeds` are pas
• Unsupported argument length: {item}
• eval.pl exited with error code {exitcode} and error message
• Prediction file {pred_file.name} does not end a sentence at
• Expect X to be 2 or 3 elements but got {repr(X)}

AI-assisted analysis of hankcs/HanLP@ddb1299bdd (2026-08-27). Data as JSON: /api/errors/cbac4f04623e1bfe. Report an issue: GitHub.