mlflow/mlflow · warning · ValueError

Malformed metric line: {metric_line!r}

Error message

Malformed metric line: {metric_line!r}

What it means

Filesystem tracking stores persist metrics as text lines of the form 'timestamp value [step]'. _parse_metric_line uses structural pattern matching to parse each line; a line that doesn't split into 2+ space-separated tokens (or has unparseable tokens) raises ValueError, surfacing a corrupted or non-MLflow metric file during fs2db migration.

Source

Thrown at mlflow/store/fs2db/_tracking.py:229

def _sanitize_metric_value(val: float) -> tuple[bool, float]:
    is_nan = math.isnan(val)
    if is_nan:
        return True, 0.0
    if math.isinf(val):
        return False, 1.7976931348623157e308 if val > 0 else -1.7976931348623157e308
    return False, val


def _parse_metric_line(metric_line: str) -> tuple[int, float, int]:
    match metric_line.strip().split(" "):
        case [ts, val]:
            return int(ts), float(val), 0
        case [ts, val, step, *_]:
            return int(ts), float(val), int(step)
        case _:
            raise ValueError(f"Malformed metric line: {metric_line!r}")


def _migrate_run_metrics(
    session: Session,
    metrics_dir: Path,
    run_uuid: str,
    stats: MigrationStats,
    *,
    batch_size: int = 5000,
) -> None:
    all_metrics = read_metric_lines(metrics_dir)
    count = 0

    for key, lines in all_metrics.items():
        # Track the "latest" metric for this key: max by (step, timestamp, value)
        latest: tuple[int, int, float] | None = None  # (step, timestamp, value)
        latest_is_nan = False

View on GitHub (pinned to 6a27f2decc)

Solutions

1. Open the offending metrics file (the path is in the migration log) and fix or remove the malformed line.
2. Remove/repair corrupted metric files for the affected run, or exclude that run from migration.
3. Regenerate the metrics by re-running the training/logging job if the source data is unavailable.

Example fix

# before (metrics/epoch_acc)
1627584.5 0.93


# after (blank lines removed)
1627584.5 0.93

def validate_metric_file(path):
    for i, line in enumerate(open(path), 1):
        parts = line.strip().split(" ")
        if len(parts) < 2:
            raise SystemExit(f"{path}:{i}: malformed metric line {line!r}")
        int(parts[0]); float(parts[1])
        if len(parts) > 2:
            int(parts[2])

def is_valid_metric_line(line: str) -> bool:
    parts = line.strip().split(" ")
    try:
        int(parts[0]); float(parts[1])
        if len(parts) > 2: int(parts[2])
        return True
    except (ValueError, IndexError):
        return False

try:
    migrate(engine, source)
except ValueError as e:
    if "Malformed metric line" in str(e):
        logger.error("Corrupt metric data in source store: %s", e)
    else:
        raise

When it happens

Trigger: Migrating a filesystem store whose metrics/<metric> files contain blank lines, extra malformed tokens handled by the catch-all, or corrupted/truncated content that fails int()/float() conversion.

Common situations: Manually edited or truncated metric files; files written by third-party tools into mlruns; interrupted writes leaving partial lines; copying mlruns with corruption.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Azure Blob upload failed: ${response.status} ${response.stat
• Aborted: the database does not have workspaces enabled. This
• Predictions response contents are not valid JSON
• INVALID_PARAMETER_VALUE
• Percentile value is required for PERCENTILE aggregation

AI-assisted analysis of mlflow/mlflow@6a27f2decc (2026-08-29). Data as JSON: /api/errors/3b52b0a2e03a0ed4. Report an issue: GitHub.