sgl-project/sglang · error · RuntimeError

Failed to parse JSON content from file {normalized_input}

Error message

Failed to parse JSON content from file {normalized_input}

What it means

The mooncake IB-device config pointed at an existing .json file, but json.load failed with JSONDecodeError — the file contents are not valid JSON (truncated, BOM, YAML/Python dict syntax, trailing commas). Raised by parse_ib_device_config with the offending path in the message, chained from the original JSONDecodeError which pinpoints the line/column.

Source

Thrown at python/sglang/srt/distributed/device_communicators/mooncake_transfer_engine.py:43

def parse_ib_device_config(
    ib_device_str: Optional[str],
) -> Optional[Union[str, Dict[int, str]]]:
    """Parse IB device config from a shared string, JSON mapping, or JSON file."""
    if ib_device_str is None or not ib_device_str.strip():
        return None

    normalized_input = ib_device_str.strip()
    if not normalized_input.endswith(".json") and not normalized_input.startswith("{"):
        return normalized_input

    if normalized_input.endswith(".json"):
        if not os.path.isfile(normalized_input):
            raise RuntimeError(f"File {normalized_input} does not exist.")
        try:
            with open(normalized_input, "r", encoding="utf-8") as file:
                mapping = json.load(file)
        except json.JSONDecodeError as exc:
            raise RuntimeError(
                f"Failed to parse JSON content from file {normalized_input}"
            ) from exc
        except (IOError, OSError) as exc:
            raise RuntimeError(
                f"Failed to read JSON file {normalized_input}: {exc}"
            ) from exc
    else:
        try:
            mapping = json.loads(normalized_input)
        except json.JSONDecodeError as exc:
            raise ValueError(f"Invalid JSON mapping: {normalized_input}") from exc

    if not isinstance(mapping, dict):
        raise ValueError(
            "Invalid format: expected a mapping from GPU id to IB device string"
        )

    normalized_mapping: Dict[int, str] = {}

View on GitHub (pinned to 0132848349)

Solutions

Validate the file with python -m json.tool /path/to/file.json; the chained JSONDecodeError shows the exact line/column
Rewrite the file as strict JSON, e.g. {"0": "mlx5_0,mlx5_1", "1": "mlx5_2"}
If the file is generated by a script, re-run the generator and verify it emits valid, fully-written JSON

Example fix

# before (gpu_ib_map.json)
{'0': 'mlx5_0', '1': 'mlx5_2'}  # Python-style quotes -> JSONDecodeError

# after
{"0": "mlx5_0", "1": "mlx5_2"}

Defensive patterns

Strategy: validation

Validate before calling

import json

def validate_ib_mapping_file(path: str) -> dict:
    with open(path) as f:
        return json.load(f)  # raises JSONDecodeError with line/column before server start

Try / catch

try:
    mapping = parse_ib_device_config(cfg)
except (RuntimeError, ValueError) as e:
    fail_deploy(f"Invalid mooncake IB config: {e}")  # abort before engine init

Prevention

Run python -m json.tool on the mapping file in CI/deploy scripts
Generate the file with a script (json.dump) instead of hand-editing
Never use Python-dict syntax (single quotes/comments) in the JSON file

When it happens

Trigger: Passing a .json file whose content json.load cannot parse — malformed JSON, empty file, hand-edited mapping with single quotes or comments, or a file corrupted/truncated during provisioning.

Common situations: Writing the mapping by hand in YAML/Python style ({'0': 'mlx5_0'}), a partially-written file from a config generator, or an empty template shipped without being filled in.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1f1c7f1b2c676df0. Report an issue: GitHub.