sgl-project/sglang · error · RuntimeError

Failed to read JSON file {normalized_input}: {exc}

Error message

Failed to read JSON file {normalized_input}: {exc}

What it means

The .json file exists and was opened, but reading it raised IOError/OSError before parsing completed — permissions denied, file removed between the isfile check and open, or an I/O error on the filesystem. parse_ib_device_config wraps the OS-level exception with the path and the original error text.

Source

Thrown at python/sglang/srt/distributed/device_communicators/mooncake_transfer_engine.py:47

    if ib_device_str is None or not ib_device_str.strip():
        return None

    normalized_input = ib_device_str.strip()
    if not normalized_input.endswith(".json") and not normalized_input.startswith("{"):
        return normalized_input

    if normalized_input.endswith(".json"):
        if not os.path.isfile(normalized_input):
            raise RuntimeError(f"File {normalized_input} does not exist.")
        try:
            with open(normalized_input, "r", encoding="utf-8") as file:
                mapping = json.load(file)
        except json.JSONDecodeError as exc:
            raise RuntimeError(
                f"Failed to parse JSON content from file {normalized_input}"
            ) from exc
        except (IOError, OSError) as exc:
            raise RuntimeError(
                f"Failed to read JSON file {normalized_input}: {exc}"
            ) from exc
    else:
        try:
            mapping = json.loads(normalized_input)
        except json.JSONDecodeError as exc:
            raise ValueError(f"Invalid JSON mapping: {normalized_input}") from exc

    if not isinstance(mapping, dict):
        raise ValueError(
            "Invalid format: expected a mapping from GPU id to IB device string"
        )

    normalized_mapping: Dict[int, str] = {}
    for gpu_key, ib_devices in mapping.items():
        normalized_key = int(gpu_key) if str(gpu_key).isdigit() else None
        if normalized_key is None or not isinstance(ib_devices, str):
            raise ValueError(

View on GitHub (pinned to 0132848349)

Solutions

  1. chmod a+r /path/to/gpu_ib_map.json (or chown to the serving user) on every node
  2. Confirm the mount is healthy and the file readable from the serving user: sudo -u <user> cat <file>
  3. Copy the mapping to local disk if a flaky network filesystem is the cause

Example fix

# before
$ ls -l /etc/sglang/gpu_ib_map.json
-rw------- 1 root root ...  # serving user can't read -> OSError

# after
$ sudo chmod 644 /etc/sglang/gpu_ib_map.json
Defensive patterns

Strategy: validation

Validate before calling

import os

def check_readable(path: str) -> None:
    if not os.access(path, os.R_OK):
        raise PermissionError(f"no read access to {path}")

Try / catch

try:
    mapping = parse_ib_device_config(cfg)
except RuntimeError as e:
    if "Failed to read JSON file" in str(e):
        fix_perms_and_retry(cfg)  # or re-provision file
    raise

Prevention

When it happens

Trigger: The sglang process runs as a user without read permission on the mapping file, the file is deleted (race) after the existence check, or NFS/storage-level read errors occur on cluster nodes.

Common situations: Config file owned by root with 600 perms while the server runs as a non-root user, or a shared filesystem mount hiccup on one worker node.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/11528a586518b8b0. Report an issue: GitHub.