sgl-project/sglang · error · ValueError

Invalid JSON in external ngram corpus at line {line_no}: {e.

Error message

Invalid JSON in external ngram corpus at line {line_no}: {e.msg}

What it means

While streaming the JSONL external corpus, a non-blank line failed json.loads. The original JSONDecodeError is chained (raise ... from e) and re-raised as a ValueError that includes the 1-based line number and the parser's message so the offending line can be located.

Source

Thrown at python/sglang/srt/speculative/cpp_ngram/external_corpus.py:34

    corpus_path = Path(path)
    if not corpus_path.is_file():
        raise ValueError(f"External ngram corpus path does not exist: {path}")
    if tokenizer is None:
        raise ValueError("A tokenizer is required to load an external ngram corpus.")
    if max_tokens <= 0:
        raise ValueError("External ngram corpus max tokens must be positive.")

    total_tokens = 0
    has_previous_doc = False
    with corpus_path.open("r", encoding="utf-8") as f:
        for line_no, line in enumerate(f, start=1):
            if not line.strip():
                continue

            try:
                record = json.loads(line)
            except json.JSONDecodeError as e:
                raise ValueError(
                    f"Invalid JSON in external ngram corpus at line {line_no}: {e.msg}"
                ) from e

            if not isinstance(record, str):
                raise ValueError(
                    "Invalid external ngram corpus record at line "
                    f"{line_no}: expected a JSON string."
                )

            token_ids = list(tokenizer.encode(record, add_special_tokens=False))
            if not token_ids:
                continue

            separator_cost = 1 if has_previous_doc else 0
            next_total_tokens = total_tokens + separator_cost + len(token_ids)
            if next_total_tokens > max_tokens:
                raise ValueError(
                    "External ngram corpus exceeds the configured token limit "

View on GitHub (pinned to 0132848349)

Solutions

Inspect the reported line number and fix or remove that line
Regenerate the corpus with json.dumps for every record, one per line
Validate the whole file up front: [json.loads(l) for l in open(p) if l.strip()]

Example fix

# before (bad corpus line)
Hello "world" 
# after
"Hello \"world\""

Defensive patterns

Strategy: validation

Validate before calling

import json
with open(path) as f:
    for i, line in enumerate(f, 1):
        if line.strip():
            json.loads(line)  # raises here with line number instead of mid-load

Try / catch

try:
    add_external_corpus(path, tok, n)
except ValueError as e:
    if "Invalid JSON" in str(e):
        # e carries the offending line number; quarantine and continue
        log.error("bad corpus: %s", e)
    else:
        raise

Prevention

Generate corpora only via json.dumps per line
Validate JSONL files in a pre-deploy lint step

When it happens

Trigger: A corpus .jsonl file containing malformed JSON on any non-empty line (trailing comma, unquoted string, truncated line from an interrupted write).

Common situations: Corpus generated by a script without proper json.dumps, file truncated mid-download, or mixed formats (plain text lines instead of JSON strings).

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d7d1ecc4b8e48f46. Report an issue: GitHub.