sgl-project/sglang · error · ValueError

prefetch_timeout_max must be number, got {type(prefetch_time

Error message

prefetch_timeout_max must be number, got {type(prefetch_timeout_max).__name__}

What it means

Validates that prefetch_timeout_max (upper bound in seconds for the linear prefetch timeout) in the parsed storage backend extra config is an int or float. Non-numeric parsed values (quoted strings, bools, nulls) raise this ValueError including the actual type name.

Source

Thrown at python/sglang/srt/mem_cache/hiradix_cache.py:777

        )  # seconds, upper bound for the linear timeout
        hicache_storage_pass_prefix_keys = extra_config.pop(
            "hicache_storage_pass_prefix_keys", False
        )

        if not isinstance(prefetch_threshold, int):
            raise ValueError(
                f"prefetch_threshold must be int, got {type(prefetch_threshold).__name__}"
            )
        if not isinstance(prefetch_timeout_base, (int, float)):
            raise ValueError(
                f"prefetch_timeout_base must be number, got {type(prefetch_timeout_base).__name__}"
            )
        if not isinstance(prefetch_timeout_per_ki_token, (int, float)):
            raise ValueError(
                f"prefetch_timeout_per_ki_token must be number, got {type(prefetch_timeout_per_ki_token).__name__}"
            )
        if not isinstance(prefetch_timeout_max, (int, float)):
            raise ValueError(
                f"prefetch_timeout_max must be number, got {type(prefetch_timeout_max).__name__}"
            )
        if not isinstance(hicache_storage_pass_prefix_keys, bool):
            raise ValueError(
                "hicache_storage_pass_prefix_keys must be bool, got "
                f"{type(hicache_storage_pass_prefix_keys).__name__}"
            )

        prefetch_timeout_config = PrefetchTimeoutConfig(
            base=float(prefetch_timeout_base),
            per_ki_token=float(prefetch_timeout_per_ki_token),
            max=float(prefetch_timeout_max),
        )

        return (
            extra_config,
            prefetch_threshold,
            prefetch_timeout_config,

View on GitHub (pinned to 0132848349)

Solutions

Set the value as a bare number: prefetch_timeout_max: 30
Audit any script that generates the config and coerce numerics before serialization
After fixing, ensure prefetch_timeout_max >= prefetch_timeout_base for a sane timeout policy

Example fix

# before
prefetch_timeout_max: "30"

# after
prefetch_timeout_max: 30

Defensive patterns

Strategy: type-guard

Validate before calling

v = cfg.get("prefetch_timeout_max", 30)
assert isinstance(v, (int, float)) and not isinstance(v, bool), "prefetch_timeout_max must be numeric"

Type guard

def is_number_not_bool(v) -> bool:
    return isinstance(v, (int, float)) and not isinstance(v, bool)

Prevention

Keep timeout fields bare numbers in config files
Ensure prefetch_timeout_max >= prefetch_timeout_base when tuning

When it happens

Trigger: Extra config with prefetch_timeout_max: "30", true, null, or a non-number (e.g. a list or map).

Common situations: Stringified numbers from templates/env substitution in generated configs; quoting values in YAML 'just to be safe'; missing key defaulting to a string in a custom config pipeline.

Understand the failure class

Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/fe83f6b97d5aba04. Report an issue: GitHub.