sgl-project/sglang · error · ValueError

prefetch_timeout_base must be number, got {type(prefetch_tim

Error message

prefetch_timeout_base must be number, got {type(prefetch_timeout_base).__name__}

What it means

Validates that prefetch_timeout_base in the parsed storage backend extra config is a number (int or float). Values that YAML/TOML parse as strings — quoted "10", or non-numeric text — fail isinstance((int, float)) and raise this ValueError with the actual type name.

Source

Thrown at python/sglang/srt/mem_cache/hiradix_cache.py:769

        prefetch_timeout_base = extra_config.pop(
            "prefetch_timeout_base", defaults.base
        )  # seconds
        prefetch_timeout_per_ki_token = extra_config.pop(
            "prefetch_timeout_per_ki_token", defaults.per_ki_token
        )  # seconds per 1024 tokens
        prefetch_timeout_max = extra_config.pop(
            "prefetch_timeout_max", defaults.max
        )  # seconds, upper bound for the linear timeout
        hicache_storage_pass_prefix_keys = extra_config.pop(
            "hicache_storage_pass_prefix_keys", False
        )

        if not isinstance(prefetch_threshold, int):
            raise ValueError(
                f"prefetch_threshold must be int, got {type(prefetch_threshold).__name__}"
            )
        if not isinstance(prefetch_timeout_base, (int, float)):
            raise ValueError(
                f"prefetch_timeout_base must be number, got {type(prefetch_timeout_base).__name__}"
            )
        if not isinstance(prefetch_timeout_per_ki_token, (int, float)):
            raise ValueError(
                f"prefetch_timeout_per_ki_token must be number, got {type(prefetch_timeout_per_ki_token).__name__}"
            )
        if not isinstance(prefetch_timeout_max, (int, float)):
            raise ValueError(
                f"prefetch_timeout_max must be number, got {type(prefetch_timeout_max).__name__}"
            )
        if not isinstance(hicache_storage_pass_prefix_keys, bool):
            raise ValueError(
                "hicache_storage_pass_prefix_keys must be bool, got "
                f"{type(hicache_storage_pass_prefix_keys).__name__}"
            )

        prefetch_timeout_config = PrefetchTimeoutConfig(
            base=float(prefetch_timeout_base),

View on GitHub (pinned to 0132848349)

Solutions

Unquote the value and make it numeric: prefetch_timeout_base: 10 or 10.5
Check for templating/env interpolation that stringifies the value and cast before writing the config
Confirm units — this is seconds, the linear timeout base — while fixing

Example fix

# before
prefetch_timeout_base: "10"

# after
prefetch_timeout_base: 10

Defensive patterns

Strategy: type-guard

Validate before calling

v = cfg.get("prefetch_timeout_base", 10)
assert isinstance(v, (int, float)) and not isinstance(v, bool), "prefetch_timeout_base must be numeric"

Type guard

def is_number_not_bool(v) -> bool:
    return isinstance(v, (int, float)) and not isinstance(v, bool)

Prevention

Never quote numeric config values in YAML
If generating config from templates, cast numbers explicitly with int()/float()

When it happens

Trigger: Extra config (YAML/TOML file or JSON string) containing prefetch_timeout_base: "10" (string), true (bool), null, or a non-numeric value.

Common situations: Quoting numbers in YAML; template engines rendering values as strings; env-var interpolation producing strings; omitting the key where a default is itself a string.

Understand the failure class

Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/db4b9f77c07149f1. Report an issue: GitHub.