sgl-project/sglang · error · ValueError

prefetch_threshold must be int, got {type(prefetch_threshold

Error message

prefetch_threshold must be int, got {type(prefetch_threshold).__name__}

What it means

After parsing the storage backend extra config (JSON string or toml/yaml file), HiRadixCache validates that prefetch_threshold is an int; YAML/TOML files can produce floats (e.g. 2.0) or strings ("2"), and this check rejects anything that is not exactly isinstance(int). Note bool is a subclass of int in Python, but floats/strings fail.

Source

Thrown at python/sglang/srt/mem_cache/hiradix_cache.py:765

                raise e

        defaults = PrefetchTimeoutConfig()
        prefetch_threshold = extra_config.pop("prefetch_threshold", 256)  # tokens
        prefetch_timeout_base = extra_config.pop(
            "prefetch_timeout_base", defaults.base
        )  # seconds
        prefetch_timeout_per_ki_token = extra_config.pop(
            "prefetch_timeout_per_ki_token", defaults.per_ki_token
        )  # seconds per 1024 tokens
        prefetch_timeout_max = extra_config.pop(
            "prefetch_timeout_max", defaults.max
        )  # seconds, upper bound for the linear timeout
        hicache_storage_pass_prefix_keys = extra_config.pop(
            "hicache_storage_pass_prefix_keys", False
        )

        if not isinstance(prefetch_threshold, int):
            raise ValueError(
                f"prefetch_threshold must be int, got {type(prefetch_threshold).__name__}"
            )
        if not isinstance(prefetch_timeout_base, (int, float)):
            raise ValueError(
                f"prefetch_timeout_base must be number, got {type(prefetch_timeout_base).__name__}"
            )
        if not isinstance(prefetch_timeout_per_ki_token, (int, float)):
            raise ValueError(
                f"prefetch_timeout_per_ki_token must be number, got {type(prefetch_timeout_per_ki_token).__name__}"
            )
        if not isinstance(prefetch_timeout_max, (int, float)):
            raise ValueError(
                f"prefetch_timeout_max must be number, got {type(prefetch_timeout_max).__name__}"
            )
        if not isinstance(hicache_storage_pass_prefix_keys, bool):
            raise ValueError(
                "hicache_storage_pass_prefix_keys must be bool, got "
                f"{type(hicache_storage_pass_prefix_keys).__name__}"

View on GitHub (pinned to 0132848349)

Solutions

  1. Change the value to a plain integer literal: prefetch_threshold: 2 (no decimal point, no quotes)
  2. If editing YAML, ensure the key isn't quoted and has no trailing .0
  3. Re-run and confirm the rest of the timeout fields also satisfy their (int, float) checks

Example fix

# before (hicache.yaml)
prefetch_threshold: 2.0

# after
prefetch_threshold: 2
Defensive patterns

Strategy: type-guard

Validate before calling

assert isinstance(cfg.get("prefetch_threshold", 2), int) and not isinstance(cfg.get("prefetch_threshold", 2), bool), "prefetch_threshold must be a plain int"

Type guard

def is_int_not_bool(v) -> bool:
    return isinstance(v, int) and not isinstance(v, bool)

Prevention

When it happens

Trigger: A .yaml/.toml extra-config file containing prefetch_threshold: 2.0 (YAML parses to float), prefetch_threshold: "2" (quoted string), or a JSON string with "prefetch_threshold": 1.5.

Common situations: Writing the extra config by hand in YAML where 2.0 is natural; copy-pasting values from docs that use floats; tools serializing numbers as floats; passing a string value from an env var.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8390bfa174313ae0. Report an issue: GitHub.