BerriAI/litellm · error · ValueError

streaming_sampling_rate must be >= 1 (got {streaming_samplin

Q: What causes "streaming_sampling_rate must be >= 1 (got {streaming_sampling_rate})" in BerriAI/litellm?

Set streaming_sampling_rate to 1 (scan every chunk) or higher, or remove it to accept the default of 5.

Error message

streaming_sampling_rate must be >= 1 (got {streaming_sampling_rate})

What it means

ValueError raised in GenericGuardrailAPI.__init__ when streaming_sampling_rate is provided with a value below 1. The parameter controls how often the streaming path samples chunks (read by UnifiedLLMGuardrails.async_post_call_streaming_iterator_hook via getattr); values < 1 are nonsensical (they would sample less than every chunk), so init rejects them. Default is 5 when unset.

Source

Thrown at litellm/proxy/guardrails/guardrail_hooks/generic_guardrail_api/generic_guardrail_api.py:222

        if not base_url.endswith("/beta/litellm_basic_guardrail_api"):
            base_url = base_url.rstrip("/")
            self.api_base = f"{base_url}/beta/litellm_basic_guardrail_api"
        else:
            self.api_base = base_url

        self.additional_provider_specific_params = additional_provider_specific_params or {}

        self.unreachable_fallback: Literal["fail_closed", "fail_open"] = unreachable_fallback

        self.fail_on_error: bool = True if fail_on_error is None else fail_on_error

        # Read by UnifiedLLMGuardrails.async_post_call_streaming_iterator_hook
        # via getattr(guardrail_to_apply, "streaming_*", default).
        self.streaming_end_of_stream_only: bool = (
            False if streaming_end_of_stream_only is None else streaming_end_of_stream_only
        )
        if streaming_sampling_rate is not None and streaming_sampling_rate < 1:
            raise ValueError(f"streaming_sampling_rate must be >= 1 (got {streaming_sampling_rate})")
        self.streaming_sampling_rate: int = 5 if streaming_sampling_rate is None else streaming_sampling_rate

        # Read by UnifiedLLMGuardrails.async_post_call_streaming_iterator_hook.
        # "block_only" (default) drops text rewrites on the streaming path;
        # "incremental_diff" emits them as synthetic deltas.
        self.streaming_transform_mode: Literal["block_only", "incremental_diff"] = (
            "block_only" if streaming_transform_mode is None else streaming_transform_mode
        )

        # Set supported event hooks
        kwargs.setdefault("supported_event_hooks", list(self.get_supported_event_hooks()))

        super().__init__(**kwargs)

        verbose_proxy_logger.debug("Generic Guardrail API initialized with api_base: %s", self.api_base)

    def _extract_user_api_key_metadata(self, request_data: dict) -> GenericGuardrailAPIMetadata:
        """

View on GitHub (pinned to 77b7c6c40c)

Solutions

Set streaming_sampling_rate to 1 (scan every chunk) or higher, or remove it to accept the default of 5.
Re-run proxy startup / config validation after the change.
Grep your config templates for streaming_sampling_rate placeholders that default to 0.

Example fix

# before
litellm_params:
  guardrail: generic_guardrail_api
  api_base: https://guardrail.internal
  streaming_sampling_rate: 0

# after
litellm_params:
  guardrail: generic_guardrail_api
  api_base: https://guardrail.internal
  streaming_sampling_rate: 1

Defensive patterns

Strategy: validation

Validate before calling

if (rate := cfg.litellm_params.get('streaming_sampling_rate')) is not None and rate < 1:
    raise SystemExit(f'invalid streaming_sampling_rate={rate}; must be >= 1')

Prevention

Treat streaming_sampling_rate as a chunk interval (>=1), not a probability.
Use config schema validation in CI to catch out-of-range numeric params.
Omit the parameter to accept the default of 5 when unsure.

When it happens

Trigger: Config such as litellm_params: { guardrail: generic_guardrail_api, streaming_sampling_rate: 0 } or 0.5; passing 0 intending 'scan everything' when 1 is the minimum.

Common situations: Copied YAML from another guardrail where 0 was allowed; misunderstanding the parameter as a probability instead of a chunk interval; config templating inserting 0 as a placeholder.

Understand the failure class

Background: Config validation failed: what "invalid value for {key}" and settings-rejection errors mean across 19 open-source libraries — this error's family across 19 libraries.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/c206c8fcb9935725. Report an issue: GitHub.