BerriAI/litellm · error · ValueError

valkey-semantic does not support cluster-mode-enabled (multi

Error message

valkey-semantic does not support cluster-mode-enabled (multi-shard) endpoints. The async cluster client cannot route the FT.* search commands reliably. Point it at a cluster-mode-disabled endpoint instead (a primary with replicas is fine; only horizontal sharding is unsupported), or pass a single redis_url. On AWS, vector search needs ElastiCache for Valkey 8.2+ on a node-based cluster.

What it means

The Valkey semantic cache only supports non-clustered endpoints. Passing startup_nodes (the multi-shard cluster topology) raises this ValueError by design: the async cluster client cannot reliably route valkey-search FT.* commands across shards, so LiteLLM refuses rather than silently corrupting search results. The message prescribes the supported alternatives, including ElastiCache for Valkey 8.2+ in node-based (non-sharded) mode.

Source

Thrown at litellm/caching/valkey_semantic_cache.py:70

        self,
        host: str | None = None,
        port: str | None = None,
        password: str | None = None,
        redis_url: str | None = None,
        similarity_threshold: float | None = None,
        embedding_model: str = "text-embedding-ada-002",
        index_name: str | None = None,
        ssl: bool = False,
        startup_nodes: list | None = None,
        sync_client: Redis | None = None,
        async_client: AsyncRedis | None = None,
        **kwargs: Any,
    ):
        if similarity_threshold is None:
            raise ValueError("similarity_threshold must be provided, passed None")

        if startup_nodes:
            raise ValueError(
                "valkey-semantic does not support cluster-mode-enabled (multi-shard) "
                "endpoints. The async cluster client cannot route the FT.* search "
                "commands reliably. Point it at a cluster-mode-disabled endpoint "
                "instead (a primary with replicas is fine; only horizontal sharding "
                "is unsupported), or pass a single redis_url. On AWS, vector search "
                "needs ElastiCache for Valkey 8.2+ on a node-based cluster."
            )

        self.similarity_threshold = similarity_threshold
        self.embedding_model = embedding_model
        self.index_name = index_name or self.DEFAULT_VALKEY_INDEX_NAME
        self.key_prefix = f"{self.index_name}:"
        self._index_dim: int | None = None

        resolved_url = None
        if sync_client is None or async_client is None:
            resolved_url = redis_url or self._build_valkey_url(host, port, password, ssl)
        self.sync_client = sync_client if sync_client is not None else Redis.from_url(resolved_url)

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Point the cache at a cluster-mode-disabled endpoint: pass a single redis_url (a primary with replicas is fine)
2. On AWS, use ElastiCache for Valkey 8.2+ on a node-based cluster (cluster mode off) for vector search
3. Remove startup_nodes from the cache config entirely and supply redis_url or host/port instead

Example fix

# before
cache = ValkeySemanticCache(similarity_threshold=0.8,
    startup_nodes=[{'host': 'shard1', 'port': 6379}, {'host': 'shard2', 'port': 6379}])

# after
cache = ValkeySemanticCache(similarity_threshold=0.8,
    redis_url='rediss://:password@primary.node.cache.amazonaws.com:6379')

if cfg.get('startup_nodes'):
    raise ValueError(
        'valkey-semantic cache cannot use cluster-mode (multi-shard) endpoints; '
        'point it at a cluster-mode-disabled endpoint via redis_url instead'
    )

When it happens

Trigger: Passing startup_nodes=[{'host': ..., 'port': ...}, ...] to ValkeySemanticCache, i.e. pointing the semantic cache at a cluster-mode-enabled Valkey/Redis (multi-shard) deployment such as AWS ElastiCache cluster-mode-on or Redis Cluster.

Common situations: Teams reuse an existing sharded ElastiCache cluster for semantic caching; migrating from redis-semantic cache config that happened to include startup_nodes.

Related errors

• similarity_threshold must be provided, passed None
• Missing required Valkey configuration. Provide host and port
• collection_name must be provided, passed None
• similarity_threshold must be provided, passed None
• Qdrant url must be provided

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/edd7a2b8bbb16810. Report an issue: GitHub.