microsoft/semantic-kernel · error · ServiceInitializationError

Dimensionality of {dimension_num} exceeds the maximum allowe

Error message

Dimensionality of {dimension_num} exceeds the maximum allowed value of {MAX_DIMENSIONALITY}.

What it means

Raised in PineconeMemoryStore.create_collection() when the resolved dimensionality (explicit dimension_num or the store's _default_dimensionality) exceeds MAX_DIMENSIONALITY (20000). Unlike the constructor check this throws ServiceInitializationError. It fires before any does_collection_exist / create_index call.

Source

Thrown at python/semantic_kernel/connectors/memory_stores/pinecone/pinecone_memory_store.py:108

    ) -> None:
        """Creates a new collection in Pinecone if it does not exist.

        This function creates an index, by default the following index
        settings are used: metric = cosine, cloud = aws, region = us-east-1.

        Args:
            collection_name (str): The name of the collection to create.
                In Pinecone, a collection is represented as an index. The concept
                of "collection" in Pinecone is just a static copy of an index.
            dimension_num (int, optional): The dimensionality of the embeddings.
            distance_type (str, optional): The distance metric to use for the index.
                (default: {"cosine"})
            index_spec (NamedTuple, optional): The index spec to use for the index.
        """
        if dimension_num is None:
            dimension_num = self._default_dimensionality
        if dimension_num > MAX_DIMENSIONALITY:
            raise ServiceInitializationError(
                f"Dimensionality of {dimension_num} exceeds " + f"the maximum allowed value of {MAX_DIMENSIONALITY}."
            )

        if not await self.does_collection_exist(collection_name):
            self.pinecone.create_index(
                name=collection_name, dimension=dimension_num, metric=distance_type, spec=index_spec
            )
            self.collection_names_cache.add(collection_name)

    async def describe_collection(self, collection_name: str) -> IndexModel | None:
        """Gets the description of the index.

        Args:
            collection_name (str): The name of the index to get.

        Returns:
            Optional[dict]: The index.
        """

View on GitHub (pinned to c028a0c7dc)

Solutions

Pass a dimension_num <= 20000 from a supported embedding model.
Do not override dimension_num and rely on a valid _default_dimensionality set at construction.
Clamp/validate the dimension at the call site against MAX_DIMENSIONALITY.
Migrate to PineconeStore + Collection.

Example fix

// before
await store.create_collection("big", dimension_num=50000)

// after
await store.create_collection("big", dimension_num=1536)

Defensive patterns

Strategy: validation

Validate before calling

from semantic_kernel.connectors.memory_stores.pinecone.pinecone_memory_store import MAX_DIMENSIONALITY
dim = dimension_num if dimension_num is not None else store._default_dimensionality
if dim > MAX_DIMENSIONALITY:
    raise ValueError(f"dimension {dim} exceeds Pinecone max {MAX_DIMENSIONALITY}")
await store.create_collection(collection_name, dimension_num=dimension_num)

Type guard

def is_valid_collection_dim(d: int | None, default_d: int) -> bool:
    return (d if d is not None else default_d) <= 20000

Try / catch

from semantic_kernel.exceptions import ServiceInitializationError
try:
    await store.create_collection(name, dimension_num=dim)
except ServiceInitializationError as e:
    raise ValueError(f"cannot create collection {name}: {e}") from e

Prevention

Pass an explicit, validated dimension_num rather than relying on the default.
Confirm the embedding model dimension before create_collection.
Clamp dimension from config.

When it happens

Trigger: Calling create_collection(collection_name, dimension_num=N) with N > 20000, or omitting dimension_num so the store falls back to a _default_dimensionality that is > 20000 (which itself would have failed at construction).

Common situations: Per-collection override with a huge dimension; embedding model changed to one exceeding the Pinecone limit; dimension read from config without clamping.

Related errors

AI-assisted analysis of microsoft/semantic-kernel@c028a0c7dc (2026-08-13). Data as JSON: /api/errors/9f8e0d760045181e. Report an issue: GitHub.