pathwaycom/pathway · error · ValueError

A Pinecone API key is required. Pass api_key=... or set the

Error message

A Pinecone API key is required. Pass api_key=... or set the PINECONE_API_KEY environment variable.

What it means

Raised by pw.io.pinecone.write() when no Pinecone API key can be resolved: the api_key argument is None/empty and the PINECONE_API_KEY environment variable is unset or empty. Without a key the connector cannot authenticate upserts.

Source

Thrown at python/pathway/io/pinecone/__init__.py:397

                    f"column {col._name!r} is used as the {role} and cannot also "
                    "be a metadata column. Remove it from metadata_columns."
                )
            metadata_names.append(col._name)

    if batch_size <= 0:
        raise ValueError(f"batch_size must be a positive integer, got {batch_size}.")

    if pk_name is not None:
        _check_primary_key_dtype(pk_name, table._get_column(pk_name).dtype)
    _check_vector_dtype(vector_name, table._get_column(vector_name).dtype)
    for col_name in metadata_names:
        _check_metadata_dtype(col_name, table._get_column(col_name).dtype)

    resolved_api_key = (
        api_key if api_key is not None else os.environ.get("PINECONE_API_KEY")
    )
    if not resolved_api_key:
        raise ValueError(
            "A Pinecone API key is required. Pass api_key=... or set the "
            "PINECONE_API_KEY environment variable."
        )

    column_index = {name_: index for index, name_ in enumerate(table.column_names())}
    vector_index = column_index[vector_name]
    metadata_indices = [column_index[name_] for name_ in metadata_names]

    data_storage = api.DataStorage(
        storage_type="pinecone",
        key_field_index=get_column_index(table, primary_key),
        max_batch_size=batch_size,
        pinecone_params=api.PineconeParams(
            api_key=resolved_api_key,
            index_name=index_name,
            vector_index=vector_index,
            metadata_indices=metadata_indices,
            namespace=namespace,

View on GitHub (pinned to fa2f74a464)

Solutions

1. Pass the key explicitly: pw.io.pinecone.write(..., api_key=os.environ['PINECONE_API_KEY']).
2. Or export the variable in the environment the pipeline runs in: export PINECONE_API_KEY=... (and add it to your CI secrets / container env).
3. Never hardcode the key in source; load it from a secrets manager.

Example fix

# before
pw.io.pinecone.write(t, "docs", vector=t.emb)  # PINECONE_API_KEY unset
# after
import os
pw.io.pinecone.write(t, "docs", vector=t.emb, api_key=os.environ["PINECONE_API_KEY"])

import os
api_key = api_key or os.environ.get("PINECONE_API_KEY")
if not api_key:
    raise RuntimeError("PINECONE_API_KEY is not set; refusing to build pipeline")

When it happens

Trigger: Calling pw.io.pinecone.write(...) without api_key= while PINECONE_API_KEY is not exported in the process environment (common in cron jobs, CI, containers, new shells).

Common situations: Works locally (env var set in .bashrc) but fails in CI/Docker/systemd where the variable is not propagated; or passing api_key="" instead of None.

Related errors

• demo.noisy_linear_stream error: nb_rows should be strictly p
• no persistent storage configured for the disk cache
• primary_key column {name!r} is nullable (type {dtype}); a Pi
• primary_key column {name!r} has unsupported type {dtype}; a
• vector column {name!r} is nullable (type {dtype}); every row

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/ec8751409270d655. Report an issue: GitHub.