BerriAI/litellm · error · ValueError

Both project_id and topic_id must be provided

Error message

Both project_id and topic_id must be provided

What it means

PubSub extended logging raises ValueError in __init__ when both project_id and topic_id cannot be resolved. Each falls back to env (GCS_PUBSUB_PROJECT_ID / GCS_PUBSUB_TOPIC_ID); the constructor runs a premium-user check first, then requires both values because publishing needs a fully-qualified topic.

Source

Thrown at litellm/integrations/gcs_pubsub/pub_sub.py:58

        Initialize Google Cloud Pub/Sub publisher

        Args:
            project_id (str): Google Cloud project ID
            topic_id (str): Pub/Sub topic ID
            credentials_path (str, optional): Path to Google Cloud credentials JSON file
        """
        from litellm.proxy.utils import _premium_user_check

        _premium_user_check()

        self.async_httpx_client = get_async_httpx_client(llm_provider=httpxSpecialProvider.LoggingCallback)

        self.project_id = project_id or os.getenv("GCS_PUBSUB_PROJECT_ID")
        self.topic_id = topic_id or os.getenv("GCS_PUBSUB_TOPIC_ID")
        self.path_service_account_json = credentials_path or os.getenv("GCS_PATH_SERVICE_ACCOUNT")

        if not self.project_id or not self.topic_id:
            raise ValueError("Both project_id and topic_id must be provided")

        self.flush_lock = asyncio.Lock()
        super().__init__(**kwargs, flush_lock=self.flush_lock)
        asyncio.create_task(self.periodic_flush())
        self.log_queue: list[SpendLogsPayload | StandardLoggingPayload] = []

    async def construct_request_headers(self) -> dict[str, str]:
        """Construct authorization headers using Vertex AI auth"""
        from litellm import vertex_chat_completion

        (
            _auth_header,
            vertex_project,
        ) = await vertex_chat_completion._ensure_access_token_async(
            credentials=self.path_service_account_json,
            project_id=self.project_id,
            custom_llm_provider="vertex_ai",
        )

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Set both env vars: GCS_PUBSUB_PROJECT_ID and GCS_PUBSUB_TOPIC_ID.
2. Or pass project_id= and topic_id= explicitly when constructing the callback.
3. Confirm a service account JSON (GCS_PATH_SERVICE_ACCOUNT) with pubsub publish rights is also configured.

Example fix

# before
# only GCS_PUBSUB_PROJECT_ID set

# after
export GCS_PUBSUB_PROJECT_ID=my-project
export GCS_PUBSUB_TOPIC_ID=llm-logs

import os

if "gcs_pubsub" in callbacks and not (os.getenv("GCS_PUBSUB_PROJECT_ID") and os.getenv("GCS_PUBSUB_TOPIC_ID")):
    raise ValueError("GCS_PUBSUB_PROJECT_ID and GCS_PUBSUB_TOPIC_ID are both required for gcs_pubsub")

def pubsub_env_ready() -> bool:
    import os
    return bool(os.getenv("GCS_PUBSUB_PROJECT_ID")) and bool(os.getenv("GCS_PUBSUB_TOPIC_ID"))

When it happens

Trigger: Instantiating the PubSub callback with neither constructor args nor env vars for project and topic — e.g. only GCS_PUBSUB_PROJECT_ID set, or callback name 'gcs_pubsub' enabled with no matching env at all.

Common situations: Env vars set in CI but not in the deployed proxy; typo'd var names; args swapped (topic passed as project).

Related errors

• GCS_BUCKET_NAME is not set in the environment, but GCS Bucke
• endpoint not set for GenericAPILogger, GENERIC_LOGGER_ENDPOI
• DD_API_KEY is not set, set 'DD_API_KEY=<>
• DD_SITE is not set in .env, set 'DD_SITE=<>
• DD_API_KEY is not set, set 'DD_API_KEY=<>'

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/153040a7b26fe47f. Report an issue: GitHub.