celery/celery · error · ImproperlyConfigured

Missing project:specify gcs_project to use gcs backend

Error message

Missing project:specify gcs_project to use gcs backend

What it means

GCSBackendBase.__init__ reads conf['gcs_project'] (the GCP project id that owns the bucket/client) and requires it; if absent/empty it raises ImproperlyConfigured. Unlike the bucket, the project is NOT inferred from the gcs:// URL, so it must be set in conf explicitly.

Source

Thrown at celery/backends/gcs.py:69

        super().__init__(**kwargs)
        self._client_lock = RLock()
        self._pid = getpid()
        self._retry_policy = DEFAULT_RETRY
        self._client = None

        conf = self.app.conf
        if self.url:
            url_params = self._params_from_url()
            conf.update(**dictfilter(url_params))

        self.bucket_name = conf.get('gcs_bucket')
        if not self.bucket_name:
            raise ImproperlyConfigured(
                'Missing bucket name: specify gcs_bucket to use gcs backend'
            )
        self.project = conf.get('gcs_project')
        if not self.project:
            raise ImproperlyConfigured(
                'Missing project:specify gcs_project to use gcs backend'
            )
        self.base_path = conf.get('gcs_base_path', '').strip('/')
        self._threadpool_maxsize = int(conf.get('gcs_threadpool_maxsize', 10))
        self.ttl = float(conf.get('gcs_ttl') or 0)
        if self.ttl < 0:
            raise ImproperlyConfigured(
                f'Invalid ttl: {self.ttl} must be greater than or equal to 0'
            )
        elif self.ttl:
            if not self._is_bucket_lifecycle_rule_exists():
                raise ImproperlyConfigured(
                    f'Missing lifecycle rule to use gcs backend with ttl on '
                    f'bucket: {self.bucket_name}'
                )

    def get(self, key):
        key = bytes_to_str(key)

View on GitHub (pinned to 571efe8120)

Solutions

Set app.conf.gcs_project = 'my-gcp-project' explicitly.
Export the env var that feeds gcs_project in the worker environment.
If using ADC, still set gcs_project to the project owning the bucket; the storage Client requires it.
Verify with `print(app.conf.gcs_project)` before starting workers.

Example fix

# before
result_backend = 'gcs://my-bucket/results'
app.conf.gcs_bucket = 'my-bucket'
# gcs_project unset -> ImproperlyConfigured

# after
app.conf.gcs_project = 'my-gcp-project'
app.conf.gcs_bucket = 'my-bucket'

Defensive patterns

Strategy: validation

Validate before calling

project = app.conf.get('gcs_project')
assert project, 'Specify gcs_project (the GCP project owning the bucket/client)'
app.conf.gcs_project = project

Type guard

def has_gcs_project(conf) -> bool:
    return bool(conf.get('gcs_project'))

Try / catch

from celery.exceptions import ImproperlyConfigured

try:
    GCSBackend(app=app, url=url)
except ImproperlyConfigured as exc:
    if 'gcs_project' in str(exc):
        log.error('Set gcs_project to the GCP project id')
    raise

Prevention

Always set gcs_project explicitly even when using Application Default Credentials.
Centralize GCP config (project, bucket) in one typed config object.
Fail fast in config loaders if the project env var is missing.

When it happens

Trigger: Configuring result_backend='gcs://bucket/path' without setting app.conf.gcs_project. The check is `if not self.project` at gcs.py:68.

Common situations: Assuming Application Default Credentials also supply the project (they may, but this backend still requires gcs_project); env var GOOGLE_CLOUD_PROJECT not exported; copy-paste config missing the project line; a multi-project account where the default differs from the target.

Related errors

AI-assisted analysis of celery/celery@571efe8120 (2026-08-04). Data as JSON: /data/errors/936653dd7638edde.json. Report an issue: GitHub.