celery/celery · error · ImproperlyConfigured

Missing aws s3 creds

Error message

Missing aws s3 creds

What it means

After building a boto3.Session with the configured keys, S3Backend._connect_to_s3 calls session.get_credentials(); if that returns None (no credentials resolved from any provider chain) it raises ImproperlyConfigured('Missing aws s3 creds'). This catches the case where neither explicit keys, env vars, shared-credentials file, nor an IAM role yielded credentials.

Source

Thrown at celery/backends/s3.py:86

    def set(self, key, value):
        key = bytes_to_str(key)
        s3_object = self._get_s3_object(key)
        s3_object.put(Body=value)

    def delete(self, key):
        key = bytes_to_str(key)
        s3_object = self._get_s3_object(key)
        s3_object.delete()

    def _connect_to_s3(self):
        session = boto3.Session(
            aws_access_key_id=self.aws_access_key_id,
            aws_secret_access_key=self.aws_secret_access_key,
            region_name=self.aws_region
        )
        if session.get_credentials() is None:
            raise ImproperlyConfigured('Missing aws s3 creds')
        return session.resource('s3', endpoint_url=self.endpoint_url)

View on GitHub (pinned to 571efe8120)

Solutions

1. Set app.conf.s3_access_key_id and app.conf.s3_secret_access_key
2. Or export AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in the worker environment
3. Or attach an IAM instance/task role when running on AWS (EC2/ECS/EKS)
4. Verify with `aws sts get-caller-identity` that credentials resolve on the host

Example fix

// before
app.conf.result_backend = 's3'
app.conf.s3_bucket = 'bucket'
# no credentials anywhere

// after
app.conf.result_backend = 's3'
app.conf.s3_bucket = 'bucket'
app.conf.s3_access_key_id = os.environ['AWS_ACCESS_KEY_ID']
app.conf.s3_secret_access_key = os.environ['AWS_SECRET_ACCESS_KEY']

import boto3
sess = boto3.Session(
    aws_access_key_id=app.conf.get('s3_access_key_id'),
    aws_secret_access_key=app.conf.get('s3_secret_access_key'),
    region_name=app.conf.get('s3_region'),
)
if sess.get_credentials() is None:
    raise SystemExit('No AWS credentials resolved; set s3_access_key_id/s3_secret_access_key or attach an IAM role')

from celery.exceptions import ImproperlyConfigured
try:
    backend = S3Backend(app=app)
except ImproperlyConfigured as e:
    if 'creds' in str(e):
        # fall back, alert, or attach role
        raise
    raise

When it happens

Trigger: Configuring the S3 backend with neither s3_access_key_id/s3_secret_access_key in conf nor AWS_* env vars nor an attached IAM role, then instantiating the backend.

Common situations: Local development without AWS env vars; ECS/EC2 task role not attached; expired STS session tokens; credentials file missing on the worker host; misspelled s3_access_key_id key.

Related errors

• You need to specify both the Access Key ID and Secret.
• You must install boto3 to use s3 backend
• Missing bucket name
• You need to install the boto3 library to use the DynamoDB ba
• Credential provider is not an instance of a redis.Credential

AI-assisted analysis of celery/celery@571efe8120 (2026-08-04). Data as JSON: /data/errors/7dfa0c1c1815942a.json. Report an issue: GitHub.