cocoindex-io/cocoindex · error · ImportError

aiobotocore is required to use the Amazon S3 source connecto

Error message

aiobotocore is required to use the Amazon S3 source connector. Please install cocoindex[amazon_s3].

What it means

The Amazon S3 source connector in cocoindex depends on aiobotocore for async S3 access. The connector module raises this ImportError at import time when aiobotocore is not installed, because it needs AioBaseClient to type and drive S3 calls. It is an intentional guard telling you to install the connector's optional dependency group.

Source

Thrown at python/cocoindex/connectors/amazon_s3/_source.py:27

from __future__ import annotations

__all__ = [
    "S3File",
    "S3FilePath",
    "S3Walker",
    "get_object",
    "list_objects",
    "read",
]

from pathlib import PurePath
from typing import AsyncIterator, overload

try:
    from aiobotocore.client import AioBaseClient  # type: ignore[import-untyped]
except ImportError as e:
    raise ImportError(
        "aiobotocore is required to use the Amazon S3 source connector. "
        "Please install cocoindex[amazon_s3]."
    ) from e

from cocoindex.resources import file


def _etag_to_fingerprint(etag: object) -> bytes | None:
    """Convert an S3 ETag (a ``str``) to the ``bytes`` content fingerprint.

    botocore returns ETags as quoted strings (e.g. ``'"d41d8cd9..."'``), but
    :class:`~cocoindex.resources.file.FileMetadata.content_fingerprint` is typed
    ``bytes``.  Storing the raw ``str`` makes the memo state's
    ``tuple[datetime, bytes]`` round-trip fail on re-run (msgspec encodes a str
    but the decoder expects bin), so encode it to bytes here.
    """
    return etag.encode("utf-8") if isinstance(etag, str) else None

View on GitHub (pinned to e84aa99b32)

Solutions

1. Install the optional extra: `pip install cocoindex[amazon_s3]` (or `uv add cocoindex[amazon_s3]`).
2. If using uv/poetry, add `cocoindex[amazon_s3]` to the project dependencies, not plain cocoindex.
3. Verify with `python -c "import aiobotocore"` that the import now succeeds before loading the connector.
4. If you cannot install aiobotocore, use a source connector without the S3 dependency (e.g. localfs).

Example fix

# before
pip install cocoindex
import cocoindex.connectors.amazon_s3 as amazon_s3  # ImportError

# after
pip install "cocoindex[amazon_s3]"
import cocoindex.connectors.amazon_s3 as amazon_s3  # ok

try:
    import aiobotocore  # noqa: F401
except ImportError:
    raise SystemExit("Install the S3 extra: pip install 'cocoindex[amazon_s3]'")

When it happens

Trigger: Importing or using anything from python/cocoindex/connectors/amazon_s3 (e.g. amazon_s3.get_object or its source connector) in an environment where aiobotocore is absent — typically after `pip install cocoindex` without the amazon_s3 extra.

Common situations: Fresh virtualenv or CI container with base cocoindex only; deploying to a slim Docker image that dropped the extra; switching machines and forgetting the extras install.

Understand the failure class

Background: "X is not installed. Please install it with pip install Y": missing optional dependency errors — ImportError/ValueError raised when a library's optional extra was never installed — this error's family across 22 libraries.

Related errors

• Invalid S3 URI {uri!r}: expected 's3://bucket/key'.
• Cannot specify both an S3 URI and a separate key. Pass eithe
• key must be provided when bucket_name_or_uri is not an S3 UR
• aiomysql is required for async Doris connections. Install it
• asyncpg is required to use the PostgreSQL connector. Please

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/b7f151478ef54088. Report an issue: GitHub.