cocoindex-io/cocoindex · error · ValueError

Partial reads are not supported for Google Drive files.

Error message

Partial reads are not supported for Google Drive files.

What it means

Google Drive files are downloaded whole via the Drive API (export_media or get_media); there is no seekable stream, so the FileLike _read_sync helper only supports reading the entire file (size=-1). Requesting a byte-limited partial read raises ValueError.

Source

Thrown at python/cocoindex/connectors/google_drive/_source.py:124

                .get(
                    fileId=self._file_id,
                    fields="size, modifiedTime",
                )
                .execute()
            )
            size_raw = response.get("size")
            size = int(size_raw) if size_raw is not None else 0
            return file.FileMetadata(
                size=size,
                modified_time=_parse_modified_time(response.get("modifiedTime")),
            )

        return await asyncio.to_thread(_fetch)

    def _read_sync(self, size: int = -1) -> bytes:
        """Synchronously read file content (internal helper)."""
        if size != -1:
            raise ValueError("Partial reads are not supported for Google Drive files.")

        if self._mime_type in _EXPORT_MIME_BY_TYPE:
            export_mime = _EXPORT_MIME_BY_TYPE[self._mime_type]
            request = self._service.files().export_media(
                fileId=self._file_id, mimeType=export_mime
            )
        else:
            request = self._service.files().get_media(fileId=self._file_id)

        fh = io.BytesIO()
        downloader = MediaIoBaseDownload(fh, request)
        done = False
        while not done:
            _, done = downloader.next_chunk()
        return fh.getvalue()

    async def _read_impl(self, size: int = -1) -> bytes:
        """Read file content via Google Drive API in a thread pool."""

View on GitHub (pinned to e84aa99b32)

Solutions

  1. Call read() with no argument (or size=-1) to fetch the entire file, then slice the bytes in memory
  2. Refactor the consumer to accept a full bytes payload instead of chunked reads
  3. Download the file to local storage first and read it with a normal file handle if chunking is essential

Example fix

// before
data = await file.read(size=1024)
// after
data = (await file.read())[:1024]
Defensive patterns

Strategy: type-guard

Validate before calling

size = getattr(read_call, "size", -1)
if size != -1:
    raise ValueError("Read the whole Google Drive file; partial reads unsupported")

Type guard

def supports_partial_read(f) -> bool:
    return not hasattr(f, "_drive_file_id")  # drive-backed files read whole

Try / catch

try:
    data = await file.read(size=n)
except ValueError:
    data = (await file.read())[:n]

Prevention

When it happens

Trigger: Calling read(size=N) with a positive N on a GoogleDrive-backed FileLike object, directly or via code that reads files in chunks.

Common situations: Generic file-processing code that reads files in bounded chunks for memory reasons; libraries that probe files with small initial reads; adapting a chunked local-file reader to Google Drive sources.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/0cdf2271d3e50c06. Report an issue: GitHub.