BerriAI/litellm · error · RuntimeError

Mavvrik FOCUS destination: GCS chunk upload failed (chunk of

Error message

Mavvrik FOCUS destination: GCS chunk upload failed (chunk offset={offset}, expected={expected_statuses}, got={resp.status_code}): {resp.text[:400]}

What it means

Raised by the Mavvrik FOCUS destination when one chunk of a GCS resumable upload does not return an expected status code. The destination streams the serialized FOCUS export in chunks to a GCS resumable-session URI; any chunk PUT that lands outside expected_statuses aborts the whole upload. The enclosing except cancels the open GCS session so it does not linger (GCS sessions can persist up to 1 week).

Source

Thrown at litellm/integrations/focus/destinations/mavvrik_destination.py:226

            while offset < total:
                chunk = gzip_bytes[offset : offset + _GCS_CHUNK_SIZE]
                chunk_end = offset + len(chunk) - 1
                is_final = (offset + len(chunk)) >= total
                content_range = f"bytes {offset}-{chunk_end}/{total}" if is_final else f"bytes {offset}-{chunk_end}/*"
                expected_statuses = {200, 201} if is_final else {308}

                resp = await self._http.client.request(
                    method="PUT",
                    url=session_uri,
                    headers={
                        "Content-Type": "application/gzip",
                        "Content-Range": content_range,
                    },
                    content=chunk,
                    timeout=120.0,
                )
                if resp.status_code not in expected_statuses:
                    raise RuntimeError(
                        f"Mavvrik FOCUS destination: GCS chunk upload failed "
                        f"(chunk offset={offset}, expected={expected_statuses}, "
                        f"got={resp.status_code}): {resp.text[:400]}"
                    )
                offset += len(chunk)
                verbose_logger.debug(
                    "Mavvrik FOCUS destination: uploaded chunk offset=%d/%d",
                    offset,
                    total,
                )
        except Exception:
            # Cancel the open GCS session so it doesn't linger for up to 1 week.
            try:
                await self._http.client.request(method="DELETE", url=session_uri, timeout=10.0)
                verbose_logger.debug("Mavvrik FOCUS destination: cancelled GCS session after error")
            except Exception:
                pass
            raise

View on GitHub (pinned to 6c2dcb801b)

Solutions

Check resp.status_code and resp.text[:400] in the log: 308 handling vs 5xx vs 401 dictates the fix — transient 5xx/network errors are resolved by re-running the export, which starts a fresh GCS session.
If the export is large, increase chunk size or reduce the window/frequency so each upload finishes before the session expires.
Verify GCS credentials (service account / signed URL issuer) still have storage.objects create/update rights on the target bucket.
Confirm the metricsMarker (error 421-424 path) was not advanced, so a re-run re-uploads the same window without data loss.

Defensive patterns

Strategy: retry

Try / catch

try:
    await destination.deliver(content=buf, time_window=w, filename=fn)
except RuntimeError as e:
    if "GCS chunk upload failed" in str(e):
        log.warning("FOCUS upload failed for window %s; will retry on next run", w)
        raise  # metricsMarker not advanced, so the re-run re-uploads this window
    raise

Prevention

Keep export windows small enough that the resumable session completes well within GCS session lifetime.
Alert on any 'GCS chunk upload failed' message and re-run the export; marker-based catch-up makes retries safe.
Monitor GCS quota and credential expiry for the service account used by the Mavvrik destination.

When it happens

Trigger: PUT {session_uri} with Content-Range for a chunk returns a status not in expected_statuses (e.g. 404 after the resumable session expired, 400 for a bad Content-Range/offset, 401/403 on revoked credentials, 5xx from GCS, or 200 when 308 Resume Incomplete was expected on intermediate chunks). Chunk offsets must advance exactly by len(chunk); a mismatch causes 400 responses.

Common situations: Very large exports where the resumable session times out mid-transfer; network interruptions between proxy and GCS; expired/revoked GCS signed-session URIs; clock-skewed or rotated credentials; retrying a session that was already cancelled by a previous failure.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/f161dd3cb3844f3a. Report an issue: GitHub.