{"record":{"id":"f161dd3cb3844f3a","repo":"BerriAI/litellm","slug":"mavvrik-focus-destination-gcs-chunk-upload-failed","errorCode":null,"errorMessage":"Mavvrik FOCUS destination: GCS chunk upload failed (chunk offset={offset}, expected={expected_statuses}, got={resp.status_code}): {resp.text[:400]}","messagePattern":"Mavvrik FOCUS destination: GCS chunk upload failed \\(chunk offset=(.+?), expected=(.+?), got=(.+?)\\): (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"litellm/integrations/focus/destinations/mavvrik_destination.py","lineNumber":226,"sourceCode":"            while offset < total:\n                chunk = gzip_bytes[offset : offset + _GCS_CHUNK_SIZE]\n                chunk_end = offset + len(chunk) - 1\n                is_final = (offset + len(chunk)) >= total\n                content_range = f\"bytes {offset}-{chunk_end}/{total}\" if is_final else f\"bytes {offset}-{chunk_end}/*\"\n                expected_statuses = {200, 201} if is_final else {308}\n\n                resp = await self._http.client.request(\n                    method=\"PUT\",\n                    url=session_uri,\n                    headers={\n                        \"Content-Type\": \"application/gzip\",\n                        \"Content-Range\": content_range,\n                    },\n                    content=chunk,\n                    timeout=120.0,\n                )\n                if resp.status_code not in expected_statuses:\n                    raise RuntimeError(\n                        f\"Mavvrik FOCUS destination: GCS chunk upload failed \"\n                        f\"(chunk offset={offset}, expected={expected_statuses}, \"\n                        f\"got={resp.status_code}): {resp.text[:400]}\"\n                    )\n                offset += len(chunk)\n                verbose_logger.debug(\n                    \"Mavvrik FOCUS destination: uploaded chunk offset=%d/%d\",\n                    offset,\n                    total,\n                )\n        except Exception:\n            # Cancel the open GCS session so it doesn't linger for up to 1 week.\n            try:\n                await self._http.client.request(method=\"DELETE\", url=session_uri, timeout=10.0)\n                verbose_logger.debug(\"Mavvrik FOCUS destination: cancelled GCS session after error\")\n            except Exception:\n                pass\n            raise","sourceCodeStart":208,"sourceCodeEnd":244,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/integrations/focus/destinations/mavvrik_destination.py#L208-L244","documentation":"Raised by the Mavvrik FOCUS destination when one chunk of a GCS resumable upload does not return an expected status code. The destination streams the serialized FOCUS export in chunks to a GCS resumable-session URI; any chunk PUT that lands outside expected_statuses aborts the whole upload. The enclosing except cancels the open GCS session so it does not linger (GCS sessions can persist up to 1 week).","triggerScenarios":"PUT {session_uri} with Content-Range for a chunk returns a status not in expected_statuses (e.g. 404 after the resumable session expired, 400 for a bad Content-Range/offset, 401/403 on revoked credentials, 5xx from GCS, or 200 when 308 Resume Incomplete was expected on intermediate chunks). Chunk offsets must advance exactly by len(chunk); a mismatch causes 400 responses.","commonSituations":"Very large exports where the resumable session times out mid-transfer; network interruptions between proxy and GCS; expired/revoked GCS signed-session URIs; clock-skewed or rotated credentials; retrying a session that was already cancelled by a previous failure.","solutions":["Check resp.status_code and resp.text[:400] in the log: 308 handling vs 5xx vs 401 dictates the fix — transient 5xx/network errors are resolved by re-running the export, which starts a fresh GCS session.","If the export is large, increase chunk size or reduce the window/frequency so each upload finishes before the session expires.","Verify GCS credentials (service account / signed URL issuer) still have storage.objects create/update rights on the target bucket.","Confirm the metricsMarker (error 421-424 path) was not advanced, so a re-run re-uploads the same window without data loss."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    await destination.deliver(content=buf, time_window=w, filename=fn)\nexcept RuntimeError as e:\n    if \"GCS chunk upload failed\" in str(e):\n        log.warning(\"FOCUS upload failed for window %s; will retry on next run\", w)\n        raise  # metricsMarker not advanced, so the re-run re-uploads this window\n    raise","preventionTips":["Keep export windows small enough that the resumable session completes well within GCS session lifetime.","Alert on any 'GCS chunk upload failed' message and re-run the export; marker-based catch-up makes retries safe.","Monitor GCS quota and credential expiry for the service account used by the Mavvrik destination."],"tags":["network","gcs","resumable-upload","mavvrik","focus-export"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}