{"record":{"id":"deefd49c0fa48df1","repo":"infiniflow/ragflow","slug":"azure-blob-failed-to-download-name-exc","errorCode":null,"errorMessage":"Azure Blob: failed to download {name}: {exc}","messagePattern":"Azure Blob: failed to download (.+?): (.+?)","errorType":"validation","errorClass":"UnexpectedValidationError","httpStatus":null,"severity":"error","filePath":"common/data_source/azure_blob_connector.py","lineNumber":329,"sourceCode":"\n                # Download blob content. A blob that was deleted between the\n                # listing and this fetch is genuinely gone — skip it. Any\n                # other failure (throttling, transient 5xx, network) must\n                # abort the run: the sync framework advances its watermark\n                # from successfully yielded docs, so silently skipping a\n                # transiently-failed blob while newer blobs succeed would\n                # move the watermark past it and drop it permanently.\n                try:\n                    blob_client = self._container_client.get_blob_client(name)\n                    data = blob_client.download_blob().readall()\n                except Exception as exc:\n                    if _is_blob_gone(exc):\n                        logger.warning(\n                            \"Azure Blob: %s vanished between listing and fetch; skipping\",\n                            name,\n                        )\n                        continue\n                    raise UnexpectedValidationError(f\"Azure Blob: failed to download {name}: {exc}\") from exc\n\n                doc_updated_at = last_modified.astimezone(timezone.utc) if last_modified else datetime.now(timezone.utc)\n\n                ext = _extension(name)\n                doc = Document(\n                    id=name,\n                    source=\"azure_blob\",\n                    semantic_identifier=name,\n                    extension=ext,\n                    blob=data,\n                    doc_updated_at=doc_updated_at,\n                    size_bytes=len(data),\n                    fingerprint=current_etag or None,\n                    metadata={\n                        \"container\": _container_name(self._container_client),\n                        \"etag\": current_etag,\n                        \"prefix\": self.prefix,\n                    },","sourceCodeStart":311,"sourceCodeEnd":347,"githubUrl":"https://github.com/infiniflow/ragflow/blob/554fb1133ac3861732235ad9c377eb5e0a770665/common/data_source/azure_blob_connector.py#L311-L347","documentation":"Raised inside _iter_documents when downloading an individual blob fails and the failure is not a 'blob vanished' condition (those are logged and skipped). The download (get_blob_client + download_blob().readall()) error is wrapped in UnexpectedValidationError, aborting the whole batch iteration — including blobs already listed but not yet fetched.","triggerScenarios":"Listing succeeded, then download_blob().readall() throws for a specific blob: blob became unreadable (lease/permission change mid-run), a transient network reset during the read, or a payload too large for memory/time limits. _is_blob_gone(exc) returned False, so it is not treated as a vanished blob.","commonSituations":"Flaky networks on large blobs, concurrent deletes that leave the blob in a partially-gone state Azure still lists, or anti-virus/middleware terminating long streaming responses.","solutions":["Retry the ingestion run — the checkpoint logic deliberately avoids advancing the watermark past failed blobs (see the comment above the try), so nothing is lost","If one specific blob always fails, download it manually (az storage blob download) to isolate whether it is corrupt, leased, or over a size limit","Add exponential backoff around the whole ingestion run for transient network causes","Check exc.__cause__ for the SDK error code (e.g. 500/timeout vs 403) to pick retry vs fix"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"# Pre-check the blobs most likely to fail (size) using listing metadata:\nfor props in container_client.list_blobs(name_starts_with=prefix):\n    if props.size > MAX_BLOB_BYTES:\n        logger.warning(\"skipping oversized blob %s (%d bytes)\", props.name, props.size)","typeGuard":null,"tryCatchPattern":"try:\n    for batch in connector.load_from_checkpoint(start, end, checkpoint):\n        process(batch)\nexcept UnexpectedValidationError as e:\n    if \"failed to download\" not in str(e):\n        raise\n    # watermark was NOT advanced past the failed blob — safe to retry the run\n    backoff_and_retry(whole_run, max_attempts=3)","preventionTips":["Rely on the connector's checkpoint design: failed downloads do not advance the watermark, so re-running is always safe","Wrap ingestion runs in retry-with-backoff for transient network errors","Isolate chronically failing blobs by downloading them out-of-band to decide skip vs fix"],"tags":["azure","azure-blob","download","transient","ingestion"],"backgroundTag":null,"analyzedSha":"554fb1133ac3861732235ad9c377eb5e0a770665","analyzedAt":"2026-08-15T09:20:16.380Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}