iflytek/astron-agent · error · RuntimeError

fetch_all_document_chunks: empty page

Error message

fetch_all_document_chunks: empty page {page} but only {len(chunks)}/{total if total is not None else '?'} chunks fetched for doc={document_id}

What it means

fetch_all_document_chunks fails closed when the server returns an empty chunk page while the accumulated chunk count is still below the reported total. The code treats this protocol anomaly as fatal rather than returning a silently incomplete chunk set.

Solutions

Re-run the fetch; if it succeeds, the original failure was a concurrent re-chunk/delete race.
Ensure no other pipeline is re-parsing the document during chunk retrieval (serialize operations per document).
Check the RAGFlow server version for known pagination/total bugs and upgrade if affected.
If unavoidable, catch this error and treat the document fetch as failed rather than re-inserting partial chunks.

Example fix

// before: partial chunks silently re-inserted
try { chunks = await fetchAllDocumentChunks(ds, doc); } catch { chunks = []; }
// after: propagate — do not persist a partial set
try {
  chunks = await fetchAllDocumentChunks(ds, doc);
} catch (e) {
  logger.error("chunk fetch incomplete, aborting upsert", e);
  throw e;
}

Defensive patterns

Strategy: try-catch

Try / catch

try:
    chunks = await fetch_all_document_chunks(dataset_id, document_id)
except RuntimeError as e:
    if "empty page" in str(e):
        logger.error("Chunk fetch incomplete (concurrent re-parse or server bug); refusing partial upsert")
    raise  # never fall back to an empty/partial chunk list

Prevention

Never replace failed chunk fetches with empty lists before re-upsert
Serialize re-parsing and chunk reads per document
Add alerting on this error — it signals a real consistency race

When it happens

Trigger: Server reports total=N but returns an empty page before all N chunks are collected: stale pagination metadata, document/chunks deleted mid-request, or server mis-reporting totals after concurrent re-chunking.

Common situations: Document re-parsed (chunks deleted/recreated) while another request was paginating through it; RAGFlow bugs or version quirks in the total field; heavy concurrency on the same document.

Related errors

AI-assisted analysis of iflytek/astron-agent@5e758547a8 (2026-09-12). Data as JSON: /api/errors/e8457a596139ee7b. Report an issue: GitHub.

Appendix: source

Thrown at core/knowledge/infra/ragflow/ragflow_client.py:733

            raise RuntimeError(
                f"fetch_all_document_chunks failed on page {page} for "
                f"doc={document_id}: code={resp.get('code')}, "
                f"message={resp.get('message')}"
            )
        data = resp.get("data") or {}
        batch = data.get("chunks") or []
        chunks.extend(batch)
        # Missing/None/non-int => keep last-known good value.
        raw_total = data.get("total")
        if isinstance(raw_total, int) and raw_total >= 0:
            total = raw_total
        if total is not None and len(chunks) >= total:
            return chunks
        if not batch:
            # Protocol anomaly (stale pagination, mid-request deletion, or
            # server mis-report): fail closed to avoid re-inserting the
            # missing chunks as if they didn't exist.
            raise RuntimeError(
                f"fetch_all_document_chunks: empty page {page} but only "
                f"{len(chunks)}/{total if total is not None else '?'} "
                f"chunks fetched for doc={document_id}"
            )
        page += 1
    raise RuntimeError(
        f"fetch_all_document_chunks exceeded max_pages={max_pages} for "
        f"doc={document_id}; server may be mis-reporting total"
    )


async def get_document_info(dataset_id: str, doc_id: str) -> Optional[Dict[str, Any]]:
    """
    Get detailed information for a single document via RAGFlow's id filter.

    Uses the ``id`` query parameter on ``/api/v1/datasets/{dataset_id}/documents``,
    which performs exact-match filtering server-side (verified against RAGFlow
    v0.20.5 ~ v0.24.0: ``DocumentService.get_list`` applies

View on GitHub (pinned to 5e758547a8)