opendataloader-project/opendataloader-pdf · error · IOException

Backend processing failed and fallback is disabled

Error message

Backend processing failed and fallback is disabled

What it means

IOException thrown from the hybrid backend path when processBackendPath throws for the ENTIRE backend batch (not a per-page partial_success) AND config.getHybridConfig().isFallbackToJava() is false. The backend exception is wrapped as the cause. This is the whole-backend failure mode; contrast error 85 which handles per-page partial failures. The library first logs the backend failure at WARNING, then either falls back to Java (if enabled) or rethrows this.

Source

Thrown at java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/processors/HybridDocumentProcessor.java:334

        // Process backend path (synchronous)
        Map<Integer, List<IObject>> backendResults;
        Set<Integer> backendFailedPages = new HashSet<>();
        // Track SemanticPicture→EnrichedImageChunk swaps so we can rekey
        // ElementMetadata after Phase 6 cross-page processors (HeaderFooter,
        // List, etc.) re-run setIDs and mutate the picture's structure id.
        Map<EnrichedImageChunk, Long> pictureSwapOriginalIds = new IdentityHashMap<>();
        try {
            backendResults = processBackendPath(inputPdfName, backendPages, config, backendFailedPages);
            // Enrich backend results: copy StreamInfos from Java-extracted content for MCID linkage
            enrichBackendResults(backendResults, filteredContents, config.getHybridConfig(),
                pictureSwapOriginalIds);
        } catch (Exception e) {
            LOGGER.log(Level.WARNING, "Backend processing failed: {0}", e.getMessage());
            if (config.getHybridConfig().isFallbackToJava()) {
                LOGGER.log(Level.INFO, "Falling back to Java processing for backend pages");
                backendResults = processJavaPath(filteredContents, backendPages, config, totalPages);
            } else {
                throw new IOException("Backend processing failed and fallback is disabled", e);
            }
        }

        // Fallback: reprocess backend-failed pages through Java path
        if (!backendFailedPages.isEmpty()) {
            // Log 1-indexed page numbers for human readability
            List<Integer> failedPages1Indexed = backendFailedPages.stream()
                .map(p -> p + 1).sorted().collect(Collectors.toList());
            if (config.getHybridConfig().isFallbackToJava()) {
                LOGGER.log(Level.WARNING, "Backend returned partial_success: {0} page(s) failed (pages {1}), falling back to Java path",
                    new Object[]{backendFailedPages.size(), failedPages1Indexed});
                Map<Integer, List<IObject>> fallbackResults = processJavaPath(
                    filteredContents, backendFailedPages, config, totalPages
                );
                backendResults.putAll(fallbackResults);
            } else {
                LOGGER.log(Level.WARNING, "Backend returned partial_success: {0} page(s) failed (pages {1}), fallback disabled — failing fast",
                    new Object[]{backendFailedPages.size(), failedPages1Indexed});

View on GitHub (pinned to a7789b8e77)

Solutions

  1. Enable fallback if partial Java output is acceptable: set config.getHybridConfig().setFallbackToJava(true) / pass --hybrid-fallback so backend pages fall back to the Java path instead of failing.
  2. Diagnose the backend: read getCause() for the HTTP/connection error and verify the backend server is up and reachable at the configured hybrid_url.
  3. If strict backend-only is intentional, treat this as a hard failure — fix/restart the backend, then retry.
  4. Confirm hybrid_url is correct and resolvable from the client host, and that the backend health endpoint responds.

Example fix

// before: fallback disabled, whole backend down -> hard fail
config.getHybridConfig().setFallbackToJava(false);
// after: tolerate backend outage by falling back to Java
config.getHybridConfig().setFallbackToJava(true);
Defensive patterns

Strategy: fallback

Validate before calling

// Before processing, decide fallback policy and check backend reachability:
if (config.getHybridConfig() != null && config.getHybridConfig().isHybridEnabled()
        && !config.getHybridConfig().isFallbackToJava()) {
    if (!isBackendReachable(config.getHybridConfig().getBackendUrl())) {
        throw new IllegalStateException("Backend unreachable and fallback disabled");
    }
}

Type guard

static boolean isBackendFailedNoFallback(IOException e) {
    return e.getMessage() != null
        && e.getMessage().equals("Backend processing failed and fallback is disabled");
}

Try / catch

try {
    HybridDocumentProcessor.process(inputPdfName, config);
} catch (IOException e) {
    if (e.getMessage() != null && e.getMessage().startsWith("Backend processing failed and fallback is disabled")) {
        log.error("Backend down, fallback off", e.getCause());
        // Either enable fallback and retry, or surface to the user.
        config.getHybridConfig().setFallbackToJava(true);
        HybridDocumentProcessor.process(inputPdfName, config);
    } else throw e;
}

Prevention

When it happens

Trigger: Running HybridDocumentProcessor with hybrid enabled, backend pages routed, and processBackendPath(inputPdfName, backendPages, config, backendFailedPages) throws (backend server unreachable, returns non-2xx, connection reset, deserialization error) while config.getHybridConfig().isFallbackToJava() == false.

Common situations: Hybrid backend server is down or the hybrid_url is wrong/misconfigured and the operator explicitly disabled fallback (--hybrid-fallback off) for strict backend-only operation. TLS/cert mismatch with the backend. Backend out of memory. Network partition between client and backend.

Related errors

AI-assisted analysis of opendataloader-project/opendataloader-pdf@a7789b8e77 (2026-08-14). Data as JSON: /api/errors/91e3f86c1e72f998. Report an issue: GitHub.