prestodb/presto · critical · PrestoException

GENERIC_INTERNAL_ERROR

GENERIC_INTERNAL_ERROR

Error message

Error loading index for join

What it means

During IndexLoader.load, if the index-building Driver's process() future completes exceptionally, the ExecutionException is rethrown as GENERIC_INTERNAL_ERROR 'Error loading index for join' with the original cause. This indicates the index build itself failed, not a timeout.

Source

Thrown at presto-main-base/src/main/java/com/facebook/presto/operator/index/IndexLoader.java:358

        {
            // Generate a RecordSet that only presents index keys that have not been cached and are deduped based on lookupSourceInputChannels
            UnloadedIndexKeyRecordSet recordSetForLookupSource = new UnloadedIndexKeyRecordSet(pipelineContext.getSession(), indexSnapshotReference.get(), lookupSourceInputChannels, indexTypes, requests, joinCompiler);

            // Drive index lookup to produce the output (landing in indexSnapshotBuilder)
            try (Driver driver = driverFactory.createDriver(pipelineContext.addDriverContext())) {
                PlanNodeId sourcePlanNodeId = driverFactory.getSourceId().get();
                ScheduledSplit split = new ScheduledSplit(0, sourcePlanNodeId, new Split(INDEX_CONNECTOR_ID, new ConnectorTransactionHandle() {}, new IndexSplit(recordSetForLookupSource)));
                driver.updateSource(new TaskSource(sourcePlanNodeId, ImmutableSet.of(split), true));
                while (!driver.isFinished()) {
                    ListenableFuture<?> process = driver.process();
                    try {
                        process.get(timeout.toMillis(), MILLISECONDS);
                    }
                    catch (TimeoutException e) {
                        throw new PrestoException(INDEX_LOADER_TIMEOUT, format("Exceeded the time limit of %s loading indexes for index join", timeout));
                    }
                    catch (ExecutionException e) {
                        throw new PrestoException(GENERIC_INTERNAL_ERROR, "Error loading index for join", e);
                    }
                    catch (InterruptedException e) {
                        Thread.currentThread().interrupt();
                        throw new PrestoException(GENERIC_INTERNAL_ERROR, "Error loading index for join", e);
                    }
                }
            }

            if (indexSnapshotBuilder.isMemoryExceeded()) {
                clearCachedData();
                return false;
            }

            // Generate a RecordSet that presents unique index keys that have not been cached
            UnloadedIndexKeyRecordSet indexKeysRecordSet = (lookupSourceInputChannels.equals(allInputChannels))
                    ? recordSetForLookupSource
                    : new UnloadedIndexKeyRecordSet(pipelineContext.getSession(), indexSnapshotReference.get(), allInputChannels, indexTypes, requests, joinCompiler);

View on GitHub (pinned to 55bb57d202)

Solutions

  1. Inspect the wrapped cause (getCause()) for the real failure
  2. Check worker logs and memory (GC/OOM) around the failure
  3. Add filters to reduce index source data; verify connector health
  4. Retry the query; if persistent, fall back to a hash join

Example fix

// before
JOIN with lookup/index join failing on connector read
// after
SET SESSION join_distribution_type = 'PARTITIONED'; -- force hash join fallback
Defensive patterns

Strategy: try-catch

Try / catch

try { run(sql); } catch (PrestoException e) { if (e.getErrorCode().getName().equals("GENERIC_INTERNAL_ERROR") && e.getMessage().equals("Error loading index for join") && e.getCause() != null) { inspectCause(e.getCause()); retryOrFallbackToHashJoin(sql); } else throw e; }

Prevention

When it happens

Trigger: Index join where the driver processing index source rows throws (connector error, bad data, OOM inside the driver), surfacing via process.get() ExecutionException.

Common situations: Connector failures while reading index source; memory pressure killing the build task; data causing exceptions in expressions evaluated during index build.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/406182b1298f1aea. Report an issue: GitHub.