apache/seatunnel · error · TiClientInternalException

Error scanning data from region.

Error message

Error scanning data from region.

What it means

ScanIterator wraps any Exception thrown while loading a batch of rows from a TiKV region into a TiClientInternalException with the fixed message 'Error scanning data from region.' The original exception is attached as the cause; this wrapper marks the failure point as the region-scan step of the iterator.

Source

Thrown at seatunnel-connectors-v2/connector-cdc/connector-cdc-tidb/src/main/java/org/tikv/common/iterator/ScanIterator.java:120

            } else if (currentCache.size() > scanLimit) {
                throw new IndexOutOfBoundsException(
                        "current cache size = "
                                + currentCache.size()
                                + ", larger than "
                                + scanLimit);
            } else {
                // Start new scan from exact next key in current region
                lastKey = Key.toRawKey(currentCache.get(currentCache.size() - 1).getKey());
                startKey = lastKey.next().toByteString();
            }
            // notify last batch if lastKey is greater than or equal to endKey
            // if startKey is empty, it indicates +∞
            if (hasEndKey && lastKey.compareTo(endKey) >= 0 || startKey.isEmpty()) {
                processingLastBatch = true;
                startKey = null;
            }
        } catch (Exception e) {
            throw new TiClientInternalException("Error scanning data from region.", e);
        }
        return false;
    }
}

View on GitHub (pinned to cf67b549a7)

Solutions

  1. Read the 'Caused by' to identify the underlying RPC/decode error and address it specifically
  2. Retry the job — region cache refreshes usually resolve stale-region errors
  3. Verify TiKV cluster health (pd-ctl health, region status) and network connectivity
  4. Update the TiKV client / region cache (recreate the client) if region topology changed
  5. Increase RPC/backoff timeouts if scans time out under load

Example fix

// before: stale region cache after rescaling causes scan failure
ScanIterator it = client.scan(...); // fails: not leader
// after: rebuild client/region cache before retrying the scan
client.close();
client = TiClient.createForScan(pdAddresses); // fresh PD routing info
ScanIterator it = client.scan(...);
Defensive patterns

Strategy: retry

Validate before calling

// Health-check TiKV/PD before long scans
Process p = new ProcessBuilder("pd-ctl", "health").start();
if (p.waitFor() != 0) throw new IllegalStateException("PD/TiKV unhealthy; postpone scan");

Try / catch

try {
    loadBatchFromRegion();
} catch (TiClientInternalException e) {
    if (retryCount++ < MAX_RETRIES) {
        refreshRegionCache(client); // re-fetch PD routing
        Thread.sleep(backoff(retryCount));
        return loadBatchFromRegion();
    }
    throw e;
}

Prevention

When it happens

Trigger: Any failure inside regionCacheLoader/feed of the scan loop: RPC errors to TiKV (region not leader, timeout), key out of region bounds after region merges/splits, deserialization failures of KV pairs, or stale region cache.

Common situations: TiKV node restart or network partition during a snapshot read; region cache stale after cluster rebalancing (region not found / not leader); scanning a hotspot region that times out under load.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/7d40f6f7a8a3a0d5. Report an issue: GitHub.