{"record":{"id":"7d40f6f7a8a3a0d5","repo":"apache/seatunnel","slug":"error-scanning-data-from-region","errorCode":null,"errorMessage":"Error scanning data from region.","messagePattern":"Error scanning data from region\\.","errorType":"exception","errorClass":"TiClientInternalException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-cdc/connector-cdc-tidb/src/main/java/org/tikv/common/iterator/ScanIterator.java","lineNumber":120,"sourceCode":"            } else if (currentCache.size() > scanLimit) {\n                throw new IndexOutOfBoundsException(\n                        \"current cache size = \"\n                                + currentCache.size()\n                                + \", larger than \"\n                                + scanLimit);\n            } else {\n                // Start new scan from exact next key in current region\n                lastKey = Key.toRawKey(currentCache.get(currentCache.size() - 1).getKey());\n                startKey = lastKey.next().toByteString();\n            }\n            // notify last batch if lastKey is greater than or equal to endKey\n            // if startKey is empty, it indicates +∞\n            if (hasEndKey && lastKey.compareTo(endKey) >= 0 || startKey.isEmpty()) {\n                processingLastBatch = true;\n                startKey = null;\n            }\n        } catch (Exception e) {\n            throw new TiClientInternalException(\"Error scanning data from region.\", e);\n        }\n        return false;\n    }\n}\n","sourceCodeStart":102,"sourceCodeEnd":125,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-cdc/connector-cdc-tidb/src/main/java/org/tikv/common/iterator/ScanIterator.java#L102-L125","documentation":"ScanIterator wraps any Exception thrown while loading a batch of rows from a TiKV region into a TiClientInternalException with the fixed message 'Error scanning data from region.' The original exception is attached as the cause; this wrapper marks the failure point as the region-scan step of the iterator.","triggerScenarios":"Any failure inside regionCacheLoader/feed of the scan loop: RPC errors to TiKV (region not leader, timeout), key out of region bounds after region merges/splits, deserialization failures of KV pairs, or stale region cache.","commonSituations":"TiKV node restart or network partition during a snapshot read; region cache stale after cluster rebalancing (region not found / not leader); scanning a hotspot region that times out under load.","solutions":["Read the 'Caused by' to identify the underlying RPC/decode error and address it specifically","Retry the job — region cache refreshes usually resolve stale-region errors","Verify TiKV cluster health (pd-ctl health, region status) and network connectivity","Update the TiKV client / region cache (recreate the client) if region topology changed","Increase RPC/backoff timeouts if scans time out under load"],"exampleFix":"// before: stale region cache after rescaling causes scan failure\nScanIterator it = client.scan(...); // fails: not leader\n// after: rebuild client/region cache before retrying the scan\nclient.close();\nclient = TiClient.createForScan(pdAddresses); // fresh PD routing info\nScanIterator it = client.scan(...);","handlingStrategy":"retry","validationCode":"// Health-check TiKV/PD before long scans\nProcess p = new ProcessBuilder(\"pd-ctl\", \"health\").start();\nif (p.waitFor() != 0) throw new IllegalStateException(\"PD/TiKV unhealthy; postpone scan\");","typeGuard":null,"tryCatchPattern":"try {\n    loadBatchFromRegion();\n} catch (TiClientInternalException e) {\n    if (retryCount++ < MAX_RETRIES) {\n        refreshRegionCache(client); // re-fetch PD routing\n        Thread.sleep(backoff(retryCount));\n        return loadBatchFromRegion();\n    }\n    throw e;\n}","preventionTips":["Always inspect the 'Caused by' of TiClientInternalException","Refresh region cache after cluster rescaling or region merges","Monitor TiKV health and network latency for scan-heavy jobs","Set generous RPC and backoff timeouts for large snapshot scans"],"tags":["tikv","scan","rpc","region-cache","wrapped-exception"],"backgroundTag":"database-query-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}