{"record":{"id":"c227136fd469efcc","repo":"pingcap/tidb","slug":"fail-to-create-iterator","errorCode":null,"errorMessage":"fail to create iterator","messagePattern":"fail to create iterator","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"pkg/ingestor/ingestctrl/iterator.go","lineNumber":170,"sourceCode":"\tkeyAdapter common.KeyAdapter,\n\topts *pebble.IterOptions,\n\tdupDB *pebble.DB,\n\tlogger log.Logger,\n\tdupOpt common.DupDetectOpt,\n\tbuf *membuf.Buffer,\n) *dupDetectIter {\n\tnewOpts := &pebble.IterOptions{TableFilter: opts.TableFilter}\n\tif len(opts.LowerBound) > 0 {\n\t\tnewOpts.LowerBound = keyAdapter.Encode(nil, opts.LowerBound, common.MinRowID)\n\t}\n\tif len(opts.UpperBound) > 0 {\n\t\tnewOpts.UpperBound = keyAdapter.Encode(nil, opts.UpperBound, common.MinRowID)\n\t}\n\n\tdetector := common.NewDupDetector(keyAdapter, dupDB.NewBatch(), logger, dupOpt)\n\titer, err := db.NewIter(newOpts)\n\tif err != nil {\n\t\tpanic(\"fail to create iterator\")\n\t}\n\treturn &dupDetectIter{\n\t\tkeyAdapter:  keyAdapter,\n\t\titer:        iter,\n\t\tbuf:         buf,\n\t\tdupDetector: detector,\n\t\tlogger:      logger,\n\t}\n}\n\ntype dupDBIter struct {\n\titer       *pebble.Iterator\n\tkeyAdapter common.KeyAdapter\n\tcurKey     []byte\n\terr        error\n}\n\nfunc (d *dupDBIter) Error() error {","sourceCodeStart":152,"sourceCodeEnd":188,"githubUrl":"https://github.com/pingcap/tidb/blob/d01f9615c13e02dfa75922745640dec9d81b802e/pkg/ingestor/ingestctrl/iterator.go#L152-L188","documentation":"newDupDetectIter in pkg/ingestor/ingestctrl (the engine behind IMPORT INTO / TiDB Lightning duplicate detection) creates a pebble iterator over the target engine to scan existing keys; if db.NewIter returns an error the constructor panics with 'fail to create iterator'. The panic aborts the import worker instead of continuing with a broken dedup scan.","triggerScenarios":"IMPORT INTO / Lightning with duplicate detection (DUP_DETECT) reaching the scan phase while the underlying pebble engine cannot create an iterator: engine already closed/closing (import cancelled concurrently), corrupted local engine files, or invalid encoded bounds produced by KeyAdapter.Encode.","commonSituations":"Cancelling an IMPORT INTO mid-flight so a shared engine closes while the dup iterator is still being built; disk corruption under the import temp dir; running duplicate detection on an engine left dirty by a previously failed import; racing duplicate-check engines across duplicate jobs sharing disk.","solutions":["Retry the IMPORT INTO after the previous one fully terminated (check import jobs; IMPORT INTO cannot run concurrently on the same table).","Clean the import working area: stop the import, clear leftover engine data under tmp paths (or run with a fresh import dir), then rerun.","If duplicate detection is not needed, import with DUPLICATE_DETECTION=OFF (or 'replace'/'ignore' error handling per your statement options) to bypass the dup iterator.","Check disk health and space under the import temp dir; corrupt SSTs there cause NewIter failures.","If it reproduces on a healthy setup, report it at github.com/pingcap/tidb with the stack."],"exampleFix":"-- before\nIMPORT INTO t FROM '/data/t.csv'; -- panics when a prior import left a dirty engine\n\n-- after\n-- 1) confirm no import is running\nSHOW IMPORT JOBS;\n-- 2) rerun cleanly; skip dup scan if duplicates are impossible\nIMPORT INTO t FROM '/data/t.csv' WITH DUPLICATE_DETECTION=OFF;","handlingStrategy":"retry","validationCode":"-- before importing, confirm no other import is active and the engine is clean\nSHOW IMPORT JOBS;\n-- expect no 'running'/'pending' job for the target table\nSHOW TABLE t REGIONS; -- sanity: table is writable, no import lock","typeGuard":null,"tryCatchPattern":"-- panics surface as a failed IMPORT job; inspect then retry on clean state\nSHOW IMPORT JOBS; SELECT * FROM information_schema.import_jobs WHERE status = 'failed';\n-- after cleanup: rerun IMPORT INTO t FROM '/data/t.csv';","preventionTips":["Never run two IMPORT INTO / Lightning tasks against the same table concurrently.","After a failed or cancelled import, clean leftover engine/dup dirs before retrying.","Use DUPLICATE_DETECTION=OFF when the source is already deduplicated.","Monitor disk health and capacity under the import temp paths."],"tags":["tidb","ingestor","import-into","lightning","pebble","panic"],"backgroundTag":null,"analyzedSha":"d01f9615c13e02dfa75922745640dec9d81b802e","analyzedAt":"2026-08-15T12:19:33.324Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}