{"record":{"id":"82ba94ce05207c7a","repo":"NationalSecurityAgency/ghidra","slug":"is-already-ingested","errorCode":null,"errorMessage":" is already ingested","messagePattern":" is already ingested","errorType":"exception","errorClass":"DatabaseNonFatalException","httpStatus":null,"severity":"warning","filePath":"Ghidra/Features/BSim/src/main/java/ghidra/features/bsim/query/client/AbstractSQLFunctionDatabase.java","lineNumber":1070,"sourceCode":"\t\t\t\t\t\tthrow new LSHException(fatalerror);\n\t\t\t\t\t}\n\t\t\t\t\tthrow new DatabaseNonFatalException(\n\t\t\t\t\t\tFunctionDatabase.constructNonfatalError(cmp, erec, tmp));\n\t\t\t\t}\n\t\t\t\tif (erec.getRowId() != null) {\n\t\t\t\t\tif (!erec.getRowId().equals(tmp.getRowId())) {\n\t\t\t\t\t\tthrow new LSHException(\n\t\t\t\t\t\t\t\"Id mismatch when inserting executable: \" + erec.getNameExec());\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\telse {\n\t\t\t\t\tinput.setExeRowId(erec, tmp.getRowId());\n\t\t\t\t}\n\t\t\t\tinput.setExeAlreadyStored(erec);\n\t\t\t\t// Just because we've seen a library before doesn't mean we should stop function insertion\n\t\t\t\t// Libraries are likely to be partially inserted multiple times\n\t\t\t\tif (!erec.isLibrary()) {\n\t\t\t\t\tthrow new DatabaseNonFatalException(\n\t\t\t\t\t\terec.getNameExec() + \" is already ingested\");\n\t\t\t\t}\n\t\t\t\tpickout_storedfuncs = true; // At least one executable has previously inserted functions\n\t\t\t}\n\t\t}\n\n\t\tif (pickout_storedfuncs) {\n\t\t\tif (!markPreviouslyStoredFunctions(input, input.listAllFunctions())) {\n\t\t\t\tthrow new DatabaseNonFatalException(\"Already inserted\");\n\t\t\t}\n\t\t}\n\t}\n\n\t/**\n\t * Do the final work of inserting new ExecutableRecords into the database. This function\n\t * assumes testExecutableDuplication has already run and marked previously ingested records\n\t * \n\t * @param input the executable descriptor","sourceCodeStart":1052,"sourceCodeEnd":1088,"githubUrl":"https://github.com/NationalSecurityAgency/ghidra/blob/d5f144c24d6bc53c9cbf4448c6d11143e7696206/Ghidra/Features/BSim/src/main/java/ghidra/features/bsim/query/client/AbstractSQLFunctionDatabase.java#L1052-L1088","documentation":"When a non-library executable's metadata matches an existing record, `testExecutableDuplication` throws DatabaseNonFatalException(\"<name> is already ingested\"). It is non-fatal because the overall insert can proceed with other executables; this one is a duplicate that should be skipped. BSim tolerates re-ingesting libraries (partial multi-inserts) but flags duplicate main executables.","triggerScenarios":"Inserting a main executable (not a library) whose metadata already matches a stored record. The metadata `compareMetadata` returned 0 (no fatal/non-fatal diff), IDs agree, but it is not a library, so re-ingestion is flagged.","commonSituations":"Re-running an ingest over the same binary set; CI pipelines re-ingesting; idempotency assumption that the DB will silently no-op.","solutions":["Treat DatabaseNonFatalException as an expected, recoverable skip rather than a hard failure.","Deduplicate the input executable list before ingest.","Track which executable MD5s are already stored and exclude them."],"exampleFix":"// before: insert throws non-fatal on a known duplicate\ntry { db.insert(manager); }\n  catch (DatabaseNonFatalException e) { /* unhandled, aborts batch */ }\n// after: classify non-fatal separately and continue\ntry { db.insert(manager); }\ncatch (DatabaseNonFatalException e) {\n    log.info(\"skipping duplicate exe: {}\", e.getMessage());\n}","handlingStrategy":"try-catch","validationCode":"// Detect already-ingested executables by MD5 before attempting insert.\nfor (ExecutableRecord erec : manager.getAllExecutables()) {\n    String md5 = erec.getMd5();\n    if (exeTable.queryMd5ExeMatch(md5) != null && !erec.isLibrary()) {\n        // will be flagged non-fatal; skip or accept the skip downstream\n        log.info(\"already ingested: {}\", erec.getNameExec());\n    }\n}","typeGuard":null,"tryCatchPattern":"try {\n    db.insert(manager);\n} catch (DatabaseNonFatalException e) {\n    // 'X is already ingested' is expected for duplicate main executables -- continue\n    log.info(\"skipping duplicate executable: {}\", e.getMessage());\n} catch (LSHException | SQLException e) {\n    throw e; // real errors propagate\n}","preventionTips":["Catch DatabaseNonFatalException separately from LSHException; treat as a skip.","Deduplicate the input executable list before ingest.","Track ingested MD5s to avoid submitting known duplicates."],"tags":["database","insert","duplicate","non-fatal","bsim"],"backgroundTag":null,"analyzedSha":"d5f144c24d6bc53c9cbf4448c6d11143e7696206","analyzedAt":"2026-08-14T01:00:57.564Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}