What causes " is already ingested" in NationalSecurityAgency/ghidra?

Catch DatabaseNonFatalException at the call site and treat it as informational — skip the executable and continue the batch.

NationalSecurityAgency/ghidra · warning · DatabaseNonFatalException

is already ingested

Error message

 is already ingested

What it means

Thrown as a DatabaseNonFatalException in insertExe when an executable already exists in the database with identical metadata. insertExecutableRecord uses op_type=create, which fails on the first insert; the existing record is then retrieved via queryMd5ExeMatch and compareMetadata returns 0 (exact match). This is expected, informational behavior signaling a no-op re-ingestion, not a genuine error.

Source

Thrown at Ghidra/Features/BSim/src/main/java/ghidra/features/bsim/query/elastic/ElasticDatabase.java:1637

			throws ElasticException, LSHException, DatabaseNonFatalException {
		RowKeyElastic eKey = updateKey(manager, exeRecord);
		String exeId = eKey.generateExeIdString();

		if (!insertExecutableRecord(exeRecord, exeId)) {	// Try to insert the executable
			JsonObject exeObj = queryMd5ExeMatch(exeRecord.getMd5());
			if (exeObj != null) {		// Try to retrieve the previous version
				ExecutableRecord oldrec = makeExecutableRecordTemp(exeObj);
				int cmp = oldrec.compareMetadata(exeRecord);
				if (cmp != 0) {
					String fatalerror =
						FunctionDatabase.constructFatalError(cmp, exeRecord, oldrec);
					if (fatalerror != null) {
						throw new LSHException(fatalerror);
					}
					throw new DatabaseNonFatalException(
						FunctionDatabase.constructNonfatalError(cmp, exeRecord, oldrec));
				}
				throw new DatabaseNonFatalException(
					exeRecord.getNameExec() + " is already ingested");
			}
			return false;		// Indicate this executable already inserted
		}
		int newIds = 0;
		long baseId = 0;
		Iterator<FunctionDescription> iter = manager.listFunctions(exeRecord);
		while (iter.hasNext()) {			// Count the functions to insert for this executable
			iter.next();
			newIds += 1;
		}
		baseId = allocateFunctionIndexSpace(newIds);		// Allocated the ids we will need
		iter = manager.listFunctions(exeRecord);
		while (iter.hasNext()) {
			manager.setFunctionDescriptionId(iter.next(), new RowKeyElastic(baseId));	// Set the (allocated) ids
			baseId += 1;
		}
		// Collect/dedup vectors and update SignatureRecords with vector ids, before writing FunctionDescriptions

View on GitHub (pinned to d5f144c24d)

Solutions

Catch DatabaseNonFatalException at the call site and treat it as informational — skip the executable and continue the batch.
Pre-check for an existing executable via queryMd5ExeMatch before attempting ingestion to avoid the exception entirely.
Log the skipped executable name and proceed with the next item in the batch.

Example fix

// before
database.query(query);  // DatabaseNonFatalException propagates uncaught

// after
try {
    database.query(query);
} catch (DatabaseNonFatalException e) {
    Msg.info(this, "Skipping already-ingested executable: " + e.getMessage());
}

Defensive patterns

Strategy: try-catch

Validate before calling

// Pre-check before ingestion
JsonObject existing = queryMd5ExeMatch(exeRecord.getMd5());
if (existing != null) {
    // Executable already exists; skip ingestion
    Msg.info(this, "Executable already ingested, skipping: " + exeRecord.getNameExec());
    return;
}

Try / catch

try {
    database.query(query);
} catch (DatabaseNonFatalException e) {
    if (e.getMessage().endsWith(" is already ingested")) {
        // Expected — skip this executable in the batch
        Msg.info(this, "Already ingested: " + e.getMessage());
        continue;
    }
    throw e;
}

Prevention

In batch ingestion scripts, catch DatabaseNonFatalException and continue to the next executable.
Pre-check with queryMd5ExeMatch before attempting ingestion to avoid the exception overhead.
Track ingested md5 values locally to skip duplicates before hitting the server.

When it happens

Trigger: Calling insertExe (via an ingest command) for an executable whose md5 already exists. The version_conflict from op_type=create is caught, the existing record is fetched, and compareMetadata returns cmp==0 (metadata is identical), so the non-fatal exception is thrown.

Common situations: Re-running an ingest script that processes the same binaries; batch ingestion pipeline that includes previously-seen executables; re-ingesting after a partial failure where the executable record was already written.

Related errors

AI-assisted analysis of NationalSecurityAgency/ghidra@d5f144c24d (2026-08-14). Data as JSON: /api/errors/f91c29f4877663db. Report an issue: GitHub.