NationalSecurityAgency/ghidra · error · LSHException

Duplicate md5 hash, different metadata

Error message

Duplicate md5 hash, different metadata

What it means

Thrown by DescriptionManager.newExecutableRecord when an executable with the same md5 hash already exists in the container but has different metadata (name, compiler, architecture, date, repo, or path). MD5 is the identity key for executables; conflicting metadata for the same md5 indicates an inconsistency the manager refuses to merge silently.

Source

Thrown at Ghidra/Features/BSim/src/main/java/ghidra/features/bsim/query/description/DescriptionManager.java:232

	 * @param md5 is the MD5 hash of the executable
	 * @param enm is the name of the executable
	 * @param cnm is the name of the compiler used to build the executable
	 * @param arc is the architecture of the executable
	 * @param dt is the date (of ingest)
	 * @param repo is the repository containing the executable
	 * @param path is the path (within the repo) to the executable
	 * @param id is the database (row) is associated with the executable (may be null)
	 * @return the new ExecutableRecord object
	 * @throws LSHException if attributes are invalid, or the executable 
	 *     already exists with different metadata
	 */
	public ExecutableRecord newExecutableRecord(String md5, String enm, String cnm, String arc,
			Date dt, String repo, String path, RowKey id) throws LSHException {
		ExecutableRecord newexe = new ExecutableRecord(md5, enm, cnm, arc, dt, id, repo, path);
		if (!exerec.add(newexe)) {
			ExecutableRecord oldexe = exerec.floor(newexe);
			if (oldexe.compareMetadata(newexe) != 0) {
				throw new LSHException("Duplicate md5 hash, different metadata");
			}
			if ((oldexe.getRowId() != null) && (id != null) && (!oldexe.getRowId().equals(id))) {
				throw new LSHException("Overwriting existing executable id");
			}
			newexe = oldexe;
		}
		return newexe;
	}

	/**
	 * Create a new "library" executable in the container.
	 * Functions in this container (will) have no body or address
	 * @param enm is the name of the library
	 * @param arc is the architecture of the library
	 * @param id is the database id associated with the library (may be null)
	 * @return the new ExecutableRecord object
	 * @throws LSHException if attributes are invalid or the
	 *   library already exists with different metadata

View on GitHub (pinned to d5f144c24d)

Solutions

1. Reconcile the metadata before re-adding: use the existing ExecutableRecord's metadata consistently across all ingest of the same md5.
2. If the new metadata is authoritative, remove the old ExecutableRecord first (and its functions) then re-add.
3. Detect the duplicate md5 upstream and reuse the existing record instead of calling newExecutableRecord again.
4. Audit the ingest pipeline to ensure deterministic metadata (same compiler/arch/path) for identical binaries.

Example fix

// before
descMgr.newExecutableRecord(md5, name2, compiler2, arch2, date2, repo2, path2, id);
// throws if md5 exists with different metadata

// after
ExecutableRecord existing = descMgr.getExecutableRecord(md5);
if (existing != null) {
    if (existing.compareMetadata(newRecord) != 0) {
        // resolve conflict or reuse existing
        return existing;
    }
    return existing;
}
return descMgr.newExecutableRecord(md5, name, compiler, arch, date, repo, path, id);

// Detect duplicate md5 and reconcile metadata before adding.
ExecutableRecord existing = descMgr.findExecutableByMd5(md5);
if (existing != null) {
    if (existing.compareMetadata(proposedRecord) != 0) {
        // reconcile or fail fast with context
        throw new IllegalStateException("Metadata conflict for md5 " + md5);
    }
    return existing; // reuse
}
return descMgr.newExecutableRecord(md5, name, comp, arch, date, repo, path, id);

try {
    return descMgr.newExecutableRecord(md5, name, comp, arch, date, repo, path, id);
} catch (LSHException e) {
    if (e.getMessage().contains("different metadata")) {
        // log the conflicting fields and surface a clear error
        throw new RuntimeException("Metadata conflict for md5 " + md5 + "; reconcile source data", e);
    }
    throw e;
}

When it happens

Trigger: Calling newExecutableRecord with an md5 that already exists in the DescriptionManager's exerec set, where oldexe.compareMetadata(newexe) returns non-zero. Happens when loading/describing the same binary twice with differing metadata fields — e.g., same md5 but different declared compiler or architecture, or different repository/path.

Common situations: Loading the same executable from two sources that annotate different metadata (compiler mismatch, name normalization differences). Inconsistent metadata in a BSim ingest pipeline. Re-describing a binary after partial changes. Merging two DescriptionManagers with conflicting metadata for shared md5s.

Related errors

• Overwriting existing executable id
• Bad category tag
• Executable
• Expecting privilege option (admin or user)
• host required

AI-assisted analysis of NationalSecurityAgency/ghidra@d5f144c24d (2026-08-14). Data as JSON: /api/errors/2d02e73a59131f05. Report an issue: GitHub.