dgraph-io/badger · error

Duplicate file found. Please delete one.

Error message

Duplicate file found. Please delete one.

What it means

Thrown by valueLog.populateFilesMap (value.go:509) when two files in the value-log directory resolve to the same uint32 file ID, i.e. duplicate '<id>.vlog' names exist. Since filesMap is keyed by fid, badger cannot disambiguate them and Open() fails.

Source

Thrown at value.go:509

	vlog.filesMap = make(map[uint32]*logFile)

	files, err := os.ReadDir(vlog.dirPath)
	if err != nil {
		return errFile(err, vlog.dirPath, "Unable to open log dir.")
	}

	found := make(map[uint64]struct{})
	for _, file := range files {
		if !strings.HasSuffix(file.Name(), ".vlog") {
			continue
		}
		fsz := len(file.Name())
		fid, err := strconv.ParseUint(file.Name()[:fsz-5], 10, 32)
		if err != nil {
			return errFile(err, file.Name(), "Unable to parse log id.")
		}
		if _, ok := found[fid]; ok {
			return errFile(err, file.Name(), "Duplicate file found. Please delete one.")
		}
		found[fid] = struct{}{}

		lf := &logFile{
			fid:      uint32(fid),
			path:     vlog.fpath(uint32(fid)),
			registry: vlog.db.registry,
		}
		vlog.filesMap[uint32(fid)] = lf
		if vlog.maxFid < uint32(fid) {
			vlog.maxFid = uint32(fid)
		}
	}
	return nil
}

func (vlog *valueLog) createVlogFile() (*logFile, error) {
	fid := vlog.maxFid + 1

View on GitHub (pinned to 2a001d466f)

Solutions

1. Identify the duplicate ids (ls the dir, strip .vlog, find repeats) and delete the older/smaller one — badger explicitly instructs 'Please delete one'.
2. Compare file sizes/mtimes and keep the newest complete file; verify with a backup if unsure.
3. Never merge two badger data directories; restore one coherent snapshot instead.
4. On case-insensitive filesystems, ensure restores don't create same-id names differing only in case.

Example fix

// before
$ ls vlogdir
5.vlog  05.vlog   # both parse to fid 5
// after
$ rm vlogdir/05.vlog   # or move it away; keep the correct one
$ ls vlogdir
5.vlog

// detect duplicate fids before opening badger
func findDuplicateVlogIDs(dir string) (map[uint64]int, error) {
    dups := map[uint64]int{}
    files, err := os.ReadDir(dir)
    if err != nil { return nil, err }
    seen := map[uint64]struct{}{}
    for _, f := range files {
        n := f.Name()
        if !strings.HasSuffix(n, ".vlog") { continue }
        id, err := strconv.ParseUint(n[:len(n)-5], 10, 32)
        if err != nil { continue }
        if _, ok := seen[id]; ok { dups[id]++ }
        seen[id] = struct{}{}
    }
    return dups, nil
}

func fidOf(name string) (uint64, bool) {
    if !strings.HasSuffix(name, ".vlog") { return 0, false }
    id, err := strconv.ParseUint(strings.TrimSuffix(name, ".vlog"), 10, 32)
    return id, err == nil
}

dups, _ := findDuplicateVlogIDs(vlogDir)
if len(dups) > 0 {
    for id := range dups {
        keep, drop := pickNewest(filepath.Join(vlogDir, fmt.Sprintf("%d.vlog", id)))
        _ = keep
        os.Remove(drop) // or move to quarantine
    }
}
db, err := badger.Open(opts)

When it happens

Trigger: Two files with identical numeric names but different casing/paths that collapse to the same fid (e.g. '5.vlog' and '05.vlog' both parse to 5), or a genuinely duplicated file restored twice under names that hash to the same id.

Common situations: Manual restores where a copy was made as '5 (copy).vlog' renamed back to '5.vlog' alongside the original on a case-insensitive filesystem, backup tooling writing duplicate names, merging two data directories.

Related errors

• Empty value: %+v
• Unable to parse log id.
• Unable to find fid: %d
• ErrNoRewrite
• Cannot use NewTransactionAt with managedDB=false. Use NewTra

AI-assisted analysis of dgraph-io/badger@2a001d466f (2026-09-05). Data as JSON: /api/errors/27f95dc0dbb95923. Report an issue: GitHub.