databendlabs/databend · error

duplicated GenericKV key found at lines

Error message

duplicated GenericKV key found at lines {} and {}: {}

What it means

While building lookup indexes over the parsed meta dump, `build_indexes` inserts every GenericKV key into `key_to_state`. If the same GenericKV key appears on two different lines, the second insert finds an existing entry and bails, reporting both line numbers and the key. The tool assumes the dump contains no duplicated GenericKV keys.

Solutions

1. Deduplicate the dump file, keeping the last occurrence of each GenericKV key before running filter_tenant.
2. Check how the dump was produced/exported and regenerate it as a single consistent snapshot.
3. If duplicates are expected in your flow, relax build_indexes to keep the latest line instead of bailing.

Example fix

// before
if let Some(prev) = self.key_to_state.insert(key.clone(), state_index) {
    anyhow::bail!("duplicated GenericKV key found at lines {} and {}: {}", ...);
}
// after
// keep the latest occurrence instead of failing
self.key_to_state.insert(key.clone(), state_index);

# pre-flight: find duplicated GenericKV keys in the dump before filtering
grep -oE 'GenericKV \{ key: [^,]+' dump.log | sort | uniq -d

match tool.build_indexes() {
    Err(e) if e.to_string().contains("duplicated GenericKV key") => {
        eprintln!("deduplicate the dump (keep last occurrence) and rerun");
        dedup_keep_last("dump.log", "dump.dedup.log")?;
        tool = Tool::load("dump.dedup.log")?;
        tool.build_indexes()?
    }
    other => other?,
}

When it happens

Trigger: Filtering a meta dump whose file contains the same GenericKV key twice — e.g. concatenating two snapshots, replaying lines twice, or a dump produced from an inconsistent state.

Common situations: Manually merged or appended dump files; re-running an export that appends instead of overwrites; snapshots taken across a Raft state-machine rebuild that legitimately contain duplicate keys.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• unsupported state machine entry at line
• key is missing from state index
• The header tree can only contain DataHeader
• indexed key is not a GenericKV state entry at line
• tenant dump filter did not mark all state machine entries…

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/d30a17df10e7fa20. Report an issue: GitHub.

Appendix: source

Thrown at src/meta/process/src/filter_tenant.rs:258

                kind,
                mark: None,
            });
            dump.report.state_machine_lines += 1;
        }

        Ok(dump)
    }

    fn build_indexes(&mut self) -> anyhow::Result<()> {
        self.key_to_state.clear();
        self.expires_by_key.clear();
        self.index_ids_by_table_id.clear();

        for (state_index, state_line) in self.state_lines.iter().enumerate() {
            match &state_line.kind {
                StateKind::GenericKV { key, .. } => {
                    if let Some(prev) = self.key_to_state.insert(key.clone(), state_index) {
                        anyhow::bail!(
                            "duplicated GenericKV key found at lines {} and {}: {}",
                            self.state_lines[prev].line_no,
                            state_line.line_no,
                            key
                        );
                    }
                }
                StateKind::Expire { key } => {
                    self.expires_by_key
                        .entry(key.clone())
                        .or_default()
                        .push(state_index);
                }
                StateKind::System { .. } => {}
            }
        }

        for state_line in &self.state_lines {

View on GitHub (pinned to 288d84d76e)