databendlabs/databend · error

required key not found

Error message

required key not found: {key}; reason: {reason}

What it means

mark_required_key marks a key whose presence in the dump is considered mandatory for the filter decision (root classification or a required dependency). It looks the key up in key_to_state; if the key is not indexed at all, the tool cannot even record a decision for it, which breaks the 'every line marked' contract, so it bails with the key and the reason that led to the lookup. (mark_optional_key tolerates missing keys; the required variant does not.)

Solutions

  1. Check the key in the error message: confirm whether it actually exists in the snapshot dump file and under what exact spelling.
  2. Verify the dump was produced by the same metasrv version as the tool; regenerate the dump if versions differ.
  3. If the key exists but was not indexed, fix the snapshot parser's key extraction so that key shape is registered in key_to_state.
  4. If the key legitimately may not exist (dangling reference), switch that call site from mark_required_key to mark_optional_key, which returns Ok(false) for missing keys.

Example fix

// before
self.mark_required_key(&child.key, decision, reason)?;
// after (child is known to be possibly-absent)
self.mark_optional_key(&child.key, decision, reason)?;
Defensive patterns

Strategy: validation

Validate before calling

fn ensure_required_key_present(idx: &HashMap<String, usize>, key: &str, reason: &str) -> anyhow::Result<()> {
    anyhow::ensure!(
        idx.contains_key(key),
        "required key {key} missing from dump (context: {reason})"
    );
    Ok(())
}

Type guard

fn required_key(idx: &HashMap<String, usize>, key: &str) -> anyhow::Result<usize> {
    idx.get(key).copied()
        .ok_or_else(|| anyhow::anyhow!("required key not found: {key}"))
}

Try / catch

match filter.mark_required_key(&key, Decision::Keep, "root record") {
    Ok(_) => {}
    Err(e) if e.to_string().contains("required key not found") => {
        // dangling reference in the dump; treat as absent and continue
        eprintln!("dangling reference: {e}");
    }
    Err(e) => return Err(e),
}

Prevention

When it happens

Trigger: filter_tenant::mark_all calls mark_required_key for every key from classify_root, and drain_mark_queue calls it for required dependency children returned by dependency_keys(key). The bail fires when such a key has no entry in key_to_state — i.e. dependency_keys or classify_root produced a key that was never parsed into an indexed state line.

Common situations: Snapshot dumps with dangling references (a record references a child key that is absent or was already consumed), version-skewed dumps where referenced keys changed format, or bugs in dependency_keys generating malformed key strings.

Understand the failure class

Background: Record Not Found Errors: "not found", RecordNotFound, and "was not found" — what they mean and how to fix them — this error's family across 28 libraries.

Related errors

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/b1dbd6374720a4f2. Report an issue: GitHub.

Appendix: source

Thrown at src/meta/process/src/filter_tenant.rs:512

        if !unmarked.is_empty() {
            anyhow::bail!(
                "tenant dump filter did not mark all state machine entries; examples: {}",
                unmarked.join("; ")
            );
        }

        Ok(())
    }

    fn mark_required_key(
        &mut self,
        key: &str,
        decision: Decision,
        reason: impl Into<String>,
    ) -> anyhow::Result<bool> {
        let reason = reason.into();
        let Some(index) = self.key_to_state.get(key).copied() else {
            anyhow::bail!("required key not found: {key}; reason: {reason}");
        };

        self.mark_existing_key(index, key, decision, reason)
    }

    fn mark_optional_key(
        &mut self,
        key: &str,
        decision: Decision,
        reason: impl Into<String>,
    ) -> anyhow::Result<bool> {
        let Some(index) = self.key_to_state.get(key).copied() else {
            return Ok(false);
        };

        self.mark_existing_key(index, key, decision, reason.into())
    }

View on GitHub (pinned to 288d84d76e)