conflicting tenant filter marks for line

Error message

conflicting tenant filter marks for line {} ({}): existing {} because {}; new {} because {}

What it means

mark_line_by_index refuses to overwrite an existing mark that disagrees with the new decision. When a second pass (root classification, dependency marking, or expiry marking) tries to Drop a line already marked Keep — or vice versa — the tool's keep/drop logic is self-contradictory, so it bails with both the existing and new decision plus their reasons. This is a deliberate consistency check: a conflicting mark would mean the filtered dump could silently remove data another kept record depends on.

Solutions

Read the existing vs new decisions and reasons in the message to identify which two code paths disagree about the line.
Inspect the key's references in the dump: if it is genuinely shared, ensure dependency marking propagates the winning (Keep) decision first, or drop the whole dependent group together.
Fix classify_root/dependency logic so the decision for a key is deterministic and order-independent (e.g. Keep wins over Drop for shared required records).
Regenerate the dump with a matching metasrv version; version-skewed dumps often contain references that look contradictory to the filter.
As a diagnostic, log the full mark_queue order for the offending key before the bail to find the second caller.

Example fix

// before: blindly marks, conflicting decisions abort
self.mark_required_key(&child.key, decision, reason)?;
// after: keep wins for shared required records
let existing = self.state_lines[self.key_to_state[&child.key]].mark.as_ref();
if existing.map(|m| m.decision) == Some(Decision::Keep) && decision == Decision::Drop {
    return Ok(false);
}
self.mark_required_key(&child.key, decision, reason)?;

Defensive patterns

Strategy: validation

Validate before calling

fn decision_for_key(filter: &TenantFilter, key: &str) -> Option<Decision> {
    filter.mark_of(key).map(|m| m.decision)
}
// call before re-marking:
if decision_for_key(&filter, &key) == Some(Decision::Keep) {
    eprintln!("key {key} already kept; skipping drop");
    return Ok(());
}

Type guard

fn is_conflict(existing: &Option<Mark>, new: Decision) -> bool {
    existing.as_ref().is_some_and(|m| m.decision != new)
}

Try / catch

match filter.mark_line_by_index(idx, decision, reason) {
    Ok(marked) => { if marked { propagate_to_dependencies(key); } }
    Err(e) if e.to_string().contains("conflicting tenant filter marks") => {
        // decide deterministically: Keep wins for shared records
        eprintln!("conflict on {key}; resolving as Keep");
        filter.mark_line_by_index(idx, Decision::Keep, "conflict resolved: keep wins")?;
    }
    Err(e) => return Err(e),
}

Prevention

Make key decisions deterministic and order-independent: define explicitly whether Keep or Drop wins for shared records.
Mark Keep decisions before Drop passes so shared dependencies are protected first.
Include the original reason string in every mark to make conflicts diagnosable from the error alone.
Test dependency graphs containing shared records (one key referenced by both kept and dropped roots).
Never hand-edit dumps; conflicting references usually come from corrupted or version-skipped exports.

When it happens

Trigger: Called from mark_all (system entries / root classification), mark_existing_key (via mark_required_key / mark_optional_key from drain_mark_queue), or mark_expires_for_key. The bail fires whenever state_lines[index].mark is Some with a different Decision than the incoming one, e.g. a dependency of a dropped key was already marked Keep by another root, or an __fd_*_expires key disagrees with its base key's decision.

Common situations: Snapshot dumps where cross-tenant or dangling references make one record both a required child of a kept key and a dependent of a dropped key; bugs in classify_root returning different decisions for keys that map to the same line; hand-edited dumps; changes to dependency_keys that create keep/drop cycles.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/bad67007d38008ca. Report an issue: GitHub.

Appendix: source

Thrown at src/meta/process/src/filter_tenant.rs:558

            self.mark_expires_for_key(key, decision, format!("expire of {key}"))?;
        }

        Ok(marked)
    }

    fn mark_line_by_index(
        &mut self,
        index: usize,
        decision: Decision,
        reason: impl Into<String>,
    ) -> anyhow::Result<bool> {
        let reason = reason.into();
        let state_line = &mut self.state_lines[index];

        match &state_line.mark {
            Some(mark) if mark.decision == decision => return Ok(false),
            Some(mark) => {
                anyhow::bail!(
                    "conflicting tenant filter marks for line {} ({}): existing {} because {}; new {} because {}",
                    state_line.line_no,
                    state_line.display_key(),
                    mark.decision.as_str(),
                    mark.reason,
                    decision.as_str(),
                    reason
                );
            }
            None => {}
        }

        state_line.mark = Some(Mark { decision, reason });

        if let StateKind::GenericKV { key, .. } = &state_line.kind {
            self.mark_queue.push_back((key.clone(), decision));
        }

View on GitHub (pinned to 288d84d76e)