risingwavelabs/risingwave · error · SinkError::LanceDb

LanceDB pre-commit sink id {} does not match coordinator sin

Error message

LanceDB pre-commit sink id {} does not match coordinator sink id {}

What it means

Same commit_data path as the epoch check: after deserializing LanceDbPreCommitMetadata, the sink verifies the sink_id recorded at pre_commit matches this coordinator instance's sink_id. A mismatch means the prepared state belongs to a different sink instance — stale metadata from a dropped/recreated sink or two sinks sharing the same state slot.

Source

Thrown at src/connector/src/sink/lancedb.rs:757

    }

    async fn commit_data(&mut self, epoch: u64, commit_metadata: Vec<u8>) -> Result<()> {
        tracing::debug!("Starting LanceDB two-phase commit in epoch {epoch}.");

        if commit_metadata.is_empty() {
            return Ok(());
        }

        let pre_commit_metadata = LanceDbPreCommitMetadata::try_from_bytes(&commit_metadata)?;
        if pre_commit_metadata.epoch != epoch {
            return Err(SinkError::LanceDb(anyhow!(
                "LanceDB pre-commit epoch {} does not match commit epoch {}",
                pre_commit_metadata.epoch,
                epoch
            )));
        }
        if pre_commit_metadata.sink_id != self.sink_id {
            return Err(SinkError::LanceDb(anyhow!(
                "LanceDB pre-commit sink id {} does not match coordinator sink id {}",
                pre_commit_metadata.sink_id,
                self.sink_id
            )));
        }

        let transaction_properties = pre_commit_metadata.transaction_properties();
        self.commit_fragments(
            epoch,
            pre_commit_metadata.fragments,
            Some(transaction_properties),
        )
        .await
    }

    async fn abort(&mut self, epoch: u64, _commit_metadata: Vec<u8>) {
        // Unreferenced files are reclaimed by Lance's old-version/orphan cleanup. This is
        // intentionally not an eager delete: the commit may have succeeded even if its result

View on GitHub (pinned to 6469eb736d)

Solutions

1. Drop and recreate the sink with clean state (remove stale pre-commit metadata or use a fresh sink state/table)
2. Ensure the recreated sink uses a distinct table or clear the old metadata so it does not inherit the previous sink_id
3. Verify no two active sinks are configured to write to the same LanceDB table/state

Example fix

// before
CREATE SINK s2 AS SELECT ... INTO same_lancedb_table ...; // stale pre-commit sink_id mismatch
// after
DROP SINK s2;
-- clear old state / use a fresh table
CREATE SINK s2 AS SELECT ... INTO fresh_lancedb_table ...;

let meta = LanceDbPreCommitMetadata::try_from_bytes(&commit_metadata)?;
if meta.sink_id != self.sink_id {
    // stale metadata from a previous sink instance; treat as invalid prepared state
}

fn is_sink_id_mismatch(err: &SinkError) -> bool {
    matches!(err, SinkError::LanceDb(e) if e.to_string().contains("does not match coordinator sink id"))
}

match commit_result {
    Err(e) if e.to_string().contains("sink id") => {
        // drop stale prepared state and re-prepare at the current epoch
    }
    other => other,
}

When it happens

Trigger: A sink with the same underlying LanceDB storage (same pre-commit bytes) but a different sink_id calls commit_data — typically after dropping and recreating a sink that reuses the same table/state key, or two sinks pointing at the same state.

Common situations: Recreating a LanceDB sink without clearing its old pre-commit metadata; misconfigured sinks sharing the same table URI/state key.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• LanceDB pre-commit epoch {} does not match commit epoch {}
• Lance fragment write task stopped before accepting a record
• LanceDB sink does not support schema change
• (underlying lancedb::Error)
• Schema change is not implemented for two-phase commit coordi

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/af310f81c2b9f2b6. Report an issue: GitHub.