risingwavelabs/risingwave · error · SinkError::LanceDb

(dataset guard acquisition error)

Error message

(dataset guard acquisition error)

What it means

After obtaining the shared Lance dataset wrapper in `LanceDbSinkWriter::new`, the writer awaits `dataset_wrapper.get()` to acquire a guard on the underlying dataset (which may require loading/refreshing it from storage). If that acquisition fails — e.g. I/O error reading the manifest, credentials rejected by the object store, or internal lancedb error — it is wrapped in `SinkError::LanceDb` with no added context message, so it surfaces as the raw inner error.

Source

Thrown at src/connector/src/sink/lancedb.rs:376

}

impl LanceDbSinkWriter {
    pub async fn new(config: LanceDbConfig, schema: Schema) -> Result<Self> {
        let arrow_schema = LanceDbConvert
            .rw_schema_to_arrow_schema(&schema)
            .map_err(|e| SinkError::LanceDb(anyhow!(e)))?;

        let conn = config.common.create_connection().await?;
        let table = config.common.open_table(&conn).await?;
        let dataset_wrapper = table.dataset().ok_or_else(|| {
            SinkError::LanceDb(anyhow!(
                "failed to get underlying lance Dataset (table may be remote)"
            ))
        })?;
        let dataset_guard = dataset_wrapper
            .get()
            .await
            .map_err(|e| SinkError::LanceDb(anyhow!(e)))?;
        let data_storage_version = dataset_guard
            .manifest
            .data_storage_format
            .lance_file_version()
            .context("failed to get LanceDB table storage version")
            .map_err(SinkError::LanceDb)?;
        let store_params =
            dataset_guard
                .storage_options_accessor()
                .map(|storage_options_accessor| ObjectStoreParams {
                    storage_options_accessor: Some(storage_options_accessor),
                    ..Default::default()
                });
        drop(dataset_guard);
        let dataset_uri = config.common.dataset_uri(&table).await?;

        Ok(Self {
            config,

View on GitHub (pinned to 6469eb736d)

Solutions

1. Check and fix the storage credentials/options in the LanceDB sink config (access keys, tokens, region) and retest.
2. Verify network reachability from the RisingWave node to the object store endpoint (DNS, firewall, proxy).
3. Inspect the wrapped error message in the SinkError chain to identify whether it is auth, I/O, or manifest corruption, and address that specific cause.
4. Validate the dataset is readable independently (e.g. open it with a small lancedb script or `lance` CLI) to rule out data corruption.

// Pre-flight: verify the dataset is reachable with current credentials
let table = config.common.open_table(&conn).await?;
let wrapper = table.dataset().ok_or_else(|| anyhow!("no native dataset"))?;
wrapper.get().await.map_err(|e| {
    anyhow!("pre-flight dataset load failed, check storage credentials/network: {e}")
})?;

// Acquire the dataset guard with bounded retries for transient storage errors
for attempt in 0..3 {
    match dataset_wrapper.get().await {
        Ok(guard) => break guard,
        Err(e) if attempt < 2 && is_transient(&e) => {
            tokio::time::sleep(Duration::from_millis(250 * (1 << attempt))).await;
        }
        Err(e) => return Err(SinkError::LanceDb(anyhow!(e).context("dataset guard acquisition"))),
    }
}

When it happens

Trigger: `dataset_wrapper.get().await` returns Err while initializing the sink writer: object-store auth failure (bad/missing AWS/GCS/Azure credentials), network partition to storage, corrupted or unreadable _manifest/_versions files, or rate limiting from the object store during dataset load.

Common situations: Missing or expired cloud storage credentials in the sink config; bucket region mismatch; IAM policy lacking s3:GetObject; firewall/proxy blocking storage endpoint; dataset files deleted or corrupted by an external process.

Related errors

• check if version hint exist failed: {}
• Fail to check exist
• failed to read Lance transaction history
• disk error: {msg}
• ObjectStore failed with IO error: {0}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/b73ef1e0255c8f13. Report an issue: GitHub.