risingwavelabs/risingwave · error · SinkError::LanceDb

failed to get underlying lance Dataset (table may be remote)

Error message

failed to get underlying lance Dataset (table may be remote)

What it means

The LanceDB sink resolves the underlying Lance dataset URI from the opened table to run compaction/optimization. Only locally-opened native tables expose a Dataset; remote tables (opened via a remote LanceDB server) return None from table.dataset(), so the sink cannot proceed and errors.

Source

Thrown at src/connector/src/sink/lancedb.rs:110

            .execute()
            .await
            .context("failed to connect to LanceDB")
            .map_err(SinkError::LanceDb)?;
        Ok(conn)
    }

    async fn open_table(&self, conn: &LanceDbConnection) -> Result<LanceDbTable> {
        conn.open_table(&self.table)
            .execute()
            .await
            .context("failed to open LanceDB table")
            .map_err(SinkError::LanceDb)
    }

    /// Get the Lance dataset URI from the opened native `LanceDB` table.
    async fn dataset_uri(&self, table: &LanceDbTable) -> Result<String> {
        let dataset_wrapper = table.dataset().ok_or_else(|| {
            SinkError::LanceDb(anyhow!(
                "failed to get underlying lance Dataset (table may be remote)"
            ))
        })?;
        let dataset_guard = dataset_wrapper
            .get()
            .await
            .map_err(|e| SinkError::LanceDb(anyhow!(e)))?;
        Ok(dataset_guard.uri().to_owned())
    }
}

#[serde_as]
#[derive(Clone, Debug, Deserialize, WithOptions)]
pub struct LanceDbConfig {
    #[serde(flatten)]
    pub common: LanceDbCommon,

    pub r#type: String,

View on GitHub (pinned to 6469eb736d)

Solutions

1. Use a local/native LanceDB table path for the sink (not a remote server URI)
2. Skip dataset optimization for remote tables (they are managed by the remote server)
3. If remote support is needed, use the LanceDB server-side optimization APIs instead of operating on the local Dataset

// before using the sink against remote tables, check connectivity mode
if table_uri.starts_with("db://") {
  return Err("local Lance dataset optimization not supported for remote tables".into());
}

fn has_local_dataset(t: &LanceDbTable) -> bool {
  t.dataset().is_some()
}
// call site: if !has_local_dataset(&table) { skip optimization }

match sink.dataset_uri(&table).await {
  Err(e) if e.to_string().contains("table may be remote") => {
    warn!("skipping dataset optimization for remote LanceDB table");
    Ok(())
  }
  r => r,
}

When it happens

Trigger: `dataset_uri(table)` called with a table opened against a remote LanceDB instance, where `table.dataset()` returns None.

Common situations: Connecting to LanceDB via a remote URI (db://... server connection) while the sink expects a local/native table path; trying to run table optimization on remote tables.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• LanceDB sink does not support schema change
• support negative scale for arrow decimal
• need url to be a file or a http based, got {url}
• invalid credentials_url scheme '{}', only file://, s3://, an
• Snowflake catalog only supports iceberg sources

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/9453786cc4e21753. Report an issue: GitHub.