risingwavelabs/risingwave · error · SinkError

Can't create deltalake sink write result from empty data!

Error message

Can't create deltalake sink write result from empty data!

What it means

DeltaLakeSinkMetadata is converted into a DeltaLakeWriteResult only when its `metadata` field contains a Some(Serialized(v)) payload; this error is thrown when that payload is absent. It means the sink metadata committed to the meta service carries no serialized delta-lake `Add` actions, so no write result can be reconstructed for commit.

Source

Thrown at src/connector/src/sink/deltalake.rs:782

    type Error = SinkError;

    fn try_from(value: &'a DeltaLakeWriteResult) -> std::result::Result<Self, Self::Error> {
        let metadata =
            serde_json::to_vec(&value.adds).context("cannot serialize deltalake sink metadata")?;
        Ok(SinkMetadata {
            metadata: Some(Serialized(SerializedMetadata { metadata })),
        })
    }
}

impl DeltaLakeWriteResult {
    fn try_from(value: &SinkMetadata) -> Result<Self> {
        if let Some(Serialized(v)) = &value.metadata {
            let adds = serde_json::from_slice::<Vec<Add>>(&v.metadata)
                .context("Can't deserialize deltalake sink metadata")?;
            Ok(DeltaLakeWriteResult { adds })
        } else {
            bail!("Can't create deltalake sink write result from empty data!")
        }
    }
}

impl From<::deltalake::DeltaTableError> for SinkError {
    fn from(value: ::deltalake::DeltaTableError) -> Self {
        SinkError::DeltaLake(anyhow!(value))
    }
}

#[cfg(all(test, not(madsim)))]
mod tests {
    use deltalake::kernel::DataType as SchemaDataType;
    use deltalake::operations::create::CreateBuilder;
    use maplit::btreemap;
    use risingwave_common::array::{Array, I32Array, Op, StreamChunk, Utf8Array};
    use risingwave_common::catalog::{Field, Schema};
    use risingwave_common::types::DataType;

View on GitHub (pinned to 6469eb736d)

Solutions

1. Ensure the DeltaLake sink writer's `serialize`/`begin_epoch` path always stores `Metadata::Serialized` with the JSON-encoded Vec<Add> before commit.
2. Check that no code path replaces or drops the metadata payload (e.g. an empty flush overwriting it with None).
3. If this occurs on a sink that wrote no data, treat empty commits specially and skip the try_from conversion.
4. Upgrade RisingWave; this is an internal invariant and recent versions may handle empty commits gracefully.

Example fix

// before
let result = DeltaLakeWriteResult::try_from(&sink_metadata)?;
// after
if matches!(&sink_metadata.metadata, Some(Serialized(_))) {
    let result = DeltaLakeWriteResult::try_from(&sink_metadata)?;
} else {
    // skip empty commit
    return Ok(());
}

fn has_serialized_metadata(m: &SinkMetadata) -> bool {
    matches!(m.metadata, Some(Serialized(_)))
}

fn as_serialized(m: &SinkMetadata) -> Option<&Vec<u8>> {
    if let Some(Serialized(v)) = &m.metadata { Some(&v.metadata) } else { None }
}

match DeltaLakeWriteResult::try_from(&meta) {
    Ok(res) => commit(res),
    Err(e) => log::warn!("empty/invalid delta sink metadata: {e}"),
}

When it happens

Trigger: Calling `DeltaLakeWriteResult::try_from(&SinkMetadata)` with a SinkMetadata whose `metadata` field is None (or not the Serialized variant), e.g. after a sink commit that never recorded serialized adds.

Common situations: Internal RisingWave sink-commit flows where the DeltaLake sink writer failed to serialize its added files before flush/commit; restoring or replaying metadata that predates serialization; bugs in the sink's `serialize` implementation.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• Row sequential scan should not have input executor!
• Source should not have input executor!
• Row sequential scan should not have input executor!
• ValuesExecutor should have no child!
• VectorIndexNearest should have an input executor!

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/49fbd0e41cd5875a. Report an issue: GitHub.