risingwavelabs/risingwave · error · SinkError

{e}

Error message

{e}

What it means

During sink validation, the connector runs `op.list(&self.path)` against the object store (via OpenDAL) to prove the configured path/location is reachable and credentials work. Any error returned by the storage backend (auth failure, missing bucket, bad endpoint) is wrapped with `anyhow!` into a generic SinkError and surfaced to the user as-is.

Source

Thrown at src/connector/src/sink/file_sink/opendal_sink.rs:171

            return Err(SinkError::Config(anyhow!(
                "File sink only supports append-only mode at present. \
                    Please change the query to append-only, and specify it \
                    explicitly after the `FORMAT ... ENCODE ...` statement. \
                    For example, `FORMAT xxx ENCODE xxx(force_append_only='true')`"
            )));
        }

        if self.format_desc.encode != SinkEncode::Parquet
            && self.format_desc.encode != SinkEncode::Json
        {
            return Err(SinkError::Config(anyhow!(
                "File sink only supports `PARQUET` and `JSON` encode at present."
            )));
        }

        match self.op.list(&self.path).await {
            Ok(_) => Ok(()),
            Err(e) => Err(anyhow!(e).into()),
        }
    }

    async fn new_log_sinker(
        &self,
        writer_param: crate::sink::SinkWriterParam,
    ) -> Result<Self::LogSinker> {
        let writer = OpenDalSinkWriter::new(
            self.op.clone(),
            &self.path,
            self.schema.clone(),
            writer_param.executor_id,
            &self.format_desc,
            self.engine_type.clone(),
            self.batching_strategy.clone(),
        )?;
        Ok(BatchingLogSinker::new(writer))
    }

View on GitHub (pinned to 6469eb736d)

Solutions

1. Verify the `path`/`location` option points to an existing, accessible prefix in the configured backend
2. Check the backend connection options (bucket, endpoint, region, root) in the sink WITH clause
3. Confirm credentials (env vars or sink options) are valid and have ListObject permission on the path
4. Test reachability of the storage endpoint from the RisingWave node (network/firewall/DNS)

Example fix

// before
WITH (connector = 'file_s3', path = 's3://my-buket/output/')
// after
WITH (connector = 'file_s3', path = 's3://my-bucket/output/', aws.region = 'us-east-1')

// Before CREATE SINK, verify the object store path is listable with the same credentials
import boto3
s3 = boto3.client('s3')
resp = s3.list_objects_v2(Bucket='my-bucket', Prefix='output/', MaxKeys=1)
assert 'Key' in resp or resp['KeyCount'] >= 0  # raises if bucket/creds/network are bad

When it happens

Trigger: Calling `CREATE SINK ... WITH (connector='file_s3'|..., path=...)` where the OpenDAL operator cannot list the target path: wrong bucket/container name, missing or invalid credentials, unreachable endpoint, or nonexistent prefix on some backends.

Common situations: Typo'd S3 bucket or path, AWS credentials absent/wrong region, HDFS/WebHDFS NameNode down, local FS path without permission, or firewalled network blocking the object store.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• `commit_checkpoint_interval` must be greater than 0
• Failed to spill out to disk
• No auth method specified for Vault backend
• Field '{}' not found in secret
• unrecognized configs: {:?}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/8e468879a4f2054e. Report an issue: GitHub.