risingwavelabs/risingwave · error · SinkError::Redshift

Failed to close manifest writer: {}

Error message

Failed to close manifest writer: {}

What it means

After writing the S3 manifest JSON entries via an OpenDAL writer, `writer.close()` failed; the sink wraps the error as a Redshift sink error. A failed close typically means the final bytes were not flushed/committed to S3, so the manifest may be missing or truncated and subsequent Redshift COPY will fail.

Source

Thrown at src/connector/src/sink/snowflake_redshift/redshift.rs:527

    async fn write_manifest_to_s3(
        s3_inner: &S3Common,
        paths: Vec<String>,
        table: &str,
    ) -> Result<String> {
        let manifest_entries: Vec<_> = paths
            .into_iter()
            .map(|path| json!({ "url": path, "mandatory": true }))
            .collect();
        let s3_operator = FileSink::<S3Sink>::new_s3_sink(s3_inner)?;
        let (mut writer, manifest_path) =
            build_opendal_writer_path(s3_inner, &s3_operator, Some("manifest"), table).await?;
        let manifest_json = json!({ "entries": manifest_entries });
        let mut chunk_buf = BytesMut::new();
        writeln!(chunk_buf, "{}", manifest_json).unwrap();
        writer.write(chunk_buf.freeze()).await?;
        writer.close().await.map_err(|e| {
            SinkError::Redshift(anyhow!(
                "Failed to close manifest writer: {}",
                e.to_report_string()
            ))
        })?;
        Ok(manifest_path)
    }

    pub async fn copy_into_from_s3_to_redshift(
        client: &JdbcJniClient,
        config: &RedShiftConfig,
        s3_inner: &S3Common,
        is_append_only: bool,
        manifest: &str,
    ) -> Result<()> {
        let all_path = format!("s3://{}/{}", s3_inner.bucket_name, manifest);

        let (table, schema_name) = if is_append_only {
            (&config.table, config.schema.as_deref())

View on GitHub (pinned to 6469eb736d)

Solutions

1. Check sink S3 config (bucket, region, endpoint, path) and AWS credentials/IAM permissions
2. Retry the sink; the manifest write is part of periodic commit and will be retried on the next epoch
3. Verify connectivity to the S3 endpoint from the compute node (e.g., MinIO address reachable)

// pre-check S3 connectivity/credentials before starting the sink
s3_client.head_bucket().bucket(bucket).send().await?;

match writer.close().await {
    Ok(()) => (),
    Err(e) => log::error!("manifest close failed: {}", e.to_report_string()), // retry next epoch
}

When it happens

Trigger: Calling `write_manifest_to_s3` when S3 is unreachable, credentials are invalid, the bucket/prefix does not exist, or the connection drops during the writer's final flush.

Common situations: Expired AWS credentials or IAM policy missing s3:PutObject; wrong `s3.region`/endpoint in sink config; network partitions between the RisingWave node and S3 (or S3-compatible storage like MinIO).

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

• S3 configuration is required for redshift s3 sink
• s3 error: {inner}
• failed to get row stream from mysql query
• Failed to spill out to disk
• Vault API returned error status: {} - {}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/37c41a9a0b63f3df. Report an issue: GitHub.