risingwavelabs/risingwave · error · SinkError::DynamoDb

failed to write items to DynamoDB sink

Error message

failed to write items to DynamoDB sink

What it means

Wraps an AWS SDK BatchWriteItem failure from the DynamoDB sink during a chunk write. The `Err` arm means the batch write request itself failed at the transport/service level (e.g. network error, throttling returning an Err rather than unprocessed items), not that individual items were rejected. The sink deliberately retries unprocessed items; this error is for hard request failures.

Source

Thrown at src/connector/src/sink/dynamodb.rs:602

                                                consumed_capacity = ?output.consumed_capacity(),
                                                "DynamoDB batch write retry succeeded"
                                            );
                                        }
                                        return Ok(());
                                    }

                                    req_items = unprocessed_items.into_values().flatten().collect();
                                    if retry_count >= batch_write_retry_times {
                                        return Err(SinkError::DynamoDb(anyhow!(
                                            "failed to write {} unprocessed items to DynamoDB sink after {} retries",
                                            req_items.len(),
                                            batch_write_retry_times,
                                        )));
                                    }
                                }
                                Err(e) => {
                                    return Err(SinkError::DynamoDb(
                                        anyhow!(e).context("failed to write items to DynamoDB sink"),
                                    ));
                                }
                            }

                            retry_count += 1;
                            let Some(delay) = retry_backoff.next() else {
                                return Err(SinkError::DynamoDb(anyhow!(
                                    "failed to write {} unprocessed items to DynamoDB sink after {} retries",
                                    req_items.len(),
                                    batch_write_retry_times,
                                )));
                            };
                            tracing::warn!(
                                retry_count,
                                delay_ms = delay.as_millis(),
                                unprocessed_items_count = req_items.len(),
                                "retrying DynamoDB batch write"
                            );

View on GitHub (pinned to 6469eb736d)

Solutions

  1. Check the wrapped AWS SDK error (`anyhow` context) for the underlying cause (AccessDenied, ResourceNotFound, throttling) and fix IAM credentials/policy accordingly.
  2. Verify the table name and region in the sink's WITH options (e.g. connection.region, endpoint).
  3. Confirm network/egress connectivity from the RisingWave node to DynamoDB endpoints.
  4. Increase table capacity or use on-demand mode if throttling is the cause.

Example fix

// before
WITH (
  'connector' = 'dynamodb',
  'table' = 'my-table'
)
// after
WITH (
  'connector' = 'dynamodb',
  'table' = 'my-table',
  aws.region = 'us-east-1',
  aws.credentials.access_key_id = '...',
  aws.credentials.secret_access_key = '...'
)
Defensive patterns

Strategy: retry

Validate before calling

aws sts get-caller-identity && aws dynamodb describe-table --table-name my-table --region us-east-1

Try / catch

// risingwave handles this internally; on failure inspect the wrapped AWS SDK error in logs and resume the sink after fixing IAM/network
match sink_error { SinkError::DynamoDb(e) => log::error!("aws cause: {:?}", e.source()), _ => {} }

Prevention

When it happens

Trigger: Calling write_chunk on the DynamoDB sink when the SDK batch_write_item call returns Err: AWS credentials invalid/expired, table not found or missing permissions, network outage, or the table exceeding provisioned throughput.

Common situations: IAM policy lacking dynamodb:BatchWriteItem on the target table; wrong region/endpoint in the connection properties; DynamoDB table deleted or renamed while the sink is running; sustained throughput throttling.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/15daaf9d3c832413. Report an issue: GitHub.