risingwavelabs/risingwave · warning

should not fail because we have set `data` and `partition_ke

Error message

should not fail because we have set `data` and `partition_key`

What it means

Building a PutRecordsRequestEntry can only fail if the required `data` or `partition_key` fields are unset. The sink always sets both, so it unwraps with expect(). This is an internal invariant violation, not a user-triggerable error.

Source

Thrown at src/connector/src/sink/kinesis.rs:410

        //
        // An unsuccessfully processed record includes ErrorCode and ErrorMessage values. ErrorCode reflects
        // the type of error and can be one of the following values: ProvisionedThroughputExceededException or
        // InternalFailure. ErrorMessage provides more detailed information about the ProvisionedThroughputExceededException
        // exception including the account ID, stream name, and shard ID of the record that was throttled.
        output
            .records
            .into_iter()
            .find_position(|entry| entry.shard_id.is_none())
    }

    fn put_record(&mut self, key: String, payload: Vec<u8>) {
        let size = key.len() + payload.len();
        self.entries.push((
            PutRecordsRequestEntry::builder()
                .partition_key(key)
                .data(Blob::new(payload))
                .build()
                .expect("should not fail because we have set `data` and `partition_key`"),
            size,
        ))
    }
}

impl FormattedSink for KinesisSinkPayloadWriter {
    type K = String;
    type V = Vec<u8>;

    async fn write_one(&mut self, k: Option<Self::K>, v: Option<Self::V>) -> Result<()> {
        self.put_record(
            k.ok_or_else(|| SinkError::Kinesis(anyhow!("no key provided")))?,
            v.unwrap_or_default(),
        );
        Ok(())
    }
}

View on GitHub (pinned to 6469eb736d)

Solutions

1. If it panics, it is a bug in the sink or aws-sdk-kinesis version — file an issue and check SDK changelog
2. Pin/upgrade aws-sdk-kinesis to a version where the builder still requires these fields
3. As a code fix, replace expect() with a proper SinkError mapping for robustness

// treat a panic here as a bug; catch at the task boundary
match tokio::spawn(sink_task).await {
  Ok(Err(e)) => error!("sink failed: {e:#}"),
  Err(join_err) => error!("sink task panicked: {join_err}"), // covers expect()
  Ok(Ok(_)) => {}
}

When it happens

Trigger: `put_record` in KinesisSinkWriter when `PutRecordsRequestEntry::builder().partition_key(key).data(Blob::new(payload)).build()` unexpectedly returns None.

Common situations: Only possible due to an SDK builder regression or code change that stops setting data/partition_key; never hit by end users in normal operation.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• error building Kinesis client: {err}
• request record count {} not match the response record count
• should not be none
• Kinesis got an unhandled error on stream {:?}, shard {:?}
• shard iterator is none

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/b35d08c7ae9da5e7. Report an issue: GitHub.