risingwavelabs/risingwave · error · ConnectorError

scan.startup.timestamp.millis is required

Error message

scan.startup.timestamp.millis is required

What it means

The Kinesis source connector's `new` parses `scan.startup.mode`; when it is set to `"timestamp"`, the property `scan.startup.timestamp.millis` (mapped to `properties.start_timestamp_millis`) must also be provided. If it is absent, `bail!` aborts source construction with this message. It enforces that a timestamp startup mode always carries an actual epoch-milliseconds value to seek from.

Solutions

  1. Add `scan.startup.timestamp.millis = '<epoch_ms>'` to the source WITH clause.
  2. If you actually want to start from the oldest data, set `scan.startup.mode = 'earliest'` instead.
  3. Validate the pair of options in the CREATE SOURCE SQL before running it.

Example fix

// before
WITH (
  connector = 'kinesis',
  stream_name = 'events',
  scan.startup.mode = 'timestamp'
)
// after
WITH (
  connector = 'kinesis',
  stream_name = 'events',
  scan.startup.mode = 'timestamp',
  scan.startup.timestamp.millis = '1700000000000'
)
Defensive patterns

Strategy: validation

Validate before calling

// Before creating/updating the source, validate the option pair
let mode = props.get("scan.startup.mode").map(|s| s.as_str());
if mode == Some("timestamp") {
    props.get("scan.startup.timestamp.millis")
        .and_then(|v| v.parse::<i64>().ok())
        .ok_or_else(|| anyhow!("scan.startup.mode='timestamp' requires scan.startup.timestamp.millis (epoch ms)"))?;
}

Prevention

When it happens

Trigger: Creating a Kinesis source with WITH option `scan.startup.mode = 'timestamp'` while omitting `scan.startup.timestamp.millis`.

Common situations: Users copy a config template that enables timestamp mode but forget the companion timestamp field; SQL changed via ALTER SOURCE adds the mode without adding the property; config validation passes because the mode value is valid but the pairing is only checked at reader construction.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/fade0ffd791a38c9. Report an issue: GitHub.

Appendix: source

Thrown at src/connector/src/source/kinesis/source/reader.rs:91

        parser_config: ParserConfig,
        source_ctx: SourceContextRef,
        _columns: Option<Vec<Column>>,
    ) -> Result<Self> {
        assert!(splits.len() == 1);

        let split = splits.into_iter().next().unwrap();

        let next_offset = match &split.next_offset {
            KinesisOffset::None => match &properties.scan_startup_mode {
                None => KinesisOffset::Earliest,
                Some(mode) => match mode.as_str() {
                    "earliest" => KinesisOffset::Earliest,
                    "latest" => KinesisOffset::Latest,
                    "timestamp" => {
                        if let Some(ts) = &properties.start_timestamp_millis {
                            KinesisOffset::Timestamp(*ts)
                        } else {
                            bail!("scan.startup.timestamp.millis is required");
                        }
                    }
                    _ => {
                        bail!("invalid scan.startup.mode, accept earliest/latest/timestamp")
                    }
                },
            },
            next_offset => next_offset.to_owned(),
        };

        if !matches!(next_offset, KinesisOffset::Timestamp(_))
            && properties.start_timestamp_millis.is_some()
        {
            // cannot bail! here because all new split readers will fail to start if user set 'scan.startup.mode' to 'timestamp'
            tracing::warn!(
                "scan.startup.mode needs to be set to 'timestamp' if you want to start with a specific timestamp, starting shard {} from the beginning",
                split.id()
            );

View on GitHub (pinned to 6469eb736d)