vectordotdev/vector · error

invalid timestamp

Error message

invalid timestamp

What it means

While decoding a protobuf-encoded Metric (Vector's internal event proto), Vector converts ts.seconds/ts.nanos to a chrono DateTime with chrono::Utc.timestamp_opt(...).single().expect("invalid timestamp") (lib/vector-core/src/event/proto.rs). timestamp_opt returns None when seconds/nanos are outside chrono's representable range (roughly years -262144..262143; nanos must be < 1e9), so a wildly out-of-range timestamp in the wire data panics the decoder.

Source

Thrown at lib/vector-core/src/event/proto.rs:224

impl From<Metric> for super::Metric {
    fn from(metric: Metric) -> Self {
        let kind = match metric.kind() {
            metric::Kind::Incremental => super::MetricKind::Incremental,
            metric::Kind::Absolute => super::MetricKind::Absolute,
        };

        let name = metric.name;

        let namespace = (!metric.namespace.is_empty()).then_some(metric.namespace);

        // Sign can never be lost as ts.nanos is always non negative (per proto spec)
        #[allow(clippy::cast_sign_loss)]
        let timestamp = metric.timestamp.map(|ts| {
            chrono::Utc
                .timestamp_opt(ts.seconds, ts.nanos as u32)
                .single()
                .expect("invalid timestamp")
        });

        let mut tags = MetricTags(
            metric
                .tags_v2
                .into_iter()
                .map(|(tag, values)| {
                    (
                        tag,
                        values
                            .values
                            .into_iter()
                            .map(|value| super::metric::TagValue::from(value.value))
                            .collect(),
                    )
                })
                .collect(),
        );

View on GitHub (pinned to 3708c39b12)

Solutions

  1. Fix the clock/timestamp source on the producing host (NTP sync); verify with date -u && timedatectl on the producer
  2. Check for unit confusion upstream: milliseconds-since-epoch in a seconds field yields year ~55897 and panics the decoder
  3. Sanitize timestamps at the producing boundary (clamp or drop metrics with |seconds| > ~8.2e15 / years outside 0000-9999) before sending
  4. If you control the producer, ensure nanos stays within 0..999,999,999 as the proto spec requires

Example fix

// before
let timestamp = metric.timestamp.map(|ts| {
    chrono::Utc.timestamp_opt(ts.seconds, ts.nanos as u32).single().expect("invalid timestamp")
});

// after: reject instead of panic
let timestamp = metric.timestamp.and_then(|ts| {
    chrono::Utc.timestamp_opt(ts.seconds, ts.nanos as u32).single()
});
Defensive patterns

Strategy: validation

Validate before calling

// Producer side: clamp/drop before serializing to proto
fn sanitize_ts(seconds: i64, nanos: i32) -> Option<(i64, i32)> {
    let within_range = chrono::DateTime::<chrono::Utc>::from_timestamp(seconds, nanos).is_some();
    within_range.then_some((seconds, nanos))
}

Prevention

When it happens

Trigger: Decoding (from_proto) a Metric whose timestamp field carries seconds beyond chrono's range — e.g. a producer with a corrupted clock, multiplied epoch values (~year 60,000), i64::MAX, or nanos >= 1_000_000_000 — arriving over Vector-to-Vector gRPC (vector topology, internal_events proto) or from any emitter of Vector's event protobuf.

Common situations: Vector-to-Vector pipelines where an upstream host has a broken RTC/NTP or epoch-unit confusion (milliseconds vs seconds); hand-written producers of Vector's proto format; corrupted payloads after transport-level truncation; bugs in custom aggregators that synthesize metric timestamps.

Related errors


AI-assisted analysis of vectordotdev/vector@3708c39b12 (2026-08-20). Data as JSON: /api/errors/6b29a089b579a466. Report an issue: GitHub.