vectordotdev/vector · error

Tried to clean schema reference that does not start with…

Error message

Tried to clean schema reference that does not start with the definition prefix: {schema_ref}

What it means

Panic from `.expect("invalid timestamp")` in `decode_ddseries_v2` (src/sources/datadog_agent/metrics.rs:324), converting a Datadog series v2 point's `timestamp` (i64, seconds since epoch) into chrono `DateTime<Utc>` via `Utc.timestamp_opt(dd_point.timestamp, 0)`. `timestamp_opt` returns None when the seconds value is outside chrono's representable range (roughly years -262144..262143), so the expect fires on a corrupt or absurd point timestamp.

Solutions

1. Bounds-check `dd_point.timestamp` (e.g. within 0..=253_402_300_799) before conversion and skip/derive the point otherwise.
2. Replace expect with `.single()` propagated via `?`/filter so out-of-range points are dropped with a warning event instead of panicking the decode task.
3. If the producer sends milliseconds, divide by 1000 before calling `timestamp_opt`.

Example fix

// before
.with_timestamp(Some(
    Utc.timestamp_opt(dd_point.timestamp, 0)
        .single()
        .expect("invalid timestamp"),
))
// after
match Utc.timestamp_opt(dd_point.timestamp, 0).single() {
    Some(ts) => { /* .with_timestamp(Some(ts)) */ }
    None => { emit!(DatadogPointTimestampInvalid { ts: dd_point.timestamp }); return None; }
}

fn valid_dd_timestamp(secs: i64) -> bool {
    (0..=253_402_300_799).contains(&secs)
}

fn dd_ts_to_utc(secs: i64) -> Option<DateTime<Utc>> {
    if !(0..=253_402_300_799).contains(&secs) { return None; }
    Utc.timestamp_opt(secs, 0).single()
}

// Propagate None and skip the point instead of expect:
let ts = Utc.timestamp_opt(dd_point.timestamp, 0).single()?;
// ... .with_timestamp(Some(ts))

When it happens

Trigger: A Datadog agent payload (protobuf v2 series) containing a point whose `timestamp` i64 is out of range — e.g. 0 interpreted with a different epoch unit, microseconds/nanoseconds instead of seconds, or negative/garbage values — reaching `Utc.timestamp_opt(dd_point.timestamp, 0).single()`.

Common situations: A non-standard or buggy Datadog-compatible agent sending epoch-milliseconds in the seconds field (values ~1.7e12 overflow chrono's valid range); fuzzed or hand-crafted requests to the Datadog agent listener; protobuf field misinterpretation after schema changes.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• unit structs should be rejected during AST parsing
• shutdown_complete_tripwire for source
• Could not build Datadog domain regex
• `duration` should be an i64
• `error` should be an i64

AI-assisted analysis of vectordotdev/vector@bdb87aeaa4 (2026-09-16). Data as JSON: /api/errors/d4122c6aadfa9ed9. Report an issue: GitHub.

Appendix: source

Thrown at lib/vector-config-common/src/schema/json_schema.rs:618

        match self {
            SingleOrVec::Single(s) => s.deref() == x,
            SingleOrVec::Vec(v) => v.contains(x),
        }
    }
}

fn is_none_or_default_true(field: &Option<Box<Schema>>) -> bool {
    match field {
        None => true,
        Some(value) => matches!(value.as_ref(), Schema::Bool(true)),
    }
}

pub fn get_cleaned_schema_reference(schema_ref: &str) -> &str {
    if let Some(cleaned) = schema_ref.strip_prefix(DEFINITIONS_PREFIX) {
        cleaned
    } else {
        panic!(
            "Tried to clean schema reference that does not start with the definition prefix: {schema_ref}"
        );
    }
}

View on GitHub (pinned to bdb87aeaa4)