vectordotdev/vector · error

target_tps cannot be Nan

Error message

target_tps cannot be Nan

What it means

The datadog_agent traces endpoint decodes the ddtrace v2 protobuf payload and copies the top-level target_tps double into the trace event as Value::Float(NotNan::new(target_tps).expect("target_tps cannot be Nan")). ordered_float::NotNan::new returns Err exactly when the f64 is NaN. Unlike JSON, a protobuf double can carry the NaN bit pattern, so the assumption 'the agent never sends NaN' is enforced with a panic.

Source

Thrown at src/sources/datadog_agent/traces.rs:150

    let enriched_events = trace_events
        .into_iter()
        .map(|mut trace_event| {
            if let Some(k) = &api_key {
                trace_event
                    .metadata_mut()
                    .set_datadog_api_key(Arc::clone(k));
            }
            trace_event.insert(
                &source.log_schema_source_type_key,
                Bytes::from("datadog_agent"),
            );
            trace_event.insert(event_path!("payload_version"), "v2".to_string());
            trace_event.insert(&source.log_schema_host_key, hostname.clone());
            trace_event.insert(event_path!("env"), env.clone());
            trace_event.insert(event_path!("agent_version"), agent_version.clone());
            trace_event.insert(
                event_path!("target_tps"),
                Value::Float(NotNan::new(target_tps).expect("target_tps cannot be Nan")),
            );
            trace_event.insert(
                event_path!("error_tps"),
                Value::Float(NotNan::new(error_tps).expect("error_tps cannot be Nan")),
            );
            if let Some(Value::Object(span_tags)) = trace_event.get_mut(event_path!("tags")) {
                span_tags.extend(tags.clone());
            } else {
                trace_event.insert(event_path!("tags"), Value::from(tags.clone()));
            }
            Event::Trace(trace_event)
        })
        .collect();
    Ok(enriched_events)
}

fn convert_dd_tracer_payload(payload: ddtrace_proto::TracerPayload) -> Vec<TraceEvent> {
    let tags = convert_tags(payload.tags);

View on GitHub (pinned to 3708c39b12)

Solutions

1. Guard before inserting: use if let Some(tps) = NotNan::new(target_tps) and skip the field insert (with a warning) on Err
2. Validate the decoded payload's float fields right after TracePayload::decode and reject the request with 422
3. Upgrade Vector once input hardening lands

Example fix

// before
trace_event.insert(
    event_path!("target_tps"),
    Value::Float(NotNan::new(target_tps).expect("target_tps cannot be Nan")),
);

// after
if let Some(tps) = NotNan::new(target_tps) {
    trace_event.insert(event_path!("target_tps"), Value::Float(tps));
} else {
    warn!(message = "dropping NaN target_tps from datadog agent payload");
}

fn finite_or_zero(v: f64) -> f64 {
    if v.is_finite() { v } else { 0.0 }
}

// before insert:
if !target_tps.is_finite() {
    warn!(value = target_tps, "non-finite target_tps from agent payload");
    target_tps = 0.0;
}

fn as_not_nan(v: f64) -> Option<ordered_float::NotNan<f64>> {
    ordered_float::NotNan::new(v).ok()
}

if let Some(tps) = NotNan::new(target_tps) {
    trace_event.insert(event_path!("target_tps"), Value::Float(tps));
} else {
    warn!(message = "dropping NaN target_tps");
}

When it happens

Trigger: POSTing a TracePayload protobuf to the datadog_agent traces endpoint with the target_tps field set to the NaN bit pattern (0x7FF8...), or a corrupted frame whose bytes decode to NaN. handle_dd_trace_payload_v1 then panics while enriching the trace events.

Common situations: Corrupt or hostile agent payloads, custom tracers built against ddtrace_proto, proxies mangling bodies. One NaN field aborts the whole trace batch conversion.

Related errors

• error_tps cannot be Nan
• Failed type coercion, {self:?} is not a trace event
• `error` should be an i64
• `duration` should be an i64
• `parent_id` should be an i64

AI-assisted analysis of vectordotdev/vector@3708c39b12 (2026-08-20). Data as JSON: /api/errors/e70ec61a4f72b1fb. Report an issue: GitHub.