vectordotdev/vector · error

error_tps cannot be Nan

Error message

error_tps cannot be Nan

What it means

Same invariant as target_tps but for the error_tps double of the ddtrace v2 protobuf payload: Value::Float(NotNan::new(error_tps).expect("error_tps cannot be Nan")). NotNan::new fails only for NaN, and protobuf doubles can encode NaN, so a payload with that bit pattern panics the trace-decoding task.

Source

Thrown at src/sources/datadog_agent/traces.rs:154

                trace_event
                    .metadata_mut()
                    .set_datadog_api_key(Arc::clone(k));
            }
            trace_event.insert(
                &source.log_schema_source_type_key,
                Bytes::from("datadog_agent"),
            );
            trace_event.insert(event_path!("payload_version"), "v2".to_string());
            trace_event.insert(&source.log_schema_host_key, hostname.clone());
            trace_event.insert(event_path!("env"), env.clone());
            trace_event.insert(event_path!("agent_version"), agent_version.clone());
            trace_event.insert(
                event_path!("target_tps"),
                Value::Float(NotNan::new(target_tps).expect("target_tps cannot be Nan")),
            );
            trace_event.insert(
                event_path!("error_tps"),
                Value::Float(NotNan::new(error_tps).expect("error_tps cannot be Nan")),
            );
            if let Some(Value::Object(span_tags)) = trace_event.get_mut(event_path!("tags")) {
                span_tags.extend(tags.clone());
            } else {
                trace_event.insert(event_path!("tags"), Value::from(tags.clone()));
            }
            Event::Trace(trace_event)
        })
        .collect();
    Ok(enriched_events)
}

fn convert_dd_tracer_payload(payload: ddtrace_proto::TracerPayload) -> Vec<TraceEvent> {
    let tags = convert_tags(payload.tags);
    payload
        .chunks
        .into_iter()
        .map(|trace| {

View on GitHub (pinned to 3708c39b12)

Solutions

1. Guard before inserting: if let Some(tps) = NotNan::new(error_tps) and log-and-skip on Err
2. Validate float fields immediately after TracePayload::decode and reject the request with 422 instead of panicking
3. Upgrade Vector once a fix lands

Example fix

// before
trace_event.insert(
    event_path!("error_tps"),
    Value::Float(NotNan::new(error_tps).expect("error_tps cannot be Nan")),
);

// after
if let Some(tps) = NotNan::new(error_tps) {
    trace_event.insert(event_path!("error_tps"), Value::Float(tps));
} else {
    warn!(message = "dropping NaN error_tps from datadog agent payload");
}

fn finite_or_zero(v: f64) -> f64 {
    if v.is_finite() { v } else { 0.0 }
}

// before insert:
if !error_tps.is_finite() {
    warn!(value = error_tps, "non-finite error_tps from agent payload");
    error_tps = 0.0;
}

fn as_not_nan(v: f64) -> Option<ordered_float::NotNan<f64>> {
    ordered_float::NotNan::new(v).ok()
}

if let Some(tps) = NotNan::new(error_tps) {
    trace_event.insert(event_path!("error_tps"), Value::Float(tps));
} else {
    warn!(message = "dropping NaN error_tps");
}

When it happens

Trigger: A TracePayload protobuf POSTed to the datadog_agent traces endpoint whose error_tps field is NaN - corrupt frames, hostile payloads, or custom senders writing uninitialized floats.

Common situations: Fuzzed or replayed trace payloads, interop with non-compliant tracers, corrupted bodies in transit. The panic loses the whole batch of trace events in that request.

Related errors

• target_tps cannot be Nan
• Failed type coercion, {self:?} is not a trace event
• `error` should be an i64
• `duration` should be an i64
• `parent_id` should be an i64

AI-assisted analysis of vectordotdev/vector@3708c39b12 (2026-08-20). Data as JSON: /api/errors/60daf8d2a58326d6. Report an issue: GitHub.