vectordotdev/vector · error · CallError
Response failed.
Error message
Response failed.
What it means
'Response failed.' is the fallback error string Vector emits in the internal CallError event when a sink request's result is classified as EventStatus::Rejected but result.err() is None — i.e., the underlying service actually returned a Response, yet the response was judged a rejection (typically an HTTP 4xx/5xx or a service-level rejection after retries were exhausted). It surfaces in logs/internal events as an Error with EventsDropped: events from that batch are dropped. The generic wording means the concrete failure detail was lost at the result_status layer.
Source
Thrown at src/sinks/util/sink.rs:459
message = "Submitting service request.",
in_flight_requests = self.in_flight.len()
);
let events_sent = register!(EventsSent::from(Output(None)));
self.service
.call(items)
.err_into()
.map(move |result| {
let status = result_status(&result);
finalizers.update_status(status);
match status {
EventStatus::Delivered => {
events_sent.emit(CountByteSize(count, json_byte_size));
// TODO: Emit a BytesSent event here too
}
EventStatus::Rejected => {
// Emit the `Error` and `EventsDropped` internal events.
// This scenario occurs after retries have been attempted.
let error = result.err().unwrap_or_else(|| "Response failed.".into());
emit!(CallError {
error,
request_id,
count,
});
}
_ => {} // do nothing
}
// If the rx end is dropped we still completed
// the request so this is a weird case that we can
// ignore for now.
_ = tx.send(());
})
.instrument(info_span!("request", %request_id).or_current())
.boxed()
}
View on GitHub (pinned to 3708c39b12)
Solutions
- Inspect the sink target directly: confirm credentials, endpoint URL, and that the service accepts the payload shape (reproduce with curl) — 4xx responses are the most common cause of a Rejected status.
- Raise retry patience and shrink batches: increase request.retry_max_duration_secs / acknowledgements settings, and lower batch.max_bytes/batch.timeout_secs so oversized or rate-limited batches fail less.
- Watch the internal metrics/logs around the CallError (component_errors_received, events_in, events_out) to identify which sink id and request it is, then test that sink alone with a minimal config.
- If the message appears with Ok responses, capture debug logs (LOG=debug or api GraphQL) for the request status to see the real HTTP status the Response carried, and report missing detail upstream — the fallback string indicates lost error context.
Example fix
# before — sink rejects after short default retries, batches too large
sinks:
out:
type: http
inputs: ["my_transform"]
uri: https://api.example.com/ingest
batch:
max_bytes: 5000000
# after — smaller batches + longer retry window so transient rejections recover
sinks:
out:
type: http
inputs: ["my_transform"]
uri: https://api.example.com/ingest
batch:
max_bytes: 1000000
timeout_secs: 1
request:
retry_max_duration_secs: 30 Defensive patterns
Strategy: validation
Validate before calling
# Pre-flight the sink endpoint exactly like the healthcheck will:
curl -sS -o /dev/null -w '%{http_code}\n' \
-H "Authorization: <same credentials as sink>" \
-H 'Content-Type: application/json' \
-d '{"events":[]}' \
https://api.example.com/ingest
# 2xx => endpoint accepts; 4xx/5xx here predicts Rejected batches at runtime.
# And validate the full topology with healthchecks enabled:
vector validate /etc/vector/vector.yaml Prevention
- Run `vector validate` (with healthchecks) against staging before deploying so rejected endpoints surface pre-rollout.
- Keep sink retry settings generous for transient failures (request.retry_max_duration_secs) and batch sizes within the endpoint's limits (batch.max_bytes).
- Monitor internal events and counters (component_errors_received, dropped events) per sink so rejections are caught by alerting instead of discovered via lost data.
- Rotate/validate credentials before expiry — 401/403 responses arrive as Ok-but-Rejected responses and produce this generic message.
When it happens
Trigger: A sink's Service::call returns Ok(response) where the response maps to Rejected (for example an HTTP 4xx like 400/413 or a 5xx that is not retryable, or a retry budget exhausted so the request future completes with a terminal rejected status). At src/sinks/util/sink.rs:459, result.err() is None, so 'Response failed.' replaces the missing error text in the emitted CallError.
Common situations: Destination rejecting payloads: HTTP sink receiving 400 (malformed batch), 401/403 (bad credentials), 413 (batch too large), or 429/5xx after retries and backoff give up; rate limits on the target; auth tokens expired; content encoding mismatch between sink and endpoint. Users see the error via `vector tap`/logs and component_errors_received / events dropped counters rising while events flow.
Related errors
- path and query should never fail to parse
- Failed type coercion, {self:?} is not a Sink
- Failed type coercion, {self:?} is not a Stream
- Invalid cache settings: {e:?}
- poll_ready must be called first
AI-assisted analysis of vectordotdev/vector@3708c39b12 (2026-08-20).
Data as JSON: /api/errors/889fb861a7904263.
Report an issue: GitHub.