qdrant/qdrant · error · CollectionError
{errors_count} of {operations_count} read operations failed{
Error message
{errors_count} of {operations_count} read operations failed{errors_separator} What it means
Read operations fan out to the shard's replica set; when every queried replica returns an error (network, timeout, replica down), qdrant aggregates them into a service error 'N of M read operations failed' listing each replica failure. It means zero healthy replicas served that shard's read — with even one healthy replica, the read succeeds.
Source
Thrown at lib/collection/src/shards/replica_set/execute_read_operation.rs:404
}
}
if responses.len() >= required_successful_results {
Ok(responses)
} else {
let errors_count = errors.len();
let operations_count = responses.len() + errors.len();
let errors_separator = if !errors.is_empty() { ":" } else { "" };
let mut message = format!(
"{errors_count} of {operations_count} read operations failed{errors_separator}"
);
for error in errors {
write!(&mut message, "\n {error}").expect("writing into String always succeeds");
}
Err(CollectionError::service_error(message))
}
}
}
View on GitHub (pinned to 74f3e85b94)
Solutions
- Check /cluster and telemetry: bring at least one replica per shard group back to Active state
- Revive a dead/out-of-sync replica or restore the shard from a snapshot/healthy replica
- Retry with backoff for transient partitions
- During maintenance, never stop all replicas of the same shard at once
Defensive patterns
Strategy: retry
Try / catch
let mut attempt = 0;
loop {
match collection.retrieve(points).await {
Err(e) if e.to_string().contains("read operations failed") && attempt < 3 => {
attempt += 1;
tokio::time::sleep(Duration::from_millis(200 * attempt)).await;
}
other => break other,
}
} Prevention
- Keep at least one replica per shard group running during maintenance windows
- Monitor /cluster for replica states and alert before all replicas of a shard go down
- Set client-side retries with backoff for read operations in distributed deployments
When it happens
Trigger: A point read (get/scroll by id) against a shard whose replicas are all stopped, unreachable, or timing out; a transient cluster-wide network partition during the read.
Common situations: Rolling restarts that stop all replicas of a shard simultaneously; misconfigured cluster addresses; replicas overloaded past request deadlines.
Related errors
AI-assisted analysis of qdrant/qdrant@74f3e85b94 (2026-08-22).
Data as JSON: /api/errors/1d58d3f5bcae98cf.
Report an issue: GitHub.