chroma-core/chroma · error
Failed to start fn-consumer service
Error message
Failed to start fn-consumer service
What it means
The tonic gRPC server bound to 0.0.0.0:<my_port> and serving the FnConsumer plus health services terminated its serve() future with an Err. This fires when the TCP bind fails (port already in use by another worker/instance, or privileged port without permission), or when the listener accept loop hits a fatal I/O error. It is a startup-time panic: the fn_consumer_service_entrypoint cannot run without a live listener, so the process aborts via expect rather than returning an error.
Source
Thrown at rust/worker/src/fn_consumer/server.rs:199
let addr = format!("0.0.0.0:{}", service_config.my_port)
.parse()
.unwrap();
println!("fn-consumer service starting on {}", addr);
// Start server (this blocks forever)
Server::builder()
.add_service(health_service)
.add_service(fn_consumer_service)
.serve(addr)
.await
.expect("Failed to start fn-consumer service");
}View on GitHub (pinned to 34f8e76bae)
Solutions
- Free the port by stopping the process that already binds my_port (find it with ss -ltnp or lsof -i :<port>) or change my_port in the service config to a free port.
- Use an unprivileged port (>1024) or grant CAP_NET_BIND_SERVICE if a low port is required.
- Add a startup pre-check that attempts TcpListener::bind on my_port before spawning the tonic server, returning a structured error instead of panicking.
- Split Server::serve into bind + serve_with_incoming so bind failures surface as distinct, actionable errors.
Example fix
Pick a free, unprivileged port for my_port (e.g. 50051+) and confirm nothing else binds it: lsof -i :<port> or ss -ltnp | grep <port>; kill the conflicting process or update service_config.my_port, then restart the worker.
Defensive patterns
Strategy: try-catch
Prevention
- Assign unique my_port values per worker instance in deployment configuration and validate them before rollout.
- Fail fast at configuration load time if my_port is privileged or malformed.
- Alert on fn-consumer process exit, since this expect panics and kills the process.
- Bind to a specific interface instead of 0.0.0.0 to reduce port conflicts on multi-tenant hosts.
When it happens
Trigger: Another process (or a duplicate instance of this worker) already holds my_port; the configured port is privileged (<1024) and the process lacks CAP_NET_BIND_SERVICE; an invalid/contended address causes bind or accept failure; the OS socket layer returns a fatal error mid-serve.
Common situations: Seen when scaling out workers without assigning distinct my_port values, after a crash leaves a socket lingering, or when a stale process from a previous deployment still listens on the port.
AI-assisted analysis of chroma-core/chroma@34f8e76bae (2026-09-03).
Data as JSON: /api/errors/088f0c9ae70da912.
Report an issue: GitHub.