{"record":{"id":"088f0c9ae70da912","repo":"chroma-core/chroma","slug":"failed-to-start-fn-consumer-service","errorCode":null,"errorMessage":"Failed to start fn-consumer service","messagePattern":"Failed to start fn-consumer service","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"rust/worker/src/fn_consumer/server.rs","lineNumber":199,"sourceCode":"let addr = format!(\"0.0.0.0:{}\", service_config.my_port)\n    .parse()\n    .unwrap();\n\nprintln!(\"fn-consumer service starting on {}\", addr);\n\n// Start server (this blocks forever)\nServer::builder()\n    .add_service(health_service)\n    .add_service(fn_consumer_service)\n    .serve(addr)\n    .await\n    .expect(\"Failed to start fn-consumer service\");\n}","sourceCodeStart":181,"sourceCodeEnd":201,"githubUrl":"https://github.com/chroma-core/chroma/blob/34f8e76bae6195b91160e78b2e47a397055aa3ab/rust/worker/src/fn_consumer/server.rs#L181-L201","documentation":"The tonic gRPC server bound to 0.0.0.0:<my_port> and serving the FnConsumer plus health services terminated its serve() future with an Err. This fires when the TCP bind fails (port already in use by another worker/instance, or privileged port without permission), or when the listener accept loop hits a fatal I/O error. It is a startup-time panic: the fn_consumer_service_entrypoint cannot run without a live listener, so the process aborts via expect rather than returning an error.","triggerScenarios":"Another process (or a duplicate instance of this worker) already holds my_port; the configured port is privileged (<1024) and the process lacks CAP_NET_BIND_SERVICE; an invalid/contended address causes bind or accept failure; the OS socket layer returns a fatal error mid-serve.","commonSituations":"Seen when scaling out workers without assigning distinct my_port values, after a crash leaves a socket lingering, or when a stale process from a previous deployment still listens on the port.","solutions":["Free the port by stopping the process that already binds my_port (find it with ss -ltnp or lsof -i :<port>) or change my_port in the service config to a free port.","Use an unprivileged port (>1024) or grant CAP_NET_BIND_SERVICE if a low port is required.","Add a startup pre-check that attempts TcpListener::bind on my_port before spawning the tonic server, returning a structured error instead of panicking.","Split Server::serve into bind + serve_with_incoming so bind failures surface as distinct, actionable errors."],"exampleFix":"Pick a free, unprivileged port for my_port (e.g. 50051+) and confirm nothing else binds it: lsof -i :<port> or ss -ltnp | grep <port>; kill the conflicting process or update service_config.my_port, then restart the worker.","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":["Assign unique my_port values per worker instance in deployment configuration and validate them before rollout.","Fail fast at configuration load time if my_port is privileged or malformed.","Alert on fn-consumer process exit, since this expect panics and kills the process.","Bind to a specific interface instead of 0.0.0.0 to reduce port conflicts on multi-tenant hosts."],"tags":[],"backgroundTag":null,"analyzedSha":"34f8e76bae6195b91160e78b2e47a397055aa3ab","analyzedAt":"2026-09-03T19:50:49.416Z","contentChangedAt":"2026-09-03T19:50:49.416Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}