{"id":"e0bff175ca09e421","repo":"apache/kafka","slug":"failed-to-construct-kafka-producer","errorCode":null,"errorMessage":"Failed to construct kafka producer","messagePattern":"Failed to construct kafka producer","errorType":"exception","errorClass":"KafkaException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java","lineNumber":524,"sourceCode":"                        PRODUCER_METRIC_GROUP_NAME,\n                        time,\n                        transactionManager,\n                        new BufferPool(this.totalMemorySize, batchSize, metrics, time, PRODUCER_METRIC_GROUP_NAME, BufferPool.AllocationMode.FULL));\n            }\n\n            this.errors = this.metrics.sensor(\"errors\");\n            this.sender = newSender(logContext, kafkaClient, this.metadata);\n            String ioThreadName = NETWORK_THREAD_PREFIX + \" | \" + clientId;\n            this.ioThread = new Sender.SenderThread(ioThreadName, this.sender, true);\n            this.ioThread.start();\n            config.logUnused();\n            AppInfoParser.registerAppInfo(JMX_PREFIX, clientId, metrics, time.milliseconds());\n            log.debug(\"Kafka producer started\");\n        } catch (Throwable t) {\n            // call close methods if internal objects are already constructed this is to prevent resource leak. see KAFKA-2121\n            close(Duration.ofMillis(0), true);\n            // now propagate the exception\n            throw new KafkaException(\"Failed to construct kafka producer\", t);\n        }\n    }\n\n    // visible for testing\n    KafkaProducer(ProducerConfig config,\n                  LogContext logContext,\n                  Metrics metrics,\n                  Serializer<K> keySerializer,\n                  Serializer<V> valueSerializer,\n                  ProducerMetadata metadata,\n                  RecordAccumulator accumulator,\n                  TransactionManager transactionManager,\n                  Sender sender,\n                  ProducerInterceptors<K, V> interceptors,\n                  Partitioner partitioner,\n                  Time time,\n                  Sender.SenderThread ioThread,\n                  Optional<ClientTelemetryReporter> clientTelemetryReporter) {","sourceCodeStart":506,"sourceCodeEnd":542,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java#L506-L542","documentation":"Catch-all wrapper thrown from the KafkaProducer constructor's outer try/catch (KAFKA-2121). Any Throwable raised while building the producer — config parsing, Metrics, serializers, transaction manager, network client, Sender thread start — is first cleaned up via close(Duration.ZERO, true) to avoid leaking threads/sockets, then rethrown wrapped as KafkaException with this message and the original as cause. The actual reason is in getCause().","triggerScenarios":"Any failure during new KafkaProducer<>(props): invalid/unknown config keys, missing or unserializable key/value.serializer.class, bad bootstrap.servers format, SSL/SASL/JAAS misconfiguration, Kerberos login failure, Metrics/MetricReporter instantiation error, enable.idempotence with an incompatible acks/max.in.flight, transactional.id set without proper broker support, classpath issues loading plugins.","commonSituations":"Wrong serializer class name; typo in bootstrap.servers (e.g. missing port); JAAS config not on classpath or wrong path; Kerberos ticket expired at producer creation; serializer jar not on classpath in a fat-jar that excluded it; conflicting client library versions; producer config copied from another service with environment-specific values that don't resolve.","solutions":["Read the wrapped cause — exception.getCause() (or the log line just above) carries the real reason; act on that, not on this message.","If the cause is ConfigException, fix the named property; if ClassNotFoundException, add the missing dependency (serializer, login module, metrics reporter).","Validate producer properties with ProducerConfig.parseAndValidate(props) or a small smoke-test main() before deploying.","For security-related causes (SASL/SSL/Kerberos), confirm the JAAS/config files are present and readable by the JVM at construction time.","Check for conflicting jars on the classpath (multiple kafka-clients versions, shaded serializer duplicates)."],"exampleFix":"// before\ntry {\n    Producer<String, byte[]> p = new KafkaProducer<>(props);\n} catch (KafkaException e) {\n    log.error(\"producer failed\", e); // message is generic, root cause hidden\n}\n\n// after — unwrap and surface the real reason\ntry {\n    Producer<String, byte[]> p = new KafkaProducer<>(props);\n} catch (KafkaException e) {\n    Throwable cause = e.getCause() != null ? e.getCause() : e;\n    log.error(\"producer construction failed: {}\", cause.getMessage(), cause);\n    throw new RuntimeException(\"cannot start producer: \" + cause.getMessage(), cause);\n}","handlingStrategy":"try-catch","validationCode":"// KafkaProducer's constructor wraps ANY failure (bad config, missing class,\n// serializer error, security, ...) in KafkaException(\"Failed to construct kafka\n// producer\", cause). You cannot fully pre-validate every internal step, so the\n// reliable defense is to catch and inspect getCause().\n// Useful pre-check: validate the Properties via ProducerConfig without building\n// the producer, which surfaces most ConfigException issues early.\ntry {\n    org.apache.kafka.clients.producer.ProducerConfig me =\n        new org.apache.kafka.clients.producer.ProducerConfig(props);\n    me.values();   // throws ConfigException on bad/unknown keys\n} catch (org.apache.kafka.common.config.ConfigException e) {\n    log.error(\"Invalid producer config, will not attempt construction\", e);\n}","typeGuard":null,"tryCatchPattern":"// Construction can fail for many reasons; always inspect the cause and free\n// any partial resources (the producer itself closes them, but your code must\n// not retain a half-built reference).\nKafkaProducer<K,V> producer;\ntry {\n    producer = new KafkaProducer<>(props, keySer, valSer);\n} catch (org.apache.kafka.common.KafkaException e) {\n    Throwable c = e.getCause();\n    if (c instanceof org.apache.kafka.common.config.ConfigException) {\n        log.error(\"Bad producer config\", c);\n    } else if (c instanceof ClassNotFoundException) {\n        log.error(\"Serializer/partitioner class not on classpath\", c);\n    } else {\n        log.error(\"Producer construction failed\", c);\n    }\n    throw e;            // or fall back to a different config / fail fast\n}","preventionTips":["Construct the producer exactly once at app startup, never per-message; surface failures immediately.","Run `new ProducerConfig(props).values()` first to catch config errors with a clear message.","Ensure serializer/partitioner classes and all JAAS/SSL config are on the classpath before startup.","Never swallow the cause — log KafkaException.getCause() so the real failure is visible."],"tags":["producer","construction","config","lifecycle","wrapper"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}