What causes "Topic must be specified by the Kafka log4j appender" in zhisheng17/flink-learning?

Add topic="your-topic" to the element.

zhisheng17/flink-learning · error · ConfigException

Topic must be specified by the Kafka log4j appender

Error message

Topic must be specified by the Kafka log4j appender

What it means

KafkaLog4j2Appender requires a target Kafka topic to which log events are produced. If the topic attribute is null after configuration parsing, it throws ConfigException during appender initialization.

Source

Thrown at flink-learning-extends/FlinkLogKafkaAppender/Log4j2KafkaAppender/src/main/java/com/zhisheng/log/appender/KafkaLog4j2Appender.java:110

                log.error("log.file Property ({}) doesn't contains yarn application id or container id", logFile);
            }
        }

        taskName = envProperties.getProperty("taskName", null);
        taskId = envProperties.getProperty("taskId", null);

        Properties props = new Properties();
        for (Property property : properties) {
            props.put(property.getName(), property.getValue());
        }

        if (bootstrapServers != null) {
            props.setProperty("bootstrap.servers", bootstrapServers);
        } else {
            throw new ConfigException("The bootstrap servers property must be specified");
        }
        if (this.topic == null) {
            throw new ConfigException("Topic must be specified by the Kafka log4j appender");
        }

        String clientIdPrefix = taskId != null ? taskId : appId;

        if (clientIdPrefix != null) {
            props.setProperty("client.id", clientIdPrefix + "_log");
        }

        if (props.getProperty("acks") == null) {
            props.setProperty("acks", "0");
        }

        if (props.getProperty("retries") == null) {
            props.setProperty("retries", "0");
        }

        if (props.getProperty("batch.size") == null) {
            props.setProperty("batch.size", "16384");

View on GitHub (pinned to d731cee761)

Solutions

Add topic="your-topic" to the <KafkaLog4j2Appender> element.
Ensure the topic already exists in Kafka or enable auto topic creation on the broker.
Double-check attribute casing/typos in log4j2.xml.
Verify with a smoke log that events are delivered to the topic.

Example fix

// before
<KafkaLog4j2Appender bootstrapServers="broker:9092"/>
// after
<KafkaLog4j2Appender bootstrapServers="broker:9092" topic="app-logs"/>

Defensive patterns

Strategy: validation

Validate before calling

if (topic == null || topic.isEmpty())
    throw new IllegalStateException("KafkaLog4j2Appender topic attribute is required");

Try / catch

try {
    LOGGER.info("startup");
} catch (Throwable t) {
    System.err.println("Kafka appender misconfigured: " + t.getMessage());
}

Prevention

Include topic in every KafkaLog4j2Appender declaration
Create topics ahead of deployment (or enable broker auto-create)
Verify attribute names against the appender's @PluginAttr definitions
Add a config lint step that parses log4j2.xml and checks required attributes

When it happens

Trigger: Configuring <KafkaLog4j2Appender bootstrapServers="..."> without the topic attribute in log4j2.xml.

Common situations: Forgotten topic attribute; topic supplied under a wrong attribute name; XML config assembled programmatically omitting the field.

Understand the failure class

Background: "X is required", "must be set", "cannot be empty": the missing-required-config error family, from Vertex AI project/location to WeChat keys — this error's family across 18 libraries.

Related errors

AI-assisted analysis of zhisheng17/flink-learning@d731cee761 (2026-09-06). Data as JSON: /api/errors/54195587859e3c0a. Report an issue: GitHub.