nathanmarz/storm · error · IllegalArgumentException

Storm conf is not valid. Must be json-serializable

Error message

Storm conf is not valid. Must be json-serializable

What it means

StormSubmitter.submitTopology validates the stormConf Map with Utils.isValidConf, which requires every value to be JSON-serializable. If the conf contains non-serializable objects (custom classes, nulls in wrong places, etc.), it throws this IllegalArgumentException before contacting Nimbus.

Solutions

  1. Replace non-serializable values in stormConf with JSON primitives (String, Number, Boolean, Map, List).
  2. Call Utils.isValidConf(conf) yourself before submitting to catch the bad key early.
  3. If you need complex objects in a bolt, pass them via component configuration alternatives or serialize them yourself.
  4. Print/inspect the conf Map and check each value's type.

Example fix

// before
conf.put("my.custom.object", new MyConfigBean());
// after
conf.put("my.custom.object", myBean.toJsonString());
Defensive patterns

Strategy: try-catch

Validate before calling

// call before submitTopology
if (!Utils.isValidConf(myConf)) {
    for (Map.Entry e : ((Map) myConf).entrySet()) {
        try { new Gson().toJsonTree(e.getValue()); }
        catch (Exception ex) { System.err.println("Non-serializable conf value at key: " + e.getKey()); }
    }
}

Try / catch

try {
    StormSubmitter.submitTopology(name, conf, topology);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("json-serializable")) {
        log.error("Conf has non-JSON-serializable values; sanitize conf", e);
    } else { throw e; }
}

Prevention

When it happens

Trigger: Putting non-JSON-serializable values into the topology conf Map (e.g. arbitrary Java objects, streams, complex types) and calling StormSubmitter.submitTopology / submitTopologyWithProgressBar.

Common situations: Programmatically building conf with typed objects (Date, custom config beans); reading conf from code that mixes Object values; injecting per-topology settings that aren't primitives/collections.

Understand the failure class

Background: "JSON serialization failed", "not JSON serializable", "Failed to serialize": why JSON marshaling errors happen and how to fix them — this error's family across 46 libraries.

Related errors

AI-assisted analysis of nathanmarz/storm@cdb116e942 (2026-09-12). Data as JSON: /api/errors/b8e79dae57a8b235. Report an issue: GitHub.

Appendix: source

Thrown at storm-core/src/jvm/backtype/storm/StormSubmitter.java:75

    public static void submitTopology(String name, Map stormConf, StormTopology topology) throws AlreadyAliveException, InvalidTopologyException {
        submitTopology(name, stormConf, topology, null);
    }    
    
    /**
     * Submits a topology to run on the cluster. A topology runs forever or until 
     * explicitly killed.
     *
     *
     * @param name the name of the storm.
     * @param stormConf the topology-specific configuration. See {@link Config}. 
     * @param topology the processing to execute.
     * @param options to manipulate the starting of the topology
     * @throws AlreadyAliveException if a topology with this name is already running
     * @throws InvalidTopologyException if an invalid topology was submitted
     */
    public static void submitTopology(String name, Map stormConf, StormTopology topology, SubmitOptions opts) throws AlreadyAliveException, InvalidTopologyException {
        if(!Utils.isValidConf(stormConf)) {
            throw new IllegalArgumentException("Storm conf is not valid. Must be json-serializable");
        }
        stormConf = new HashMap(stormConf);
        stormConf.putAll(Utils.readCommandLineOpts());
        Map conf = Utils.readStormConfig();
        conf.putAll(stormConf);
        try {
            String serConf = JSONValue.toJSONString(stormConf);
            if(localNimbus!=null) {
                LOG.info("Submitting topology " + name + " in local mode");
                localNimbus.submitTopology(name, null, serConf, topology);
            } else {
                NimbusClient client = NimbusClient.getConfiguredClient(conf);
                if(topologyNameExists(conf, name)) {
                    throw new RuntimeException("Topology with name `" + name + "` already exists on cluster");
                }
                submitJar(conf);
                try {
                    LOG.info("Submitting topology " +  name + " in distributed mode with conf " + serConf);

View on GitHub (pinned to cdb116e942)