apache/druid · error · IAE

Invalid format specification

Error message

Invalid format specification

What it means

InputFormats.convert wraps Jackson's convertValue of the format JSON map into the concrete InputFormat class; any mapping failure (unknown field, wrong types, bad structure) is rethrown as IllegalArgumentException with 'Invalid format specification'. The original exception is attached as the cause.

Source

Thrown at server/src/main/java/org/apache/druid/catalog/model/table/InputFormats.java:116

          .stream()
          .map(col -> col.name())
          .collect(Collectors.toList());
      jsonMap.put("columns", cols);
    }

    /**
     * Convert a generic Java map of input format properties to an input format object.
     */
    public InputFormat convert(
        final Map<String, Object> jsonMap,
        final ObjectMapper jsonMapper
    )
    {
      try {
        return jsonMapper.convertValue(jsonMap, inputFormatClass());
      }
      catch (Exception e) {
        throw new IAE(e, "Invalid format specification");
      }
    }

    @Override
    public InputFormat convertFromTable(ResolvedExternalTable table)
    {
      return convert(table.inputFormatMap, table.resolvedTable().jsonMapper());
    }
  }

  /**
   * Definition of a flat text (CSV and delimited text) input format.
   * <p>
   * Note that not all the fields in
   * {@link org.apache.druid.data.input.impl.FlatTextInputFormat
   * FlatTextInputFormat} appear here:
   * <ul>
   * <li>{@code findColumnsFromHeader} - not yet supported in MSQ.</li>

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Inspect the cause chain of the IAE for Jackson's actual mapping error and fix the offending field
2. Validate the format JSON against the schema for that format type before conversion
3. Correct field names and value types to match the target InputFormat class

Example fix

// before
{"type":"csv","seperator":"|"}   // misspelled option
// after
{"type":"csv","separator":"|"}

// verify required fields for the format type before conversion
def Map<String,Object> sanitizeCsv(Map<String,Object> m) {
  if (!"csv".equals(m.get("type"))) throw new IllegalArgumentException("unsupported format type");
  m.keySet().retainAll(List.of("type","separator","listDelimiter","skipHeader"));
  return m;
}

try { format.convert(jsonMap); } catch (IAE e) {
  Throwable cause = e.getCause();
  // log cause (Jackson error) to identify the bad field, then fix the format JSON
}

When it happens

Trigger: Calling convert on an InputFormats.FormatDefn with a jsonMap that does not conform to the target inputFormatClass, e.g. wrong field types or unknown/misspelled fields for the format type.

Common situations: Typo in a format option name; supplying a string where an object/array is expected; using options from one format type (e.g. CSV delimiter) on another; format JSON hand-edited incorrectly.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• Failed to deserialize a DB object
• Empty columns
• useJsonNodeReader cannot be set to true when assumeNewlineDe
• First token should be START_ARRAY, but it is actually [%s]
• Must provide at least one range

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/cd3ba0e1b9ac549f. Report an issue: GitHub.