{"record":{"id":"638cd580422227e2","repo":"apache/druid","slug":"if-an-external-table-provides-a-format-it-must-al","errorCode":null,"errorMessage":"If an external table provides a format, it must also provide columns","messagePattern":"If an external table provides a format, it must also provide columns","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"server/src/main/java/org/apache/druid/catalog/model/table/FormattedInputSourceDefn.java","lineNumber":69,"sourceCode":"  private Map<String, InputFormatDefn> formats;\n\n  @Override\n  public void bind(TableDefnRegistry registry)\n  {\n    formats = registry.formats();\n    super.bind(registry);\n  }\n\n  @Override\n  public void validate(ResolvedExternalTable table)\n  {\n    final boolean hasColumns = !CollectionUtils.isNullOrEmpty(table.resolvedTable().spec().columns());\n    final boolean hasFormat = table.inputFormatMap != null;\n    if (hasColumns && !hasFormat) {\n      throw new IAE(\"If an external table provides columns, it must also provide a format\");\n    }\n    if (!hasColumns && hasFormat) {\n      throw new IAE(\"If an external table provides a format, it must also provide columns\");\n    }\n    super.validate(table);\n  }\n\n  @Override\n  protected AdHocTableFunction defineAdHocTableFunction()\n  {\n    List<ParameterDefn> fullTableParams = adHocTableFnParameters();\n    List<ParameterDefn> allParams = addFormatParameters(fullTableParams);\n    return new AdHocTableFunction(allParams);\n  }\n\n  /**\n   * Overridden by subclasses to provide the list of table function parameters for\n   * this specific input format. This list is combined with parameters for input\n   * formats. The method is called only once per run.\n   */\n  protected abstract List<ParameterDefn> adHocTableFnParameters();","sourceCodeStart":51,"sourceCodeEnd":87,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/server/src/main/java/org/apache/druid/catalog/model/table/FormattedInputSourceDefn.java#L51-L87","documentation":"The mirror condition of the columns-without-format check: FormattedInputSourceDefn.validate rejects an external table that declares an input format but no columns. For formatted sources Druid requires the schema to be supplied alongside the format.","triggerScenarios":"Validating an external table where inputFormatMap is non-null but spec().columns() is empty; e.g. saving a spec with {\"inputFormat\":{\"type\":\"json\"}} and no column definitions.","commonSituations":"Providing format via an API that does not auto-populate columns; skipping the schema-discovery step in a wizard; hand-written specs that include format but omit columns.","solutions":["Add column definitions to the table spec matching the data.","Run schema discovery/sampling to populate columns before saving.","Remove the format if columns will be inferred elsewhere (only if the source defn allows).","Validate the spec locally before submission: columns XOR format is never accepted for formatted sources."],"exampleFix":"// before\n{\"inputFormat\":{\"type\":\"json\"}}\n// after\n{\"inputFormat\":{\"type\":\"json\"},\"columns\":[{\"name\":\"a\",\"type\":\"bigint\"}]}","handlingStrategy":"validation","validationCode":"boolean hasCols = spec.getColumns() != null && !spec.getColumns().isEmpty();\nboolean hasFmt = spec.getInputFormat() != null;\nif (!hasCols && hasFmt) throw new IllegalArgumentException(\"format requires column definitions\");","typeGuard":null,"tryCatchPattern":"try { saveExternalTable(spec); } catch (IAE e) { if (e.getMessage().contains(\"must also provide columns\")) { runSchemaDiscovery(spec); saveExternalTable(spec); } }","preventionTips":["Run schema discovery before saving formatted external tables","Never submit format-only specs","Keep both fields in your spec templates","Verify columns survive partial-update APIs"],"tags":["druid","external-table","format","validation"],"backgroundTag":"schema-validation-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}