apache/beam · error · InvalidTableException

Table with type 'text' must have format 'csv' or 'lines' or…

Error message

Table with type 'text' must have format 'csv' or 'lines' or 'json'

What it means

TextTableProvider.buildBeamSqlTable validates the 'format' table property when creating a Beam SQL table over text files. Only 'csv', 'lines', and 'json' formats are supported; any other value is rejected via InvalidTableException. This prevents silently misinterpreting file contents at read time.

Solutions

Set the table property format to one of 'csv', 'lines', or 'json'.
Fix casing/typos in the DDL: FORMAT TYPE 'csv' (values are matched literally).
If the data is not line-delimited text, choose a different table provider type (e.g. parquet, avro) instead of 'text'.

Example fix

// before
CREATE EXTERNAL TABLE orders (id INT) TYPE 'text' LOCATION '/data/orders.txt' TBLPROPERTIES '{"format":"txt"}'
// after
CREATE EXTERNAL TABLE orders (id INT) TYPE 'text' LOCATION '/data/orders.txt' TBLPROPERTIES '{"format":"csv"}'

Defensive patterns

Strategy: validation

Validate before calling

String fmt = (String) table.getProperties().get("format");
if (!"csv".equals(fmt) && !"lines".equals(fmt) && !"json".equals(fmt)) {
  throw new IllegalArgumentException("text table format must be csv|lines|json, got: " + fmt);
}

Prevention

Keep table properties in constants/enums rather than raw strings.
Copy format values from working examples in Beam docs.
Validate DDL properties in CI with a table-creation smoke test.

When it happens

Trigger: Calling createTable/BeamSqlEnv with a Table whose getType() is 'text' and whose properties map has a 'format' key with a value other than csv/lines/json (or a misspelled/uppercase value, since matching is on the switch of the format string).

Common situations: Typos like format='txt' or 'JSON' in a CREATE EXTERNAL TABLE DDL statement; copying config from another provider (e.g. parquet) into a text table; programmatic Table builders built with arbitrary property maps.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6c7e4e28c1d423f3. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/text/TextTableProvider.java:127

                : (legacyCsvFormat != null
                    ? CSVFormat.valueOf(legacyCsvFormat)
                    : CSVFormat.DEFAULT);
        return new TextTable(
            schema, filePattern, new CsvToRow(schema, csvFormat), new RowToCsv(csvFormat));
      case "json":
        return new TextJsonTable(
            schema, filePattern, JsonToRow.create(schema, deadLetterFile), RowToJson.create());
      case "lines":
        if (!(schema.getFieldCount() == 1
            && schema.getField(0).getType().getTypeName().equals(TypeName.STRING))) {
          throw new InvalidTableException(
              "Table with type 'text' and format 'lines' "
                  + "must have exactly one STRING/VARCHAR/CHAR column ");
        }
        return new TextTable(
            schema, filePattern, new LinesReadConverter(), new LinesWriteConverter());
      default:
        throw new InvalidTableException(
            "Table with type 'text' must have format 'csv' or 'lines' or 'json'");
    }
  }

  /** Write-side converter for for {@link TextTable} with format {@code 'lines'}. */
  public static class LinesWriteConverter extends PTransform<PCollection<Row>, PCollection<String>>
      implements Serializable {

    public LinesWriteConverter() {}

    @Override
    public PCollection<String> expand(PCollection<Row> input) {
      return input.apply(
          "rowsToLines",
          MapElements.into(TypeDescriptors.strings()).via((Row row) -> row.getString(0) + "\n"));
    }
  }

View on GitHub (pinned to 12126d8942)