apache/flink · error · ValidationException

Table options do not contain an option key '%s' for discover

Error message

Table options do not contain an option key '%s' for discovering a format.

What it means

FileSystemTableFactory.formatFactoryExists checks whether a format factory is available for a given format identifier. It first reads the 'format' option (FactoryUtil.FORMAT) from the table options. If the option is absent (null), it throws a ValidationException indicating the required 'format' key is missing. This is a table-creation-time validation.

Source

Thrown at flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/FileSystemTableFactory.java:191

    private <I, F extends EncodingFormatFactory<I>> EncodingFormat<I> discoverEncodingFormat(
            Context context, Class<F> formatFactoryClass) {
        FactoryUtil.TableFactoryHelper helper = FactoryUtil.createTableFactoryHelper(this, context);
        if (formatFactoryExists(context, formatFactoryClass)) {
            return helper.discoverEncodingFormat(formatFactoryClass, FactoryUtil.FORMAT);
        } else {
            return null;
        }
    }

    /**
     * Returns true if the format factory can be found using the given factory base class and
     * identifier.
     */
    private boolean formatFactoryExists(Context context, Class<?> factoryClass) {
        Configuration options = Configuration.fromMap(context.getCatalogTable().getOptions());
        String identifier = options.get(FactoryUtil.FORMAT);
        if (identifier == null) {
            throw new ValidationException(
                    String.format(
                            "Table options do not contain an option key '%s' for discovering a format.",
                            FactoryUtil.FORMAT.key()));
        }

        final List<Factory> factories = new LinkedList<>();
        ServiceLoader.load(Factory.class, context.getClassLoader())
                .iterator()
                .forEachRemaining(factories::add);

        final List<Factory> foundFactories =
                factories.stream()
                        .filter(f -> factoryClass.isAssignableFrom(f.getClass()))
                        .collect(Collectors.toList());

        final List<Factory> matchingFactories =
                foundFactories.stream()
                        .filter(f -> f.factoryIdentifier().equals(identifier))

View on GitHub (pinned to 2f3c205e92)

Solutions

Add the 'format' option to the table DDL: 'format'='parquet' (or csv, json, orc, avto, etc.).
Verify the option key is exactly 'format' (not 'file-format', 'formats', or other variants).
Ensure the format identifier matches a supported format (parquet, csv, json, orc, avro, etc.).

Example fix

-- before
CREATE TABLE t (a INT, b STRING) WITH (
  'connector'='filesystem',
  'path'='file:///data'
);

-- after
CREATE TABLE t (a INT, b STRING) WITH (
  'connector'='filesystem',
  'path'='file:///data',
  'format'='parquet'
);

Defensive patterns

Strategy: validation

Validate before calling

// Validate table options before creating the table
Map<String, String> options = catalogTable.getOptions();
if (!options.containsKey("format")) {
    throw new ValidationException(
        "Filesystem connector requires the 'format' option. "
        + "Supported: parquet, csv, json, orc, avro.");
}

Prevention

Always include 'format' in filesystem table DDLs.
Use a DDL template or linter that checks for required options.
Document supported format identifiers in your project's table creation guide.

When it happens

Trigger: A CREATE TABLE statement with a filesystem connector does not include the 'format' option. For example: CREATE TABLE t (...) WITH ('connector'='filesystem', 'path'='...') without specifying 'format'='parquet' or 'format'='csv'.

Common situations: Omitting the 'format' option in a filesystem table DDL. Typo in the option key (e.g. 'formats' instead of 'format'). Migrating a table definition and forgetting to carry over the format option.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/4e083a25874689b7. Report an issue: GitHub.