apache/beam · error · IllegalArgumentException

A schema was provided without a data format (or viceversa).

Error message

A schema was provided without a data format (or viceversa). Please provide both of these parameters to read from Pubsub, or if you would like to use the Pubsub schema service, please leave both of these blank.

What it means

For non-RAW formats, PubsubReadSchemaTransformProvider requires the 'schema' and 'format' configuration fields to be provided together or not at all. It throws when exactly one of the two is set, because it cannot know how to decode payloads with only a schema (no format) or only a format (no schema).

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubReadSchemaTransformProvider.java:93

  @Override
  public SchemaTransform from(PubsubReadSchemaTransformConfiguration configuration) {
    if (configuration.getSubscription() == null && configuration.getTopic() == null) {
      throw new IllegalArgumentException(
          "To read from Pubsub, a subscription name or a topic name must be provided");
    }

    if (configuration.getSubscription() != null && configuration.getTopic() != null) {
      throw new IllegalArgumentException(
          "To read from Pubsub, a subscription name or a topic name must be provided. Not both.");
    }

    if (!"RAW".equals(configuration.getFormat())) {
      if ((Strings.isNullOrEmpty(configuration.getSchema())
              && !Strings.isNullOrEmpty(configuration.getFormat()))
          || (!Strings.isNullOrEmpty(configuration.getSchema())
              && Strings.isNullOrEmpty(configuration.getFormat()))) {
        throw new IllegalArgumentException(
            "A schema was provided without a data format (or viceversa). Please provide "
                + "both of these parameters to read from Pubsub, or if you would like to use the Pubsub schema service,"
                + " please leave both of these blank.");
      }
    }

    Schema payloadSchema;
    SerializableFunction<byte[], Row> payloadMapper;

    String format =
        configuration.getFormat() == null ? null : configuration.getFormat().toUpperCase();
    if ("RAW".equals(format)) {
      payloadSchema = Schema.of(Schema.Field.of("payload", Schema.FieldType.BYTES));
      payloadMapper = input -> Row.withSchema(payloadSchema).addValue(input).build();
    } else if ("JSON".equals(format)) {
      payloadSchema = JsonUtils.beamSchemaFromJsonSchema(configuration.getSchema());
      payloadMapper = JsonUtils.getJsonBytesToRowFunction(payloadSchema);
    } else if ("AVRO".equals(format)) {

View on GitHub (pinned to 12126d8942)

Solutions

Set both 'schema' and 'format' in the configuration Row (e.g. an Avro/JSON schema plus format=AVRO/JSON).
Alternatively, leave BOTH empty to let the reader use the Pubsub schema service.
If payloads are opaque bytes, set format=RAW, which bypasses this check.
Check that empty-string template variables aren't silently filling only one of the two fields.

Example fix

// before
config = Config.builder().setSchema(avroSchemaString).build(); // format missing
// after
config = Config.builder().setSchema(avroSchemaString).setFormat("AVRO").build();

Defensive patterns

Strategy: validation

Validate before calling

boolean hasSchema = cfg.getSchema() != null && !cfg.getSchema().isEmpty();
boolean hasFormat = cfg.getFormat() != null && !cfg.getFormat().isEmpty();
if (!"RAW".equals(cfg.getFormat()) && hasSchema != hasFormat) { throw new IllegalArgumentException("schema and format must be provided together or both blank"); }

Try / catch

try { return provider.from(cfg); } catch (IllegalArgumentException e) { if (e.getMessage().startsWith("A schema was provided without a data format")) { /* set both fields or both blank */ } throw e; }

Prevention

Always set schema and format as a pair from a single config object.
Remember the third valid option: leave both blank to use the Pubsub schema service.
Watch for template variables rendering as empty strings and filling only one field.

When it happens

Trigger: Calling from() with a configuration Row where (schema is set and format is empty) OR (format is set and schema is empty), while format != "RAW".

Common situations: Users supplying a JSON schema but forgetting format=JSON; users setting format=AVRO but leaving the schema field empty; template variables that resolve to empty strings for one of the two fields.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d81a9649cc9dd45b. Report an issue: GitHub.