apache/beam · error · IllegalArgumentException

To read from Pubsub, a subscription name or a topic name mus

Error message

To read from Pubsub, a subscription name or a topic name must be provided

What it means

PubsubReadSchemaTransformProvider.from() validates its configuration before building the SchemaTransform. A read needs exactly one of a subscription or a topic; if both are null it throws IllegalArgumentException with this message.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubReadSchemaTransformProvider.java:79

  public static final String VALID_FORMATS_STR = "RAW,AVRO,JSON";
  public static final Set<String> VALID_DATA_FORMATS =
      Sets.newHashSet(VALID_FORMATS_STR.split(","));

  public static final TupleTag<Row> OUTPUT_TAG = new TupleTag<Row>() {};
  public static final TupleTag<Row> ERROR_TAG = new TupleTag<Row>() {};
  public static final Schema ERROR_SCHEMA =
      Schema.builder().addStringField("error").addNullableByteArrayField("row").build();

  @Override
  public Class<PubsubReadSchemaTransformConfiguration> configurationClass() {
    return PubsubReadSchemaTransformConfiguration.class;
  }

  @Override
  public SchemaTransform from(PubsubReadSchemaTransformConfiguration configuration) {
    if (configuration.getSubscription() == null && configuration.getTopic() == null) {
      throw new IllegalArgumentException(
          "To read from Pubsub, a subscription name or a topic name must be provided");
    }

    if (configuration.getSubscription() != null && configuration.getTopic() != null) {
      throw new IllegalArgumentException(
          "To read from Pubsub, a subscription name or a topic name must be provided. Not both.");
    }

    if (!"RAW".equals(configuration.getFormat())) {
      if ((Strings.isNullOrEmpty(configuration.getSchema())
              && !Strings.isNullOrEmpty(configuration.getFormat()))
          || (!Strings.isNullOrEmpty(configuration.getSchema())
              && Strings.isNullOrEmpty(configuration.getFormat()))) {
        throw new IllegalArgumentException(
            "A schema was provided without a data format (or viceversa). Please provide "
                + "both of these parameters to read from Pubsub, or if you would like to use the Pubsub schema service,"
                + " please leave both of these blank.");
      }

View on GitHub (pinned to 12126d8942)

Solutions

1. Set exactly one of subscription or topic in the configuration (YAML key 'subscription' or 'topic').
2. Check for typo'd or ignored config keys so the value actually lands on the configuration object.
3. When setting programmatically, call configuration.setTopic(...) or setSubscription(...) before from().

Example fix

// before
map:
  format: JSON   # no subscription/topic
// after
map:
  subscription: projects/my-project/subscriptions/my-sub
  format: JSON

PubsubReadSchemaTransformConfiguration c = ...;
if (c.getSubscription() == null && c.getTopic() == null) {
  throw new IllegalArgumentException("Set exactly one of subscription or topic for Pubsub read");
}
if (c.getSubscription() != null && c.getTopic() != null) {
  throw new IllegalArgumentException("Set subscription OR topic, not both");
}

try {
  transform = provider.from(configuration);
} catch (IllegalArgumentException e) {
  // populate subscription or topic in the config and retry
}

When it happens

Trigger: Creating a Pubsub read SchemaTransform via PubsubReadSchemaTransformProvider.from(configuration) with a configuration object where neither getSubscription() nor getTopic() is set — e.g. empty YAML/config mapping or programmatically built configuration with no fields populated.

Common situations: YAML pipeline definitions missing the 'subscription' or 'topic' key, typo'd config keys (e.g. 'topicName'), or building the configuration bean in Java and forgetting setSubscription()/setTopic().

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• Need to set either the topic or the subscription for a Pubsu
• Can't set both the topic and the subscription for a PubsubIO
• Unrecognized value for stable unique names:
• Failed to validate %s
• Secret option string cannot be null

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/09ec3721a2fd276e. Report an issue: GitHub.