apache/beam · error · IllegalArgumentException

Unsupported format for Pubsub topic

Error message

Unsupported format for Pubsub topic: '{linkedResource}'

What it means

PubsubTableFactory.getLocation throws IllegalArgumentException when a Data Catalog entry's linkedResource URI path does not match the expected Pubsub topic pattern (topics/{project}/{topic}). The path after the first slash is used as the Pubsub location.

Solutions

1. Ensure the entry's linkedResource is a full Pubsub topic URL like projects/_/topics/my-topic (or projects/p/topics/t)
2. Check the URI path matches 'topics/<project>/<topic>' after the scheme/host
3. Use the correct TableFactory (GCS) for fileset entries

Example fix

// before
linkedResource: 'https://pubsub.googleapis.com/subscriptions/my-sub'
// after
linkedResource: 'https://pubsub.googleapis.com/projects/my-project/topics/my-topic'

URI uri = URI.create(entry.getLinkedResource());
String path = uri.getPath();
if (path == null || !path.matches("/topics/[^/]+/[^/]+")) {
  throw new IllegalArgumentException("Not a Pubsub topic linkedResource: " + entry.getLinkedResource());
}

static boolean isPubsubTopicEntry(Entry e) {
  String p = URI.create(e.getLinkedResource()).getPath();
  return p != null && p.startsWith("/topics/");
}

try {
  Table t = pubsubFactory.tableBuilder(entry);
} catch (IllegalArgumentException ex) {
  LOG.error("linkedResource must be a Pubsub topic URL", ex);
}

When it happens

Trigger: entry.getLinkedResource() parses to a URI whose path does not match PS_PATH_PATTERN, e.g. a non-Pubsub linked resource or a malformed topic URL.

Common situations: Using a PubsubTableFactory against a GCS fileset entry, a topic URL missing 'topics/', or a linkedResource with subscription instead of topic path.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Related errors

• A pubsub message attribute key must not exceed 256 bytes.
• A pubsub message attribute value must not exceed 1024 bytes
• A pubsub message data field must not exceed 10MB
• A pubsub message must not have more than 100 attributes.
• A pubsub message ordering key must not exceed 1024 bytes.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/abd6ac826d49047f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/sql/datacatalog/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/datacatalog/PubsubTableFactory.java:56

    if (!URI.create(entry.getLinkedResource()).getAuthority().equalsIgnoreCase(PUBSUB_API)) {
      return Optional.empty();
    }

    return Optional.of(
        Table.builder()
            .location(getLocation(entry))
            .properties(TableUtils.emptyProperties())
            .type("pubsub")
            .comment(""));
  }

  private static String getLocation(Entry entry) {
    URI entryName = URI.create(entry.getLinkedResource());
    String psPath = entryName.getPath();

    Matcher bqPathMatcher = PS_PATH_PATTERN.matcher(psPath);
    if (!bqPathMatcher.matches()) {
      throw new IllegalArgumentException(
          "Unsupported format for Pubsub topic: '" + entry.getLinkedResource() + "'");
    }

    return psPath.substring(1);
  }
}

View on GitHub (pinned to 12126d8942)