apache/beam · error · IllegalArgumentException

Unknown spanner type + spannerType

Error message

Unknown spanner type + spannerType

What it means

In SpannerSchema.parseSpannerType, after exhausting the known GOOGLE_SQL dialect type names (including PROTO and ENUM prefixes), any unrecognized type string throws IllegalArgumentException("Unknown spanner type " + spannerType). The parser has a hardcoded list of Spanner types and fails on anything newer or nonstandard.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerSchema.java:238

              return Type.array(itemType);
            } else {
              // Handle the case where the regex doesn't match (invalid ARRAY type)
              throw new IllegalArgumentException("Invalid ARRAY type: " + originalSpannerType);
            }
          }
          if (spannerType.startsWith("PROTO")) {
            // Substring "PROTO<xxx>"
            String spannerProtoType =
                originalSpannerType.substring(6, originalSpannerType.length() - 1);
            return Type.proto(spannerProtoType);
          }
          if (spannerType.startsWith("ENUM")) {
            // Substring "ENUM<xxx>"
            String spannerEnumType =
                originalSpannerType.substring(5, originalSpannerType.length() - 1);
            return Type.protoEnum(spannerEnumType);
          }
          throw new IllegalArgumentException("Unknown spanner type " + spannerType);
        case POSTGRESQL:
          Pattern pattern = Pattern.compile("([^\\[]+)\\[\\]");
          Matcher m = pattern.matcher(spannerType);
          if (m.find()) {
            // Substring "xxx[]" or "xxx[] vector length yyy"
            // Must check array type first
            String spannerArrayType = m.group(1);
            Type itemType = parseSpannerType(spannerArrayType, dialect);
            return Type.array(itemType);
          }
          type = POSTGRES_TYPE_MAP.get(spannerType);
          if (type != null) {
            return type;
          }
          if (spannerType.startsWith("CHARACTER VARYING")) {
            return Type.string();
          }
          if (spannerType.startsWith("NUMERIC")) {

View on GitHub (pinned to 12126d8942)

Solutions

1. Upgrade beam-sdks-java-io-google-cloud-platform to a version that supports the type shown in the message.
2. Exclude the unsupported column via withColumns(...) on the read transform.
3. Check the message for the exact type and confirm its support status in your Beam version's SpannerSchema source.
4. If it's a custom/unexpected type string, fix the column DDL.

Example fix

// before: table has TOKENLIST column, older Beam
SpannerSchema.create(schema, Dialect.GOOGLESQL); // throws Unknown spanner type TOKENLIST
// after
SpannerRead.of(config).withColumns("id", "payload"); // skip the TOKENLIST column

Set<String> supported = Set.of("BOOL","BYTES","DATE","FLOAT64","FLOAT32","INT64","NUMERIC","STRING","STRUCT","TIMESTAMP","JSON","PROTO","ENUM");
if (columns.stream().anyMatch(c -> !supported.contains(c.type.toUpperCase()))) { /* exclude or upgrade */ }

try { schema = SpannerSchema.create(s, dialect); } catch (IllegalArgumentException e) { if (e.getMessage().startsWith("Unknown spanner type")) { LOG.error("Unsupported type: {}", e.getMessage()); } throw e; }

When it happens

Trigger: Schema inference (SpannerSchema.create / itemType) against a GOOGLE_SQL database containing a column type not in the parser's list — e.g., newly introduced Spanner types (VECTOR, TOKENLIST) or an older Beam SDK predating JSON/PROTO support.

Common situations: Using an older Beam version against a database created with newer Spanner features; querying a table with experimental types; migrating schemas across environments with divergent Spanner feature versions.

Related errors

• Invalid ARRAY type: + originalSpannerType
• Null collection element type at field {}
• Exception while trying to retrieve schema
• Cannot find Spanner table.
• Exception while trying to retrieve schema

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e93691a419b3c657. Report an issue: GitHub.