apache/beam · error · IllegalArgumentException

Invalid ARRAY type: + originalSpannerType

Error message

Invalid ARRAY type: + originalSpannerType

What it means

SpannerSchema.parseSpannerType parses a Spanner column type string for the GOOGLE_SQL dialect; for ARRAY<...> types it applies a regex to extract the element type, and throws IllegalArgumentException("Invalid ARRAY type: ...") when the regex fails to match. This usually means the type string is malformed or an array-of-array/proto form the parser doesn't support.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerSchema.java:223

            return Type.string();
          }
          if (spannerType.startsWith("BYTES")) {
            return Type.bytes();
          }
          if (spannerType.startsWith("ARRAY")) {
            // find 'xxx' in string ARRAY<xxxxx>
            // Graph DBs may have suffixes, eg ARRAY<FLOAT32>(vector_length=>256)
            //
            Pattern pattern = Pattern.compile("ARRAY<([^>]+)>");
            Matcher matcher = pattern.matcher(originalSpannerType);

            if (matcher.find()) {
              String spannerArrayType = matcher.group(1).trim();
              Type itemType = parseSpannerType(spannerArrayType, dialect);
              return Type.array(itemType);
            } else {
              // Handle the case where the regex doesn't match (invalid ARRAY type)
              throw new IllegalArgumentException("Invalid ARRAY type: " + originalSpannerType);
            }
          }
          if (spannerType.startsWith("PROTO")) {
            // Substring "PROTO<xxx>"
            String spannerProtoType =
                originalSpannerType.substring(6, originalSpannerType.length() - 1);
            return Type.proto(spannerProtoType);
          }
          if (spannerType.startsWith("ENUM")) {
            // Substring "ENUM<xxx>"
            String spannerEnumType =
                originalSpannerType.substring(5, originalSpannerType.length() - 1);
            return Type.protoEnum(spannerEnumType);
          }
          throw new IllegalArgumentException("Unknown spanner type " + spannerType);
        case POSTGRESQL:
          Pattern pattern = Pattern.compile("([^\\[]+)\\[\\]");
          Matcher m = pattern.matcher(spannerType);

View on GitHub (pinned to 12126d8942)

Solutions

1. Inspect the offending column type from the message and check if it is a nested array; Beam's Spanner schema mapper only handles one level — flatten the schema via a view/TVF.
2. Upgrade Beam to the latest version, which supports more Spanner types.
3. Exclude the unsupported column from the columns list when configuring the Spanner read.
4. If the type string looks genuinely malformed, fix the table DDL in Spanner.

Example fix

// before: schema has ARRAY<ARRAY<INT64>> column
SpannerSchema.create(spannerSchema, Dialect.GOOGLESQL); // throws
// after: project out the nested column
SpannerRead.of(config).withColumns("id", "name"); // exclude nested-array column

// Skip array columns the mapper can't handle
List<String> safeColumns = columns.stream()
    .filter(c -> !c.type.toUpperCase().startsWith("ARRAY<ARRAY"))
    .map(c -> c.name).collect(toList());

try { schema = SpannerSchema.create(spannerSchema, dialect); } catch (IllegalArgumentException e) { if (e.getMessage().startsWith("Invalid ARRAY type")) { /* drop or flatten that column */ } throw e; }

When it happens

Trigger: Reading a Spanner schema (SpannerSchema.create / schema retrieval for SQL sources) where a column's declared type is an ARRAY whose text doesn't match the expected ARRAY<x> pattern — e.g., ARRAY<ARRAY<INT64>> nested arrays, malformed DDL, or types like ARRAY<PROTO<...>> the parser can't decompose.

Common situations: Schemas using nested arrays (Spanner supports ARRAY<STRUCT<...>> but nesting is limited); newer Spanner types (e.g., VECTOR) appearing in ARRAY position not supported by the Beam version; hand-written/DDL-migrated type strings with unusual spacing.

Related errors

• Unknown spanner type + spannerType
• Null collection element type at field {}
• Exception while trying to retrieve schema
• Cannot find Spanner table.
• Exception while trying to retrieve schema

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c38e182d4534466a. Report an issue: GitHub.