{"record":{"id":"e93691a419b3c657","repo":"apache/beam","slug":"unknown-spanner-type-spannertype","errorCode":null,"errorMessage":"Unknown spanner type + spannerType","messagePattern":"Unknown spanner type \\+ spannerType","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerSchema.java","lineNumber":238,"sourceCode":"              return Type.array(itemType);\n            } else {\n              // Handle the case where the regex doesn't match (invalid ARRAY type)\n              throw new IllegalArgumentException(\"Invalid ARRAY type: \" + originalSpannerType);\n            }\n          }\n          if (spannerType.startsWith(\"PROTO\")) {\n            // Substring \"PROTO<xxx>\"\n            String spannerProtoType =\n                originalSpannerType.substring(6, originalSpannerType.length() - 1);\n            return Type.proto(spannerProtoType);\n          }\n          if (spannerType.startsWith(\"ENUM\")) {\n            // Substring \"ENUM<xxx>\"\n            String spannerEnumType =\n                originalSpannerType.substring(5, originalSpannerType.length() - 1);\n            return Type.protoEnum(spannerEnumType);\n          }\n          throw new IllegalArgumentException(\"Unknown spanner type \" + spannerType);\n        case POSTGRESQL:\n          Pattern pattern = Pattern.compile(\"([^\\\\[]+)\\\\[\\\\]\");\n          Matcher m = pattern.matcher(spannerType);\n          if (m.find()) {\n            // Substring \"xxx[]\" or \"xxx[] vector length yyy\"\n            // Must check array type first\n            String spannerArrayType = m.group(1);\n            Type itemType = parseSpannerType(spannerArrayType, dialect);\n            return Type.array(itemType);\n          }\n          type = POSTGRES_TYPE_MAP.get(spannerType);\n          if (type != null) {\n            return type;\n          }\n          if (spannerType.startsWith(\"CHARACTER VARYING\")) {\n            return Type.string();\n          }\n          if (spannerType.startsWith(\"NUMERIC\")) {","sourceCodeStart":220,"sourceCodeEnd":256,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerSchema.java#L220-L256","documentation":"In SpannerSchema.parseSpannerType, after exhausting the known GOOGLE_SQL dialect type names (including PROTO and ENUM prefixes), any unrecognized type string throws IllegalArgumentException(\"Unknown spanner type \" + spannerType). The parser has a hardcoded list of Spanner types and fails on anything newer or nonstandard.","triggerScenarios":"Schema inference (SpannerSchema.create / itemType) against a GOOGLE_SQL database containing a column type not in the parser's list — e.g., newly introduced Spanner types (VECTOR, TOKENLIST) or an older Beam SDK predating JSON/PROTO support.","commonSituations":"Using an older Beam version against a database created with newer Spanner features; querying a table with experimental types; migrating schemas across environments with divergent Spanner feature versions.","solutions":["Upgrade beam-sdks-java-io-google-cloud-platform to a version that supports the type shown in the message.","Exclude the unsupported column via withColumns(...) on the read transform.","Check the message for the exact type and confirm its support status in your Beam version's SpannerSchema source.","If it's a custom/unexpected type string, fix the column DDL."],"exampleFix":"// before: table has TOKENLIST column, older Beam\nSpannerSchema.create(schema, Dialect.GOOGLESQL); // throws Unknown spanner type TOKENLIST\n// after\nSpannerRead.of(config).withColumns(\"id\", \"payload\"); // skip the TOKENLIST column","handlingStrategy":"validation","validationCode":"Set<String> supported = Set.of(\"BOOL\",\"BYTES\",\"DATE\",\"FLOAT64\",\"FLOAT32\",\"INT64\",\"NUMERIC\",\"STRING\",\"STRUCT\",\"TIMESTAMP\",\"JSON\",\"PROTO\",\"ENUM\");\nif (columns.stream().anyMatch(c -> !supported.contains(c.type.toUpperCase()))) { /* exclude or upgrade */ }","typeGuard":null,"tryCatchPattern":"try { schema = SpannerSchema.create(s, dialect); } catch (IllegalArgumentException e) { if (e.getMessage().startsWith(\"Unknown spanner type\")) { LOG.error(\"Unsupported type: {}\", e.getMessage()); } throw e; }","preventionTips":["Keep Beam SDK versions current relative to Spanner feature adoption.","Exclude exotic columns (TOKENLIST, VECTOR) from reads.","Check SpannerSchema source of your Beam version for the supported type list."],"tags":["java","beam","spanner","schema","type-parsing"],"backgroundTag":"unsupported-dtype","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}