apache/iceberg · error · UnsupportedOperationException

Unsupported variant: shredded typed_value array

Error message

Unsupported variant: shredded typed_value array

What it means

The vectorized Variant reader supports reading shredded Variant columns only up to primitives and typed_value scalars. When a shredded array typed_value is encountered (a Parquet group with a repeated element), this visitor hook deliberately throws UnsupportedOperationException because the vectorized path has no implementation for shredded arrays yet.

Source

Thrown at arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorizedVariantVisitor.java:120

      GroupType value, VectorizedReader<?> valueResult, VectorizedReader<?> typedResult) {
    if (typedResult != null) {
      throw new UnsupportedOperationException(
          "Unsupported variant: shredded typed_value primitive");
    }
    return valueResult;
  }

  @Override
  public VectorizedReader<?> object(
      GroupType object, VectorizedReader<?> valueResult, List<VectorizedReader<?>> fieldResults) {
    throw new UnsupportedOperationException(
        "Unsupported variant: shredded typed_value object with " + fieldResults.size() + " fields");
  }

  @Override
  public VectorizedReader<?> array(
      GroupType array, VectorizedReader<?> valueResult, VectorizedReader<?> elementResult) {
    throw new UnsupportedOperationException("Unsupported variant: shredded typed_value array");
  }

  private ColumnDescriptor resolveDescriptor(PrimitiveType primitive) {
    // Build full column path: variant group path + primitive name
    // e.g., ["v1", "metadata"] or ["v2", "value"]
    String[] path = new String[variantGroupPath.length + 1];
    System.arraycopy(variantGroupPath, 0, path, 0, variantGroupPath.length);
    path[variantGroupPath.length] = primitive.getName();

    try {
      return parquetSchema.getColumnDescription(path);
    } catch (InvalidRecordException e) {
      return null;
    }
  }

  private Types.NestedField findVariantField(GroupType variant) {
    if (variant.getId() != null) {

View on GitHub (pinned to 86d9c8fc54)

Solutions

  1. Fall back to the non-vectorized variant reader for these files (disable vectorization for the variant column).
  2. Rewrite the data to keep array variant values unshredded so the generic Variant path handles them.
  3. Check for an updated Iceberg version implementing shredded array support in the vectorized reader.

Example fix

// before (vectorized read)
SparkContext.getConf().set("spark.sql.iceberg.handle-timestamp-without-timezone", ...)
.read.format("iceberg").load("t") // vectorized variant read fails on shredded array
// after
spark.read.format("iceberg").option("vectorization-enabled", "false").load("t")
Defensive patterns

Strategy: fallback

Validate before calling

// Inspect shredded variant schema before enabling vectorization
boolean hasShreddedArray = variantSchema.fields().stream()
    .anyMatch(f -> f.type() instanceof Types.NestedField
        && ((Types.NestedField) f.type()).type().isListType());
if (hasShreddedArray) { /* disable vectorization or use generic variant reader */ }

Try / catch

// catch (UnsupportedOperationException e) {
//   if (e.getMessage().contains("shredded typed_value")) { fallbackToNonVectorizedRead(); }
//   else throw e;
// }

Prevention

When it happens

Trigger: Reading a Parquet Variant column whose shredded variant_metadata/variant_value group contains an array-typed typed_value field, via the vectorized reader (VectorizedVariantVisitor.array).

Common situations: Tables written with variant shredding that produced ARRAY values; queries using vectorized reads against schema-evolved Variant data containing arrays.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/bcddf4df8f67e1de. Report an issue: GitHub.