apache/iceberg · error · UnsupportedOperationException

Unsupported type - short

Error message

Unsupported type - short

What it means

IcebergArrowColumnVector does not produce short-typed Arrow accessors, so getShort() always throws UnsupportedOperationException. Iceberg reads 16-bit values through other paths, making short access unsupported on this vector.

Source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/IcebergArrowColumnVector.java:95

  @Override
  public boolean isNullAt(int rowId) {
    return nullabilityHolder.isNullAt(rowId) == 1;
  }

  @Override
  public boolean getBoolean(int rowId) {
    return accessor.getBoolean(rowId);
  }

  @Override
  public byte getByte(int rowId) {
    throw new UnsupportedOperationException("Unsupported type - byte");
  }

  @Override
  public short getShort(int rowId) {
    throw new UnsupportedOperationException("Unsupported type - short");
  }

  @Override
  public int getInt(int rowId) {
    return accessor.getInt(rowId);
  }

  @Override
  public long getLong(int rowId) {
    return accessor.getLong(rowId);
  }

  @Override
  public float getFloat(int rowId) {
    return accessor.getFloat(rowId);
  }

  @Override

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Read the column as IntegerType and cast to short after the scan
2. Disable vectorized reads for the query (read.split.vectorization.enabled=false)
3. Adjust the table schema to avoid short-mapped columns if feasible
4. Upgrade Iceberg — short accessor support is revisited across releases

Example fix

// before
val df = spark.table("t") // smallint column, vectorized read
// after
spark.conf.set("read.split.vectorization.enabled", "false")
val df = spark.table("t").withColumn("c", col("c").cast("short"))

if (schema.fields().anyMatch(f -> f.dataType() == ShortType)) { spark.conf.set("read.split.vectorization.enabled", "false"); }

boolean shortSafe(ColumnVector v) { return !(v instanceof IcebergArrowColumnVector); }

try { s = vector.getShort(rowId); } catch (UnsupportedOperationException e) { int widened = vector.getInt(rowId); s = (short) widened; }

When it happens

Trigger: Spark's columnar reader invokes getShort(rowId) on an Arrow-backed vector for a column resolved to Spark ShortType in a vectorized batch scan.

Common situations: Tables whose schema maps to Spark ShortType being read with vectorization enabled; Spark plans that cast to short inside the batch reader.

Related errors

• Unsupported type - byte
• Unsupported type - byte
• Unsupported type - short
• Cannot read unsupported column types:
• Unsupported type: boolean

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/4389f0067cdc3a17. Report an issue: GitHub.