{"record":{"id":"4389f0067cdc3a17","repo":"apache/iceberg","slug":"unsupported-type-short-4389f0","errorCode":null,"errorMessage":"Unsupported type - short","messagePattern":"Unsupported type - short","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/IcebergArrowColumnVector.java","lineNumber":95,"sourceCode":"\n  @Override\n  public boolean isNullAt(int rowId) {\n    return nullabilityHolder.isNullAt(rowId) == 1;\n  }\n\n  @Override\n  public boolean getBoolean(int rowId) {\n    return accessor.getBoolean(rowId);\n  }\n\n  @Override\n  public byte getByte(int rowId) {\n    throw new UnsupportedOperationException(\"Unsupported type - byte\");\n  }\n\n  @Override\n  public short getShort(int rowId) {\n    throw new UnsupportedOperationException(\"Unsupported type - short\");\n  }\n\n  @Override\n  public int getInt(int rowId) {\n    return accessor.getInt(rowId);\n  }\n\n  @Override\n  public long getLong(int rowId) {\n    return accessor.getLong(rowId);\n  }\n\n  @Override\n  public float getFloat(int rowId) {\n    return accessor.getFloat(rowId);\n  }\n\n  @Override","sourceCodeStart":77,"sourceCodeEnd":113,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/IcebergArrowColumnVector.java#L77-L113","documentation":"IcebergArrowColumnVector does not produce short-typed Arrow accessors, so getShort() always throws UnsupportedOperationException. Iceberg reads 16-bit values through other paths, making short access unsupported on this vector.","triggerScenarios":"Spark's columnar reader invokes getShort(rowId) on an Arrow-backed vector for a column resolved to Spark ShortType in a vectorized batch scan.","commonSituations":"Tables whose schema maps to Spark ShortType being read with vectorization enabled; Spark plans that cast to short inside the batch reader.","solutions":["Read the column as IntegerType and cast to short after the scan","Disable vectorized reads for the query (read.split.vectorization.enabled=false)","Adjust the table schema to avoid short-mapped columns if feasible","Upgrade Iceberg — short accessor support is revisited across releases"],"exampleFix":"// before\nval df = spark.table(\"t\") // smallint column, vectorized read\n// after\nspark.conf.set(\"read.split.vectorization.enabled\", \"false\")\nval df = spark.table(\"t\").withColumn(\"c\", col(\"c\").cast(\"short\"))","handlingStrategy":"type-guard","validationCode":"if (schema.fields().anyMatch(f -> f.dataType() == ShortType)) { spark.conf.set(\"read.split.vectorization.enabled\", \"false\"); }","typeGuard":"boolean shortSafe(ColumnVector v) { return !(v instanceof IcebergArrowColumnVector); }","tryCatchPattern":"try { s = vector.getShort(rowId); } catch (UnsupportedOperationException e) { int widened = vector.getInt(rowId); s = (short) widened; }","preventionTips":["Read Iceberg integers as int and cast to short after the scan","Validate schema for ShortType columns when enabling vectorized reads"],"tags":["spark","vectorized-read","arrow","unsupported-type"],"backgroundTag":"unsupported-dtype","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}