apache/iceberg · error · IllegalArgumentException

Invalid primitive type for decimal

Error message

Invalid primitive type for decimal: ${primitiveType}

What it means

Thrown by VariantWriterBuilder's decimal logical-type visitor when the typed_value column's physical primitive is not one of the decimal-capable primitives the writer supports (INT32/INT64, per the reader's DECIMAL4/DECIMAL8 mapping). The builder cannot create a decimal Variant writer for the given primitive type.

Solutions

1. Restrict shredded decimals to precision that fits INT32 (<=9) or INT64 (<=18) and back them with those primitives.
2. Leave higher-precision decimals unshredded inside the Variant binary.
3. Adjust the shredding-schema derivation to exclude decimals that cannot be backed by INT32/INT64.

Example fix

// before
Types.required(PrimitiveTypeName.BINARY).as(DecimalLogicalType.decimalType(38, 12)).named("typed_value")

// after
Types.required(PrimitiveTypeName.INT64).as(DecimalLogicalType.decimalType(18, 12)).named("typed_value")

PrimitiveTypeName p = desc.getPrimitiveType().getPrimitiveTypeName();
Preconditions.checkArgument(p == INT32 || p == INT64,
    "Decimal typed_value must be INT32/INT64-backed, got %s", p);

boolean isWritableShreddedDecimal(Types.NestedField f) {
  return switch (f.type().typeId()) {
    case DECIMAL d -> d.precision() <= 18;
    default -> false;
  };
}

try {
  buildVariantWriter(schema);
} catch (IllegalArgumentException e) {
  if (e.getMessage().startsWith("Invalid primitive type for decimal")) {
    excludeFromShredding(columnName);
  } else throw e;
}

When it happens

Trigger: visit(DecimalLogicalTypeAnnotation) where desc.getPrimitiveType() is neither of the supported decimal-backed primitives (e.g. FLBA/BINARY-backed decimal with high precision) while writing a shredded Variant.

Common situations: Shredding decimals with precision > 18 that default to fixed-length backing; schema propagation from engines that emit BINARY decimals; misconfigured shredding schemas.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Invalid primitive type for decimal
• Invalid bit width for int
• Invalid bit width for int
• Unknown primitive physical type
• Unsupported shredded value type

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/668af1a62e026db5. Report an issue: GitHub.

Appendix: source

Thrown at parquet/src/main/java/org/apache/iceberg/parquet/VariantWriterBuilder.java:232

              ParquetVariantWriters.decimal(
                  ParquetValueWriters.decimalAsLong(
                      desc, decimal.getPrecision(), decimal.getScale()),
                  decimal.getPrecision(),
                  decimal.getScale(),
                  PhysicalType.DECIMAL8);
          return Optional.of(writer);
        case INT32:
          writer =
              ParquetVariantWriters.decimal(
                  ParquetValueWriters.decimalAsInteger(
                      desc, decimal.getPrecision(), decimal.getScale()),
                  decimal.getPrecision(),
                  decimal.getScale(),
                  PhysicalType.DECIMAL4);
          return Optional.of(writer);
      }

      throw new IllegalArgumentException(
          "Invalid primitive type for decimal: " + desc.getPrimitiveType());
    }

    @Override
    public Optional<ParquetValueWriter<?>> visit(DateLogicalTypeAnnotation ignored) {
      ParquetValueWriter<VariantValue> writer =
          ParquetVariantWriters.primitive(ParquetValueWriters.ints(desc), PhysicalType.DATE);
      return Optional.of(writer);
    }

    @Override
    public Optional<ParquetValueWriter<?>> visit(TimeLogicalTypeAnnotation time) {
      ParquetValueWriter<VariantValue> writer =
          ParquetVariantWriters.primitive(ParquetValueWriters.longs(desc), PhysicalType.TIME);
      return Optional.of(writer);
    }

    @Override

View on GitHub (pinned to 86d9c8fc54)