Couldn't set Arrow properties, which may impact read perform

Error message

Couldn't set Arrow properties, which may impact read performance

What it means

The VectorizedSparkParquetReaders class sets Arrow allocator properties (unsafe memory access, disabled null checks on get) in a static initializer to speed up vectorized reads. If setting them fails, it logs this warning and falls back to slower safe behavior rather than failing reads.

Source

Thrown at spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/VectorizedSparkParquetReaders.java:49

import org.apache.iceberg.spark.SparkUtil;
import org.apache.parquet.schema.MessageType;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class VectorizedSparkParquetReaders {

  private static final Logger LOG = LoggerFactory.getLogger(VectorizedSparkParquetReaders.class);
  private static final String ENABLE_UNSAFE_MEMORY_ACCESS = "arrow.enable_unsafe_memory_access";
  private static final String ENABLE_UNSAFE_MEMORY_ACCESS_ENV = "ARROW_ENABLE_UNSAFE_MEMORY_ACCESS";
  private static final String ENABLE_NULL_CHECK_FOR_GET = "arrow.enable_null_check_for_get";
  private static final String ENABLE_NULL_CHECK_FOR_GET_ENV = "ARROW_ENABLE_NULL_CHECK_FOR_GET";

  static {
    try {
      enableUnsafeMemoryAccess();
      disableNullCheckForGet();
    } catch (Exception e) {
      LOG.warn("Couldn't set Arrow properties, which may impact read performance", e);
    }
  }

  private VectorizedSparkParquetReaders() {}

  public static ColumnarBatchReader buildReader(
      Schema expectedSchema,
      MessageType fileSchema,
      Map<Integer, ?> idToConstant,
      BufferAllocator bufferAllocator) {
    return (ColumnarBatchReader)
        TypeWithSchemaVisitor.visit(
            expectedSchema.asStruct(),
            fileSchema,
            new ReaderBuilder(
                expectedSchema,
                fileSchema,
                NullCheckingForGet.NULL_CHECKING_ENABLED,

View on GitHub (pinned to 86d9c8fc54)

Solutions

Verify Arrow runtime version matches the one Iceberg was built against
Add JVM flags enabling unsafe access (e.g. --add-opens java.base/java.nio=ALL-UNNAMED)
Check the chained exception in the log to identify the failing property call

Defensive patterns

Strategy: fallback

Validate before calling

// check JVM unsafe access: boolean ok = io.netty.util.internal.PlatformDependent.hasUnsafe();

Try / catch

try { Class.forName("org.apache.iceberg.spark.data.vectorized.VectorizedSparkParquetReaders"); } catch (Throwable t) { /* falls back to non-vectorized path; add --add-opens flags */ }

Prevention

Run on a JDK where sun.misc.Unsafe is available
Pin the Arrow version matching the Iceberg Spark runtime
Add --add-opens java.base/java.nio=ALL-UNNAMED to driver/executor JVM options

When it happens

Trigger: Static class initialization where the Arrow reflection-based unsafe-access configuration throws (e.g. missing sun.misc.Unsafe access, restricted JDK modules, or Arrow version mismatch).

Common situations: Running on JDKs with --illegal-access denied / strong encapsulation without --add-opens, hardened JVMs, or shaded/conflicting Arrow versions on the classpath.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/03c5572d9acc8944. Report an issue: GitHub.