Couldn't set Arrow properties, which may impact read perform

Error message

Couldn't set Arrow properties, which may impact read performance

What it means

Static-initializer WARN in VectorizedSparkParquetReaders: the class tries to set Arrow/Spark unsafe-memory and null-check flags (enableUnsafeMemoryAccess, disableNullCheckForGet) via reflection to speed up vectorized reads. If any of these fail — typically a Spark/Arrow version incompatibility — it logs this warning with the cause and continues with slower safe paths rather than crashing.

Source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/VectorizedSparkParquetReaders.java:49

import org.apache.iceberg.spark.SparkUtil;
import org.apache.parquet.schema.MessageType;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class VectorizedSparkParquetReaders {

  private static final Logger LOG = LoggerFactory.getLogger(VectorizedSparkParquetReaders.class);
  private static final String ENABLE_UNSAFE_MEMORY_ACCESS = "arrow.enable_unsafe_memory_access";
  private static final String ENABLE_UNSAFE_MEMORY_ACCESS_ENV = "ARROW_ENABLE_UNSAFE_MEMORY_ACCESS";
  private static final String ENABLE_NULL_CHECK_FOR_GET = "arrow.enable_null_check_for_get";
  private static final String ENABLE_NULL_CHECK_FOR_GET_ENV = "ARROW_ENABLE_NULL_CHECK_FOR_GET";

  static {
    try {
      enableUnsafeMemoryAccess();
      disableNullCheckForGet();
    } catch (Exception e) {
      LOG.warn("Couldn't set Arrow properties, which may impact read performance", e);
    }
  }

  private VectorizedSparkParquetReaders() {}

  public static ColumnarBatchReader buildReader(
      Schema expectedSchema,
      MessageType fileSchema,
      Map<Integer, ?> idToConstant,
      BufferAllocator bufferAllocator) {
    return (ColumnarBatchReader)
        TypeWithSchemaVisitor.visit(
            expectedSchema.asStruct(),
            fileSchema,
            new ReaderBuilder(
                expectedSchema,
                fileSchema,
                NullCheckingForGet.NULL_CHECKING_ENABLED,

View on GitHub (pinned to 86d9c8fc54)

Solutions

Align the Iceberg runtime jar's Spark version (iceberg-spark-runtime-<sparkVer>) with the cluster's Spark version.
Check the logged cause for the exact reflective call that failed and verify the corresponding Arrow class exists at that version.
Accept the warning if correctness matters more than read performance — vectorized reads still work, just slower.
If a security manager/JPMS blocks Unsafe, relax module access or use the non-vectorized reader.

Example fix

// before (Spark 3.5 cluster)
spark.jars.packages org.apache.iceberg:iceberg-spark-runtime-4.0_2.13:...
// after: match runtime to Spark
spark.jars.packages org.apache.iceberg:iceberg-spark-runtime-3.5_2.13:...

Defensive patterns

Strategy: fallback

Validate before calling

// verify jar matches Spark: check iceberg-spark-runtime-<sparkVersion>_2.13 on classpath

Try / catch

try { Class.forName("org.apache.iceberg.spark.data.vectorized.VectorizedSparkParquetReaders"); } catch (Throwable t) { /* fall back to row-based parquet reader */ }

Prevention

Pin iceberg-spark-runtime to the exact cluster Spark version
After Spark upgrades, re-check vectorized read warnings
Treat the warning as a perf signal, not a correctness issue

When it happens

Trigger: Class-loading VectorizedSparkParquetReaders on a Spark build where the internal Arrow APIs it patches differ (Spark version mismatch between iceberg-spark module and the runtime Spark), or a security manager blocking unsafe access.

Common situations: Running iceberg-spark-4.0 jar on a mismatched Spark/Arrow version; shaded or custom Arrow distributions; restricted JVM environments where sun.misc.Unsafe access is limited.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/849627cdda8ac454. Report an issue: GitHub.