{"record":{"id":"849627cdda8ac454","repo":"apache/iceberg","slug":"couldn-t-set-arrow-properties-which-may-impact-re-849627","errorCode":null,"errorMessage":"Couldn't set Arrow properties, which may impact read performance","messagePattern":"Couldn't set Arrow properties, which may impact read performance","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/VectorizedSparkParquetReaders.java","lineNumber":49,"sourceCode":"import org.apache.iceberg.spark.SparkUtil;\nimport org.apache.parquet.schema.MessageType;\nimport org.slf4j.Logger;\nimport org.slf4j.LoggerFactory;\n\npublic class VectorizedSparkParquetReaders {\n\n  private static final Logger LOG = LoggerFactory.getLogger(VectorizedSparkParquetReaders.class);\n  private static final String ENABLE_UNSAFE_MEMORY_ACCESS = \"arrow.enable_unsafe_memory_access\";\n  private static final String ENABLE_UNSAFE_MEMORY_ACCESS_ENV = \"ARROW_ENABLE_UNSAFE_MEMORY_ACCESS\";\n  private static final String ENABLE_NULL_CHECK_FOR_GET = \"arrow.enable_null_check_for_get\";\n  private static final String ENABLE_NULL_CHECK_FOR_GET_ENV = \"ARROW_ENABLE_NULL_CHECK_FOR_GET\";\n\n  static {\n    try {\n      enableUnsafeMemoryAccess();\n      disableNullCheckForGet();\n    } catch (Exception e) {\n      LOG.warn(\"Couldn't set Arrow properties, which may impact read performance\", e);\n    }\n  }\n\n  private VectorizedSparkParquetReaders() {}\n\n  public static ColumnarBatchReader buildReader(\n      Schema expectedSchema,\n      MessageType fileSchema,\n      Map<Integer, ?> idToConstant,\n      BufferAllocator bufferAllocator) {\n    return (ColumnarBatchReader)\n        TypeWithSchemaVisitor.visit(\n            expectedSchema.asStruct(),\n            fileSchema,\n            new ReaderBuilder(\n                expectedSchema,\n                fileSchema,\n                NullCheckingForGet.NULL_CHECKING_ENABLED,","sourceCodeStart":31,"sourceCodeEnd":67,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/VectorizedSparkParquetReaders.java#L31-L67","documentation":"Static-initializer WARN in VectorizedSparkParquetReaders: the class tries to set Arrow/Spark unsafe-memory and null-check flags (enableUnsafeMemoryAccess, disableNullCheckForGet) via reflection to speed up vectorized reads. If any of these fail — typically a Spark/Arrow version incompatibility — it logs this warning with the cause and continues with slower safe paths rather than crashing.","triggerScenarios":"Class-loading VectorizedSparkParquetReaders on a Spark build where the internal Arrow APIs it patches differ (Spark version mismatch between iceberg-spark module and the runtime Spark), or a security manager blocking unsafe access.","commonSituations":"Running iceberg-spark-4.0 jar on a mismatched Spark/Arrow version; shaded or custom Arrow distributions; restricted JVM environments where sun.misc.Unsafe access is limited.","solutions":["Align the Iceberg runtime jar's Spark version (iceberg-spark-runtime-<sparkVer>) with the cluster's Spark version.","Check the logged cause for the exact reflective call that failed and verify the corresponding Arrow class exists at that version.","Accept the warning if correctness matters more than read performance — vectorized reads still work, just slower.","If a security manager/JPMS blocks Unsafe, relax module access or use the non-vectorized reader."],"exampleFix":"// before (Spark 3.5 cluster)\nspark.jars.packages org.apache.iceberg:iceberg-spark-runtime-4.0_2.13:...\n// after: match runtime to Spark\nspark.jars.packages org.apache.iceberg:iceberg-spark-runtime-3.5_2.13:...","handlingStrategy":"fallback","validationCode":"// verify jar matches Spark: check iceberg-spark-runtime-<sparkVersion>_2.13 on classpath","typeGuard":null,"tryCatchPattern":"try { Class.forName(\"org.apache.iceberg.spark.data.vectorized.VectorizedSparkParquetReaders\"); } catch (Throwable t) { /* fall back to row-based parquet reader */ }","preventionTips":["Pin iceberg-spark-runtime to the exact cluster Spark version","After Spark upgrades, re-check vectorized read warnings","Treat the warning as a perf signal, not a correctness issue"],"tags":["logging","spark","arrow","vectorized-read","performance"],"backgroundTag":"dependency-version-incompatible","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}