apache/iceberg · error · UnsupportedOperationException

Not implemented: copyWithoutStats

Error message

Not implemented: copyWithoutStats

What it means

SparkContentFile does not implement copyWithoutStats(), the method used to clone a file record while dropping statistics for memory reuse in manifest scanning. Because the wrapper delegates to a shared InternalRow, it cannot cheaply produce a stats-less copy, so it throws UnsupportedOperationException.

Solutions

  1. Use core implementations (GenericDataFile / BaseFile) when copyWithoutStats is required.
  2. Read needed fields directly from the wrapper instead of copying.
  3. Wrap results into core file objects before handing them to APIs that call copyWithoutStats.

Example fix

// before
ContentFile<?> statFree = sparkFile.copyWithoutStats(); // throws
// after
ContentFile<?> statFree = genericDataFile.copyWithoutStats(); // core DataFile
Defensive patterns

Strategy: fallback

Type guard

boolean supportsCopyWithoutStats(org.apache.iceberg.ContentFile<?> f) {
  return !(f instanceof org.apache.iceberg.spark.SparkContentFile);
}

Try / catch

try {
  statFree = file.copyWithoutStats();
} catch (UnsupportedOperationException e) {
  statFree = toCoreDataFile(file).copyWithoutStats();
}

Prevention

When it happens

Trigger: Calling copyWithoutStats() on a SparkContentFile/SparkDeleteFile, typically in parallel manifest-reading or delete-file grouping code that holds references and needs stat-free copies.

Common situations: Custom scan planning or delete-file loaders built on the Spark wrapper; code copying the ManifestReader pattern of copyWithoutStats onto Spark-backed files.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/b93fb91b5fd3f68e. Report an issue: GitHub.

Appendix: source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/SparkContentFile.java:222

  public Map<Integer, ByteBuffer> upperBounds() {
    Map<?, ?> upperBounds =
        wrapped.isNullAt(upperBoundsPosition) ? null : wrapped.getJavaMap(upperBoundsPosition);
    return convert(upperBoundsType, upperBounds);
  }

  @Override
  public ByteBuffer keyMetadata() {
    return convert(keyMetadataType, wrapped.get(keyMetadataPosition));
  }

  @Override
  public F copy() {
    throw new UnsupportedOperationException("Not implemented: copy");
  }

  @Override
  public F copyWithoutStats() {
    throw new UnsupportedOperationException("Not implemented: copyWithoutStats");
  }

  @Override
  public List<Long> splitOffsets() {
    return wrapped.isNullAt(splitOffsetsPosition) ? null : wrapped.getList(splitOffsetsPosition);
  }

  @Override
  public Integer sortOrderId() {
    return wrapped.getAs(sortOrderIdPosition);
  }

  @Override
  public List<Integer> equalityFieldIds() {
    return wrapped.isNullAt(equalityIdsPosition) ? null : wrapped.getList(equalityIdsPosition);
  }

  public String referencedDataFile() {

View on GitHub (pinned to 86d9c8fc54)