apache/iceberg · error · UnsupportedOperationException

Not implemented: copy

Error message

Not implemented: copy

What it means

SparkContentFile wraps a Spark InternalRow as a ContentFile and does not implement copy(). Copying requires constructing a fully independent file object with its own stats, which this wrapper deliberately does not support since it delegates to a shared row. Any code path that calls copy() on a Spark-backed ContentFile hits this UnsupportedOperationException.

Solutions

  1. Convert to a core DataFile/DeleteFile (e.g. via GenericDataFile builders) before copying.
  2. Restructure code so the underlying row is read in place instead of copying.
  3. If in your own catalog code, implement copy() by building a new instance from the wrapped row's fields.

Example fix

// before
ContentFile<?> owned = sparkFile.copy(); // throws
// after
DataFile owned = new GenericDataFile(
    sparkFile.location(), sparkFile.partition(), sparkFile.recordCount(),
    sparkFile.fileSizeInBytes(), sparkFile.fileOffsetInBytes(), sparkFile.rowPositions());
Defensive patterns

Strategy: fallback

Type guard

boolean supportsCopy(org.apache.iceberg.ContentFile<?> f) {
  return !(f instanceof org.apache.iceberg.spark.SparkContentFile);
}

Try / catch

try {
  owned = file.copy();
} catch (UnsupportedOperationException e) {
  owned = buildGenericDataFile(file); // convert wrapper to core file first
}

Prevention

When it happens

Trigger: Calling copy() on a SparkContentFile/SparkDeleteFile instance, e.g. when a scan or delete-processing API requires an owned copy of the file record rather than a live view over the row.

Common situations: Custom delete-file processing or scan planning code that reuses Iceberg patterns like copyWithoutStats on Spark wrapper objects; engine integration code assuming all ContentFile implementations support copy.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/60218884dcdfd552. Report an issue: GitHub.

Appendix: source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/SparkContentFile.java:217

        wrapped.isNullAt(lowerBoundsPosition) ? null : wrapped.getJavaMap(lowerBoundsPosition);
    return convert(lowerBoundsType, lowerBounds);
  }

  @Override
  public Map<Integer, ByteBuffer> upperBounds() {
    Map<?, ?> upperBounds =
        wrapped.isNullAt(upperBoundsPosition) ? null : wrapped.getJavaMap(upperBoundsPosition);
    return convert(upperBoundsType, upperBounds);
  }

  @Override
  public ByteBuffer keyMetadata() {
    return convert(keyMetadataType, wrapped.get(keyMetadataPosition));
  }

  @Override
  public F copy() {
    throw new UnsupportedOperationException("Not implemented: copy");
  }

  @Override
  public F copyWithoutStats() {
    throw new UnsupportedOperationException("Not implemented: copyWithoutStats");
  }

  @Override
  public List<Long> splitOffsets() {
    return wrapped.isNullAt(splitOffsetsPosition) ? null : wrapped.getList(splitOffsetsPosition);
  }

  @Override
  public Integer sortOrderId() {
    return wrapped.getAs(sortOrderIdPosition);
  }

  @Override

View on GitHub (pinned to 86d9c8fc54)