apache/iceberg · error · java.lang.UnsupportedOperationException

Cannot convert Spark geography with mixed SRID to Iceberg

Error message

Cannot convert Spark geography with mixed SRID to Iceberg

What it means

SparkTypeToType rejects Spark GeographyType columns whose coordinates use mixed SRIDs, since an Iceberg GeographyType must carry exactly one CRS/SRID. The UnsupportedOperationException is thrown during Spark-to-Iceberg type conversion instead of producing a lossy mapping.

Source

Thrown at spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/SparkTypeToType.java:180

    } else if (atomic instanceof TimestampNTZType) {
      return Types.TimestampType.withoutZone();

    } else if (atomic instanceof DecimalType) {
      return Types.DecimalType.of(
          ((DecimalType) atomic).precision(), ((DecimalType) atomic).scale());
    } else if (atomic instanceof BinaryType) {
      return Types.BinaryType.get();
    } else if (atomic instanceof GeometryType) {
      GeometryType geometry = (GeometryType) atomic;
      if (geometry.isMixedSrid()) {
        throw new UnsupportedOperationException(
            "Cannot convert Spark geometry with mixed SRID to Iceberg");
      }
      return Types.GeometryType.of(geometry.crs());
    } else if (atomic instanceof GeographyType) {
      GeographyType geography = (GeographyType) atomic;
      if (geography.isMixedSrid()) {
        throw new UnsupportedOperationException(
            "Cannot convert Spark geography with mixed SRID to Iceberg");
      }
      return Types.GeographyType.of(geography.crs(), convertAlgorithm(geography.algorithm()));
    } else if (atomic instanceof NullType) {
      return Types.UnknownType.get();
    }

    throw new UnsupportedOperationException("Not a supported type: " + atomic.catalogString());
  }

  // Translates Spark's edge-interpolation algorithm to Iceberg's, mirroring
  // TypeToSparkType#convertAlgorithm. Spark supports only the spherical algorithm today; anything
  // else is rejected loudly rather than silently defaulting, so a new Spark algorithm surfaces here
  // instead of being dropped.
  private static EdgeAlgorithm convertAlgorithm(EdgeInterpolationAlgorithm algorithm) {
    switch (algorithm.toString().toUpperCase(Locale.ROOT)) {
      case "SPHERICAL":
        return EdgeAlgorithm.SPHERICAL;

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Reproject all geography values to one SRID (typically EPSG:4326) before conversion
2. Separate values by SRID into distinct geography columns
3. Convert to geometry or binary if a single-SRID geography contract cannot be enforced
4. Add an upstream validation step asserting a single SRID before writing

Example fix

// before
// schema: geography(geom) with mixed SRIDs -> throws on conversion
// after
df.withColumn("geom", functions.expr("st_transform(geom, 'EPSG:4326')")); // uniform SRID geography

if (sparkGeogType.isMixedSrid()) {
  throw new IllegalStateException("Normalize geography SRID before Iceberg conversion");
}

if (dt instanceof org.apache.spark.sql.types.GeographyType t && t.isMixedSrid()) {
  return false; // not convertible until reprojected
}

try {
  icebergType = converter.convert(sparkSchema);
} catch (UnsupportedOperationException e) {
  if (e.getMessage().contains("mixed SRID")) {
    // reproject to one SRID (typically EPSG:4326) and retry
  }
}

When it happens

Trigger: Schema conversion of a Spark DataFrame containing a geography column where geometry instances reference more than one SRID.

Common situations: Ingesting geodetic data combined from different sources (e.g. EPSG:4326 and EPSG:4269) into one geography column; upstream libraries that leave SRID unset on some geometries, triggering mixed detection.

Related errors

• Cannot convert Spark geography with mixed SRID to Iceberg
• Cannot convert Spark geometry with mixed SRID to Iceberg
• Cannot convert Spark geometry with mixed SRID to Iceberg
• Iceberg does not support Spark geography edge algorithm:
• Iceberg does not support Spark geography edge algorithm:

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/66746dcc970eaeab. Report an issue: GitHub.