{"record":{"id":"fdc4f56c2c4d0a24","repo":"apache/iceberg","slug":"spark-does-not-support-geography-edge-algorithm-fdc4f5","errorCode":null,"errorMessage":"Spark does not support geography edge algorithm: ","messagePattern":"Spark does not support geography edge algorithm: ","errorType":"exception","errorClass":"java.lang.UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/TypeToSparkType.java","lineNumber":192,"sourceCode":"    // IllegalArgumentException).\n    return GeometryType$.MODULE$.apply(geometry.crs());\n  }\n\n  private DataType geographyType(Types.GeographyType geography) {\n    // The spec requires a geography CRS to be geographic; Spark recognizes only OGC:CRS84, so any\n    // other CRS throws SparkIllegalArgumentException (an IllegalArgumentException).\n    return GeographyType$.MODULE$.apply(geography.crs(), convertAlgorithm(geography.algorithm()));\n  }\n\n  // Translates Iceberg's edge-interpolation algorithm to Spark's. Spark supports only the spherical\n  // algorithm (the Iceberg default); every other algorithm is rejected with a clear error. Shared\n  // with PruneColumnsWithoutReordering, which compares algorithms across the two type systems.\n  static EdgeInterpolationAlgorithm convertAlgorithm(EdgeAlgorithm algorithm) {\n    switch (algorithm) {\n      case SPHERICAL:\n        return SPARK_SPHERICAL;\n      default:\n        throw new UnsupportedOperationException(\n            \"Spark does not support geography edge algorithm: \" + algorithm);\n    }\n  }\n\n  private Metadata fieldMetadata(int fieldId) {\n    if (MetadataColumns.metadataFieldIds().contains(fieldId)) {\n      return new MetadataBuilder().putBoolean(METADATA_COL_ATTR_KEY, true).build();\n    }\n\n    return Metadata.empty();\n  }\n}\n","sourceCodeStart":174,"sourceCodeEnd":205,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/TypeToSparkType.java#L174-L205","documentation":"Iceberg geography columns carry an edge interpolation algorithm, but Spark's geography support only recognizes the spherical algorithm. convertAlgorithm throws when a geography column specifies any other edge algorithm.","triggerScenarios":"Reading a table whose schema has a geography column with an edge algorithm other than spherical (e.g. vincenty or a future spec algorithm) through the Spark type converter.","commonSituations":"Tables written by geospatial-enabled engines using non-spherical edges, later queried in Spark; spec evolution introducing new algorithms ahead of Spark support.","solutions":["Rewrite the geography column with edge algorithm spherical (or use geometry instead of geography)","Skip/exclude the geography column from Spark queries","Upgrade to an Iceberg/Spark version supporting the algorithm"],"exampleFix":"// before\nTypes.GeographyType.get(\"srid\", EdgeAlgorithm.VINCENTY);\n// after\nTypes.GeographyType.get(\"srid\", EdgeAlgorithm.SPHERICAL);","handlingStrategy":"validation","validationCode":"table.schema().columns().stream()\n  .filter(c -> c.type() instanceof Types.GeographyType)\n  .map(c -> (Types.GeographyType) c.type())\n  .forEach(g -> Preconditions.checkArgument(\n      g.algorithm() == null || g.algorithm().name().equals(\"SPHERICAL\"),\n      \"Non-spherical geography not supported in Spark: %s\", g));","typeGuard":null,"tryCatchPattern":"try { df = spark.read().load(\"iceberg_geo_table\"); }\ncatch (UnsupportedOperationException e) {\n  if (e.getMessage().contains(\"edge algorithm\")) { /* rewrite column or exclude */ }\n  else throw e;\n}","preventionTips":["Use spherical edges for geography columns","Prefer geometry when non-spherical edges are needed","Keep geo schemas within Spark-supported algorithms"],"tags":["spark","geography","type-conversion"],"backgroundTag":"unsupported-operation","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}