{"record":{"id":"006836a8d06fb357","repo":"apache/iceberg","slug":"spark-does-not-support-time-fields","errorCode":null,"errorMessage":"Spark does not support time fields","messagePattern":"Spark does not support time fields","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/TypeToSparkType.java","lineNumber":108,"sourceCode":"  }\n\n  @Override\n  public DataType primitive(Type.PrimitiveType primitive) {\n    switch (primitive.typeId()) {\n      case BOOLEAN:\n        return BooleanType$.MODULE$;\n      case INTEGER:\n        return IntegerType$.MODULE$;\n      case LONG:\n        return LongType$.MODULE$;\n      case FLOAT:\n        return FloatType$.MODULE$;\n      case DOUBLE:\n        return DoubleType$.MODULE$;\n      case DATE:\n        return DateType$.MODULE$;\n      case TIME:\n        throw new UnsupportedOperationException(\"Spark does not support time fields\");\n      case TIMESTAMP:\n        Types.TimestampType ts = (Types.TimestampType) primitive;\n        if (ts.shouldAdjustToUTC()) {\n          return TimestampType$.MODULE$;\n        } else {\n          return TimestampNTZType$.MODULE$;\n        }\n      case STRING:\n        return StringType$.MODULE$;\n      case UUID:\n        // use String\n        return StringType$.MODULE$;\n      case FIXED:\n        return BinaryType$.MODULE$;\n      case BINARY:\n        return BinaryType$.MODULE$;\n      case DECIMAL:\n        Types.DecimalType decimal = (Types.DecimalType) primitive;","sourceCodeStart":90,"sourceCodeEnd":126,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/TypeToSparkType.java#L90-L126","documentation":"Iceberg supports a TIME primitive type but Spark SQL has no corresponding time-of-day data type. TypeToSparkType (used when converting Iceberg schemas to Spark schemas for reads/writes) therefore throws UnsupportedOperationException when a schema contains a Types.TimeType field.","triggerScenarios":"Reading or writing an Iceberg table whose schema contains a TIME-typed column via the Spark connector, e.g. spark.read().load(\"...\") or a DataFrame write to an Iceberg table with a time field.","commonSituations":"Tables written by engines that do support TIME (e.g. Flink or custom writers) being consumed in Spark; schema evolved to add a time column; copying a schema from another catalog into Iceberg and querying it from Spark.","solutions":["Exclude or transform the TIME column before conversion: select all columns except the time field","Cast/rewrite the time value as a string or as microseconds-since-midnight stored in a long/integer column","Change the table schema to use TIMESTAMP or another Spark-compatible type"],"exampleFix":"// before\nDataset<Row> df = spark.read().load(\"tbl\"); // fails on time_col\n// after\nDataset<Row> df = spark.read().load(\"tbl\").drop(\"time_col\");","handlingStrategy":"validation","validationCode":"boolean hasTime = table.schema().columns().stream()\n    .anyMatch(c -> c.type().typeId() == Types.TimeType.get().typeId());\nif (hasTime) throw new IllegalArgumentException(\"Schema contains TIME, unsupported in Spark\");","typeGuard":"static boolean isSparkCompatible(Types.NestedField f) { return f.type().typeId() != Types.TimeType.get().typeId(); }","tryCatchPattern":"try { return TypeToSparkType.convert(schema); } catch (UnsupportedOperationException e) { throw new AnalysisException(\"Unsupported Iceberg type for Spark: \" + e.getMessage(), e); }","preventionTips":["Avoid TIME columns in tables intended for Spark consumption; use timestamp or long-since-midnight instead","Validate cross-engine schemas before writing shared tables","Add a schema compatibility check in CI for tables consumed by multiple engines"],"tags":["spark","schema","type-conversion"],"backgroundTag":"unsupported-operation","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}