{"record":{"id":"ad0992d82003b6bd","repo":"apache/seatunnel","slug":"unknown-arrow-type-defaulting-to-utf8","errorCode":null,"errorMessage":"Unknown Arrow type: {}, defaulting to utf8","messagePattern":"Unknown Arrow type: (.+?), defaulting to utf8","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-connectors-v2/connector-lance/src/main/java/org/apache/seatunnel/connectors/seatunnel/lance/catalog/LanceCatalog.java","lineNumber":473,"sourceCode":"            } else if (arrowType\n                    instanceof org.apache.arrow.vector.types.pojo.ArrowType.FloatingPoint) {\n                org.apache.arrow.vector.types.pojo.ArrowType.FloatingPoint fp =\n                        (org.apache.arrow.vector.types.pojo.ArrowType.FloatingPoint) arrowType;\n                if (fp.getPrecision()\n                        == org.apache.arrow.vector.types.FloatingPointPrecision.SINGLE) {\n                    jsonType.setType(\"float32\");\n                } else {\n                    jsonType.setType(\"float64\");\n                }\n            } else if (arrowType instanceof org.apache.arrow.vector.types.pojo.ArrowType.Binary) {\n                jsonType.setType(\"binary\");\n            } else if (arrowType instanceof org.apache.arrow.vector.types.pojo.ArrowType.Date) {\n                jsonType.setType(\"date32\");\n            } else if (arrowType\n                    instanceof org.apache.arrow.vector.types.pojo.ArrowType.Timestamp) {\n                jsonType.setType(\"timestamp\");\n            } else {\n                log.warn(\"Unknown Arrow type: {}, defaulting to utf8\", arrowType);\n                jsonType.setType(\"utf8\");\n            }\n\n            jsonField.setType(jsonType);\n            fields.add(jsonField);\n        }\n\n        jsonArrowSchema.setFields(fields);\n        return jsonArrowSchema;\n    }\n\n    private Schema convertJsonArrowSchemaToArrowSchema(JsonArrowSchema jsonArrowSchema) {\n        if (jsonArrowSchema == null || jsonArrowSchema.getFields() == null) {\n            return null;\n        }\n\n        List<Field> arrowFields = new ArrayList<>();\n        for (JsonArrowField jsonField : jsonArrowSchema.getFields()) {","sourceCodeStart":455,"sourceCodeEnd":491,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-lance/src/main/java/org/apache/seatunnel/connectors/seatunnel/lance/catalog/LanceCatalog.java#L455-L491","documentation":"LanceCatalog.convertArrowSchemaToJsonArrowSchema maps each Arrow field type to a JSON Arrow type string. When it encounters an ArrowType it does not recognize (not time/timestamp/date/etc.), it logs this warning and falls back to utf8. The resulting catalog schema may lose the original type semantics.","triggerScenarios":"Reading a Lance table whose schema contains an Arrow type not covered by the conversion chain (e.g. Duration, Interval, Union, large_binary variants, or Decimal in an older mapping) during getTable catalog loading.","commonSituations":"A Lance dataset written by a newer library version with types this connector version does not know; user-defined or nested extension types; schema evolution adding exotic column types.","solutions":["Upgrade the connector-lance / lance-JNI dependency to a version supporting the encountered Arrow type","Identify the offending column from the warning's arrowType value and change the writer to use a supported type (e.g. timestamp/date32/utf8)","If utf8 fallback is acceptable, ignore the warning; otherwise cast the column at write time","Add the missing instanceof branch to convertArrowSchemaToJsonArrowSchema and rebuild"],"exampleFix":"// before\n} else {\n    log.warn(\"Unknown Arrow type: {}, defaulting to utf8\", arrowType);\n    jsonType.setType(\"utf8\");\n}\n// after\n} else if (arrowType instanceof org.apache.arrow.vector.types.pojo.ArrowType.Duration) {\n    jsonType.setType(\"duration\");\n} else {\n    log.warn(\"Unknown Arrow type: {}, defaulting to utf8\", arrowType);\n    jsonType.setType(\"utf8\");\n}","handlingStrategy":"fallback","validationCode":"// validate schema types before catalog read\narrowSchema.getFields().stream()\n    .map(f -> f.getType())\n    .filter(t -> !(t instanceof ArrowType.Utf8 || t instanceof ArrowType.Timestamp || t instanceof ArrowType.Date))\n    .forEach(t -> System.out.println(\"Unsupported Arrow type: \" + t));","typeGuard":null,"tryCatchPattern":"try { Table table = catalog.getTable(ident); } catch (Exception e) { log.warn(\"Table schema degraded by unknown Arrow types\", e); }","preventionTips":["Keep connector-lance and lance JNI versions aligned","Avoid exotic Arrow types (Duration, Union) in Lance datasets consumed by SeaTunnel","Cast unsupported columns to string/timestamp at write time"],"tags":["arrow","lance","schema","type-mapping"],"backgroundTag":"unsupported-dtype","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}