apache/seatunnel · warning

Unknown Arrow type: {}, defaulting to utf8

Error message

Unknown Arrow type: {}, defaulting to utf8

What it means

LanceCatalog.convertArrowSchemaToJsonArrowSchema maps each Arrow field type to a JSON Arrow type string. When it encounters an ArrowType it does not recognize (not time/timestamp/date/etc.), it logs this warning and falls back to utf8. The resulting catalog schema may lose the original type semantics.

Source

Thrown at seatunnel-connectors-v2/connector-lance/src/main/java/org/apache/seatunnel/connectors/seatunnel/lance/catalog/LanceCatalog.java:473

            } else if (arrowType
                    instanceof org.apache.arrow.vector.types.pojo.ArrowType.FloatingPoint) {
                org.apache.arrow.vector.types.pojo.ArrowType.FloatingPoint fp =
                        (org.apache.arrow.vector.types.pojo.ArrowType.FloatingPoint) arrowType;
                if (fp.getPrecision()
                        == org.apache.arrow.vector.types.FloatingPointPrecision.SINGLE) {
                    jsonType.setType("float32");
                } else {
                    jsonType.setType("float64");
                }
            } else if (arrowType instanceof org.apache.arrow.vector.types.pojo.ArrowType.Binary) {
                jsonType.setType("binary");
            } else if (arrowType instanceof org.apache.arrow.vector.types.pojo.ArrowType.Date) {
                jsonType.setType("date32");
            } else if (arrowType
                    instanceof org.apache.arrow.vector.types.pojo.ArrowType.Timestamp) {
                jsonType.setType("timestamp");
            } else {
                log.warn("Unknown Arrow type: {}, defaulting to utf8", arrowType);
                jsonType.setType("utf8");
            }

            jsonField.setType(jsonType);
            fields.add(jsonField);
        }

        jsonArrowSchema.setFields(fields);
        return jsonArrowSchema;
    }

    private Schema convertJsonArrowSchemaToArrowSchema(JsonArrowSchema jsonArrowSchema) {
        if (jsonArrowSchema == null || jsonArrowSchema.getFields() == null) {
            return null;
        }

        List<Field> arrowFields = new ArrayList<>();
        for (JsonArrowField jsonField : jsonArrowSchema.getFields()) {

View on GitHub (pinned to cf67b549a7)

Solutions

1. Upgrade the connector-lance / lance-JNI dependency to a version supporting the encountered Arrow type
2. Identify the offending column from the warning's arrowType value and change the writer to use a supported type (e.g. timestamp/date32/utf8)
3. If utf8 fallback is acceptable, ignore the warning; otherwise cast the column at write time
4. Add the missing instanceof branch to convertArrowSchemaToJsonArrowSchema and rebuild

Example fix

// before
} else {
    log.warn("Unknown Arrow type: {}, defaulting to utf8", arrowType);
    jsonType.setType("utf8");
}
// after
} else if (arrowType instanceof org.apache.arrow.vector.types.pojo.ArrowType.Duration) {
    jsonType.setType("duration");
} else {
    log.warn("Unknown Arrow type: {}, defaulting to utf8", arrowType);
    jsonType.setType("utf8");
}

// validate schema types before catalog read
arrowSchema.getFields().stream()
    .map(f -> f.getType())
    .filter(t -> !(t instanceof ArrowType.Utf8 || t instanceof ArrowType.Timestamp || t instanceof ArrowType.Date))
    .forEach(t -> System.out.println("Unsupported Arrow type: " + t));

try { Table table = catalog.getTable(ident); } catch (Exception e) { log.warn("Table schema degraded by unknown Arrow types", e); }

When it happens

Trigger: Reading a Lance table whose schema contains an Arrow type not covered by the conversion chain (e.g. Duration, Interval, Union, large_binary variants, or Decimal in an older mapping) during getTable catalog loading.

Common situations: A Lance dataset written by a newer library version with types this connector version does not know; user-defined or nested extension types; schema evolution adding exotic column types.

Related errors

• Unknown JsonArrow type: {}, defaulting to utf8
• seaTunnelDataType cannot be null
• TABLE_DATASET_WRITE_ST_ROW_EXCEPTION
• Map field must have key and value child fields
• UNSUPPORTED_DATA_TYPE

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/ad0992d82003b6bd. Report an issue: GitHub.