apache/seatunnel · error · FileConnectorException

FileConnectorErrorCode.DATA_DESERIALIZE_FAILED

FileConnectorErrorCode.DATA_DESERIALIZE_FAILED

Error message

Deserialize this jsonFile data [%s] failed, please check the origin data

What it means

The JSON file read strategy failed to deserialize a line of a JSON file into a SeaTunnelRow, wrapping the underlying IOException. SeaTunnel throws this because the origin data in the JSON file does not conform to the schema the user defined in the config. The failing line's raw text is included in the message for inspection.

Source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/reader/JsonReadStrategy.java:141

                            line -> {
                                try {
                                    SeaTunnelRow seaTunnelRow =
                                            deserializationSchema.deserialize(
                                                    line.getBytes(StandardCharsets.UTF_8));
                                    if (isMergePartition) {
                                        int index = seaTunnelRowType.getTotalFields();
                                        for (String value : partitionsMap.values()) {
                                            seaTunnelRow.setField(index++, value);
                                        }
                                    }
                                    seaTunnelRow.setTableId(split.getTableId());
                                    output.collect(seaTunnelRow);
                                } catch (IOException e) {
                                    String errorMsg =
                                            String.format(
                                                    "Deserialize this jsonFile data [%s] failed, please check the origin data",
                                                    line);
                                    throw new FileConnectorException(
                                            FileConnectorErrorCode.DATA_DESERIALIZE_FAILED,
                                            errorMsg,
                                            e);
                                }
                            });
        }
    }

    @Override
    public SeaTunnelRowType getSeaTunnelRowTypeInfo(String path) throws FileConnectorException {
        throw new FileConnectorException(
                CommonErrorCodeDeprecated.UNSUPPORTED_OPERATION,
                "User must defined schema for json file type");
    }
}

View on GitHub (pinned to cf67b549a7)

Solutions

1. Open the file and inspect the offending line printed in the error message; fix or remove the malformed record.
2. Validate each line parses as JSON (e.g. jq) before running the job.
3. Check the user-defined schema matches the actual JSON field names and types, and correct schema in the source config.
4. If files are multi-line pretty-printed JSON, convert them to JSON Lines (one object per line).
5. Pre-clean or filter dirty records upstream before the file source.

Example fix

// before: schema declares id as int but file line is {"id":"abc"}
schema = {
  fields {
    id = int
    name = string
  }
}
// after: align schema with real data or fix the data
// data: {"id":1,"name":"abc"}

// validate every line before running the job
for line in file.lines: JSON.parse(line) // or: jq -c . file.json > /dev/null

function isValidJsonLine(line) { try { const o = JSON.parse(line); return o && typeof o === 'object'; } catch { return false; } }

try { reader.read(); } catch (FileConnectorException e) { if (e.getCode() == FileConnectorErrorCode.DATA_DESERIALIZE_FAILED) { log.error("Bad JSON line: {}", extractLine(e.getMessage())); skipOrQuarantine(); } else { throw e; } }

When it happens

Trigger: JsonReadStrategy.readProcess() calls the deserializer for each line inside a row-output collector; any IOException while parsing a line (malformed JSON, field type mismatch against the configured schema, missing required field) throws FileConnectorException(DATA_DESERIALIZE_FAILED).

Common situations: JSON files with trailing commas or concatenated (non-newline-delimited) JSON objects; values whose type differs from the user-defined schema (e.g. string where int declared); empty/corrupt lines from a truncated upload; files written with a different schema than schema { fields { ... } }.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

• CommonErrorCodeDeprecated.UNSUPPORTED_OPERATION
• DATA_DESERIALIZE_FAILED
• Failed to deserialize python source stdout line [{}]
• UNSUPPORTED_DATA_TYPE
• Could not find field with name ${fieldName} .

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/589cb4f1198b42c8. Report an issue: GitHub.