apache/seatunnel · error · SeaTunnelException

Unexpected JSON payload format from Firebase

Error message

Unexpected JSON payload format from Firebase

What it means

processJsonPayload validates that the Firebase REST response starts with '{' or '[' before parsing; otherwise it throws this SeaTunnelException. It guards against payloads that are neither JSON objects/arrays — e.g. bare strings, numbers, or HTML error pages returned in place of data.

Source

Thrown at seatunnel-connectors-v2/connector-firebase/src/main/java/org/apache/seatunnel/connectors/seatunnel/firebase/source/FirebaseSourceReader.java:174

            SeaTunnelRowType rowType = catalogTable.getTableSchema().toPhysicalRowDataType();
            if (rowType != null && rowType.getFieldNames() != null) {
                return Collections.unmodifiableSet(
                        new HashSet<>(Arrays.asList(rowType.getFieldNames())));
            }
        }
        return Collections.emptySet();
    }

    /** Helper method that consistently handles single records, record maps, and JSON arrays. */
    private void processJsonPayload(String jsonPayload, Collector<SeaTunnelRow> output)
            throws Exception {
        if (jsonPayload == null || jsonPayload.trim().equalsIgnoreCase("null")) {
            return;
        }
        String trimmed = jsonPayload.trim();

        if (!trimmed.startsWith("{") && !trimmed.startsWith("[")) {
            throw new SeaTunnelException("Unexpected JSON payload format from Firebase");
        }

        // Work queue to store objects (Maps, Lists) pending evaluation
        Queue<Object> workQueue = new ArrayDeque<>();

        // Parse initial payload once into Java Map/List
        Object initialParsed = OBJECT_MAPPER.readValue(trimmed, Object.class);
        workQueue.add(initialParsed);

        while (!workQueue.isEmpty()) {
            Object current = workQueue.poll();
            if (current == null) {
                continue;
            }

            if (current instanceof Map) {
                @SuppressWarnings("unchecked")
                Map<String, Object> recordMap = (Map<String, Object>) current;

View on GitHub (pinned to cf67b549a7)

Solutions

1. Point the path at an object/array node rather than a scalar leaf value.
2. Curl the exact REST URL to inspect the raw body the reader receives.
3. Fix authentication so Firebase returns JSON data instead of error text.
4. Add a transform/normalization step or wrap scalars client-side if leaf values are intended.

Example fix

// before
path = "/users/alice/email" // leaf: "alice@example.com" -> throws
// after
path = "/users/alice" // object node: {"email": "alice@example.com", ...}

String body = fetchRaw(url);
String t = body == null ? "" : body.trim();
if (!t.startsWith("{") && !t.startsWith("[")) throw new IllegalStateException("expected JSON object/array, got: " + t);

try { reader.readSplit(split); } catch (SeaTunnelException e) { if (e.getMessage().contains("Unexpected JSON payload")) { inspectRawResponse(); } throw e; }

When it happens

Trigger: Path pointing at a scalar leaf node (Firebase returns just "value" or 42 without braces); response being an HTML/plain-text error page that passed earlier status checks; empty or whitespace-only payload shaped unexpectedly.

Common situations: Misconfigured path selecting a leaf instead of a container node; auth issues returning non-JSON bodies; proxies injecting content; users expecting object-wrapped data from a scalar endpoint.

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

• Failed to parse shallow keys from Firebase response
• AmazonDocumentDB option '' must be a valid BSON/JSON documen
• timeout_ms must be greater than 0
• Invalid Firebase REST URI constructed. Check parameter forma
• Invalid sort JSON:

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/bcbac901f6983c60. Report an issue: GitHub.