Shallow key discovery failed on path

Error message

Shallow key discovery failed on path [{}]. Falling back to Single Path Split. Reason: {}

What it means

The Firebase source split enumerator first attempts a shallow scan of the configured base path to discover child keys and generate one split per child. If the shallow scan throws any exception (or returns no keys), the enumerator logs this warning and falls back to generating a single split covering the whole base path instead of failing the job. Data is still read, but with reduced parallelism since only one split is produced.

Solutions

Check that the configured base path exists and is readable under the database's security rules; fix the path or rules so the shallow scan succeeds.
Inspect the 'Reason' in the warning log to identify the underlying exception (auth, network, not-found) and fix that root cause.
Accept the fallback if single-path parallelism is sufficient for your data volume.
Retry the job if the failure was transient (network blip); the enumerator will re-run shallow discovery.

Example fix

// before (path blocked by rules)
basePath = "/orders/2026"
// after (grant read on that subtree in Firebase rules)
{ "rules": { "orders": { "2026": { ".read": true } } } }

Defensive patterns

Strategy: fallback

Validate before calling

// before submitting the job, verify the path is readable
Firebase foo = new Firebase("https://<db>.firebaseio.com");
foo.child(basePath).once("value", snap => console.log(snap.numChildren()), err => console.error(err));

Try / catch

try {
    enumerator.run();
} catch (Exception e) {
    // shallow-discovery failures are internally handled; reaching here means deeper failure
    log.error("Split enumeration failed beyond fallback: {}", e.getMessage(), e);
}

Prevention

Validate the Firebase base path exists and security rules allow read before starting the job.
Watch for this warning and treat it as a parallelism degradation signal, not silent success.
Grant read permission at the parent level so shallow scans of children succeed.
Monitor the Reason field in the log to distinguish transient network errors from config errors.

When it happens

Trigger: Calling run() on FirebaseSourceSplitEnumerator when the shallow key discovery REST/SDK call throws — e.g. the base path does not exist, network/permission errors during the shallow scan, or the scan unexpectedly returns an empty key list.

Common situations: Misconfigured or renamed Firebase Realtime Database path; restrictive database security rules blocking read at the parent path; transient network failures to Firebase; genuinely empty child collections at the base path.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/4654bd187bd1d9b6. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-connectors-v2/connector-firebase/src/main/java/org/apache/seatunnel/connectors/seatunnel/firebase/source/FirebaseSourceSplitEnumerator.java:109

            }
        }
        String basePath = config.get(FirebaseSourceOptions.PATH);
        List<FirebaseSourceSplit> generatedSplits = new ArrayList<>();

        try {
            log.info("Attempting automatic key discovery (shallow scan) on path [{}]", basePath);
            List<String> keys = httpClient.fetchShallowKeys();
            if (!keys.isEmpty()) {
                generatedSplits = partitionKeysIntoSplits(basePath, keys);
                log.info(
                        "Key discovery succeeded. Partitioned path into {} splits.",
                        generatedSplits.size());
            } else {
                log.info("Shallow scan returned no child keys. Falling back to Single Path Split.");
                generatedSplits.add(createSinglePathSplit(basePath));
            }
        } catch (Exception e) {
            log.warn(
                    "Shallow key discovery failed on path [{}]. "
                            + "Falling back to Single Path Split. Reason: {}",
                    basePath,
                    e.getMessage());
            generatedSplits.add(createSinglePathSplit(basePath));
        }
        synchronized (this) {
            pendingSplits.addAll(generatedSplits);
            log.info("pending Splits : {}", pendingSplits);
            this.discoveryDone = true;
            assignSplits();
        }
    }

    @Override
    public synchronized void registerReader(int subtaskId) {
        log.info("Reader subtask [{}] registered.", subtaskId);
        assignSplits();

View on GitHub (pinned to cf67b549a7)