apache/seatunnel · critical · SQLException

Failed to append EdgeEvent to WAL: generated id is missing.

Error message

Failed to append EdgeEvent to WAL: generated id is missing.

What it means

SqliteWalStore.append() inserts the event row and retrieves the auto-generated row id; if the insert succeeds but no generated key is returned, the store rolls back and throws this SQLException. A WAL row without an id is unusable for ordering and replay, so the append is treated as failed.

Solutions

Verify the sqlite JDBC driver version supports generated-key retrieval and is up to date.
Ensure the WAL table uses an INTEGER PRIMARY KEY autoincrement column so sqlite exposes the generated rowid.
Confirm the insert is executed with Statement.RETURN_GENERATED_KEYS / prepareStatement(sql, RETURN_GENERATED_KEYS).
Check schema drift — recreate the WAL table with the expected schema if it was altered externally.

Defensive patterns

Strategy: try-catch

Try / catch

try { return store.append(event); } catch (SQLException e) { if (e.getMessage().contains("generated id is missing")) { log.error("JDBC generated-keys unsupported; check driver/table schema", e); } throw e; }

Prevention

Pin a sqlite JDBC driver version with generated-keys support.
Keep the WAL table's INTEGER PRIMARY KEY column intact.
Always prepare inserts with RETURN_GENERATED_KEYS.
Add a startup check that an INSERT into the WAL table returns a generated key.

When it happens

Trigger: Calling walStore.append(event) when Statement.RETURN_GENERATED_KEYS (or equivalent) yields no key after the insert — e.g. a driver/table configuration that does not expose rowids, or an INSERT statement the driver cannot report generated keys for.

Common situations: Using a sqlite JDBC driver whose generated-key support is disabled or incompatible; schema changes replacing the rowid-based table with one lacking auto-generated keys; driver version downgrades.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/db8167efba4ff909. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-edge-agent/seatunnel-edge-agent-starter/src/main/java/org/apache/seatunnel/edge/agent/starter/wal/sqlite/SqliteWalStore.java:101

                statement.setBytes(2, payload);
                statement.setLong(3, event.getEventTime());
                statement.setBytes(4, MetadataSerde.serialize(event.getMetadata()));
                statement.setString(5, WalRecordStatus.PENDING.name());
                statement.setInt(6, 0);
                statement.setLong(7, now);
                statement.setLong(8, now);
                statement.setLong(9, batchId);
                statement.executeUpdate();
                try (ResultSet generatedKeys = statement.getGeneratedKeys()) {
                    if (generatedKeys.next()) {
                        long rowId = generatedKeys.getLong(1);
                        connection.commit();
                        return rowId;
                    }
                }
            }
            connection.rollback();
            throw new SQLException("Failed to append EdgeEvent to WAL: generated id is missing.");
        } catch (Exception ex) {
            connection.rollback();
            throw ex;
        } finally {
            connection.setAutoCommit(autoCommit);
        }
    }

    @Override
    public List<WalRecord> claimPending(int maxRecords, int maxAttempts) throws Exception {
        if (maxRecords < 1 || maxAttempts < 1) {
            return new ArrayList<>();
        }
        boolean autoCommit = connection.getAutoCommit();
        connection.setAutoCommit(false);
        try {
            List<WalRecord> records = selectPendingOrderByIdAsc(maxRecords, maxAttempts);
            long now = System.currentTimeMillis();

View on GitHub (pinned to cf67b549a7)