What causes "Failed to advance next_batch_id" in apache/seatunnel?

Ensure only one agent process accesses the sqlite file at a time (sqlite allows only a single writer).

apache/seatunnel · critical · SQLException

Failed to advance next_batch_id

Error message

Failed to advance next_batch_id

What it means

allocateNext() advances the next_batch_id row by writing current+1 and requires exactly one affected row. If the UPDATE affects 0 rows, the meta key was deleted (or the row vanished) between the read and the write, so the allocator cannot persist the id advance and throws this SQLException.

Solutions

Ensure only one agent process accesses the sqlite file at a time (sqlite allows only a single writer).
Restore the next_batch_id row in edge_agent_meta and restart the agent.
Avoid external maintenance scripts that delete rows from edge_agent_meta while the agent is running.
Enable sqlite WAL journal mode and retry the allocation if transient locking is suspected.

Defensive patterns

Strategy: retry

Validate before calling

try (PreparedStatement ps = connection.prepareStatement("SELECT COUNT(*) FROM edge_agent_meta WHERE key='next_batch_id'")) { try (ResultSet rs = ps.executeQuery()) { rs.next(); if (rs.getInt(1) != 1) { /* restore row before allocating */ } } }

Try / catch

catch (SQLException e) { if (e.getMessage().contains("Failed to advance next_batch_id")) { recreateMetaRow(); retryAllocate(); } else { throw e; } }

Prevention

Run only one agent process against the sqlite file.
Avoid concurrent maintenance scripts on edge_agent_meta.
Use sqlite WAL journal mode for safer concurrent reads.
Wrap meta updates in short transactions to reduce race windows.

When it happens

Trigger: Calling allocateNext(connection) where UPDATE_VALUE on key 'next_batch_id' matches no rows — typically because the edge_agent_meta row was deleted concurrently or the table was recreated mid-call.

Common situations: Concurrent access to the same sqlite database from two agent processes; manual DELETE/maintenance scripts touching edge_agent_meta while the agent runs; a transaction/rollback race wiping the key.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/e7e874b4e749dc0e. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-edge-agent/seatunnel-edge-agent-starter/src/main/java/org/apache/seatunnel/edge/agent/starter/wal/sqlite/SqliteBatchIdAllocator.java:45

    /**
     * Returns the next batch id and advances the counter. Must run inside an open transaction on
     * {@code connection}.
     */
    public static long allocateNext(Connection connection) throws SQLException {
        Long current = readNextBatchId(connection);
        if (current == null) {
            seedMetaFromWal(connection);
            current = readNextBatchId(connection);
        }
        if (current == null || current <= 0) {
            throw new SQLException("Failed to initialize next_batch_id in edge_agent_meta");
        }
        try (PreparedStatement statement =
                connection.prepareStatement(MetaSqlStatements.UPDATE_VALUE)) {
            statement.setLong(1, current + 1);
            statement.setString(2, MetaSqlStatements.KEY_NEXT_BATCH_ID);
            if (statement.executeUpdate() != 1) {
                throw new SQLException("Failed to advance next_batch_id");
            }
        }
        return current;
    }

    public static Long peekNextBatchId(Connection connection) throws SQLException {
        return readNextBatchId(connection);
    }

    public static void seedMetaFromWal(Connection connection) throws SQLException {
        try (PreparedStatement statement =
                connection.prepareStatement(MetaSqlStatements.SEED_NEXT_BATCH_ID_FROM_WAL)) {
            statement.executeUpdate();
        }
    }

    private static Long readNextBatchId(Connection connection) throws SQLException {
        try (PreparedStatement statement =

View on GitHub (pinned to cf67b549a7)