What causes "Inventory dump failed on %s" in apache/shardingsphere?

Inspect the logged cause (log.error prints the exception) and proxy logs to identify whether it is connectivity, privileges, timeout, or SQL/dialect failure.

apache/shardingsphere · error · IngestException

Inventory dump failed on %s

Error message

Inventory dump failed on %s

What it means

InventoryDumper.dump wraps any SQLException or RuntimeException from the full-dump phase (calculator-based or streaming) of a pipeline job in IngestException('Inventory dump failed on <table>'). The root cause is always in the attached exception/log — connectivity loss, privilege errors, SQL dialect problems, or invalid unique-key assumptions during the initial table snapshot.

Source

Thrown at kernel/data-pipeline/core/src/main/java/org/apache/shardingsphere/data/pipeline/core/ingest/dumper/inventory/InventoryDumper.java:114

    
    @Override
    protected void runBlocking() {
        IngestPosition position = dumperContext.getCommonContext().getPosition();
        if (position instanceof IngestFinishedPosition) {
            log.info("Ignored because of already finished.");
            return;
        }
        try {
            if (dumperContext.hasUniqueKey()) {
                dumpByCalculator();
            } else {
                dumpWithStreamingQuery();
            }
            // CHECKSTYLE:OFF
        } catch (final SQLException | RuntimeException ex) {
            // CHECKSTYLE:ON
            log.error("Inventory dump failed on {}", dumperContext.getActualTableName(), ex);
            throw new IngestException("Inventory dump failed on " + dumperContext.getActualTableName(), ex);
        }
    }
    
    private void dumpByCalculator() {
        String schemaName = dumperContext.getCommonContext().getTableAndSchemaNameMapper().getSchemaName(dumperContext.getLogicTableName());
        QualifiedTable table = new QualifiedTable(schemaName, dumperContext.getActualTableName());
        IngestPosition initialPosition = dumperContext.getCommonContext().getPosition();
        log.info("Dump by calculator start, dataSource={}, table={}, initialPosition={}", dumperContext.getCommonContext().getDataSourceName(), table, initialPosition);
        List<String> columnNames = dumperContext.getQueryColumnNames();
        TableInventoryCalculateParameter calculateParam = new TableInventoryCalculateParameter(dataSource, table,
                columnNames, dumperContext.getUniqueKeyColumns(), QueryType.RANGE_QUERY, null);
        Range<?> range = Range.closed(((UniqueKeyIngestPosition<?>) initialPosition).getLowerBound(), ((UniqueKeyIngestPosition<?>) initialPosition).getUpperBound());
        calculateParam.setRange(range);
        RecordTableInventoryDumpCalculator dumpCalculator = new RecordTableInventoryDumpCalculator(dumperContext.getBatchSize());
        long rowCount = 0L;
        try {
            JobRateLimitAlgorithm rateLimitAlgorithm = dumperContext.getRateLimitAlgorithm();
            String firstUniqueKey = calculateParam.getFirstUniqueKey().getName();

View on GitHub (pinned to e952770a21)

Solutions

Inspect the logged cause (log.error prints the exception) and proxy logs to identify whether it is connectivity, privileges, timeout, or SQL/dialect failure.
Fix the source-side issue: grant privileges, stabilize connectivity, or tune chunk-size / unique-key selection, then restart or resume the pipeline job.
For timeout-class causes, reduce chunk-size and ensure the unique key is indexed.
If the table was modified during the dump, re-run inventory for that table after the schema stabilizes.

Defensive patterns

Strategy: try-catch

Validate before calling

// preflight before starting a migration job
try (Connection c = sourceDataSource.getConnection();
     Statement s = c.createStatement();
     ResultSet rs = s.executeQuery("SELECT 1 FROM `" + tableName + "` LIMIT 1")) {
    // table readable -> inventory dump preconditions hold
}

Try / catch

try {
    dumper.dump();
} catch (final IngestException ex) {
    Throwable cause = ex.getCause();
    // classify: SQLException -> connectivity/privilege/timeout; rethrow for job retry
}

Prevention

Verify pipeline-user privileges (SELECT on all source tables) before starting the job.
Pre-check unique-key presence and index coverage for range-chunked dumps.
Tune chunk-size down when source queries time out; monitor proxy logs during snapshot.

When it happens

Trigger: Running the inventory (full) dump stage of a migration job where the SELECT over the actual table (range query by unique key or streaming query) throws: connection reset, lock timeout, column-type mapping failure, missing table, or a unique-key column that cannot be range-queried.

Common situations: Source database restarts or network blips mid-snapshot; insufficient SELECT/REPLICATION privileges on the source; very wide chunk queries timing out; table dropped or renamed while the job ran; a unique key on a type the dialect cannot order consistently.

Related errors

AI-assisted analysis of apache/shardingsphere@e952770a21 (2026-08-14). Data as JSON: /api/errors/3da91ac18a29b57c. Report an issue: GitHub.