influxdata/influxdb · critical · MigrationError

Unexpected error while migrating v2 catalog (wraps…

Error message

Unexpected error while migrating v2 catalog (wraps apply_records failure)

What it means

`MigrationError::Unexpected` wrapping a failure of `apply_records` during the v2->v3 catalog migration. `apply_records` replays synthesized v2 snapshot records into the fresh v3 catalog; any error there (malformed record, constraint violation, internal bug) is wrapped as an unexpected/non-categorized migration error via anyhow.

Solutions

1. Read the wrapped `anyhow` cause in the error chain to identify which record type failed to apply.
2. Validate/repair the v2 snapshot data (dump it via `synthesize_records` paths or restore from a known-good backup).
3. Upgrade to the latest influxdb3 release — migration record-compat fixes are common.
4. If the v2 store is disposable, clear it (or the checkpoint) so the migration short-circuits with NothingToMigrate.

Example fix

// before
apply_records(...).map_err(|e| MigrationError::Unexpected(anyhow::anyhow!(e)))?;
// after (diagnose first)
match apply_records(...) {
    Ok(()) => {},
    Err(e) => { log::error!("apply_records failed: {e:#?}"); return Err(MigrationError::Unexpected(anyhow::anyhow!(e))); }
}

match check_and_migrate_v2_to_v3(prefix, store).await {
    Ok(MigrationResult::Migrated) => info!("migrated"),
    Err(e) => {
        // inspect the wrapped anyhow chain to find the failing record
        error!("migration failed: {e:#}");
        return Err(e);
    }
}

When it happens

Trigger: Calling `check_and_migrate_v2_to_v3` when replaying v2 snapshot records into the v3 in-memory catalog fails — e.g. a v2 snapshot containing records the v3 `apply_records` parser rejects, duplicate ids, or out-of-order sequences.

Common situations: Migrating catalogs written by older v2 builds whose snapshots are not fully compatible with the current migration code; hand-edited or partially restored catalog objects; a bug in `synthesize_records`/`apply_records` surfaced by unusual v2 content.

Related errors

• generation duration overflows u64 nanoseconds
• last cache size overflows u64
• row_delete_predicate_version exceeds u64
• v2 catalog should exist since checkpoint was found
• cache error

AI-assisted analysis of influxdata/influxdb@06200ef96b (2026-09-19). Data as JSON: /api/errors/8a139e6dc624353d. Report an issue: GitHub.

Appendix: source

Thrown at influxdb3_catalog/src/catalog/migrations/v3.rs:115

    }

    let v2_catalog = ser::v2::load_catalog(Arc::clone(&prefix), &v2_store)
        .await?
        .expect("v2 catalog should exist since checkpoint was found");

    let catalog_uuid = v2_catalog.catalog_uuid;
    let catalog_sequence = v2_catalog.sequence_number();
    let v2_snapshot = v2_catalog.snapshot();
    let batch = synthesize_records(&v2_snapshot);

    let mut v3_inner = V3InnerCatalog::new(Arc::clone(&prefix), catalog_uuid);
    apply_records(
        batch.as_slice(),
        &mut v3_inner,
        catalog_sequence,
        RestorePreload::empty(),
    )
    .map_err(|e| MigrationError::Unexpected(anyhow::anyhow!(e)))?;
    let v3_snapshot_bytes = v3_inner.create_snapshot();

    if !v2_catalog.has_upgraded {
        let next_sequence = catalog_sequence.next();
        if let PersistCatalogResult::AlreadyExists =
            v2_store.persist_upgrade_log(next_sequence).await?
        {
            return Err(MigrationError::UpgradeLogAlreadyExists);
        }
    } else {
        info!("skipping write of UpgradedLog, as it is already present");
    }

    info!("writing the v3 catalog snapshot");
    match v3_store.initialize_snapshot(v3_snapshot_bytes).await? {
        MaybePutCatalogFile::Success(_) | MaybePutCatalogFile::AlreadyExists => {
            Ok(MigrationResult::Migrated)
        }

View on GitHub (pinned to 06200ef96b)