influxdata/influxdb · error · TableIndexCacheError

Failed to persist updated table index

Error message

Failed to persist updated table index: {:?}

What it means

A TableIndexCacheError::Unexpected wrapping an anyhow message 'Failed to persist updated table index: {:?}' — raised in purge_expired when the updated TableIndex could not be written back to the object store via updated_index.persist(...). The expiry purge succeeded in memory but the result could not be durably saved, so the purge is not effective until retried.

Solutions

1. Read the debug-formatted cause embedded in the message ({:?} of the TableIndexError) to classify I/O vs serialization failure.
2. Retry purge_expired — persistence is the last step, so a retry reuses the in-memory expiry result.
3. Verify bucket write permissions and quotas for the node's service account.
4. Check object store health/status page if failures cluster in time.

Example fix

// before: one-shot persist, panic on failure
updated_index.persist(Arc::clone(&object_store)).await.map_err(...)?;
// after: bounded retries
for attempt in 0..3 {
  match updated_index.persist(Arc::clone(&object_store)).await {
    Ok(()) => break,
    Err(e) if attempt == 2 => return Err(map_err(e)),
    Err(_) => tokio::time::sleep(backoff(attempt)).await,
  }
}

// Rust: probe write access before running the purge
object_store.put(&probe_path, Bytes::from_static(b"ok")).await
  .expect("object store not writable; postpone purge_expired");

fn is_persist_unexpected(e: &TableIndexCacheError) -> bool {
  matches!(e, TableIndexCacheError::Unexpected(msg) if msg.contains("Failed to persist updated table index"))
}

match purge_expired().await {
  Err(e) if is_persist_unexpected(&e) => {
    log::warn!("persist failed, retrying purge: {e:?}");
    retry_with_backoff(3, purge_expired).await
  }
  other => other,
}

When it happens

Trigger: Running the purge_expired flow (public) on a TableIndexCache where persist() to the object store returns a TableIndexError — store write failure, serialization failure, or connectivity loss at write time.

Common situations: Object store bucket write quota exceeded or throttling (S3 503 Slow Down); credentials rotated and losing write access mid-run; network partition to the blob store during expiry processing; snapshot size or request limits hit on very large indexes.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Failed to create table index from object store
• Failed to delete parquet file
• Failed to delete table index
• Failed to list object store metadata
• Failed to list table index snapshots from object store

AI-assisted analysis of influxdata/influxdb@06200ef96b (2026-09-19). Data as JSON: /api/errors/53d3f9ba69567cb2. Report an issue: GitHub.

Appendix: source

Thrown at influxdb3_write/src/table_index_cache.rs:938

        }

        // Create updated table index with only remaining files
        let updated_index = CoreTableIndex {
            id: core_index.id.clone(),
            files: remaining_files,
            latest_snapshot_sequence_number: core_index.latest_snapshot_sequence_number,
            metadata: core_index.metadata,
        };

        // Drop the lock before we do more operations
        drop(core_index);

        // Persist the updated index to object store
        updated_index
            .persist(Arc::clone(&self.inner.object_store))
            .await
            .map_err(|e| {
                TableIndexCacheError::Unexpected(anyhow::anyhow!(
                    "Failed to persist updated table index: {:?}",
                    e
                ))
            })?;

        // Update the cache with the new index
        let new_cached = CachedTableIndex::new(updated_index);
        {
            let mut indices = self.inner.indices.write().await;
            let db_map = indices.entry((node_id, db_id)).or_default();
            db_map.insert(table_id, new_cached);
        }

        info!(
            ?table_index_id,
            cutoff_time_ns,
            deleted_file_count = expired_files.len(),
            "Successfully purged expired files from table"

View on GitHub (pinned to 06200ef96b)