influxdata/influxdb · critical · TableIndexCacheError

Object store operation failed

Error message

Object store operation failed

What it means

TableIndexCacheError::ObjectStore wraps an object_store::Error raised while the TableIndexCache performs object store I/O outside its more specific variants (list/get/put/delete helpers). It indicates the backing store failed an operation while loading, updating, or persisting table index data through the LRU cache.

Solutions

1. Read the wrapped #[source] object_store::Error to identify the concrete failure (auth, throttling, NotFound) and fix that first.
2. Verify object store credentials are valid and not expired (check IAM session/token lifetime).
3. Confirm the configured bucket/container still exists and is reachable from the node.
4. For rate limiting, lower TableIndexCacheConfig::concurrency_limit or enable/raise retry backoff.
5. If transient, re-run TableIndexCache::initialize or restart the node once the store is healthy.

Example fix

// before: default concurrency overwhelming the store
TableIndexCacheConfig::default()
// after: reduced concurrency
TableIndexCacheConfig { concurrency_limit: 5, ..Default::default() }

// cheap liveness probe of the object store before cache-heavy operations
let _ = object_store.head(&probe_path).await.or_else(|e| match e {
    object_store::Error::NotFound { .. } => Ok(()),
    e => Err(e),
})?;

fn is_store_unavailable(e: &TableIndexCacheError) -> bool {
    matches!(e, TableIndexCacheError::ObjectStore(_))
}

match cache.get_or_load(&table_id).await {
    Err(TableIndexCacheError::ObjectStore(e)) if retryable(&e) => {
        tokio::time::sleep(backoff).await;
        cache.get_or_load(&table_id).await?
    }
    r => r?,
}

When it happens

Trigger: Calls through TableIndexCache (get_or_load, update_from_object_store, initialize, purge paths) hit a RetryableObjectStore operation that returns a non-NotFound error after default retries.

Common situations: Transient S3/GCS outages, expired cloud credentials mid-run, hitting request-rate limits during concurrent cache warm-up, or a bucket that was deleted or renamed while the server was running.

Related errors

• object store error
• object_store error
• Object store operation failed
• another process has written to the WAL ahead of this one
• cache not found

AI-assisted analysis of influxdata/influxdb@06200ef96b (2026-09-19). Data as JSON: /api/errors/0580253dc33bc98e. Report an issue: GitHub.

Appendix: source

Thrown at influxdb3_write/src/table_index_cache.rs:42

};

use influxdb3_id::{DbId, TableId, TableIndexId};
use influxdb3_wal::SnapshotSequenceNumber;

use crate::{
    ParquetFile,
    paths::{
        SnapshotInfoFilePath, TableIndexConversionCompletedPath, TableIndexPath,
        TableIndexSnapshotPath,
    },
    table_index::{CoreTableIndex, IndexMetadata, TableIndex, TableIndexSnapshot},
};

use thiserror::Error;

#[derive(Debug, Error)]
pub enum TableIndexCacheError {
    #[error("Object store operation failed")]
    ObjectStore(#[source] object_store::Error),

    #[error("JSON serialization/deserialization failed")]
    Json(#[source] serde_json::Error),

    #[error("TableIndex operation failed")]
    TableIndex(#[source] crate::table_index::TableIndexError),

    #[error("Object meta is missing filename")]
    MissingFilename,

    #[error("Failed to parse snapshot sequence number from filename")]
    InvalidSnapshotSequenceNumber,

    #[error("Unexpected error")]
    Unexpected(#[source] anyhow::Error),

    #[error("Table snapshot persistence task failed")]

View on GitHub (pinned to 06200ef96b)