influxdata/influxdb · error · QueryExecutorError

unable to compose record batches from retention policies

Error message

unable to compose record batches from retention policies: {0}

What it means

This QueryExecutorError::RetentionPoliciesToRecordBatch variant wraps an ArrowError raised while building RecordBatches from retention policy metadata (InfluxQL-era SHOW RETENTION POLICIES style results). It mirrors DatabasesToRecordBatch but for retention-policy rows.

Solutions

1. Inspect the wrapped ArrowError to find which field of the retention policy batches fails
2. Verify retention policy metadata in the catalog is complete and well-formed
3. Re-run against a single database to isolate the offending retention policy
4. Update influxdb3 to a version with corrected retention-policy batch composition

Example fix

// before
let rp = executor.show_retention_policies(db).await?; // ArrowError inside
// after
match executor.show_retention_policies(db).await {
    Ok(rp) => rp,
    Err(QueryExecutorError::RetentionPoliciesToRecordBatch(e)) => {
        eprintln!("RP batch compose failed: {e}");
        return Err(e.into());
    }
    Err(e) => return Err(e),
}

// validate retention policy metadata shape before composing
if retention_policies.iter().any(|rp| rp.name.is_empty() || rp.duration.is_none()) {
    return Err("malformed retention policy metadata".into());
}

fn is_rp_batch_err(e: &QueryExecutorError) -> Option<&ArrowError> {
    if let QueryExecutorError::RetentionPoliciesToRecordBatch(ae) = e { Some(ae) } else { None }
}

match executor.retention_policies_to_record_batch(&rps) {
    Ok(b) => Ok(b),
    Err(QueryExecutorError::RetentionPoliciesToRecordBatch(ae)) => {
        log::warn!("retention policy batch failed: {ae}");
        Err(QueryExecutorError::RetentionPoliciesToRecordBatch(ae))
    }
    Err(e) => Err(e),
}

When it happens

Trigger: Executing a retention-policy query through the internal executor where Arrow fails while composing batches from retention policy records (schema construction or batch building error).

Common situations: Legacy InfluxQL queries against a 3.x instance where retention policy metadata is partially populated; custom retention policy entries with unexpected field values causing Arrow batch build failures.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

• Error creating record batch
• unable to compose record batches from databases
• Field not found in table buffer
• Invalid Flatbuffer
• Message with header of type dictionary batch could not…

AI-assisted analysis of influxdata/influxdb@06200ef96b (2026-09-19). Data as JSON: /api/errors/806f542395b90c35. Report an issue: GitHub.

Appendix: source

Thrown at influxdb3_internal_api/src/query_executor.rs:33

use schema::INFLUXQL_MEASUREMENT_COLUMN_NAME;
use std::fmt::Debug;
use std::sync::Arc;
use trace::ctx::SpanContext;
use trace::span::{Span, SpanExt};
use trace_http::ctx::RequestLogContext;
use tracker::InstrumentedAsyncOwnedSemaphorePermit;

#[derive(Debug, thiserror::Error)]
pub enum QueryExecutorError {
    #[error("database not found: {db_name}")]
    DatabaseNotFound { db_name: String },
    #[error("error while planning query: {0}")]
    QueryPlanning(#[source] DataFusionError),
    #[error("error while executing plan: {0}")]
    ExecuteStream(#[source] DataFusionError),
    #[error("unable to compose record batches from databases: {0}")]
    DatabasesToRecordBatch(#[source] ArrowError),
    #[error("unable to compose record batches from retention policies: {0}")]
    RetentionPoliciesToRecordBatch(#[source] ArrowError),
    #[error("invokded a method that is not implemented: {0}")]
    MethodNotImplemented(&'static str),
    #[error(transparent)]
    Anyhow(#[from] anyhow::Error),
}

#[async_trait]
pub trait QueryExecutor: QueryDatabase + Debug + Send + Sync + 'static {
    async fn get_db_namespace(
        &self,
        database_name: &str,
        span_ctx: &Option<SpanContext>,
    ) -> Result<Arc<dyn QueryNamespace>, QueryExecutorError> {
        self.namespace(
            database_name,
            span_ctx.child_span("get_db_namespace"),
            false,

View on GitHub (pinned to 06200ef96b)