influxdata/influxdb · error

num_columns_in_parallel should be above zero

Error message

num_columns_in_parallel should be above zero

What it means

ParquetStorage::with_parallel_write_settings configures parallel Parquet writing and wraps both arguments in NonZeroUsize. The second argument, num_column_writers_across_row_groups, must be at least 1; passing 0 makes NonZeroUsize::new return None and the .expect() panics with 'num_columns_in_parallel should be above zero'. The panic is documented under '# Panics' on the method, so it is intended behavior for invalid input, not an internal bug.

Source

Thrown at core/parquet_file/src/storage.rs:249

    }

    /// Provide settings for parallelized writes. Settings determine the
    /// amount of parallelization per row group and per column.
    ///
    /// # Panics
    ///
    /// This will panic if an invalid usize (not > 0) is used.
    pub fn with_parallel_write_settings(
        self,
        num_row_group_writers: usize,
        num_column_writers_across_row_groups: usize,
    ) -> Self {
        Self {
            parquet_write_parallelization_settings: ParallelParquetWriterOptions::new(
                NonZeroUsize::new(num_row_group_writers)
                    .expect("num_row_groups_in_parallel should be above zero"),
                NonZeroUsize::new(num_column_writers_across_row_groups)
                    .expect("num_columns_in_parallel should be above zero"),
            ),
            ..self
        }
    }

    /// Get underlying object store.
    pub fn object_store(&self) -> &Arc<DynObjectStore> {
        &self.object_store
    }

    /// Get ID.
    pub fn id(&self) -> StorageId {
        self.id
    }

    /// Fake DataFusion context for testing that contains this store
    pub fn test_df_context(&self) -> SessionContext {
        // set up "fake" DataFusion session

View on GitHub (pinned to d28e26e048)

Solutions

1. Pass a value >= 1 for num_column_writers_across_row_groups (1 means 'no column-level parallelism').
2. Trace where the argument originates: if it is computed (e.g. num_cpus * something, len() of a collection), guard that computation so it cannot produce 0.
3. Change the calling code's own API to accept NonZeroUsize so the invalid state is unrepresentable and this panic can never fire.
4. If the value comes from user config, validate it at config-load time and return a descriptive error instead of panicking deep in storage setup.

Example fix

// before
let storage = ParquetStorage::new(store, id)
    .with_parallel_write_settings(4, config.column_writers); // panics if 0

// after
let storage = ParquetStorage::new(store, id)
    .with_parallel_write_settings(4, config.column_writers.max(1));

// better: make it unrepresentable at the boundary
fn build(num_row_groups: NonZeroUsize, num_cols: NonZeroUsize) -> ParquetStorage {
    ParquetStorage::new(store, id)
        .with_parallel_write_settings(num_row_groups.get(), num_cols.get())
}

// Before calling with_parallel_write_settings:
let cols = num_column_writers_across_row_groups;
assert!(cols > 0, "num_column_writers_across_row_groups must be >= 1, got {cols}");
let storage = ParquetStorage::new(store, id)
    .with_parallel_write_settings(num_row_groups.max(1), cols.max(1));

fn valid_write_settings(row_groups: usize, cols: usize) -> bool {
    row_groups >= 1 && cols >= 1
}

When it happens

Trigger: Calling ParquetStorage::with_parallel_write_settings(n, c) where c == 0, e.g. storage.with_parallel_write_settings(4, 0). Typically the 0 comes from a computed value (num_shards * num_columns where a factor is 0) or from an unset config field whose default is 0, not from a literal 0.

Common situations: A new config knob (env var, TOML field) for column-parallel writes that defaults to 0; multiplying a shard/column count where one operand is 0; passing a value derived from a test fixture or a degraded cluster topology that yields an empty set; copy-pasting a call site and dropping one argument.

Related errors

• invalid IOx metadata
• failed to read encoded schema
• invalid statistics
• no time column in metadata statistics
• no min time statistic

AI-assisted analysis of influxdata/influxdb@d28e26e048 (2026-08-16). Data as JSON: /api/errors/006529e62a160663. Report an issue: GitHub.