influxdata/influxdb · error · UploadError

failed to properly configure parquet upload

Error message

failed to properly configure parquet upload: {0}

What it means

UploadError::Config, a #[from] DataFusionError wrapper for configuration errors when setting up parquet upload. Raised when the parquet upload path is configured with invalid options (e.g. invalid writer properties parsed into DataFusion config).

Solutions

1. Read the wrapped DataFusionError to see which configuration key/value was rejected
2. Fix the parquet writer options (valid compression codec, sane row group/page sizes) before constructing the upload sink
3. Validate configuration at startup, before any write is attempted
4. Check for DataFusion version changes that renamed or removed config options

Example fix

// before
"datafusion.execution.parquet.compression" => "brotli9x" // invalid value
// after
"datafusion.execution.parquet.compression" => "zstd(3)" // valid config value

// validate parquet writer config at startup
let opts = ParquetOptions::try_from_config(&config)
    .map_err(|e| format!("invalid parquet upload config: {e}"))?;

match res {
    Err(UploadError::Config(e)) => {
        error!(%e, "parquet upload misconfigured; fix options before retrying")
    }
    other => other?,
}

When it happens

Trigger: Constructing the parquet upload sink with invalid configuration — e.g. malformed parquet writer options supplied through DataFusion config, invalid enum values, or out-of-range settings.

Common situations: Typo'd or unsupported parquet writer option keys in config, invalid compression names, max row group sizes that violate constraints, config supplied as an unsupported DataFusion type.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• failed to construct IOx parquet metadata
• num_columns_in_parallel should be above zero
• num_row_groups_in_parallel should be above zero
• An unexpected error occurred in the client library
• at least one expr

AI-assisted analysis of influxdata/influxdb@06200ef96b (2026-09-19). Data as JSON: /api/errors/a5e54bbbc289a319. Report an issue: GitHub.

Appendix: source

Thrown at core/parquet_file/src/storage.rs:62

///
/// [`RecordBatch`]: arrow::record_batch::RecordBatch
#[derive(Debug, Error)]
pub enum UploadError {
    /// A codec failure during serialisation.
    #[error(transparent)]
    Serialise(#[from] CodecError),

    /// An error during Parquet metadata conversion when attempting to
    /// instantiate a valid [`IoxParquetMetaData`] instance.
    #[error("failed to construct IOx parquet metadata: {0}")]
    Metadata(crate::metadata::Error),

    /// Uploading the Parquet file to object store failed.
    #[error("failed to upload to object storage: {0}")]
    Upload(#[from] object_store::Error),

    /// Error in configuration.
    #[error("failed to properly configure parquet upload: {0}")]
    Config(#[from] DataFusionError),
}

impl From<UploadError> for DataFusionError {
    fn from(value: UploadError) -> Self {
        match value {
            UploadError::Serialise(e) => {
                Self::Context(String::from("serialize"), Box::new(e.into()))
            }
            UploadError::Metadata(e) => Self::External(Box::new(e)),
            UploadError::Upload(e) => Self::ObjectStore(Box::new(e)),
            UploadError::Config(e) => Self::External(e.into()),
        }
    }
}

/// ID for an object store hooked up into DataFusion.
#[derive(Debug, PartialEq, Eq, Hash, Clone, Copy)]

View on GitHub (pinned to 06200ef96b)