influxdata/influxdb · error · UploadError

failed to upload to object storage

Error message

failed to upload to object storage: {0}

What it means

UploadError::Upload, a #[from] object_store::Error wrapper raised when uploading the Parquet file to object storage fails. Any object_store client failure (network, permissions, quota, throttling) is surfaced here.

Solutions

1. Check the wrapped object_store::Error source for the provider-specific cause
2. Verify credentials and bucket/container configuration for the object store
3. Retry the upload with backoff if the error is transient (5xx, throttling, timeout)
4. Confirm network connectivity and that the target path respects store naming/size rules

Example fix

// before: single attempt
store.put(&path, payload).await?;
// after: bounded retry with backoff
backoff::retry(_backoff::ExponentialBackoff::default(), || async {
    match store.put(&path, payload.clone()).await {
        Ok(_) => Ok(()),
        Err(e) if e.is_timeout() || is_throttling(&e) => Err(backoff::Error::transient(e)),
        Err(e) => Err(backoff::Error::permanent(e)),
    }
}).await?;

// pre-flight: verify credentials/bucket before writing
store.head(&Location::from_url(&bucket_probe_url)).await?;

match res {
    Err(UploadError::Upload(e)) if is_transient(&e) => {
        retry_with_backoff(upload_closure).await?
    }
    other => other?,
}

When it happens

Trigger: put/multipart upload call to the configured object store (S3, GCS, Azure, local) returns an error during the parquet upload path.

Common situations: Expired or missing cloud credentials, bucket not existing, network outages, rate limiting (SlowDown/Throttling), disk full for local stores, object size limits exceeded.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

• object_store error
• client disconnected
• client error
• Deadline expired before operation could complete
• error fetching plugin from repository

AI-assisted analysis of influxdata/influxdb@06200ef96b (2026-09-19). Data as JSON: /api/errors/17a9af69cfa82fd3. Report an issue: GitHub.

Appendix: source

Thrown at core/parquet_file/src/storage.rs:58

/// Errors returned during a Parquet "put" operation, covering [`RecordBatch`]
/// pull from the provided stream, encoding, and finally uploading the bytes to
/// the object store.
///
/// [`RecordBatch`]: arrow::record_batch::RecordBatch
#[derive(Debug, Error)]
pub enum UploadError {
    /// A codec failure during serialisation.
    #[error(transparent)]
    Serialise(#[from] CodecError),

    /// An error during Parquet metadata conversion when attempting to
    /// instantiate a valid [`IoxParquetMetaData`] instance.
    #[error("failed to construct IOx parquet metadata: {0}")]
    Metadata(crate::metadata::Error),

    /// Uploading the Parquet file to object store failed.
    #[error("failed to upload to object storage: {0}")]
    Upload(#[from] object_store::Error),

    /// Error in configuration.
    #[error("failed to properly configure parquet upload: {0}")]
    Config(#[from] DataFusionError),
}

impl From<UploadError> for DataFusionError {
    fn from(value: UploadError) -> Self {
        match value {
            UploadError::Serialise(e) => {
                Self::Context(String::from("serialize"), Box::new(e.into()))
            }
            UploadError::Metadata(e) => Self::External(Box::new(e)),
            UploadError::Upload(e) => Self::ObjectStore(Box::new(e)),
            UploadError::Config(e) => Self::External(e.into()),
        }
    }

View on GitHub (pinned to 06200ef96b)