influxdata/influxdb · error · CodecError

failed to serialize iox metadata

Error message

failed to serialize iox metadata: {0}

What it means

CodecError::MetadataSerialisation, a #[from] prost::EncodeError wrapper raised when protobuf-encoding the IoxMetadata (the custom IOx metadata stored alongside the Parquet data) fails. The underlying prost encode error is included in the message.

Solutions

1. Check the wrapped prost::EncodeError for the exact field that failed to encode
2. Ensure generated protobuf code and the prost crate version are compatible (re-generate with the matching prost-build)
3. Validate the IoxMetadata contents (required fields populated, no invalid enums) before writing
4. Upgrade prost if the encode failure is a known bug in your pinned version

Example fix

// before: mixing prost versions
gen.prost(ProstConfig::default()); // generated with prost 0.12 while Cargo.toml pins prost 0.11
// after: align versions
gen.prost(ProstConfig::default()); // regenerate and pin the same prost version in Cargo.toml

// ensure metadata encodes before writing
let _ = iox_metadata.clone().encode_to_vec()
    .map_err(|e| format!("iox metadata not encodable: {e}"))?;

match res {
    Err(CodecError::MetadataSerialisation(e)) => {
        error!(%e, "protobuf encode of IoxMetadata failed; check prost version / metadata contents")
    }
    other => other?,
}

When it happens

Trigger: Calling the Parquet write path where encode_to_vec/encode of the IoxMetadata protobuf message fails, e.g. prost encoding an invalid or oversized message.

Common situations: Prost version mismatches between generated code and the prost runtime, corrupted or hand-built IoxMetadata that violates proto invariants, or hitting internal prost encoding limits.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• Invalid InfluxQL metadata
• arrow error
• bitcode error
• catalog format error
• catalog records cannot be generic — the persisted bytes…

AI-assisted analysis of influxdata/influxdb@06200ef96b (2026-09-19). Data as JSON: /api/errors/a917150252284695. Report an issue: GitHub.

Appendix: source

Thrown at core/parquet_file/src/serialize.rs:76

    /// The result stream contained at least one [`RecordBatch`] and all
    /// instances yielded by the stream contained 0 rows.
    ///
    /// This would result in an empty file being uploaded to object store.
    ///
    /// [`RecordBatch`]: arrow::record_batch::RecordBatch
    #[error("no rows to serialise")]
    NoRows,

    /// A DataFusion error during the plan execution.
    ///
    /// Of note: a ResourcesExhaused error likely means the buffer
    /// used for parquet data became too large.
    #[error(transparent)]
    DataFusion(Box<DataFusionError>),

    /// Serialising the [`IoxMetadata`] to protobuf-encoded bytes failed.
    #[error("failed to serialize iox metadata: {0}")]
    MetadataSerialisation(#[from] prost::EncodeError),

    /// Writing the parquet file failed with the specified error.
    #[error("failed to build parquet file: {0}")]
    Writer(#[from] ParquetError),

    /// Attempting to clone a handle to the provided write sink failed.
    #[error("failed to obtain writer handle clone: {0}")]
    CloneSink(std::io::Error),
}

impl From<CodecError> for DataFusionError {
    fn from(value: CodecError) -> Self {
        match value {
            e @ (CodecError::NoRecordBatches
            | CodecError::NoRows
            | CodecError::MetadataSerialisation(_)
            | CodecError::CloneSink(_)) => Self::External(Box::new(e)),

View on GitHub (pinned to 06200ef96b)