{"record":{"id":"efed3d3ba813bc78","repo":"influxdata/influxdb","slug":"failed-to-build-parquet-file-0","errorCode":null,"errorMessage":"failed to build parquet file: {0}","messagePattern":"failed to build parquet file: (.+?)","errorType":"error_code","errorClass":"CodecError","httpStatus":null,"severity":"error","filePath":"core/parquet_file/src/serialize.rs","lineNumber":80,"sourceCode":"    /// This would result in an empty file being uploaded to object store.\n    ///\n    /// [`RecordBatch`]: arrow::record_batch::RecordBatch\n    #[error(\"no rows to serialise\")]\n    NoRows,\n\n    /// A DataFusion error during the plan execution.\n    ///\n    /// Of note: a ResourcesExhaused error likely means the buffer\n    /// used for parquet data became too large.\n    #[error(transparent)]\n    DataFusion(Box<DataFusionError>),\n\n    /// Serialising the [`IoxMetadata`] to protobuf-encoded bytes failed.\n    #[error(\"failed to serialize iox metadata: {0}\")]\n    MetadataSerialisation(#[from] prost::EncodeError),\n\n    /// Writing the parquet file failed with the specified error.\n    #[error(\"failed to build parquet file: {0}\")]\n    Writer(#[from] ParquetError),\n\n    /// Attempting to clone a handle to the provided write sink failed.\n    #[error(\"failed to obtain writer handle clone: {0}\")]\n    CloneSink(std::io::Error),\n}\n\nimpl From<CodecError> for DataFusionError {\n    fn from(value: CodecError) -> Self {\n        match value {\n            e @ (CodecError::NoRecordBatches\n            | CodecError::NoRows\n            | CodecError::MetadataSerialisation(_)\n            | CodecError::CloneSink(_)) => Self::External(Box::new(e)),\n            CodecError::Writer(e) => Self::ParquetError(Box::new(e)),\n            CodecError::DataFusion(e) => *e,\n        }\n    }","sourceCodeStart":62,"sourceCodeEnd":98,"githubUrl":"https://github.com/influxdata/influxdb/blob/06200ef96ba82c5f6727e5038a83af8e722c6875/core/parquet_file/src/serialize.rs#L62-L98","documentation":"CodecError::Writer, a #[from] ParquetError wrapper raised when the underlying Parquet writer fails while building the file (schema mismatch, I/O error on the sink, buffer exhaustion, etc.). Note per docs: a ResourcesExhausted error likely means the in-memory buffer used for parquet data grew too large.","triggerScenarios":"RecordBatch serialization hits ParquetError during arrow-to-parquet write: schema/Arrow type mismatch with the writer schema, sink write failures, or memory budget (ResourcesExhausted) exceeded.","commonSituations":"Writing very large single files exhausting the parquet buffer memory budget; Arrow schema not matching the schema the ParquetWriter was created with; underlying object-store/sink failing mid-write.","solutions":["If the message mentions ResourcesExhausted, reduce batch size / file size or increase the memory budget for parquet writing","Verify the Arrow record batch schema exactly matches the schema given to the Parquet writer","Check the sink (file/object store) health and permissions; surface the inner ParquetError details","Retry the write if the failure was transient I/O"],"exampleFix":"// before: one giant batch exhausting the buffer\nlet batch = concat_batches(&schema, all_batches)?;\nwrite(batch)?;\n// after: write in bounded chunks\nfor chunk in all_batches.chunks(64) {\n    let batch = concat_batches(&schema, chunk)?;\n    write(batch)?;\n}","handlingStrategy":"try-catch","validationCode":"if batch.schema() != expected_schema {\n    return Err(format!(\"batch schema {:?} != writer schema {:?}\",\n        batch.schema(), expected_schema));\n}","typeGuard":null,"tryCatchPattern":"match res {\n    Err(CodecError::Writer(ParquetError::General(msg)))\n        if msg.contains(\"ResourcesExhausted\") =>\n    {\n        warn!(\"parquet buffer exhausted; reduce file size or raise memory budget\")\n    }\n    other => other?,\n}","preventionTips":["Bound batch/file sizes so the parquet buffer cannot exhaust memory","Assert record batch schema equals the writer schema before writing","Monitor sink (object store/disk) health for mid-write failures"],"tags":["parquet","writer","memory","io"],"backgroundTag":"file-write-failed","analyzedSha":"06200ef96ba82c5f6727e5038a83af8e722c6875","analyzedAt":"2026-09-19T12:55:30.003Z","contentChangedAt":"2026-09-19T12:55:30.003Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}