{"record":{"id":"8bf33d932c7fccce","repo":"influxdata/influxdb","slug":"invalid-iox-metadata","errorCode":null,"errorMessage":"invalid IOx metadata","messagePattern":"invalid IOx metadata","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"core/parquet_file/src/metadata.rs","lineNumber":452,"sourceCode":"            compaction_level: CompactionLevel::Initial,\n            sort_key: None,\n            max_l0_created_at: MaxL0CreatedAt::NotCompacted,\n        }\n    }\n\n    /// Create a corresponding iox catalog's ParquetFile\n    pub fn to_parquet_file<F>(\n        &self,\n        partition_id: PartitionId,\n        partition_hash_id: PartitionHashId,\n        file_size_bytes: u64,\n        metadata: &IoxParquetMetaData,\n        column_id_map: F,\n    ) -> ParquetFileParams\n    where\n        F: for<'a> Fn(&'a str) -> ColumnId,\n    {\n        let decoded = metadata.decode().expect(\"invalid IOx metadata\");\n        trace!(\n            ?partition_id,\n            ?decoded,\n            \"DecodedIoxParquetMetaData decoded from its IoxParquetMetaData\"\n        );\n        let row_count = decoded.row_count();\n        if decoded.md.row_groups().is_empty() {\n            debug!(\n                ?partition_id,\n                \"Decoded IoxParquetMetaData has no row groups to provide useful statistics\"\n            );\n        }\n\n        // Derive the min/max timestamp from the Parquet column statistics.\n        let schema = decoded\n            .read_schema()\n            .expect(\"failed to read encoded schema\");\n        let stats = decoded","sourceCodeStart":434,"sourceCodeEnd":470,"githubUrl":"https://github.com/influxdata/influxdb/blob/d28e26e048401c53cbb98cf2d6ab0cf1e98048ca/core/parquet_file/src/metadata.rs#L434-L470","documentation":"TableMetaData::to_parquet_file converts an IoxParquetMetaData (thrift-encoded, zstd-compressed parquet footer bytes kept separate from the data) into catalog ParquetFileParams, and starts by calling metadata.decode().expect(\"invalid IOx metadata\"). decode() is fallible; it fails when the stored metadata bytes are not valid IOx parquet metadata — corrupted objects, foreign parquet files, or bytes written by an incompatible IOx version.","triggerScenarios":"Calling to_parquet_file on an IoxParquetMetaData whose bytes fail decoding: truncated/corrupted object-store payloads, parquet files produced by non-IOx writers (pandas/arrow/spark lack the IOx footer), or deserialization format changes between writer and reader versions.","commonSituations":"Catalog/compaction code loading files whose metadata blob was bit-rotted or partially written; ingesting externally-written parquet; upgrading IOx across a metadata encoding change; test fixtures with hand-built metadata bytes.","solutions":["Pre-validate before to_parquet_file: call metadata.decode() yourself (it returns Result) and handle the Err instead of letting the expect panic.","Verify the object's integrity (size, checksum/ETag) against the catalog record to catch corruption/truncation.","Confirm the file was written by the IOx write path with the same version; re-write or re-ingest foreign parquet files through IOx.","File/track the upstream conversion expecting to take Result and propagate the error properly."],"exampleFix":"// before\nlet params = table_meta.to_parquet_file(pid, phid, size, &iox_meta, &col_map); // panics via expect\n\n// after\nmatch iox_meta.decode() {\n    Ok(_) => table_meta.to_parquet_file(pid, phid, size, &iox_meta, &col_map),\n    Err(e) => {\n        tracing::error!(%e, \"object has invalid IOx metadata; skipping\");\n        return Err(e.into());\n    }\n}","handlingStrategy":"validation","validationCode":"// pre-validate the metadata bytes before the panicking conversion\nuse parquet_file::metadata::IoxParquetMetaData;\n\nfn metadata_decodes(m: &IoxParquetMetaData) -> bool {\n    m.decode().is_ok()\n}\n\nif !metadata_decodes(&iox_meta) {\n    return quarantine(object_store_id, \"undecodable IOx metadata\");\n}\nlet params = table_meta.to_parquet_file(pid, phid, size, &iox_meta, col_map);","typeGuard":null,"tryCatchPattern":"// if you must call it on untrusted data, contain the panic and classify it\nlet params = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {\n    table_meta.to_parquet_file(pid, phid, size, &iox_meta, &col_map)\n}));\nlet params = match params {\n    Ok(p) => p,\n    Err(payload) => {\n        debug_assert_payload_mentions(&payload, \"invalid IOx metadata\");\n        return Err(CatalogError::InvalidParquetMetadata.into());\n    }\n};","preventionTips":["Treat to_parquet_file as trusting: only feed it metadata from objects you wrote via the IOx write path.","Verify object size/checksum against the catalog before conversion to catch corruption early.","Route external parquet through IOx ingest rather than direct catalog insertion.","Track upstream migration of these expects to Result-returning APIs when upgrading."],"tags":["rust","influxdb-iox","parquet","metadata","deserialization","data-corruption","panic"],"backgroundTag":"metadata-decode-failed","analyzedSha":"d28e26e048401c53cbb98cf2d6ab0cf1e98048ca","analyzedAt":"2026-08-16T19:53:34.623Z","schemaVersion":2},"datasetVersion":"2026-08-16T23:17:17.608Z"}