{"record":{"id":"2b23a6052b304f43","repo":"influxdata/influxdb","slug":"failed-to-read-encoded-schema","errorCode":null,"errorMessage":"failed to read encoded schema","messagePattern":"failed to read encoded schema","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"core/parquet_file/src/metadata.rs","lineNumber":469,"sourceCode":"    {\n        let decoded = metadata.decode().expect(\"invalid IOx metadata\");\n        trace!(\n            ?partition_id,\n            ?decoded,\n            \"DecodedIoxParquetMetaData decoded from its IoxParquetMetaData\"\n        );\n        let row_count = decoded.row_count();\n        if decoded.md.row_groups().is_empty() {\n            debug!(\n                ?partition_id,\n                \"Decoded IoxParquetMetaData has no row groups to provide useful statistics\"\n            );\n        }\n\n        // Derive the min/max timestamp from the Parquet column statistics.\n        let schema = decoded\n            .read_schema()\n            .expect(\"failed to read encoded schema\");\n        let stats = decoded\n            .read_statistics(&schema)\n            .expect(\"invalid statistics\");\n        let columns: Vec<_> = stats.iter().map(|v| column_id_map(&v.name)).collect();\n\n        // Extract the min/max timestamps.\n        let TimestampRange {\n            min: min_time,\n            max: max_time,\n        } = derive_min_max_time(stats);\n\n        ParquetFileParams {\n            namespace_id: self.namespace_id,\n            table_id: self.table_id,\n            partition_id,\n            partition_hash_id,\n            object_store_id: self.object_store_id,\n            min_time,","sourceCodeStart":451,"sourceCodeEnd":487,"githubUrl":"https://github.com/influxdata/influxdb/blob/d28e26e048401c53cbb98cf2d6ab0cf1e98048ca/core/parquet_file/src/metadata.rs#L451-L487","documentation":"After decoding, to_parquet_file derives the arrow/IOx Schema via decoded.read_schema().expect(\"failed to read encoded schema\"). read_schema() deserializes the IOx schema that the IOx write path embeds in the parquet key-value metadata. It fails when that embedded schema is missing or malformed — typically parquet data produced outside the IOx write path, or metadata that passed decode() but lacks/has-corrupt schema bytes.","triggerScenarios":"to_parquet_file on files whose thrift footer decoded but the embedded IOx schema key-value is absent (non-IOx writer) or unparseable (version skew in the schema serialization, truncated custom metadata).","commonSituations":"Loading externally generated parquet into IOx catalog flows; test batches serialized without IOx schema metadata (the codebase docs call this exact failure out); rolling upgrades where the schema encoding changed.","solutions":["Pre-flight the metadata: run decode()?.read_schema() yourself and skip/quarantine files that error, instead of calling to_parquet_file blindly.","Re-ingest the parquet through the IOx write path so the schema (and IOx field-type metadata like the time column) is embedded correctly.","Check writer vs reader versions after upgrades; rewrite old files if the embedded schema format changed.","Push for to_parquet_file to return Result rather than expect (upstream improvement)."],"exampleFix":"// before\nlet params = table_meta.to_parquet_file(pid, phid, size, &iox_meta, &col_map);\n\n// after\nif let Err(e) = iox_meta.decode().and_then(|d| d.read_schema()) {\n    tracing::warn!(%e, \"file lacks valid IOx schema metadata\");\n    return quarantine(object_store_id);\n}\nlet params = table_meta.to_parquet_file(pid, phid, size, &iox_meta, &col_map);","handlingStrategy":"validation","validationCode":"// confirm the embedded IOx schema exists and parses before converting\nfn has_iox_schema(m: &IoxParquetMetaData) -> bool {\n    match m.decode() {\n        Ok(d) => d.read_schema().is_ok(),\n        Err(_) => false,\n    }\n}\n\nif !has_iox_schema(&iox_meta) {\n    return quarantine(object_store_id, \"missing/unparseable IOx schema metadata\");\n}","typeGuard":null,"tryCatchPattern":"let ok = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {\n    table_meta.to_parquet_file(pid, phid, size, &iox_meta, &col_map)\n}));\nif let Err(_) = ok {\n    return Err(CatalogError::MissingIoxSchema.into());\n}","preventionTips":["Only catalog files written by the IOx write path (it embeds the schema in footer key-value metadata).","In tests, build batches via iox test helpers rather than plain arrow writers.","After version upgrades, spot-check that decode()+read_schema() still succeed on a sample of existing objects.","Keep a quarantine path for objects failing pre-validation instead of letting them panic the catalog job."],"tags":["rust","influxdb-iox","parquet","schema","deserialization","panic"],"backgroundTag":"metadata-decode-failed","analyzedSha":"d28e26e048401c53cbb98cf2d6ab0cf1e98048ca","analyzedAt":"2026-08-16T19:53:34.623Z","schemaVersion":2},"datasetVersion":"2026-08-16T23:17:17.608Z"}