{"record":{"id":"da4b8c98b3941598","repo":"quickwit-oss/quickwit","slug":"source-is-defined-more-than-once","errorCode":null,"errorMessage":"source `{}` is defined more than once","messagePattern":"source `(.+?)` is defined more than once","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-metastore/src/metastore/index_metadata/serialize.rs","lineNumber":87,"sourceCode":"    pub index_uid: IndexUid,\n    #[schema(value_type = VersionedIndexConfig)]\n    pub index_config: IndexConfig,\n    #[schema(value_type = Object)]\n    pub checkpoint: IndexCheckpoint,\n    #[serde(default = \"utc_now_timestamp\")]\n    pub create_timestamp: i64,\n    #[schema(value_type = Vec<VersionedSourceConfig>)]\n    pub sources: Vec<SourceConfig>,\n}\n\nimpl TryFrom<IndexMetadataV0_8> for IndexMetadata {\n    type Error = anyhow::Error;\n\n    fn try_from(v0_8: IndexMetadataV0_8) -> anyhow::Result<Self> {\n        let mut sources: HashMap<String, SourceConfig> = Default::default();\n        for source in v0_8.sources {\n            if sources.contains_key(&source.source_id) {\n                anyhow::bail!(\"source `{}` is defined more than once\", source.source_id);\n            }\n            sources.insert(source.source_id.clone(), source);\n        }\n        Ok(Self {\n            index_uid: v0_8.index_uid,\n            index_config: v0_8.index_config,\n            checkpoint: v0_8.checkpoint,\n            create_timestamp: v0_8.create_timestamp,\n            sources,\n        })\n    }\n}\n","sourceCodeStart":69,"sourceCodeEnd":100,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-metastore/src/metastore/index_metadata/serialize.rs#L69-L100","documentation":"Thrown when deserializing a v0.8 index metadata document that contains two sources with the same source_id. The migration code converts the source list to a map keyed by source_id and treats duplicates as corrupted input rather than silently dropping one.","triggerScenarios":"Loading/migrating an old IndexMetadataV0_8 whose `sources` array contains two entries with identical source_id, e.g. a hand-edited or corrupted metastore JSON file.","commonSituations":"Upgrading Quickwit from an old version with metastore files edited manually or written by a buggy tool; JSON merge conflicts producing duplicated source entries.","solutions":["Inspect the metastore index JSON file and remove/merge the duplicate source entries with the same source_id","Fix whatever tool or manual edit produced the duplicate and re-export the metadata","Restore the index metadata file from a backup taken before the corruption"],"exampleFix":"// before\n\"sources\": [{\"source_id\":\"kafka\"},{\"source_id\":\"kafka\"}]\n// after\n\"sources\": [{\"source_id\":\"kafka\"}]","handlingStrategy":"validation","validationCode":"let ids: HashSet<&str> = v08.sources.iter().map(|s| s.source_id.as_str()).collect();\nif ids.len() != v08.sources.len() {\n    panic!(\"duplicate source_id in index metadata\");\n}","typeGuard":"fn has_unique_sources(sources: &[SourceConfig]) -> bool {\n    let ids: HashSet<_> = sources.iter().map(|s| s.source_id.as_str()).collect();\n    ids.len() == sources.len()\n}","tryCatchPattern":"match IndexMetadata::try_from(v0_8) {\n    Err(e) if e.to_string().contains(\"defined more than once\") => {\n        eprintln!(\"corrupted metadata: {e}; restore from backup\");\n    }\n    other => other?,\n}","preventionTips":["Never hand-edit metastore index metadata JSON","Validate uniqueness of source_id before writing metadata files","Keep backups before Quickwit version upgrades"],"tags":["metastore","migration","deserialization","duplicate"],"backgroundTag":"schema-validation-failed","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}