{"record":{"id":"8682bf54dac0dacc","repo":"apache/iceberg","slug":"can-t-index-multiple-dvs-for-s-s-and-s","errorCode":null,"errorMessage":"Can't index multiple DVs for %s: %s and %s","messagePattern":"Can't index multiple DVs for (.+?): (.+?) and (.+?)","errorType":"validation","errorClass":"ValidationException","httpStatus":null,"severity":"critical","filePath":"core/src/main/java/org/apache/iceberg/DeleteFileIndex.java","lineNumber":572,"sourceCode":"            throw new UnsupportedOperationException(\"Unsupported content: \" + file.content());\n        }\n        ScanMetricsUtil.indexedDeleteFile(scanMetrics, file);\n      }\n\n      return new DeleteFileIndex(\n          globalDeletes.isEmpty() ? null : globalDeletes,\n          eqDeletesByPartition.isEmpty() ? null : eqDeletesByPartition,\n          posDeletesByPartition.isEmpty() ? null : posDeletesByPartition,\n          posDeletesByPath.isEmpty() ? null : posDeletesByPath,\n          dvByPath.isEmpty() ? null : dvByPath,\n          specsById.isEmpty() ? null : specsById);\n    }\n\n    private void add(Map<String, DeleteFile> dvByPath, DeleteFile dv) {\n      String path = dv.referencedDataFile();\n      DeleteFile existingDV = dvByPath.putIfAbsent(path, dv);\n      if (existingDV != null) {\n        throw new ValidationException(\n            \"Can't index multiple DVs for %s: %s and %s\",\n            path, ContentFileUtil.dvDesc(dv), ContentFileUtil.dvDesc(existingDV));\n      }\n    }\n\n    private void add(\n        Map<String, PositionDeletes> deletesByPath,\n        PartitionMap<PositionDeletes> deletesByPartition,\n        DeleteFile file) {\n      String path = ContentFileUtil.referencedDataFileLocation(file);\n\n      PositionDeletes deletes;\n      if (path != null) {\n        deletes = deletesByPath.computeIfAbsent(path, ignored -> new PositionDeletes());\n      } else {\n        int specId = file.specId();\n        StructLike partition = file.partition();\n        deletes = deletesByPartition.computeIfAbsent(specId, partition, PositionDeletes::new);","sourceCodeStart":554,"sourceCodeEnd":590,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/DeleteFileIndex.java#L554-L590","documentation":"DeleteFileIndex.Builder.add refuses to index more than one deletion vector (DV) for the same referenced data file path. Iceberg allows at most one DV per data file; `putIfAbsent` returns the previously stored DV when a second one maps to the same path, and the builder throws ValidationException naming both files. This guards the commit path against producing a snapshot whose DV index is ambiguous.","triggerScenarios":"Adding two deletion vectors whose `referencedDataFile()` returns the same path, e.g. building a snapshot where an overwrite/rewrite operation re-registered a DV for a data file that already has one, or re-adding the same DV delete file to the index builder twice via different AddDeleteFiles calls.","commonSituations":"Concurrent or duplicated compaction/rewrite jobs both producing DVs for the same data file; a buggy writer emitting the same DV path twice; replaying or re-applying delete files when constructing a snapshot; manually assembled table metadata referencing duplicate DVs.","solutions":["Inspect the reported DV descriptors (path, content offsets) and remove the duplicate delete file so only one DV per data file remains before committing.","Ensure the writing engine (Spark/Flink) does not re-run the same rewrite/compaction plan concurrently against the same data files; use retries with snapshot conflict re-planning.","If a DV was written twice at different offsets, keep the newest one and expire/delete the stale DV file, then re-run the operation.","Check for double-adding the same DeleteFile in custom catalog/commit code that calls DeleteFileIndex.Builder."],"exampleFix":"// before: appending a DV without checking the data file already has one\nindexBuilder.add(dv);\n// after: guard by tracking DVs per referenced data file\nif (dvPaths.add(dv.referencedDataFile())) {\n  indexBuilder.add(dv);\n} else {\n  throw new IllegalStateException(\"DV already indexed for \" + dv.referencedDataFile());\n}","handlingStrategy":"validation","validationCode":"java.util.Set<String> seen = new HashSet<>();\nfor (DeleteFile dv : candidateDvs) {\n  if (!seen.add(dv.referencedDataFile())) {\n    throw new IllegalStateException(\"Duplicate DV for data file: \" + dv.referencedDataFile());\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  append.commit();\n} catch (ValidationException e) {\n  if (e.getMessage().startsWith(\"Can't index multiple DVs\")) {\n    // re-plan the operation against the latest snapshot, dropping the duplicate DV\n  } else throw e;\n}","preventionTips":["Never allow two concurrent rewrite/compaction jobs to produce DVs for the same data files.","Track referenced data files with a set before registering DVs in custom commit code.","Rebase operations on the latest snapshot before committing to avoid re-adding stale DVs."],"tags":["deletion-vector","duplicate-file","validation","commit"],"backgroundTag":"internal-invariant-violation","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}