{"record":{"id":"9d5af49b79c3d9a3","repo":"apache/hadoop","slug":"repetitive-schemas-in-ec-policy-configuration-file","errorCode":null,"errorMessage":"Repetitive schemas in EC policy configuration file: {schemaId}","messagePattern":"Repetitive schemas in EC policy configuration file: (.+?)","errorType":"validation","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ECPolicyLoader.java","lineNumber":173,"sourceCode":"   * Load schemas from root element in the XML configuration file.\n   * @param root root element\n   * @return EC schema map\n   */\n  private Map<String, ECSchema> loadSchemas(Element root) {\n    NodeList elements = root.getElementsByTagName(\"schemas\")\n        .item(0).getChildNodes();\n    Map<String, ECSchema> schemas = new HashMap<String, ECSchema>();\n    for (int i = 0; i < elements.getLength(); i++) {\n      Node node = elements.item(i);\n      if (node instanceof Element) {\n        Element element = (Element) node;\n        if (\"schema\".equals(element.getTagName())) {\n          String schemaId = element.getAttribute(\"id\");\n          ECSchema schema = loadSchema(element);\n          if (!schemas.containsValue(schema)) {\n            schemas.put(schemaId, schema);\n          } else {\n            throw new RuntimeException(\"Repetitive schemas in EC policy\"\n                + \" configuration file: \" + schemaId);\n          }\n        } else {\n          throw new RuntimeException(\"Bad element in EC policy\"\n              + \" configuration file: \" + element.getTagName());\n        }\n      }\n    }\n\n    return schemas;\n  }\n\n  /**\n   * Load EC policies from root element in the XML configuration file.\n   * @param root root element\n   * @param schemas schema map\n   * @return EC policy list\n   */","sourceCodeStart":155,"sourceCodeEnd":191,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ECPolicyLoader.java#L155-L191","documentation":"ECPolicyLoader.loadSchemas() rejects duplicate schemas: before inserting each <schema id=\"...\"> it checks schemas.containsValue(schema), i.e. equality of the parsed ECSchema (codec, k/m, options), not the id. Two <schema> entries that produce equal ECSchema objects — even under different ids — make the second one throw this RuntimeException. The message reports the id of the offending (second) entry.","triggerScenarios":"`hdfs ec -loadPolicy` where the <schemas> block defines the same erasure schema twice: identical id and parameters, or different ids (e.g. 'rs-6-3' and 'rs-6-3-copy') with the same codec and k/m values.","commonSituations":"Copy-pasting a schema block to tweak a name but leaving k/m/codec unchanged, then referencing both ids from policies; merging policy files that each define RS(6,3).","solutions":["Delete the duplicate <schema> entry and keep one canonical definition per distinct codec/k/m combination","Give each remaining schema a unique id and reference that id from every <policy><schema> element","If two policies need the same schema, share one <schema> definition — ids are referenced, not redefined"],"exampleFix":"<!-- before -->\n<schema id=\"rs-6-3\">\n  <k>6</k><m>3</m><codec>rs</codec>\n</schema>\n<schema id=\"rs-6-3-alt\">\n  <k>6</k><m>3</m><codec>rs</codec>\n</schema>\n\n<!-- after -->\n<schema id=\"rs-6-3\">\n  <k>6</k><m>3</m><codec>rs</codec>\n</schema>\n<!-- policies both reference <schema>rs-6-3</schema> -->","handlingStrategy":"validation","validationCode":"Set<String> seen = new HashSet<>();\nfor (Element s : schemaElements(doc)) {\n  String key = s.getAttribute(\"codec\") + \":\" + textOf(s, \"k\") + \":\" + textOf(s, \"m\");\n  if (!seen.add(key)) {\n    throw new IllegalArgumentException(\"duplicate schema params (codec:k:m) at id \" + s.getAttribute(\"id\"));\n  }\n}","typeGuard":null,"tryCatchPattern":"try { new ECPolicyLoader().loadPolicy(path); }\ncatch (RuntimeException e) {\n  LOG.error(\"EC policy file {} rejected: {}\", path, e.getMessage());\n}","preventionTips":["Define each codec/k/m combination exactly once and reference it by id","Remember equality is by schema content, not id — renaming a duplicate does not fix it"],"tags":["hdfs","erasure-coding","ec-policy","xml","duplicate-entry"],"backgroundTag":"duplicate-config-entry","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}