apache/hadoop · error · RuntimeException

Failed to load EC policy file: {policyFilePath}

Error message

Failed to load EC policy file: {policyFilePath}

What it means

ECPolicyLoader.loadPolicy(path) locates the EC policy XML (a missing file is only a warning returning an empty list) and parses it with a secure DocumentBuilder; the main caller is the `hdfs ec -addPolicies -policyFile` command. Any ParserConfigurationException, IOException or SAXException during the parse is rethrown as a bare RuntimeException 'Failed to load EC policy file: <path>' - the original cause is not attached to the exception, so the message alone tells you little beyond the file path.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ECPolicyLoader.java:73

      = LoggerFactory.getLogger(ECPolicyLoader.class);

  private static final int LAYOUT_VERSION = 1;

  /**
   * Load user defined EC policies from a XML configuration file.
   * @param policyFilePath path of EC policy file
   * @return all valid EC policies in EC policy file
   */
  public List<ErasureCodingPolicy> loadPolicy(String policyFilePath) {
    try {
      File policyFile = getPolicyFile(policyFilePath);
      if (!policyFile.exists()) {
        LOG.warn("Not found any EC policy file");
        return Collections.emptyList();
      }
      return loadECPolicies(policyFile);
    } catch (ParserConfigurationException | IOException | SAXException e) {
      throw new RuntimeException("Failed to load EC policy file: "
          + policyFilePath);
    }
  }

  /**
   * Load EC policies from a XML configuration file.
   * @param policyFile EC policy file
   * @return list of EC policies
   * @throws ParserConfigurationException if ParserConfigurationException happen
   * @throws IOException if no such EC policy file
   * @throws SAXException if the xml file has some invalid elements
   */
  private List<ErasureCodingPolicy> loadECPolicies(File policyFile)
      throws ParserConfigurationException, IOException, SAXException {

    LOG.info("Loading EC policy file " + policyFile);

    // Read and parse the EC policy file.

View on GitHub (pinned to 2add963021)

Solutions

1. Validate syntax first: xmllint --noout <policyFile>
2. Remove any DOCTYPE/DTD - XMLUtils.newSecureDocumentBuilderFactory rejects doctypes/external entities
3. Diff against the stock user-ec-policies.xml shipped in etc/hadoop of your distribution and restore the expected structure
4. If syntax is fine, fix permissions or the path so the process can open the file

Example fix

<!-- before: malformed (unclosed elements, DOCTYPE) -->
<!DOCTYPE configuration>
<configuration>
  <layoutversion>1</layoutversion>
  <schemas>...</schemas>
  <policies><policy><name>RS-10-4-1024k</name></policy>

<!-- after: well-formed, no DOCTYPE -->
<configuration>
  <layoutversion>1</layoutversion>
  <schemas><!-- schema definitions --></schemas>
  <policies>
    <policy><name>RS-10-4-1024k</name><!-- schemaId, cellsize... --></policy>
  </policies>
</configuration>

// Pre-validate before `hdfs ec -addPolicies`: well-formed XML, no DOCTYPE
DocumentBuilderFactory dbf = XMLUtils.newSecureDocumentBuilderFactory();
try { dbf.newDocumentBuilder().parse(new File(policyFilePath)); }
catch (Exception e) { /* reject the file before it reaches the CLI */ }

try {
  List<ErasureCodingPolicy> ps = new ECPolicyLoader().loadPolicy(path);
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Failed to load EC policy file")) {
    // cause is not chained: run xmllint --noout on the path and fix the XML
  } else { throw e; }
}

When it happens

Trigger: Running `hdfs ec -addPolicies -policyFile <path>` (or calling ECPolicyLoader.loadPolicy directly) where the XML is malformed (SAXException), contains a DOCTYPE/DTD the hardened parser rejects, or cannot be read.

Common situations: Hand-edited policy files with unclosed tags; files pasted from the web with encoding damage; DOCTYPE declarations forbidden by the secure parser; wrong permissions on the file.

Related errors

• Bad EC policy configuration file: top-level element not <con
• Bad EC policy configuration file: no <layoutVersion> element
• Value of <layoutVersion> is null
• Bad EC policy configuration file: no <policies> element
• Bad EC policy configuration file: no <schemas> element

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1a28087493076626. Report an issue: GitHub.