apache/hadoop · error · IllegalArgumentException

Bad rule definition: {bad_lines}

Error message

Bad rule definition: {bad_lines}

What it means

HostRestrictingAuthorizationFilter (host-based authorization for WebHDFS/HTTPFS) parses the rule string from the config 'dfs.web.authentication.host.allow.rules' (RESTRICTION_CONFIG under prefix dfs.web.authentication.). Rules are separated by '|' or newlines and each must split on commas into exactly 3 parts (user, network/bits, path). If grouping the splits by length yields anything other than {3}, the filter re-materializes the offending lines and throws IllegalArgumentException('Bad rule definition: ...') during initialization.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HostRestrictingAuthorizationFilter.java:173

  private void loadRuleMap(String ruleString) throws IllegalArgumentException {
    if (ruleString == null || ruleString.equals("")) {
      LOG.debug("Got no rules - will disallow anyone access");
    } else {
      // value: user1,network/bits1,path_glob1|user2,network/bits2,path_glob2...
      Pattern comma_split = Pattern.compile(",");
      Pattern rule_split = Pattern.compile("\\||\n");
      // split all rule lines
      Map<Integer, List<String[]>> splits = rule_split.splitAsStream(ruleString)
          .map(x -> comma_split.split(x, 3))
          .collect(Collectors.groupingBy(x -> x.length));
      // verify all rules have three parts
      if (!splits.keySet().equals(Collections.singleton(3))) {
        // instead of re-joining parts, re-materialize lines which do not split
        // correctly for the exception
        String bad_lines = rule_split.splitAsStream(ruleString)
            .filter(x -> comma_split.split(x, 3).length != 3)
            .collect(Collectors.joining("\n"));
        throw new IllegalArgumentException("Bad rule definition: " + bad_lines);
      }
      // create a list of Rules
      int user = 0;
      int cidr = 1;
      int path = 2;
      BiFunction<CopyOnWriteArrayList<Rule>, CopyOnWriteArrayList<Rule>,
          CopyOnWriteArrayList<Rule>> arrayListMerge = (v1, v2) -> {
        v1.addAll(v2);
        return v1;
      };
      for (String[] split : splits.get(3)) {
        LOG.debug("Loaded rule: user: {}, network/bits: {} path: {}",
            split[user], split[cidr], split[path]);
        Rule rule = (split[cidr].trim().equals("*") ? new Rule(null,
            split[path]) : new Rule(new SubnetUtils(split[cidr]).getInfo(),
            split[path]));
        // Rule map is {"user": [rule1, rule2, ...]}, update the user's array
        CopyOnWriteArrayList<Rule> arrayListRule =

View on GitHub (pinned to 2add963021)

Solutions

1. Rewrite every rule as a strict 'user,cidr,path' triplet, e.g. '*,127.0.0.0/8,/webhdfs/v1', separated by '|' or newlines
2. Validate the rule string with the same split logic (Pattern "\\||\n" then split(',', 3).length == 3) in a pre-deploy check before restarting the service
3. After fixing, restart the httpfs/webhdfs service so the filter re-initializes, and confirm the 'Loaded rule' debug lines appear

Example fix

# before (dfs.web.authentication.host.allow.rules)
alice,10.0.0.0/8          # missing path field -> Bad rule definition

# after
alice,10.0.0.0/8,/webhdfs/v1|bob,192.168.0.0/16,/webhdfs/v1

static boolean isValidRuleString(String ruleString) {
  return Pattern.compile("\\||\n").splitAsStream(ruleString)
      .allMatch(line -> line.split(",", 3).length == 3
          && !line.startsWith(",") && !line.endsWith(","));
}

if (!isValidRuleString(rules)) throw new IllegalArgumentException("Bad rule definition: " + rules);

static boolean isWellFormedRuleLine(String line) {
  String[] parts = line.split(",", 3);
  return parts.length == 3
      && !parts[0].isEmpty() && !parts[1].isEmpty() && !parts[2].isEmpty();
}

try {
  filter.init(filterConfig);
} catch (IllegalArgumentException e) {
  if (e.getMessage().startsWith("Bad rule definition")) {
    failDeployment("host.allow.rules malformed: " + e.getMessage()); // config error, not transient
  } else throw e;
}

When it happens

Trigger: Deploying the filter with a rule line containing only 2 comma-separated fields (missing path), 4+ fields where the first two commas are consumed but later validation of structure fails, empty fields, or a stray line from newline mangling — note 'a,b,c,d' splits to length 3 only when the limit-3 split keeps the tail together, but 'a,b' or 'a' yields a non-3 length and trips the check.

Common situations: Editing host.allow.rules and dropping the path column; XML config that collapses or breaks newlines so two rules merge; quoting errors when the pipe separator is interpreted by shells or config templating.

Related errors

• Failed to parse "{str}" for the parameter {varName}. The va
• getErasureCodeCodecs is not supported for HttpFs on {0}. Ple
• getTrashRoots is not supported for HttpFs on {0}. Please che
• Cannot determine UGI from request or conf
• Usernames not matched: expecting null but name={name}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ae2e3a45583f598b. Report an issue: GitHub.