What causes "Auth '{}' not of expected form scheme:auth" in apache/hadoop?

Format every entry as scheme:auth, e.g. "digest:user:password" or "kerberos:user@REALM".

apache/hadoop · error · BadAuthFormatException

Auth '{}' not of expected form scheme:auth

Error message

Auth '{}' not of expected form scheme:auth

What it means

ZKUtil.stringToAuth splits each comma-separated auth entry on ':' with a limit of 2, expecting 'scheme:auth'. If an entry contains no colon at all, the split yields a single-element array and the method throws BadAuthFormatException (a HadoopIllegalArgumentException) with this message. With the limit-2 split, 'digest:user:pass' is fine — everything after the first colon is the auth blob — but 'digest' alone is not.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/ZKUtil.java:147

   * @param authString the comma-separated auth mechanisms
   * @return a list of parsed authentications
   * @throws BadAuthFormatException if the auth format is invalid
   */
  public static List<ZKAuthInfo> parseAuth(String authString) throws
      BadAuthFormatException{
    List<ZKAuthInfo> ret = Lists.newArrayList();
    if (authString == null) {
      return ret;
    }
    
    List<String> authComps = Lists.newArrayList(
        Splitter.on(',').omitEmptyStrings().trimResults()
        .split(authString));
    
    for (String comp : authComps) {
      String parts[] = comp.split(":", 2);
      if (parts.length != 2) {
        throw new BadAuthFormatException(
            "Auth '" + comp + "' not of expected form scheme:auth");
      }
      ret.add(new ZKAuthInfo(parts[0],
          parts[1].getBytes(StandardCharsets.UTF_8)));
    }
    return ret;
  }
  
  /**
   * Because ZK ACLs and authentication information may be secret,
   * allow the configuration values to be indirected through a file
   * by specifying the configuration as "@/path/to/file". If this
   * syntax is used, this function will return the contents of the file
   * as a String.
   * 
   * @param valInConf the value from the Configuration 
   * @return either the same value, or the contents of the referenced
   * file if the configured value starts with "@"

View on GitHub (pinned to 2add963021)

Solutions

Format every entry as scheme:auth, e.g. "digest:user:password" or "kerberos:user@REALM".
Fix the auth config property so each comma-separated entry contains a colon.
Pre-validate entries with ^[^:]+:.+$ before calling stringToAuths.
Catch BadAuthFormatException at config load and name the offending entry.

Example fix

// before
List<ZKAuthInfo> auth = ZKUtil.stringToAuths("digest");
// throws: Auth 'digest' not of expected form scheme:auth

// after
List<ZKAuthInfo> auth = ZKUtil.stringToAuths("digest:alice:secret");

Defensive patterns

Strategy: validation

Validate before calling

private static final Pattern AUTH_ENTRY =
    Pattern.compile("^[^:,]+:.+$");

static void validateAuthString(String auth) {
  for (String entry : auth.split(",")) {
    if (!AUTH_ENTRY.matcher(entry).matches()) {
      throw new IllegalArgumentException(
          "Auth entry '" + entry + "' must be scheme:auth");
    }
  }
}

validateAuthString(authConf); // before ZKUtil.stringToAuths

Try / catch

try {
  auths = ZKUtil.stringToAuths(authString);
} catch (ZKUtil.BadAuthFormatException e) {
  throw new ConfigurationException("Bad zk auth config: " + e.getMessage(), e);
}

Prevention

Template zk auth values as scheme:auth (digest:user:pass, kerberos:principal).
Validate auth config at startup with a scheme:auth regex.
Never assume a bare scheme name is accepted; the colon separator is mandatory.

When it happens

Trigger: ZKUtil.stringToAuths("digest") — scheme only; "kerberos" without a principal; "digest:user@realm" intended as credential but missing ':pass' is actually fine ('user@realm' becomes the auth blob) while "digest:" is fine too (empty auth); the failing case is strictly zero colons.

Common situations: ZooKeeper auth configuration (e.g. YARN registry 'hadoop.registry.zk.auth') written as a bare scheme; auth entries assembled from variables where the separator or credential part was lost; using '@file' indirection syntax inside the wrong property instead of as the whole value.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/96aa8acb84accf12. Report an issue: GitHub.