apache/hadoop · error · IOException

Unable to parse permission string {}: expected 3 components,

Error message

Unable to parse permission string {}: expected 3 components, but only had {}

What it means

permissionXmlToU64 splits an inode's <permission> string on ':' and requires exactly three components - user, group, and symbolic mode (e.g. hdfs:supergroup:rwxr-xr-x) - to pack into the 64-bit permission long. The split produced a different component count, so the string cannot be encoded and the inode cannot be written.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageReconstructor.java:1610

        bld.build().writeDelimitedTo(out);
      }
      expectTagEnd(SNAPSHOT_DIFF_SECTION_FILE_DIFF_ENTRY);
    }
  }

  /**
   * Permission is serialized as a 64-bit long. [0:24):[25:48):[48:64)
   * (in Big Endian).  The first and the second parts are the string ids
   * of the user and group name, and the last 16 bits are the permission bits.
   *
   * @param perm           The permission string from the XML.
   * @return               The 64-bit value to use in the fsimage for permission.
   * @throws IOException   If we run out of string IDs in the string table.
   */
  private long permissionXmlToU64(String perm) throws IOException {
    String components[] = perm.split(":");
    if (components.length != 3) {
      throw new IOException("Unable to parse permission string " + perm +
          ": expected 3 components, but only had " + components.length);
    }
    String userName = components[0];
    String groupName = components[1];
    String modeString = components[2];
    long userNameId = registerStringId(userName);
    long groupNameId = registerStringId(groupName);
    long mode = new FsPermission(modeString).toShort();
    return (userNameId << 40) | (groupNameId << 16) | mode;
  }

  /**
   * The FSImage contains a string table which maps strings to IDs.
   * This is a simple form of compression which takes advantage of the fact
   * that the same strings tend to occur over and over again.
   * This function will return an ID which we can use to represent the given
   * string.  If the string already exists in the string table, we will use
   * that ID; otherwise, we will allocate a new one.

View on GitHub (pinned to 2add963021)

Solutions

1. Set every <permission> to full 'USER:GROUP:MODE' form, e.g. hdfs:supergroup:rwxr-xr-x
2. Grep the XML for values that do not match ^[^:]+:[^:]+:[rwxstX-]{9,10}$ and fix each
3. If a username legitimately contains ':', it cannot be represented in this format - rewrite it or regenerate from the original image
4. Validate all permission strings with a streaming pre-check before ReverseXML

Example fix

<!-- before -->
<permission>rwxr-xr-x</permission>
<!-- after -->
<permission>hdfs:supergroup:rwxr-xr-x</permission>

# python: all <permission> values must be USER:GROUP:MODE
import re, xml.etree.ElementTree as ET
PERM_RE = re.compile(r'^[^:]+:[^:]+:[rwxstX-]{9,10}$')

def permissions_ok(path):
    bad = []
    for ev, el in ET.iterparse(path, events=('end',)):
        if el.tag == 'permission' and not PERM_RE.match(el.text or ''):
            bad.append(el.text)
    return not bad

def is_valid_permission(s: str) -> bool:
    """Narrow a <permission> text node to the USER:GROUP:MODE shape oiv accepts."""
    parts = s.split(':')
    return len(parts) == 3 and all(parts) and set(parts[2]) <= set('rwxstX-')

# catch the parse failure naming the offending string, fix that
# <permission> element, remove partial output, re-run

When it happens

Trigger: A <permission> value like 'rwxr-xr-x' (mode only), 'hdfs' (user only), 'hdfs:supergroup' (missing mode), or one with extra colons; any hand-edited or script-injected permission field that breaks the user:group:mode shape.

Common situations: Permissions rewritten during bulk edits; empty <permission/> elements from faulty transforms; usernames or groupnames that themselves contain ':' (unsupported by this format).

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Only read ${actualNumSnapshots} <snapshot> entries out of ${
• <snapshot> section was missing <id>
• Got unexpected end tag for ${name}
• SnapshotDiffSection contained unexpected tag ${tagName}
• <dirDiffEntry> contained no <inodeId> entry.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d88c46f66ac3dfc4. Report an issue: GitHub.