{"record":{"id":"26f0f380d633aeff","repo":"apache/hadoop","slug":"could-not-parse-line-line","errorCode":null,"errorMessage":"Could not parse line: ${line}","messagePattern":"Could not parse line: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/DF.java","lineNumber":199,"sourceCode":"    }\n    if (!tokens.hasMoreTokens()) {            // for long filesystem name\n      if (output.size() > 2) {\n        line = output.get(2);\n      } else {\n        throw new IOException(\"Expecting additional output after line: \"\n            + line);\n      }\n      tokens = new StringTokenizer(line, \" \\t\\n\\r\\f%\");\n    }\n\n    try {\n      Long.parseLong(tokens.nextToken()); // capacity\n      Long.parseLong(tokens.nextToken()); // used\n      Long.parseLong(tokens.nextToken()); // available\n      Integer.parseInt(tokens.nextToken()); // pct used\n      this.mount = tokens.nextToken();\n    } catch (NoSuchElementException e) {\n      throw new IOException(\"Could not parse line: \" + line);\n    } catch (NumberFormatException e) {\n      throw new IOException(\"Could not parse line: \" + line);\n    }\n  }\n\n  private void verifyExitCode() throws IOException {\n    if (getExitCode() != 0) {\n      StringBuilder sb =\n          new StringBuilder(\"df could not be run successfully: \");\n      for (String line : output) {\n        sb.append(line);\n      }\n      throw new IOException(sb.toString());\n    }\n  }\n\n  public static void main(String[] args) throws Exception {\n    String path = \".\";","sourceCodeStart":181,"sourceCodeEnd":217,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/DF.java#L181-L217","documentation":"After locating the numeric row, DF parses five fields in order (capacity, used, available, pct, mount) via StringTokenizer; if a token is missing (NoSuchElementException) it rethrows IOException 'Could not parse line'. The df line simply had fewer columns than the expected layout - usually a percentage or mount-point field that this OS/filesystem combination does not emit.","triggerScenarios":"DF.parseOutput() over 'df -k <dir>' output whose data row lacks a trailing field: pseudo-filesystems or FUSE mounts without a mount token, df variants that drop the '%used' column, or the wrapped-device fallback line still missing its tail.","commonSituations":"Data dirs on FUSE (s3fs, goofys, sshfs) or exotic mounts whose df rows are malformed; minimal Busy df variants; the same wrapped-long-device case as error 517 landing here when the third line exists but is short.","solutions":["Run sudo -u <daemon> df -k <datadir> and count columns against 'Filesystem 1K-blocks Used Available Use% Mounted on'; if a column is missing, that mount is unusable for Hadoop data dirs on this version.","Move data dirs to a standard local filesystem (ext4/xfs) and keep FUSE/exotic mounts out of dfs.datanode.data.dir.","Upgrade Hadoop for more tolerant DF parsing, or avoid DF-based checks on such volumes in your own code."],"exampleFix":"# before\n# sshfs mount: df -k /mnt/remote prints a row with no mount-point column\n# DF.parseOutput(): IOException: Could not parse line: sshfs# ...\n\n# after\n# datanode uses a real local volume instead\n# hdfs-site.xml: dfs.datanode.data.dir = /grid1/dn   (ext4/xfs, full df row)","handlingStrategy":"try-catch","validationCode":"// pre-flight: data row must have >= 6 columns matching\n// 'Filesystem 1K-blocks Used Available Use% Mounted on'\nProcess p = new ProcessBuilder(\"df\", \"-k\", dir).start();\n// assert token count of the data row before letting DF parse it","typeGuard":null,"tryCatchPattern":"try {\n  df.getMount();\n} catch (IOException e) { // 'Could not parse line:'\n  // exclude this mount from data dirs; use a standard local filesystem volume\n}","preventionTips":["Do not place dfs.datanode.data.dir / nodemanager dirs on FUSE or exotic mounts.","Keep an allowlist of storage types (ext4/xfs) in cluster validation tooling."],"tags":["hadoop","disk","df","parsing","fuse","missing-field"],"backgroundTag":"df-command-output-parse-failure","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}