apache/hadoop · error · IOException

Error executing command. %s Process exited with exit code %d

Error message

Error executing command. %s Process exited with exit code %d.

What it means

The private runCommand helper used by unTar(InputStream, ...) on non-Windows pipes the input stream into an external process, and after draining stdout/stderr futures and process.waitFor(), a non-zero exit code yields IOException("Error executing command. <command> Process exited with exit code <n>."). The command array is the tar invocation ('bash','-c','tar -xf - ...'). So this error means the spawned tar (or bash) failed on your archive.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java:963

            inputStream, process.getOutputStream());
      } finally {
        process.getOutputStream().close();
      }

      // Wait for both stdout and stderr futures to finish
      error.get();
      output.get();
    } finally {
      // Clean up the threads
      if (executor != null) {
        executor.shutdown();
      }
      // Wait to avoid leaking the child process
      exitCode = process.waitFor();
    }

    if (exitCode != 0) {
      throw new IOException(
          String.format(
              "Error executing command. %s " +
                  "Process exited with exit code %d.",
              command, exitCode));
    }
  }

  /**
   * Given a Tar File as input it will untar the file in a the untar directory
   * passed as the second parameter
   *
   * This utility will untar ".tar" files and ".tar.gz","tgz" files.
   *
   * @param inputStream The tar file as input.
   * @param untarDir The untar directory where to untar the tar file.
   * @param gzipped The input stream is gzipped
   *                TODO Use magic number and PusbackInputStream to identify
   * @throws IOException an exception occurred

View on GitHub (pinned to 2add963021)

Solutions

1. Verify the archive integrity first: gzip -t / tar -tf locally or via checksum against the source
2. Confirm the node has bash and tar on PATH (containers based on distroless/scratch often do not)
3. Pass the correct gzipped flag — the API relies on it rather than magic-number detection (see TODO in the source)
4. Catch the IOException, inspect the exit code in the message, and fall back to unTarUsingJava if the platform tar is the problem

Example fix

// before
FileUtil.unTar(Files.newInputStream(tgz.toPath()), outDir, true);
// IOException: Error executing command. ... exit code 2

// after: validate gzip integrity before untarring
try (GzipCompressorInputStream g =
         new GzipCompressorInputStream(Files.newInputStream(tgz.toPath()))) {
  byte[] buf = new byte[8192];
  while (g.read(buf) != -1) { /* drain to verify */ }
}
FileUtil.unTar(Files.newInputStream(tgz.toPath()), outDir, true);

// verify the tar binary exists and the stream is the expected format
if (new File("/bin/tar").canExecute() || Shell.checkIsBashSupported()) { /* ok */ }
byte[] magic = new byte[2];
try (PushbackInputStream pb = new PushbackInputStream(in, 2)) {
  pb.read(magic);
  pb.unread(magic);
  boolean gz = (magic[0] & 0xff) == 0x1f && (magic[1] & 0xff) == 0x8b;
  FileUtil.unTar(pb, untarDir, gz); // stop guessing the gzipped flag
}

try {
  FileUtil.unTar(in, untarDir, gzipped);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("exit code")) {
    // parse the exit code: 127 = tar missing, 1/2 = corrupt archive
    // verify archive, install tar, or fall back to a Java-based untar
  } else throw e;
}

When it happens

Trigger: unTar(stream, untarDir, gzipped) with a truncated/corrupt tar or tar.gz (tar exits 1/2); 'tar' binary missing from PATH (shell exit 127); gzipped=false on a .gz stream; target dir unwritable (tar cannot write).

Common situations: Partially downloaded tarballs; archives corrupted in transit; minimal container images without tar installed; wrong gzipped flag derived from filename heuristics.

Related errors

• Error untarring file " + inFile + ". Tar process exited with
• Mkdirs failed to create " + untarDir
• expanding " + entry.getName() + " would create entry outside
• Mkdirs failed to create tar internal dir " + outputDir
• PipeMapRed.waitOutputThreads(): subprocess failed with code

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8f5b327ac4970e35. Report an issue: GitHub.