apache/hadoop · error · PathIOException

Not a valid manifest file; file status = {status}

Error message

Not a valid manifest file; file status = {status}

What it means

While loading task manifests at job commit, fetchTaskManifest() sanity-checks each manifest file's FileStatus and throws PathIOException('Not a valid manifest file; file status = <status>') when the file is zero-length or is not a regular file. Since manifests are written via temp-file-plus-rename, a 0-byte or non-file manifest means the write was interrupted, the FS corrupted it, or something foreign sits in the manifests directory.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/stages/LoadManifestsStage.java:259

          directories.putIfAbsent(entry.getDir(), entry);
        });
      }
    }
    return toCreate.size();
  }

  /**
   * Precommit preparation of a single manifest file.
   * To reduce the memory foot print, the IOStatistics and
   * extra data of each manifest is cleared.
   * @param status status of file.
   * @return number of files.
   * @throws IOException IO Failure.
   */
  private TaskManifest fetchTaskManifest(FileStatus status)
      throws IOException {
    if (status.getLen() == 0 || !status.isFile()) {
      throw new PathIOException(status.getPath().toString(),
          "Not a valid manifest file; file status = " + status);
    }
    // load the manifest, which includes validation.
    final TaskManifest manifest = loadManifest(status);
    final String id = manifest.getTaskAttemptID();
    final int filecount = manifest.getFilesToCommit().size();
    final long size = manifest.getTotalFileSize();
    LOG.info("{}: Task Attempt {} file {}: File count: {}; data size={}",
        getName(), id, status.getPath(), filecount, size);

    // record file size for tracking of memory consumption, work etc.
    final IOStatisticsStore iostats = getIOStatistics();
    iostats.addSample(COMMITTER_TASK_MANIFEST_FILE_SIZE, status.getLen());
    iostats.addSample(COMMITTER_TASK_FILE_COUNT_MEAN, filecount);
    iostats.addSample(COMMITTER_TASK_DIRECTORY_COUNT_MEAN,
        manifest.getDestDirectories().size());
    return manifest;
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Inspect the path and status in the message: a 0-length file points to an interrupted write; a directory points to external interference.
2. Remove the offending attempt's stale manifest directory and any partial state, then rerun the job.
3. Verify nothing writes into the committer's directory tree (manifests live under the output path's committer-managed area).
4. If it recurs, check filesystem health and task-kill logs around the task attempt id shown.

// optional pre-commit audit: every listed manifest must be a non-empty regular file
for (FileStatus s : fs.listStatus(manifestsDir)) {
  if (s.getLen() == 0 || !s.isFile()) {
    throw new IOException("Invalid manifest entry " + s.getPath() + " - investigate before commit");
  }
}

try {
  job.waitForCompletion(true);
} catch (PathIOException e) {
  if (e.getMessage() != null && e.getMessage().contains("Not a valid manifest file")) {
    // corrupt/stale manifest: remove that attempt's state and rerun; retrying cannot fix bad bytes
  }
}

When it happens

Trigger: A path under the job's task-manifests directory whose status has getLen()==0 or isFile()==false: truncated manifest from a killed/crashed task attempt that still got listed, a directory placed where a manifest file belongs, or FS-level corruption.

Common situations: Task killed exactly during manifest publication; external processes or users writing into the committer's manifest directory; disk/FS corruption on the store; mixing committer versions that changed manifest write behavior.

Related errors

• Failed to parse "{path}" : {e}
• Failed to write manifest data for task {taskId}to local file
• Expected a file renamed from {sourcePath}; found {destStatus
• Path must be absolute: " + path
• Rename dir to self is forbidden

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/19b6e1629e802edf. Report an issue: GitHub.