apache/hadoop · error · EOFException

No data in ${path}

Error message

No data in ${path}

What it means

The cluster overload load(FileSystem, Path, FileStatus) trusts the caller-supplied FileStatus to skip a metadata call; when it reports getLen() == 0 the method throws EOFException("No data in <path>") without opening the file. Passing a null status is legal and defers to the read path instead.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/JsonSerialization.java:268

  }

  /**
   * Load from a Hadoop filesystem.
   * If a file status is supplied, it's passed in to the openFile()
   * call so that FS implementations can optimize their opening.
   * @param fs filesystem
   * @param path path
   * @param status status of the file to open.
   * @return a loaded object
   * @throws PathIOException JSON parse problem
   * @throws EOFException file status references an empty file
   * @throws IOException IO problems
   */
  public T load(FileSystem fs, Path path, @Nullable FileStatus status)
      throws IOException {

    if (status != null && status.getLen() == 0) {
      throw new EOFException("No data in " + path);
    }
    FutureDataInputStreamBuilder builder = fs.openFile(path)
        .opt(FS_OPTION_OPENFILE_READ_POLICY,
            FS_OPTION_OPENFILE_READ_POLICY_WHOLE_FILE);
    if (status != null) {
      builder.withFileStatus(status);
    }
    try (FSDataInputStream dataInputStream =
             awaitFuture(builder.build())) {
      return fromJsonStream(dataInputStream);
    } catch (JsonProcessingException e) {
      throw new PathIOException(path.toString(),
          "Failed to read JSON file " + e, e);
    }
  }

  /**
   * Save to a Hadoop filesystem.

View on GitHub (pinned to 2add963021)

Solutions

1. Check the object itself (hdfs dfs -ls or an object-store listing): a 0-byte object means the writer failed - regenerate or delete it.
2. Pass null instead of a possibly stale status; the reader then fails with the more precise Jackson error.
3. In recovery loops, treat zero-length state files as 'no state' and skip or quarantine them.

Example fix

// before
MyType t = serializer.load(fs, path, cachedStatus);

// after
if (cachedStatus == null) {
  cachedStatus = fs.getFileStatus(path);
}
if (cachedStatus.getLen() == 0) {
  return defaults(); // empty state file: writer failed, treat as absent
}
return serializer.load(fs, path, cachedStatus);

FileStatus st = (status != null) ? status : fs.getFileStatus(path);
if (st.getLen() == 0) {
  return defaults(); // or quarantine and continue recovery
}
return serializer.load(fs, path, st);

When it happens

Trigger: Calling load(fs, path, status) where status reports zero length: a newly created HDFS/S3 object with no data written yet, an aborted write leaving an empty file, or a cached/stale status of a truncated file.

Common situations: Recovery code scanning state or journal objects where one is empty after an unclean failure; racing a writer between create and first flush; a listing-derived FileStatus reused after the file was rewritten empty.

Related errors

• File is empty: ${jsonFile}
• Failed to read JSON file ${e}
• Mkdirs failed to create directory {dirName}
• Could not rename {oldDir} to {newDir}
• Failed to add {}: You cannot have a rack and a non-rack node

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3d20ed4c3ebde291. Report an issue: GitHub.