apache/hadoop · error · IOException

Input file does not contain valid topology data.

Error message

Input file does not contain valid topology data.

What it means

IOException('Input file does not contain valid topology data.') from ClusterTopologyReader.readTopology: the JsonParser's getNext() returned null on the first record, meaning the file had no parseable LoggedNetworkTopology object at its head — either the file is empty or its first JSON value is a different/malformed shape and the mapper could not produce one.

Source

Thrown at hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ClusterTopologyReader.java:38

import java.io.IOException;
import java.io.InputStream;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;

/**
 * Reading JSON-encoded cluster topology and produce the parsed
 * {@link LoggedNetworkTopology} object.
 */
public class ClusterTopologyReader {
  private LoggedNetworkTopology topology;

  private void readTopology(JsonObjectMapperParser<LoggedNetworkTopology> parser)
      throws IOException {
    try {
      topology = parser.getNext();
      if (topology == null) {
        throw new IOException(
            "Input file does not contain valid topology data.");
      }
    } finally {
      parser.close();
    }
  }

  /**
   * Constructor.
   * 
   * @param path
   *          Path to the JSON-encoded topology file, possibly compressed.
   * @param conf
   * @throws IOException
   */
  public ClusterTopologyReader(Path path, Configuration conf)
      throws IOException {
    JsonObjectMapperParser<LoggedNetworkTopology> parser = new JsonObjectMapperParser<LoggedNetworkTopology>(

View on GitHub (pinned to 2add963021)

Solutions

1. Check the file size and head the file — it must start with a JSON object carrying host/rack fields ('{"name": ..., "children": ...}') not job records
2. If the file is empty or truncated, regenerate it from the previous pipeline stage (e.g. re-run HadoopLogsAnalyzer -write-topology or the Anonymizer)
3. Confirm you passed the topology file where a topology is expected, not the job trace
4. Verify integrity if compressed: gunzip -t file.json.gz

Example fix

# before
hadoop ... Folder -topology jobtrace.json ...   # wrong input kind

# after - feed the machine-generated topology, not the trace
hadoop ... Folder - topology: hadoop ... Folder -topology cluster-topology.json ...  # see next line for exact form
# correct: -topology <topology.json> where topology.json starts with {"name":"...","children":[...]}

// verify head record shape before constructing the reader
try (BufferedReader r = new BufferedReader(
    new InputStreamReader(fs.open(path), StandardCharsets.UTF_8))) {
  String first = r.readLine();
  if (first == null || !first.trim().startsWith("{")) {
    throw new IllegalArgumentException("Not a topology JSON file: " + path);
  }
}

try {
  new ClusterTopologyReader(conf, topologyInputPath, statePool);
} catch (IOException e) {
  if (e.getMessage().contains("valid topology")) {
    LOG.error("Topology file empty/wrong kind: " + path);
  }
}

When it happens

Trigger: Constructing ClusterTopologyReader (e.g. from Folder with -topology, or directly) pointing at an empty file, a truncated/garbled file, a gz file whose compression ended early, or a trace file (job records) supplied where a topology file was expected.

Common situations: Swapping the -topology and trace arguments on the command line; upstream anonymizer/folder runs crashed before writing the file, leaving a 0-byte artifact; CR/LF or BOM mangling; text mode FTP corrupting .gz topology files.

Related errors

• Can not open a folder
• Failed to rename %s to %s, %s is a file
• Failed to rename %s to %s, file already exists or not empty!
• Truncate is not supported by BaiduBosFileSystem
• user accessKey and secretAccessKey should not be null

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8d25821c1c7faf03. Report an issue: GitHub.