apache/hadoop · warning · IOException

NameNode initialization not yet complete. FSImage has not be

Error message

NameNode initialization not yet complete. FSImage has not been set in the NameNode.

What it means

ImageServlet.getAndValidateFSImage returns HTTP 403 and throws IOException("NameNode initialization not yet complete. FSImage has not been set in the NameNode.") when an image-transfer HTTP request arrives after the NameNode's HTTP port is open but before the FSImage object has been published to the servlet context. It is a startup-window race: the endpoint exists but is not yet serviceable.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ImageServlet.java:124

   * due to image upload delay, or minor machine clock skew) can cause ANN to
   * reject a fsImage too aggressively.
   */
  private static double recentImageCheckTimePrecision = 0.75;

  @VisibleForTesting
  static void setRecentImageCheckTimePrecision(double ratio) {
    recentImageCheckTimePrecision = ratio;
  }

  private FSImage getAndValidateFSImage(ServletContext context,
      final HttpServletResponse response)
      throws IOException {
    final FSImage nnImage = NameNodeHttpServer.getFsImageFromContext(context);
    if (nnImage == null) {
      String errorMsg = "NameNode initialization not yet complete. "
          + "FSImage has not been set in the NameNode.";
      sendError(response, HttpServletResponse.SC_FORBIDDEN, errorMsg);
      throw new IOException(errorMsg);
    }
    return nnImage;
  }

  @Override
  public void doGet(final HttpServletRequest request,
      final HttpServletResponse response) throws ServletException, IOException {
    try {
      final ServletContext context = getServletContext();
      final FSImage nnImage = getAndValidateFSImage(context, response);
      final GetImageParams parsedParams = new GetImageParams(request, response);
      final Configuration conf = (Configuration) context
          .getAttribute(JspHelper.CURRENT_CONF);
      final NameNodeMetrics metrics = NameNode.getNameNodeMetrics();

      validateRequest(context, conf, request, response, nnImage,
          parsedParams.getStorageInfoString());

View on GitHub (pinned to 2add963021)

Solutions

1. Retry the request after the NameNode finishes startup — poll /jmx (State, Safemode) or HA state until it is ready
2. Gate automated checkpoints/bootstrap on NN readiness rather than fixed timers
3. If the error persists well after startup, check NN logs: the image may be failing to load entirely

Example fix

# before
curl -f http://nn:9870/getimage?getimage=1   # during NN startup -> 403

# after: gate on readiness, then transfer
until curl -sf http://nn:9870/jmx | grep -q '"State" : "active"'; do sleep 5; done
curl -f http://nn:9870/getimage?getimage=1

// Gate image-transfer scripts on NN readiness before the first request
# bash
until curl -sf http://nn:9870/jmx \
  | grep -q '"State" : "\(active\|standby\)"'; do
  sleep 5
done
curl -f 'http://nn:9870/getimage?getimage=1&latest=1'

int attempts = 12;
for (int i = 0; i < attempts; i++) {
  try {
    return fetchImage(url);
  } catch (IOException e) {
    if (!e.getMessage().contains("initialization not yet complete")
        || i == attempts - 1) { throw e; }
    Thread.sleep(10_000L);   // NN still loading: back off and retry
  }
}

When it happens

Trigger: SecondaryNameNode or a bootstrapping Standby immediately requests /getimage right after the NN process restarts; monitoring/backup scripts polling the image-transfer endpoint during a long fsimage load; NN restart storms where checkpointers retry aggressively.

Common situations: Restart automation without readiness gating; very large fsimages extending the load window while SNN timers fire; health checks that hit transfer servlets instead of status endpoints.

Related errors

• Cannot import image from a checkpoint. NameNode already con
• Could not find image with txid " + txid
• Unknown nameservice: {}
• Configuration has multiple addresses that match local node's
• Configuration dfs.namenode.rpc-address must be suffixed with

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/cea57c1ad4459ca3. Report an issue: GitHub.