apache/hadoop · error · IOException

Could not find jobid

Error message

Could not find jobid

What it means

Error "Could not find jobid" thrown in apache/hadoop.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleChannelHandler.java:452

  @SuppressWarnings("checkstyle:VisibilityModifier")
  static class MapOutputInfo {
    final Path mapOutputFileName;
    final IndexRecord indexRecord;

    MapOutputInfo(Path mapOutputFileName, IndexRecord indexRecord) {
      this.mapOutputFileName = mapOutputFileName;
      this.indexRecord = indexRecord;
    }
  }

  protected void verifyRequest(String appid, ChannelHandlerContext ctx,
                               HttpRequest request, HttpResponse response, URL requestUri)
      throws IOException {
    SecretKey tokenSecret = handlerCtx.secretManager.retrieveTokenSecret(appid);
    if (null == tokenSecret) {
      LOG.info("Request for unknown token {}, channel id: {}", appid, ctx.channel().id());
      throw new IOException("Could not find jobid");
    }
    // encrypting URL
    String encryptedURL = SecureShuffleUtils.buildMsgFrom(requestUri);
    // hash from the fetcher
    String urlHashStr =
        request.headers().get(SecureShuffleUtils.HTTP_HEADER_URL_HASH);
    if (urlHashStr == null) {
      LOG.info("Missing header hash for {}, channel id: {}", appid, ctx.channel().id());
      throw new IOException("fetcher cannot be authenticated");
    }
    if (LOG.isDebugEnabled()) {
      int len = urlHashStr.length();
      LOG.debug("Verifying request. encryptedURL:{}, hash:{}, channel id: " +
              "{}", encryptedURL,
          urlHashStr.substring(len - len / 2, len - 1), ctx.channel().id());
    }
    // verify - throws exception
    SecureShuffleUtils.verifyReply(urlHashStr, encryptedURL, tokenSecret);

View on GitHub (pinned to 2add963021)

Solutions

1. Include a valid jobId in the shuffle request URL; check the fetcher constructs the request path correctly.
2. Verify the shuffle handler is serving the correct application and the map output index files exist.

When it happens

Trigger: Thrown by ShuffleChannelHandler when a shuffle HTTP request does not carry a valid jobId parameter. The reducer's shuffle URL is malformed; verify the shuffle request URI includes mapId, jobId and reduceId parameters.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ae90495d8bf01a85. Report an issue: GitHub.