apache/hadoop · error · IOException

FileMetadata not match key:" + key

Error message

FileMetadata not match key:" + key

What it means

BosNativeFileSystemStore.retrieve(key, byteRangeStart, byteRangeEnd, meta) enforces an internal invariant before issuing the GET: meta must be non-null and meta.getKey() must equal the requested key, else IOException 'FileMetadata not match key:<key>'. It protects the byte-range read against metadata cached for a different object.

Source

Thrown at hadoop-cloud-storage-project/hadoop-bos/src/main/java/org/apache/hadoop/fs/bos/BosNativeFileSystemStore.java:400

   * @param key the object key
   * @param byteRangeStart the start of the byte range
   * @param byteRangeEnd the end of the byte range
   * @param meta the file metadata for verification
   * @return an input stream for reading the range, or null
   *         if the key is not found
   * @throws IOException if an I/O error occurs
   */
  public InputStream retrieve(
      String key, long byteRangeStart,
      long byteRangeEnd, FileMetadata meta)
      throws IOException {
    log.debug(
        "Getting key: {} from bucket: {} with"
            + " byteRangeStart: {} and"
            + " byteRangeEnd: {}.",
        key, bucketName, byteRangeStart, byteRangeEnd);
    if (meta == null || !meta.getKey().equals(key)) {
      throw new IOException(
          "FileMetadata not match key:" + key);
    }
    GetObjectRequest request =
        new GetObjectRequest(bucketName, key);
    // Due to the InvalidRange exception of bos, we
    // shouldn't set range for empty file
    if (meta.getLength() != 0) {
      request.setRange(byteRangeStart, byteRangeEnd);
    }
    BosObject object =
        bosClientProxy.getObject(request);
    return object.getObjectContent();
  }

  /**
   * Converts a prefix to a directory-style key by appending
   * a trailing delimiter if not already present.
   *

View on GitHub (pinned to 2add963021)

Solutions

  1. Always obtain FileMetadata via getFileMetadata(key) for the exact same key string used in retrieve
  2. Never reuse one FileMetadata instance across different keys
  3. If you control the boundary, log meta.getKey() versus key where they are produced to find where they diverge

Example fix

// before
store.retrieve(key, start, end, recycledMeta); // from another file

// after
FileMetadata meta = store.retrieveMetadata(key);
store.retrieve(key, start, end, meta);
Defensive patterns

Strategy: validation

Validate before calling

if (meta == null || !meta.getKey().equals(key)) {
  meta = store.retrieveMetadata(key); // re-fetch metadata for THIS key
}
store.retrieve(key, start, end, meta);

Type guard

static boolean metadataMatches(FileMetadata meta, String key) {
  return meta != null && meta.getKey() != null && meta.getKey().equals(key);
}

Try / catch

catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("FileMetadata not match key")) {
    meta = store.retrieveMetadata(key); // invariant broken: refresh and retry once
  } else { throw e; }
}

Prevention

When it happens

Trigger: The stream was opened for one key but retrieve receives FileMetadata belonging to another: subclasses/wrappers recycling a metadata instance, hand-built FileMetadata in tests, or a key-normalization difference (trailing slash, encoding) between the metadata key and the fetch key.

Common situations: Custom extensions of the store or the input stream passing stale metadata; connector-internal races when a file is replaced between open and read; unit tests constructing FileMetadata directly with a mismatched key.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3e4c46ce890f919f. Report an issue: GitHub.