apache/hadoop · error · IOException

%s : %s

Error message

%s : %s

What it means

handleException is the central error wrapper of CosNativeFileSystemStore: it takes the failing COS key, builds the fully qualified cosn://bucket+key URI, and rethrows the original exception text as a new IOException formatted "%s : %s" (uri : cause). Seeing this message means an underlying COS operation (metadata fetch, read, delete, upload part) failed and its cause was flattened into the message text. The real root cause (NoSuchKey, AccessDenied, throttling, timeout) must be read from the portion after the ' : ' separator.

Source

Thrown at hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/CosNativeFileSystemStore.java:659

      handleException(new Exception(errMsg), srcKey);
    }
  }

  @Override
  public void purge(String prefix) throws IOException {
    throw new IOException("purge not supported");
  }

  @Override
  public void dump() throws IOException {
    throw new IOException("dump not supported");
  }

  // process Exception and print detail
  private void handleException(Exception e, String key) throws IOException {
    String cosPath = CosNFileSystem.SCHEME + "://" + bucketName + key;
    String exceptInfo = String.format("%s : %s", cosPath, e.toString());
    throw new IOException(exceptInfo);
  }

  @Override
  public long getFileLength(String key) throws IOException {
    LOG.debug("Get file length. COS key: {}", key);
    GetObjectMetadataRequest getObjectMetadataRequest =
        new GetObjectMetadataRequest(bucketName, key);
    try {
      ObjectMetadata objectMetadata =
          (ObjectMetadata) callCOSClientWithRetry(getObjectMetadataRequest);
      return objectMetadata.getContentLength();
    } catch (Exception e) {
      String errMsg = String.format("Getting file length occurs an exception." +
              "COS key: %s, exception: %s", key,
          e.toString());
      LOG.error(errMsg);
      handleException(new Exception(errMsg), key);
      return 0; // never will get here

View on GitHub (pinned to 2add963021)

Solutions

1. Parse the suffix after the first ' : ' in the message to identify the real cause, then fix that (missing object, permissions, network)
2. Verify the object exists with fs.exists()/getFileStatus before reading the failing path
3. Check the fs.cosn.userinfo.appid / bucket credentials have GetObject/HeadObject rights on that key
4. For transient causes seen in the suffix, retry the operation — the store only retries SDK-level 5xx, not every path

Example fix

// before
long len = fs.getFileStatus(new Path("cosn://bucket/missing-key")).getLen();
// -> IOException "cosn://bucket/missing-key : com.qcloud.cos.exception.CosServiceException: ... NoSuchKey"

// after
Path p = new Path("cosn://bucket/missing-key");
if (!fs.exists(p)) {
  throw new FileNotFoundException(p.toString());
}
long len = fs.getFileStatus(p).getLen();

// Avoid the common NoSuchKey path before reading
Path p = new Path("cosn://bucket/key");
if (!fs.exists(p)) {
  throw new FileNotFoundException(p.toString());
}

try {
  long len = store.getFileLength(key);
} catch (IOException e) {
  // message is "cosn://bucket/key : <cause>" — split on first " : " to recover the cause text
  String cause = e.getMessage().substring(e.getMessage().indexOf(" : ") + 3);
  if (cause.contains("NoSuchKey")) { /* treat as missing */ }
  else { throw e; }
}

When it happens

Trigger: Any store operation routed through handleException fails on the backend, e.g. getFileLength(key) when the object does not exist or metadata HEAD fails, copy/read failures inside the block-based input/output streams, or object deletes that error out. The wrapper fires only on the catch paths that call it, converting Exception to IOException with path context.

Common situations: Reading a stale/cosn path after the object was deleted or renamed; wrong bucket or key prefix in fs.cosn configuration; credentials without permission on the bucket; transient Tencent COS network errors surfacing mid-read.

Related errors

• Call cos sdk failed, call method: %s, exception: %s
• dump not supported
• no such method
• No COS Credential Providers
• Credentials requested after provider list was closed

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b1b2f1ede20e7011. Report an issue: GitHub.