{"record":{"id":"7dd263ac2ad2d67c","repo":"apache/flink","slug":"failed-to-get-metadata-for-key","errorCode":null,"errorMessage":"Failed to get metadata for key: {}","messagePattern":"Failed to get metadata for key: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/writer/NativeS3ObjectOperations.java","lineNumber":449,"sourceCode":"            return;\n        }\n        try {\n            stream.abort();\n        } catch (RuntimeException e) {\n            LOG.debug(\"Error aborting S3 response stream\", e);\n        }\n        IOUtils.closeQuietly(stream);\n    }\n\n    public ObjectMetadata getObjectMetadata(String key) throws IOException {\n        try {\n            HeadObjectRequest request =\n                    HeadObjectRequest.builder().bucket(bucketName).key(key).build();\n            HeadObjectResponse response = s3Client.headObject(request);\n            return new ObjectMetadata(\n                    response.contentLength(), response.eTag(), response.lastModified());\n        } catch (S3Exception e) {\n            throw new IOException(\"Failed to get metadata for key: \" + key, e);\n        }\n    }\n\n    public String getBucketName() {\n        return bucketName;\n    }\n\n    /**\n     * Extracts the S3 object key from a Flink Path.\n     *\n     * <p>Expected URI format: {@code s3://bucket-name/path/to/object}\n     *\n     * <p><b>Limitations:</b> This method only supports the standard S3 URI format. Other URI\n     * formats are NOT supported:\n     *\n     * <ul>\n     *   <li>{@code https://bucket.s3.amazonaws.com/path/to/object} (virtual-hosted style)\n     *   <li>{@code https://s3.amazonaws.com/bucket/path/to/object} (path style)","sourceCodeStart":431,"sourceCodeEnd":467,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/writer/NativeS3ObjectOperations.java#L431-L467","documentation":"Thrown by NativeS3ObjectOperations.getObjectMetadata when the underlying s3Client.headObject call raises an S3Exception. A HEAD request is used to cheaply fetch object size, ETag, and last-modied time; any rejection (missing key, no permission, connectivity) is wrapped into an IOException naming the key.","triggerScenarios":"Calling getObjectMetadata(key) — used to check existence/size of parts or target objects during commit/recovery — when the key does not exist (404), the caller lacks s3:GetObject permission, the bucket/endpoint is misconfigured, or the network fails.","commonSituations":"Committing a multipart upload whose already-uploaded part objects were removed; IAM policy missing head-object permission; s3.endpoint configured for the wrong region; clock-skew induced 403s; concurrent cleanup deleting the object between listing and HEAD.","solutions":["Inspect the nested S3Exception status code: 404 means the key is gone, 403 means permissions, otherwise suspect network/endpoint.","Verify with: aws s3api head-object --bucket <bucket> --key <key> using the same credentials Flink uses.","Fix IAM/bucket policy to allow s3:GetObject on the bucket for HEAD requests.","Check s3.endpoint and s3.path.style.access settings match the bucket's actual region/layout.","Retry on 5xx/timeout — HEAD is idempotent."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"try {\n    s3Client.headObject(HeadObjectRequest.builder().bucket(bucket).key(key).build());\n} catch (NoSuchKeyException e) {\n    // key absent — decide before calling getObjectMetadata\n}","typeGuard":null,"tryCatchPattern":"try {\n    ObjectMetadata md = s3AccessHelper.getObjectMetadata(key);\n} catch (IOException e) {\n    if (e.getCause() instanceof S3Exception\n            && ((S3Exception) e.getCause()).statusCode() == 404) {\n        // treat as absent\n        return Optional.empty();\n    }\n    throw e;\n}","preventionTips":["Grant s3:GetObject on the checkpoint bucket in IAM.","Use HEAD-based existence checks instead of GET where possible.","Align s3.endpoint with the bucket region."],"tags":["s3","filesystem","metadata","permissions","io"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}