{"record":{"id":"6e719127a0b8cac8","repo":"apache/hadoop","slug":"failed-to-compute-unencrypted-length","errorCode":null,"errorMessage":"Failed to compute unencrypted length","messagePattern":"Failed to compute unencrypted length","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/CSEUtils.java","lineNumber":147,"sourceCode":"          && !plaintextLength.isEmpty()) {\n        return Long.parseLong(plaintextLength);\n      }\n    }\n\n    // identify the length by doing a ranged GET operation.\n    if (contentLength >= CSE_PADDING_LENGTH) {\n      long minPlaintextLength = contentLength - CSE_PADDING_LENGTH;\n      if (minPlaintextLength < 0) {\n        minPlaintextLength = 0;\n      }\n      try (InputStream is = store.getRangedS3Object(key, minPlaintextLength, contentLength)) {\n        int i = 0;\n        while (is.read() != -1) {\n          i++;\n        }\n        return minPlaintextLength + i;\n      } catch (Exception e) {\n        throw new IOException(\"Failed to compute unencrypted length\", e);\n      }\n    }\n    return contentLength;\n  }\n\n  /**\n   * Creates encryption materials for client-side encryption based on the specified algorithm.\n   *\n   * Supports two types of client-side encryption:\n   * <ul>\n   *   <li>CSE_KMS: Uses AWS KMS for key management</li>\n   *   <li>CSE_CUSTOM: Uses a custom cryptographic implementation</li>\n   * </ul>\n   *\n   * @param conf      The configuration containing encryption settings\n   * @param bucket    The S3 bucket name for which encryption materials are being created\n   * @param algorithm The encryption algorithm to use (CSE_KMS or CSE_CUSTOM)\n   * @return         CSEMaterials configured with the appropriate encryption settings","sourceCodeStart":129,"sourceCodeEnd":165,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/CSEUtils.java#L129-L165","documentation":"IOException from CSEUtils when computing the plaintext length of a client-side-encrypted (CSE) object. For CSE, S3A stores a padding block at the tail, so for contentLength >= CSE_PADDING_LENGTH the code issues a ranged GET over [contentLength - padding, contentLength] and counts the returned bytes to recover the true unencrypted size. Any failure of that ranged GET (permission, throttling, network, deserialization of the encrypted tail) is caught and rethrown as this wrapping IOException with the original as cause.","triggerScenarios":"Calling getFileStatus (or any API that needs the length) on an object written with fs.s3a.encryption.algorithm=CSE-KMS or CSE-CUSTOM when the tail ranged read fails: no s3:GetObject permission on the key material, KMS key disabled or inaccessible, object deleted mid-operation, or a transient S3 error.","commonSituations":"Reader lacks grants to the KMS key used to encrypt the data; cross-account access without key policy updates; throttled S3 responses during metadata-heavy workloads; object lifecycle deleted the object between listing and stat.","solutions":["Inspect the wrapped cause (e.getCause()) -- AccessDenied points to KMS/key-policy issues, NoSuchKey to a deleted object, 503/timeout to transient S3 conditions","Verify the reading principal can decrypt with the key that encrypted the object (kms:Decrypt grant and key policy for cross-account)","Retry the operation for transient causes; the length computation is idempotent","If the object may be changing concurrently, quiesce writers before statting CSE objects"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  FileStatus st = fs.getFileStatus(csePath);\n} catch (IOException e) {\n  if (\"Failed to compute unencrypted length\".equals(e.getMessage())\n      && e.getCause() instanceof AccessDeniedException /* or SDK AccessDenied */) {\n    // key-material permission problem: not retryable, fix KMS/key policy\n    throw new SecurityException(\"Cannot decrypt CSE object tail\", e);\n  }\n  // otherwise transient: retry with backoff\n}","preventionTips":["Grant readers kms:Decrypt and verify key policies before deploying CSE-encrypted datasets","Log the cause chain fully -- the wrapping message alone hides the real failure","Avoid statting CSE objects while writers are mid-replace of the same key"],"tags":["s3a","client-side-encryption","cse","kms","metadata"],"backgroundTag":"s3-decryption-failure","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}