{"record":{"id":"71f100678b06e162","repo":"apache/hadoop","slug":"encoded-sha256-hash-must-be-provided-for-global-en","errorCode":null,"errorMessage":"Encoded SHA256 hash must be provided for global encryption","messagePattern":"Encoded SHA256 hash must be provided for global encryption","errorType":"exception","errorClass":"PathIOException","httpStatus":null,"severity":"critical","filePath":"hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java","lineNumber":1891,"sourceCode":"          abfsConfiguration.getRawConfiguration());\n    }\n\n    // Encryption setup\n    EncryptionContextProvider encryptionContextProvider = null;\n    if (isSecure) {\n      encryptionContextProvider =\n          abfsConfiguration.createEncryptionContextProvider();\n      if (encryptionContextProvider != null) {\n        if (abfsConfiguration.getEncodedClientProvidedEncryptionKey() != null) {\n          throw new PathIOException(uri.getPath(),\n              \"Both global key and encryption context are set, only one allowed\");\n        }\n        encryptionContextProvider.initialize(\n            abfsConfiguration.getRawConfiguration(), accountName,\n            fileSystemName);\n      } else if (abfsConfiguration.getEncodedClientProvidedEncryptionKey() != null) {\n        if (abfsConfiguration.getEncodedClientProvidedEncryptionKeySHA() == null) {\n          throw new PathIOException(uri.getPath(),\n              \"Encoded SHA256 hash must be provided for global encryption\");\n        }\n      }\n    }\n\n    LOG.trace(\"Initializing AbfsClientHandler for {}\", baseUrl);\n    this.clientHandler = new AbfsClientHandler(baseUrl, creds,\n        abfsConfiguration,\n        tokenProvider, sasTokenProvider, encryptionContextProvider,\n        populateAbfsClientContext());\n\n    this.setClient(getClientHandler().getClient());\n    LOG.trace(\"AbfsClient init complete\");\n  }\n\n  private AbfsServiceType getAbfsServiceTypeFromUrl() {\n    if (uri.toString().contains(ABFS_BLOB_DOMAIN_NAME)) {\n      return AbfsServiceType.BLOB;","sourceCodeStart":1873,"sourceCodeEnd":1909,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java#L1873-L1909","documentation":"Client-provided encryption keys (CPPK) require the service to verify key integrity via x-ms-encryption-key-sha256, so Hadoop demands both halves of the pair. At initialization, if fs.azure.encryption.encoded.client-provided-key is set on a secure connection but fs.azure.encryption.encoded.client-provided-key-sha is missing, PathIOException 'Encoded SHA256 hash must be provided for global encryption' aborts client construction.","triggerScenarios":"Secure abfss:// mount (no encryption-context provider) with fs.azure.encryption.encoded.client-provided-key configured but the matching fs.azure.encryption.encoded.client-provided-key-sha property absent or misspelled (e.g. missing the account-specific overlay).","commonSituations":"Key rotated but hash line forgotten; typo in the -sha property name; copy-paste from docs that show only the key; config management dropping the second property.","solutions":["Compute SHA-256 of the raw key bytes and add fs.azure.encryption.encoded.client-provided-key-sha=<base64 hash>.","Both values are Base64: key = base64(raw AES key), sha = base64(SHA256(raw key bytes)) - not SHA256 of the base64 string.","Check for account-scoped variants (fs.azure.account.<acct>.) overriding or shadowing the cluster-wide properties.","Remember CPPK applies only on secure (abfss/https) connections; on insecure endpoints the pair is ignored rather than validated."],"exampleFix":"# compute the required sha (linux)\nprintf '%s' '<raw-key-bytes>' | openssl dgst -sha256 -binary | base64\n\n<property>\n  <name>fs.azure.encryption.encoded.client-provided-key-sha</name>\n  <value>BASE64_OF_SHA256_OF_RAW_KEY</value>\n</property>","handlingStrategy":"validation","validationCode":"String key = conf.get(\"fs.azure.encryption.encoded.client-provided-key\");\nString sha = conf.get(\"fs.azure.encryption.encoded.client-provided-key-sha\");\nif (key != null && sha == null) {\n  throw new IllegalArgumentException(\"fs.azure.encryption.encoded.client-provided-key-sha is required with the key\");\n}\n// sanity: sha must decode to 32 bytes\nif (sha != null && java.util.Base64.getDecoder().decode(sha).length != 32) {\n  throw new IllegalArgumentException(\"-sha must be base64 of a 32-byte SHA-256\");\n}","typeGuard":null,"tryCatchPattern":"try { fs = FileSystem.get(conf); } catch (PathIOException e) { if (e.getMessage().contains(\"SHA256\")) { /* add the -sha property: base64(SHA256(raw key bytes)) */ } throw e; }","preventionTips":["Generate key and sha together in one script; store them as a pair.","sha = base64(SHA256(raw key bytes)), not a hash of the base64 string.","Decode-check the sha (must be 32 bytes) in config validation."],"tags":["azure","abfs","encryption","client-side-encryption","sha256","configuration"],"backgroundTag":"missing-sha256-hash","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}