{"record":{"id":"efb942b8142ab2cc","repo":"shadowsocks/shadowsocks-windows","slug":"openssl-fail-to-encrypt-aead","errorCode":null,"errorMessage":"openssl: fail to encrypt AEAD","messagePattern":"openssl: fail to encrypt AEAD","errorType":"exception","errorClass":"CryptoErrorException","httpStatus":null,"severity":"critical","filePath":"shadowsocks-csharp/Encryption/AEAD/AEADOpenSSLEncryptor.cs","lineNumber":97,"sourceCode":"                null,\r\n                isEncrypt ? OpenSSL.OPENSSL_ENCRYPT : OpenSSL.OPENSSL_DECRYPT);\r\n            if (ret != 1) throw new System.Exception(\"openssl: cannot set key\");\r\n            OpenSSL.EVP_CIPHER_CTX_set_padding(ctx, 0);\r\n        }\r\n\r\n        public override void cipherEncrypt(byte[] plaintext, uint plen, byte[] ciphertext, ref uint clen)\r\n        {\r\n            OpenSSL.SetCtxNonce(_encryptCtx, _encNonce, true);\r\n            // buf: all plaintext\r\n            // outbuf: ciphertext + tag\r\n            int ret;\r\n            int tmpLen = 0;\r\n            clen = 0;\r\n            var tagBuf = new byte[tagLen];\r\n\r\n            ret = OpenSSL.EVP_CipherUpdate(_encryptCtx, ciphertext, out tmpLen,\r\n                plaintext, (int) plen);\r\n            if (ret != 1) throw new CryptoErrorException(\"openssl: fail to encrypt AEAD\");\r\n            clen += (uint) tmpLen;\r\n            // For AEAD cipher, it should not output anything\r\n            ret = OpenSSL.EVP_CipherFinal_ex(_encryptCtx, ciphertext, ref tmpLen);\r\n            if (ret != 1) throw new CryptoErrorException(\"openssl: fail to finalize AEAD\");\r\n            if (tmpLen > 0)\r\n            {\r\n                throw new System.Exception(\"openssl: fail to finish AEAD\");\r\n            }\r\n\r\n            OpenSSL.AEADGetTag(_encryptCtx, tagBuf, tagLen);\r\n            Array.Copy(tagBuf, 0, ciphertext, clen, tagLen);\r\n            clen += (uint) tagLen;\r\n        }\r\n\r\n        public override void cipherDecrypt(byte[] ciphertext, uint clen, byte[] plaintext, ref uint plen)\r\n        {\r\n            OpenSSL.SetCtxNonce(_decryptCtx, _decNonce, false);\r\n            // buf: ciphertext + tag\r","sourceCodeStart":79,"sourceCodeEnd":115,"githubUrl":"https://github.com/shadowsocks/shadowsocks-windows/blob/891d971682eefcaa2e640258d3b352a3ad3b2233/shadowsocks-csharp/Encryption/AEAD/AEADOpenSSLEncryptor.cs#L79-L115","documentation":"Thrown as CryptoErrorException when OpenSSL's EVP_CipherUpdate returns a value other than 1 during AEAD encryption. EVP_CipherUpdate feeds plaintext into the native cipher context (_encryptCtx) and writes ciphertext; a non-1 return means libcrypto rejected the operation. This wraps the EVP AEAD pipeline used for AES-GCM and ChaCha20-Poly1305 in Shadowsocks.","triggerScenarios":"cipherEncrypt() is called before InitCipher(), so _encryptCtx is IntPtr.Zero or uninitialized when SetCtxNonce / EVP_CipherUpdate run. The native EVP context pointer was freed or corrupted (use-after-free, double Dispose). Two threads call cipherEncrypt() on the same encryptor instance simultaneously (EVP contexts are not thread-safe). The ciphertext buffer is smaller than plen + tagLen, or the subkey/nonce were derived from a bad salt.","commonSituations":"Bundled libcrypto is the wrong architecture (32 vs 64-bit) or an incompatible OpenSSL major version that silently fails CIPHER operations. An encryptor instance is reused across multiple sockets/connections without synchronization. A build where InitCipher's EVP_CipherInit_ex failed but the error was swallowed. Heap corruption from a concurrent native call clobbers the context.","solutions":["Verify InitCipher(salt, isEncrypt: true, isUdp) was invoked exactly once before the first cipherEncrypt call and that none of its EVP_* calls returned non-1.","Ensure each AEADOpenSSLEncryptor instance is bound to a single connection/thread; never share one across sockets.","Confirm the bundled libcrypto matches the process architecture and the OpenSSL version expected by the wrapper, and that the native DLL loads at startup.","Allocate the ciphertext buffer to at least plen + tagLen bytes before calling cipherEncrypt."],"exampleFix":"// before\nenc.cipherEncrypt(plain, (uint)plain.Length, cipher, ref clen); // _encryptCtx not initialised\n\n// after\nenc.InitCipher(salt, true, isUdp);                       // must init first\nif (cipher.Length < plain.Length + enc.tagLen)\n    throw new InvalidOperationException(\"ciphertext buffer too small\");\nenc.cipherEncrypt(plain, (uint)plain.Length, cipher, ref clen);","handlingStrategy":"try-catch","validationCode":"// ensure the context is initialised and the buffer is large enough before encrypting\nif (_encryptCtx == IntPtr.Zero)\n    throw new InvalidOperationException(\"encryptor not initialised\");\nif (ciphertext == null || ciphertext.Length < plen + tagLen)\n    throw new ArgumentException(\"ciphertext buffer too small\");","typeGuard":null,"tryCatchPattern":"try {\n    enc.cipherEncrypt(plain, (uint)plain.Length, cipher, ref clen);\n} catch (CryptoErrorException ex) {\n    // the crypto context is now unusable — do not reuse it\n    logger.Error(ex, \"AEAD encrypt failed\");\n    enc.Dispose();\n    throw;\n}","preventionTips":["Call InitCipher exactly once before the first cipherEncrypt.","Never share one encryptor instance across threads or sockets.","Dispose encryptors to free native EVP contexts promptly.","Verify the OpenSSL native library loads at startup."],"tags":["openssl","crypto","aead","encryption","native-interop"],"backgroundTag":null,"analyzedSha":"891d971682eefcaa2e640258d3b352a3ad3b2233","analyzedAt":"2026-08-13T10:12:34.434Z","schemaVersion":2},"datasetVersion":"2026-08-13T14:17:21.547Z"}