shadowsocks/shadowsocks-windows · critical · CryptoErrorException

ret is

Error message

ret is {0}

What it means

Thrown as CryptoErrorException from cipherEncrypt when MbedTLS.cipher_auth_encrypt (AES-GCM authenticated encryption) returns non-zero. The formatted message includes the native return code. Non-zero from an AEAD encrypt call means the operation could not complete — typically an internal GCM failure rather than bad input, since the inputs were validated earlier during setup.

Solutions

1. Log the exact ret value and look it up in mbedTLS error codes (e.g. MBEDTLS_ERR_CIPHER_*).
2. Ensure the encrypt context is freshly initialized per session and not shared across threads.
3. Chunk very large plaintexts instead of passing them in one cipher_auth_encrypt call.
4. Confirm _encNonce is being incremented (IncrementNonce) and not reused.

Example fix

// before
if (ret != 0) throw new CryptoErrorException(String.Format("ret is {0}", ret));

// after
if (ret != 0)
    throw new CryptoErrorException($"mbedTLS cipher_auth_encrypt failed (ret={ret}, plen={plen})");

// Sanity-check plaintext length against a per-call cap
const int MaxChunk = 1 << 20;
if (plen > MaxChunk) /* chunk before calling cipher_auth_encrypt */

try { cipherEncrypt(plain, plen, cipher, ref clen); }
catch (CryptoErrorException ex)
{ /* log ret code; close the relay, the session key/nonce state is suspect */ }

When it happens

Trigger: Plaintext length exceeding the cipher's per-call limits; nonce reuse collision detected internally; the encrypt context was corrupted between setup and this call; plen/tagLen combination the GCM implementation rejects.

Common situations: Encrypting an unusually large buffer in a single call; a context being reused after a partial failure left it in a bad state; nonce counter overflow; native memory corruption.

Related errors

• Cannot initialize mbed TLS cipher context
• failed to finish preparation
• failed to generate session key
• failed to set key
• method not found

AI-assisted analysis of shadowsocks/shadowsocks-windows@891d971682 (2026-08-13). Data as JSON: /api/errors/dd95e45e1b5f9f24. Report an issue: GitHub.

Appendix: source

Thrown at shadowsocks-csharp/Encryption/AEAD/AEADMbedTLSEncryptor.cs:91

            // buf: all plaintext
            // outbuf: ciphertext + tag
            int ret;
            byte[] tagbuf = new byte[tagLen];
            uint olen = 0;
            switch (_cipher)
            {
                case CIPHER_AES:
                    ret = MbedTLS.cipher_auth_encrypt(_encryptCtx,
                        /* nonce */
                        _encNonce, (uint) nonceLen,
                        /* AD */
                        IntPtr.Zero, 0,
                        /* plain */
                        plaintext, plen,
                        /* cipher */
                        ciphertext, ref olen,
                        tagbuf, (uint) tagLen);
                    if (ret != 0) throw new CryptoErrorException(String.Format("ret is {0}", ret));
                    Debug.Assert(olen == plen);
                    // attach tag to ciphertext
                    Array.Copy(tagbuf, 0, ciphertext, (int) plen, tagLen);
                    clen = olen + (uint) tagLen;
                    break;
                default:
                    throw new System.Exception("not implemented");
            }
        }

        public override void cipherDecrypt(byte[] ciphertext, uint clen, byte[] plaintext, ref uint plen)
        {
            // buf: ciphertext + tag
            // outbuf: plaintext
            int ret;
            uint olen = 0;
            // split tag
            byte[] tagbuf = new byte[tagLen];

View on GitHub (pinned to 891d971682)