XTLS/Xray-core · error

failed to compute mod inverse

Error message

failed to compute mod inverse

What it means

DeriveRSAKey could not compute d = e^-1 mod totient(p-1)(q-1) with e = 65537. ModInverse returns nil only when e and the totient are not coprime. Because derivePrime explicitly searches for primes p, q with gcd(p-1, 65537) == 1, this branch is a near-unreachable internal invariant failure: in practice it signals a logic regression in the prime-derivation code rather than a bad password.

Source

Thrown at transport/internet/finalmask/xmc/derivation.go:89

				e := big.NewInt(65537)
				gcd := new(big.Int).GCD(nil, nil, qMinus1, e)
				if gcd.Cmp(big.NewInt(1)) == 0 {
					break
				}
			}
			q.Add(q, big.NewInt(2))
		}
	}

	n := new(big.Int).Mul(p, q)
	pMinus1 := new(big.Int).Sub(p, big.NewInt(1))
	qMinus1 := new(big.Int).Sub(q, big.NewInt(1))
	totient := new(big.Int).Mul(pMinus1, qMinus1)

	e := big.NewInt(65537)
	d := new(big.Int).ModInverse(e, totient)
	if d == nil {
		return nil, fmt.Errorf("failed to compute mod inverse")
	}

	priv := &rsa.PrivateKey{
		PublicKey: rsa.PublicKey{
			N: n,
			E: 65537,
		},
		D:      d,
		Primes: []*big.Int{p, q},
	}
	priv.Precompute()

	return priv, nil
}

View on GitHub (pinned to 7d214f8b09)

Solutions

1. If you maintain a fork, re-add the gcd(p-1, 65537) == 1 check to every loop that advances a prime candidate (derivePrime and the p==q adjustment in DeriveRSAKey)
2. If using stock code, report it upstream with the exact binary/version: stock derivation cannot produce non-coprime primes
3. Do not retry with a different password; the failure is deterministic in the derivation logic, not the input

Example fix

// before
q.Add(q, big.NewInt(2)) // step without gcd check

// after
q.Add(q, big.NewInt(2))
qMinus1 := new(big.Int).Sub(q, big.NewInt(1))
if new(big.Int).GCD(nil, nil, qMinus1, big.NewInt(65537)).Cmp(big.NewInt(1)) != 0 {
    continue
}

key, err := xmc.DeriveRSAKey(password)
if err != nil {
    // deterministic derivation failure: do not retry, surface immediately
    return nil, fmt.Errorf("rsa derivation broken: %w", err)
}

When it happens

Trigger: Calling DeriveRSAKey(password) after modifying derivePrime or the p != q adjustment loop so that the gcd(p-1, e) == 1 guarantee is lost; any password triggers it once the invariant is broken.

Common situations: Forks of the transport that change the exponent, prime search step (Add 2), or the q-dedup loop; unit tests that stub out the SHA-256 stream with degenerate seeds; essentially never seen on unmodified code.

Related errors

• parse server public key: %w
• parse server public key: not rsa
• encrypt shared secret: %w
• encrypt verify token: %w
• Failed to build REALITY config.

AI-assisted analysis of XTLS/Xray-core@7d214f8b09 (2026-08-15). Data as JSON: /api/errors/1a2b1303eb436fdc. Report an issue: GitHub.