xai-org/x-algorithm · critical · Exception

Kerberos renewal failed: {stderr.decode()}

Error message

Kerberos renewal failed: {stderr.decode()}

What it means

_renew_kerberos_ticket shells out to kinit; a non-zero return code (with captured stderr) triggers a bare Exception whose message embeds kinit's stderr. The renewal loop and the public renew() both surface it. Because it is a generic Exception raised after logging, callers cannot narrowly catch a typed error.

Source

Thrown at grox/libs/kerberos_cli/kerberos.py:80

            self.config.principal,
        ]
        logger.info(
            f"Start renewing Kerberos ticket with command: {' '.join(kinit_cmd)}"
        )
        try:
            process = await asyncio.create_subprocess_exec(
                *kinit_cmd,
                stdout=asyncio.subprocess.PIPE,
                stderr=asyncio.subprocess.PIPE,
            )
            _, stderr = await process.communicate()
            if process.returncode == 0:
                logger.info(
                    f"Finished renewing Kerberos ticket for {self.config.principal}"
                )
            else:
                logger.error(f"Kerberos renewal failed: {stderr.decode()}")
                raise Exception(f"Kerberos renewal failed: {stderr.decode()}")
        except Exception:
            logger.error(f"Error during Kerberos renewal: {traceback.format_exc()}")
            raise

    async def renew(self):
        await self._renew_kerberos_ticket()

    def start(self):
        logger.info("Starting Kerberos renewer")
        if self._renewer is not None:
            logger.warning("Kerberos renewer already started, skipping")
            return
        self._renewer = asyncio.create_task(self._kerberos_renewal_loop())
        logger.info("Kerberos renewer started")

    def stop(self):
        if self._renewer is not None:
            logger.warning("Stopping Kerberos renewer")

View on GitHub (pinned to 24c60942c5)

Solutions

1. Read the embedded stderr — kinit's message (e.g. 'Preauthentication failed', 'Clock skew too great') identifies the root cause.
2. Verify keytab matches principal: klist -kt /path/keytab and compare against the principal string.
3. Check clock sync (ntpd/chrony) and krb5.conf REALM/KDC entries in the container.
4. After rotating the keytab in the KDC, restart pods so the mounted Secret is the fresh one.

Example fix

# before
await client.renew()  # Exception: Kerberos renewal failed: Preauthentication failed

# after
try:
    await client.renew()
except Exception as e:
    logger.error('kinit failed: %s', e)
    await alert_ops('kerberos-renewal-failed')
    raise

# preflight: verify keytab matches principal before relying on renewal
import subprocess
out = subprocess.run(['klist', '-kt', kt], capture_output=True, text=True)
if principal not in out.stdout:
    raise SystemExit('keytab does not contain principal; rotation needed')

for attempt in range(3):
    try:
        await client.renew()
        break
    except Exception as e:
        if 'Clock skew' in str(e):
            sync_clock(); continue
        logger.error('kinit failed: %s', e)
        raise
else:
    alert_ops('kerberos-renewal-exhausted')

When it happens

Trigger: kinit failing: wrong keytab for the principal, principal expired/disabled in KDC, clock skew beyond allowed skew, krb5.conf misconfigured (wrong REALM/KDC), or keytab permissions/ownership preventing read.

Common situations: Keytab rotated but pod still has the old secret; principal renamed; krb5.conf missing in a slim container; container clock drift; DNS issues reaching the KDC.

Related errors

• Keytab file not found: {self.config.keytab_path}
• API key is required: set EapiModelConfig.api_key or the XAI_
• Renew interval must be greater than 0
• Attempts must be greater than 0
• Retry interval must be greater than 0

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/f59007af26bda632. Report an issue: GitHub.