{"record":{"id":"f59007af26bda632","repo":"xai-org/x-algorithm","slug":"kerberos-renewal-failed-stderr-decode","errorCode":null,"errorMessage":"Kerberos renewal failed: {stderr.decode()}","messagePattern":"Kerberos renewal failed: (.+?)","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"critical","filePath":"grox/libs/kerberos_cli/kerberos.py","lineNumber":80,"sourceCode":"            self.config.principal,\n        ]\n        logger.info(\n            f\"Start renewing Kerberos ticket with command: {' '.join(kinit_cmd)}\"\n        )\n        try:\n            process = await asyncio.create_subprocess_exec(\n                *kinit_cmd,\n                stdout=asyncio.subprocess.PIPE,\n                stderr=asyncio.subprocess.PIPE,\n            )\n            _, stderr = await process.communicate()\n            if process.returncode == 0:\n                logger.info(\n                    f\"Finished renewing Kerberos ticket for {self.config.principal}\"\n                )\n            else:\n                logger.error(f\"Kerberos renewal failed: {stderr.decode()}\")\n                raise Exception(f\"Kerberos renewal failed: {stderr.decode()}\")\n        except Exception:\n            logger.error(f\"Error during Kerberos renewal: {traceback.format_exc()}\")\n            raise\n\n    async def renew(self):\n        await self._renew_kerberos_ticket()\n\n    def start(self):\n        logger.info(\"Starting Kerberos renewer\")\n        if self._renewer is not None:\n            logger.warning(\"Kerberos renewer already started, skipping\")\n            return\n        self._renewer = asyncio.create_task(self._kerberos_renewal_loop())\n        logger.info(\"Kerberos renewer started\")\n\n    def stop(self):\n        if self._renewer is not None:\n            logger.warning(\"Stopping Kerberos renewer\")","sourceCodeStart":62,"sourceCodeEnd":98,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/grox/libs/kerberos_cli/kerberos.py#L62-L98","documentation":"_renew_kerberos_ticket shells out to kinit; a non-zero return code (with captured stderr) triggers a bare Exception whose message embeds kinit's stderr. The renewal loop and the public renew() both surface it. Because it is a generic Exception raised after logging, callers cannot narrowly catch a typed error.","triggerScenarios":"kinit failing: wrong keytab for the principal, principal expired/disabled in KDC, clock skew beyond allowed skew, krb5.conf misconfigured (wrong REALM/KDC), or keytab permissions/ownership preventing read.","commonSituations":"Keytab rotated but pod still has the old secret; principal renamed; krb5.conf missing in a slim container; container clock drift; DNS issues reaching the KDC.","solutions":["Read the embedded stderr — kinit's message (e.g. 'Preauthentication failed', 'Clock skew too great') identifies the root cause.","Verify keytab matches principal: klist -kt /path/keytab and compare against the principal string.","Check clock sync (ntpd/chrony) and krb5.conf REALM/KDC entries in the container.","After rotating the keytab in the KDC, restart pods so the mounted Secret is the fresh one."],"exampleFix":"# before\nawait client.renew()  # Exception: Kerberos renewal failed: Preauthentication failed\n\n# after\ntry:\n    await client.renew()\nexcept Exception as e:\n    logger.error('kinit failed: %s', e)\n    await alert_ops('kerberos-renewal-failed')\n    raise","handlingStrategy":"retry","validationCode":"# preflight: verify keytab matches principal before relying on renewal\nimport subprocess\nout = subprocess.run(['klist', '-kt', kt], capture_output=True, text=True)\nif principal not in out.stdout:\n    raise SystemExit('keytab does not contain principal; rotation needed')","typeGuard":null,"tryCatchPattern":"for attempt in range(3):\n    try:\n        await client.renew()\n        break\n    except Exception as e:\n        if 'Clock skew' in str(e):\n            sync_clock(); continue\n        logger.error('kinit failed: %s', e)\n        raise\nelse:\n    alert_ops('kerberos-renewal-exhausted')","preventionTips":["Monitor the renewal loop and alert on first failure","Keep clocks synced (chrony/ntp) in kerberos-enabled containers","Rotate keytabs by restarting pods after KDC-side changes"],"tags":["kerberos","kinit","authentication","subprocess","renewal"],"backgroundTag":"kerberos-auth-failed","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}