k3s-io/k3s · error

%s error ID %05d

Error message

%s error ID %05d

What it means

SendErrorWithID generates a random 5-digit ID, logs the real error server-side ('<component> error ID 01234: <detail>'), and sends the client only '<component> error ID 01234' with HTTP 500 unless an explicit status was passed. The design avoids information disclosure: the REST response carries no stack or detail, but the ID correlates it to the server log entry containing the root cause.

Source

Thrown at pkg/util/apierrors.go:28

	"github.com/k3s-io/api/pkg/generated/clientset/versioned/scheme"
	"github.com/sirupsen/logrus"
	apierrors "k8s.io/apimachinery/pkg/api/errors"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/apimachinery/pkg/runtime/schema"
	"k8s.io/apiserver/pkg/endpoints/handlers/responsewriters"
)

var ErrAPINotReady = errors.New("apiserver not ready")
var ErrAPIDisabled = errors.New("apiserver disabled")
var ErrCoreNotReady = errors.New("runtime core not ready")

// SendErrorWithID sends and logs a random error ID so that logs can be correlated
// between the REST API (which does not provide any detailed error output, to avoid
// information disclosure) and the server logs.
func SendErrorWithID(err error, component string, resp http.ResponseWriter, req *http.Request, status ...int) {
	errID, _ := rand.Int(rand.Reader, big.NewInt(99999))
	logrus.Errorf("%s error ID %05d: %v", component, errID, err)
	SendError(fmt.Errorf("%s error ID %05d", component, errID), resp, req, status...)
}

// SendError sends a properly formatted error response
func SendError(err error, resp http.ResponseWriter, req *http.Request, status ...int) {
	var code int
	if len(status) == 1 {
		code = status[0]
	}
	if code == 0 || code == http.StatusOK {
		code = http.StatusInternalServerError
	}

	// Don't log "apiserver not ready" or "apiserver disabled" errors, they are frequent during startup
	if !errors.Is(err, ErrAPINotReady) && !errors.Is(err, ErrAPIDisabled) {
		logrus.Errorf("Sending %s %d response to %s: %v", req.Proto, code, req.RemoteAddr, err)
	}

	var serr *apierrors.StatusError

View on GitHub (pinned to 6ba341e396)

Solutions

1. Take the exact ID from the HTTP body and grep the server logs for it (journalctl, kubectl logs, or container logs) to see the underlying error
2. If it occurs right after startup, wait for the apiserver/runtime core to become ready and retry
3. If persistent, fix the root cause shown in the log line (config, certificates, connectivity) - not the wrapper
4. If you operate the client, log the error ID in your own logs/support bundle for correlation with the server

var errIDRe = regexp.MustCompile(`error ID (\d{5})`)

func extractErrorID(body string) (string, bool) {
	m := errIDRe.FindStringSubmatch(body)
	if m == nil {
		return "", false
	}
	return m[1], true
}

resp, err := client.Do(req)
if err != nil {
	return err
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
if resp.StatusCode == http.StatusInternalServerError {
	if id, ok := extractErrorID(string(body)); ok {
		log.Printf("server error %s: correlate with server log 'error ID %s'", resp.Status, id)
	}
	// internal errors may be transient (startup, etcd blip): retry with backoff
	return retryWithBackoff(req)
}

When it happens

Trigger: Any handler using SendErrorWithID hitting an internal failure, e.g. the apiserver/runtime core not yet ready, etcd unavailability, or an unexpected error while processing the request.

Common situations: Hitting the REST API while the embedded control plane is still booting; transient etcd/apiserver unavailability; handler bugs surfacing as opaque 500s that confuse users because the body has no detail.

Related errors

• invalid snapshot operation
• invalid username/password combination
• token must not be empty
• etcd datastore disabled
• hijacking not supported

AI-assisted analysis of k3s-io/k3s@6ba341e396 (2026-08-15). Data as JSON: /api/errors/e4461682ada334bc. Report an issue: GitHub.