caddyserver/caddy · warning

storing STEK gob: %v

Error message

storing STEK gob: %v

What it means

storeSTEK writes the gob-encoded STEK bytes to the configured storage via Store(); write failures are wrapped as 'storing STEK gob'. This is an infrastructure-level failure: the storage backend rejected the write.

Source

Thrown at modules/caddytls/distributedstek/distributedstek.go:141

		return sg, err // don't wrap, in case error is certmagic.ErrNotExist
	}
	dec := gob.NewDecoder(bytes.NewReader(gobBytes))
	err = dec.Decode(&sg)
	if err != nil {
		return sg, fmt.Errorf("STEK gob corrupted: %v", err)
	}
	return sg, nil
}

func (s *Provider) storeSTEK(dstek distributedSTEK) error {
	var buf bytes.Buffer
	err := gob.NewEncoder(&buf).Encode(dstek)
	if err != nil {
		return fmt.Errorf("encoding STEK gob: %v", err)
	}
	err = s.storage.Store(s.ctx, stekFileName, buf.Bytes())
	if err != nil {
		return fmt.Errorf("storing STEK gob: %v", err)
	}
	return nil
}

// getSTEK locks and loads the current STEK from storage. If none
// currently exists, a new STEK is created and persisted. If the
// current STEK is outdated (NextRotation time is in the past),
// then it is rotated and persisted. The resulting STEK is returned.
func (s *Provider) getSTEK() (distributedSTEK, error) {
	err := s.storage.Lock(s.ctx, stekLockName)
	if err != nil {
		return distributedSTEK{}, fmt.Errorf("failed to acquire storage lock: %v", err)
	}

	//nolint:errcheck
	defer s.storage.Unlock(s.ctx, stekLockName)

	// load the current STEKs from storage

View on GitHub (pinned to 50e54ee279)

Solutions

  1. Check the wrapped backend error — it names the real cause (ENOSPC, EACCES, connection refused, etc.)
  2. Free space / fix permissions on the storage root (commonly /var/lib/caddy) for file storage
  3. Restore connectivity or credentials for the storage plugin, then reload Caddy so STEK rotation retries
  4. If using network storage under heavy multi-instance contention, verify the lock timeout settings of the storage plugin

Example fix

# before: storage volume full
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1        8G   8G     0 100% /

# after: free space, then reload
docker system prune -f   # or grow the volume
systemctl reload caddy
Defensive patterns

Strategy: retry

Validate before calling

// Pre-deploy capacity check for file storage (the common backend)
// (shell) df -h /var/lib/caddy && test -w /var/lib/caddy

Try / catch

// Ops automation: on 'storing STEK gob' alerts, inspect the wrapped cause, repair the backend
// (free space / restore Redis), then reload Caddy — the provider retries rotation on next cycle.
if alert.Msg matches "storing STEK gob" {
	repairStorageBackend()
	systemctl reload caddy
}

Prevention

When it happens

Trigger: File storage: disk full, permission denied on the storage dir, read-only filesystem. Plugin storage (Redis/S3/etc.): connection lost, credentials expired, quota exceeded. Also possible when the storage lock expires mid-write under contention.

Common situations: Containers with small ephemeral volumes filling up; Redis evictions or restarts; IAM credential rotation on object storage; NFS stale handles.

Related errors


AI-assisted analysis of caddyserver/caddy@50e54ee279 (2026-08-15). Data as JSON: /api/errors/bc1b29f51b6e5b11. Report an issue: GitHub.