crowdsecurity/crowdsec · error

storing metrics snapshot for '%s' at %s: %w

Error message

storing metrics snapshot for '%s' at %s: %w

What it means

CreateMetric inserts a usage-metrics snapshot row (generatedBy, receivedAt, payload) into the local SQLite DB via ent. When the ent Save() fails, the original error is logged as a warning and a wrapped InsertFail ('unable to insert row') sentinel is returned, so callers only see the generic insert failure, not the underlying cause (check logs for details).

Source

Thrown at pkg/database/metrics.go:22

	"context"
	"fmt"
	"time"

	"github.com/crowdsecurity/crowdsec/pkg/database/ent"
	"github.com/crowdsecurity/crowdsec/pkg/database/ent/metric"
)

func (c *Client) CreateMetric(ctx context.Context, generatedType metric.GeneratedType, generatedBy string, receivedAt time.Time, payload string) (*ent.Metric, error) {
	metric, err := c.Ent.Metric.
		Create().
		SetGeneratedType(generatedType).
		SetGeneratedBy(generatedBy).
		SetReceivedAt(receivedAt).
		SetPayload(payload).
		Save(ctx)
	if err != nil {
		c.Log.Warningf("CreateMetric: %s", err)
		return nil, fmt.Errorf("storing metrics snapshot for '%s' at %s: %w", generatedBy, receivedAt, InsertFail)
	}

	return metric, nil
}

func (c *Client) GetLPUsageMetricsByMachineID(ctx context.Context, machineId string, toSend bool) ([]*ent.Metric, error) {
	query := c.Ent.Metric.Query().
		Where(
			metric.GeneratedTypeEQ(metric.GeneratedTypeLP),
			metric.GeneratedByEQ(machineId),
		)

	if toSend {
		query = query.Where(metric.PushedAtIsNil())
	}

	metrics, err := query.All(ctx)
	if err != nil {

View on GitHub (pinned to 909b515798)

Solutions

1. Check the accompanying Warningf log line 'CreateMetric: <err>' for the real cause (e.g. 'database is locked', 'no such table').
2. Ensure only one crowdsec instance uses the same DB and increase contention tolerance (WAL mode) or reduce writers.
3. Run 'cscli db migrate' / restart crowdsec so ent schema migrations create/update the metrics table.
4. Free disk space and verify the DB file ('cscli db doctor' or sqlite3 integrity_check).
5. Retry the operation; CreateMetric is called from the periodic usage-metrics loop, so a transient failure self-heals on the next tick.

Example fix

// before: only generic sentinel returned, real cause only in logs
if err != nil {
    return nil, fmt.Errorf("storing metrics snapshot for '%s' at %s: %w", generatedBy, receivedAt, InsertFail)
}
// after (caller side): check the underlying causes with errors.Is
var insertFail = db.InsertFail
if errors.Is(err, insertFail) {
    log.Warnf("metrics insert failed, will retry next cycle: %v", err)
}

// pre-check DB writability before the insert
if _, err := os.Stat(dbPath); err != nil { return err }
if fi, err := os.Stat(dbPath); err == nil && fi.Size() > 0 {
    if f, err := os.OpenFile(dbPath, os.O_WRONLY, 0); err == nil { f.Close() } else { return err }
}

metric, err := client.CreateMetric(ctx, generatedBy, receivedAt, payload)
if err != nil {
    if errors.Is(err, database.InsertFail) {
        log.Warnf("metric insert failed (see warning log for cause), retrying next cycle: %v", err)
        return nil // non-fatal; metrics are periodic
    }
    return err
}

When it happens

Trigger: Calling CreateMetric when the underlying database INSERT fails: DB file locked by another writer, disk full, schema out of date (missing metric table), corrupted SQLite file, or context cancellation during Save.

Common situations: Multiple crowdsec/cscli processes contending for the SQLite write lock; disk quota exhausted on /var/lib/crowdsec; upgrading crowdsec without running the schema migration so the metrics table is stale.

Related errors

• getting LP usage metrics by origin %s: %w
• getting bouncer usage metrics by name %s: %w
• marking usage metrics as sent: %w
• getting unsent usage metrics: %w
• marking stale usage metrics as sent: %w

AI-assisted analysis of crowdsecurity/crowdsec@909b515798 (2026-09-06). Data as JSON: /api/errors/5ba3e8d288525d0a. Report an issue: GitHub.