VictoriaMetrics/VictoriaMetrics · error

cannot create gcs client with credsFile %q: %w

Error message

cannot create gcs client with credsFile %q: %w

What it means

Init creates a Google Cloud Storage client using a credentials JSON file (option.WithCredentialsFile). When storage.NewClient fails with that credentials file, this error wraps the underlying reason - typically an unreadable file or invalid JSON/service-account key. The client is required for all subsequent GCS operations.

Source

Thrown at lib/backup/gcsremote/gcs.go:68

	if fs.bkt != nil {
		logger.Panicf("BUG: fs.Init has been already called")
	}

	fs.ctx, fs.cancel = context.WithCancel(ctx)

	for strings.HasPrefix(fs.Dir, "/") {
		fs.Dir = fs.Dir[1:]
	}
	if !strings.HasSuffix(fs.Dir, "/") {
		fs.Dir += "/"
	}

	var client *storage.Client
	if len(fs.CredsFilePath) > 0 {
		creds := option.WithCredentialsFile(fs.CredsFilePath)
		c, err := storage.NewClient(fs.ctx, creds)
		if err != nil {
			return fmt.Errorf("cannot create gcs client with credsFile %q: %w", fs.CredsFilePath, err)
		}
		client = c
	} else {
		c, err := storage.NewClient(fs.ctx)
		if err != nil {
			return fmt.Errorf("cannot create default gcs client: %w", err)
		}
		client = c
	}

	client.SetRetry(
		storage.WithPolicy(storage.RetryAlways),
		storage.WithBackoff(gax.Backoff{
			Initial:    time.Second,
			Max:        time.Minute * 3,
			Multiplier: 3,
		}))
	fs.bkt = client.Bucket(fs.Bucket)

View on GitHub (pinned to 5079fb58f1)

Solutions

1. Verify the file exists and is readable: os.Stat the CredsFilePath and check permissions in the runtime environment (container/pod).
2. Validate the credentials with `gcloud auth activate-service-account --key-file=<path>` or `gcloud projects get-iam-policy` to confirm the key is valid and not revoked.
3. Use an absolute path for CredsFilePath and confirm the process working directory.
4. Check the wrapped error text for specifics like 'could not find default credentials' vs JSON parse errors and fix accordingly.

Example fix

// before
fs.CredsFilePath = "gcs-key.json" // relative; fails in service context
// after
fs.CredsFilePath = "/etc/mybackup/secrets/gcs-key.json"

creds := fs.CredsFilePath
if creds != "" {
	if _, err := os.Stat(creds); err != nil {
		return fmt.Errorf("credentials file missing/unreadable: %w", err)
	}
	var v map[string]any
	b, _ := os.ReadFile(creds)
	if json.Unmarshal(b, &v) != nil {
		return errors.New("credentials file is not valid JSON")
	}
	if v["type"] != "service_account" {
		return fmt.Errorf("unexpected credential type %v", v["type"])
	}
}

if err := fs.Init(); err != nil {
	if strings.Contains(err.Error(), "credsFile") {
		// surface config problem: check path, mount, key validity
	}
	return fmt.Errorf("gcs init failed: %w", err)
}

When it happens

Trigger: Calling fs.Init() with fs.CredsFilePath set where the file does not exist, is unreadable by the process user, contains malformed JSON, is not a valid service-account key, or the storage.NewClient construction itself fails (bad endpoint/option).

Common situations: Wrong path in config (relative path resolved from the wrong working directory); k8s secret not mounted or mounted with wrong name; key downloaded from a different project or revoked; JSON truncated during secret rotation.

Related errors

• cannot create default gcs client: %w
• error in SetAttrSelection: %w
• unexpected prefix for gcs key %q; want %q
• algorithm is not supported
• missing AWS secret_key; it may be set via env var AWS_SECRET

AI-assisted analysis of VictoriaMetrics/VictoriaMetrics@5079fb58f1 (2026-09-03). Data as JSON: /api/errors/5d691b41f1a554b6. Report an issue: GitHub.