crowdsecurity/crowdsec · error

query is mandatory (at least start_date and end_date or back

Error message

query is mandatory (at least start_date and end_date or backlog)

What it means

ConfigureByDSN parses a cloudwatch DSN of the form cloudwatch:///group:stream?query. The DSN must contain exactly one '?' separating path from query parameters (which must include date ranges or backlog). Any DSN that doesn't split into exactly 2 parts on '?' is rejected.

Source

Thrown at pkg/acquisition/modules/cloudwatch/config.go:207

		targetStream = *s.Config.StreamRegexp
	} else if s.Config.StreamName != nil {
		targetStream = *s.Config.StreamName
	}

	s.logger.Infof("Adding cloudwatch group '%s' (stream:%s) to datasources", s.Config.GroupName, targetStream)

	return nil
}

func (s *Source) ConfigureByDSN(ctx context.Context, dsn string, labels map[string]string, logger *log.Entry, uuid string) error {
	s.logger = logger

	dsn = strings.TrimPrefix(dsn, s.GetName()+"://")

	args := strings.Split(dsn, "?")
	if len(args) != 2 {
		return errors.New("query is mandatory (at least start_date and end_date or backlog)")
	}

	frags := strings.Split(args[0], ":")
	if len(frags) != 2 {
		return errors.New("cloudwatch path must contain group and stream : /my/group/name:stream/name")
	}

	s.Config.GroupName = frags[0]
	s.Config.StreamName = &frags[1]
	s.Config.Labels = labels
	s.Config.UniqueId = uuid

	u, err := url.ParseQuery(args[1])
	if err != nil {
		return fmt.Errorf("while parsing %s: %w", dsn, err)
	}

	for k, v := range u {

View on GitHub (pinned to 909b515798)

Solutions

1. Append a query string with start_date & end_date or backlog, e.g. cloudwatch:///my/group:stream?backlog=1h
2. URL-encode parameter values so no stray '?' breaks the split
3. Use the two-part form path?query exactly once

Example fix

// before
cloudwatch:///my/log/group:my-stream
// after
cloudwatch:///my/log/group:my-stream?backlog=1h

u, _ := url.Parse(dsn)
if !strings.Contains(u.RawQuery, "backlog=") && !strings.Contains(u.RawQuery, "start_date=") {
    return errors.New("cloudwatch DSN needs ?backlog= or start_date/end_date")
}

func validCWDSN(dsn string) bool { parts := strings.Split(strings.TrimPrefix(dsn, "cloudwatch://"), "?"); return len(parts) == 2 }

if err := src.ConfigureByDSN(dsn, logger, labels, uuid); err != nil {
    if strings.Contains(err.Error(), "query is mandatory") { /* append ?backlog= or date range */ }
    return err
}

When it happens

Trigger: ConfigureByDSN called with a DSN containing no '?' or more than one '?', e.g. cloudwatch:///my/group:stream (no params) or a URL-encoded '?' inside a parameter value.

Common situations: Building DSNs programmatically without query parameters; forgetting to URL-encode values so an extra '?' appears; old/example DSN strings copied without the query part.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Related errors

• cloudwatch path must contain group and stream : /my/group/na
• while parsing %s: %w
• group_name is mandatory for CloudwatchSource
• aws_region is not specified, specify it or aws_config_dir
• empty loki host

AI-assisted analysis of crowdsecurity/crowdsec@909b515798 (2026-09-06). Data as JSON: /api/errors/ca67ae361ec06e3d. Report an issue: GitHub.