SigNoz/signoz · error · model.ApiError

error in reading data

Error message

error in reading data

What it means

ApiError (ErrorInternal) from GetTopLevelOperations when rows.Scan of (name, serviceName, timestamp) fails while iterating results of the top-level operations query — the query executed but returned rows incompatible with the scan targets.

Source

Thrown at pkg/query-service/app/clickhouseReader/reader.go:329

	query := fmt.Sprintf(`SELECT name, serviceName, max(time) as ts FROM %s.%s WHERE time >= @start`, r.TraceDB, r.topLevelOperationsTable)
	if len(services) > 0 {
		query += ` AND serviceName IN @services`
	}
	query += ` GROUP BY name, serviceName ORDER BY ts DESC LIMIT 5000`

	rows, err := r.db.Query(ctx, query, clickhouse.Named("start", start), clickhouse.Named("services", services))

	if err != nil {
		r.logger.Error("Error in processing sql query", errorsV2.Attr(err))
		return nil, &model.ApiError{Typ: model.ErrorExec, Err: fmt.Errorf("error in processing sql query")}
	}

	defer rows.Close()
	for rows.Next() {
		var name, serviceName string
		var t time.Time
		if err := rows.Scan(&name, &serviceName, &t); err != nil {
			return nil, &model.ApiError{Typ: model.ErrorInternal, Err: fmt.Errorf("error in reading data")}
		}
		if _, ok := operations[serviceName]; !ok {
			operations[serviceName] = []string{"overflow_operation"}
		}
		operations[serviceName] = append(operations[serviceName], name)
	}
	return &operations, nil
}

func (r *ClickHouseReader) buildResourceSubQuery(ctx context.Context, orgID valuer.UUID, tags []model.TagQueryParam, svc string, start, end time.Time) (string, error) {
	// assuming all will be resource attributes.
	// and resource attributes are string for traces
	filterSet := v3.FilterSet{}
	for _, tag := range tags {
		// skip the collector id as we don't add it to traces
		if tag.Key == "signoz.collector.id" {
			continue
		}

View on GitHub (pinned to 5069bf80b0)

Solutions

1. Inspect the wrapped scan error in debugging/logs
2. DESCRIBE the traces table and verify name/serviceName/ts column types match expectations
3. Upgrade/align the clickhouse driver and server versions
4. Re-run against a healthy ClickHouse node

// Verify expected columns/types before scanning-heavy calls
cols := describeTable(ctx, ch, r.TraceDB, r.traceTableName)
requireColumns(cols, map[string]string{"name":"String","serviceName":"String"})

func isReadDataErr(apiErr *model.ApiError) bool {
    return apiErr != nil && apiErr.Typ == model.ErrorInternal && strings.Contains(apiErr.Err.Error(), "error in reading data")
}

ops, apiErr := reader.GetTopLevelOperations(ctx, params)
if isReadDataErr(apiErr) {
    // likely type drift: pin driver/server versions and re-verify schema
    return nil, apiErr
}

When it happens

Trigger: Result columns for name/serviceName/time arrive with unexpected types or NULLs — typically after a schema/type change in the traces table or a driver deserialization mismatch — causing Scan to error inside the rows.Next loop.

Common situations: Version mismatches between clickhouse-go driver and server, schema drift after upgrades altering column types, or corrupted/partial results from an unhealthy CH node.

Related errors

• couldn't scan attrib value rows: %w
• error while scanning metric name: %s
• error while scanning meter name: %s
• error in processing sql query
• error while scanning rows: %s

AI-assisted analysis of SigNoz/signoz@5069bf80b0 (2026-08-28). Data as JSON: /api/errors/75d358a9a0390eea. Report an issue: GitHub.