What causes "failed to execute query on workspace %s: %w" in wtfutil/wtf?

Inspect the wrapped error for the HTTP status code and Azure error code to identify the root cause

failed to execute query on workspace %s: %w

Error message

failed to execute query on workspace %s: %w

What it means

RunQuery wraps the error returned by the Azure Log Analytics QueryClient's Query operation (Resources.Query on the workspace) with this message. The Azure SDK call itself failed — network error, HTTP error status, auth rejection, or invalid query/request body. The raw SDK error is preserved via %w.

Source

Thrown at modules/azurelogs/query.go:76

			LogQueryClients[qf.SubscriptionID], err = CreateLogsClient(sess, qf.SubscriptionID)
			if err != nil {
				clientsMutex.Unlock()
				return nil, fmt.Errorf("failed to create Azure Logs client for subscription %s: %w", qf.SubscriptionID, err)
			}
		}
		client = LogQueryClients[qf.SubscriptionID]
		clientsMutex.Unlock()
	}

	res, err := client.QueryWorkspace(
		context.Background(),
		qf.WorkspaceID,
		azquery.Body{
			Query: to.Ptr(qf.Query),
		},
		nil)
	if err != nil {
		return nil, fmt.Errorf("failed to execute query on workspace %s: %w", qf.WorkspaceID, err)
	}

	if res.Error != nil {
		return nil, res.Error
	}

	switch len(res.Tables) {
	case 0:
		return nil, fmt.Errorf("query returned no data tables: %s", qf.Query)
	case 1:
		if len(res.Tables[0].Columns) == 0 {
			return nil, fmt.Errorf("query returned table with no columns: %s", qf.Query)
		}
	default:
		return nil, fmt.Errorf("query returned %d tables, expected 1: %s", len(res.Tables), qf.Query)
	}

	// Process each row of data

View on GitHub (pinned to bb838c1ccb)

Solutions

Inspect the wrapped error for the HTTP status code and Azure error code to identify the root cause
Test the same KQL query manually in the Azure Portal Log Analytics query editor
Re-authenticate if the error indicates 401/expired token
Check workspace ID correctness and that the identity has Log Analytics Reader access
Add backoff/retry for 429 (throttled) responses

Example fix

// before
res, err := client.Query(ctx, workspaceID, azquery.Body{Query: qf.Query}, nil)
// after: validate the query and retry transient failures
if isThrottlingError(err) { time.Sleep(backoff); res, err = client.Query(ctx, workspaceID, azquery.Body{Query: qf.Query}, nil) }

Defensive patterns

Strategy: retry

Validate before calling

// validate KQL basics and workspace access before calling RunQuery
// e.g. run a cheap `Heartbeat | take 1` probe query first
probeRes, err := client.Query(ctx, workspaceID, azquery.Body{Query: to.Ptr("Heartbeat | take 1")}, nil)
if err != nil { return fmt.Errorf("workspace unreachable: %w", err) }

Try / catch

var resp *azquery.QueryResponse
for attempt := 0; attempt < 3; attempt++ {
    resp, err = RunQuery(sess)
    if err == nil || !isRetryable(err) { break } // retry 429/5xx only
    time.Sleep(time.Duration(1<<attempt) * time.Second)
}

Prevention

Test KQL in the Azure Portal query editor before shipping it in config
Implement exponential backoff for 429/5xx responses
Keep tokens fresh; long-running jobs should re-create credentials on 401
Monitor Azure Log Analytics throttling limits for your workspace

When it happens

Trigger: RunQuery or fetchDataAsync invoking client.Query(workspaceID, azquery.Body{Query: ...}, nil) when the HTTP request to the Log Analytics API fails: 401/403 auth failure, 400 invalid KQL, workspace not found, throttling (429), or network outage.

Common situations: KQL syntax error rejected with 400, token expired mid-run, workspace deleted or ID wrong, region/network egress blocked, or API throttling under heavy polling.

Related errors

AI-assisted analysis of wtfutil/wtf@bb838c1ccb (2026-09-03). Data as JSON: /api/errors/a33f556272aa9878. Report an issue: GitHub.