argoproj/argo-workflows · error

error listing blobs in Azure Blob Storage container

Error message

error listing blobs %s in Azure Blob Storage container: %w

What it means

ListObjects iterates the List Blobs Flat pager; any failure from pager.NextPage (non-2xx response, auth failure, network error) is wrapped as 'error listing blobs %s in Azure Blob Storage container'. Note only the first failing page aborts the loop.

Solutions

  1. Grant the credential 'Storage Blob Data Reader' role (or SAS with list permission) on the container.
  2. Verify the container name exists: az storage container list on the account.
  3. Check for 403 vs 404 vs 503 in the wrapped error and fix accordingly (permissions / name / throttling).
  4. If throttled, reduce listing pressure or retry with backoff; the SDK default retries may not be enough for sustained 503s.
  5. Confirm network path: test from the same pod with curl to https://<account>.blob.core.windows.net.
Defensive patterns

Strategy: retry

Validate before calling

// cheap pre-check that the container is listable
_, err := driver.ListObjects(ctx, &wfv1.Artifact{ArtifactLocation: wfv1.ArtifactLocation{
    Azure: &wfv1.AzureArtifact{Endpoint: ep, Container: c, Blob: probePrefix}}})
if err != nil { return err }

Try / catch

err := retry.Do(ctx, func() error {
    files, err = driver.ListObjects(ctx, artifact)
    var respErr *azcore.ResponseError
    if err != nil && errors.As(err, &respErr) && respErr.StatusCode == 403 {
        return retry.Fatal(err) // don't retry auth errors
    }
    return err
})

Prevention

When it happens

Trigger: Listing blobs under an artifact prefix when Azure returns 403 (no list permission), 404 (container missing), throttling (503) beyond retries, or a connection reset/DNS failure mid-pagination.

Common situations: Identity has only Reader (control plane) but not 'Storage Blob Data Reader' (data plane); container typo; private endpoint/firewall blocking egress; SAS token lacking 'l' (list) permission; storage account throttling large prefix listings.

Related errors


AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/d49c509abf2f4294. Report an issue: GitHub.

Appendix: source

Thrown at workflow/artifacts/azure/azure.go:435

	logger.WithField("endpoint", artifact.Azure.Endpoint).
		WithField("container", artifact.Azure.Container).
		WithField("blob", artifact.Azure.Blob).
		Info(ctx, "Listing blobs in Azure Blob Storage")

	containerClient, err := azblobDriver.newAzureContainerClient(ctx)
	if err != nil {
		return nil, fmt.Errorf("unable to create Azure Blob Container client: %w", err)
	}

	listOpts := azblob.ListBlobsFlatOptions{
		Prefix: &artifact.Azure.Blob,
		Marker: nil,
	}
	pager := containerClient.NewListBlobsFlatPager(&listOpts)
	for pager.More() {
		resp, err := pager.NextPage(ctx)
		if err != nil {
			return nil, fmt.Errorf("error listing blobs %s in Azure Blob Storage container: %w", artifact.Azure.Blob, err)
		}
		for _, v := range resp.Segment.BlobItems {
			files = append(files, *v.Name)
		}
	}
	return files, nil
}

// IsDirectory indicates whether or not the artifact represents a directory or a single file.
func (azblobDriver *ArtifactDriver) IsDirectory(ctx context.Context, artifact *wfv1.Artifact) (bool, error) {
	blobPrefix := artifact.Azure.Blob

	if blobPrefix == "" {
		return true, nil
	}
	if !strings.HasSuffix(blobPrefix, "/") {
		blobPrefix += "/"
	}

View on GitHub (pinned to 35bff19146)