argoproj/argo-workflows · error

plugin %s get capabilities failed: %w

Error message

plugin %s get capabilities failed: %w

What it means

The artifact plugin driver calls the plugin's GetCapabilities RPC (via supportsSaveStream) before SaveStream to check whether the plugin advertises streaming upload support. If the RPC fails with any gRPC status other than Unimplemented (which is deliberately mapped to a silent fallback to buffered Save for old plugins), the driver wraps the error as "plugin %s get capabilities failed" and aborts. This is a transport- or server-side failure of the capability probe, not the artifact upload itself.

Source

Thrown at workflow/artifacts/plugin/plugin.go:313

	}
	if !resp.Success {
		return fmt.Errorf("plugin %s save stream failed: %s", d.pluginName, resp.Error)
	}
	return nil
}

// supportsSaveStream reports whether the plugin advertises streaming support.
// A plugin that predates GetCapabilities returns codes.Unimplemented, which maps to
// (false, nil) so SaveStream falls back to the buffered Save. Any other error is
// returned rather than silently downgrading to buffering a potentially large artifact
// to disk before the real failure would resurface via Save.
func (d *Driver) supportsSaveStream(ctx context.Context) (bool, error) {
	resp, err := d.client.GetCapabilities(ctx, &artifact.GetCapabilitiesRequest{})
	if err != nil {
		if status.Code(err) == codes.Unimplemented {
			return false, nil
		}
		return false, fmt.Errorf("plugin %s get capabilities failed: %w", d.pluginName, err)
	}
	return resp.GetSupportsSaveStream(), nil
}

// saveStreamViaTempFile is the fallback used when the plugin doesn't implement
// streaming SaveStream: buffer to a temp file and call the existing unary Save.
func (d *Driver) saveStreamViaTempFile(ctx context.Context, reader io.Reader, outputArtifact *wfv1.Artifact) error {
	return common.SaveStreamViaTempFile(reader, "plugin-upload-*", func(path string) error {
		return d.Save(ctx, path, outputArtifact)
	})
}

// Delete implements ArtifactDriver.Delete by calling the plugin service
func (d *Driver) Delete(ctx context.Context, artifactRef *wfv1.Artifact) error {
	grpcArtifact := convertToGRPC(artifactRef)
	resp, err := d.client.Delete(ctx, &artifact.DeleteArtifactRequest{
		Artifact: grpcArtifact,
	})

View on GitHub (pinned to 35bff19146)

Solutions

1. Check the plugin container/pod is running and its logs for a crash (kubectl logs on the plugin sidecar or plugin deployment), then restart it.
2. Verify the plugin's gRPC address/socket configuration so the client can connect (correct port, unix socket path, service name).
3. Inspect the wrapped error (%w) for the underlying gRPC status code and address it specifically (e.g. Unavailable = connectivity, DeadlineExceeded = increase timeout/ctx).
4. If the plugin is an older build that should fall back to buffered Save, ensure it returns codes.Unimplemented for GetCapabilities rather than another error (rebuild/upgrade the plugin against the current artifact plugin API).
5. Retry the workflow; transient Unavailable errors during plugin rollout resolve once the plugin is healthy.

Example fix

// plugin predating GetCapabilities but crashing with a generic error
// before: plugin's GetCapabilities returns codes.Internal from a panic
// after: implement GetCapabilities correctly, or if intentionally unsupported:
func (s *pluginServer) GetCapabilities(ctx context.Context, req *artifact.GetCapabilitiesRequest) (*artifact.GetCapabilitiesResponse, error) {
	return &artifact.GetCapabilitiesResponse{SupportsSaveStream: false}, nil // never return a non-Unimplemented error for 'unsupported'
}

// probe plugin health before streaming
conn, err := grpc.Dial(pluginAddr, grpc.WithTransportCredentials(insecure.NewCredentials()))
if err != nil { return fmt.Errorf("plugin unreachable: %w", err) }
client := artifact.NewArtifactServiceClient(conn)
_, err = client.GetCapabilities(ctx, &artifact.GetCapabilitiesRequest{})
if err != nil && status.Code(err) != codes.Unimplemented {
	return fmt.Errorf("plugin not ready: %v (code=%s)", err, status.Code(err))
}

func isPluginConnectivityErr(err error) bool {
	code := status.Code(err)
	return code == codes.Unavailable || code == codes.Unimplemented || code == codes.DeadlineExceeded
}

ok, err := d.supportsSaveStream(ctx)
if err != nil {
	if isPluginConnectivityErr(err) {
		// fall back to buffered Save via temp file, or retry after plugin restart
		return d.saveStreamViaTempFile(ctx, reader, outputArtifact)
	}
	return err
}

When it happens

Trigger: Calling Driver.SaveStream when the plugin's GetCapabilities RPC returns a non-Unimplemented gRPC error: codes.Unavailable (plugin container not running/ready), codes.DeadlineExceeded (ctx cancelled or plugin too slow), codes.Internal, codes.PermissionDenied, or a connection failure (Uninstantiated: connection refused / no transport).

Common situations: Plugin sidecar crashed or was OOM-killed right before the upload; plugin image predates streaming but fails the call for another reason; plugin socket/address misconfigured so the gRPC client can't connect; context deadline too short for a slow plugin startup; plugin returned PermissionDenied due to RBAC/auth changes after a version upgrade.

Related errors

• plugin %s stream error: %s
• plugin %s save failed: %w
• plugin %s save stream failed to open: %w
• plugin %s save stream failed %s: %w
• plugin %s save stream failed: %w

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/8c736266582129e0. Report an issue: GitHub.