apache/beam · error · IOException

Copying blobs requires that a SAS token, connection string…

Error message

Copying blobs requires that a SAS token, connection string, or account key be provided.

What it means

generateSasToken() builds a legacy CloudStorageAccount SAS signature that requires one of: a SAS token, a connection string, or an account access key. When none of these credentials are configured in AzureOptions (e.g. only an OAuth access token is set), it throws this IOException. Server-side blob copy via SAS needs a writable credential to sign the shared access policy.

Solutions

1. Provide the account key: options.setAccessKey("...") (or via --azureAccessKey) before running copy/rename.
2. Provide a SAS token with read/write/delete permissions via setSasToken().
3. Provide a connection string if your setup uses one.
4. If you must stay on OAuth tokens, implement copy manually (download then upload) instead of relying on SAS-based copy.

Example fix

// before
AzureOptions options = PipelineOptionsFactory.as(AzureOptions.class);
options.setAzureAccessToken(oauthToken); // copy() will throw
// after
AzureOptions options = PipelineOptionsFactory.as(AzureOptions.class);
options.setAccessKey(accountKey); // enables SAS generation for copy/rename

AzureOptions options = ...;
if (options.getAccessKey() == null && options.getSasToken() == null /* and no connection string */) {
  throw new IllegalStateException("copy/rename on azfs requires accessKey or sasToken");
}

try {
  FileSystems.rename(srcIds, dstIds);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("SAS token")) {
    // reconfigure credentials (accessKey/sasToken) before retrying
  } else { throw e; }
}

When it happens

Trigger: Calling FileSystems.copy()/rename() on azfs paths while AzureOptions has only setAzureAccessToken() (or no credential at all) instead of setAccessKey(), setSasToken(), or a connection string.

Common situations: Pipelines authenticated with Azure AD/OAuth tokens (common on Dataflow) attempting blob rename/move; users unaware that copy requires account-key-class credentials rather than bearer tokens; credentials omitted entirely in the worker environment.

Related errors

• Failed to get application default credential.
• The copy source does not exist.
• Unable to get application default credentials. Please see…
• Unable to obtain credential
• API Key is required for writing events.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a75267d565057da8. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/azure/src/main/java/org/apache/beam/sdk/io/azure/blobstore/AzureBlobStoreFileSystem.java:382

    sharedAccessAccountPolicy.setPermissionsFromString(DEFAULT_PERMISSIONS);
    sharedAccessAccountPolicy.setSharedAccessStartTime(new Date(date));
    sharedAccessAccountPolicy.setSharedAccessExpiryTime(new Date(expiryDate));
    sharedAccessAccountPolicy.setResourceTypeFromString(DEFAULT_RESOURCE_TYPES);
    sharedAccessAccountPolicy.setServiceFromString(DEFAULT_SERVICES);

    String storageConnectionString;
    if (!Strings.isNullOrEmpty(options.getAzureConnectionString())) {
      storageConnectionString = options.getAzureConnectionString();
    } else if (!Strings.isNullOrEmpty(options.getAccessKey())) {
      storageConnectionString =
          "DefaultEndpointsProtocol=https;AccountName="
              + client.get().getAccountName()
              + ";AccountKey="
              + options.getAccessKey()
              + ";EndpointSuffix=core.windows.net";
    } else {
      throw new IOException(
          "Copying blobs requires that a SAS token, connection string, or account key be provided.");
    }

    try {
      CloudStorageAccount storageAccount = CloudStorageAccount.parse(storageConnectionString);
      return "?" + storageAccount.generateSharedAccessSignature(sharedAccessAccountPolicy);
    } catch (Exception e) {
      throw (IOException) e.getCause();
    }
  }

  @Override
  protected void rename(
      List<AzfsResourceId> srcResourceIds,
      List<AzfsResourceId> destResourceIds,
      MoveOptions... moveOptions)
      throws IOException {
    if (moveOptions.length > 0) {

View on GitHub (pinned to 12126d8942)