apache/beam · error · IOException

Could not create a temporary directory for storing…

Error message

Could not create a temporary directory for storing credentials: {credentialsDir.getAbsolutePath()}

What it means

TransformServiceLauncher's constructor prepares a directory to hold Google Application Default Credentials for the transform service. When the chosen directory does not exist and mkdir() fails (permissions, existing file at path, full disk), it throws an IOException with this message including the absolute path.

Solutions

1. Check/write-enable the parent directory or choose a writable credentialsDir.
2. Remove any regular file occupying the credentialsDir path.
3. Set a writable temp dir (e.g. TMPDIR=/tmp) when running in locked-down containers.
4. Verify available disk space and that the process user has write permission.

Example fix

// before
export TMPDIR=/var/ro-tmp
// after
export TMPDIR=/tmp && rm -f /tmp/transform-service-creds # if a stale file exists

File dir = new File(base, "creds"); if (!dir.exists() && !(dir.mkdirs() || dir.isDirectory())) { throw new IOException("cannot prepare credentials dir: " + dir); }

try { new TransformServiceLauncher(...); } catch (IOException e) { if (e.getMessage().contains("temporary directory")) { /* fall back to an explicitly writable dir */ } throw e; }

When it happens

Trigger: Constructing TransformServiceLauncher when credentialsDir cannot be created: parent directory not writable, path already exists as a regular file, or filesystem is read-only/full.

Common situations: Running in a read-only container filesystem, TMPDIR pointing to a non-writable location, a stale file where the directory should be, and restricted service accounts without write access to the configured dir.

Understand the failure class

Background: mkdir permission denied (EACCES): failed to create directory errors explained — this error's family across 32 libraries.

Related errors

• Could not create a temporary directory for storing…
• Could not create the new requirements file
• Error matching file spec
• Error when finding
• Failed to create directory:

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5c719666d9480671. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/transform-service/launcher/src/main/java/org/apache/beam/sdk/transformservice/launcher/TransformServiceLauncher.java:98

    try (FileOutputStream fout = new FileOutputStream(dockerComposeFile)) {
      ByteStreams.copy(getClass().getResourceAsStream("/docker-compose.yml"), fout);
    }

    File envFile = Paths.get(tmpDir, ".env").toFile();
    try (FileOutputStream fout = new FileOutputStream(envFile)) {
      ByteStreams.copy(getClass().getResourceAsStream("/.env"), fout);
    }

    // Setting up the credentials directory.
    File credentialsDir = Paths.get(tmpDir, "credentials_dir").toFile();
    if (credentialsDir.exists()) {
      LOG.info("Reusing the existing credentials directory {}", credentialsDir.getAbsolutePath());
    } else {
      LOG.info(
          "Creating a temporary directory for storing credentials: {}",
          credentialsDir.getAbsolutePath());
      if (!credentialsDir.mkdir()) {
        throw new IOException(
            "Could not create a temporary directory for storing credentials: "
                + credentialsDir.getAbsolutePath());
      }

      LOG.info("Copying the Google Application Default Credentials file.");

      File applicationDefaultCredentialsFileCopied =
          Paths.get(credentialsDir.getAbsolutePath(), "application_default_credentials.json")
              .toFile();

      boolean isWindows =
          System.getProperty("os.name").toLowerCase(Locale.ENGLISH).contains("windows");
      String applicationDefaultFilePathSuffix =
          isWindows
              ? "\\gcloud\\application_default_credentials.json"
              : "/.config/gcloud/application_default_credentials.json";
      String applicationDefaultFilePath =
          System.getProperty("user.home") + applicationDefaultFilePathSuffix;

View on GitHub (pinned to 12126d8942)