apache/druid · critical · RE

Cannot create tempDir [%s] for google storage connector

Error message

Cannot create tempDir [%s] for google storage connector

What it means

GoogleStorageConnector's constructor requires a local scratch tempDir (from GoogleStorageDruidModule config) and calls FileUtils.mkdirp to create it. If creation fails (permissions, existing non-directory file, read-only FS), the constructor throws this RuntimeException, so the connector (and any job using it) cannot start.

Source

Thrown at extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/output/GoogleStorageConnector.java:74

  public GoogleStorageConnector(
      GoogleOutputConfig config,
      GoogleStorage googleStorage,
      GoogleInputDataConfig inputDataConfig
  )
  {
    this.storage = googleStorage;
    this.config = config;
    this.inputDataConfig = inputDataConfig;

    Preconditions.checkNotNull(config, "config is null");
    Preconditions.checkNotNull(config.getTempDir(), "tempDir is null in google config");

    try {
      FileUtils.mkdirp(config.getTempDir());
    }
    catch (IOException e) {
      throw new RE(
          e,
          StringUtils.format("Cannot create tempDir [%s] for google storage connector", config.getTempDir())
      );
    }
  }

  @Override
  public boolean pathExists(String path)
  {
    return storage.exists(config.getBucket(), objectPath(path));
  }

  @Override
  public OutputStream write(String path)
  {
    return storage.getObjectOutputStream(config.getBucket(), objectPath(path), config.getChunkSize().getBytesInInt());
  }

View on GitHub (pinned to 9b90983fd2)

Solutions

Set druid.google.output.tempDir (google config tempDir) to a writable path, e.g. a mounted volume like /opt/druid/var/tmp or /tmp.
Pre-create the directory manually with correct ownership: mkdir -p <dir> && chown druid:druid <dir>.
If the path exists as a file, remove it first.
For containers, mount an emptyDir/PV at the tempDir path and avoid read-only rootfs.
Run the process as a user that owns the configured directory.

Example fix

// before
druid.storage.google.tempDir=/var/druid/shared-tmp   // not writable in container
// after
druid.storage.google.tempDir=/tmp/google-connector-tmp
// (or mount a writable volume at the configured path)

Defensive patterns

Strategy: validation

Validate before calling

File tmp = config.getTempDir();
if (tmp.exists() && !tmp.isDirectory()) {
  throw new IllegalStateException("tempDir exists but is not a directory: " + tmp);
}
if (!tmp.canWrite() && !tmp.mkdirs() && !tmp.isDirectory()) {
  throw new IllegalStateException("tempDir not writable/creatable: " + tmp);
}

Try / catch

try { new GoogleStorageConnector(fs, config); }
catch (RE e) { alert("Fix druid.storage.google tempDir: " + e.getMessage()); }

Prevention

Configure a guaranteed-writable tempDir (mounted volume) before deploying.
Pre-create and chown the directory in the container image/entrypoint.
Avoid read-only root filesystems without an explicit writable mount.
Verify writability as the same user Druid runs as.

When it happens

Trigger: Constructing GoogleStorageConnector when config.getTempDir() cannot be created: parent dir not writable, path exists as a regular file, disk full, or container running as non-root on a read-only filesystem.

Common situations: Kubernetes containers with read-only root filesystems and default druid/tmp paths; running Druid as an unprivileged user after another user created the dir; docker-compose configs without volume mounts for the temp dir.

Understand the failure class

Background: mkdir permission denied (EACCES): failed to create directory errors explained — this error's family across 32 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/e4895dac3bc23d78. Report an issue: GitHub.