apache/hadoop · error · IOException

Cannot create a file whose name looks like a directory: '%s'

Error message

Cannot create a file whose name looks like a directory: '%s'

What it means

GoogleCloudStorageFileSystem.create throws when StorageResourceId.fromUriPath(path, allowEmptyObjectName=true) yields a directory id — an empty object name, i.e. a path with a trailing '/'. In GCS, directories are naming conventions (prefixes and placeholder objects), so a file whose name 'looks like' a directory is rejected before any parent-conflict checks run.

Source

Thrown at hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorageFileSystem.java:113

    return new GoogleCloudStorage(configuration, credentials);
  }

  GoogleCloudStorageFileSystem(final GoogleHadoopFileSystemConfiguration configuration,
      final Credentials credentials) throws IOException {
    this.configuration = configuration;
    gcs = createCloudStorage(configuration, credentials);
  }

  WritableByteChannel create(final URI path, final CreateFileOptions createOptions)
      throws IOException {
    LOG.trace("create(path: {}, createOptions: {})", path, createOptions);
    checkNotNull(path, "path could not be null");
    StorageResourceId resourceId =
        StorageResourceId.fromUriPath(path, /* allowEmptyObjectName=*/ true);

    if (resourceId.isDirectory()) {
      throw new IOException(
          String.format("Cannot create a file whose name looks like a directory: '%s'",
              resourceId));
    }

    // Because create call should create parent directories too, before creating an actual file
    // we need to check if there are no conflicting items in the directory tree:
    // - if there are no conflicting files with the same name as any parent subdirectory
    // - if there are no conflicting directory with the name as a file
    //
    // For example, for a new `gs://bucket/c/d/f` file:
    // - files `gs://bucket/c` and `gs://bucket/c/d` should not exist
    // - directory `gs://bucket/c/d/f/` should not exist
    if (configuration.isEnsureNoConflictingItems()) {
      // Check if a directory with the same name exists.
      StorageResourceId dirId = resourceId.toDirectoryId();
      Boolean conflictingDirExist = false;
      if (createOptions.isEnsureNoDirectoryConflict()) {
        // TODO: Do this concurrently

View on GitHub (pinned to 2add963021)

Solutions

Normalize the path and strip trailing '/' from the object name before create.
Build URIs with path utilities (UriPaths) instead of string concatenation so files and directories cannot blur.
If a directory was intended, call mkdirs instead of create.

Example fix

// before
URI path = URI.create("gs://bucket/data/part-0000/");
gcsFs.create(path, CreateFileOptions.DEFAULT); // 'looks like a directory'

// after
String p = path.toString().replaceAll("/+$", "");
gcsFs.create(URI.create(p), CreateFileOptions.DEFAULT);

Defensive patterns

Strategy: validation

Validate before calling

static URI asFilePath(URI u) {
  String s = u.toString();
  checkArgument(!s.equals("gs://"), "need bucket + object");
  return URI.create(s.replaceAll("/+$", "")); // strip trailing '/' for file create
}
gcsFs.create(asFilePath(path), CreateFileOptions.DEFAULT);

Prevention

Build URIs with path utilities rather than string concatenation.
Normalize user-supplied filenames (strip trailing slashes and empty segments) at the API boundary.
Add a unit test asserting create() is never called with a path ending in '/'.

When it happens

Trigger: Calling create(gs://bucket/some/dir/, options) — any path whose object component is empty because of a trailing slash; URIs assembled by joining path segments that already end with '/'.

Common situations: String-built paths with duplicated or trailing separators; user-supplied filenames not normalized; code ported from local filesystems where a trailing slash is ignored; configuration values with accidental trailing '/'.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/73416bbd850dd4d3. Report an issue: GitHub.