apache/hadoop · error · IllegalArgumentException

Missing keyfile property ('%s') for authentication type '%s'

Error message

Missing keyfile property ('%s') for authentication type '%s'

What it means

HadoopCredentialsConfiguration.getCredentials: when the authentication type property is SERVICE_ACCOUNT_JSON_KEYFILE, the keyfile property (key suffix .auth.service.account.json.keyfile, resolved under configured prefixes such as fs.gs and the google.cloud base prefix - e.g. fs.gs.auth.service.account.json.keyfile) must be non-empty. If it is null/empty, IllegalArgumentException names the exact missing key.

Source

Thrown at hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/HadoopCredentialsConfiguration.java:142

    GoogleCredentials credentials = getCredentialsInternal(config, keyPrefixes);
    return credentials == null ? null : configureCredentials(config, keyPrefixes, credentials);
  }

  private static GoogleCredentials getCredentialsInternal(
          Configuration config, List<String> keyPrefixes) throws IOException {
    AuthenticationType authenticationType =
            AUTHENTICATION_TYPE_SUFFIX.withPrefixes(keyPrefixes).get(config, config::getEnum);
    switch (authenticationType) {
    case APPLICATION_DEFAULT:
      return GoogleCredentials.getApplicationDefault();
    case COMPUTE_ENGINE:
      return ComputeEngineCredentials.newBuilder().build();
    case SERVICE_ACCOUNT_JSON_KEYFILE:
      String keyFile = SERVICE_ACCOUNT_JSON_KEYFILE_SUFFIX
              .withPrefixes(keyPrefixes).get(config, config::get);

      if (Strings.isNullOrEmpty(keyFile)) {
        throw new IllegalArgumentException(String.format(
                "Missing keyfile property ('%s') for authentication type '%s'",
                SERVICE_ACCOUNT_JSON_KEYFILE_SUFFIX.getKey(),
                authenticationType));
      }

      try (FileInputStream fis = new FileInputStream(keyFile)) {
        return ServiceAccountCredentials.fromStream(fis);
      }
    case USER_CREDENTIALS:
      String clientId = AUTH_CLIENT_ID_SUFFIX.withPrefixes(keyPrefixes).get(config, config::get);
      RedactedString clientSecret =
              AUTH_CLIENT_SECRET_SUFFIX.withPrefixes(keyPrefixes).getPassword(config);
      RedactedString refreshToken =
              AUTH_REFRESH_TOKEN_SUFFIX.withPrefixes(keyPrefixes).getPassword(config);

      return UserCredentials.newBuilder()
              .setClientId(clientId)
              .setClientSecret(clientSecret.getValue())

View on GitHub (pinned to 2add963021)

Solutions

Set fs.gs.auth.service.account.json.keyfile (and/or google.cloud.auth.service.account.json.keyfile) to the absolute path of the service-account JSON key in core-site.xml or the job Configuration.
Verify the path is readable on every node that instantiates the FileSystem, not just the client.
If no keyfile is intended, switch the auth type back (e.g. APPLICATION_DEFAULT or COMPUTE_ENGINE).
Check for typos/prefix mismatches against the exact key name printed in the exception message.

Example fix

<!-- before (core-site.xml) -->
<property><name>fs.gs.auth.type</name><value>SERVICE_ACCOUNT_JSON_KEYFILE</value></property>
<!-- keyfile property missing -> IllegalArgumentException -->

<!-- after -->
<property><name>fs.gs.auth.type</name><value>SERVICE_ACCOUNT_JSON_KEYFILE</value></property>
<property>
  <name>fs.gs.auth.service.account.json.keyfile</name>
  <value>/etc/hadoop/conf/gcs-service-account.json</value>
</property>

Defensive patterns

Strategy: validation

Validate before calling

String type = conf.get("fs.gs.auth.type", "google.cloud.auth.type" /*fallback*/);
if ("SERVICE_ACCOUNT_JSON_KEYFILE".equals(conf.get("fs.gs.auth.type"))) {
  String keyfile = conf.get("fs.gs.auth.service.account.json.keyfile",
      conf.get("google.cloud.auth.service.account.json.keyfile"));
  if (keyfile == null || keyfile.isEmpty()) {
    throw new IllegalArgumentException("Set fs.gs.auth.service.account.json.keyfile");
  }
}

Try / catch

catch IllegalArgumentException with message startsWith("Missing keyfile property") - the message names the exact key to set; add it to core-site.xml / job conf and re-initialize the FileSystem.

Prevention

Whenever switching fs.gs.auth.type to SERVICE_ACCOUNT_JSON_KEYFILE, set the keyfile property in the same change.
Distribute auth properties to every node's config, not only the client.
Use config linting or startup validation for required auth property pairs.

When it happens

Trigger: Setting fs.gs.auth.type (or google.cloud.auth.type) to SERVICE_ACCOUNT_JSON_KEYFILE without setting fs.gs.auth.service.account.json.keyfile / google.cloud.auth.service.account.json.keyfile; setting the keyfile under the wrong prefix so resolution falls through to empty.

Common situations: Switching a cluster from default COMPUTE_ENGINE auth (the built-in default) to keyfile auth but only editing one property; typos in the long property name; keyfile path configured on the client but missing from NodeManager/DataNode configs; CI using stripped-down core-site.xml.

Understand the failure class

Authentication and authorization failures — expired tokens, bad credentials, and missing scopes.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/720ff86e8fbc561e. Report an issue: GitHub.