{"record":{"id":"6225fbaa9e2ffa21","repo":"apache/hadoop","slug":"component-invalid-aws-credentials-in-credentia","errorCode":null,"errorMessage":"{component}: Invalid AWS credentials in {credentials} required: {typeRequired}","messagePattern":"(.+?): Invalid AWS credentials in (.+?) required: (.+?)","errorType":"exception","errorClass":"NoAuthWithAWSException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/MarshalledCredentialBinding.java","lineNumber":168,"sourceCode":"   * SDK references out of that class, the logic is implemented here instead,\n   * @param marshalled marshalled credentials\n   * @param typeRequired type of credentials required\n   * @param component component name for exception messages.\n   * @return a new set of credentials\n   * @throws NoAuthWithAWSException validation failure\n   * @throws NoAwsCredentialsException the credentials are actually empty.\n   */\n  public static AwsCredentials toAWSCredentials(\n      final MarshalledCredentials marshalled,\n      final MarshalledCredentials.CredentialTypeRequired typeRequired,\n      final String component)\n      throws NoAuthWithAWSException, NoAwsCredentialsException {\n\n    if (marshalled.isEmpty()) {\n      throw new NoAwsCredentialsException(component, NO_AWS_CREDENTIALS);\n    }\n    if (!marshalled.isValid(typeRequired)) {\n      throw new NoAuthWithAWSException(component + \":\" +\n          marshalled.buildInvalidCredentialsError(typeRequired));\n    }\n    final String accessKey = marshalled.getAccessKey();\n    final String secretKey = marshalled.getSecretKey();\n    if (marshalled.hasSessionToken()) {\n      // a session token was supplied, so return session credentials\n      return AwsSessionCredentials.create(accessKey, secretKey,\n          marshalled.getSessionToken());\n    } else {\n      // these are full credentials\n      return AwsBasicCredentials.create(accessKey, secretKey);\n    }\n  }\n\n  /**\n   * Request a set of credentials from an STS endpoint.\n   * @param parentCredentials the parent credentials needed to talk to STS\n   * @param configuration AWS client configuration","sourceCodeStart":150,"sourceCodeEnd":186,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/MarshalledCredentialBinding.java#L150-L186","documentation":"Thrown by MarshalledCredentialBinding.toAWSCredentials when the marshalled static S3A credentials fail validation for the credential type the component requires. S3A converts the fs.s3a.access.key / fs.s3a.secret.key / fs.s3a.session.token triple into AWS SDK AwsBasicCredentials or AwsSessionCredentials; before doing so it checks that all fields the required type needs are present and non-empty. This is the connector's fail-fast guard against half-configured static credentials (note: a completely empty set throws NoAwsCredentialsException instead).","triggerScenarios":"A component (filesystem init, delegation-token binding) calls toAWSCredentials(marshalled, typeRequired, component) and marshalled.isValid(typeRequired) returns false. Concrete case: CredentialTypeRequired.Session demanded but fs.s3a.session.token is missing/blank, or fs.s3a.access.key / fs.s3a.secret.key is empty or contains surrounding whitespace.","commonSituations":"Hand-edited core-site.xml with a missing or typo'd secret key; switching between long-lived keys and STS session credentials and leaving one of the three fields behind; pasting keys with trailing whitespace/newlines; expecting S3A to read AWS_ACCESS_KEY_ID env vars (it does not for static keys); cluster-wide config differing from the job-level override.","solutions":["Set the complete triple for the required type: fs.s3a.access.key + fs.s3a.secret.key, plus fs.s3a.session.token when session credentials are required, all non-empty and trimmed","If you intend to authenticate via IAM instance roles or environment variables instead, delete the static keys and set fs.s3a.aws.credentials.provider to the matching provider (e.g. InstanceProfileCredentialsProvider or EnvironmentVariableCredentialsProvider)","Print the effective config on the exact node/executor that fails (conf.getTrimmed for each key) to find which override is incomplete","For STS-derived session credentials, re-generate them: expired sessions often get replaced by half-updated configs"],"exampleFix":"<!-- before: session type required but token missing -->\n<property><name>fs.s3a.access.key</name><value>AKIA...</value></property>\n<property><name>fs.s3a.secret.key</name><value>secret...</value></property>\n\n<!-- after: complete session triple, or drop all three and use fs.s3a.aws.credentials.provider instead -->\n<property><name>fs.s3a.access.key</name><value>AKIA...</value></property>\n<property><name>fs.s3a.secret.key</name><value>secret...</value></property>\n<property><name>fs.s3a.session.token</name><value>FwoGZXIv...</value></property>","handlingStrategy":"try-catch","validationCode":"import org.apache.hadoop.fs.s3a.auth.MarshalledCredentials;\nimport org.apache.hadoop.fs.s3a.auth.MarshalledCredentialBinding;\n\nMarshalledCredentials mc = MarshalledCredentials.fromConfiguration(conf);\nMarshalledCredentials.CredentialTypeRequired required =\n    MarshalledCredentials.CredentialTypeRequired.Session; // whatever your component needs\nif (!mc.isValid(required)) {\n  throw new IOException(\"Refusing to start: incomplete S3A credentials - \"\n      + mc.buildInvalidCredentialsError(required));\n}","typeGuard":null,"tryCatchPattern":"try {\n  FileSystem fs = FileSystem.get(new URI(\"s3a://bucket\"), conf);\n} catch (NoAuthWithAWSException | NoAwsCredentialsException e) {\n  // config problem, not transient: report which component/type failed and stop\n  throw new JobSetupException(\"S3A credentials rejected: \" + e.getMessage(), e);\n}","preventionTips":["Lint core-site.xml: all three credential keys present and non-empty when session credentials are used","Prefer IAM roles / credential providers over static keys in config","Trim credential values when injecting programmatically; whitespace fails validation","Validate with MarshalledCredentials.fromConfiguration(conf).isValid(...) at job submission time"],"tags":["aws","s3a","credentials","authentication","configuration"],"backgroundTag":"invalid-aws-credentials","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}