{"record":{"id":"0106de73a6063e46","repo":"apache/hadoop","slug":"filesystem-is-configured-to-use-unknown-s3guard-st","errorCode":null,"errorMessage":"Filesystem is configured to use unknown S3Guard store \" + classname + \" origin \" + origin","messagePattern":"Filesystem is configured to use unknown S3Guard store \" \\+ classname \\+ \" origin \" \\+ origin","errorType":"exception","errorClass":"PathIOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/S3Guard.java","lineNumber":115,"sourceCode":"      LOG.warn(\"Ignoring S3Guard store option of {} -no longer needed or supported. \"\n              + \"Origin {}\",\n          S3GUARD_METASTORE_LOCAL, origin);\n      break;\n    case S3GUARD_METASTORE_DYNAMO:\n      // this is the dangerous one, as it is a sign that a config is in use where\n      // older releases will use DDB for listing metadata, yet this\n      // client will not update it.\n      final String message = String.format(\"S3Guard is no longer needed/supported,\"\n              + \" yet %s is configured to use DynamoDB as the S3Guard metadata store.\"\n              + \" This is no longer needed or supported. \" +\n              \"Origin of setting is %s\",\n          fsPath, origin);\n      LOG.error(message);\n      throw new PathIOException(fsPath, message);\n\n    default:\n      // an unknown store entirely.\n      throw new PathIOException(fsPath,\n          \"Filesystem is configured to use unknown S3Guard store \" + classname\n              + \" origin \" + origin);\n    }\n\n    // an option was set, but it was harmless\n    return true;\n  }\n\n\n  /**\n   * Get the authoritative paths of a filesystem.\n   *\n   * @param uri FS URI\n   * @param conf configuration\n   * @param qualifyToDir a qualification operation\n   * @return list of URIs valid for this FS.\n   */\n  @VisibleForTesting","sourceCodeStart":97,"sourceCodeEnd":133,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/S3Guard.java#L97-L133","documentation":"Companion to the DynamoDB rejection in S3Guard.checkNoS3Guard(): when fs.s3a.metadatastore.impl is set to any value that is neither empty, the NullMetadataStore, the LocalMetadataStore, nor DynamoDBMetadataStore, S3A initialization throws PathIOException('unknown S3Guard store <classname> origin <source>'). Post-3.4 S3A recognizes no metadata store implementations at all, so a custom/typo'd class name is fatal.","triggerScenarios":"fs.s3a.metadatastore.impl set to a typo, a custom third-party MetadataStore class (e.g. from an old internal fork), or a class name copied from S3Guard-era docs; initialization of any s3a:// filesystem then fails before the bucket is touched.","commonSituations":"Leftover or hand-edited S3Guard config after upgrading to Hadoop 3.4+; forks that once shipped custom metadata stores; property inheritance from a parent cluster's core-site.xml (origin string in the message points at the file).","solutions":["Unset fs.s3a.metadatastore.impl entirely (S3Guard is removed; no metadata store is used).","Use the 'origin' in the message to find and edit the exact config file (or programmatic source) supplying the bad class name.","Remove remaining fs.s3a.s3guard.* / fs.s3a.metadatastore.* options to avoid the DynamoDB sibling failure.","Sweep configs cluster-wide: grep -r 'metadatastore' across core-site.xml and job submission templates."],"exampleFix":"<!-- before -->\n<property>\n  <name>fs.s3a.metadatastore.impl</name>\n  <value>com.myco.s3.MyMetadataStore</value>\n</property>\n\n<!-- after: property removed; no metadata store on Hadoop 3.4+ -->","handlingStrategy":"validation","validationCode":"String store = conf.getTrimmed(\"fs.s3a.metadatastore.impl\", \"\");\nif (!store.isEmpty()) {\n  // only Null/Local/DynamoDB names are even recognized (all S3Guard-era);\n  // on 3.4+ just unset everything\n  conf.unset(\"fs.s3a.metadatastore.impl\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  FileSystem fs = path.getFileSystem(conf);\n} catch (PathIOException e) {\n  if (e.getMessage().contains(\"unknown S3Guard store\")) {\n    // remove fs.s3a.metadatastore.impl from the origin file, retry\n  } else { throw e; }\n}","preventionTips":["Remove every fs.s3a.metadatastore.* and fs.s3a.s3guard.* property before Hadoop 3.4+.","Check conf.getPropertySources to find inherited settings from parent configs.","Add a startup config lint that rejects S3Guard keys on 3.4+ deployments."],"tags":["hadoop-aws","s3guard","configuration","hadoop-3-4-upgrade","unknown-class"],"backgroundTag":"removed-feature-config-error","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}