{"record":{"id":"3d5ca33fd0db4729","repo":"apache/hadoop","slug":"conditional-writes-unavailable","errorCode":null,"errorMessage":"Conditional Writes Unavailable","messagePattern":"Conditional Writes Unavailable","errorType":"exception","errorClass":"PathIOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java","lineNumber":2125,"sourceCode":"     This seems complicated, but comes down to\n     \"if explicitly requested and the FS enables it, use\".\n     */\n    // create file attributes\n    boolean cCreate = options.isConditionalOverwrite();\n    boolean cEtag = options.isConditionalOverwriteEtag();\n    boolean createPerf = options.isPerformance();\n    boolean overwrite = flags.contains(CreateFlag.OVERWRITE);\n\n    // path attributes\n    boolean magic = isUnderMagicCommitPath(path);\n\n    // store options\n    // is CC available.\n    boolean ccAvailable = conditionalCreateEnabled;\n\n    if (!ccAvailable && (cCreate || cEtag)) {\n      // fail fast if conditional creation is requested on an FS without it.\n      throw new PathIOException(path.toString(), \"Conditional Writes Unavailable\");\n    }\n\n    // probes to evaluate.\n    Set<StatusProbeEnum> probes = EnumSet.of(\n        StatusProbeEnum.List, StatusProbeEnum.Head);\n\n\n    // the PUT is conditional if requested, or if one of the\n    // this is a performance creation, overwrite has not been requested,\n    // this is not and etag write *and* conditional creation is available.\n    // write is NOT conditional etag write.\n    boolean conditionalPut = cCreate\n        || !(overwrite || cEtag) && ccAvailable && createPerf;\n\n    // skip the HEAD check for many reasons\n    // old: the path is magic, it's an overwrite or the \"create\" performance is set.\n    // new: also skip if any conditional create operation is in progress\n","sourceCodeStart":2107,"sourceCodeEnd":2143,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L2107-L2143","documentation":"innerCreateFile() fails fast when the caller requested conditional creation (options isConditionalOverwrite()/isConditionalOverwriteEtag(), driven by the fs.option.create.conditional.overwrite / .overwrite.etag create options) but the filesystem was initialized with fs.s3a.create.conditional.enabled=false - the conditionalCreateEnabled field read once in initialize(). PathIOException('Conditional Writes Unavailable') is raised before any probe or PUT, because proceeding would silently drop the no-clobber guarantee the caller asked for.","triggerScenarios":"Using createFile(...) with the conditional-overwrite (or etag) create option on a cluster whose core-site.xml sets fs.s3a.create.conditional.enabled=false; frameworks or committers that request conditional writes running against such a configuration.","commonSituations":"Conditional create disabled for compatibility with stores lacking If-None-Match support (older MinIO/Ceph/S3 clones) while application code assumes the default (true); rolling upgrades where new code meets old cluster config; enforcing overwrite=false atomically on S3.","solutions":["Set fs.s3a.create.conditional.enabled=true (the default) in the cluster or job configuration if the store supports conditional PUTs","Probe capability first: fs.hasPathCapability(path, \"fs.s3a.create.conditional.enabled\") and fall back to a plain create with overwrite=false when it returns false","For pure no-clobber semantics on stores without conditional writes, pre-check existence and use a unique destination name"],"exampleFix":"// before: assumes conditional writes exist on this FS\nFSDataOutputStreamBuilder b = fs.createFile(dst);\nb.opt(\"fs.option.create.conditional.overwrite\", true);\n\n// after\nif (fs.hasPathCapability(dst, \"fs.s3a.create.conditional.enabled\")) {\n  FSDataOutputStreamBuilder b = fs.createFile(dst);\n  b.opt(\"fs.option.create.conditional.overwrite\", true);\n  out = b.build();\n} else {\n  out = fs.create(dst, false); // probe-based no-overwrite\n}","handlingStrategy":"validation","validationCode":"// Probe the store capability before requesting conditional semantics\nboolean cc = fs.hasPathCapability(path, \"fs.s3a.create.conditional.enabled\");\nFSDataOutputStreamBuilder b = fs.createFile(path);\nif (cc) {\n  b.opt(\"fs.option.create.conditional.overwrite\", true);\n} else {\n  // fall back to probe-based create\n  out = fs.create(path, false);\n}","typeGuard":null,"tryCatchPattern":"Catch PathIOException from the create call and test whether the message is 'Conditional Writes Unavailable'; treat as configuration drift (enable fs.s3a.create.conditional.enabled or downgrade the create mode) - never as a transient error.","preventionTips":["Keep fs.s3a.create.conditional.enabled aligned with the application's create options across cluster configs","Assert the capability in integration tests on the target store","For third-party stores, verify If-None-Match PUT support before enabling conditional writes"],"tags":["s3a","conditional-writes","create","configuration","capability"],"backgroundTag":"store-capability-unsupported","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}