{"record":{"id":"63d57c66a230d54a","repo":"apache/hadoop","slug":"flushorsync-has-error-bs-pre-write-obs-s-has","errorCode":null,"errorMessage":"flushOrSync has error. bs : pre write obs[%s] has error.","messagePattern":"flushOrSync has error\\. bs : pre write obs\\[(.+?)\\] has error\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSBlockOutputStream.java","lineNumber":537,"sourceCode":"    // hflush hsyn same\n    flushOrSync();\n  }\n\n  /**\n   * Flush local file or multipart to obs. focus: not posix bucket is not\n   * support\n   *\n   * @throws IOException io exception\n   */\n  private synchronized void flushOrSync() throws IOException {\n\n    checkOpen();\n    if (hasException.get()) {\n      String flushWarning = String.format(\n          \"flushOrSync has error. bs : pre write obs[%s] has error.\",\n          key);\n      LOG.warn(flushWarning);\n      throw new IOException(flushWarning);\n    }\n    if (fs.isFsBucket()) {\n      // upload\n      flushCurrentBlock();\n\n      // clear\n      clearHFlushOrSync();\n    } else {\n      LOG.warn(\"not posix bucket, not support hflush or hsync.\");\n      flush();\n    }\n  }\n\n  /**\n   * Clear for hflush or hsync.\n   */\n  private synchronized void clearHFlushOrSync() {\n    appendAble.set(true);","sourceCodeStart":519,"sourceCodeEnd":555,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSBlockOutputStream.java#L519-L555","documentation":"OBSBlockOutputStream.flushOrSync() (backing hflush/hsync) first checks filesystem state, then hasException: if a previous operation failed, it throws IOException('flushOrSync has error. bs : pre write obs[<key>] has error.') and will not attempt the flush. Note the sibling behavior in the same method: on a non-POSIX (object) bucket it only warns 'not posix bucket, not support hflush or hsync' and buffers — so for real durability semantics you also need an fs (POSIX-enabled) bucket, but THIS error specifically means the stream was already poisoned by an earlier failure.","triggerScenarios":"hflush()/hsync() called after a failed write or block upload on the same stream; commit-time sync in mapreduce output committers touching a stream whose background part upload already failed; checkpoint writers (Flink/Kafka-connect style) syncing periodically and hitting the poisoned flag set by an earlier transient network error.","commonSituations":"Streaming sinks that call hflush every N seconds; exactly-once pipelines assuming hflush makes data visible after an earlier silent failure; mixed buckets where users also confuse the 'not posix bucket' warning with this error.","solutions":["Find the first exception for the key in the logs and fix the root cause (most often expired temporary AK/SK or network resets in long jobs).","Abort/close the stream and restart the write from the last known-good checkpoint — a stream that hit hasException cannot be recovered.","If durability semantics matter, ensure the bucket is a POSIX (fs) bucket via fs.obs.bucket type so hflush/hsync actually upload rather than only buffer.","If using temporary security tokens, schedule credential rotation shorter than job duration so the first failure never happens."],"exampleFix":"// before\nout.write(record);            // earlier async upload already failed\nif (++n % 1000 == 0) out.hflush(); // -> 'flushOrSync has error'\n\n// after\ntry {\n  out.write(record);\n  if (++n % 1000 == 0) out.hflush();\n} catch (IOException e) {\n  checkpoint.markFailed();   // trigger sink-level replay from checkpoint\n  throw e;                   // let the framework restart the writer","handlingStrategy":"try-catch","validationCode":"if (streamFailed || fsClosed) {\n  skipHflushAndFailFast(); // do not call hflush on a poisoned stream\n}","typeGuard":null,"tryCatchPattern":"try {\n  out.hflush();\n} catch (IOException e) {\n  if (String.valueOf(e.getMessage()).contains(\"flushOrSync has error\")) {\n    triggerCheckpointReplay(); // stream unrecoverable\n    throw e;\n  }\n  throw e;\n}","preventionTips":["Use fs (POSIX) buckets when hflush/hsync durability is required.","Surface the FIRST stream exception to checkpointing logic so syncs never run on a poisoned stream.","Schedule security-token refresh inside job lifetime; expired tokens are the top root cause."],"tags":["obs","huaweicloud","hflush","hsync","fail-fast"],"backgroundTag":"broken-output-stream","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}