apache/hadoop · error · IOException

flushOrSync has error. bs : pre write obs

Error message

flushOrSync has error. bs : pre write obs[%s] has error.

What it means

OBSBlockOutputStream.flushOrSync() (backing hflush/hsync) first checks filesystem state, then hasException: if a previous operation failed, it throws IOException('flushOrSync has error. bs : pre write obs[<key>] has error.') and will not attempt the flush. Note the sibling behavior in the same method: on a non-POSIX (object) bucket it only warns 'not posix bucket, not support hflush or hsync' and buffers — so for real durability semantics you also need an fs (POSIX-enabled) bucket, but THIS error specifically means the stream was already poisoned by an earlier failure.

Solutions

1. Find the first exception for the key in the logs and fix the root cause (most often expired temporary AK/SK or network resets in long jobs).
2. Abort/close the stream and restart the write from the last known-good checkpoint — a stream that hit hasException cannot be recovered.
3. If durability semantics matter, ensure the bucket is a POSIX (fs) bucket via fs.obs.bucket type so hflush/hsync actually upload rather than only buffer.
4. If using temporary security tokens, schedule credential rotation shorter than job duration so the first failure never happens.

Example fix

// before
out.write(record);            // earlier async upload already failed
if (++n % 1000 == 0) out.hflush(); // -> 'flushOrSync has error'

// after
try {
  out.write(record);
  if (++n % 1000 == 0) out.hflush();
} catch (IOException e) {
  checkpoint.markFailed();   // trigger sink-level replay from checkpoint
  throw e;                   // let the framework restart the writer

if (streamFailed || fsClosed) {
  skipHflushAndFailFast(); // do not call hflush on a poisoned stream
}

try {
  out.hflush();
} catch (IOException e) {
  if (String.valueOf(e.getMessage()).contains("flushOrSync has error")) {
    triggerCheckpointReplay(); // stream unrecoverable
    throw e;
  }
  throw e;
}

When it happens

Trigger: hflush()/hsync() called after a failed write or block upload on the same stream; commit-time sync in mapreduce output committers touching a stream whose background part upload already failed; checkpoint writers (Flink/Kafka-connect style) syncing periodically and hitting the poisoned flag set by an earlier transient network error.

Common situations: Streaming sinks that call hflush every N seconds; exactly-once pipelines assuming hflush makes data visible after an earlier silent failure; mixed buckets where users also confuse the 'not posix bucket' warning with this error.

Related errors

• write has error. bs : pre upload obs
• Bucket does not exist
• Bucket cannot be deleted
• Can't make directory for path
• Cannot delete root path

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/63d57c66a230d54a. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSBlockOutputStream.java:537

    // hflush hsyn same
    flushOrSync();
  }

  /**
   * Flush local file or multipart to obs. focus: not posix bucket is not
   * support
   *
   * @throws IOException io exception
   */
  private synchronized void flushOrSync() throws IOException {

    checkOpen();
    if (hasException.get()) {
      String flushWarning = String.format(
          "flushOrSync has error. bs : pre write obs[%s] has error.",
          key);
      LOG.warn(flushWarning);
      throw new IOException(flushWarning);
    }
    if (fs.isFsBucket()) {
      // upload
      flushCurrentBlock();

      // clear
      clearHFlushOrSync();
    } else {
      LOG.warn("not posix bucket, not support hflush or hsync.");
      flush();
    }
  }

  /**
   * Clear for hflush or hsync.
   */
  private synchronized void clearHFlushOrSync() {
    appendAble.set(true);

View on GitHub (pinned to 2add963021)