apache/hadoop · error · IOException

flushOrSync has error. bs : pre write obs[%s] has error.

Error message

flushOrSync has error. bs : pre write obs[%s] has error.

What it means

OBSBlockOutputStream.flushOrSync() (backing hflush/hsync) first checks filesystem state, then hasException: if a previous operation failed, it throws IOException('flushOrSync has error. bs : pre write obs[<key>] has error.') and will not attempt the flush. Note the sibling behavior in the same method: on a non-POSIX (object) bucket it only warns 'not posix bucket, not support hflush or hsync' and buffers — so for real durability semantics you also need an fs (POSIX-enabled) bucket, but THIS error specifically means the stream was already poisoned by an earlier failure.

Source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSBlockOutputStream.java:537

    // hflush hsyn same
    flushOrSync();
  }

  /**
   * Flush local file or multipart to obs. focus: not posix bucket is not
   * support
   *
   * @throws IOException io exception
   */
  private synchronized void flushOrSync() throws IOException {

    checkOpen();
    if (hasException.get()) {
      String flushWarning = String.format(
          "flushOrSync has error. bs : pre write obs[%s] has error.",
          key);
      LOG.warn(flushWarning);
      throw new IOException(flushWarning);
    }
    if (fs.isFsBucket()) {
      // upload
      flushCurrentBlock();

      // clear
      clearHFlushOrSync();
    } else {
      LOG.warn("not posix bucket, not support hflush or hsync.");
      flush();
    }
  }

  /**
   * Clear for hflush or hsync.
   */
  private synchronized void clearHFlushOrSync() {
    appendAble.set(true);

View on GitHub (pinned to 2add963021)

Solutions

1. Find the first exception for the key in the logs and fix the root cause (most often expired temporary AK/SK or network resets in long jobs).
2. Abort/close the stream and restart the write from the last known-good checkpoint — a stream that hit hasException cannot be recovered.
3. If durability semantics matter, ensure the bucket is a POSIX (fs) bucket via fs.obs.bucket type so hflush/hsync actually upload rather than only buffer.
4. If using temporary security tokens, schedule credential rotation shorter than job duration so the first failure never happens.

Example fix

// before
out.write(record);            // earlier async upload already failed
if (++n % 1000 == 0) out.hflush(); // -> 'flushOrSync has error'

// after
try {
  out.write(record);
  if (++n % 1000 == 0) out.hflush();
} catch (IOException e) {
  checkpoint.markFailed();   // trigger sink-level replay from checkpoint
  throw e;                   // let the framework restart the writer

if (streamFailed || fsClosed) {
  skipHflushAndFailFast(); // do not call hflush on a poisoned stream
}

try {
  out.hflush();
} catch (IOException e) {
  if (String.valueOf(e.getMessage()).contains("flushOrSync has error")) {
    triggerCheckpointReplay(); // stream unrecoverable
    throw e;
  }
  throw e;
}

When it happens

Trigger: hflush()/hsync() called after a failed write or block upload on the same stream; commit-time sync in mapreduce output committers touching a stream whose background part upload already failed; checkpoint writers (Flink/Kafka-connect style) syncing periodically and hitting the poisoned flag set by an earlier transient network error.

Common situations: Streaming sinks that call hflush every N seconds; exactly-once pipelines assuming hflush makes data visible after an earlier silent failure; mixed buckets where users also confuse the 'not posix bucket' warning with this error.

Related errors

• write has error. bs : pre upload obs[%s] has error.
• Proxy error: %s or %s set without the other.
• From option %s %s
• Filesystem %s closed
• closed has error. bs : pre write obs[%s] has error.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/63d57c66a230d54a. Report an issue: GitHub.