apache/hadoop · error · UnsupportedOperationException

hflush not supported by {out}

Error message

hflush not supported by {out}

What it means

BufferedIOStatisticsOutputStream wraps an inner OutputStream to add buffering and IOStatistics collection. Its hflush() only works when the inner stream implements org.apache.hadoop.fs.Syncable: the buffer is flushed and the call forwarded. With a non-Syncable inner stream and downgradeSyncable=false (a constructor flag) it throws UnsupportedOperationException('hflush not supported by <out>').

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/statistics/BufferedIOStatisticsOutputStream.java:125

  /**
   * If the inner stream is Syncable, flush the buffer and then
   * invoke the inner stream's hflush() operation.
   *
   * Otherwise: throw an exception, unless the stream was constructed with
   * {@link #downgradeSyncable} set to true, in which case the stream
   * is just flushed.
   * @throws IOException IO Problem
   * @throws UnsupportedOperationException if the inner class is not syncable
   */
  @Override
  public void hflush() throws IOException {
    if (out instanceof Syncable) {
      flush();
      ((Syncable) out).hflush();
    } else {
      if (!downgradeSyncable) {
        throw new UnsupportedOperationException("hflush not supported by "
            + out);
      } else {
        flush();
      }
    }
  }

  /**
   * If the inner stream is Syncable, flush the buffer and then
   * invoke the inner stream's hsync() operation.
   *
   * Otherwise: throw an exception, unless the stream was constructed with
   * {@link #downgradeSyncable} set to true, in which case the stream
   * is just flushed.
   * @throws IOException IO Problem
   * @throws UnsupportedOperationException if the inner class is not syncable
   */
  @Override

View on GitHub (pinned to 2add963021)

Solutions

1. Construct with downgradeSyncable=true when the durability guarantee is not required — the call degrades to flush(), matching FSDataOutputStream behavior
2. Make the inner stream implement Syncable (e.g. wrap it in FSDataOutputStream)
3. Probe 'out instanceof Syncable' before calling hflush()

Example fix

// before
BufferedIOStatisticsOutputStream out =
    new BufferedIOStatisticsOutputStream(inner, false);
out.hflush(); // UnsupportedOperationException when inner is not Syncable

// after
BufferedIOStatisticsOutputStream out =
    new BufferedIOStatisticsOutputStream(inner, true);
out.hflush(); // degrades to flush()

if (out instanceof Syncable) {
  statsStream.hflush();
} else {
  statsStream.flush(); // or construct the wrapper with downgradeSyncable=true
}

static boolean supportsHflush(OutputStream out) {
  return out instanceof org.apache.hadoop.fs.Syncable;
}

If you cannot control construction, catch UnsupportedOperationException from hflush() and degrade to flush() — but prefer the downgradeSyncable=true constructor flag so the behavior is explicit rather than exception-driven.

When it happens

Trigger: Constructing the wrapper around a stream that does not implement Syncable (compression codec streams, CipherOutputStream, plain in-memory streams, some test mocks) with downgradeSyncable=false, then calling hflush().

Common situations: Adding statistics instrumentation on top of encryption or compression layers; unit tests wrapping mock streams; pipelines where the bottom stream is not an FSDataOutputStream.

Related errors

• hsync not supported by {out}
• write (b[{b.length}], {off}, {len})
• catch exception when allocating BosBlockBuffer:
• Stream closed
• %s: Stream is closed!

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d57a2fcd20aad7f5. Report an issue: GitHub.