apache/hadoop · error
ENOTSUP
ENOTSUP
Error message
ERROR: cannot open an hdfs file in O_RDWR mode
What it means
hdfsOpenFile rejects accmode O_RDWR with ENOTSUP before any JNI call: HDFS files are write-once - a stream is either an FSDataInputStream (O_RDONLY) or an FSDataOutputStream (O_WRONLY/O_APPEND) - so read-write random-access opens are impossible in the storage model, not merely unimplemented by the client.
Source
Thrown at hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/hdfs.c:1138
jint jBufferSize = bufferSize;
jshort jReplication = replication;
/* The hadoop java api/signature */
const char *method = NULL;
const char *signature = NULL;
/* Get the JNIEnv* corresponding to current thread */
JNIEnv* env = getJNIEnv();
if (env == NULL) {
errno = EINTERNAL;
return NULL;
}
if (accmode == O_RDONLY || accmode == O_WRONLY) {
/* yay */
} else if (accmode == O_RDWR) {
fprintf(stderr, "ERROR: cannot open an hdfs file in O_RDWR mode\n");
errno = ENOTSUP;
return NULL;
} else {
fprintf(stderr, "ERROR: cannot open an hdfs file in mode 0x%x\n",
accmode);
errno = EINVAL;
return NULL;
}
if ((flags & O_CREAT) && (flags & O_EXCL)) {
fprintf(stderr,
"WARN: hdfs does not truly support O_CREATE && O_EXCL\n");
}
if (accmode == O_RDONLY) {
method = "open";
signature = JMETHOD2(JPARAM(HADOOP_PATH), "I", JPARAM(HADOOP_FSDISTRM));
} else if (flags & O_APPEND) {View on GitHub (pinned to 2add963021)
Solutions
- Split into phases: write the file with O_WRONLY (optionally O_CREAT/O_APPEND), hdfsCloseFile, then reopen with O_RDONLY to read back.
- For update-in-place semantics, rewrite to write-a-new-file + atomic rename (hdfsRename), or use HBase/Kudu instead of raw HDFS files.
- Audit ported code for O_RDWR early - it will never work, so fail fast with your own error before calling libhdfs.
Example fix
// before - ENOTSUP: HDFS files are either input or output, never both hdfsFile f = hdfsOpenFile(fs, "/x", O_RDWR | O_CREAT, 0, 0, 0); // after - write, close, reopen read-only hdfsFile out = hdfsOpenFile(fs, "/x", O_WRONLY | O_CREAT, 0, 0, 0); hdfsWrite(fs, out, buf, n); hdfsCloseFile(fs, out); hdfsFile in = hdfsOpenFile(fs, "/x", O_RDONLY, 0, 0, 0);
Defensive patterns
Strategy: validation
Validate before calling
int flags = O_WRONLY | O_CREAT; /* or O_RDONLY - never O_RDWR */
if ((flags & O_ACCMODE) == O_RDWR) {
fail_fast("HDFS files cannot be opened read-write");
}
hdfsFile f = hdfsOpenFile(fs, path, flags, 0, 0, 0); Type guard
static int hdfs_open_mode_supported(int flags) {
int m = flags & O_ACCMODE;
return m == O_RDONLY || m == O_WRONLY; /* O_RDWR is ENOTSUP in libhdfs */
} Try / catch
hdfsFile f = hdfsOpenFile(fs, path, flags, 0, 0, 0);
if (!f && errno == ENOTSUP && (flags & O_ACCMODE) == O_RDWR) {
/* structural limitation: switch to write-then-reopen, do not retry */
f = open_write_then_read_phases(fs, path);
} Prevention
- Fail fast on O_RDWR in your own wrappers - retrying can never succeed.
- Design for write-once semantics: write file, close, reopen read-only; use write-temp + hdfsRename for atomic replace.
- Do not port POSIX open() flag words blindly; audit for O_RDWR during HDFS migrations.
- For true random-access read/write data, choose HBase/Kudu rather than HDFS files.
When it happens
Trigger: hdfsOpenFile(fs, path, O_RDWR, ...), including combined flags like O_RDWR|O_CREAT; POSIX code ported verbatim that uses open(path, O_RDWR); code that opens a file for update-in-place (seek + write + read on one handle).
Common situations: Porting file-oriented C programs (config writers, state files) or POSIX-style libraries to libhdfs; database-style workloads (SQLite, dbm) that fundamentally need r/w - these cannot run on HDFS files at all.
Related errors
AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22).
Data as JSON: /api/errors/076505e90eb3888b.
Report an issue: GitHub.