What causes "hadoopReadZero failed with error code %d (%s)\n" in apache/hadoop?

Enable dfs.client.read.shortcircuit=true and set dfs.domain.socket.path on the client config used by vecsum.

hadoopReadZero failed with error code %d (%s)\n

Error message

hadoopReadZero failed with error code %d (%s)\n

What it means

In zero-copy-read mode vecsum calls hadoopReadZero() (hdfsReadZero API); a NULL return means the zero-copy read failed and errno holds the reason. Zero-copy reads require short-circuit reads to be enabled and working (dfs.client.read.shortcircuit=true plus a valid dfs.domain.socket.path) and the file opened O_RDONLY; otherwise you get ENOTSUP/EPROTONOSUPPORT-style failures.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/vecsum.c:591

}

#endif

static int vecsum_zcr_loop(int pass, struct libhdfs_data *ldata,
        struct hadoopRzOptions *zopts,
        const struct options *opts)
{
    int32_t len;
    double sum = 0.0;
    const double *buf;
    struct hadoopRzBuffer *rzbuf = NULL;
    int ret;

    while (1) {
        rzbuf = hadoopReadZero(ldata->file, zopts, ZCR_READ_CHUNK_SIZE);
        if (!rzbuf) {
            ret = errno;
            fprintf(stderr, "hadoopReadZero failed with error "
                "code %d (%s)\n", ret, strerror(ret));
            goto done;
        }
        buf = hadoopRzBufferGet(rzbuf);
        if (!buf) break;
        len = hadoopRzBufferLength(rzbuf);
        if (len < ZCR_READ_CHUNK_SIZE) {
            fprintf(stderr, "hadoopReadZero got a partial read "
                "of length %d\n", len);
            ret = EINVAL;
            goto done;
        }
        sum += vecsum(buf,
            ZCR_READ_CHUNK_SIZE / sizeof(double));
        hadoopRzBufferFree(ldata->file, rzbuf);
    }
    printf("finished zcr pass %d.  sum = %g\n", pass, sum);
    ret = 0;

View on GitHub (pinned to 2add963021)

Solutions

Enable dfs.client.read.shortcircuit=true and set dfs.domain.socket.path on the client config used by vecsum.
Ensure the DataNode is configured with a matching dfs.datanode.domain.socket.path and that the socket directory is owned/group-accessible for the DFS clients (commonly root:hadoop 755 or 0755 parents).
Verify with the short-circuit smoke test or check DataNode logs for unix domain socket errors.
Until configured, run 'vecsum -t libhdfs' (normal buffered reads) instead of -t zcr.

Example fix

# before: hdfs-site.xml used by the client
<property><name>dfs.client.read.shortcircuit</name><value>false</value></property>
# after
<property><name>dfs.client.read.shortcircuit</name><value>true</value></property>
<property><name>dfs.domain.socket.path</name><value>/var/lib/hadoop-hdfs/dn_socket</value></property>

Defensive patterns

Strategy: fallback

Validate before calling

/* probe the precondition: this config must be present for zero-copy */
/* (libhdfs has no direct getter — validate the XML the binary will load) */
/* e.g. hdfs getconf -confKey dfs.client.read.shortcircuit  => must print true */

Try / catch

rzbuf = hadoopReadZero(file, zopts, ZCR_READ_CHUNK_SIZE);
if (!rzbuf) {
    int err = errno;
    if (err == ENOTSUP || err == EPROTONOSUPPORT) {
        /* zero-copy unavailable on this setup: degrade to buffered reads */
        ret = vecsum_libhdfs(ldata, opts);
    } else {
        /* real failure: report err */
    }
}

Prevention

Before -t zcr runs, confirm dfs.client.read.shortcircuit=true and a valid dfs.domain.socket.path on both client and DataNode.
Verify the domain-socket directory ownership/permissions (accessible to the DFS client user) before starting benchmarks.
Smoke-test zero-copy with a tiny aligned file before long runs.
Keep a -t libhdfs fallback path in benchmark scripts for clusters where short-circuit is disabled.

When it happens

Trigger: Running 'vecsum -t zcr' against a cluster where dfs.client.read.shortcircuit is false, dfs.domain.socket.path is unset/misconfigured or its directory has wrong ownership/permissions, the DataNode does not have the matching dfs.datanode.domain.socket.path, or the domain socket connection failed at read time.

Common situations: Performance-testing zero-copy reads on a cluster where domain sockets were never set up; the domain socket directory was created by root so the hadoop user cannot connect; secure-cluster configs where short-circuit was disabled for security.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a05d4229bcf8913d. Report an issue: GitHub.