What causes "vecsum_zcr_loop pass %d failed with error %d\n" in apache/hadoop?

Read the stderr line just above this one — it carries the root cause (hadoopReadZero errno or partial-read length).

vecsum_zcr_loop pass %d failed with error %d\n

Error message

vecsum_zcr_loop pass %d failed with error %d\n

What it means

Wrapper error: vecsum_zcr() reports that zero-copy pass number N failed with the error code returned by vecsum_zcr_loop() and aborts the remaining passes. The real cause is always printed immediately before by the loop itself (hadoopReadZero failure or partial read).

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/vecsum.c:642

    if (!zopts) {
        fprintf(stderr, "hadoopRzOptionsAlloc failed.\n");
        ret = ENOMEM;
        goto done;
    }
    if (hadoopRzOptionsSetSkipChecksum(zopts, 1)) {
        ret = errno;
        perror("hadoopRzOptionsSetSkipChecksum failed: ");
        goto done;
    }
    if (hadoopRzOptionsSetByteBufferPool(zopts, NULL)) {
        ret = errno;
        perror("hadoopRzOptionsSetByteBufferPool failed: ");
        goto done;
    }
    for (pass = 0; pass < opts->passes; ++pass) {
        ret = vecsum_zcr_loop(pass, ldata, zopts, opts);
        if (ret) {
            fprintf(stderr, "vecsum_zcr_loop pass %d failed "
                "with error %d\n", pass, ret);
            goto done;
        }
        hdfsSeek(ldata->fs, ldata->file, 0);
    }
    ret = 0;
done:
    if (zopts)
        hadoopRzOptionsFree(zopts);
    return ret;
}

tSize hdfsReadFully(hdfsFS fs, hdfsFile f, void* buffer, tSize length)
{
    uint8_t *buf = buffer;
    tSize ret, nread = 0;

    while (length > 0) {

View on GitHub (pinned to 2add963021)

Solutions

Read the stderr line just above this one — it carries the root cause (hadoopReadZero errno or partial-read length).
Fix short-circuit config (dfs.client.read.shortcircuit + dfs.domain.socket.path) or align -l to 8 MiB as indicated by that underlying message.
Re-run the benchmark.

Defensive patterns

Strategy: try-catch

Validate before calling

/* prevent the two known loop failures up front */
if (opts->length % ZCR_READ_CHUNK_SIZE) return EINVAL;  /* partial-read guard */
/* plus: short-circuit config must be enabled before selecting zcr mode */

Try / catch

ret = vecsum_zcr_loop(pass, ldata, zopts, opts);
if (ret) {
    /* ret is the errno from hadoopReadZero or EINVAL from a partial chunk —
       dispatch on it rather than re-running blindly */
    if (ret == ENOTSUP) { /* config problem: stop, do not retry more passes */ }
    else if (ret == EINVAL) { /* alignment problem: stop */ }
    else { /* transient I/O: safe to restart from pass 0 after hdfsSeek */ }
}

Prevention

Treat this as an aggregator: always log and interpret the underlying loop error printed just before it.
Do not retry remaining passes after ENOTSUP/EINVAL — those are deterministic failures.
Re-seek to 0 (hdfsSeek) before any pass retry, mirroring what the loop itself does between successful passes.

When it happens

Trigger: Any nonzero return from vecsum_zcr_loop(): hadoopReadZero() returning NULL (short-circuit/domain-socket misconfiguration) or a partial chunk (file length not a multiple of ZCR_READ_CHUNK_SIZE). Failing on an early pass also skips all later passes, so a single transient error kills the whole zcr benchmark.

Common situations: First run of 'vecsum -t zcr' on a cluster without short-circuit reads enabled; running with an unaligned file length; DataNode restart mid-benchmark dropping the domain socket.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/4ed88f1bceb88b26. Report an issue: GitHub.