What causes "vecsum_normal_loop pass %d failed with error %d\n" in apache/hadoop?

Look at the stderr line immediately preceding this one (hdfsRead failed with error N / partial read of length N) — that is the actionable error.

apache/hadoop · error

vecsum_normal_loop pass %d failed with error %d\n

Error message

vecsum_normal_loop pass %d failed with error %d\n

What it means

Wrapper error: vecsum_libhdfs() reports that normal-read pass N failed with the code returned by vecsum_normal_loop() and returns immediately, skipping the remaining passes. The root cause was printed one line earlier inside the loop.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/vecsum.c:719

    printf("finished normal pass %d.  sum = %g\n", pass, sum);
    return 0;
}

static int vecsum_libhdfs(struct libhdfs_data *ldata,
            const struct options *opts)
{
    int pass;

    ldata->buf = malloc(NORMAL_READ_CHUNK_SIZE);
    if (!ldata->buf) {
        fprintf(stderr, "failed to malloc buffer of size %d\n",
            NORMAL_READ_CHUNK_SIZE);
        return ENOMEM;
    }
    for (pass = 0; pass < opts->passes; ++pass) {
        int ret = vecsum_normal_loop(pass, ldata, opts);
        if (ret) {
            fprintf(stderr, "vecsum_normal_loop pass %d failed "
                "with error %d\n", pass, ret);
            return ret;
        }
        hdfsSeek(ldata->fs, ldata->file, 0);
    }
    return 0;
}

static void vecsum_local(struct local_data *cdata, const struct options *opts)
{
    int pass;

    for (pass = 0; pass < opts->passes; pass++) {
        double sum = vecsum(cdata->mmap, cdata->length / sizeof(double));
        printf("finished vecsum_local pass %d.  sum = %g\n", pass, sum);
    }
}

View on GitHub (pinned to 2add963021)

Solutions

Look at the stderr line immediately preceding this one (hdfsRead failed with error N / partial read of length N) — that is the actionable error.
Fix per that message (cluster stability, aligned length, file exists), then re-run.

Defensive patterns

Strategy: try-catch

Validate before calling

/* prevent the underlying loop errors */
if (opts->length % NORMAL_READ_CHUNK_SIZE) return EINVAL;
if (hdfsExists(ldata->fs, opts->path) != 0) return ENOENT;

Try / catch

ret = vecsum_normal_loop(pass, ldata, opts);
if (ret) {
    /* ret == errno from hdfsRead or EINVAL from partial read;
       only transient errno values justify a full-pass retry after hdfsSeek */
}

Prevention

Interpret the printed error code together with the line printed just above it by the loop.
Fix deterministic causes (alignment, missing file) before any retry.
Keep per-pass error codes in scripts so you can classify failures automatically.

When it happens

Trigger: Any nonzero return from vecsum_normal_loop(): hdfsReadFully() failing (errno case) or a partial read because the file length is not a multiple of NORMAL_READ_CHUNK_SIZE.

Common situations: Same as the underlying read errors: DataNode/network problems mid-benchmark, unaligned -l length, file deleted during the run.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b68b35479f13d5aa. Report an issue: GitHub.