What causes "vecsum_length: stat(%s) failed: error %d (%s)\n" in apache/hadoop?

Place the local test file outside auto-cleaned directories like /tmp.

vecsum_length: stat(%s) failed: error %d (%s)\n

Error message

vecsum_length: stat(%s) failed: error %d (%s)\n

What it means

After the timed run finishes, vecsum_length() stats the local backing file (VECSUM_LOCAL mode) to compute throughput for the stopwatch report. If stat() fails it prints this message and returns -EIO; main() then simply skips the rate report — the benchmark result itself has already completed.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/vecsum.c:745

static void vecsum_local(struct local_data *cdata, const struct options *opts)
{
    int pass;

    for (pass = 0; pass < opts->passes; pass++) {
        double sum = vecsum(cdata->mmap, cdata->length / sizeof(double));
        printf("finished vecsum_local pass %d.  sum = %g\n", pass, sum);
    }
}

static long long vecsum_length(const struct options *opts,
                const struct libhdfs_data *ldata)
{
    if (opts->ty == VECSUM_LOCAL) {
        struct stat st_buf = { 0 };
        if (stat(opts->path, &st_buf)) {
            int err = errno;
            fprintf(stderr, "vecsum_length: stat(%s) failed: "
                "error %d (%s)\n", opts->path, err, strerror(err));
            return -EIO;
        }
        return st_buf.st_size;
    } else {
        return ldata->length;
    }
}

/*
 * vecsum is a microbenchmark which measures the speed of various ways of
 * reading from HDFS.  It creates a file containing floating-point 'doubles',
 * and computes the sum of all the doubles several times.  For some CPUs,
 * assembly optimizations are used for the summation (SSE, etc).
 */
int main(void)
{
    int ret = 1;

View on GitHub (pinned to 2add963021)

Solutions

Place the local test file outside auto-cleaned directories like /tmp.
Ensure the file persists and stays readable for the whole vecsum run.
If it only recurs, check for other actors: lsof / audit on the path.

Defensive patterns

Strategy: validation

Validate before calling

/* in local mode, stat the file before starting the run */
struct stat st;
if (stat(opts->path, &st) || st.st_size != opts->length) {
    fprintf(stderr, "local file %s missing or wrong size\n", opts->path);
    return -1;
}

Try / catch

long long len = vecsum_length(opts, ldata);
if (len < 0) {
    /* -EIO: only the throughput report is lost; the run result is still valid.
       Log and continue rather than failing the whole benchmark. */
}

Prevention

Put local benchmark files outside /tmp and other auto-cleaned directories.
Ensure the file's ownership/permissions stay stable for the full run.
Treat vecsum_length failure as non-fatal — the timing data is already collected.

When it happens

Trigger: The local mmap-backed file (opts->path) was deleted or made inaccessible between the run and the measurement, e.g. tmpwatch/systemd-tmpfiles cleaning /tmp, another process removing the file, or permission changes.

Common situations: Running vecsum in local mode against a file in /tmp on hosts with aggressive tmp cleaners; running in a directory removed by a cleanup script mid-run.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a5a834164fdfd298. Report an issue: GitHub.