{"record":{"id":"eba571e579bc8aaf","repo":"apache/hadoop","slug":"hdfsread-failed-with-error-d-s-n","errorCode":null,"errorMessage":"hdfsRead failed with error %d (%s)\\n","messagePattern":"hdfsRead failed with error (.+?) \\((.+?)\\)\\\\n","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/vecsum.c","lineNumber":689,"sourceCode":"        length -= ret;\n        buf += ret;\n    }\n    return nread;\n}\n\nstatic int vecsum_normal_loop(int pass, const struct libhdfs_data *ldata,\n            const struct options *opts)\n{\n    double sum = 0.0;\n\n    while (1) {\n        int res = hdfsReadFully(ldata->fs, ldata->file, ldata->buf,\n                NORMAL_READ_CHUNK_SIZE);\n        if (res == 0) // EOF\n            break;\n        if (res < 0) {\n            int err = errno;\n            fprintf(stderr, \"hdfsRead failed with error %d (%s)\\n\",\n                err, strerror(err));\n            return err;\n        }\n        if (res < NORMAL_READ_CHUNK_SIZE) {\n            fprintf(stderr, \"hdfsRead got a partial read of \"\n                \"length %d\\n\", res);\n            return EINVAL;\n        }\n        sum += vecsum(ldata->buf,\n                  NORMAL_READ_CHUNK_SIZE / sizeof(double));\n    }\n    printf(\"finished normal pass %d.  sum = %g\\n\", pass, sum);\n    return 0;\n}\n\nstatic int vecsum_libhdfs(struct libhdfs_data *ldata,\n            const struct options *opts)\n{","sourceCodeStart":671,"sourceCodeEnd":707,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/vecsum.c#L671-L707","documentation":"In the normal (buffered) read loop vecsum calls hdfsReadFully() for NORMAL_READ_CHUNK_SIZE (8 MiB) bytes; a negative return means the read failed and errno holds the reason (checksum error EIO, connection reset ECONNRESET, lease/auth failures, etc.). The error code is returned up and aborts the pass.","triggerScenarios":"A DataNode dies or is restarted mid-read, network interruption between client and DataNode, the file is deleted while being read, or a Kerberos token expires during a long benchmark run.","commonSituations":"Long multi-pass benchmark runs that overlap with cluster maintenance (DN restarts/decommission); flaky networks; running vecsum against a file another job is actively deleting.","solutions":["Use the errno text to classify: ECONNRESET/EOF on stream -> DataNode/network issue, check DataNode logs; EACCES/EPERM -> auth/permission; ESTALE/ENOENT -> file removed.","Re-run the benchmark after the cluster is stable; reduce --passes to shorten exposure.","Verify the file still exists and is readable: hdfs dfs -ls <path>."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"/* before a long benchmark: confirm readability end-to-end */\ntSize_t cap = hdfsGetDefaultBlockSize(fs);\nhdfsFileInfo *i = hdfsGetPathInfo(fs, path);\nif (!i) { /* do not start the run */ }\nhdfsFreeFileInfo(i, 1);","typeGuard":null,"tryCatchPattern":"int res = hdfsReadFully(fs, file, buf, CHUNK);\nif (res < 0) {\n    int err = errno;\n    if (err == ECONNRESET || err == EIO) {\n        /* transient DataNode/network failure: reopen the file and retry the pass */\n    } else {\n        /* auth/permission/stale: report and stop */\n    }\n}","preventionTips":["Schedule long multi-pass runs away from DataNode restarts/decommissioning windows.","Retry at pass granularity: reopen the stream and seek back to offset 0 rather than failing the whole run.","Distinguish retryable errnos (ECONNRESET, EAGAIN-ish) from fatal ones (EACCES, ESTALE) before retrying."],"tags":["hdfs","libhdfs","c","errno","read","vecsum"],"backgroundTag":"hdfs-read-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}