{"record":{"id":"825d02599a26cb3f","repo":"apache/hadoop","slug":"threadcreate-pthread-create-failed-with-error-d","errorCode":null,"errorMessage":"threadCreate: pthread_create failed with error %d\n","messagePattern":"threadCreate: pthread_create failed with error (.+?)\n","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/os/posix/thread.c","lineNumber":41,"sourceCode":"\n/**\n * Defines a helper function that adapts function pointer provided by caller to\n * the type required by pthread_create.\n *\n * @param toRun thread to run\n * @return void* result of running thread (always NULL)\n */\nstatic void* runThread(void *toRun) {\n  const thread *t = toRun;\n  t->start(t->arg);\n  return NULL;\n}\n\nint threadCreate(thread *t) {\n  int ret;\n  ret = pthread_create(&t->id, NULL, runThread, t);\n  if (ret) {\n    fprintf(stderr, \"threadCreate: pthread_create failed with error %d\\n\", ret);\n  }\n  return ret;\n}\n\nint threadJoin(const thread *t) {\n  int ret = pthread_join(t->id, NULL);\n  if (ret) {\n    fprintf(stderr, \"threadJoin: pthread_join failed with error %d\\n\", ret);\n  }\n  return ret;\n}\n","sourceCodeStart":23,"sourceCodeEnd":53,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/os/posix/thread.c#L23-L53","documentation":"libhdfs' POSIX threadCreate() reports pthread_create failed. The number is an errno-style code: EAGAIN (out of resources — thread/process limit hit, or no memory to map the default 8MB stack), EINVAL (invalid attributes), EPERM (scheduling permission). The thread never starts, so the owning logic (or a later join) fails downstream.","triggerScenarios":"Calling threadCreate when the process is at RLIMIT_NPROC or the cgroup pids.max, or when ulimit -v / container memory prevents stack allocation (EAGAIN both ways); directly or via libhdfs test/utility code.","commonSituations":"CI containers with pids cgroup caps; unbounded thread-per-task designs; valgrind runs (larger effective stacks); low ulimit -v in hardened profiles.","solutions":["Check `ulimit -u` and the cgroup pids.max; raise the limit or shrink the thread count","Check ulimit -v / container memory for stack mapping failure; unset restrictive -v limits or raise memory","Bound the worker pool instead of spawning one thread per task","If EINVAL appears (attributes are NULL here), suspect build mismatch or corruption"],"exampleFix":"# before: one thread per task exhausts the pids cgroup\nfor (i = 0; i < 10000; i++) threadCreate(&t[i]);\n\n# after: fixed pool sized to the limit\nfor (i = 0; i < 16; i++) threadCreate(&pool[i]);  /* headroom under ulimit -u */","handlingStrategy":"validation","validationCode":"#include <sys/resource.h>\nstruct rlimit rl;\ngetrlimit(RLIMIT_NPROC, &rl);\nsize_t pool = (wanted > rl.rlim_cur - 8) ? 8 : wanted; /* headroom under the limit */","typeGuard":null,"tryCatchPattern":"thread t = { .start = my_fn, .arg = NULL };\nif (threadCreate(&t) != 0) {\n    /* code already printed with the message */\n    run_inline_or_retry_with_smaller_pool();\n}","preventionTips":["Size thread pools against `ulimit -u` and the cgroup pids.max","Avoid thread-per-task designs in containers with low pids/memory caps","Check ulimit -v before relying on default 8MB thread stacks"],"tags":["libhdfs","pthread","thread-creation","resource-limits","posix"],"backgroundTag":"thread-creation-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}