What causes "threadCreate: pthread_create failed with error %d " in apache/hadoop?

Check `ulimit -u` and the cgroup pids.max; raise the limit or shrink the thread count

threadCreate: pthread_create failed with error %d

Error message

threadCreate: pthread_create failed with error %d

What it means

libhdfs' POSIX threadCreate() reports pthread_create failed. The number is an errno-style code: EAGAIN (out of resources — thread/process limit hit, or no memory to map the default 8MB stack), EINVAL (invalid attributes), EPERM (scheduling permission). The thread never starts, so the owning logic (or a later join) fails downstream.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/os/posix/thread.c:41

/**
 * Defines a helper function that adapts function pointer provided by caller to
 * the type required by pthread_create.
 *
 * @param toRun thread to run
 * @return void* result of running thread (always NULL)
 */
static void* runThread(void *toRun) {
  const thread *t = toRun;
  t->start(t->arg);
  return NULL;
}

int threadCreate(thread *t) {
  int ret;
  ret = pthread_create(&t->id, NULL, runThread, t);
  if (ret) {
    fprintf(stderr, "threadCreate: pthread_create failed with error %d\n", ret);
  }
  return ret;
}

int threadJoin(const thread *t) {
  int ret = pthread_join(t->id, NULL);
  if (ret) {
    fprintf(stderr, "threadJoin: pthread_join failed with error %d\n", ret);
  }
  return ret;
}

View on GitHub (pinned to 2add963021)

Solutions

Check `ulimit -u` and the cgroup pids.max; raise the limit or shrink the thread count
Check ulimit -v / container memory for stack mapping failure; unset restrictive -v limits or raise memory
Bound the worker pool instead of spawning one thread per task
If EINVAL appears (attributes are NULL here), suspect build mismatch or corruption

Example fix

# before: one thread per task exhausts the pids cgroup
for (i = 0; i < 10000; i++) threadCreate(&t[i]);

# after: fixed pool sized to the limit
for (i = 0; i < 16; i++) threadCreate(&pool[i]);  /* headroom under ulimit -u */

Defensive patterns

Strategy: validation

Validate before calling

#include <sys/resource.h>
struct rlimit rl;
getrlimit(RLIMIT_NPROC, &rl);
size_t pool = (wanted > rl.rlim_cur - 8) ? 8 : wanted; /* headroom under the limit */

Try / catch

thread t = { .start = my_fn, .arg = NULL };
if (threadCreate(&t) != 0) {
    /* code already printed with the message */
    run_inline_or_retry_with_smaller_pool();
}

Prevention

Size thread pools against `ulimit -u` and the cgroup pids.max
Avoid thread-per-task designs in containers with low pids/memory caps
Check ulimit -v before relying on default 8MB thread stacks

When it happens

Trigger: Calling threadCreate when the process is at RLIMIT_NPROC or the cgroup pids.max, or when ulimit -v / container memory prevents stack allocation (EAGAIN both ways); directly or via libhdfs test/utility code.

Common situations: CI containers with pids cgroup caps; unbounded thread-per-task designs; valgrind runs (larger effective stacks); low ulimit -v in hardened profiles.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/825d02599a26cb3f. Report an issue: GitHub.