What causes "threadJoin: pthread_join failed with error %d " in apache/hadoop?

Audit for double-join: after a successful join, invalidate the handle and never join it again

threadJoin: pthread_join failed with error %d

Error message

threadJoin: pthread_join failed with error %d

What it means

threadJoin() wraps pthread_join; failure codes are ESRCH (no such thread — already joined or handle reused), EINVAL (thread not joinable, e.g. detached), EDEADLK (thread joining itself). After this error the thread's state is undefined; joining an already-reaped thread id risks reaping an unrelated, reused thread id.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/os/posix/thread.c:49

static void* runThread(void *toRun) {
  const thread *t = toRun;
  t->start(t->arg);
  return NULL;
}

int threadCreate(thread *t) {
  int ret;
  ret = pthread_create(&t->id, NULL, runThread, t);
  if (ret) {
    fprintf(stderr, "threadCreate: pthread_create failed with error %d\n", ret);
  }
  return ret;
}

int threadJoin(const thread *t) {
  int ret = pthread_join(t->id, NULL);
  if (ret) {
    fprintf(stderr, "threadJoin: pthread_join failed with error %d\n", ret);
  }
  return ret;
}

View on GitHub (pinned to 2add963021)

Solutions

Audit for double-join: after a successful join, invalidate the handle and never join it again
If EINVAL, find who detached the thread and pick one ownership model (join OR detach, never both)
If EDEADLK, restructure shutdown so a thread never joins itself
Wrap joins in a helper that clears the stored id, making double-join impossible

Example fix

/* before */
threadJoin(&t);
/* error path may join the same t again */

/* after: join exactly once, then invalidate */
int ret = threadJoin(&t);
t.id = 0;  /* mark joined */
if (ret) log_join_error(ret);

Defensive patterns

Strategy: validation

Validate before calling

typedef struct { thread t; int joined; } thread_once;

static int join_once(thread_once *w) {
    if (w->joined) return 0;   /* block double-join at the source */
    w->joined = 1;
    return threadJoin(&w->t);
}

Try / catch

int ret = threadJoin(&t);
if (ret != 0) {
    /* ESRCH: already joined; EINVAL: detached; handle without re-joining */
    log_warn("join failed code=%d", ret);
}

Prevention

Invalidate thread handles right after a successful join
Choose one ownership model per thread: joinable or detached, never both
Never join the running thread itself

When it happens

Trigger: Calling threadJoin twice on the same handle; joining a thread that was detached elsewhere; racing threadCreate reuse of the same thread struct before the join completes.

Common situations: Cleanup loops that join every thread including ones already joined in an error branch; refactors that introduce detach; stress tests recycling thread structs.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/021a6ee83b6b115a. Report an issue: GitHub.