{"record":{"id":"1fae4d936dbd55c5","repo":"apache/hadoop","slug":"threadlocalstorageget-pthread-key-create-failed-w","errorCode":null,"errorMessage":"threadLocalStorageGet: pthread_key_create failed with error %d\n","messagePattern":"threadLocalStorageGet: pthread_key_create failed with error (.+?)\n","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/os/posix/thread_local_storage.c","lineNumber":172,"sourceCode":"  struct ThreadLocalState *state;\n  state = (struct ThreadLocalState*)malloc(sizeof(struct ThreadLocalState));\n  if (state == NULL) {\n    fprintf(stderr,\n      \"threadLocalStorageCreate: OOM - Unable to allocate thread local state\\n\");\n    return NULL;\n  }\n  state->lastExceptionStackTrace = NULL;\n  state->lastExceptionRootCause = NULL;\n  return state;\n}\n\nint threadLocalStorageGet(struct ThreadLocalState **state)\n{\n  int ret = 0;\n  if (!gTlsKeyInitialized) {\n    ret = pthread_key_create(&gTlsKey, hdfsThreadDestructor);\n    if (ret) {\n      fprintf(stderr,\n        \"threadLocalStorageGet: pthread_key_create failed with error %d\\n\",\n        ret);\n      return ret;\n    }\n    gTlsKeyInitialized = 1;\n  }\n  *state = pthread_getspecific(gTlsKey);\n  return ret;\n}\n\nint threadLocalStorageSet(struct ThreadLocalState *state)\n{\n  int ret = pthread_setspecific(gTlsKey, state);\n  if (ret) {\n    fprintf(stderr,\n      \"threadLocalStorageSet: pthread_setspecific failed with error %d\\n\",\n      ret);\n    hdfsThreadDestructor(state);","sourceCodeStart":154,"sourceCodeEnd":190,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/os/posix/thread_local_storage.c#L154-L190","documentation":"On the first thread-local access libhdfs lazily creates its process-wide pthread TLS key, registering hdfsThreadDestructor. pthread_key_create failed: EAGAIN means PTHREAD_KEYS_MAX (typically 1024) keys already exist in the process; ENOMEM means no memory for the key. threadLocalStorageGet returns the error, so getJNIEnv fails for the calling thread.","triggerScenarios":"Embedding libhdfs in a process whose TLS key space is already exhausted by many other runtime libraries/plugins, or under ENOMEM conditions.","commonSituations":"Large plugin hosts (scripting-language extensions plus many native libs); libraries that leak keys by re-initializing after dlopen/dlclose cycles; general memory exhaustion.","solutions":["If EAGAIN: find key hogs — trace pthread_key_create calls (ltrace/gdb breakpoint) and reduce or fix libraries that create keys per-init without pthread_key_delete","If ENOMEM: address memory exhaustion (limits or leaks)","Load and initialize libhdfs early so it claims its single key before the space fills","Avoid dlopen/dlclose cycles of libhdfs — the key and JVM state are process-wide and only reclaimed at exit"],"exampleFix":null,"handlingStrategy":"validation","validationCode":null,"typeGuard":null,"tryCatchPattern":"/* After initializing libhdfs once, cheap calls confirm TLS is healthy */\nif (hdfsConfGetStr(\"dfs.replication\", &val) != 0) {\n    /* TLS key path broken; report instead of hammering */\n    fprintf(stderr, \"libhdfs TLS init failed\\n\");\n}","preventionTips":["Load and initialize libhdfs early, before plugins exhaust the 1024 TLS-key space","Avoid dlopen/dlclose cycles of libhdfs; treat it as process-wide state","Audit native dependencies that allocate pthread keys per initialization"],"tags":["libhdfs","pthread","tls","thread-local-storage","resource-limits"],"backgroundTag":"tls-key-exhausted","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}