{"record":{"id":"67f32e8627171d31","repo":"apache/hadoop","slug":"threadlocalstoragecreate-oom-unable-to-allocate","errorCode":null,"errorMessage":"threadLocalStorageCreate: OOM - Unable to allocate thread local state\n","messagePattern":"threadLocalStorageCreate: OOM - Unable to allocate thread local state\n","errorType":"console","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/os/posix/thread_local_storage.c","lineNumber":157,"sourceCode":"  snprintf(id, max, \"%s:%ld\", thr_name_str, thr_id);\n\n  // Release the char*\n  (*env)->ReleaseStringUTFChars(env, thr_name, thr_name_str);\n\ndone:\n  destroyLocalReference(env, thr);\n  destroyLocalReference(env, thr_name);\n\n  // Make sure the id is null terminated in case we overflow the max length\n  id[max - 1] = '\\0';\n}\n\nstruct ThreadLocalState* threadLocalStorageCreate()\n{\n  struct ThreadLocalState *state;\n  state = (struct ThreadLocalState*)malloc(sizeof(struct ThreadLocalState));\n  if (state == NULL) {\n    fprintf(stderr,\n      \"threadLocalStorageCreate: OOM - Unable to allocate thread local state\\n\");\n    return NULL;\n  }\n  state->lastExceptionStackTrace = NULL;\n  state->lastExceptionRootCause = NULL;\n  return state;\n}\n\nint threadLocalStorageGet(struct ThreadLocalState **state)\n{\n  int ret = 0;\n  if (!gTlsKeyInitialized) {\n    ret = pthread_key_create(&gTlsKey, hdfsThreadDestructor);\n    if (ret) {\n      fprintf(stderr,\n        \"threadLocalStorageGet: pthread_key_create failed with error %d\\n\",\n        ret);\n      return ret;","sourceCodeStart":139,"sourceCodeEnd":175,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/os/posix/thread_local_storage.c#L139-L175","documentation":"threadLocalStorageCreate's malloc for the per-thread ThreadLocalState (env pointer plus exception-string buffers) returned NULL — the C heap is exhausted. getJNIEnv aborts for this thread (its caller also logs 'Unable to create ThreadLocalState') and the current HDFS call fails. This is purely an out-of-memory condition.","triggerScenarios":"First libhdfs use on a thread when malloc fails: cgroup memory cap reached, RLIMIT_AS/RLIMIT_DATA hit, or a severe native leak consuming the heap.","commonSituations":"Containers with tight MemoryMax; long-running daemons with slow leaks; test suites under valgrind's memory overhead.","solutions":["Compare process RSS with cgroup MemoryMax / ulimit -v / -d; raise the limit or restart the leaked process","Profile with valgrind/heaptrack for native leaks if it recurs","Retry the operation after pressure clears; the allocation itself is tiny"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"if (!hdfsConnect(nn, port) && errno == ENOMEM) {\n    sleep(2);\n    fs = hdfsConnect(nn, port); /* one bounded retry */\n}","preventionTips":["Monitor native RSS against cgroup/rlimit memory ceilings","Run periodic leak checks (valgrind/heaptrack) on long-lived daemons","Fail after one bounded retry rather than spinning on ENOMEM"],"tags":["libhdfs","out-of-memory","malloc","thread-local-storage"],"backgroundTag":"out-of-memory","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}