{"record":{"id":"f421f3934bc693e5","repo":"apache/hadoop","slug":"enomem-f421f3","errorCode":"ENOMEM","errorMessage":"calcEffectiveURI: out of memory","messagePattern":"calcEffectiveURI: out of memory","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/hdfs.c","lineNumber":668,"sourceCode":"        return EINVAL;\n    scheme = (strstr(bld->nn, \"://\")) ? \"\" : \"hdfs://\";\n    if (bld->port == 0) {\n        suffix[0] = '\\0';\n    } else {\n        lastColon = strrchr(bld->nn, ':');\n        if (lastColon && (strspn(lastColon + 1, \"0123456789\") ==\n                          strlen(lastColon + 1))) {\n            fprintf(stderr, \"port %d was given, but URI '%s' already \"\n                \"contains a port!\\n\", bld->port, bld->nn);\n            return EINVAL;\n        }\n        snprintf(suffix, sizeof(suffix), \":%d\", bld->port);\n    }\n\n    uriLen = strlen(scheme) + strlen(bld->nn) + strlen(suffix);\n    u = malloc((uriLen + 1) * (sizeof(char)));\n    if (!u) {\n        fprintf(stderr, \"calcEffectiveURI: out of memory\");\n        return ENOMEM;\n    }\n    snprintf(u, uriLen + 1, \"%s%s%s\", scheme, bld->nn, suffix);\n    *uri = u;\n    return 0;\n}\n\nstatic const char *maybeNull(const char *str)\n{\n    return str ? str : \"(NULL)\";\n}\n\nstatic const char *hdfsBuilderToStr(const struct hdfsBuilder *bld,\n                                    char *buf, size_t bufLen)\n{\n    snprintf(buf, bufLen, \"forceNewInstance=%d, nn=%s, port=%d, \"\n             \"kerbTicketCachePath=%s, userName=%s\",\n             bld->forceNewInstance, maybeNull(bld->nn), bld->port,","sourceCodeStart":650,"sourceCodeEnd":686,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/hdfs.c#L650-L686","documentation":"calcEffectiveURI mallocs the assembled 'hdfs://' + namenode + ':port' string before the JVM is contacted; when that allocation fails, hdfsBuilderConnect aborts with ENOMEM and returns NULL. This is client-side memory exhaustion in the process embedding libhdfs - the allocation itself is tiny, so the heap was already exhausted.","triggerScenarios":"hdfsConnect/hdfsConnectAsUser/hdfsBuilderConnect while the C heap of the embedding process is exhausted: rlimit-as/cgroup memory limits hit, a leak elsewhere in the process, or a 32-bit address space fully mapped.","commonSituations":"Long-running services with slow leaks that finally surface on connect; fuse_dfs on memory-capped hosts; 32-bit clients with large mappings; OOM-adjacent incidents where the first visible failure is a trivial allocation.","solutions":["Compare process RSS against cgroup/rlimit limits; the URI allocation is the victim, not the cause.","Profile for leaks (valgrind/heap profilers) in the embedding process - libhdfs is rarely the consumer.","Raise memory limits or restart the leaking service, then retry the connect.","On 32-bit hosts, move to 64-bit - this allocation is only the first of many that will fail under pressure."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// cheap preflight: fail or defer before calling connect when memory is gone\nvoid *probe = malloc(4096);\nif (!probe) {\n    /* heap exhausted: shed load instead of calling hdfsBuilderConnect */\n    return -ENOMEM;\n}\nfree(probe);\nhdfsFS fs = hdfsConnect(nn, port);","typeGuard":null,"tryCatchPattern":"hdfsFS fs = hdfsBuilderConnect(bld);\nif (!fs && errno == ENOMEM) {\n    release_native_caches();          /* application-level relief */\n    fs = hdfsBuilderConnect(hdfsNewBuilder_configured());  /* rebuild + retry once */\n}\nif (!fs) {\n    /* permanent failure: report, do not loop */\n}","preventionTips":["Set memory limits (cgroup/rlimit) with headroom above measured peak so trivial allocations never fail.","Fix leaks in the embedding process; libhdfs connection setup allocates almost nothing.","Retry ENOMEM at most once after releasing memory - repeated retries just thrash.","On 32-bit builds under pressure, move to 64-bit."],"tags":["c","libhdfs","oom","enomem","connection"],"backgroundTag":"native-oom-allocation","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}