xai-org/x-algorithm · warning · std::runtime_error

could not resolve loaded NCCL library path

Error message

could not resolve loaded NCCL library path

What it means

The nccl_library_path binding uses dladdr on ncclGetVersion to find the loaded NCCL shared object path. If dladdr fails (returns 0) or yields a null filename, a std::runtime_error 'could not resolve loaded NCCL library path' is thrown.

Source

Thrown at phoenix/xrex/cuda/async_emb/src/async_emb_api.cc:840

          throw std::invalid_argument("async_emb context not initialized");
        }
        ctx->resetTableBinding();
      },
      nb::call_guard<nb::gil_scoped_release>()
  );
  m.def("_test_snapshot", &testSnapshot);
  m.def("nccl_version", [] {
    int version = 0;
    ncclResult_t result = ncclGetVersion(&version);
    if (result != ncclSuccess) {
      throw std::runtime_error(ncclGetErrorString(result));
    }
    return version;
  });
  m.def("nccl_library_path", [] {
    Dl_info info{};
    if (dladdr(reinterpret_cast<void*>(&ncclGetVersion), &info) == 0 || info.dli_fname == nullptr) {
      throw std::runtime_error("could not resolve loaded NCCL library path");
    }
    return std::string(info.dli_fname);
  });
}

}

View on GitHub (pinned to 24c60942c5)

Solutions

1. Fall back to inspecting /proc/self/maps (or ldd on the module .so) to find the NCCL DSO manually
2. Avoid preloads/shims that hide NCCL symbol provenance
3. If NCCL is statically linked, expect this to fail and use build metadata instead

Example fix

# before
path = async_emb.nccl_library_path()  # throws if dladdr fails

# after
import subprocess
path = next(l for l in open('/proc/self/maps') if 'libnccl' in l).split()[-1]

def ncclPath():
    try:
        return async_emb.nccl_library_path()
    except RuntimeError:
        return next((l.split()[-1] for l in open('/proc/self/maps') if 'libnccl' in l), None)

try:
    path = async_emb.nccl_library_path()
except RuntimeError as e:
    path = next((l.split()[-1] for l in open('/proc/self/maps') if 'libnccl' in l), None)
    if path is None: raise

When it happens

Trigger: Calling async_emb.nccl_library_path() when the NCCL symbols cannot be mapped back to a loaded DSO — e.g. statically linked NCCL, symbols resolved via a shim/trampoline, or unusual loader setups where dladdr info is unavailable.

Common situations: Static NCCL linkage in custom builds; LD_PRELOAD interposition of NCCL symbols; stripped or unusual ELF setups; calling in restricted sandbox environments where dladdr info is limited.

Related errors

• NCCL version query failed: ${ncclGetErrorString(result)}
• async_emb: ${label} timed out waiting for step ${step}
• NCCL ${operation} failed: ${ncclGetErrorString(result)}
• async_emb context not initialized
• unknown async_emb snapshot region: ${region}

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/1dce3da3e48cfeee. Report an issue: GitHub.