janhq/jan · error

Failed to load llamacpp backend

Error message

Failed to load llamacpp backend

What it means

Thrown by the llamacpp extension's device-listing routine when querying available GPU/compute devices via the underlying backend fails for any reason. The original error is logged, then replaced with this generic message, so the root cause (driver, backend library, IPC failure) is only visible in logs. It indicates the llamacpp backend could not be initialized enough to enumerate devices.

Solutions

1. Check extension logs for the 'Failed to query devices' entry to find the root cause error
2. Update or reinstall GPU drivers (Vulkan/CUDA/Metal) appropriate for your hardware
3. Reinstall or update the llamacpp extension so backend binaries match your platform
4. Fall back to CPU-only configuration by removing GPU device overrides
5. Report the logged root cause to the extension maintainers if drivers are healthy

Example fix

// before
const devices = await getDevices()
// after
let devices = []
try {
  devices = await getDevices()
} catch (e) {
  logger.warn('GPU device query failed, falling back to CPU', e)
}

// Precheck GPU availability before querying devices
const hasGpu = navigator.gpu !== undefined // or platform driver check
if (!hasGpu) console.warn('No GPU detected; backend device query may fail')

try {
  const devices = await getDevices()
} catch (e) {
  logger.warn('Device query failed, continuing without GPU devices', e)
}

When it happens

Trigger: Calling the device-list API (e.g. listDevices/providers) when the native llama.cpp backend fails to load or its device query throws — missing/Corrupt Vulkan/CUDA/Metal drivers, incompatible backend binaries, or a crashed native process.

Common situations: Users on machines without required GPU drivers, after upgrading llama.cpp backend binaries, on WSL/VMs without GPU passthrough, or when the backend shared library fails to dlopen.

Related errors

• Backend update failed
• Backend update failed
• Current backend not found
• Extension does not support backend updates
• LlamaCpp extension not found

AI-assisted analysis of janhq/jan@7205d770c1 (2026-09-17). Data as JSON: /api/errors/7d87f476e128fc30. Report an issue: GitHub.

Appendix: source

Thrown at extensions/llamacpp-extension/src/index.ts:2831

                        return { ...dev, mem: total, free }
                      }
                    }
                  }
                }
                return dev
              })
              return adjusted
            }
          }
        }
      } catch (e) {
        logger.warn('Device memory override (AMD/Linux) failed:', e)
      }

      return dList
    } catch (error) {
      logger.error('Failed to query devices:\n', error)
      throw new Error('Failed to load llamacpp backend')
    }
  }

  /**
   * Resolves the default/preferred embedding model, importing and loading
   * sentence-transformer-mini as the fallback, then ensures a session exists.
   * Shared by embed() and getEmbeddingContextSize() so both agree on which
   * model is "the" embedding model.
   */
  private async ensureEmbeddingModelLoaded(): Promise<SessionInfo> {
    const downloadedModelList = await this.list()
    const installedEmbedding = downloadedModelList.filter(
      (m) => (m as any).embedding === true
    )
    const hasMini = downloadedModelList.some(
      (m) => m.id === FALLBACK_EMBEDDING_MODEL_ID
    )
    let preferred = await getDefaultEmbeddingModelId('llamacpp')

View on GitHub (pinned to 7205d770c1)