janhq/jan · error · Error

Failed to load llamacpp backend

Error message

Failed to load llamacpp backend

What it means

Catch-all thrown by getDevices() when invoke('plugin:llamacpp|get_devices', ...) rejects. By this point version_backend parsed and ensureBackendReady() succeeded, so the backend binary exists; the failure is in executing it with --list-devices or in the Tauri plugin layer. The original error is logged but not surfaced.

Source

Thrown at extensions/llamacpp-extension/src/index.ts:4089

                        return { ...dev, mem: total, free }
                      }
                    }
                  }
                }
                return dev
              })
              return adjusted
            }
          }
        }
      } catch (e) {
        logger.warn('Device memory override (AMD/Linux) failed:', e)
      }

      return dList
    } catch (error) {
      logger.error('Failed to query devices:\n', error)
      throw new Error('Failed to load llamacpp backend')
    }
  }

  /**
   * Resolves the default/preferred embedding model, importing and loading
   * sentence-transformer-mini as the fallback, then ensures a session exists.
   * Shared by embed() and getEmbeddingContextSize() so both agree on which
   * model is "the" embedding model.
   */
  private async ensureEmbeddingModelLoaded(): Promise<SessionInfo> {
    const downloadedModelList = await this.list()
    const installedEmbedding = downloadedModelList.filter(
      (m) => (m as any).embedding === true
    )
    const hasMini = downloadedModelList.some(
      (m) => m.id === FALLBACK_EMBEDDING_MODEL_ID
    )
    let preferred = await getDefaultEmbeddingModelId('llamacpp')

View on GitHub (pinned to fad3f12a14)

Solutions

1. Install the required GPU runtime for the selected backend (Vulkan loader, CUDA, or ROCm packages).
2. Re-download the backend via the app's backend management so the binary is not corrupt/truncated.
3. Run the backend binary manually with --list-devices from a terminal to see the real native error the plugin swallowed.
4. Switch the llamacpp backend variant to one matching your CPU arch / available GPU stack.

Example fix

// before
return await engine.getDevices()

// after (surface native error for diagnosis)
try {
  return await engine.getDevices()
} catch (e) {
  console.error('get_devices failed; run', backendPath, '--list-devices manually')
  throw e
}

import { getBackendExePath } from './backends'
import { fs } from '@janhub/core'

async function backendBinaryReady(backend: string, version: string): Promise<boolean> {
  try {
    const p = await getBackendExePath(backend, version)
    return fs.existsSync(p)
  } catch { return false }
}

try {
  return await engine.getDevices()
} catch (e) {
  if (/Failed to load llamacpp backend/.test(String(e))) {
    // run the binary directly to capture the native error
    logNativeError(backendPath, '--list-devices')
  }
  throw e
}

When it happens

Trigger: The backend executable is present but cannot run (wrong architecture, missing system libs like Vulkan/ROCm/CUDA runtime, exec permission bit unset); the --list-devices invocation crashes; the Tauri plugin command itself errors (serialization, permission scope).

Common situations: Missing GPU driver/runtime (Vulkan loader, CUDA toolkit, ROCm) on Linux; running an x86 backend on ARM or vice versa; a partially downloaded/corrupted backend binary; OS permission or sandbox blocking process spawn.

Related errors

• Failed to decompress archive: ${String(e)}
• No supported backend binaries found for this system. Backend
• Failed to fetch supported backends: ${error instanceof Error
• Invalid backend string: ${targetBackendString} supplied to u
• Invalid backend string format: "${targetBackendString}". Exp

AI-assisted analysis of janhq/jan@fad3f12a14 (2026-08-12). Data as JSON: /api/errors/7d87f476e128fc30. Report an issue: GitHub.