rustdesk/rustdesk · error · anyhow::Error

Failed to create decoder

Error message

Failed to create decoder

What it means

HwRamDecoder::new found a CodecInfo for the requested format (info was Some) but hwcodec's Decoder::new(ctx) failed to instantiate it. As a side effect HwCodecConfig::clear(false, false) wipes the cached hardware config so the next detection re-probes, then 'Failed to create decoder' returns. The underlying cause is dropped by map_err, so only the generic message survives.

Source

Thrown at libs/scrap/src/common/hwcodec.rs:363

        info
    }

    pub fn new(format: CodecFormat) -> ResultType<Self> {
        let info = HwRamDecoder::try_get(format);
        log::info!("try create {info:?} ram decoder");
        let Some(info) = info else {
            bail!("unsupported format: {:?}", format);
        };
        let ctx = DecodeContext {
            name: info.name.clone(),
            device_type: info.hwdevice.clone(),
            thread_count: codec_thread_num(16) as _,
        };
        match Decoder::new(ctx) {
            Ok(decoder) => Ok(HwRamDecoder { decoder, info }),
            Err(_) => {
                HwCodecConfig::clear(false, false);
                Err(anyhow!(format!("Failed to create decoder")))
            }
        }
    }
    pub fn decode<'a>(&'a mut self, data: &[u8]) -> ResultType<Vec<HwRamDecoderImage<'a>>> {
        match self.decoder.decode(data) {
            Ok(v) => Ok(v.iter().map(|f| HwRamDecoderImage { frame: f }).collect()),
            Err(e) => Err(anyhow!(e)),
        }
    }
}

pub struct HwRamDecoderImage<'a> {
    frame: &'a DecodeFrame,
}

impl HwRamDecoderImage<'_> {
    // rgb [in/out] fmt and stride must be set in ImageRgb
    pub fn to_fmt(&self, rgb: &mut ImageRgb, i420: &mut Vec<u8>) -> ResultType<()> {

View on GitHub (pinned to 7aa98d43cf)

Solutions

1. Log the discarded inner error (keep e in the anyhow! message) to see the driver's refusal reason.
2. Retry creation once after HwCodecConfig::clear has invalidated the cache (the code already clears it for you).
3. Fall back to software decoding for this stream (vpx path) when hardware decode creation fails.
4. Check concurrency: close other decode sessions or reduce the number of simultaneous video streams.

Example fix

// before
Err(_) => {
    HwCodecConfig::clear(false, false);
    Err(anyhow!(format!("Failed to create decoder")))
}

// after
Err(e) => {
    HwCodecConfig::clear(false, false);
    Err(anyhow!("Failed to create decoder: {}", e))
}

// confirm the decoder device can actually open before the session
let info = hwcodec::Decoder::info(0, is_hevc).ok_or_else(|| anyhow!("no hw decoder listed"))?;
let probe = hwcodec::Decoder::new(DecodeContext { name: info.name.clone(), device_type: info.hwdevice.clone(), thread_count: 1 });

match Decoder::new(ctx) {
    Ok(decoder) => Ok(HwRamDecoder { decoder, info }),
    Err(e) => { HwCodecConfig::clear(false, false); Err(anyhow!("Failed to create decoder: {}", e)) }
}

When it happens

Trigger: Decoder open fails although it was listed: driver/session limits (max concurrent decode sessions on NVDEC/vaapi), device lost/reset, unsupported profile of the incoming stream, or a stale detection cache pointing at a device that no longer exists.

Common situations: Multiple simultaneous sessions exhausting hardware decode slots; GPU driver update or TDR reset between detection and creation; running in containers without GPU device access; virtual machines.

Related errors

• Failed to create encoder
• don't support h264!
• don't support h265!
• failed to get format from name:{}
• encoder type mismatch

AI-assisted analysis of rustdesk/rustdesk@7aa98d43cf (2026-08-16). Data as JSON: /api/errors/363bece9057b6e9e. Report an issue: GitHub.