rustdesk/rustdesk · error

rotation not supported

Error message

rotation not supported

What it means

encode_to_message extracts the captured texture plus a rotation value; if rotation != 0 it bails with `rotation not supported`. The VRAM hardware encoder path has no implementation for rotated displays — both the encoder configuration and the reported display width/height would need to change, which is still a to-do in the code. So encoding a rotated capture through the hardware path is explicitly unsupported.

Source

Thrown at libs/scrap/src/common/vram.rs:105

                        same_bad_len_counter: 0,
                    }),
                    Err(_) => Err(anyhow!(format!("Failed to create encoder"))),
                }
            }
            _ => Err(anyhow!("encoder type mismatch")),
        }
    }

    fn encode_to_message(
        &mut self,
        frame: EncodeInput,
        ms: i64,
    ) -> ResultType<base::message_proto::VideoFrame> {
        let (texture, rotation) = frame.texture()?;
        if rotation != 0 {
            // to-do: support rotation
            // Both the encoder and display(w,h) information need to be changed.
            bail!("rotation not supported");
        }
        let mut vf = VideoFrame::new();
        let mut frames = Vec::new();
        for frame in self
            .encode(texture, ms)
            .with_context(|| "Failed to encode")?
        {
            frames.push(EncodedVideoFrame {
                data: Bytes::from(frame.data),
                pts: frame.pts,
                key: frame.key == 1,
                ..Default::default()
            });
        }
        if frames.len() > 0 {
            // This kind of problem is occurred after a period of time when using AMD encoding,
            // the encoding length is fixed at about 40, and the picture is still
            const MIN_BAD_LEN: usize = 100;

View on GitHub (pinned to 91c9fccbb0)

Solutions

1. Set the remote display rotation back to 0 degrees (landscape default) so the capture reports rotation == 0.
2. Fall back to the software capture/encoder path, which handles rotated displays, instead of the VRAM path.
3. Implement rotation support in vram.rs: reconfigure the encoder and swap the reported display w/h when rotation is non-zero (the code's own to-do).

Example fix

// before
let (texture, rotation) = frame.texture()?;
if rotation != 0 {
    bail!("rotation not supported");
}
// after
let (texture, rotation) = frame.texture()?;
if rotation != 0 {
    log::warn!("rotated display on vram path, falling back to software encoder");
    return software_encoder.encode_to_message(frame, ms);
}

let (_, rotation) = frame.texture()?;
if rotation != 0 {
    // switch to software encoder path before calling vram encode_to_message
}

fn rotation_supported(rotation: i32) -> bool { rotation == 0 }

// Rust: match on Result
match vram_encoder.encode_to_message(frame, ms) {
    Ok(vf) => vf,
    Err(e) if e.to_string() == "rotation not supported" => software_encoder.encode_to_message(frame, ms)?,
    Err(e) => return Err(e),
}

When it happens

Trigger: Calling encode_to_message when the frame's texture reports a non-zero rotation — i.e. capturing a display whose orientation is rotated 90/180/270 degrees on a machine using the VRAM (hardware) encoder.

Common situations: Remote machine has a monitor in portrait mode or rotated via display settings; user rotates a monitor mid-session and the next capture carries rotation while the VRAM encoder is active.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• encoder type mismatch
• {:?} not supported
• no valid frame
• encoder type mismatch
• no valid frame

AI-assisted analysis of rustdesk/rustdesk@91c9fccbb0 (2026-09-10). Data as JSON: /api/errors/dd417b394365b007. Report an issue: GitHub.