hpcaitech/Open-Sora · error · NotImplementedError

Downsample during project_in is not supported for video

Error message

Downsample during project_in is not supported for video

What it means

When the encoder's project_in has spatial factor 2, the code falls back to a pixel-unshuffle-style downsample that is not implemented for 5D video tensors; constructing a video DCAE with in_video mode and factor=2 project_in raises NotImplementedError. Video configs should use the video-capable project_in path (factor 1 conv or the video branch).

Source

Thrown at opensora/models/dc_ae/models/dc_ae.py:269


def build_encoder_project_in_block(
    in_channels: int, out_channels: int, factor: int, downsample_block_type: str, is_video: bool
):
    if factor == 1:
        block = ConvLayer(
            in_channels=in_channels,
            out_channels=out_channels,
            kernel_size=3,
            stride=1,
            use_bias=True,
            norm=None,
            act_func=None,
            is_video=is_video,
        )
    elif factor == 2:
        if is_video:
            raise NotImplementedError("Downsample during project_in is not supported for video")
        block = build_downsample_block(
            block_type=downsample_block_type, in_channels=in_channels, out_channels=out_channels, shortcut=None
        )
    else:
        raise ValueError(f"downsample factor {factor} is not supported for encoder project in")
    return block


def build_encoder_project_out_block(
    in_channels: int,
    out_channels: int,
    norm: Optional[str],
    act: Optional[str],
    shortcut: Optional[str],
    is_video: bool,
):
    block = OpSequential(
        [

View on GitHub (pinned to 7ad6a96a13)

Solutions

  1. Set the encoder project_in factor appropriate for video configs (use the video-capable setting, typically factor 1 / conv-only)
  2. Use a video DCAE config shipped with the repo
  3. Keep is_video=False when using factor 2 project_in on images

Example fix

# before
project_in_factor: 2   # with is_video: true
# after
project_in_factor: 1
Defensive patterns

Strategy: validation

Validate before calling

if is_video:
    assert project_in_factor == 1, 'factor-2 project_in downsample is not implemented for video'

Prevention

When it happens

Trigger: Building a DCAE with is_video=True and project_in_factor: 2 in the encoder config; build_encoder_project_in_block is called from Encoder __init__.

Common situations: Reusing an image-model config (which uses factor 2 to pre-downsample) with is_video enabled for video generation/encoding.

Related errors

AI-assisted analysis of hpcaitech/Open-Sora@7ad6a96a13 (2026-08-28). Data as JSON: /api/errors/e47642be83aec8b2. Report an issue: GitHub.