Comfy-Org/ComfyUI · error · NotImplementedError

{type(model).__name__} must implement map_context_window_to_

Error message

{type(model).__name__} must implement map_context_window_to_modalities for multimodal context windows.

What it means

Multimodal context-window scheduling (e.g. audio+video generation) must map the primary modality's window indices onto each secondary modality. This is done by calling model.map_context_window_to_modalities(...); the code catches AttributeError and re-raises NotImplementedError, so any model class that lacks that method cannot be used with a multimodal ContextWindowChunker — only specific multimodal models (e.g. certain Wan/Omni-style architectures) implement it.

Source

Thrown at comfy/context_windows.py:247

        Non-multimodal contexts return the input window unchanged.
        """
        if not self.is_multimodal:
            return window

        x = self.latents[0]
        primary_total = self.latent_shapes[0][self.dim]
        primary_overlap = window.context_overlap
        map_shapes = self.latent_shapes
        if x.size(self.dim) != primary_total:
            map_shapes = list(self.latent_shapes)
            video_shape = list(self.latent_shapes[0])
            video_shape[self.dim] = x.size(self.dim)
            map_shapes[0] = torch.Size(video_shape)
        try:
            per_modality_indices = model.map_context_window_to_modalities(
                window.index_list, map_shapes, self.dim)
        except AttributeError:
            raise NotImplementedError(
                f"{type(model).__name__} must implement map_context_window_to_modalities for multimodal context windows.")
        modality_windows = {}
        for mod_idx in range(1, len(self.latents)):
            modality_total_frames = self.latents[mod_idx].shape[self.dim]
            ratio = modality_total_frames / primary_total if primary_total > 0 else 1
            modality_overlap = max(round(primary_overlap * ratio), 0)
            modality_windows[mod_idx] = IndexListContextWindow(
                per_modality_indices[mod_idx], dim=self.dim,
                total_frames=modality_total_frames,
                context_overlap=modality_overlap)
        return IndexListContextWindow(
            window.index_list, dim=self.dim, total_frames=x.shape[self.dim],
            modality_windows=modality_windows, context_overlap=primary_overlap)

    def slice_for_window(self, window: IndexListContextWindow, retain_index_list: list[int], device=None) -> tuple[list[torch.Tensor], list[int]]:
        """Slice latents for a context window, injecting guide frames where applicable.
        For multimodal contexts, uses the modality-specific windows derived in prepare_window().
        """

View on GitHub (pinned to 1c6d8d45b3)

Solutions

1. Use the multimodal-capable model variant the workflow was designed for (the one implementing map_context_window_to_modalities).
2. If developing a model, implement map_context_window_to_modalities(window_indices, latent_shapes, dim) returning per-modality IndexList windows.
3. Drop the extra modality input if multimodal scheduling is not intended.

Example fix

# model integration: add the required method
class MyMultiModalModel(nn.Module):
    def map_context_window_to_modalities(self, index_list, latent_shapes, dim):
        primary = torch.tensor(index_list.index_list if hasattr(index_list, 'index_list') else index_list)
        # scale indices per modality by frame ratio
        return [primary, self._scale_to_audio(primary, latent_shapes)]

if not hasattr(model, "map_context_window_to_modalities"):
    raise SystemExit(f"{type(model).__name__} lacks multimodal context-window support")

def supports_multimodal_context_windows(model) -> bool:
    return callable(getattr(model, "map_context_window_to_modalities", None))

try:
    window = chunker.map_window(model, window, x)
except NotImplementedError as e:
    raise SystemExit("use the multimodal-capable model variant for audio+video contexts") from e

When it happens

Trigger: Building a multimodal context window (multiple latents/modalities) with a model class that does not define map_context_window_to_modalities; running an audio+video workflow where the video DiT is not the multimodal-capable variant; a custom model subclass missing the method.

Common situations: Wrong model variant loaded for an omni/multimodal workflow; new model integration not yet implementing multimodal index mapping; passing a second modality into a video-only pipeline.

Related errors

• Unknown context_schedule '{context_schedule}'.
• Unknown fuse_method '{fuse_method}'.
• ar_video sampler requires 5-D video latents [B,C,T,H,W], got
• Image-token count {image_idx.shape[0]} != ViT output count {
• MiniMax H3 supports batch size 1

AI-assisted analysis of Comfy-Org/ComfyUI@1c6d8d45b3 (2026-08-14). Data as JSON: /api/errors/a0a7247583f0e41f. Report an issue: GitHub.