ultralytics/ultralytics · error · NotImplementedError

Please use the corresponding methods in SAM2VideoPredictor f

Error message

Please use the corresponding methods in SAM2VideoPredictor for inference.See notebooks/video_predictor_example.ipynb for an example.

What it means

NotImplementedError from SAM2VideoPredictor.forward(): video segmentation models in the SAM2 family do not support plain tensor forward passes — video inference is stateful (per-frame memory, per-object prompts, propagation), so __call__/forward is intentionally disabled and the message directs users to the dedicated streaming API (init_state, add_new_points_or_box, propagate_in_video) shown in the official video predictor notebook.

Source

Thrown at ultralytics/models/sam/modules/sam.py:352

        # Model compilation
        if compile_image_encoder:
            # Compile the forward function (not the full module) to allow loading checkpoints.
            LOGGER.info("Image encoder compilation is enabled. First forward pass will be slow.")
            self.image_encoder.forward = torch.compile(
                self.image_encoder.forward,
                mode="max-autotune",
                fullgraph=True,
                dynamic=False,
            )

    @property
    def device(self):
        """Return the device on which the model's parameters are stored."""
        return next(self.parameters()).device

    def forward(self, *args, **kwargs):
        """Process image and prompt inputs to generate object masks and scores in video sequences."""
        raise NotImplementedError(
            "Please use the corresponding methods in SAM2VideoPredictor for inference."
            "See notebooks/video_predictor_example.ipynb for an example."
        )

    def _build_sam_heads(self):
        """Build SAM-style prompt encoder and mask decoder for image segmentation tasks."""
        self.sam_prompt_embed_dim = self.hidden_dim
        self.sam_image_embedding_size = self.image_size // self.backbone_stride

        # Build PromptEncoder and MaskDecoder from SAM (hyperparameters like `mask_in_chans=16` are from SAM code)
        self.sam_prompt_encoder = PromptEncoder(
            embed_dim=self.sam_prompt_embed_dim,
            image_embedding_size=(
                self.sam_image_embedding_size,
                self.sam_image_embedding_size,
            ),
            input_image_size=(self.image_size, self.image_size),
            mask_in_chans=16,

View on GitHub (pinned to 0449ea011c)

Solutions

1. Use the stateful API: state = predictor.init_state(video_path); predictor.add_new_points_or_box(state, frame_idx=0, obj_id=1, points=..., labels=...); then iterate predictor.propagate_in_video(state).
2. For single images, use the image predictor (SAM/SAM2 image model) instead of the video predictor.
3. In the ultralytics facade, use model(source=video_path) track/predict paths which drive these methods internally rather than raw forward.

Example fix

# before
out = sam2_video_predictor(frame_tensor, points)  # NotImplementedError

# after
state = sam2_video_predictor.init_state("video.mp4")
sam2_video_predictor.add_new_points_or_box(state, frame_idx=0, obj_id=1, points=np.array([[500, 375]]), labels=np.array([1]))
for out_frame_idx, out_obj_ids, out_mask_logits in sam2_video_predictor.propagate_in_video(state):
    ...

from ultralytics.models.sam.modules.sam import SAM2VideoPredictor

def is_video_predictor(predictor) -> bool:
    return isinstance(predictor, SAM2VideoPredictor)

if is_video_predictor(model):
    raise TypeError("video predictors need the stateful API, not forward()")

from ultralytics.models.sam.modules.sam import SAM2VideoPredictor

def uses_stateful_api(model) -> bool:
    """True for models that must be driven via init_state/add_prompts/propagate."""
    return isinstance(model, SAM2VideoPredictor)

When it happens

Trigger: Calling model(frames_tensor) or model(image, prompts) directly on a SAM2VideoPredictor instance — e.g. reusing generic inference code that treats every ultralytics model as callable.

Common situations: Plugging a SAM2 video model into code written for image models; wrapper frameworks that abstract inference as forward(); copy-pasting image-SAM usage onto the video predictor.

Related errors

• No points are provided; please add points first
• Cannot add new object id {obj_id} after tracking starts. All
• Cannot remove object id {obj_id} as it doesn't exist. All ex
• '{k}={v}' is invalid. Use (0.0, 1.0] for fraction; [0.0, 1.0
• RandomGridShuffle cannot preserve polygon or keypoint topolo

AI-assisted analysis of ultralytics/ultralytics@0449ea011c (2026-08-15). Data as JSON: /api/errors/b722f466ce6152d8. Report an issue: GitHub.