sgl-project/sglang · error · ValueError

camera_actions event payload must be list[list[str]]

Error message

camera_actions event payload must be list[list[str]]

What it means

_validate_camera_actions rejects a camera_actions payload whose top level is not a list. The API expects list[list[str]] — one list of action strings per frame.

Source

Thrown at python/sglang/multimodal_gen/runtime/entrypoints/openai/realtime/adapters/lingbot_world_realtime_adapter.py:138

    async def on_init(
        self,
        session: GenerateSession,
        request: RealtimeVideoGenerationsRequest,
    ) -> None:
        condition_inputs = request.condition_inputs or {}
        camera_actions = condition_inputs.get(LINGBOT_CAMERA_ACTIONS_CONDITION)
        if camera_actions is not None:
            state = self._state(session)
            state.receive_camera_action_script(
                self._validate_camera_actions(camera_actions)
            )

        await save_realtime_first_frame(session, request)

    @staticmethod
    def _validate_camera_actions(payload: Any) -> list[list[str]]:
        if not isinstance(payload, list):
            raise ValueError("camera_actions event payload must be list[list[str]]")
        normalized = []
        for frame_actions in payload:
            if not isinstance(frame_actions, list):
                raise ValueError("camera_actions event payload must be list[list[str]]")
            normalized.append(list(frame_actions))
        return normalized

    def ingest_event(
        self,
        session: GenerateSession,
        event: RealtimeEvent,
    ) -> str:
        state = self._state(session)
        if event.kind == "camera_actions":
            return self._ingest_camera_actions(state, event.payload, event.event_id)
        elif event.kind == "prompt":
            return self._ingest_prompt(state, event.payload, event.event_id)
        elif event.kind == COMPOSITE_INPUT_EVENT_KIND:

View on GitHub (pinned to 0132848349)

Solutions

1. Send camera_actions as an array of arrays of strings, e.g. [["turn_left"],["zoom_in"]]
2. If you have a single script string, use the 'action' condition input instead
3. Validate the payload shape client-side before sending

Example fix

// before
{"camera_actions": "turn_left,zoom_in"}
// after
{"camera_actions": [["turn_left"],["zoom_in"]]}

assert isinstance(payload, list), 'camera_actions must be a list of frames'

def is_camera_actions(x: object) -> bool:
    return isinstance(x, list) and all(isinstance(f, list) and all(isinstance(a, str) for a in f) for f in x)

When it happens

Trigger: on_init receiving condition_inputs.camera_actions as a dict/string, or ingest_event with a camera_actions payload that is not a JSON array.

Common situations: Sending camera actions as an object keyed by frame ({"0": [...]}) or a flat string script instead of nested arrays; schema drift between client and adapter.

Related errors

• composite_input input_types must contain non-empty strings
• prompt event payload must be a string
• prompt event payload must be a non-empty string
• composite_input event payload must be a map
• composite_input event payload requires non-empty input_types

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2d2b8f73a16f3274. Report an issue: GitHub.