Unity-Technologies/ml-agents · error · BufferException

Unable to shuffle if the fields are not of same length

Error message

Unable to shuffle if the fields are not of same length

What it means

AgentBuffer.shuffle randomly permutes whole sequences across the requested fields; this only makes sense if all fields have the same number of elements so rows stay aligned. check_length detects a mismatch and raises BufferException.

Source

Thrown at ml-agents/mlagents/trainers/buffer.py:394

            if key not in self._fields:
                return False
            if (length is not None) and (length != len(self[key])):
                return False
            length = len(self[key])
        return True

    def shuffle(
        self, sequence_length: int, key_list: List[AgentBufferKey] = None
    ) -> None:
        """
        Shuffles the fields in key_list in a consistent way: The reordering will
        be the same across fields.
        :param key_list: The fields that must be shuffled.
        """
        if key_list is None:
            key_list = list(self._fields.keys())
        if not self.check_length(key_list):
            raise BufferException(
                "Unable to shuffle if the fields are not of same length"
            )
        s = np.arange(len(self[key_list[0]]) // sequence_length)
        np.random.shuffle(s)
        for key in key_list:
            buffer_field = self[key]
            tmp: List[np.ndarray] = []
            for i in s:
                tmp += buffer_field[i * sequence_length : (i + 1) * sequence_length]
            buffer_field.set(tmp)

    def make_mini_batch(self, start: int, end: int) -> "AgentBuffer":
        """
        Creates a mini-batch from buffer.
        :param start: Starting index of buffer.
        :param end: Ending index of buffer.
        :return: Dict of mini batch.
        """

View on GitHub (pinned to 3ecb446f75)

Solutions

1. Check that every key in key_list exists and has identical length before shuffling: buffer.check_length(key_list)
2. Verify all experience-collection code appends to every field on every step
3. Rebuild the buffer from a complete episode set, dropping fields with missing entries

Example fix

// before
buffer.shuffle(sequence_length)  # fields unequal
// after
key_list = [k for k in buffer.keys() if len(buffer[k]) == buffer[buffer.keys()[0]].length]
assert buffer.check_length(key_list)
buffer.shuffle(sequence_length, key_list)

key_list = key_list or list(buffer._fields.keys())
lengths = {k: len(buffer[k]) for k in key_list}
if len(set(lengths.values())) != 1:
    raise ValueError(f"Unequal field lengths before shuffle: {lengths}")
buffer.shuffle(sequence_length, key_list)

try:
    buffer.shuffle(sequence_length, key_list)
except BufferException:
    key_list = [k for k in key_list if len(buffer[k]) == min(len(buffer[j]) for j in key_list)]
    buffer.shuffle(sequence_length, key_list)

When it happens

Trigger: Calling buffer.shuffle(sequence_length, key_list) where one of the fields was appended fewer/more times than the others (e.g. missing reward fields for some agents, or a partial update_batch).

Common situations: Training with multiple reward signals where one signal produced no entries, custom experience-collection code appending to only some buffer fields, or off-by-one sequence_length handling.

Related errors

• The length of the fields {key_list} were not of same length
• {key} has type ({type(key0)}, {type(key1)})
• {key} is a {type(key)}
• Unable to convert {encoded_key} to an AgentBufferKey
• Index out of bounds, expected a number between 0 and {Length

AI-assisted analysis of Unity-Technologies/ml-agents@3ecb446f75 (2026-09-02). Data as JSON: /api/errors/f2243f510cc8ce65. Report an issue: GitHub.