MemPalace/mempalace · error · ValueError

chunk_overlap ({chunk_overlap}) must be at most chunk_size /

Error message

chunk_overlap ({chunk_overlap}) must be at most chunk_size // 2 ({chunk_size // 2}); a larger overlap can loop forever on short-line content (#2056)

What it means

Error "chunk_overlap ({chunk_overlap}) must be at most chunk_size // 2 ({chunk_size // 2}); a larger overlap can loop forever on short-line content (#2056)" thrown in MemPalace/mempalace.

Source

Thrown at mempalace/miner.py:676

    # enforces these and falls back to defaults on bad config.json
    # values, but ``chunk_text`` is a public function — direct callers
    # (tests, library users, future caller paths) might still pass
    # values that would loop forever. Fail fast and loud rather than
    # hang. See review feedback on #1024.
    if not isinstance(chunk_size, int) or chunk_size <= 0:
        raise ValueError(f"chunk_size must be a positive int, got {chunk_size!r}")
    if not isinstance(chunk_overlap, int) or chunk_overlap < 0:
        raise ValueError(f"chunk_overlap must be a non-negative int, got {chunk_overlap!r}")
    if chunk_overlap > chunk_size // 2:
        # The windowing loop pulls ``end`` back to a boundary only when that
        # boundary is past ``start + chunk_size // 2`` (the rfind guards below),
        # so a pulled chunk always spans more than ``chunk_size // 2`` chars.
        # ``start = end - chunk_overlap`` therefore advances only while
        # ``chunk_overlap <= chunk_size // 2``; a larger overlap makes ``start``
        # stall or move backward and the loop spins forever on short-line
        # content (#2056). The largest safe value is ``chunk_size // 2`` (half,
        # rounded down for odd sizes), which still advances and stays allowed.
        raise ValueError(
            f"chunk_overlap ({chunk_overlap}) must be at most chunk_size // 2 "
            f"({chunk_size // 2}); a larger overlap can loop forever on "
            f"short-line content (#2056)"
        )
    if not isinstance(min_chunk_size, int) or min_chunk_size < 0:
        raise ValueError(f"min_chunk_size must be a non-negative int, got {min_chunk_size!r}")

    # Clean up
    content = content.strip()
    if not content:
        return []

    chunks = []
    start = 0
    chunk_index = 0

    # Running newline tallies for the 1-indexed (line_start, line_end)
    # locators emitted below. These replace the previous per-chunk

View on GitHub (pinned to 06cb6987f0)

Solutions

1. Reduce chunk_overlap to at most half of chunk_size

When it happens

Trigger: Thrown at mempalace/miner.py:676 when the library encounters an invalid state.

Common situations: Overlap larger than half the chunk size can loop forever on short-line content.

AI-assisted analysis of MemPalace/mempalace@06cb6987f0 (2026-08-15). Data as JSON: /api/errors/297140e37e48ed72. Report an issue: GitHub.