shareAI-lab/learn-claude-code · error · ValueError

consolidation returned empty or duplicate records

Error message

consolidation returned empty or duplicate records

What it means

Raised during memory consolidation in s09_memory/code.py:487 as a self-check on the model's response. After the consolidating LLM returns a JSON array, each item is validated with validate_memory_record(); if every item fails validation (consolidated is empty) or two surviving records slugify to the same filename (duplicate names), the routine aborts before deleting any existing files. The duplicate check exists because each record is written as {slug}.md and duplicates would overwrite each other.

Source

Thrown at s09_memory/code.py:487

        if len(catalog) > CONSOLIDATE_INPUT_CHAR_LIMIT:
            raise ValueError(
                "memory store is too large for one consolidation pass"
            )
        response = client.messages.create(
            model=MODEL,
            messages=[{"role": "user", "content": prompt}],
            max_tokens=3000,
        )
        consolidated = [
            validated
            for item in extract_json_array(
                message_text({"content": response.content})
            )
            if (validated := validate_memory_record(item)) is not None
        ]
        slugs = [memory_slug(record["name"]) for record in consolidated]
        if not consolidated or len(slugs) != len(set(slugs)):
            raise ValueError(
                "consolidation returned empty or duplicate records"
            )

        snapshot = {
            record["filename"]: memory_path(record["filename"]).read_text()
            for record in records
        }
        try:
            for path in MEMORY_DIR.glob("*.md"):
                if path.name != MEMORY_INDEX.name:
                    try:
                        memory_path(path.name).unlink()
                    except ValueError:
                        continue
            for record in consolidated:
                path = memory_path(f"{memory_slug(record['name'])}.md")
                path.write_text(memory_document(
                    record["name"],

View on GitHub (pinned to 985456f4ad)

Solutions

1. Retry consolidation once or twice — output variance often resolves a schema miss; consider lowering the record count or adding an explicit schema echo to the prompt.
2. Strengthen the prompt: require each item to have exactly name, type in {user,feedback,project,reference}, description, body, and unique names.
3. Post-process instead of failing: on duplicate slugs, merge the two records deterministically (append bodies) rather than aborting.
4. Switch to a model/temperature setting that adheres to JSON structure more reliably.

Example fix

# before
if not consolidated or len(slugs) != len(set(slugs)):
    raise ValueError('consolidation returned empty or duplicate records')

# after: retry once, then merge duplicates
for attempt in range(2):
    consolidated = call_consolidate(client, prompt)
    slugs = [memory_slug(r['name']) for r in consolidated]
    if consolidated and len(slugs) == len(set(slugs)):
        break
else:
    merged = merge_by_slug(consolidated)  # append bodies on slug collision
    consolidated = merged

def consolidation_is_usable(records) -> bool:
    if not records:
        return False
    slugs = [memory_slug(r['name']) for r in records]
    return len(slugs) == len(set(slugs))

for attempt in range(3):
    try:
        return run_consolidation(client, records)
    except ValueError as e:
        if 'empty or duplicate' not in str(e) or attempt == 2:
            raise
        time.sleep(1)  # vary output on retry

When it happens

Trigger: The model returns malformed items that all fail validate_memory_record (missing name/type/description/body, unknown type); the model returns an empty array; the model merges two records but keeps identical or slug-colliding names (e.g. 'Editor prefs' and 'editor-prefs'), making len(slugs) != len(set(slugs)). It only triggers after a successful API call.

Common situations: Non-deterministic model output drifting outside the requested schema; names differing only in punctuation/case colliding after memory_slug normalization; weaker models ignoring the 'at most 30 records, fields name/type/description/body' instruction.

Related errors

• memory store is too large for one consolidation pass
• Invalid memory filename: {filename}
• The memory index is not a memory record
• Memory directory escapes the workspace
• Memory name cannot be empty

AI-assisted analysis of shareAI-lab/learn-claude-code@985456f4ad (2026-08-14). Data as JSON: /api/errors/efb77c8c29c9ca0e. Report an issue: GitHub.