unslothai/unsloth · error · RuntimeError

Folder contains more than the {config.FOLDER_MAX_FILES} supp

Error message

Folder contains more than the {config.FOLDER_MAX_FILES} supported files limit

What it means

_scan aborts once the count of collected files exceeds config.FOLDER_MAX_FILES (default 10000, env RAG_FOLDER_MAX_FILES). Each found file records path/size/mtime/device/inode for reconciliation, so unbounded folders would exhaust memory and job time; the limit is checked incrementally during the walk.

Source

Thrown at studio/backend/core/rag/folder_sync.py:1007

                        st = os.lstat(full)
                        from_path = True
                    except OSError:
                        # A file that vanished mid-scan must still reach _snapshot as a failure
                        # rather than abort the scan, which is never authoritative for deletion.
                        pass
                rel = os.path.relpath(full, root).replace(os.sep, "/")
                found[rel] = {
                    "path": full,
                    "size_bytes": st.st_size,
                    "mtime_ns": st.st_mtime_ns,
                    "device": st.st_dev,
                    "inode": st.st_ino,
                    # A recovered identity is comparable to the next scan's, but not to os.fstat's:
                    # shared-folder and WebDAV drivers report different ids for the two call paths.
                    "identity_from_path": from_path,
                }
                if config.FOLDER_MAX_FILES and len(found) > config.FOLDER_MAX_FILES:
                    raise RuntimeError(
                        f"Folder contains more than the {config.FOLDER_MAX_FILES} supported files limit"
                    )
    if _root_identity(root) != identity:
        raise RuntimeError("Linked folder root identity changed during scan")
    return found, identity


def _snapshot(root: str, metadata: dict) -> str:
    source = metadata["path"]
    resolved = os.path.realpath(source)
    if not _is_within(root, resolved):
        raise RuntimeError("File escaped the linked folder")
    # os.fdopen already forces this descriptor binary on Windows; O_BINARY only guards a raw os.read.
    flags = os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_BINARY", 0)
    fd = os.open(source, flags)
    target = None
    try:
        ext = os.path.splitext(source)[1].lower()

View on GitHub (pinned to 203007d190)

Solutions

Split the data: link smaller subdirectories so each stays under the cap.
Exclude bulky subtrees (caches, node_modules, datasets) from the linked root by restructuring.
If the deployment can afford it, raise the limit via RAG_FOLDER_MAX_FILES env var and re-run the sync job.

Example fix

# before
create_folder(..., path="/data/all")  # 250k files

# after
create_folder(..., path="/data/all/docs")
# or raise the cap for this deployment:
#   RAG_FOLDER_MAX_FILES=50000

Defensive patterns

Strategy: validation

Validate before calling

import os
from core.rag import config

def file_count_ok(root: str) -> bool:
    count = 0
    for _, _, files in os.walk(root):
        count += len(files)
        if config.FOLDER_MAX_FILES and count > config.FOLDER_MAX_FILES:
            return False
    return True

Try / catch

try:
    create_folder_with_sync(...)
except RuntimeError as e:
    if "files limit" in str(e):
        show_user(f"Folder has more than {config.FOLDER_MAX_FILES} files; split or prune it.")
    else:
        raise

Prevention

Count supported files in the tree before registering it.
Keep caches, node_modules, and dataset dumps out of linked roots.
Set RAG_FOLDER_MAX_FILES deliberately per deployment instead of discovering the default mid-sync.

When it happens

Trigger: Linking a directory tree with more than 10000 supported files (media libraries, dataset dumps, node_modules included) so len(found) passes the cap mid-walk.

Common situations: First link of a large Documents/Datasets folder; accidentally linking a folder that contains caches or dependency trees; lowering RAG_FOLDER_MAX_FILES in deployment while folders previously synced fine.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/8b0411dbe37bf870. Report an issue: GitHub.