jackwener/OpenCLI · error · ArgumentError

hf datasets limit must be <= 100

Error message

hf datasets limit must be <= 100

What it means

The hf datasets command caps limit at 100 to keep the Hugging Face API query bounded; any limit above 100 throws ArgumentError. The cap mirrors the practical page-size limit of the datasets listing endpoint.

Source

Thrown at clis/hf/datasets.js:37

    browser: false,
    args: [
        { name: 'sort', type: 'string', default: 'downloads', help: `Sort key: ${SORT_OPTIONS.join(', ')}` },
        { name: 'search', type: 'string', required: false, help: 'Optional name/owner substring filter.' },
        { name: 'limit', type: 'int', default: 20, help: 'Max datasets (max 100; one API page).' },
    ],
    columns: ['rank', 'id', 'author', 'downloads', 'likes', 'tags', 'lastModified', 'url'],
    func: async (args) => {
        const sortRaw = String(args.sort ?? 'downloads').toLowerCase();
        const sort = SORT_ALIAS[sortRaw] ?? sortRaw;
        if (!SORT_OPTIONS.includes(sort)) {
            throw new ArgumentError(`hf datasets sort must be one of ${SORT_OPTIONS.join(', ')}`);
        }
        const limit = Number(args.limit ?? 20);
        if (!Number.isInteger(limit) || limit <= 0) {
            throw new ArgumentError('hf datasets limit must be a positive integer');
        }
        if (limit > 100) {
            throw new ArgumentError('hf datasets limit must be <= 100');
        }

        const url = new URL('https://huggingface.co/api/datasets');
        url.searchParams.set('sort', sort);
        url.searchParams.set('direction', '-1');
        url.searchParams.set('limit', String(limit));
        url.searchParams.set('full', 'true');
        if (args.search) url.searchParams.set('search', String(args.search));

        let resp;
        try {
            resp = await fetch(url, {
                headers: {
                    'Accept': 'application/json',
                    'User-Agent': 'opencli/1.0 (+https://github.com/jackwener/opencli)',
                },
            });
        } catch (error) {

View on GitHub (pinned to 49907e53dc)

Solutions

1. Use --limit 100 or less per call
2. Issue multiple calls with different sort/filter criteria to cover more datasets
3. Query the HF API directly with pagination (Link header) if you need more than 100 results
4. Adjust scripts that assumed unbounded limits

Example fix

// before
hf datasets --limit 500
// after
hf datasets --limit 100  # repeat with different filters as needed

const limit = Number(rawLimit ?? 20);
if (limit > 100) throw new Error('limit must be <= 100');

function isBoundedLimit(v) { const n = Number(v); return Number.isInteger(n) && n > 0 && n <= 100; }

try { await hfDatasets({ limit }); } catch (e) { if (e instanceof ArgumentError && e.message.includes('limit must be <= 100')) { limit = 100; return hfDatasets({ limit }); } throw e; }

When it happens

Trigger: Calling hf datasets with --limit 101 or higher, e.g. --limit 500 to try to dump many datasets in one call.

Common situations: Users attempting bulk exports in one request; scripts with large default limits; misunderstanding that the CLI paginates (it does not — it slices a single API call).

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• ${label} must be <= ${maxValue}
• dblp ${label} must be <= ${maxValue}
• ${label} is required
• ${label} must be a positive integer
• arxiv ${label} must be <= ${maxValue}

AI-assisted analysis of jackwener/OpenCLI@49907e53dc (2026-08-29). Data as JSON: /api/errors/06bc8493a7839330. Report an issue: GitHub.