LykosAI/StabilityMatrix · error · InvalidOperationException

CivArchive list page was missing pageProps

Error message

CivArchive list page was missing pageProps

What it means

SearchAsync in CivArchiveApiClient throws InvalidOperationException when the Next.js __NEXT_DATA__ payload fetched from a CivArchive list/search page has no PageProps property. The client relies on pageProps to carry search results and filters; its absence means the page shape changed or the request did not return a real list page.

Solutions

1. Check the raw JSON the client fetches to see if pageProps moved or was renamed
2. Update CivArchiveApiClient / CivArchiveListPageResponse to match the current site schema
3. Retry later if the site is temporarily serving an error page
4. Report/track the mismatch since this is a third-party site scraping dependency

Example fix

// before
var pageProps = response.PageProps ?? throw new InvalidOperationException("CivArchive list page was missing pageProps");
// after
if (response.PageProps is null) { log.Warn("CivArchive list page had no pageProps"); return CivArchiveSearchResponse.Empty; }

var html = await http.GetStringAsync(searchUrl);
bool hasPageProps = html.Contains("__NEXT_DATA__") && html.Contains("pageProps");

bool HasPageProps(CivArchiveListPageResponse r) => r?.PageProps is not null;

try { return await api.SearchAsync(filters, ct); }
catch (InvalidOperationException) { return cachedResults ?? CivArchiveSearchResponse.Empty; }

When it happens

Trigger: Calling CivArchiveApiClient.SearchAsync when the scraped /search page's __NEXT_DATA__ JSON lacks a pageProps object (page moved, site redesign, redirect/error page returned instead of the list).

Common situations: CivArchive website layout changes breaking the scraped contract, CDN serving an error/challenge page, network middleware returning HTML other than the expected Next.js page.

Related errors

• CivArchive detail page was missing pageProps
• CivArchive detail page was missing model data
• GetUserAccount did not contain an id
• Relative URL is required
• Relative URL did not contain a route path

AI-assisted analysis of LykosAI/StabilityMatrix@af93d6ef57 (2026-09-12). Data as JSON: /api/errors/81d0935ad2005964. Report an issue: GitHub.

Appendix: source

Thrown at StabilityMatrix.Core/Api/CivArchiveApiClient.cs:91

    {
        ArgumentNullException.ThrowIfNull(filters);

        // /search is the only list route without server-side fixed filters — the curated
        // routes (/top-models, /hot-models, …) force their own period/sort and silently
        // override whatever the query string says (e.g. /top-models pins period=quarter).
        var routePath = string.IsNullOrWhiteSpace(filters.RoutePath) ? "/search" : filters.RoutePath;
        var relativePath = BuildSearchDataPath(routePath, filters);

        if (searchCache.Get(relativePath) is { } cached && cached.IsFresh(SearchCacheTtl))
        {
            return cached.Value;
        }

        var response = await GetNextDataAsync<CivArchiveListPageResponse>(relativePath, cancellationToken);

        var pageProps =
            response.PageProps
            ?? throw new InvalidOperationException("CivArchive list page was missing pageProps");

        var effectiveFilters = pageProps.Filters?.ToSearchFilters() ?? filters;

        var result = new CivArchiveSearchResponse
        {
            Results = pageProps.Data?.Results ?? [],
            FilterOptions = new CivArchiveFilterOptions
            {
                BaseModels = pageProps.FilterOptions?.BaseModels ?? [],
                ModelTypes = pageProps.FilterOptions?.ModelTypes ?? [],
            },
            EffectiveFilters = effectiveFilters,
            CanonicalUrl = pageProps.CanonicalUrl ?? string.Empty,
            TotalHits = pageProps.Data?.TotalHits ?? 0,
        };

        searchCache.Add(
            relativePath,

View on GitHub (pinned to af93d6ef57)