projectdiscovery/katana · error

failed to get origin page state: %w

Error message

failed to get origin page state: %w

What it means

isCorrectNavigation failed to load the origin page state from the crawl graph before comparing SimHash distances. This error wraps the underlying GetPageState failure, so the cause is whatever the crawl graph store reported (typically the origin vertex no longer exists or the store errored). Navigation correctness cannot be evaluated without the origin state, so the check aborts.

Source

Thrown at pkg/engine/headless/crawler/state.go:35

var emptyPageHash = sha256Hash("")

const simhashThreshold = 2 // Allow up to 2 bits difference

func (c *Crawler) isCorrectNavigation(page *browser.BrowserPage, action *types.Action) (string, *types.PageState, error) {
	currentPageHash, pageState, err := getPageHash(page)
	if err != nil {
		return "", nil, err
	}

	if currentPageHash == action.OriginID {
		return currentPageHash, pageState, nil
	}

	// Get the origin page state to compare SimHash
	originPageState, err := c.crawlGraph.GetPageState(action.OriginID)
	if err != nil {
		return "", pageState, fmt.Errorf("failed to get origin page state: %w", err)
	}

	if pageState != nil && originPageState != nil {
		distance := simhash.Distance(pageState.SimHash, originPageState.SimHash)
		if distance <= simhashThreshold {
			c.logger.Debug("Page is similar enough to origin, proceeding",
				slog.String("current_hash", currentPageHash),
				slog.String("origin_hash", action.OriginID),
				slog.Uint64("simhash_distance", uint64(distance)),
			)
			// Treat this page as the origin state to avoid creating a new vertex
			return originPageState.UniqueID, pageState, nil
		}
	}

	return "", pageState, fmt.Errorf("failed to navigate back to origin page: %s != %s", currentPageHash, action.OriginID)
}

View on GitHub (pinned to e3e742739c)

Solutions

  1. Verify action.OriginID is a valid, still-present vertex ID in the crawl graph before invoking navigation (log GetPageState for that ID).
  2. Disable or increase graph state pruning/eviction so origin states survive until their actions are processed.
  3. If resuming from persisted state, regenerate or re-crawl the origin page so its state exists in the graph.
  4. Inspect the wrapped error (%w) for the root cause (not-found vs I/O) and fix the store accordingly.

Example fix

// before
originPageState, err := c.crawlGraph.GetPageState(action.OriginID)
// after
originPageState, err := c.crawlGraph.GetPageState(action.OriginID)
if errors.Is(err, ErrPageStateNotFound) {
    c.logger.Warn("origin state missing, re-crawling origin", "origin", action.OriginID)
    if _, err := c.crawlGraph.AddPageState(action.OriginID, page); err != nil {
        return "", pageState, err
    }
    originPageState, err = c.crawlGraph.GetPageState(action.OriginID)
}
Defensive patterns

Strategy: try-catch

Validate before calling

if _, err := c.crawlGraph.GetPageState(action.OriginID); err != nil {
    // origin state unavailable; re-crawl or skip this action before navigation
}

Type guard

func hasOriginState(g *crawlGraph, id string) bool {
    s, err := g.GetPageState(id)
    return err == nil && s != nil
}

Try / catch

result, err := crawler.checkNavigation(action)
if err != nil {
    var navErr *NavigationError
    if errors.As(err, &navErr) && errors.Is(navErr.Unwrap(), ErrPageStateNotFound) {
        // re-crawl origin or mark action stale
    }
}

Prevention

When it happens

Trigger: Calling tryBrowserHistoryNavigation or tryShortestPathNavigation with an action whose OriginID references a page state that crawlGraph.GetPageState cannot return: OriginID never persisted, graph evicted/pruned the vertex, or the store returned an I/O error.

Common situations: Resuming a crawl from a persisted session file where origin vertices were pruned or never saved; using an action struct copied from an older crawl run; in-memory graph reset between navigation attempts; store backend (file/db) corruption or permission errors.

Related errors

AI-assisted analysis of projectdiscovery/katana@e3e742739c (2026-09-03). Data as JSON: /api/errors/4aca1e22e9292767. Report an issue: GitHub.