projectdiscovery/katana · critical

failed to create text normalizer

Error message

failed to create text normalizer

What it means

Error from normalizer.New(): NewTextNormalizer() returned an error while constructing the text normalization pipeline, wrapped here as "failed to create text normalizer". This is the root cause surfaced by the crawler package's "failed to create domnormalizer" init error, and it prevents any Normalizer (and thus any headless Crawler) from being created.

Source

Thrown at pkg/engine/headless/crawler/normalizer/normalizer.go:27

	"strings"

	"github.com/PuerkitoBio/goquery"
	"github.com/pkg/errors"
	htmlpkg "golang.org/x/net/html"
)

var whiteSpacesRegex = regexp.MustCompile(`[\r\n]+|\s+`)

type Normalizer struct {
	dom  *DOMNormalizer
	text *TextNormalizer
}

// New returns a new Normalizer
func New() (*Normalizer, error) {
	textNormalizer, err := NewTextNormalizer()
	if err != nil {
		return nil, errors.Wrap(err, "failed to create text normalizer")
	}
	domNormalizer := NewDOMNormalizer()
	return &Normalizer{
		dom:  domNormalizer,
		text: textNormalizer,
	}, nil
}

// Apply applies the normalizers to the given content
//
// It normalizes the given content by:
// - Applying the DOM normalizer
// - Applying the text normalizer
// - Denormalizing it
func (n *Normalizer) Apply(text string) (string, error) {
	first := normalizeDocument(text)

	firstpass, err := n.dom.Apply(first)

View on GitHub (pinned to e3e742739c)

Solutions

1. Reinstall katana so normalizer resources exist and are readable
2. Inspect the wrapped error (print with %+v) to identify the exact missing resource
3. Fix file permissions or working directory if assets load from disk
4. Rebuild from source ensuring embedded assets are included

// check the text normalizer can be constructed before use
if _, err := normalizer.New(); err != nil {
    log.Fatalf("normalizer unavailable: %+v", err)
}

func normalizerReady(n *normalizer.Normalizer) bool { return n != nil }

norm, err := normalizer.New()
if err != nil {
    log.Fatalf("text normalizer init failed: %+v", err)
}

When it happens

Trigger: 1) NewTextNormalizer fails to load its backing data (missing embedded/disk resources, unreadable file). 2) Corrupted or incomplete installation where expected assets are absent. 3) Environment/permission issues preventing the resource load.

Common situations: Trimmed container images; broken katana builds; running the binary from a directory without access to data files; version upgrades that moved assets.

Related errors

• failed to create domnormalizer
• captcha solver init: %w
• unsupported captcha solver provider: %s
• error compiling pattern %s: %v

AI-assisted analysis of projectdiscovery/katana@e3e742739c (2026-09-03). Data as JSON: /api/errors/43e407d9480b703b. Report an issue: GitHub.