rouge-ruby/rouge · error · RuntimeError

unknown token: #{tok.inspect} for #{safe_val.inspect}

Error message

unknown token: #{tok.inspect} for #{safe_val.inspect}

What it means

Error "unknown token: #{tok.inspect} for #{safe_val.inspect}" thrown in rouge-ruby/rouge.

Source

Thrown at lib/rouge/formatters/html.rb:40

      tag 'html'

      # @yield the html output.
      def stream(tokens, &b)
        tokens.each { |tok, val| yield span(tok, val) }
      end

      def span(tok, val)
        return val if escape?(tok)

        safe_span(tok, escape_special_html_chars(val))
      end

      def safe_span(tok, safe_val)
        if tok == Token::Tokens::Text
          safe_val
        else
          shortname = tok.shortname or raise "unknown token: #{tok.inspect} for #{safe_val.inspect}"

          "<span class=\"#{shortname}\">#{safe_val}</span>"
        end
      end

      private

      # A performance-oriented helper method to escape `&`, `<` and `>` for the rendered
      # HTML from this formatter.
      #
      # `String#gsub` will always return a new string instance irrespective of whether
      # a substitution occurs. This method however invokes `String#gsub` only if
      # a substitution is imminent.
      #
      # Returns either the given `value` argument string as is or a new string with the
      # special characters replaced with their escaped counterparts.
      def escape_special_html_chars(value)
        return value unless value =~ ESCAPE_REGEX

View on GitHub (pinned to bd7dad7b09)

When it happens

Trigger: Thrown at lib/rouge/formatters/html.rb:40 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rouge-ruby/rouge@bd7dad7b09 (2026-08-23). Data as JSON: /api/errors/e3bf87c77648e35e. Report an issue: GitHub.