{"record":{"id":"c64cbb1d0bfb96ef","repo":"stanfordnlp/CoreNLP","slug":"bad-character-encoding","errorCode":null,"errorMessage":"Bad character encoding","messagePattern":"Bad character encoding","errorType":"exception","errorClass":"RuntimeIOException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/international/spanish/process/SpanishTokenizer.java","lineNumber":496,"sourceCode":"            writer.newLine();\n            printSpace = false;\n          }\n        } else {\n          String outputToken = toLower ? word.toLowerCase(es) : word;\n          if (onePerLine) {\n            writer.write(outputToken);\n            writer.newLine();\n          } else {\n            if (printSpace) {\n              writer.write(\" \");\n            }\n            writer.write(outputToken);\n            printSpace = true;\n          }\n        }\n      }\n    } catch (UnsupportedEncodingException e) {\n      throw new RuntimeIOException(\"Bad character encoding\", e);\n    } catch (IOException e) {\n      throw new RuntimeIOException(e);\n    }\n    long elapsedTime = System.nanoTime() - startTime;\n    double linesPerSec = (double) nLines / (elapsedTime / 1e9);\n    System.err.printf(\"Done! Tokenized %d lines (%d tokens) at %.2f lines/sec%n\", nLines, nTokens, linesPerSec);\n  } // end main()\n\n}\n","sourceCodeStart":478,"sourceCodeEnd":506,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/international/spanish/process/SpanishTokenizer.java#L478-L506","documentation":"SpanishTokenizer.main (the command-line tokenizer entry point) wraps its output loop in a try/catch; an UnsupportedEncodingException thrown while constructing the output writer with the requested encoding is rethrown as a RuntimeIOException with the message 'Bad character encoding'. This means the -encoding value given on the command line is not a supported charset name for the JVM.","triggerScenarios":"Running java edu.stanford.nlp.international.spanish.process.SpanishTokenizer with an -encoding option whose value is not a valid charset name (e.g. 'utf8' misspellings unsupported by the JVM, 'ISO-8859-2' unavailable, or an empty/garbled value).","commonSituations":"Command-line scripts with a typo in the encoding flag value, running on a JVM with a restricted charset provider, or copying an encoding string from a locale that differs from the machine's installed charsets.","solutions":["Use a standard charset name exactly: 'UTF-8' or 'ISO-8859-1'.","Run Charset.availableCharsets() (or check JVM docs) to list charsets supported by your JVM and pick a valid one.","Omit the -encoding option to use the platform default encoding.","Ensure the file is not read with a mismatched encoding that cascades into writer construction."],"exampleFix":"// before\njava ... SpanishTokenizer -encoding utf-16leish -file in.txt\n// after\njava ... SpanishTokenizer -encoding UTF-8 -file in.txt","handlingStrategy":"validation","validationCode":"String enc = cmdLineEncoding;\nif (enc != null && !Charset.isSupported(enc)) {\n  throw new IllegalArgumentException(\"Unsupported encoding: \" + enc);\n}","typeGuard":null,"tryCatchPattern":"try {\n  SpanishTokenizer.main(args);\n} catch (RuntimeIOException e) {\n  if (\"Bad character encoding\".equals(e.getMessage())) {\n    System.err.println(\"Fix the -encoding flag; use UTF-8 or ISO-8859-1\");\n  } else throw e;\n}","preventionTips":["Always pass canonical charset names (UTF-8, ISO-8859-1) exactly as spelled by java.nio.charset.Charset.","Validate the encoding flag with Charset.isSupported() in wrapper scripts before invoking the JVM.","Prefer omitting -encoding and controlling encoding via -Dfile.encoding or the platform default."],"tags":["encoding","cli","tokenizer"],"backgroundTag":"invalid-env-var-value","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}