What causes "length < 0" in infinilabs/analysis-ik?

Q: What causes "length < 0" in infinilabs/analysis-ik?

If you construct Lexeme yourself, compute length as Math.max(0, end - begin) or assert end >= begin before calling the constructor

infinilabs/analysis-ik · error · IllegalArgumentException

length < 0

Error message

length < 0

What it means

The Lexeme constructor (a 'lexeme' is one token produced by the IK segmenter: begin offset + length + type) rejects any token whose character length is negative. A negative length is structurally impossible for a valid token, so this IllegalArgumentException signals that the caller computed begin/end offsets inconsistently. It is an argument-contract guard inside the public constructor org.wltea.analyzer.core.Lexeme#Lexeme(int,int,int,int).

Source

Thrown at core/src/main/java/org/wltea/analyzer/core/Lexeme.java:69

	public static final int TYPE_CQUAN = 48;
	
	//词元的起始位移
	private int offset;
    //词元的相对起始位置
    private int begin;
    //词元的长度
    private int length;
    //词元文本
    private String lexemeText;
    //词元类型
    private int lexemeType;
    
    
	public Lexeme(int offset , int begin , int length , int lexemeType){
		this.offset = offset;
		this.begin = begin;
		if(length < 0){
			throw new IllegalArgumentException("length < 0");
		}
		this.length = length;
		this.lexemeType = lexemeType;
	}
	
    /*
     * 判断词元相等算法
     * 起始位置偏移、起始位置、终止位置相同
     * @see java.lang.Object#equals(Object o)
     */
	public boolean equals(Object o){
		if(o == null){
			return false;
		}
		
		if(this == o){
			return true;
		}

View on GitHub (pinned to 6d2d70fd1a)

Solutions

If you construct Lexeme yourself, compute length as Math.max(0, end - begin) or assert end >= begin before calling the constructor
Audit the two inputs that produce length: in library-internal paths the exception means the AnalyzeContext cursor/offset bookkeeping got corrupted — capture the input text and the segmenter that was active (useSmart on/off) and reduce to a minimal repro
If reproducible against the stock segmenters with normal Chinese text, file an upstream issue with the failing input string — stock paths should never produce negative lengths
Never wrap the call in a silent catch that substitutes length=0; that masks offset corruption and produces wrong term positions downstream

Example fix

// before
int end = computeEnd();
Lexeme l = new Lexeme(offset, begin, end - begin, Lexeme.TYPE_CJK_NORMAL); // throws when end < begin

// after
int end = computeEnd();
if (end < begin) throw new IllegalStateException("corrupted offsets: begin=" + begin + ", end=" + end);
Lexeme l = new Lexeme(offset, begin, end - begin, Lexeme.TYPE_CJK_NORMAL);

Defensive patterns

Strategy: validation

Validate before calling

if (end - begin < 0) {
    throw new IllegalStateException("invalid lexeme bounds: begin=" + begin + ", end=" + end);
}
Lexeme lexeme = new Lexeme(offset, begin, end - begin, Lexeme.TYPE_CJK_NORMAL);

Type guard

private static boolean isValidLexemeLength(int begin, int end) {
    return end >= begin; // length = end - begin must be >= 0
}

Try / catch

try {
    Lexeme lexeme = new Lexeme(offset, begin, length, type);
} catch (IllegalArgumentException e) {
    if (!"length < 0".equals(e.getMessage())) throw e;
    throw new IllegalStateException("segmenter produced inverted bounds at offset " + offset, e);
}

Prevention

Treat negative length as a symptom of corrupted offset bookkeeping, never as a data value to sanitize
When writing custom segmenters, always derive length from end - begin and assert end >= begin
Log the input buffer window around the cursor when the exception fires — it identifies the offending segmenter

When it happens

Trigger: Calling new Lexeme(offset, begin, length, lexemeType) with a negative length argument. Inside the library this happens when a segmenter or LexemePath merge computes length as (end - begin) while end < begin — e.g. overlapping lexemes merged in the wrong order, or a custom segmenter appending Lexeme objects with hand-computed offsets. User code that builds Lexeme instances directly (custom similarity/LexemePath experiments) and passes begin > end arithmetic hits it immediately.

Common situations: Writing a custom Segmenter plugin that emits Lexemes with begin beyond the current cursor; porting code from an IK version where length was unchecked; unit tests that construct Lexemes with placeholder offsets like (0, 5, -1, 0); downstream code that derives length from two independently computed positions that drift apart when the char buffer is refilled.

Related errors

AI-assisted analysis of infinilabs/analysis-ik@6d2d70fd1a (2026-08-14). Data as JSON: /api/errors/dafe021b2026f021. Report an issue: GitHub.