Solved : “these characters need to be tokenized” for netbeans lang development

Solved : “these characters need to be tokenized” for netbeans lang development

2018/03/10 Netbeans 0

When developing highlight for new language, the antlr grammar has to return all tokens, you can’t skip token, otherwise your netbeans will throw this exception:

java.lang.IllegalStateException: Lexer com.github.mcheung63.netbeans.antlr.syntax.antlr3.Antlr3EditorLexer@7da1fd06
returned null token but lexerInput.readLength()=1
lexer-state: null
tokenStartOffset=1101, readOffset=1102, lookaheadOffset=1103
Chars: "\n" - these characters need to be tokenized.
Fix the lexer to not return null token in this state

Leave a Reply

Your email address will not be published. Required fields are marked *