Package jsyntaxpane.lexers
Class DefaultJFlexLexer
java.lang.Object
jsyntaxpane.lexers.DefaultJFlexLexer
- All Implemented Interfaces:
Lexer
- Direct Known Subclasses:
BashLexer
,CLexer
,ClojureLexer
,CppLexer
,DOSBatchLexer
,GroovyLexer
,JavaLexer
,JavaScriptLexer
,JFlexLexer
,LuaLexer
,PropertiesLexer
,PythonLexer
,RubyLexer
,ScalaLexer
,SqlLexer
,TALLexer
,XHTMLLexer
,XmlLexer
,XPathLexer
This is a default, and abstract implemenatation of a Lexer using JFLex
with some utility methods that Lexers can implement.
-
Field Summary
FieldsModifier and TypeFieldDescriptionprotected int
protected int
protected int
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionvoid
The DefaultJFlexLexer simply calls the yylex method of a JFLex compatible Lexer and adds the tokens obtained to an ArrayList.protected Token
Create and return a Token of given type.protected Token
Create and return a Token of given type and pairValue.protected Token
Create and return a Token of given type from start with length offset is added to startprotected Token
Helper method to create and return a new Token from of TokenType tokenStart and tokenLength will be modified to the newStart and newLength paramsabstract int
yychar()
Return the char number from beginning of input stream.abstract char
yycharat
(int pos) Returns the character at position pos from the matched text.abstract int
yylength()
Returns the length of the matched text region.abstract Token
yylex()
This is called to return the next Token from the Input Readerabstract void
This will be called to reset the the lexer.abstract String
yytext()
Returns the text matched by the current regular expression.
-
Field Details
-
tokenStart
protected int tokenStart -
tokenLength
protected int tokenLength -
offset
protected int offset
-
-
Constructor Details
-
DefaultJFlexLexer
public DefaultJFlexLexer()
-
-
Method Details
-
token
Helper method to create and return a new Token from of TokenType tokenStart and tokenLength will be modified to the newStart and newLength params- Parameters:
type
-tStart
-tLength
-newStart
-newLength
-- Returns:
-
token
Create and return a Token of given type from start with length offset is added to start- Parameters:
type
-start
-length
-- Returns:
-
token
Create and return a Token of given type. start is obtained fromyychar()
and length fromyylength()
offset is added to start- Parameters:
type
-- Returns:
-
token
Create and return a Token of given type and pairValue. start is obtained fromyychar()
and length fromyylength()
offset is added to start- Parameters:
type
-pairValue
-- Returns:
-
parse
The DefaultJFlexLexer simply calls the yylex method of a JFLex compatible Lexer and adds the tokens obtained to an ArrayList.- Specified by:
parse
in interfaceLexer
- Parameters:
segment
- Text to parse.ofst
- offset to add to start of each token (useful for nesting)tokens
- List of Tokens to be added. This is done so that the caller creates the appropriate List implementation and size. The parse method just adds to the list
-
yyreset
This will be called to reset the the lexer. This is created automatically by JFlex.- Parameters:
reader
-
-
yylex
This is called to return the next Token from the Input Reader- Returns:
- next token, or null if no more tokens.
- Throws:
IOException
-
yycharat
public abstract char yycharat(int pos) Returns the character at position pos from the matched text. It is equivalent to yytext().charAt(pos), but faster- Parameters:
pos
- the position of the character to fetch. A value from 0 to yylength()-1.- Returns:
- the character at position pos
-
yylength
public abstract int yylength()Returns the length of the matched text region. This method is automatically implemented by JFlex lexers- Returns:
-
yytext
Returns the text matched by the current regular expression. This method is automatically implemented by JFlex lexers- Returns:
-
yychar
public abstract int yychar()Return the char number from beginning of input stream. This is NOT implemented by JFLex, so the code must be added to create this and return the private yychar field- Returns:
-