Class DefaultJFlexLexer

java.lang.Object
jsyntaxpane.lexers.DefaultJFlexLexer
All Implemented Interfaces:
Lexer
Direct Known Subclasses:
BashLexer, CLexer, ClojureLexer, CppLexer, DOSBatchLexer, GroovyLexer, JavaLexer, JavaScriptLexer, JFlexLexer, LuaLexer, PropertiesLexer, PythonLexer, RubyLexer, ScalaLexer, SqlLexer, TALLexer, XHTMLLexer, XmlLexer, XPathLexer

public abstract class DefaultJFlexLexer extends Object implements Lexer
This is a default, and abstract implemenatation of a Lexer using JFLex with some utility methods that Lexers can implement.
  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected int
     
    protected int
     
    protected int
     
  • Constructor Summary

    Constructors
    Constructor
    Description
     
  • Method Summary

    Modifier and Type
    Method
    Description
    void
    parse(Segment segment, int ofst, List<Token> tokens)
    The DefaultJFlexLexer simply calls the yylex method of a JFLex compatible Lexer and adds the tokens obtained to an ArrayList.
    protected Token
    Create and return a Token of given type.
    protected Token
    token(TokenType type, int pairValue)
    Create and return a Token of given type and pairValue.
    protected Token
    token(TokenType type, int start, int length)
    Create and return a Token of given type from start with length offset is added to start
    protected Token
    token(TokenType type, int tStart, int tLength, int newStart, int newLength)
    Helper method to create and return a new Token from of TokenType tokenStart and tokenLength will be modified to the newStart and newLength params
    abstract int
    Return the char number from beginning of input stream.
    abstract char
    yycharat(int pos)
    Returns the character at position pos from the matched text.
    abstract int
    Returns the length of the matched text region.
    abstract Token
    This is called to return the next Token from the Input Reader
    abstract void
    yyreset(Reader reader)
    This will be called to reset the the lexer.
    abstract String
    Returns the text matched by the current regular expression.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
  • Field Details

    • tokenStart

      protected int tokenStart
    • tokenLength

      protected int tokenLength
    • offset

      protected int offset
  • Constructor Details

    • DefaultJFlexLexer

      public DefaultJFlexLexer()
  • Method Details

    • token

      protected Token token(TokenType type, int tStart, int tLength, int newStart, int newLength)
      Helper method to create and return a new Token from of TokenType tokenStart and tokenLength will be modified to the newStart and newLength params
      Parameters:
      type -
      tStart -
      tLength -
      newStart -
      newLength -
      Returns:
    • token

      protected Token token(TokenType type, int start, int length)
      Create and return a Token of given type from start with length offset is added to start
      Parameters:
      type -
      start -
      length -
      Returns:
    • token

      protected Token token(TokenType type)
      Create and return a Token of given type. start is obtained from yychar() and length from yylength() offset is added to start
      Parameters:
      type -
      Returns:
    • token

      protected Token token(TokenType type, int pairValue)
      Create and return a Token of given type and pairValue. start is obtained from yychar() and length from yylength() offset is added to start
      Parameters:
      type -
      pairValue -
      Returns:
    • parse

      public void parse(Segment segment, int ofst, List<Token> tokens)
      The DefaultJFlexLexer simply calls the yylex method of a JFLex compatible Lexer and adds the tokens obtained to an ArrayList.
      Specified by:
      parse in interface Lexer
      Parameters:
      segment - Text to parse.
      ofst - offset to add to start of each token (useful for nesting)
      tokens - List of Tokens to be added. This is done so that the caller creates the appropriate List implementation and size. The parse method just adds to the list
    • yyreset

      public abstract void yyreset(Reader reader)
      This will be called to reset the the lexer. This is created automatically by JFlex.
      Parameters:
      reader -
    • yylex

      public abstract Token yylex() throws IOException
      This is called to return the next Token from the Input Reader
      Returns:
      next token, or null if no more tokens.
      Throws:
      IOException
    • yycharat

      public abstract char yycharat(int pos)
      Returns the character at position pos from the matched text. It is equivalent to yytext().charAt(pos), but faster
      Parameters:
      pos - the position of the character to fetch. A value from 0 to yylength()-1.
      Returns:
      the character at position pos
    • yylength

      public abstract int yylength()
      Returns the length of the matched text region. This method is automatically implemented by JFlex lexers
      Returns:
    • yytext

      public abstract String yytext()
      Returns the text matched by the current regular expression. This method is automatically implemented by JFlex lexers
      Returns:
    • yychar

      public abstract int yychar()
      Return the char number from beginning of input stream. This is NOT implemented by JFLex, so the code must be added to create this and return the private yychar field
      Returns: