Class AlphabeticTokenizer

  • All Implemented Interfaces:
    java.io.Serializable, java.util.Enumeration, OptionHandler, RevisionHandler

    public class AlphabeticTokenizer
    extends Tokenizer
    Alphabetic string tokenizer, tokens are to be formed only from contiguous alphabetic sequences.

    Version:
    $Revision: 1.2 $
    Author:
    Asrhaf M. Kibriya (amk14@cs.waikato.ac.nz), FracPete (fracpete at waikato dot ac dot nz)
    See Also:
    Serialized Form
    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      java.lang.String getRevision()
      Returns the revision string.
      java.lang.String globalInfo()
      Returns a string describing the stemmer
      boolean hasMoreElements()
      returns whether there are more elements still
      static void main​(java.lang.String[] args)
      Runs the tokenizer with the given options and strings to tokenize.
      java.lang.Object nextElement()
      returns the next element
      void tokenize​(java.lang.String s)
      Sets the string to tokenize.
      • Methods inherited from class java.lang.Object

        equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
      • Methods inherited from interface java.util.Enumeration

        asIterator
    • Constructor Detail

      • AlphabeticTokenizer

        public AlphabeticTokenizer()
    • Method Detail

      • globalInfo

        public java.lang.String globalInfo()
        Returns a string describing the stemmer
        Specified by:
        globalInfo in class Tokenizer
        Returns:
        a description suitable for displaying in the explorer/experimenter gui
      • hasMoreElements

        public boolean hasMoreElements()
        returns whether there are more elements still
        Specified by:
        hasMoreElements in interface java.util.Enumeration
        Specified by:
        hasMoreElements in class Tokenizer
        Returns:
        true if there are still more elements
      • nextElement

        public java.lang.Object nextElement()
        returns the next element
        Specified by:
        nextElement in interface java.util.Enumeration
        Specified by:
        nextElement in class Tokenizer
        Returns:
        the next element
      • tokenize

        public void tokenize​(java.lang.String s)
        Sets the string to tokenize. Tokenization happens immediately.
        Specified by:
        tokenize in class Tokenizer
        Parameters:
        s - the string to tokenize
      • getRevision

        public java.lang.String getRevision()
        Returns the revision string.
        Returns:
        the revision
      • main

        public static void main​(java.lang.String[] args)
        Runs the tokenizer with the given options and strings to tokenize. The tokens are printed to stdout.
        Parameters:
        args - the commandline options and strings to tokenize