Package weka.core.tokenizers
Class AlphabeticTokenizer
- java.lang.Object
-
- weka.core.tokenizers.Tokenizer
-
- weka.core.tokenizers.AlphabeticTokenizer
-
- All Implemented Interfaces:
java.io.Serializable
,java.util.Enumeration
,OptionHandler
,RevisionHandler
public class AlphabeticTokenizer extends Tokenizer
Alphabetic string tokenizer, tokens are to be formed only from contiguous alphabetic sequences.- Version:
- $Revision: 1.2 $
- Author:
- Asrhaf M. Kibriya (amk14@cs.waikato.ac.nz), FracPete (fracpete at waikato dot ac dot nz)
- See Also:
- Serialized Form
-
-
Constructor Summary
Constructors Constructor Description AlphabeticTokenizer()
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description java.lang.String
getRevision()
Returns the revision string.java.lang.String
globalInfo()
Returns a string describing the stemmerboolean
hasMoreElements()
returns whether there are more elements stillstatic void
main(java.lang.String[] args)
Runs the tokenizer with the given options and strings to tokenize.java.lang.Object
nextElement()
returns the next elementvoid
tokenize(java.lang.String s)
Sets the string to tokenize.-
Methods inherited from class weka.core.tokenizers.Tokenizer
getOptions, listOptions, runTokenizer, setOptions, tokenize
-
-
-
-
Method Detail
-
globalInfo
public java.lang.String globalInfo()
Returns a string describing the stemmer- Specified by:
globalInfo
in classTokenizer
- Returns:
- a description suitable for displaying in the explorer/experimenter gui
-
hasMoreElements
public boolean hasMoreElements()
returns whether there are more elements still- Specified by:
hasMoreElements
in interfacejava.util.Enumeration
- Specified by:
hasMoreElements
in classTokenizer
- Returns:
- true if there are still more elements
-
nextElement
public java.lang.Object nextElement()
returns the next element- Specified by:
nextElement
in interfacejava.util.Enumeration
- Specified by:
nextElement
in classTokenizer
- Returns:
- the next element
-
tokenize
public void tokenize(java.lang.String s)
Sets the string to tokenize. Tokenization happens immediately.
-
getRevision
public java.lang.String getRevision()
Returns the revision string.- Returns:
- the revision
-
main
public static void main(java.lang.String[] args)
Runs the tokenizer with the given options and strings to tokenize. The tokens are printed to stdout.- Parameters:
args
- the commandline options and strings to tokenize
-
-