FileTokenizer, LineTokenizer, StringTokenizer, TokenFilter.FileTokenizer, TokenFilter.StringTokenizerpublic interface Tokenizer
| Modifier and Type | Method | Description |
|---|---|---|
java.lang.String |
getPostToken() |
return the string between tokens, after the
previous token.
|
java.lang.String |
getToken(java.io.Reader in) |
get the next token from the input stream
|
java.lang.String getToken(java.io.Reader in)
throws java.io.IOException
in - the input streamjava.io.IOException - if an error occursjava.lang.String getPostToken()