java.lang.Cloneable, TokenizerTokenFilter.FileTokenizerpublic class FileTokenizer extends ProjectComponent implements Tokenizer
description, location, project| Constructor | Description |
|---|---|
FileTokenizer() |
| Modifier and Type | Method | Description |
|---|---|---|
java.lang.String |
getPostToken() |
Return the intra-token string
|
java.lang.String |
getToken(java.io.Reader in) |
Get the complete input as a string
|
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitclone, getDescription, getLocation, getProject, log, log, setDescription, setLocation, setProjectpublic java.lang.String getToken(java.io.Reader in)
throws java.io.IOException
public java.lang.String getPostToken()
getPostToken in interface Tokenizer