Package ddf.catalog.pubsub.criteria.contextual

Class ContextualAnalyzer

java.lang.Object
org.apache.lucene.analysis.Analyzer
ddf.catalog.pubsub.criteria.contextual.ContextualAnalyzer
All Implemented Interfaces:
Closeable, AutoCloseable
public class ContextualAnalyzer extends org.apache.lucene.analysis.Analyzer
Filters StandardTokenizer with StandardFilter, LowerCaseFilter and StopFilter, using a list of English stop words.

You must specify the required Version compatibility when creating StandardAnalyzer:

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final int
    DEFAULT_MAX_TOKEN_LENGTH
    Default maximum allowed token length
    protected static final Set<?>
    STOP_WORDS_SET
    An unmodifiable set containing some common English words that are usually not useful for searching.

    Fields inherited from class org.apache.lucene.analysis.Analyzer

    overridesTokenStreamMethod

  • Constructor Summary

    Constructors
    Constructor
    Description
    ContextualAnalyzer(org.apache.lucene.util.Version matchVersion)
    Builds an analyzer with the default stop words (STOP_WORDS_SET).
    ContextualAnalyzer(org.apache.lucene.util.Version matchVersion, File stopwords)
    Builds an analyzer with the stop words from the given file.
    ContextualAnalyzer(org.apache.lucene.util.Version matchVersion, Reader stopwords)
    Builds an analyzer with the stop words from the given reader.
    ContextualAnalyzer(org.apache.lucene.util.Version matchVersion, Set<?> stopWords)
    Builds an analyzer with the given stop words.

  • Method Summary

    Modifier and Type
    Method
    Description
    int
    getMaxTokenLength()
     
    org.apache.lucene.analysis.TokenStream
    reusableTokenStream(String fieldName, Reader reader)
     
    void
    setMaxTokenLength(int length)
    Set maximum allowed token length.
    org.apache.lucene.analysis.TokenStream
    tokenStream(String fieldName, Reader reader)
    Constructs a StandardTokenizer filtered by a StandardFilter, a LowerCaseFilter and a StopFilter.

    Methods inherited from class org.apache.lucene.analysis.Analyzer

    close, getOffsetGap, getPositionIncrementGap, getPreviousTokenStream, setOverridesTokenStreamMethod, setPreviousTokenStream

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • STOP_WORDS_SET

      protected static final Set<?> STOP_WORDS_SET
      An unmodifiable set containing some common English words that are usually not useful for searching.

    • DEFAULT_MAX_TOKEN_LENGTH

      public static final int DEFAULT_MAX_TOKEN_LENGTH
      Default maximum allowed token length
      See Also:

  • Constructor Details

    • ContextualAnalyzer

      public ContextualAnalyzer(org.apache.lucene.util.Version matchVersion)
      Builds an analyzer with the default stop words (STOP_WORDS_SET).
      Parameters:
      matchVersion - Lucene version to match See {@link <a href="#version">above</a>}

    • ContextualAnalyzer

      public ContextualAnalyzer(org.apache.lucene.util.Version matchVersion, Set<?> stopWords)
      Builds an analyzer with the given stop words.
      Parameters:
      matchVersion - Lucene version to match See {@link <a href="#version">above</a>}
      stopWords - stop words

    • ContextualAnalyzer

      public ContextualAnalyzer(org.apache.lucene.util.Version matchVersion, File stopwords) throws IOException
      Builds an analyzer with the stop words from the given file.
      Parameters:
      matchVersion - Lucene version to match See {@link <a href="#version">above</a>}
      stopwords - File to read stop words from
      Throws:
      IOException
      See Also:
      • WordlistLoader.getWordSet(File)

    • ContextualAnalyzer

      public ContextualAnalyzer(org.apache.lucene.util.Version matchVersion, Reader stopwords) throws IOException
      Builds an analyzer with the stop words from the given reader.
      Parameters:
      matchVersion - Lucene version to match See {@link <a href="#version">above</a>}
      stopwords - Reader to read stop words from
      Throws:
      IOException
      See Also:
      • WordlistLoader.getWordSet(Reader)

  • Method Details

    • tokenStream

      public org.apache.lucene.analysis.TokenStream tokenStream(String fieldName, Reader reader)
      Constructs a StandardTokenizer filtered by a StandardFilter, a LowerCaseFilter and a StopFilter.
      Specified by:
      tokenStream in class org.apache.lucene.analysis.Analyzer

    • getMaxTokenLength

      public int getMaxTokenLength()
      See Also:

    • setMaxTokenLength

      public void setMaxTokenLength(int length)
      Set maximum allowed token length. If a token is seen that exceeds this length then it is discarded. This setting only takes effect the next time tokenStream or reusableTokenStream is called.

    • reusableTokenStream

      public org.apache.lucene.analysis.TokenStream reusableTokenStream(String fieldName, Reader reader) throws IOException
      Overrides:
      reusableTokenStream in class org.apache.lucene.analysis.Analyzer
      Throws:
      IOException