Class LMJelinekMercerSimilarity
java.lang.Object
org.apache.lucene.search.similarities.Similarity
org.apache.lucene.search.similarities.SimilarityBase
org.apache.lucene.search.similarities.LMSimilarity
org.apache.lucene.search.similarities.LMJelinekMercerSimilarity
Language model based on the Jelinek-Mercer smoothing method. From Chengxiang Zhai and John
Lafferty. 2001. A study of smoothing methods for language models applied to Ad Hoc information
retrieval. In Proceedings of the 24th annual international ACM SIGIR conference on Research and
development in information retrieval (SIGIR '01). ACM, New York, NY, USA, 334-342.
The model has a single parameter, λ. According to said paper, the optimal value depends
on both the collection and the query. The optimal value is around 0.1 for title queries
and 0.7 for long queries.
Values should be between 0 (exclusive) and 1 (inclusive). Values near zero act score more like a conjunction (coordinate level matching), whereas values near 1 behave the opposite (more like pure disjunction).
- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.lucene.search.similarities.LMSimilarity
LMSimilarity.CollectionModel, LMSimilarity.DefaultCollectionModel, LMSimilarity.LMStatsNested classes/interfaces inherited from class org.apache.lucene.search.similarities.Similarity
Similarity.SimScorer -
Field Summary
Fields inherited from class org.apache.lucene.search.similarities.LMSimilarity
collectionModel -
Constructor Summary
ConstructorsConstructorDescriptionLMJelinekMercerSimilarity(float lambda) Instantiates with the specified λ parameter.LMJelinekMercerSimilarity(LMSimilarity.CollectionModel collectionModel, boolean discountOverlaps, float lambda) Instantiates with the specified collectionModel and parameters.LMJelinekMercerSimilarity(LMSimilarity.CollectionModel collectionModel, float lambda) Instantiates with the specified collectionModel and λ parameter. -
Method Summary
Modifier and TypeMethodDescriptionprotected voidexplain(List<Explanation> subs, BasicStats stats, double freq, double docLen) Subclasses should implement this method to explain the score.protected Explanationexplain(BasicStats stats, Explanation freq, double docLen) Explains the score.floatReturns the λ parameter.getName()Returns the name of the LM method.protected doublescore(BasicStats stats, double freq, double docLen) Scores the documentdoc.Methods inherited from class org.apache.lucene.search.similarities.LMSimilarity
fillBasicStats, newStats, toStringMethods inherited from class org.apache.lucene.search.similarities.SimilarityBase
log2, scorerMethods inherited from class org.apache.lucene.search.similarities.Similarity
computeNorm, getDiscountOverlaps
-
Constructor Details
-
LMJelinekMercerSimilarity
Instantiates with the specified collectionModel and λ parameter. -
LMJelinekMercerSimilarity
public LMJelinekMercerSimilarity(LMSimilarity.CollectionModel collectionModel, boolean discountOverlaps, float lambda) Instantiates with the specified collectionModel and parameters. -
LMJelinekMercerSimilarity
public LMJelinekMercerSimilarity(float lambda) Instantiates with the specified λ parameter.
-
-
Method Details
-
score
Description copied from class:SimilarityBaseScores the documentdoc.Subclasses must apply their scoring formula in this class.
- Specified by:
scorein classSimilarityBase- Parameters:
stats- the corpus level statistics.freq- the term frequency.docLen- the document length.- Returns:
- the score.
-
explain
Description copied from class:SimilarityBaseSubclasses should implement this method to explain the score.explalready contains the score, the name of the class and the doc id, as well as the term frequency and its explanation; subclasses can add additional clauses to explain details of their scoring formulae.The default implementation does nothing.
- Overrides:
explainin classLMSimilarity- Parameters:
subs- the list of details of the explanation to extendstats- the corpus level statistics.freq- the term frequency.docLen- the document length.
-
explain
Description copied from class:SimilarityBaseExplains the score. The implementation here provides a basic explanation in the format score(name-of-similarity, doc=doc-id, freq=term-frequency), computed from:, and attaches the score (computed via theSimilarityBase.score(BasicStats, double, double)method) and the explanation for the term frequency. Subclasses content with this format may add additional details inSimilarityBase.explain(List, BasicStats, double, double).- Overrides:
explainin classSimilarityBase- Parameters:
stats- the corpus level statistics.freq- the term frequency and its explanation.docLen- the document length.- Returns:
- the explanation.
-
getLambda
public float getLambda()Returns the λ parameter. -
getName
Description copied from class:LMSimilarityReturns the name of the LM method. The values of the parameters should be included as well.Used in
LMSimilarity.toString().- Specified by:
getNamein classLMSimilarity
-