Summary 04 — IR Modeling, Part II (Probabilistic Model & Alternative Models)

Source: 03_Modeling_Part-II.pdf

1. Table of Contents (Topics Covered)

Gist (2–3 paragraphs)

The Probabilistic Model frames retrieval as estimating the probability that a document is relevant to a query. It imagines an ideal answer set R containing exactly the relevant documents; querying specifies the properties of R. Using Bayes’ rule, assuming binary weights and term independence, the similarity reduces to a sum over query terms of log-odds of a term appearing in relevant vs. non-relevant documents. Since R is unknown initially, estimates are bootstrapped (often R=ri=0), yielding an IDF-like ranking; with relevance data the Robertson–Sparck Jones equation applies (with +0.5 smoothing). Its weaknesses: needs initial guesses, ignores TF, and lacks document length normalization.

Among the three classic models, Boolean is weakest (no partial match); the Vector model generally outperforms the probabilistic model on general collections, though this is debated. Because each classic model has limits, many alternative models exist.

These alternatives span three families. Set-theoretic: Set-Based (exploits term dependence via termsets), Extended Boolean, Fuzzy Set. Algebraic: Generalized Vector (non-orthogonal terms), Latent Semantic Indexing (retrieve by concepts not just terms, via LSA), Neural Network models. Probabilistic: BM25 (adds TF + length normalization to the probabilistic model — the modern workhorse), Language Models (probability of generating the query), Divergence from Randomness, and Bayesian Networks (combining evidence). The lecture also previews Web-based models (PageRank, Hubs-Authorities), Hypertext, Structured Text, Multimedia, and Enterprise/Vertical search.

2. Key Terminologies — Meaning & Use Cases

Term Meaning Use Case
Probabilistic Model Ranks by estimated probability of relevance Foundation of BM25
Ideal answer set R The (unknown) set of exactly relevant docs Target of estimation
Probabilistic Ranking Principle Rank by decreasing probability of relevance Optimal ordering theory
Bayes’ rule Inverts conditional probabilities Deriving the ranking
Contingency table N, ni, R, ri counts for term incidence Estimating probabilities
Robertson–Sparck Jones eq. Classic probabilistic ranking formula (+0.5 smoothing) Ranking with relevance info
BM25 (Okapi) Probabilistic ranking with TF, IDF & length normalization Modern default baseline
Language Model Probability distribution over word sequences Query-likelihood retrieval
Latent Semantic Indexing (LSI) Retrieval by latent concepts (LSA) Matching synonyms/concepts
Generalized Vector Model Allows non-orthogonal term vectors Capturing term correlation
Set-Based Model Uses termsets (correlated term groups) Exploiting dependence
Fuzzy Set Model Degree of membership for vague matching Approximate matching
Divergence from Randomness Weight = divergence of actual vs. random term distribution Parameter-light weighting
Bayesian Network Model Combines multiple evidence sources cleanly Feedback, multi-query
Hypertext Model Nodes + directed links (graph) for non-sequential reading HTML / web navigation

3. Process Workflow Diagram

flowchart TD
    A[User Query] --> B[Retrieve initial doc set]
    B --> C[User inspects top docs
marks relevant]
    C --> D[Estimate piR, qiR
contingency table]
    D --> E[Compute Robertson-Sparck Jones score]
    E --> F[Re-rank documents]
    F --> G{Refine R?}
    G -- Yes --> C
    G -- No --> H[Final ranking]
    subgraph NoRelevanceInfo[Bootstrap: no relevance data]
        I[Assume R=ri=0] --> J[IDF-like ranking]
    end

4. ELI5 — Complex Terms Explained Simply