Summary 03 — IR Modeling, Part I (Boolean & Vector Models, TF-IDF)

Source: 03_Modeling_Part-I.pdf

1. Table of Contents (Topics Covered)

Gist (2–3 paragraphs)

IR modeling produces a ranking function R(qi, dj) that scores documents against queries. Formally an IR model is a quadruple [D, Q, F, R]: logical views of documents, logical views of queries, a framework, and the ranking function. Documents and queries are represented by index terms drawn from the collection vocabulary V. The lecture covers two of the three classic models.

The Boolean Model uses set theory and Boolean algebra: terms have binary weights {0,1}, and queries are Boolean expressions (e.g., ka ∧ (kb ∨ ¬kc)) converted to disjunctive normal form. It is precise and intuitive but has serious drawbacks — no partial matching, no ranking, awkward query formulation, and it often returns too few or too many results.

The Vector Model fixes this by giving terms non-binary weights (typically TF-IDF) and representing documents/queries as vectors in a t-dimensional space. Similarity is the cosine of the angle between vectors, enabling partial matching and ranking by degree of similarity. TF-IDF combines term frequency (Luhn: frequent-in-doc = important, usually log-scaled) with inverse document frequency (rare-across-collection = selective, grounded in Zipf’s law). Document length normalization (dividing by the vector norm) prevents long documents from being unfairly favored. The vector model is simple, performs well on general collections, but assumes term independence.

2. Key Terminologies — Meaning & Use Cases

Term Meaning Use Case
IR model [D,Q,F,R] Formal quadruple defining doc/query views, framework, ranking Foundation of any retrieval engine
Ranking function R(qi,dj) Assigns a real-valued score to (query, doc) Ordering results
Index term Keyword (often a noun) representing document content Indexing & querying
Vocabulary V Set of all distinct index terms in the collection Defines the term space
Term-document matrix Matrix of term frequencies fi,j Computing weights
Boolean Model Binary weights + Boolean query logic Precise filtering, legal search
DNF (disjunctive normal form) Query as OR of conjunctive components Boolean matching
Term frequency (TF) How often a term appears in a document (often 1+log f) Local importance
Inverse document frequency (IDF) log(N/ni) — rarer terms score higher Global selectivity
Zipf’s law Document frequency ≈ power law of rank (n(r)~r^-α) Justifies IDF
TF-IDF (1+log f) × log(N/ni) — combined weight Standard term weighting
Term-term correlation matrix C = M·Mᵀ, co-occurrence of term pairs Capturing dependencies
Document length normalization Divide score by document vector norm Fair scoring across sizes
Vector Model Docs/queries as weighted vectors; cosine similarity Most classic ranking
Cosine similarity Cosine of angle between query & doc vectors (0–1) Degree of relevance

3. Process Workflow Diagram

flowchart TD
    A[Document Collection] --> B[Extract Index Terms
build Vocabulary V]
    B --> C[Term-Document Matrix
frequencies fi,j]
    C --> D[Compute TF
1+log f]
    C --> E[Compute IDF
log N/ni]
    D --> F[TF-IDF Weights]
    E --> F
    F --> G[Document Length Normalization
vector norm]
    H[User Query] --> I[Query Vector
TF-IDF weights]
    G --> J["Cosine Similarity
sim = q·d / (|q| |d|)"]
    I --> J
    J --> K[Rank docs by similarity]

4. ELI5 — Complex Terms Explained Simply