Softmax is one of those operations that looks trivial on paper but turns into a genuine systems problem on GPU. The math is:

The $\max(x)$ subtraction keeps things numerically stable — without it you overflow exp for large logits. What makes this interesting to benchmark is that softmax is completely memory-bound. The arithmetic is dirt cheap; every cycle you’re waiting on HBM. And since transformers run softmax on every attention head at every layer, squeezing bandwidth out of it actually matters.

I wrote four versions — naive PyTorch, torch.compile, a hand-written Triton kernel, and a persistent variant — then benchmarked them across a range of matrix shapes. The results had a few surprises.

The Bandwidth Formula

The Naive Softmax implementation is straightforward to analyze. For an M×N input, torch.max reads M×N elements, the subtraction reads M×N again and writes M×N for exp_logits, torch.exp touches M×N in-place, torch.sum reads M×N, and the final divide reads and writes M×N. Total traffic is about 5MN elements, or 20MN bytes for float32. The bandwidth formula is:

The fused kernels are more efficient. They load the input once, write the output once, and do all the arithmetic in registers. The max reduction and sum reduction are both done in-register with no extra HBM traffic. The only HBM reads/writes are the initial load and the final store of the output. For M×N, that’s 2MN elements or 8MN bytes, so the formula is:

The naive implementation is worse: torch.max reads the full matrix, the subtraction reads it again, torch.exp is in-place, torch.sum reads again, and the final divide reads and writes. Rough traffic is closer to $8MN + 4M$ bytes, which at M=4096, N=4096 is about 4× the fused kernel’s footprint. To keep the benchmark comparable across all four implementations, the code uses 3 × numel × 4 as a shared normalization — a conventional approximation from the Triton tutorial. The actual numbers favor the fused kernels more than the reported GB/s suggests.

Four Implementations

Naive PyTorch

def naive_softmax(logits: torch.Tensor) -> torch.Tensor:
    max_logits, _ = torch.max(logits, dim=1, keepdim=True)
    exp_logits = torch.exp(logits - max_logits)
    sum_exp_logits = torch.sum(exp_logits, dim=1, keepdim=True)
    return exp_logits / sum_exp_logits

Each line here is a separate CUDA kernel. torch.max writes max_logits to HBM. The subtraction reads logits and max_logits back, writes exp_logits. torch.exp touches it again. torch.sum makes another full pass. Five dispatch calls total, each materializing intermediate tensors to global memory. On a GPU with 2 TB/s HBM bandwidth, this is the worst way to do it.

torch.compile

torch_softmax_compiled = torch.compile(
    lambda logits: torch.nn.functional.softmax(logits, dim=1)
)

The Inductor backend traces through the operation graph and emits a fused Triton kernel automatically. It does AOT compilation on the first call, so you need to run it once before benchmarking. This is what Inductor generates for a 4096×256 input:

@triton.jit
def triton_per_fused__softmax_prepare_softmax_online_0(
    in_ptr0, out_ptr2, xnumel, r0_numel, XBLOCK: tl.constexpr
):
    xnumel = 4096
    r0_numel = 256
    R0_BLOCK: tl.constexpr = 256

    xoffset = tl.program_id(0) * XBLOCK
    xindex  = xoffset + tl.arange(0, XBLOCK)[:, None]
    r0_index = tl.arange(0, R0_BLOCK)[None, :]

    tmp0  = tl.load(in_ptr0 + (r0_index + 256 * xindex), None)
    tmp1  = tl.broadcast_to(tmp0, [XBLOCK, R0_BLOCK])
    tmp5  = triton_helpers.max2(tmp1, 1)[:, None].to(tl.float32)
    tmp6  = tmp1 - tmp5
    tmp7  = libdevice.exp(tmp6)
    tmp10 = tl.sum(tl.broadcast_to(tmp7, [XBLOCK, R0_BLOCK]), 1)[:, None].to(tl.float32)
    tmp13 = libdevice.exp(tmp0 - tmp5) / tmp10
    tl.store(out_ptr2 + (r0_index + 256 * xindex), tmp13, None)

The kernel tiles in 2D — XBLOCK rows by R0_BLOCK=256 columns per CTA. When all 256 columns fit in R0_BLOCK, this is fast and efficient. The problem shows up when N grows past 256.

Triton — One Block Per Row

@triton.jit
def softmax_kernel(
    input_ptr, output_ptr, n_rows, n_cols,
    input_stride_row, output_stride_row,
    BLOCK_SIZE: tl.constexpr
):
    row_idx = tl.program_id(axis=0)
    row_start_ptr = input_ptr + row_idx * input_stride_row
    offsets = tl.arange(0, BLOCK_SIZE)
    mask = offsets < n_cols

    x = tl.load(row_start_ptr + offsets, mask=mask)
    max_x = tl.max(x, axis=0)
    numerator = tl.exp(x - max_x)
    denominator = tl.sum(numerator, axis=0)
    tl.store(output_ptr + row_idx * output_stride_row + offsets,
             numerator / denominator, mask=mask)

BLOCK_SIZE = triton.next_power_of_2(n_cols)
grid = (n_rows,)  # one CTA per row

One CTA owns one row. The entire row loads into registers, max reduces, exp happens, sum reduces, output stores — never touching HBM between steps. BLOCK_SIZE = next_power_of_2(n_cols) means the kernel is always compiled for the right size, masked for anything not a power of 2. For n_cols=6272 that means BLOCK_SIZE=8192 with 1920 lanes masked out — wasteful in register terms, but the data shows this is fine in practice.

The downside is that n_rows CTAs get launched. At large row counts the GPU scheduler handles thousands of CTAs and wave quantization starts adding up.

Persistent Triton Kernel

@triton.jit
def presistant_softmax_kernel(input_ptr, output_ptr, n_rows, n_cols,
                              input_stride_row, output_stride_row,
                              BLOCK_SIZE: tl.constexpr, n_stages: tl.constexpr):
    row_start = tl.program_id(axis=0)
    row_step  = tl.num_programs(axis=0)

    for row_idx in tl.range(row_start, n_rows, row_step, num_stages=n_stages):
        row_start_ptr = input_ptr + row_idx * input_stride_row
        offsets = tl.arange(0, BLOCK_SIZE)
        mask = offsets < n_cols

        x = tl.load(row_start_ptr + offsets, mask=mask)
        max_x = tl.max(x, axis=0)
        numerator = tl.exp(x - max_x)
        denominator = tl.sum(numerator, axis=0)
        tl.store(output_ptr + row_idx * output_stride_row + offsets,
                 numerator / denominator, mask=mask)

The kernel logic is identical. What changes is how many CTAs are launched, and the grid-stride loop. Instead of n_rows CTAs, it launches exactly occupancy × N_SMS:

kernel._init_handles()
n_regs    = kernel.n_regs
size_smem = kernel.metadata.shared

occupancy    = NUM_REGS // (n_regs * num_warps * WARP_SIZE)
occupancy    = min(occupancy, SHARED_MEM // size_smem)
num_programs = min(occupancy * N_SMS, n_rows)

The warmup call compiles the kernel and exposes n_regs and size_smem. From those, you can compute the same occupancy number CUDA’s occupancy calculator would give you — how many CTAs can simultaneously live on each SM without register spilling or shared memory overflow. Launch exactly that many, let them loop over all rows with a stride, and you get continuous SM utilization with zero wave quantization overhead. num_stages=4 adds software pipelining: Triton prefetches the next row’s memory while the current row is computing, hiding the round-trip HBM latency.

Results

Varying N (columns), M = 4096 fixed

The torch.compile cliff is the most striking thing here. It starts at 223 GB/s at N=256 — actually beating non-persistent Triton at that point — then falls off a cliff, hitting ~119 GB/s by N=1024 and staying flat. At N=6272, it’s delivering 51% of what the hand-written kernel achieves, and both are Triton kernels under the hood.

The reason is that fixed R0_BLOCK=256. When N grows past 256 columns, Inductor can’t fit the whole row in one reduction block and has to fall back to a multi-wave strategy. The hand-written kernel gets around this entirely by setting BLOCK_SIZE = next_power_of_2(n_cols) — each new shape gets a fresh compilation with the right block size.

Persistent softmax starts below non-persistent at small N (197 vs 215 GB/s) because the occupancy calculation and loop overhead don’t pay off when there are already 4096 rows saturating the GPU. By N=2048 both Triton variants are neck and neck at ~233 GB/s.

Naive sits flat at 54–60 GB/s the whole way — about 4× slower — completely bottlenecked by five kernel launches and repeated HBM reads.

Varying M (rows), N = 4096 fixed

Everything improves as rows increase — more CTAs means better SM saturation. At M=32 all four implementations are well below their peaks simply because the GPU is mostly idle. At M=3168, both Triton variants hit ~234 GB/s while torch compiled settles around 225 GB/s.

The interesting thing at low M is that persistent actually trails non-persistent slightly (142 vs 149 at M=32). With num_programs = min(occupancy * N_SMS, n_rows), at M=32 the clamp kicks in and the persistent kernel degenerates to essentially the same as non-persistent — but with the added overhead of the loop and stage management. That gap closes completely by M=64.

Latency

Fused implementations grow linearly with M, as you’d expect for a memory-bound kernel. Triton and torch compiled are within 5% of each other in absolute milliseconds, the gap is mainly in bandwidth efficiency.

Naive softmax’s curve bends upward — super-linear on a log scale. At M=512 it’s already taking 4× longer than any fused kernel. Extrapolate that across 32 transformer layers with multi-head attention and the wall time adds up fast.

What the Persistent Kernel is Actually Doing

The occupancy calculation is the part worth slowing down on:

occupancy = NUM_REGS // (n_regs * num_warps * WARP_SIZE)
occupancy = min(occupancy, SHARED_MEM // size_smem)
num_programs = occupancy * N_SMS

n_regs is read directly from the compiled kernel object after kernel._init_handles(). This is the same number CUDA’s occupancy API would give you, but computed in Python without any manual guessing. The two constraints — register file size and shared memory — are both checked, and whichever is tighter limits occupancy. Launch occupancy × N_SMS CTAs, each running a grid-stride loop, and SMs stay continuously loaded with no wave overhead.

The num_stages=4 in tl.range is what enables the software pipeline. Triton emits instructions to prefetch iteration i+1’s memory while arithmetic for iteration i is still executing, effectively hiding the ~200+ cycle HBM round trip. This shows up as a 1–2 GB/s edge over non-persistent at intermediate row counts.

Both Triton kernels plateau around 234 GB/s regardless of shape. That’s the achievable bandwidth ceiling for this access pattern on this GPU — not the advertised peak, but the real number after memory controller overhead, row-buffer effects, and the read-write mix.

What Didn’t Go as Expected

The torch.compile cliff was steeper than I anticipated. The Triton tutorial frames torch.compile as a strong baseline that custom kernels only marginally beat, so I expected a gradual degradation as N grew. What actually happened was a near-halving of throughput between N=256 and N=1024, then a complete plateau. Once I looked at the generated kernel and saw R0_BLOCK: tl.constexpr = 256 baked in as a compile-time constant, it made sense — but nothing in the tutorial hints that this is the mechanism. I would have expected Inductor to retrace with a larger block when the shape changed. It doesn’t, at least not in this version.

The persistent kernel at small M was a subtler surprise. My prior was that persistent kernels are strictly better — they avoid wave quantization and get L2 reuse, so why wouldn’t they win everywhere? The answer is that at M=32, min(occupancy * N_SMS, n_rows) clamps the grid to 32 CTAs regardless, so you get the loop overhead without any of the benefits. The non-persistent kernel launches the same 32 CTAs but with a simpler dispatch path, and edges it slightly. It’s a small gap (149 vs 142 GB/s) but the direction is counterintuitive until you trace through the math. The persistent design only earns its cost when the row count is large enough to give each SM multiple iterations of work.

Wrapping Up

The takeaway from this isn’t that torch.compile is bad — for most shapes and workloads it’s great. The issue is specifically variable-length reductions. When the compiled kernel specializes on a shape with 256 columns and you then feed it 4096 columns, the blocking strategy it baked in stops working. A hand-written kernel that sets BLOCK_SIZE dynamically doesn’t have this problem because Triton JITs a fresh specialization for each power-of-2 size.

The persistent kernel’s value is less about raw throughput — at saturation it matches non-persistent exactly — and more about correctness of SM utilization across shapes. The occupancy-aware launch pattern is what FlashAttention and similar kernels use under the hood, and it’s worth understanding because it transfers directly to any kernel that loops over a reduction dimension.

The next step from here is online softmax (Milakov & Gimelshein 2018), which fuses the max and sum passes into one, opening the door to fusing softmax with the preceding matmul — which is exactly what FlashAttention does.

Basic RAG implementations—embed query, retrieve documents, pass to LLM, return response—break down when applied to complex domains requiring nuanced understanding. The standard pipeline provides no quality control, making it unsuitable for querying philosophical texts where accuracy and grounding matter.

Building a RAG system for Marcus Aurelius’ “Meditations” exposed these limitations clearly. Stoic philosophy requires synthesizing insights across multiple passages and careful interpretation of context. A simple retrieval pipeline would either miss relevant passages or return superficially similar but contextually wrong content.

The solution is an agentic architecture that treats retrieval and generation as iterative processes with built-in quality validation. This system uses LangGraph to orchestrate multi-step reasoning, hybrid search to improve retrieval, and an evaluation loop to catch hallucinations before they reach users.

Live Demo: https://meditations-rag-180347172582.asia-south1.run.app GitHub: https://github.com/MrudhuhasM/meditations-rag

Agentic Architecture with LangGraph

The system operates through four specialized nodes that form a feedback loop rather than a linear pipeline.

Controller Node

Analyzes incoming queries to determine retrieval strategy. For factual lookups (“What does Marcus say about anger?”), the controller routes directly to dense vector search. For philosophical synthesis (“How do Stoic principles apply to modern work?”), it enables hybrid search and sets higher thresholds for the evaluator.

Retriever Node: Hybrid Search

The retriever combines two search methods through weighted score fusion:

Dense Vector Search uses semantic embeddings to find conceptually similar passages. This captures meaning beyond exact keyword matches—”dealing with difficult people” retrieves passages about “managing relationships” even without shared vocabulary.

Question-Based Retrieval matches user queries against synthetic questions generated during document ingestion. For each text chunk, an LLM generates questions the passage could answer. User query “How should I handle anger?” matches generated question “What techniques does Marcus suggest for managing anger?”

Score fusion combines both methods:

final_score = (alpha * dense_score) + (beta * question_score)

Setting $\alpha = 0.6$ and $\beta = 0.4$ weights semantic similarity higher while still capturing question-format matches. This prevents missing passages where users phrase questions differently than the source text.

Generator Node

Synthesizes information across retrieved passages while maintaining source attribution. The generator receives ranked passages and must cite specific text when making claims. This constraint reduces hallucination by forcing grounding in retrieved content.

Evaluator Node

The evaluator validates three properties:

• Groundedness: Claims must be supported by retrieved passages
• Relevance: Response must address the query
• Completeness: All query aspects must be covered

Evaluation uses an LLM to assess these properties against retrieved context. If any check fails, the system can request additional context, reformulate the response, or return to the controller for a different strategy. This feedback loop catches roughly 15-20% of responses that would otherwise be irrelevant or hallucinated.

Production Engineering

Deploying RAG systems exposes issues invisible during local development. Rate limiting prevents abuse, authentication controls access, and structured logging enables debugging production failures.

Rate Limiting

SlowAPI implements token bucket rate limiting at 10 requests per minute per IP:

@limiter.limit("10/minute")
async def query_endpoint(request: Request):
    # Handle query
    pass

This prevents resource exhaustion while allowing burst traffic from legitimate users.

Authentication

Middleware validates API keys before processing requests. The system supports three tiers: demo keys (limited queries), authenticated keys (standard rate limits), and premium keys (higher limits, priority queue). Usage tracking per key enables cost analysis and abuse detection.

Logging

Structured logging captures query metadata for debugging and performance analysis:

logger.info(
    "Query processed",
    extra={
        "query": query_text,
        "retrieval_time_ms": retrieval_time,
        "num_chunks": len(chunks),
        "evaluation_score": eval_score
    }
)

This data reveals retrieval patterns, identifies slow queries, and tracks evaluation scores over time.

Error Handling

The system handles three failure modes: LLM service unavailability triggers fallback to alternative providers (OpenAI → Gemini → local models), vector DB connection failures serve cached responses when available, and malformed queries return clear error messages with suggested reformulations.

Data Processing: Semantic Chunking and Metadata

Retrieval quality depends on how text is chunked during ingestion. Fixed-size chunks (512 tokens) break mid-sentence or mid-thought, degrading semantic coherence. Semantic chunking addresses this by grouping sentences based on embedding similarity.

Semantic Chunking

The algorithm:

1. Split text into sentences
2. Generate embeddings for each sentence
3. Compute cosine similarity between adjacent sentences
4. Group sentences where similarity $> 0.7$
5. Create chunks from groups

This produces chunks that maintain topical coherence. A passage discussing anger management stays together rather than being split across chunks based on arbitrary token counts.

Metadata Extraction

For each chunk, an LLM extracts structured metadata:

metadata = {
    "book": 3,
    "chapter": 12,
    "topics": ["anger", "self-control", "virtue"],
    "philosophical_concepts": ["Stoicism", "reason"],
    "synthetic_questions": [
        "How should one handle anger?",
        "What does Marcus say about self-control?"
    ]
}

This metadata enables filtered retrieval (“passages about virtue from Book 3”), question matching (user query → synthetic questions), and topic exploration. The upfront cost of metadata extraction—about 2 seconds per chunk using GPT-3.5—pays off through improved retrieval precision.

Implementation Details

LangGraph orchestrates the agentic workflow with state management. Each node modifies shared state containing the query, retrieved passages, generated response, and evaluation scores.

Qdrant serves as the vector database, providing sub-100ms similarity search with metadata filtering. The system stores ~500 chunks from “Meditations” with 1536-dimensional embeddings (OpenAI ada-002).

Language model support spans cloud providers (OpenAI GPT-4/3.5, Gemini) and local inference (llama.cpp, vLLM). Multi-provider support enables cost/quality tradeoffs: GPT-4 for generation, GPT-3.5 for evaluation, local models for development.

FastAPI handles HTTP requests with async support. Docker containerization enables reproducible deployments across development and production environments.

Lessons from Building Production RAG

Hybrid search is essential. Pure semantic search misses exact matches; pure keyword search misses conceptual similarity. Weighted fusion delivers better results than either method alone.

Evaluation catches hallucinations. The evaluator node catches 15-20% of responses that would have been irrelevant or hallucinated. This quality gate justifies the extra latency.

Metadata drives retrieval quality. Rich metadata transforms retrieval from “find similar text” to “find relevant knowledge.” The investment in metadata extraction—2 seconds per chunk during ingestion—pays off in retrieval precision.

Simplicity wins. The initial workflow had 7 nodes. Testing revealed that 4 nodes cover essential decision points without unnecessary complexity. Over-engineering the agent introduced latency without improving quality.

Production features require time. Rate limiting, authentication, logging, and error handling consumed 30% of development time but differentiate a usable system from a demo.

Performance Characteristics

Query Latency:

• p50: 2.5 seconds
• p95: 4.5 seconds
• p99: 7 seconds

Retrieval Accuracy (evaluated on 100 test queries):

• Relevant chunks in top 5: 87%
• Answer grounded in sources: 92%
• Answer addresses question: 89%

Scalability: The stateless API design enables horizontal scaling. Current deployment handles 50 concurrent requests with Qdrant providing sub-100ms vector search.

Future Work

Multi-Document Support: Extend beyond “Meditations” to query across Stoic texts (Epictetus, Seneca) with comparative analysis.

Conversation Memory: Maintain dialogue history to handle follow-up questions and enable deeper exploration.

Citation Visualization: Build UI showing which passages contributed to each part of the answer.

Fine-Tuned Embeddings: Train a custom embedding model on philosophical texts for improved semantic understanding.

Feedback Loop: Collect user ratings on answer quality to improve evaluation criteria.

Conclusion

Moving from basic RAG to production systems requires addressing quality control, retrieval precision, and operational reliability. The agentic architecture with LangGraph enables multi-step reasoning and evaluation loops that catch hallucinations. Hybrid search improves retrieval accuracy. Production engineering—rate limiting, authentication, logging, error handling—makes the system deployable.

The complete implementation is available at:

Live Demo: https://meditations-rag-180347172582.asia-south1.run.app GitHub: https://github.com/MrudhuhasM/meditations-rag

When we predict product sales based on advertising spend, we instinctively seek that straight-line relationship in our data. Simple linear regression quantifies this intuition – transforming visual patterns into mathematical certainty. As data scientists, we need to master its inner workings to validate assumptions and interpret predictions correctly. Walk alongside me as we implement regression from scratch, revealing what happens beneath the surface of every scikit-learn .fit() call.

Here’s the journey we’ll take together:  

1. Generating and visualizing synthetic regression data  

2. Mathematical derivation of Ordinary Least Squares estimation  

3. Implementing coefficient calculation from scratch  

4. Visual validation against the true relationship  

5. Quantifying uncertainty through confidence intervals  

6. Hypothesis testing for statistical significance  

7. Measuring model fit with R² metrics  

1. Designing and Visualizing Synthetic Data

We begin by engineering a controlled experimental environment. Using Python’s scikit-learn:

from sklearn.datasets import make_regression

import matplotlib.pyplot as plt

  

X, y, true_coef = make_regression(

    n_samples=5000,  # Sample size statistical reliability

    n_features=1,    # Single predictor focus

    noise=10,        # Real-world measurement noise

    random_state=12, # Reproducibility seed

    coef=True        # Ground truth for validation

)

X = X.flatten()      # Simplified array structure

Why synthetic data? We get a laboratory-perfect environment where we know the true relationship (β₁ = 14.5238) before estimation. The noise=10 parameter replicates real-world observational variance – those unpredictable forces affecting outcomes despite our input controls. Unlike messy real data, this lets us verify our methods with precision.

Visual confirmation comes next:

plt.figure(figsize=(12, 6))

plt.scatter(X, y, color='royalblue', alpha=0.4, s=15)

plt.title('Synthetic Linear Relationship', fontsize=14)

plt.xlabel('Feature Value (X)', fontsize=12)

plt.ylabel('Target Value (y)', fontsize=12)

plt.grid(alpha=0.2)

plt.gca().spines[['top','right']].set_visible(False)

Blue dots: 5,000 synthetic observations. The diagonal grouping shows the underlying linear pattern, while vertical dispersion represents noise – reflecting real-world unpredictability.

Observe how points cluster near an invisible diagonal? That visual intuition precedes our mathematical formalization. At scale, these patterns become mathematical relationships we can quantify, test, and deploy.

2. Mathematical Foundations

The population relationship follows a precise architectural blueprint:  

\(y = \beta_0 + \beta_1 x + \epsilon\)  

Where:  

• $\beta_0$: True intercept (vertical offset when X=0)  

• $\beta_1$: True slope (change in y per 1-unit x-change)  

• $\epsilon$: Independent random noise ($E[\epsilon]=0$, $\text{Var}(\epsilon)=\sigma^2$)  

But how do we estimate $\beta$ from samples? Enter Ordinary Least Squares (OLS) – the workhorse of linear modeling. Why choose OLS among alternatives?

1. Unbiasedness: Under Gauss-Markov assumptions, OLS provides Best Linear Unbiased Estimators  

2. Optimality: Minimizes mean-squared error of predictions  

3. Interpretability: Closed-form solutions facilitate insight  

OLS estimates $\hat{\beta}_0,\hat{\beta}_1$ by minimizing Residual Sum of Squares:  

\(RSS(\hat{\beta}_0, \hat{\beta}_1) = \sum_{i=1}^n (y_i - \hat{y}_i)^2 = \sum_{i=1}^n \left(y_i - (\hat{\beta}_0 + \hat{\beta}_1 x_i)\right)^2\)  

graph TD

A[Observed Value y] --> B[Prediction ŷ]

B --> C[Residual e = y − ŷ]

C --> D[RSS = Σe² Minimization]

Setting partial derivatives ∂RSS/∂β⃕ to zero yields the optimal solutions:  

These constitute the normal equations of linear regression – the analytical bedrock we’ll implement next.

3. Implementing OLS Estimation

Translating theory into Python:

# Sample statistics - anchors of estimation

mean_X = X.mean()  # ≈ -0.02

mean_y = y.mean()  # ≈ -0.41

  

# Calculate slope = Covariance / Variance

numerator = ((X - mean_X) * (y - mean_y)).sum()  # Covariance component

denominator = ((X - mean_X)**2).sum()            # Variance component

beta_1_hat = numerator / denominator

  

# y-intercept = Mean adjustment for X-centering

beta_0_hat = mean_y - beta_1_hat * mean_X

  

# Compare estimates to ground truth

print(f"True slope coefficient: {true_coef:.4f}")

print(f"Estimated slope β̂₁: {beta_1_hat:.4f}")

print(f"Estimated intercept β̂₀: {beta_0_hat:.4f}")

True slope coefficient: 14.5238

Estimated slope β̂₁: 14.5467

Estimated intercept β̂₀: -0.1667

Output Interpretation:

• 0.16% error on slope estimate (

  14.5467-14.5238

  /14.5238≈0.0016)

• Near-zero intercept confirms data centering near origin  

• Minor discrepancies stem from deliberate $σ^2=100$ noise  

Our estimators already show strong accuracy at n=5000 samples, demonstrating Law of Large Numbers in action.

4. Visual Validation

How well does our estimated line ($\hat{y}=\hat{\beta}_0+\hat{\beta}_1x$) capture reality? Side-by-side comparison:

plt.figure(figsize=(14,7))

plt.scatter(X, y, color='royalblue', alpha=0.15, s=15, label='Data')

  

# True population trajectory (known by design)

regression_line_true = true_coef * X  # True β₀≈0

plt.plot(X, regression_line_true, color='limegreen',

         linewidth=3.5, alpha=0.7, label='True Relationship')

  

# OLS-estimated trajectory

regression_line_estimate = beta_0_hat + beta_1_hat * X

plt.plot(X, regression_line_estimate, 'r--', linewidth=1.8,

         alpha=0.95, label='OLS Estimate')

Green: The hidden relationship. Red dashes: Our estimate from data. Their near-overlap visually confirms OLS effectiveness at scale.

What we see aligns with theory – but eyes can deceive. We need quantitative uncertainty metrics to statistically validate our inferences.

5. Statistical Assessment

5.1 Confidence Intervals and Standard Errors

OLS estimates have inherent sampling variability. We quantify precision via:

Residual Standard Error (RSE):

Estimates the standard deviation σ of $\epsilon$, governing prediction error scale.

Interpreted as: “Actual responses deviate ±9.95 units from predictions on average”

Then derive coefficient standard errors:

residuals = y - (beta_0_hat + beta_1_hat * X)

RSS = (residuals**2).sum()

RSE = (RSS / (len(X) - 2))**0.5  # n-2 for linear regression

  

denom = ((X - mean_X)**2).sum()

SE_beta1 = RSE / denom**0.5

SE_beta0 = RSE * (1/len(X) + mean_X**2/denom)**0.5

  

# 95% confidence intervals (normal approximation)

CI_beta1 = (beta_1_hat - 2*SE_beta1, beta_1_hat + 2*SE_beta1)  

CI_beta0 = (beta_0_hat - 2*SE_beta0, beta_0_hat + 2*SE_beta0)  

  

print(f"RSE: {RSE:.4f}  (≈ ±1σ vertical deviation)")

print(f"95% CI for β₁: [{CI_beta1[0]:.4f}, {CI_beta1[1]:.4f}]")

print(f"95% CI for β₀: [{CI_beta0[0]:.4f}, {CI_beta0[1]:.4f}]")

RSE: 9.9478  (≈ ±1σ vertical deviation)

95% CI for β₁: [14.2639, 14.8294]

95% CI for β₀: [-0.4481, 0.1147]

Statistical Perspective:

• True β₁=14.5238 lies squarely within [14.26, 14.83] → Estimation accurate  

• CI for β₀ crosses zero (-0.45 to 0.11) → Intercept non-significant  

• Interval widths show precise slope estimation (SE=0.1414)

5.2 Formal Hypothesis Testing

Does X truly predict Y? We reformulate scientifically:

\(H_0: \beta_1 = 0 \quad \text{(X has no effect)}\)  

\(H_1: \beta_1 \neq 0 \quad \text{(Systematic relationship exists)}\)  

Compute t-statistic using standard error:

from scipy.stats import t as t_dist

t_stat = beta_1_hat / SE_beta1

p_value = 2 * (1 - t_dist.cdf(abs(t_stat), df=len(X)-2))

  

print(f"t-statistic = {t_stat:.2f}, df={len(X)-2}, p-value = {p_value:.5e}")

print("\nFormal report:")

print("| Parameter | Estimate | Std. Error | t-value   | p-value    | 95% CI          |")

print("|-----------|----------|------------|-----------|------------|-----------------|")

print(f"| β₁       | {beta_1_hat:.4f} | {SE_beta1:.4f}    | {t_stat:.2f} | {p_value:.3e} | [{CI_beta1[0]:.4f}, {CI_beta1[1]:.4f}] |")

print(f"| β₀       | {beta_0_hat:.4f} | {SE_beta0:.4f}    | {'N/A':^9} | {'N/A':<11} | [{CI_beta0[0]:.4f}, {CI_beta0[1]:.4f}] |")

t-statistic = 102.89, df=4998, p-value = 0.00000e+00

Formal report:

Evidence Synthesis:

• p-value < 10^{-100} → Overwhelming evidence against H₀  

• t=102.89 indicates slope is 103 SEs from zero - vanishingly unlikely if null true  

• Statistical certainty confirms what data generation intentionally built

5.3 Model Fit: R-Squared Analysis

While parameters test relationships, R² quantifies how well the model fits:

TSS = ((y - mean_y)**2).sum()

R_squared = 1 - (RSS / TSS)

  

print(f"Total Variance (TSS): {TSS:.0f}")

print(f"Unexplained Variance (RSS): {RSS:.0f}")

print(f"Explained Variance: R² = {R_squared:.4f}")

Total Variance (TSS): 1542149

Unexplained Variance (RSS): 494592

Explained Variance: R² = 0.6793

pie showData

    title Variance Breakdown

    "Explained by model" : 67.93

    "Unexplained noise" : 32.07

Final visualization places this in context:

plt.figure(figsize=(14,7))

plt.scatter(X, y, color='royalblue', alpha=0.08, s=18, label='Data')

plt.plot(X, regression_line_estimate, 'r-', linewidth=1.5, alpha=0.9)

The R²=0.68 indicates a moderate-to-strong model fit – capturing 68% of target variance. This quantitative assessment aligns with our visual impression of point clustering around the regression line.

In predictive model scoring, 0.68 represents a substantial linear signal – significantly better than naive models, with room for improvement through feature engineering.

Where To Next?

This foundation supports multiple extensions:

Key Takeaways  

• Core Equation: Simple linear regression models $y = \beta_0 + \beta_1x + \epsilon$ using a fixed slope and intercept  

• Estimation Strategy: OLS minimizes residual sum of squares, providing provably optimal linear estimators  

• Uncertainty Quantification: Standard errors and confidence intervals reveal estimate precision  

• Model Validation: Hypothesis testing formally evaluates predictor significance  

• Fit Metrics: R² quantifies explanatory power (here ≈68% = moderate-to-strong)  

• Implementation: From-scratch Python implementation requires <15 lines of core math  

While real-world data brings complications, these principles remain the bedrock of predictive modeling. Master them to build, diagnose, and reliably interpret regression models across applications.

Content-based image retrieval finds visually similar images without textual metadata. Traditional approaches using handcrafted features (SIFT, SURF, color histograms) require domain expertise and struggle with semantic similarity. Deep learning feature extraction from pre-trained CNNs captures high-level visual patterns enabling semantic similarity search.

This system uses ResNet50 pre-trained on ImageNet to extract 100,352-dimensional feature vectors, reduced through PCA and visualized with t-SNE. Nearest neighbor search retrieves similar images from 5,000 samples of the Flickr30k dataset.

Kaggle Notebook: Finding Similar Images using ResNet + Nearest Neighbors

Architecture

Feature Extraction Pipeline:

1. Load image, resize to 224×224 (ResNet input)
2. Extract features from final convolutional layer (7×7×2048 = 100,352D)
3. Flatten and L2-normalize feature vector
4. Index in k-NN structure for similarity search

Retrieval:

1. Query image → feature extraction
2. k-NN search in feature space (Euclidean distance)
3. Return k most similar images

Feature Extraction

ResNet50 Pre-trained Model

ResNet50 trained on ImageNet learns hierarchical visual representations. Early layers detect edges and textures; deeper layers recognize objects and scenes. Using the final convolutional layer (before classification) provides generic visual features transferable to new domains.

model = ResNet50(
    include_top=False,  # Remove classification head
    input_shape=(224,224,3),
    weights='imagenet'
)

Setting include_top=False removes the 1000-class classifier, retaining convolutional layers that output spatial features.

Normalization

def feature_extraction(img_path, model):
    img = image.load_img(img_path, target_size=(224,224))
    img_array = image.img_to_array(img)
    img_batch = np.expand_dims(img_array, axis=0)
    img_processed = preprocess_input(img_batch)
    features = model.predict(img_processed)
    features_flatten = features.flatten()  # 100,352D vector
    normalized_features = features_flatten / norm(features_flatten)
    return normalized_features

L2 normalization ensures unit-length vectors, making Euclidean distance equivalent to cosine similarity:

for normalized vectors $|\mathbf{x}| = |\mathbf{y}| = 1$.

Processing Dataset

root_dir = '../input/flickr-image-dataset/flickr30k_images/'
file_names = get_files(root_dir)  # 5,000 images
feature_list = []
for file_name in tqdm(file_names):
    feature_list.append(feature_extraction(file_name, model))

Extracting features for 5,000 images takes ~15 minutes on GPU. For larger datasets, batch processing and offline indexing would be necessary.

Nearest Neighbor Search

k-NN Index

neighbors = NearestNeighbors(
    n_neighbors=5,
    algorithm='brute',
    metric='euclidean'
).fit(feature_list)

Parameters:

• n_neighbors=5: Return 5 most similar images
• algorithm='brute': Exhaustive search (guaranteed exact results)
• metric='euclidean': Distance metric

For production systems with millions of images, approximate methods (LSH, HNSW, FAISS) trade accuracy for speed. Brute-force search is $O(n)$ per query; approximate methods achieve $O(\log n)$ or $O(1)$ with indexing.

Query and Retrieval

distances, indices = neighbors.kneighbors([feature_list[query_idx]])

Returns indices of k nearest neighbors and their distances. The nearest neighbor (index 0) is always the query image itself with distance ≈0.

Results

Query Image: Person standing on rocky terrain

Similar Images:

1. Original (distance: 0.00)
2. Mountain landscape (distance: 0.42)
3. Rock formation (distance: 0.48)
4. Outdoor scene (distance: 0.51)
5. Person outdoors (distance: 0.53)

The system retrieves images with similar visual characteristics: outdoor settings, natural landscapes, rocky textures. Semantic similarity emerges from deep features without explicit object detection.

Query Image: Group of people indoors

Similar Images: Other indoor group scenes, similar compositions, comparable lighting conditions.

Query Image: Urban street scene

Similar Images: Cityscapes, buildings, street-level photography.

ResNet features capture scene type, composition, and object categories, enabling semantic retrieval beyond low-level visual similarity.

Dimensionality Reduction and Visualization

PCA Compression

100,352 dimensions are computationally expensive and contain redundancy. PCA projects to 100 dimensions retaining most variance:

pca = PCA(n_components=100)
feature_list_compressed = pca.transform(feature_list[:300])

Reduced dimensionality accelerates nearest neighbor search and enables visualization.

t-SNE Visualization

t-SNE maps high-dimensional features to 2D while preserving local neighborhood structure:

tsne_results = TSNE(
    n_components=2,
    metric='euclidean'
).fit_transform(feature_list_compressed)

Scatter Plot:

Points represent images in 2D embedding space. Proximity indicates visual similarity. Clusters emerge for scene categories (indoor/outdoor), object types (people/landscapes), composition patterns.

Image Grid:

def tsne_to_grid_plotter(x, y, image_paths):
    # Map t-SNE coordinates to 2D grid
    # Place image thumbnails at grid positions
    plot_images_in_2d(x_grid, y_grid, image_paths)

Arranging actual images on the t-SNE grid reveals semantic structure. Similar images cluster spatially: beach scenes together, urban scenes together, portraits together.

t-SNE visualization confirms that ResNet features encode semantic relationships. Images cluster by content despite no explicit supervision on the Flickr30k dataset.

Applications

Reverse Image Search: User uploads image, system finds visually similar images in database. E-commerce applications: find similar products.

Image Deduplication: Detect near-duplicate images by thresholding distance. Useful for cleaning datasets or detecting copyright violations.

Image Organization: Cluster images by visual similarity for automatic album creation or content moderation.

Recommendation Systems: “Users who viewed this image also viewed…” based on feature similarity.

Retrieval-Augmented Generation: Image features as context for multimodal LLMs. Similar to text RAG, retrieve relevant images for visual question answering or captioning.

Limitations

Computational Cost: ResNet50 inference requires GPU for real-time performance. 100K-dimensional features are memory-intensive at scale.

Domain Shift: ImageNet pre-training may not transfer well to specialized domains (medical imaging, satellite imagery). Fine-tuning on domain data would improve features.

Lack of Semantic Understanding: Features capture visual patterns but not high-level semantics. “Dog playing fetch” and “person throwing frisbee” may not be similar despite shared concept of throwing.

No Text Integration: Cannot search “sunset on beach” without combining with captioning models or CLIP-style vision-language embeddings.

Improvements

Better Embeddings: Use models explicitly trained for similarity (SimCLR, MoCo) or multimodal models (CLIP) enabling text-to-image search.

Approximate NN: Replace brute-force search with FAISS, Annoy, or HNSW for sub-linear query time on large datasets.

Fine-tuning: Adapt ResNet50 on target domain with triplet loss or contrastive learning to improve domain-specific similarity.

Hybrid Retrieval: Combine visual features with metadata (tags, captions, location) for more comprehensive search.

Feature Compression: Use learned compression (autoencoders) instead of PCA to reduce dimensionality while preserving discriminative information.

Conclusion

Image similarity search using ResNet50 features and k-NN retrieval achieves semantic matching on Flickr30k. The system finds visually and semantically similar images (landscapes with landscapes, portraits with portraits) without explicit labels.

Pre-trained CNN features transfer well to similarity tasks despite being trained for classification. The final convolutional layer representations capture high-level visual patterns generalizing across domains.

t-SNE visualization reveals semantic clusters in feature space, confirming that deep features encode meaningful relationships. Images with similar content, composition, and scene types cluster together.

For production systems, approximate nearest neighbor search and more efficient embeddings (CLIP) would enable scalable image retrieval. The core principle—deep feature extraction + similarity search—remains effective across domains from e-commerce to content moderation.

Full implementation: Kaggle Notebook

Sentiment analysis requires converting text into numerical representations. Traditional approaches like Bag of Words (BOW) treat words as independent tokens, discarding order and semantic relationships. Word embeddings (Word2Vec, FastText) capture semantic similarity in dense vector spaces, potentially improving classification performance.

This analysis compares three approaches on IMDb movie reviews: BOW with TF-IDF weighting, Word2Vec embeddings, and FastText embeddings. Results show embedding methods achieve similar accuracy (88-89%) to BOW (87%) while providing semantic representations useful beyond classification.

Kaggle Notebook: Movie Sentiment Analysis: BOW, Word2Vec, FastText

Dataset and Preprocessing

IMDb movie reviews dataset: 40,000 samples balanced between positive and negative sentiment.

data = pd.read_csv('movie.csv')
print(data.label.value_counts())
# 0 (negative): 20,019
# 1 (positive): 19,981

Preprocessing Pipeline:

def clean_data(text):
    text = text.replace('br','')  # Remove HTML breaks
    doc = load_model(text)  # spaCy processing
    text = " ".join([token.lemma_ for token in doc])  # Lemmatization
    preprocess_functions = [
        to_lower,
        remove_special_character,
        remove_number,
        normalize_unicode,
        remove_punctuation,
        expand_contraction,
        remove_stopword
    ]
    preprocessed_text = preprocess_text(text, preprocess_functions)
    return preprocessed_text

Steps: HTML cleaning → lemmatization → lowercase → special character/number removal → contraction expansion → stopword removal.

Train/test split: 32,000 train, 8,000 test (80/20).

Bag of Words with TF-IDF

BOW represents documents as word frequency vectors. TF-IDF weights down common words appearing across many documents:

where $N$ is total documents and $df(t)$ is document frequency of term $t$.

vect = CountVectorizer(max_features=10000)
X_train_bow = vect.fit_transform(X_train)
X_test_bow = vect.transform(X_test)

Vocabulary limited to 10,000 most frequent terms, producing sparse 10,000-dimensional vectors.

Multinomial Naive Bayes

Probabilistic classifier assuming feature independence:

classifier = MultinomialNB()
classifier.fit(X_train_bow, y_train)
predictions = classifier.predict(X_test_bow)

Performance:

• Accuracy: 87.2%

• Precision (Negative): 0.88

  Recall: 0.87

• Precision (Positive): 0.86

  Recall: 0.88

Strong baseline with simple model. Independence assumption holds reasonably well for sentiment despite ignoring word order.

Logistic Regression

Linear classifier with balanced class weights:

logreg_classifier = LogisticRegression(class_weight='balanced', solver='liblinear')
logreg_classifier.fit(X_train_bow, y_train)

Performance:

• Accuracy: 88.0%

• Precision (Negative): 0.89

  Recall: 0.87

• Precision (Positive): 0.87

  Recall: 0.89

Slight improvement over Naive Bayes. Class balancing prevents bias toward majority class.

Linear SVM

Maximum-margin classifier:

vect = CountVectorizer(max_features=1000)  # Reduced features
svm_classifier = LinearSVC()
svm_classifier.fit(X_train_bow, y_train)

Performance:

• Accuracy: 86.6%

• Precision (Negative): 0.86

  Recall: 0.88

• Precision (Positive): 0.87

  Recall: 0.85

Comparable to Naive Bayes. Vocabulary reduction to 1,000 features limits performance but improves speed.

Word2Vec Embeddings

Word2Vec learns dense vector representations where semantically similar words cluster together. Skip-gram variant predicts context words from target word.

X_train_tokens = X_train.apply(word_tokenize)
w2v = Word2Vec(
    X_train_tokens,
    vector_size=200,
    window=5,
    min_count=2,
    workers=3,
    sg=1  # Skip-gram
)

Semantic Relationships:

w2v.wv.most_similar('good')
# [('goodand', 0.73), ('serviceable', 0.71), ('halfdecent', 0.70), ...]

The model captures semantic similarity: “good” relates to “serviceable”, “halfdecent”, “soso”—all expressing quality assessment.

Document Representation:

def text2vec(list_tokens):
    DIMENSION = 200
    features = []
    for tokens in list_tokens:
        row_feat = np.zeros(DIMENSION)
        c = 1e-5
        for token in tokens:
            if token in w2v.wv:
                row_feat += w2v.wv[token]
                c += 1
        features.append(row_feat / c)  # Mean pooling
    return features

Document vector = mean of constituent word vectors. Simple aggregation loses word order but captures overall semantic content.

Logistic Regression with W2V

log_reg_emb = LogisticRegression(solver='liblinear')
log_reg_emb.fit(X_train_vect, y_train)

Performance:

• Accuracy: 88.4%

• Precision (Negative): 0.88

  Recall: 0.89

• Precision (Positive): 0.89

  Recall: 0.88

Marginal improvement over BOW (88.4% vs 88.0%). Dense representations capture semantic similarity but mean pooling discards syntactic structure.

FastText Embeddings

FastText extends Word2Vec by representing words as bags of character n-grams. This handles out-of-vocabulary words and morphological variations.

Training:

# Format: text __label__class
train_file = './train.csv'
X_train.to_csv(train_file, header=None, index=False)

model = fasttext.train_supervised(
    input=train_file,
    lr=1.0,
    epoch=75,
    loss='hs',  # Hierarchical softmax
    wordNgrams=2,
    dim=200,
    thread=2
)

Performance:

• Test Accuracy: 88.98%

• Precision: 88.98%

  Recall: 88.98%

FastText achieves highest accuracy through character n-gram modeling. Subword information helps with rare words and typos common in reviews.

Comparative Analysis

Key Observations:

1. Marginal Gains: Embeddings improve accuracy by ~1% over BOW (88.4-89.0% vs 87.2-88.0%). For sentiment analysis, bag-of-words captures sufficient signal.

2. Semantic Benefits: Word embeddings provide semantic similarity beyond classification. “good” → “serviceable” relationships enable analogical reasoning unavailable in BOW.

3. Computational Trade-offs: BOW with 10K features is sparse but fast. Embeddings are dense (200D) but require pre-training or external models.

4. Subword Modeling: FastText’s character n-grams handle misspellings and rare words better than Word2Vec. Review data contains informal language where this matters.

5. Mean Pooling Limitation: Averaging word vectors loses sequential information. More sophisticated aggregation (weighted averaging, RNN encoding) might improve embedding performance further.

Prediction Examples

Negative Review:

“The movie was totally boring. The story was dull and the editor did a horrible job at editing this movie.”

• BOW (SVM): Negative ✓
• Word2Vec (LR): Negative ✓
• FastText: Negative ✓

All models correctly identify negative sentiment from words like “boring”, “dull”, “horrible”.

Positive Review:

“I loved the way all the Spider-Man movies were pulled into 1 story. I also loved how Spider-Man saved all his villains.”

• BOW (SVM): Positive ✓
• Word2Vec (LR): Positive ✓
• FastText: Positive ✓

“Loved” appears twice, providing strong positive signal recognized by all approaches.

Conclusion

Sentiment classification on movie reviews achieves 87-89% accuracy across BOW and embedding approaches. FastText performs best (88.98%) through subword modeling, while BOW + Logistic Regression provides competitive results (88.0%) with simpler implementation.

The marginal accuracy gain from embeddings (1-2%) must be weighed against increased complexity. For production systems requiring only classification, BOW remains effective. For applications needing semantic similarity (recommendation, search), embeddings justify the additional effort despite modest classification improvement.

Word2Vec and FastText provide interpretable semantic spaces where “good” relates to “serviceable” and “halfdecent”. This semantic structure enables downstream tasks beyond classification, making embeddings valuable even when classification accuracy is similar to BOW.

Full implementation: Kaggle Notebook

Introduction to Text Classification with Pre-trained Models

Text classification is a core task in Natural Language Processing (NLP), where the goal is to assign predefined labels to text. In recent years, pre-trained models like BERT, GPT, and their variants have greatly advanced the state of text classification, enabling strong performance across many NLP tasks. These models, trained on vast amounts of data, can either be fine-tuned on a specific task or applied directly for inference with little modification.

In this post, we will focus on several approaches to text classification:

• Using pre-trained models like BERT.
• Applying zero-shot classification where the model is able to classify text into labels it hasn’t seen during training.
• Classifying text using embeddings and traditional machine learning models.
• Exploring prompt-based classification using generative models like GPT.

Loading the Dataset and Necessary Libraries

Before diving into the models, let’s start by loading the IMDb movie reviews dataset, which contains reviews labeled as either positive or negative. We’ll also import the necessary libraries.

import pandas as pd
from transformers import pipeline
from sentence_transformers import SentenceTransformer

# Paths to the datasets
splits = {'train': 'IMDB_train.csv', 'validation': 'IMDB_validation.csv', 'test': 'IMDB_test.csv'}

# Loading the data from a remote source (Hugging Face Datasets)
train_df = pd.read_csv("hf://datasets/jahjinx/IMDb_movie_reviews/" + splits["train"])
test_df = pd.read_csv("hf://datasets/jahjinx/IMDb_movie_reviews/" + splits["test"])
valid_df = pd.read_csv("hf://datasets/jahjinx/IMDb_movie_reviews/" + splits["validation"])

Using Pre-trained Models for Text Classification

Pre-trained models are language models trained on extensive corpora, such as BERT or DistilBERT. Instead of training from scratch, we can use these models to classify text directly by fine-tuning them or applying them with minimal changes. Hugging Face’s pipeline API makes this process straightforward.

Let’s use the nlptown/bert-base-multilingual-uncased-sentiment model for sentiment classification of movie reviews.

from transformers import pipeline

model_path = "nlptown/bert-base-multilingual-uncased-sentiment"

pipe = pipeline(
    model=model_path,
    tokenizer=model_path,
    return_all_scores=True, 
    device="cuda:0"
)

In this setup:

• model: We use the pre-trained BERT model fine-tuned for sentiment classification.
• return_all_scores: This ensures the model outputs scores for all sentiment classes, not just the top prediction.
• device: Using GPU for faster inference.

Classifying the Text Data

Now that we’ve set up the pipeline, let’s classify some text from our validation dataset.

import numpy as np
from tqdm import tqdm

# Store predictions
y_pred = []

# Extract the list of texts from the validation dataset
texts = valid_df['text'].tolist()

# Classify each text
for text in tqdm(texts, total=len(texts)):
    output = pipe(text, max_length=512, truncation=True, padding=True)[0]
    scores = {entry['label']: entry['score'] for entry in output}
    
    # Assign positive or negative sentiment based on score
    negative_score = scores.get('NEGATIVE', 0)
    positive_score = scores.get('POSITIVE', 0)
    assignment = 1 if positive_score > negative_score else 0
    y_pred.append(assignment)

Evaluating Model Performance

We evaluate the model using common classification metrics, such as precision, recall, and F1 score:

from sklearn.metrics import classification_report

# Assuming y_true contains the ground truth labels
classification_report(
    y_true, y_pred,
    target_names=["Negative Review", "Positive Review"]
)

The model achieves an accuracy of 87%, demonstrating strong performance with minimal tuning.

Zero-Shot Classification

Zero-shot classification allows models to predict labels they haven’t been explicitly trained on. Instead of needing a fixed set of labels, the model can classify text into new categories based on its understanding of language semantics. This is especially useful when we don’t have labeled training data for every possible class.

Example: Zero-Shot Sentiment Classification

With Hugging Face’s zero-shot-classification model, we can classify sentiment without explicitly training for this task.

# Initialize the zero-shot classification pipeline
zero_shot_pipe = pipeline(
    model="zero-shot-classification",
    device=0
)

# Define potential labels
labels = ["positive", "negative"]

y_pred = []

# Classify text in the validation set
texts = valid_df['text'].tolist()

for text in tqdm(texts, total=len(texts)):
    output = zero_shot_pipe(text, labels)
    y_pred.append(output['labels'][0])

In this example, we ask the model to classify reviews as either “positive” or “negative,” even though the model wasn’t explicitly trained for these labels.

Evaluating the Zero-Shot Model

classification_report(
    y_true, y_pred,
    target_names=["Negative Review", "Positive Review"]
)

With 87% accuracy, this demonstrates the power of zero-shot learning, allowing the model to generalize to new tasks without specific fine-tuning.

Embeddings-Based Classification

Another approach is to use embeddings to represent text in a high-dimensional space, where similar texts are closer together. These embeddings can then be used as features for traditional machine learning classifiers, such as logistic regression.

We’ll use Sentence Transformers to generate embeddings for the text data.

Generating Embeddings with Sentence Transformers

from sentence_transformers import SentenceTransformer

# Load the pre-trained SentenceTransformer model
model = SentenceTransformer("w601sxs/b1ade-embed-kd_3", trust_remote_code=True)

# Generate embeddings for train, validation, and test datasets
train_embeddings = model.encode(train_df["text"], show_progress_bar=True)
test_embeddings = model.encode(test_df["text"], show_progress_bar=True)
valid_embeddings = model.encode(valid_df["text"], show_progress_bar=True)

Classification Using Logistic Regression

Once we have the embeddings, we can classify the text using a simple machine learning algorithm like logistic regression.

from sklearn.linear_model import LogisticRegression

# Train a logistic regression classifier on the training embeddings
clf = LogisticRegression(random_state=42)
clf.fit(train_embeddings, train_df["label"].values)

# Predict labels for the test embeddings
y_pred = clf.predict(test_embeddings)

Evaluating the Logistic Regression Model

classification_report(
    test_df["label"].values, y_pred,
    target_names=["Negative Review", "Positive Review"]
)

Even with logistic regression, the embeddings-based model achieves an 87% accuracy, showcasing the effectiveness of using embeddings for text classification.

Prompt-Based Classification with Generative Models

Beyond pre-trained models and embeddings, generative models like GPT can also be used for classification. By crafting specific prompts, we can frame the task as a question-answer scenario, where the model generates the class label.

How Prompt-Based Classification Works

We provide the model with an input (e.g., a movie review) and ask it a question about the input’s class label (e.g., “Is the sentiment positive or negative?”). The model generates a response with the classification label.

Example: Sentiment Classification Using GPT-2

from transformers import pipeline

# Load the GPT-2 model and tokenizer using Hugging Face's pipeline
gpt_pipe = pipeline(
    model="gpt2",
    tokenizer="gpt2",
    device=0  # Use GPU if available
)

# Example text to classify
text = "This movie was absolutely fantastic! The plot was gripping and the characters were well-developed."

# Crafting a prompt for the model
prompt = f"{text}\nQuestion: Is the sentiment of this review positive or negative?\nAnswer:"

# Generate the classification result
response = gpt_pipe(prompt, max_length=50, num_return_sequences=1)[0]['generated_text']

# Output the response
print(response)

Here, the model generates a response to the question embedded in the prompt, such as “positive.”

Crafting Effective Prompts

The success of this method depends heavily on how the prompt is designed. You can:

• Be Clear and Specific: The model works best when given clear, specific questions (e.g., “Is the sentiment positive or negative?”).
• Multi-Class Classification: You can handle more than two classes by adjusting the prompt (e.g., “Is the sentiment very positive, positive, neutral, negative, or very negative?”).

Conclusion

In this post, we explored multiple techniques for text classification:

1. Pre-trained models like BERT allow for easy, high-performing classification with minimal fine-tuning.
2. Zero-shot classification enables models to classify text into labels they haven’t explicitly seen during training.
3. Embeddings-based classification offers a flexible way to use pre-trained embeddings and traditional machine learning algorithms.
4. Generative models like GPT can be repurposed for classification using prompt-based techniques.

We achieved up to 87% accuracy across several approaches. Each method has its strengths and trade-offs, making them suitable for different scenarios.

In the next post, we will explore deep learning models like LSTMs and Transformers, diving deeper into custom-built text classification systems. Stay tuned for more hands-on examples!

Text classification is a core task in natural language processing (NLP) that involves categorizing text into predefined labels or categories. This process is used across many applications, such as sentiment analysis, spam detection, language identification, and topic categorization.

In this blog post, we’ll walk through the progression of text classification methods, starting from the basic Bag of Words (BOW) model, moving through TF-IDF, XGBoost, and concluding with deep learning models such as CNNs, RNNs, and Transformers. By examining each of these techniques, you will understand the pros and cons of each and gain a solid foundation for choosing the right model for your task.

For this post, we’ll use the IMDB movie reviews dataset from Hugging Face’s dataset repository to showcase the implementation and evaluation of each method. You can find the dataset here.

Introduction to Text Classification

At its core, text classification assigns a label to a piece of text. For example, given a movie review, a text classification model can predict whether the review is positive or negative.

A typical pipeline for text classification includes the following steps:

1. Preprocessing: Clean and tokenize the text.
2. Feature Extraction: Convert the text into a numerical representation that machine learning models can interpret.
3. Model Training: Train the model on labeled text data to learn the mapping between text and labels.
4. Evaluation: Measure the model’s performance on test data.
5. Prediction: Use the trained model to predict labels for new data.

There are different approaches to each of these steps, and over the years, various techniques have been introduced to improve the classification performance. Let’s start with the Bag of Words (BOW) model.

Bag of Words (BOW) Model

What is BOW?

Bag of Words (BOW) is one of the simplest techniques for text classification. It represents text as an unordered collection of words, completely ignoring word order and treating each word as an independent feature. Each word in the text is represented by its frequency in the document.

For example, given two sentences:

• “The dog chased the cat.”
• “The cat chased the dog.”

The BOW representation will treat both sentences the same, because it ignores word order. This leads to some key limitations, which we’ll discuss shortly.

Limitations of BOW

1. No Context or Word Order: The BOW model loses important information by ignoring word order. For instance, “The dog chased the cat” is different from “The cat chased the dog,” but BOW would treat them identically.
2. High Dimensionality: Each unique word is treated as a feature, leading to high-dimensional feature spaces, which can be computationally expensive.
3. Sparse Representation: Since most documents only contain a small fraction of the entire vocabulary, the resulting feature matrix is sparse (containing many zeros).

Implementing BOW

Let’s now see how to implement the Bag of Words model using the IMDB movie reviews dataset. First, we load the dataset and inspect it.

import pandas as pd

# Load the dataset
splits = {'train': 'IMDB_train.csv', 'validation': 'IMDB_validation.csv', 'test': 'IMDB_test.csv'}
train_df = pd.read_csv("hf://datasets/jahjinx/IMDb_movie_reviews/" + splits["train"])
test_df = pd.read_csv("hf://datasets/jahjinx/IMDb_movie_reviews/" + splits["test"])
valid_df = pd.read_csv("hf://datasets/jahjinx/IMDb_movie_reviews/" + splits["validation"])

# Example review and label
print(f"text :{train_df.iloc[0,0]}")
print(f"label :{train_df.iloc[0,1]}")

text :Beautifully photographed and ably acted, generally, but the writing is very slipshod. There are scenes of such unbelievability that there is no joy in the watching. The fact that the young lover has a twin brother, for instance, is so contrived that I groaned out loud. And the "emotion-light bulb connection" seems gimmicky, too.<br /><br />I don't know, though. If you have a few glasses of wine and feel like relaxing with something pretty to look at with a few flaccid comedic scenes, this is a pretty good movie. No major effort on the part of the viewer required. But Italian film, especially Italian comedy, is usually much, much better than this.
label :0

Each review in the dataset has a corresponding label (0 for negative, 1 for positive). Next, we’ll preprocess the data.

Preprocessing and Vectorization

We will clean the text by removing punctuation, converting the text to lowercase, and tokenizing it into words. Afterward, we will use CountVectorizer from the scikit-learn library to create a Bag of Words representation of the text.

import re
from sklearn.feature_extraction.text import CountVectorizer
from nltk.tokenize import word_tokenize

# Preprocess function to clean text
def clean_text(text):
    return re.sub(r"[^\w\s+]", '', text)

# Initialize CountVectorizer for BOW representation
vectorizer = CountVectorizer(lowercase=True, preprocessor=clean_text,
                             tokenizer=word_tokenize, stop_words='english', max_features=300)

# Fit and transform the training data
train_text_array = vectorizer.fit_transform(train_df['text'])
test_text_array = vectorizer.transform(test_df['text'])

The output of the above code gives a sparse matrix where each row represents a document (movie review), and each column represents a word from the vocabulary (with a maximum of 300 words). The matrix stores the frequency of words in each document.

Classification with Logistic Regression

Now, we will use Logistic Regression to classify the reviews as positive or negative based on the Bag of Words representation.

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report

# Train logistic regression model
model = LogisticRegression()
model.fit(train_text_array, train_df['label'].values)

# Predict and evaluate on the test set
preds = model.predict(test_text_array)
print("Accuracy: ", model.score(test_text_array, test_df['label'].values))
print(classification_report(test_df['label'].values, preds))

Results

As we can see, the Bag of Words model performs reasonably well, achieving an accuracy of around 79%. However, there is room for improvement, especially in capturing the semantic meaning and context of the text. This is where TF-IDF comes into play.

TF-IDF with Logistic Regression

What is TF-IDF?

TF-IDF (Term Frequency-Inverse Document Frequency) is a more refined version of BOW. It calculates the importance of a word by considering how frequently it appears in a document (TF) and how unique it is across all documents in the corpus (IDF). Words that appear frequently but are not common across many documents receive higher weights, allowing TF-IDF to better capture the importance of specific words.

The formula for TF-IDF is:

Where:

• TF(t, d): Term frequency, the number of times term t appears in document d.
• DF(t): Document frequency, the number of documents that contain the term t.
• N: Total number of documents.

Implementing TF-IDF

We will now apply TF-IDF to the movie review dataset using TfidfVectorizer from scikit-learn.

from sklearn.feature_extraction.text import TfidfVectorizer

# Apply TF-IDF transformation
tfidf_vectorizer = TfidfVectorizer(lowercase=True, preprocessor=clean_text,
                                   tokenizer=word_tokenize, stop_words='english', max_features=300)
train_text_tfidf = tfidf_vectorizer.fit_transform(train_df['text'])
test_text_tfidf = tfidf_vectorizer.transform(test_df['text'])

# Train and evaluate logistic regression
model.fit(train_text_tfidf, train_df['label'].values)
preds = model.predict(test_text_tfidf)
print("Accuracy: ", model.score(test_text_tfidf, test_df['label'].values))
print(classification_report(test_df['label'].values, preds))

Results

TF-IDF achieves nearly the same performance as Bag of Words. However, TF-IDF often performs better on larger and more complex datasets by giving more weight to important but rare words.

Gradient Boosting with XGBoost

XGBoost is a popular machine learning algorithm that uses gradient boosting. It is known for its high performance in many machine learning competitions and its ability to handle large datasets efficiently. By iteratively improving weak learners (usually decision trees), XGBoost creates a strong model.

XGBoost can handle sparse input data, making it a good fit for text classification using BOW or TF-IDF features.

Implementing XGBoost

We will train an XGBoost model on the TF-IDF features of the movie reviews.

import xgboost as xgb

# Prepare data in DMatrix format
dtrain = xgb.DMatrix(train_text_tfidf, label=train_df['label'].values)
dtest = xgb.DMatrix(test_text_tfidf, label=test_df['label'].values)

# Define model parameters
param = {'max_depth': 6, 'eta': 0.3, 'objective': 'binary:logistic'}
num_round = 10

# Train XGBoost model
import xgboost as xgb
from ray import train, tune
from ray.tune.schedulers import ASHAScheduler
from ray.tune.integration.xgboost import TuneReportCheckpointCallback

def train_model(config,train_x,train_y,test_x,test_y):
    dtrain = xgb.DMatrix(train_x,label=train_y)
    dtest = xgb.DMatrix(test_x,label=test_y)
    results = {}
    bst = xgb.train(config,
                    dtrain,
                    num_boost_round=10,
                    evals=[(dtest,'test')],
                    evals_result=results,
                    callbacks=[TuneReportCheckpointCallback(frequency=1)]
                    )
    accuracy = 1-results['test']['error'][-1]
    train.report({"Mean Accuracy": accuracy, "done": True})

def get_best_model_checkpoint(results):
    best_result = results.get_best_result()
    best_bst = TuneReportCheckpointCallback.get_model(best_result.checkpoint)
    accuracy = 1-best_result.metrics['test-error']
    print(f"Best model parameters: {best_result.config}")
    print(f"Best model found has an accuracy of {accuracy:4f}")
    return best_bst


def tune_model(train_x,train_y,test_x,test_y,smoke_test=False):
    search_space = {
        'objective':'binary:logistic',
        'eval_metric': ['error','logloss'],
        'max_depth': tune.randint(1,9),
        'max_child_weight': tune.choice([1,2,3]),
        'subsample': tune.uniform(0.5,1),
        'eta': tune.loguniform(1e-4,1e-1),
    }

    scheduler = ASHAScheduler(
        max_t=10,
        grace_period=1,
        reduction_factor=2
    )

    tuner = tune.Tuner(
        tune.with_parameters(train_model,
                            train_x=train_x,
                            train_y=train_y,
                            test_x=test_x,
                            test_y=test_y),
        tune_config=tune.TuneConfig(metric="test-logloss",
                                    mode="min",
                                    scheduler=scheduler,
                                    num_samples= 1 if smoke_test else 100,
                                    ),
        param_space=search_space

    )

    results = tuner.fit()
    return results

results = tune_model(
    train_text_array,
    train_df['label'].values,
    test_text_array,
    test_df['label'].values,
    smoke_test=False
)

Results

• Accuracy: 71.7%

XGBoost performs worse than logistic regression in this case, but it may excel on other datasets or with more in depth hyperparameter tuning.

Deep Learning Models: CNNs and RNNs

Traditional models like Logistic Regression and XGBoost are limited in their ability to capture the sequential nature of text. Deep learning models, particularly Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), can automatically learn patterns in text and account for both local and global dependencies.

• CNNs are good at detecting local patterns, such as word n-grams.
• RNNs can capture long-range dependencies and are excellent for modeling sequences.

Convolutional Neural Networks (CNNs)

A CNN applies filters across the text to detect patterns such as combinations of words. These patterns are then used for classification.

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader, TensorDataset

# Define CNN model
class CNNClassifier(nn.Module):
    def __init__(self, input_dim):
        super(CNNClassifier, self).__init__()
        self.conv1 = nn.Conv1d(in_channels=1, out_channels=128, kernel_size=3)
        self.fc1 = nn.Linear(128, 1)

    def forward(self, x):
        x = self.conv1(x.unsqueeze(1))
        x = torch.relu(x)
        x = torch.max(x, 2)[0]
        x = self.fc1(x)
        return x

Recurrent Neural Networks (RNNs)

RNNs are designed to handle sequential data. They can maintain information about previous words in the sequence, which helps them capture dependencies in text.

class RNNClassifier(nn.Module):
    def __init__(self, input_dim):
        super(RNNClassifier, self).__init__()
        self.rnn = nn.RNN(input_dim, 128, 2, batch_first=True)
        self.fc1 = nn.Linear(128, 1)

    def forward(self, x):
        _, hn = self.rnn(x.unsqueeze(1))
        x = self.fc1(hn[-1])
        return x

Results of CNN and RNN

RNNs perform significantly better than CNNs in this case, likely due to their ability to model the sequential nature of text more effectively and the performance of RNN is along the lines of the logistic regression model.

The poor performance of the CNN model is expected, we may see improvement by using n-grams instead of single words

The Transformer Era

Transformers revolutionized NLP by introducing the self-attention mechanism, which allows models to capture long-range dependencies in text more efficiently than RNNs. Models like BERT and GPT are built on the transformer architecture and have set new benchmarks in NLP tasks.

Transformers are capable of:

• Understanding context better: They capture both local and global relationships in the text.
• Scaling better: Transformers can be trained in parallel, unlike RNNs which process sequences one step at a time.

We will explore transformers like BERT and GPT in the next blog post and learn how to implement them for text classification.

Conclusion and Next Steps

In this blog post, we walked through the progression of text classification techniques:

1. Bag of Words (BOW): Simple but loses word order and context.
2. TF-IDF: Adds importance to rare but significant words.
3. XGBoost: A strong boosting algorithm that handles sparse data.
4. Deep Learning (CNNs and RNNs): Models that capture local and sequential patterns.
5. Transformers: The state-of-the-art models for NLP.

Each technique offers different advantages, but the choice of model depends on the task at hand. Transformers represent the cutting edge of NLP and are typically the best choice for complex text classification tasks.

In the next post, we will explore transformers like BERT and GPT in detail and learn how to implement them for text classification. We’ll also dive into the concept of word embeddings and how they enhance deep learning models in NLP.

Stay tuned!

Today, we’ll look at two popular methods for understanding text using topic vectors—Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA). These approaches will help us break free from traditional keyword searches and allow us to capture the deeper context behind words.

Beyond Keyword Search: Why Topic Modeling?

Imagine you’re searching for something online, but you’re struggling to find the exact words to describe what you’re looking for. Traditional keyword search engines might not return helpful results. Now, imagine a search engine that can understand your intent, even if your query doesn’t perfectly match the keywords in a document. That’s where semantic search powered by topic modeling shines!

Topic vectors provide a way to represent words and documents in terms of topics rather than just words. This shift opens the door to incredible applications:

• Semantic Search: Find documents based on their meaning, not just exact word matches.
• Keyword Extraction: Automatically identify the most relevant words and phrases that summarize a document’s content.
• Document Comparison: Measure how close two documents are in meaning, even if they don’t use the same words.

In this post, we’ll explore how topic vectors work, the challenges of traditional methods like TF-IDF, and how algorithms like LSA and LDA help us build more intelligent systems.

The Problem with Word Frequency and TF-IDF

Before we dive into topic vectors, let’s talk about TF-IDF (Term Frequency-Inverse Document Frequency) and why it’s limited in understanding meaning.

TF-IDF is a popular technique that counts how often a word appears in a document and adjusts that frequency by how rare the word is across all documents. While it’s useful, it has some serious drawbacks when it comes to capturing the meaning behind words:

• Synonyms Are Ignored: Words like “happy” and “joyful” might mean the same thing, but TF-IDF treats them as completely unrelated.
• Polysemy Creates Confusion: Many English words have multiple meanings. For example, the word “band” can refer to a music group or something you wear in your hair. TF-IDF doesn’t understand these nuances and can mix up unrelated contexts.

Homonyms and Polysemy: A Headache for TF-IDF

Let’s take a closer look at how polysemy (words with multiple meanings) creates problems:

• Homonyms: Words that are spelled and pronounced the same but have different meanings, like “band” (music group) and “band” (hair accessory).
• Homographs: Words that are spelled the same but pronounced differently, like “object” (noun) and “object” (verb).
• Zeugma: A fun example where a single word is used with two meanings in the same sentence, like “She stole his heart and his wallet.”

In all of these cases, TF-IDF struggles to capture the true meaning behind the words. We need something more sophisticated—something that can dive deeper into the relationships between words. This is where topic vectors come into play.

Topic Vectors: Capturing Meaning Beyond Words

Now that we understand the limitations of TF-IDF, let’s introduce topic vectors. These vectors help us represent the meaning of documents based on the topics they discuss, rather than focusing on individual word counts.

So, how do we build topic vectors? Imagine we’ve already tokenized our text and created a Bag-of-Words (BOW) or TF-IDF vector. The next step is to transform these vectors into topic space, where each dimension represents a specific topic.

How Topic Vectors Work

Let’s say we have three topics: music, fashion, and politics. Each word in our document will contribute to one or more of these topics. For example, words like “guitar” and “concert” would contribute more to the music topic, while words like “vote” and “president” would contribute to the politics topic.

Now, let’s imagine a document that talks about a musician who used to be a model and now plays a role in politics. Our document contains words like “guitar,” “model,” and “president.” We can calculate how much each word contributes to each topic and create a topic vector:

# Sample topic vector calculation
topic = {}
doc = "The Politician used to be a model before he became a politician, he plays guitar and is a member of a band"
tfidf = dict(list(zip("politician model guitar band".split(), [0.5, 0.1, 0.4, 0.6])))

# Weights for each topic
topic['music'] = 0*tfidf['politician'] + 0*tfidf['model'] + 0.5*tfidf['guitar'] + 0.5*tfidf['band']
topic['politics'] = 0.6*tfidf['politician'] + 0.5*tfidf['model'] + 0*tfidf['guitar'] + 0*tfidf['band']
topic['fashion'] = 0.1*tfidf['politician'] + 0.6*tfidf['model'] + 0*tfidf['guitar'] + 0*tfidf['band']

Resulting Topic Vectors:

{'music': 0.5, 'politics': 0.35, 'fashion': 0.11}

This tells us that this document is primarily about music and politics, with a smaller connection to fashion. With this kind of representation, we can easily compare documents, search based on topics, and understand their overall meaning.

Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA)

Now, how do we actually create these topic vectors from real-world text? There are two common methods: Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA).

Latent Semantic Analysis (LSA)

LSA uses a mathematical technique called Singular Value Decomposition (SVD) to decompose a large matrix of word co-occurrences (like a TF-IDF matrix) into two smaller matrices—one representing words and the other representing topics. By reducing the dimensions, LSA can group together words that appear in similar contexts, which helps to capture the latent meaning behind words.

Latent Dirichlet Allocation (LDA)

LDA takes a different approach. It assumes that every document is a mix of topics, and every word in the document is drawn from one of these topics. The LDA algorithm then works to figure out the most likely topics for each document and the most likely words for each topic. Unlike LSA, LDA is a probabilistic model, which makes it particularly good at handling complex text corpora.

Both LSA and LDA are unsupervised learning algorithms, meaning they don’t require labeled data to work their magic. They help us discover the hidden structure within text—perfect for tasks like topic detection and semantic search.

Wrapping Up: Why Topic Vectors Matter

In today’s data-rich world, understanding the meaning behind words is more important than ever. Simple keyword searches and word counts often miss the mark, but with topic vectors, we can capture the deeper semantic relationships between words and documents.

Whether you’re building a search engine, a recommendation system, or just exploring a large collection of text, topic modeling techniques like LSA and LDA can uncover insights that would otherwise be hidden.

By transforming documents into topic vectors, we can:

• Perform Semantic Search: Find documents based on meaning, not just exact word matches.
• Compare Document Similarity: Measure how closely two documents are related, even if they use different words.
• Extract Keywords: Automatically identify the most relevant words or phrases to summarize a document.

So the next time you’re working with text, think beyond keywords. Dive into the world of topic vectors and discover the hidden meanings behind the words.

In this blog, we’ll explore the foundations of n-gram models, how they work, their strengths and limitations, and how they compare to more sophisticated approaches. Understanding these classical techniques provides valuable insights into the evolution of language modeling and why more advanced models were developed.

For example, given the sentence:

The quick brown fox jumps over the lazy ____

We might predict the next word as “dog,” “cat,” or “river” based on the context, but not function words like “the” or “this.”

A language model is a machine learning model that predicts the next word in a sequence and assigns probabilities to each possible word. One of the simplest types of language models is the n-gram model.

An n-gram is a sequence of n words:

• A 1-gram (unigram) is a single word.
• A 2-gram (bigram) is a sequence of two words.
• A 3-gram (trigram) is a sequence of three words, and so on.

When we refer to an n-gram model, we mean a model that predicts the $n^{th}$ word given the previous $n-1$ words. These models provide a simple yet effective way to estimate probabilities in a text sequence and serve as a foundation for more advanced techniques.

How Do N-gram Models Work?

To understand n-gram models, let’s begin with the fundamental task of computing the probability of a word $w$ given some history $h$:

where:

• $C(w,h)$ is the count of occurrences of the sequence $(h, w)$ in a corpus.
• $C(h)$ is the count of the history $h$ appearing in the corpus.

For example, given the sentence:

The quick brown fox jumps over the lazy ____

we want to compute the probability of the word “dog” appearing next:

If we had a large enough dataset, we could estimate these probabilities by counting occurrences. However, natural language is highly diverse, and many word combinations might rarely appear in training data, making it difficult to estimate probabilities directly. To address this, we need a more practical way to approximate probabilities.

The Markov Assumption and N-gram Approximation

This is where n-gram models come in. They simplify the problem by making a key assumption: the probability of a word depends only on the previous $n-1$ words, rather than the entire history. This is known as the Markov assumption:

This assumption drastically reduces the complexity of computing probabilities, making it feasible to estimate them using limited data.

For example, instead of:

we approximate it as:

if we use a bigram model (n=2), meaning we consider only the last word. Similarly, for a trigram model (n=3), we would consider the last two words:

Thus, in general, an n-gram model estimates probabilities as:

where:

• Bigram model (n=2):
  \(P(w|h) \approx P(w|h_{n-1})\)
• Trigram model (n=3):
  \(P(w|h) \approx P(w|h_{n-1}, h_{n-2})\)

By using this approximation, n-gram models allow us to compute word probabilities efficiently while capturing local word dependencies.

Estimating N-gram Probabilities

Given the Markov assumption, we can estimate n-gram probabilities using maximum likelihood estimation (MLE).

We get MLE estimates by counting occurrences of n-grams in a corpus and normalizing them to obtain probabilities. For example, to estimate bigram probabilities, we count the occurrences of each word pair $(h, w)$ and divide by the total count of history $h$:

where:

• $C(h, w)$ is the count of the bigram $(h, w)$.
• $C(h)$ is the count of the history $h$.

Practical Example: Estimating a Bigram Probability

Let’s go through a practical example using The Verdict - Wikisource, the free online library.

Loading the Corpus

import re
from collections import Counter
from nltk.util import ngrams

with open("the-verdict.txt", "r") as f:
    data = f.read()

words = data.replace("--", " ").lower().split()
len(words)

3731

Our corpus contains a total of 3731 words. Now, let’s implement a bigram model.

Extracting Bigrams

We first extract all possible bigrams from the text.

bigrams = list(ngrams(words, 2))
bigrams[:10]

[('i', 'had'),
 ('had', 'always'),
 ('always', 'thought'),
 ('thought', 'jack'),
 ('jack', 'gisburn'),
 ('gisburn', 'rather'),
 ('rather', 'a'),
 ('a', 'cheap'),
 ('cheap', 'genius'),
 ('genius', 'though')]

Now, let’s say we want to compute $P(\text{“gisburn”}|\text{“jack”})$, the probability of the word gisburn appearing after jack.

Step 1: Count All Bigrams Starting with ‘Jack’

We extract all bigrams where "jack" is the first word to find possible next words.

jack_bigrams = [bigram for bigram in bigrams if bigram[0] == 'jack']
print(jack_bigrams)

[('jack', 'gisburn'), ('jack', 'gisburn!'), ('jack', 'one'), ('jack', 'himself,'), ('jack', 'himself')]

There are 5 bigrams where "jack" is the first word.

Step 2: Count Occurrences of (‘Jack’, ‘Gisburn’)

We count how many times "gisburn" follows "jack" in the corpus.

jack_gisburn_count = sum(1 for bigram in jack_bigrams if bigram[1] == 'gisburn')
jack_gisburn_count

1

The bigram ("jack", "gisburn") appears once.

Step 3: Compute Probability

Finally, we compute the probability:

jack_count = len(jack_bigrams)
jack_gisburn_prob = jack_gisburn_count / jack_count
jack_gisburn_prob

0.2

Thus, the probability of “gisburn” appearing after “jack” is 0.2 (or 20%).

Relative Frequency and Maximum Likelihood Estimation (MLE)

The ratio

is also called the relative frequency.

Using relative frequencies to estimate probabilities is a simple yet effective approach for computing maximum likelihood estimation (MLE) in n-gram models. However, directly multiplying probabilities can introduce computational challenges.

The Problem of Underflow and Log Probabilities

Language models can be very large, leading to practical issues—one of which is computing probabilities. Since probabilities are small values between 0 and 1, multiplying many probabilities together results in very small numbers, leading to computational underflow. To mitigate this, we often use log probabilities, which are more numerically stable.

Instead of:

we compute:

By summing log probabilities instead of multiplying them, we avoid numerical instability while preserving the relationships between probabilities.

Evaluating N-gram Models

To assess the performance of an n-gram model, we first split our data into training, development, and test sets. The model is trained on the training set, fine-tuned on the development set, and evaluated on the test set, which measures how well the model generalizes to unseen data.

Training set: Data used to learn the model parameters.

Development set (Dev set): A separate dataset used to tune hyperparameters and compare different models before final evaluation.

Test set: A held-out dataset used to assess the final model’s performance.

The development set is crucial because it helps prevent overfitting. If we tuned hyperparameters directly on the test set, we might unknowingly optimize the model for that specific dataset rather than ensuring generalization. By using a separate development set, we ensure that the test set remains an unbiased measure of the model’s actual performance.

A good language model should generalize well to unseen text. A model that fits the training data perfectly but fails on new data is overfitting, making it ineffective in real-world applications. We measure the quality of an n-gram model by its performance on an unseen test corpus.

Perplexity

To evaluate n-gram models, we compare how well they assign probabilities to a test set. However, using raw probabilities isn’t ideal because:

• The probability of a test set decreases as the text length increases.
• Comparing models based on raw probability is not straightforward.

Instead, we use perplexity, a function of probability that normalizes for text length.

Definition of Perplexity

The perplexity (PP or PPL) of a language model on a test set is:

where:

• $PP(W)$ is the perplexity of the test set $W$.
• $P(w_1, w_2, w_3, …, w_N)$ is the probability assigned to the test set.
• $N$ is the total number of words in the test set.

Interpreting Perplexity

Perplexity quantifies how uncertain or ‘surprised’ a model is when predicting text. A lower perplexity indicates:

• The model assigns higher probabilities to correct word sequences.
• The model is more confident in its predictions.

For example:

• A perfect model that assigns a probability of 1 to the correct test set would have a perplexity of 1.
• A random model that assigns equal probability to all words would have a high perplexity.
• A better-performing model will have a lower perplexity than a weaker one.

Since different language models yield different perplexities for the same text, perplexity is a useful metric for comparing models.

Limitations of N-gram Models

While n-gram models are simple and computationally efficient, they have several limitations:

Data Sparsity: As n increases, the number of possible n-grams grows exponentially, making it difficult to collect enough data to estimate probabilities accurately.

Fixed Context Window: N-gram models only consider a limited number of preceding words, ignoring long-range dependencies in language.

Poor Generalization: They rely on exact word matches, meaning they struggle with unseen n-grams or slight variations in phrasing.

High Memory Usage: Storing large n-gram counts requires significant memory, especially for higher-order models.

To address these limitations, modern language models use smoothing techniques (like Laplace and Kneser-Ney smoothing) and neural-based approaches like Recurrent Neural Networks (RNNs), LSTMs, and Transformers.

Conclusion

N-gram language models represent a foundational approach to language modeling. They offer a simple and interpretable way to estimate word probabilities but struggle with long-range dependencies and data sparsity. Despite their limitations, they have played a crucial role in NLP and continue to be useful in applications like text compression, speech recognition, and information retrieval.

While deep learning models like Transformers have largely replaced n-gram models for complex NLP tasks, understanding n-grams provides a valuable perspective on the evolution of language modeling techniques. By combining classical approaches with modern advancements, we can build more efficient and effective NLP systems.

</div>

In this post, we’ll take a journey through the various vectorization techniques used in NLP, including one-hot encoding, Bag of Words (BOW), and TF-IDF. Let’s dive in and explore how we can convert text into numbers that power everything from spam filters to recommendation systems!

The Challenge: Why Do We Need Vectorization?

Imagine trying to teach a computer to recognize spam emails. You can’t just feed it the raw email text—it would have no idea what to do with it! What we need is a way to turn text into a format that machine learning models can digest—numbers. That’s where vectorization comes in. It converts words into vectors (arrays of numbers) that a machine learning algorithm can understand and learn from.

Let’s say we have these three simple sentences:

1. “I love NLP”
2. “I hate NLP”
3. “I enjoy NLP”

Our task is to convert these sentences into vectors that capture their essence. This way, a model can easily process them, spot patterns, and learn from the data.

One-Hot Encoding: The Basic Building Block

Let’s start with one-hot encoding, the simplest and most intuitive way to represent words. Each word in our vocabulary gets its own unique vector. In this case, our vocabulary is ["I", "love", "hate", "enjoy", "NLP"]. For each word, we create a vector that has a 1 in the position of the word and 0 elsewhere. Here’s what that looks like:

• “I”: [1, 0, 0, 0, 0]
• “love”: [0, 1, 0, 0, 0]
• “hate”: [0, 0, 1, 0, 0]
• “enjoy”: [0, 0, 0, 1, 0]
• “NLP”: [0, 0, 0, 0, 1]

We can represent each sentence as a collection of these one-hot vectors:

# Sentence: "I love NLP"
[[1, 0, 0, 0, 0],  # "I"
 [0, 1, 0, 0, 0],  # "love"
 [0, 0, 0, 0, 1]]  # "NLP"

While one-hot encoding is easy to understand, it has limitations. First, the vectors can get huge for large vocabularies. If you have 10,000 words, each vector will be 10,000 elements long! Worse, this method doesn’t capture any relationships between words. In one-hot encoding, “love” and “hate” are as unrelated as “apple” and “neutron star,” even though they are opposites.

Bag of Words (BOW): Counting Word Occurrences

A more sophisticated approach is the Bag of Words (BOW) model. Here, we ignore word order and grammar, focusing only on word frequency. Each document (sentence, paragraph, or entire text) is represented by a vector that counts the number of times each word in the vocabulary appears.

Here’s how it looks with our three sentences:

from sklearn.feature_extraction.text import CountVectorizer

sentences = [
    "I loved the movie",
    "Movie was awesome, movie characters were awesome",
    "Movie was terrible"
]

vectorizer = CountVectorizer()
X = vectorizer.fit_transform(sentences)
print(vectorizer.get_feature_names_out())
print(X.toarray())

Output:

['awesome', 'characters', 'loved', 'movie', 'terrible', 'the', 'was', 'were']
[[0 0 1 1 0 1 0 0]
 [2 1 0 2 0 0 1 1]
 [0 0 0 1 1 0 1 0]]

In this example, the words "movie", "awesome", and "was" show up multiple times, and each sentence is converted into a vector of word counts.

Limitations of BOW: While BOW captures word frequency, it ignores the word order and context. For instance, “I hate NLP” and “I love NLP” would appear quite similar in a BOW representation, even though they have opposite meanings!

TF-IDF: Adding Weight to Words

Enter TF-IDF (Term Frequency-Inverse Document Frequency), a more advanced vectorization technique that weighs words by how important they are in the document, compared to how common they are in the entire dataset.

The TF part measures how often a word appears in a document, while IDF measures how rare the word is across all documents. Words that are common across many documents (like “the” or “is”) get lower scores, while unique words get higher scores.

Term Frequency (TF) is calculated as follows:

where:

• $f_{t,d}$ is the frequency of term $t$ in document $d$.
• $\sum_{t’ \in d} f_{t’,d}$ is the total number of terms in document $d$.

Inverse Document Frequency (IDF) is calculated as follows:

where:

• $N$ is the total number of documents.
• $n_t$ is the number of documents containing term $t$.

The final TF-IDF score is the product of TF and IDF:

Here’s how TF-IDF works in action:

from sklearn.feature_extraction.text import TfidfVectorizer

sentences = [
    "I love NLP",
    "I hate NLP",
    "I enjoy NLP"
]

vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(sentences)
print(vectorizer.get_feature_names_out())
print(X.toarray())

Output:

['enjoy', 'hate', 'love', 'nlp']
[[0.         0.         0.861037   0.50854232]
 [0.         0.861037   0.         0.50854232]
 [0.861037   0.         0.         0.50854232]]

As you can see, TF-IDF gives higher importance to the unique words (“love,” “hate,” “enjoy”) while giving less weight to common words like “NLP” that appear in all three sentences.

What About Word Embeddings?

So far, we’ve been dealing with sparse, high-dimensional vectors that don’t capture relationships between words. But what if we want vectors that understand word meaning? That’s where word embeddings like Word2Vec, GloVe, and BERT come in.

Word embeddings create dense vectors, where words with similar meanings are close to each other in vector space. For example, the words “king” and “queen” might be close, and “man” and “woman” might also be near each other.

Unlike one-hot encoding or BOW, word embeddings can understand that words like “dog” and “puppy” are more similar than “dog” and “car.” These dense vectors are incredibly powerful for tasks like text classification, sentiment analysis, and even machine translation.

Use Case: Finding Similar Documents

Vectorization isn’t just for building models—it’s also handy for finding similar documents. Imagine you’re running a recommendation system and want to suggest articles that are similar to one a user has read.

We can use cosine similarity to compare the vectors of documents and find the most similar ones. Here’s an example of how we can use cosine similarity with BOW:

from sklearn.metrics.pairwise import cosine_similarity

sentences = [
    "Cat is drinking milk",
    "Dog is running behind the cat",
    "A man is riding a horse",
    "Cat is playing with the mouse",
]

vectorizer = CountVectorizer()
X = vectorizer.fit_transform(sentences)

new_doc = "Cat and mouse are playing"
Y = vectorizer.transform([new_doc])

cs = cosine_similarity(X, Y)

sorted_index = np.argsort(cs, axis=0)[::-1].flatten()
for idx in sorted_index:
    print(sentences[idx], cs[idx])

Output:

Cat is playing with the mouse [0.70710678]
Cat is drinking milk [0.28867513]
Dog is running behind the cat [0.23570226]
A man is riding a horse [0.]

The output shows how similar the new document is to existing ones, based on their BOW vector representation. This is the backbone of document recommendation systems!

Wrapping It Up: From Sparse to Dense Vectors

In this blog, we explored how vectorization transforms words into numbers through different techniques. One-hot encoding, Bag of Words, and TF-IDF are all powerful methods for turning text into data that machine learning models can process. While these methods have limitations (like not capturing word meaning), they are great for many text processing tasks.

If you want your model to understand word semantics, word embeddings are the next step in NLP, offering dense representations that carry rich information about words and their relationships.

Whatever your text-processing task—whether it’s document classification, sentiment analysis, or building a search engine—vectorization is a critical tool in your NLP toolkit.

How do you approach vectorization in your NLP projects? Do you prefer TF-IDF or word embeddings? Share your thoughts and experiences in the comments below!