NLP Foundations and the Transformer
- Text Processing: Tokenization, Stemming, Lemmatization
- Bag of Words, TF-IDF & Text Representation
- Word Embeddings: Word2Vec from Scratch
- GloVe, FastText & Subword Embeddings
- Sentiment Analysis
- Named Entity Recognition (NER)
- POS Tagging & Syntactic Parsing
- Text Classification — CNNs & RNNs for Text
- Sequence-to-Sequence Models
- Attention Mechanism — The Breakthrough
- Machine Translation
- Text Summarization
- Question Answering Systems
- Information Retrieval & Search
- Topic Modeling: LDA, BERTopic
- Text Generation
- Chatbots: Rule-Based to Neural
- Multilingual NLP
- Subword Tokenization: BPE, WordPiece, Unigram, SentencePiece
- Structured Outputs & Constrained Decoding
- NLI & Textual Entailment
- Embedding Models Deep Dive
- Chunking Strategies for RAG
- Coreference Resolution
- Entity Linking & Disambiguation
- Relation Extraction & Knowledge Graph Construction
- LLM Evaluation: RAGAS, DeepEval, G-Eval
- Long-Context Evaluation: NIAH, RULER, LongBench, MRCR
- Dialogue State Tracking
- Why Transformers: The Problems with RNNs
- Self-Attention from Scratch
- Multi-Head Attention
- Positional Encoding: Sinusoidal, RoPE, ALiBi
- The Full Transformer: Encoder + Decoder
- BERT — Masked Language Modeling
- GPT — Causal Language Modeling
- T5, BART — Encoder-Decoder Models
- Vision Transformers (ViT)
- Audio Transformers — Whisper Architecture
- Mixture of Experts (MoE)
- KV Cache, Flash Attention & Inference Optimization
- Scaling Laws
- Build a Transformer from Scratch
- Attention Variants — Sliding Window, Sparse, Differential
- Speculative Decoding — Draft, Verify, Repeat