RSRahul Sai Indeevar Vinrahul-ai.hashnode.dev·6h ago · 10 min readParameter-Efficient Fine-Tuning: From LoRA to QLoRAFine-tuning Large Language Models (LLMs) used to be an option only for organizations with massive compute budgets. Full fine-tuning requires updating and storing every parameter, gradient, and optimiz20
RSRahul Sai Indeevar Vinrahul-ai.hashnode.dev·Sep 8 · 14 min readLLM Post-Training: From Full Fine-Tuning to PEFT, Adapters, and Soft PromptsLarge Language Models (LLMs) arrive with immense foundational knowledge acquired during pre-training. However, deploying them to specialized real-world domains—such as legal document parsing, medical 10
RSRahul Sai Indeevar Vinrahul-ai.hashnode.dev·Sep 8 · 12 min readDeep Dive into LLM Pre-Training: Paradigms, Mathematical Formulations, and Training ObjectivesBuilding effective Transformer-based Pretrained Language Models (T-PTLMs) requires understanding how models acquire general linguistic representations and adapt to specialized domains. Pre-training si10
RSRahul Sai Indeevar Vinrahul-ai.hashnode.dev·Sep 7 · 15 min readAdvanced Transformers: Positional Encodings (RoPE) & Efficient Attention Mechanisms (KV Caching, MQA, GQA, MLA)In modern Large Language Model (LLM) architectures, two foundational engineering bottlenecks dictate model scalability and inference latency: Representing Order: How self-attention mechanisms—which a13A
RSRahul Sai Indeevar Vinrahul-ai.hashnode.dev·Sep 5 · 9 min readLLM Architectures: From BERT and GPT-2 to XLNet and T5Understanding the internal mechanics of Large Language Model (LLM) architectures is crucial for building, fine-tuning, or deploying modern AI models. Why does GPT excel at text generation, while BERT 10
