Feb 9 · 6 min read · Hello 😊, We learned the core philosophy behind BERT in the introductory chapter of our previous series chapter. In this chapter, we’ll learn more about BERT and another pivotal process in Machine Learning known as General Language Pre-training, GLP....
Join discussion
Feb 8 · 5 min read · Pretraining an LLM from scratch usually sounds like “big-lab-only” territory. I wanted to test how far a smaller, practical setup can go while keeping the process transparent and reproducible. This post documents an end-to-end run of training a ~360M...
Join discussionAug 13, 2025 · 2 min read · Pre-training uses massive datasets and computational resources—often thousands of GPUs running for weeks or months—making it a domain dominated by top AI companies. Post-training is much lighter in cost and time (often days instead of months) and foc...
Join discussion
Apr 16, 2025 · 6 min read · In this series, we’ll explore everything about pre-training in the Generative AI (GenAI) pipeline — including how models are trained based on specific objectives or goals, and how to design an effective pre-training pipeline. But before diving into ...
Join discussion
Oct 18, 2024 · 10 min read · Training a GPT-2 model from scratch is a rewarding experience, especially if you want to learn about natural language processing and get hands-on with machine learning models. This guide will walk you through the process step-by-step, with simplified...
Join discussion
