Llama2 From Scratch with Pytorch LightningDive into the implementation of Llama2 using PyTorch Lightning. Extending my Llama implementation to include grouped-query attention.7 min read·afterhoursresearch.comNov 1, 2023
Llama From Scratch with Pytorch LightningExplore building Llama from scratch with PyTorch Lightning, Hydra and Wandb. Learn BPE tokenizer, RMSNorm, RoPE and SwiGLU. Includes detailed attention mech21 min read·afterhoursresearch.comNov 1, 2023
Batch Normalization, Layer Normalization and Root Mean Square Layer Normalization: A Comprehensive Guide with Python ImplementationsUnderstanding BatchNorm, LayerNorm, and RMSNorm: A Deep Dive into Neural Network Normalization Techniques with Python Implementations6 min read·afterhoursresearch.comOct 10, 2023
RoPE - Rotary Positional EmbeddingUnlock the power of Rotary Positional Embeddings in your NLP models with our comprehensive guide and PyTorch code examples. From theory to implementation, l5 min read·afterhoursresearch.comOct 4, 2023