@AfterHoursResearch
Machine Learning Research - Focusing on LLMs
Nothing here yet.
Nov 1, 2023 · 7 min read · In our previous blog post, we built the Llama LLM with PyTorch Lightning, with Weights & Biases for experiment tracking and Hydra for configuration management. Now, we turn our attention to Llama 2, the successor to Llama. Let's look at the differenc...
Join discussion
Nov 1, 2023 · 21 min read · Welcome to this deep dive into building Llama from scratch. This project is inspired by Llama from scratch, but it diverges in several ways. For instance, we make various architectural adjustments, such as modifications to the placement of residuals ...
Join discussion
Oct 10, 2023 · 6 min read · Introduction Stabilizing and accelerating the training of neural networks often hinge on the normalization techniques employed. While the theory behind normalization appears straightforward, its practical applications come in various flavours, each w...
Join discussionOct 4, 2023 · 5 min read · Introduction The transformer architecture has seen a meteoric rise in its applications across various domains of machine learning. However, the architecture lacks an inherent understanding of the order or sequence of tokens. This necessitates some fo...
Join discussion