---
title: "Discussion on \"Llama From Scratch with Pytorch Lightning\" | Hashnode"
description: "Discussion on \"Llama From Scratch with Pytorch Lightning\". Explore building Llama from scratch with PyTorch Lightning, Hydra and Wandb. Learn BPE tokenizer, RMSNorm, RoPE and SwiGLU. Includes detailed attention mech"
url: "https://hashnode.com/posts/llama-from-scratch-with-pytorch-lightning/65425ac9379eb266bfbbb6ce"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Llama From Scratch with Pytorch Lightning](https://afterhoursresearch.com/llama-from-scratch-with-pytorch-lightning?utm_source=hashnode&utm_medium=feed)

Explore building Llama from scratch with PyTorch Lightning, Hydra and Wandb. Learn BPE tokenizer, RMSNorm, RoPE and SwiGLU. Includes detailed attention mech

Nov 1, 2023 · 21 min read · afterhoursresearch.com  
Read the article

-   [#pytorch](https://hashnode.com/tag/pytorch)
-   [#pytorch-lightning](https://hashnode.com/tag/pytorch-lightning)
-   [#llm](https://hashnode.com/tag/llm)
-   [#llama](https://hashnode.com/tag/llama)

## Written by

[After Hours Research](https://hashnode.com/@AfterHoursResearch)

@AfterHoursResearch

  
[View profile](https://hashnode.com/@AfterHoursResearch)

Machine Learning Research - Focusing on LLMs

After work-hours ML Research lab, composed of: Brad &amp; Sara

## Responses

Comments are turned off for this post.
