---
title: "#pre-training articles | Hashnode"
description: "Explore 6 articles tagged #pre-training on Hashnode. Tutorials, guides, and how-tos from the community."
url: "https://hashnode.com/tag/pre-training"
---

Tag

# #pre-training

6 posts · 0 followers

[Abstract Algorithms](https://hashnode.com/@abstractalgorithms) · [Mar 9](https://hashnode.com/posts/pre-training-llms-guide/69ae35fb0fe4dd6eff460424)

### [A Guide to Pre-training Large Language Models](https://abstractalgorithms.hashnode.dev/pre-training-llms-guide?utm_source=hashnode&utm_medium=feed)

Pre-training is the most expensive part of building an LLM. We explain the data pipeline, the 'Ne...

15 min read · [abstractalgorithms.hashnode.dev](https://abstractalgorithms.hashnode.dev/pre-training-llms-guide?utm_source=hashnode&utm_medium=feed)

[Retzam Tarle](https://hashnode.com/@retzam) · [Feb 9](https://hashnode.com/posts/bert-and-glp/698999dc265d96fdd47cb1e8)

### [BERT and GLP](https://retzam.hashnode.dev/bert-and-glp?utm_source=hashnode&utm_medium=feed)

AI Series - Chapter 33

6 min read · [retzam.hashnode.dev](https://retzam.hashnode.dev/bert-and-glp?utm_source=hashnode&utm_medium=feed)

[Kashif](https://hashnode.com/@ifkash) · [Feb 8](https://hashnode.com/posts/smol-llama-pretraining/698898652fb502a9301f0a63)

### [I Pretrained a 360M LLaMA-Style Language Model from Scratch on 6B FineWeb Tokens (Single H100)](https://blog.ifkash.dev/smol-llama-pretraining?utm_source=hashnode&utm_medium=feed)

Explore pretraining a 360M LLaMA-style language model on 6B tokens using a single H100 GPU, with reproducible results

5 min read · [blog.ifkash.dev](https://blog.ifkash.dev/smol-llama-pretraining?utm_source=hashnode&utm_medium=feed)

[Anni Huang](https://hashnode.com/@HuangAnni) · [Aug 13, 2025](https://hashnode.com/posts/beyond-pre-training-the-power-of-rlhf-in-llm-alignment/689cc42ea3cc115547a50c37)

### [Beyond Pre-training: The Power of RLHF in LLM Alignment](https://huanganni.hashnode.dev/beyond-pre-training-the-power-of-rlhf-in-llm-alignment?utm_source=hashnode&utm_medium=feed)

If LLM is compared to a person, then the early training is like general education, and the later training is like vocational training.

2 min read · [huanganni.hashnode.dev](https://huanganni.hashnode.dev/beyond-pre-training-the-power-of-rlhf-in-llm-alignment?utm_source=hashnode&utm_medium=feed)

[Jayasri](https://hashnode.com/@JayasriJ) · [Apr 16, 2025](https://hashnode.com/posts/llm-pre-training-in-genai/67ffec0b987372437e452ce1)

### [LLM Pre-Training in GenAI](https://j-jayasri.hashnode.dev/llm-pre-training-in-genai?utm_source=hashnode&utm_medium=feed)

pretraining llm

6 min read · [j-jayasri.hashnode.dev](https://j-jayasri.hashnode.dev/llm-pre-training-in-genai?utm_source=hashnode&utm_medium=feed)

[Rafal Jackiewicz](https://hashnode.com/@cocofrut) · [Oct 18, 2024](https://hashnode.com/posts/training-gpt-2-from-scratch-a-beginner-friendly-step-by-step-guide/6712dd638064c803bff9c5bf)

### [Training GPT-2 From Scratch: A Beginner-Friendly Step-by-Step Guide](https://jackiewicz.hashnode.dev/training-gpt-2-from-scratch-a-beginner-friendly-step-by-step-guide?utm_source=hashnode&utm_medium=feed)

Training a GPT-2 model from scratch is a rewarding experience, especially if you want to learn about natural language processing and get hands-on with machine learning models. This guide will walk you through the process step-by-step, with simplified...

10 min read · [jackiewicz.hashnode.dev](https://jackiewicz.hashnode.dev/training-gpt-2-from-scratch-a-beginner-friendly-step-by-step-guide?utm_source=hashnode&utm_medium=feed)
