---
title: "Deepanshi (@dpanshi) | Hashnode"
description: "Deepanshi (@dpanshi) writes on Hashnode. Read their latest articles and follow their work."
url: "https://hashnode.com/@dpanshi"
---

@dpanshi

# Deepanshi

@dpanshi · San F · Joined August 2026

### [Does a 1B Model Fit on 8×A100-40GB? Memory Arithmetic for DDP and ZeRO 1/2/3](https://dpanshi.hashnode.dev/does-a-1b-model-fit-on-8-a100-40gb-memory-arithmetic-for-ddp-and-zero-1-2-3?utm_source=hashnode&utm_medium=feed)

The question: a 1B-parameter model, trained with AdamW in bf16 mixed precision on 8×A100-40GB. Does it fit under plain DDP? ZeRO-1? ZeRO-2? ZeRO-3? And what micro-batch size can each one handle? Assum

5 min read · [dpanshi.hashnode.dev](https://dpanshi.hashnode.dev/does-a-1b-model-fit-on-8-a100-40gb-memory-arithmetic-for-ddp-and-zero-1-2-3?utm_source=hashnode&utm_medium=feed)

[1d ago](https://hashnode.com/posts/does-a-1b-model-fit-on-8-a100-40gb-memory-arithmetic-for-ddp-and-zero-1-2-3/6afd1027818f8fb17f206ada)

### [Let's Talk SSL](https://dpanshi.hashnode.dev/let-s-talk-ssl?utm_source=hashnode&utm_medium=feed)

My rambling on state of art model comparison in self supervised learning and everything in between: No math just thoughts, opinions and vibes

6 min read · [dpanshi.hashnode.dev](https://dpanshi.hashnode.dev/let-s-talk-ssl?utm_source=hashnode&utm_medium=feed)

[Aug 4](https://hashnode.com/posts/let-s-talk-ssl/6a71779a6e4ea5fb89702db4)

### [Transformers from scratch: things that silently broke](https://dpanshi.hashnode.dev/transformers-from-scratch-things-that-silently-broke?utm_source=hashnode&utm_medium=feed)

Over the last three weeks I built a decoder-only GPT, a Vision Transformer, RoPE, and GQA, all from raw PyTorch, no copy-pasting a reference implementation. The models train and the loss curves look r

5 min read · [dpanshi.hashnode.dev](https://dpanshi.hashnode.dev/transformers-from-scratch-things-that-silently-broke?utm_source=hashnode&utm_medium=feed)

[Aug 1](https://hashnode.com/posts/transformers-from-scratch-things-that-silently-broke/6a6e7297d785bec6d04c20ed)
