---
title: "Discussion on \"LoRA & DoRA: The Math, Memory, and Trade-offs\" | Hashnode"
description: "Discussion on \"LoRA & DoRA: The Math, Memory, and Trade-offs\". LoRA and DoRA explained: matrix shapes, the geometry of weight decomposition, exact memory math for a 27B model, and what the DoRA benchmarks support."
url: "https://hashnode.com/posts/lora-dora-the-math-memory-and-trade-offs/6ab7ee6c0c027d01840127f8"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [LoRA & DoRA: The Math, Memory, and Trade-offs](https://gfactor.hashnode.dev/lora-dora-the-math-memory-and-trade-offs?utm_source=hashnode&utm_medium=feed)

LoRA and DoRA explained: matrix shapes, the geometry of weight decomposition, exact memory math for a 27B model, and what the DoRA benchmarks support.

Sep 26 · 10 min read · gfactor.hashnode.dev  
Updated Sep 28  
Read the article

-   [#machine-learning](https://hashnode.com/tag/machine-learning)
-   [#artificial-intelligence](https://hashnode.com/tag/artificial-intelligence)
-   [#llm](https://hashnode.com/tag/llm)
-   [#deep-learning](https://hashnode.com/tag/deep-learning)
-   [#fine-tuning](https://hashnode.com/tag/fine-tuning)

## Written by

[Aleksei Romanov](https://hashnode.com/@gfactor)

@gfactor · United States

  
[View profile](https://hashnode.com/@gfactor)

Ex-Apple GenAI engineer, building an LLM post-training platform at g factor technologies

Ex-Apple GenAI engineer, building an LLM post-training platform at g factor technologies

## Responses

Markdown supported · ⌘↵ to post

No responses yet.
