---
title: "Discussion on \"Building an AI That Never Stops Learning on 8GB of VRAM\" | Hashnode"
description: "Discussion on \"Building an AI That Never Stops Learning on 8GB of VRAM\". How mini-AGI trains a byte-level, mixture-of-experts model from scratch on a single 8GB GPU, and solves catastrophic forgetting with one learning-rate tweak."
url: "https://hashnode.com/posts/building-an-ai-that-never-stops-learning-on-8gb-of-vram/6abc783398502f910ec6d19a"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Building an AI That Never Stops Learning on 8GB of VRAM](https://blog.zyvop.com/building-an-ai-that-never-stops-learning-on-8gb-of-vram?utm_source=hashnode&utm_medium=feed)

How mini-AGI trains a byte-level, mixture-of-experts model from scratch on a single 8GB GPU, and solves catastrophic forgetting with one learning-rate tweak.

Sep 30 · 8 min read · blog.zyvop.com  
Read the article

-   [#continuallearning](https://hashnode.com/tag/continuallearning)
-   [#localai](https://hashnode.com/tag/localai)
-   [#machinelearning](https://hashnode.com/tag/machinelearning)
-   [#mixtureofexperts](https://hashnode.com/tag/mixtureofexperts)
-   [#opensource](https://hashnode.com/tag/opensource)

## Written by

[ZyVOP](https://hashnode.com/@zyvop)

@zyvop · Delhi, India

  
[View profile](https://hashnode.com/@zyvop)

The Developer Publishing Hub

One platform to write, cross-post to Dev.to, Medium, WordPress, Hashnode, and Bluesky, and protect your SEO with canonical links. Zero paywalls and full content ownership.

## Responses

Markdown supported · ⌘↵ to post

No responses yet.