---
title: "Discussion on \"4-bit NF4 quantization for llama guard 3\" | Hashnode"
description: "Discussion on \"4-bit NF4 quantization for llama guard 3\". this quantization will help for users with low vram and get desired result but note using 4 bit quantization has its drawback.\nimport torch\nfrom transformers import AutoTokenizer, AutoModelForCausalLM"
url: "https://hashnode.com/posts/4-bit-nf4-quantization-for-llama-guard-3/6af3153ea84302f8dc4ddd89"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [4-bit NF4 quantization for llama guard 3](https://chris001.hashnode.dev/4-bit-nf4-quantization-for-llama-guard-3?utm_source=hashnode&utm_medium=feed)

this quantization will help for users with low vram and get desired result but note using 4 bit quantization has its drawback. import torch from transformers import AutoTokenizer, AutoModelForCausalLM

1d ago · 1 min read · chris001.hashnode.dev  
Read the article

## Written by

[obliterate.exe](https://hashnode.com/@chrisoo1)

@chrisoo1 · Paris

  
[View profile](https://hashnode.com/@chrisoo1)

hope!!!!!!

just a matter of time

## Responses

Markdown supported · ⌘↵ to post

No responses yet.