---
title: "#gqa articles | Hashnode"
description: "Explore 6 articles tagged #gqa on Hashnode. Tutorials, guides, and how-tos from the community."
url: "https://hashnode.com/tag/gqa"
---

Tag

# #gqa

6 posts · 0 followers

[Rishi](https://hashnode.com/@rishi2220) · [Jul 22](https://hashnode.com/posts/towards-the-modern-transformer-architecture/6a6090024093ae09cf4fb380)

### [Towards the Modern Transformer Architecture](https://rishi2220.hashnode.dev/towards-the-modern-transformer-architecture?utm_source=hashnode&utm_medium=feed)

A deep dive into how GPT-1's architecture evolved: pre-norm, RMSNorm, SwiGLU, RoPE, and GQA — with the math behind every change.

36 min read · [rishi2220.hashnode.dev](https://rishi2220.hashnode.dev/towards-the-modern-transformer-architecture?utm_source=hashnode&utm_medium=feed)

[1](https://hashnode.com/posts/towards-the-modern-transformer-architecture/6a6090024093ae09cf4fb380)

[Sisir Dhakal](https://hashnode.com/@sisirdhakal) · [Feb 9, 2025](https://hashnode.com/posts/multi-head-multi-query-and-grouped-query-attention-which-one-should-you-use/67a8a1cd90b0d9a7ab753aba)

### [Multi-Head, Multi-Query, and Grouped-Query Attention: Which One Should You Use?](https://sisirdhakal.hashnode.dev/multi-head-multi-query-and-grouped-query-attention-which-one-should-you-use?utm_source=hashnode&utm_medium=feed)

Multi-Head, Multi-Query, and Grouped-Query Attention: Which One Should You Use?

6 min read · [sisirdhakal.hashnode.dev](https://sisirdhakal.hashnode.dev/multi-head-multi-query-and-grouped-query-attention-which-one-should-you-use?utm_source=hashnode&utm_medium=feed)

[Jonas Kim](https://hashnode.com/@bits-bytes-nn) · [Jan 4, 2024](https://hashnode.com/posts/deepseek-llm-scaling-open-source-language-models-with-longtermism/67e2c222dc0e88ba125abf9f)

### [DeepSeek LLM: Scaling Open-Source Language Models with Longtermism 논문 리뷰](https://bits-bytes-nn.hashnode.dev/deepseek-llm-scaling-open-source-language-models-with-longtermism?utm_source=hashnode&utm_medium=feed)

DeepSeek AI는 영어와 중국어 분야에서 우수한 성능을 보이는 모델을 선보이는 동시에, 하이퍼파라미터와 모델·데이터 간 연산량 할당에 관한 정교한 스케일링 법칙도 제시했습니다.

21 min read · [bits-bytes-nn.hashnode.dev](https://bits-bytes-nn.hashnode.dev/deepseek-llm-scaling-open-source-language-models-with-longtermism?utm_source=hashnode&utm_medium=feed)

[Jonas Kim](https://hashnode.com/@bits-bytes-nn) · [Sep 9, 2023](https://hashnode.com/posts/mistral-7b/67dd606518faecacf3bee98a)

### [Mistral 7B 논문 리뷰](https://bits-bytes-nn.hashnode.dev/mistral-7b?utm_source=hashnode&utm_medium=feed)

Mistral AI가 2023년 10월에 처음 선보인 Mistral 7B는 효율적인 아키텍처와 뛰어난 성능으로, 소형 오픈소스 언어 모델 분야에 새로운 변화를 가져왔습니다.

20 min read · [bits-bytes-nn.hashnode.dev](https://bits-bytes-nn.hashnode.dev/mistral-7b?utm_source=hashnode&utm_medium=feed)

[Jonas Kim](https://hashnode.com/@bits-bytes-nn) · [Jul 17, 2023](https://hashnode.com/posts/llama-2-open-foundation-and-fine-tuned-chat-models/67da243517f9d13e7168c4a2)

### [Llama 2: Open Foundation and Fine-Tuned Chat Models 논문 리뷰](https://bits-bytes-nn.hashnode.dev/llama-2-open-foundation-and-fine-tuned-chat-models?utm_source=hashnode&utm_medium=feed)

2023년 7월 Meta AI가 출시한 Llama 2는 거부 샘플링, RLHF, 레드팀 테스팅 등의 기법을 통해 대화 안전성 향상에 중점을 두었습니다. 이 모델은 유해 콘텐츠 제한과 유용성 사이의 균형을 찾는 새로운 기준을 제시했습니다.

34 min read · [bits-bytes-nn.hashnode.dev](https://bits-bytes-nn.hashnode.dev/llama-2-open-foundation-and-fine-tuned-chat-models?utm_source=hashnode&utm_medium=feed)
