---
title: "Discussion on \"DeepSeek R1: Efficient Reinforcement Learning with GRPO\" | Hashnode"
description: "Discussion on \"DeepSeek R1: Efficient Reinforcement Learning with GRPO\". DeepSeek R1 uses GRPO for cost-efficient AI training, boosting reasoning capabilities and reducing hardware expenses across diverse tasks"
url: "https://hashnode.com/posts/deepseek-r1-efficient-reinforcement-learning-with-grpo/6798621c024aa8f31b407afb"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [DeepSeek R1: Efficient Reinforcement Learning with GRPO](https://blog.dataopslabs.com/deepseek-r1-efficient-reinforcement-learning-with-grpo?utm_source=hashnode&utm_medium=feed)

DeepSeek R1 uses GRPO for cost-efficient AI training, boosting reasoning capabilities and reducing hardware expenses across diverse tasks

Jan 28, 2025 · 4 min read · blog.dataopslabs.com  
Read the article

-   [#deepseekr1](https://hashnode.com/tag/deepseekr1)

## Written by

[DataOps Labs](https://hashnode.com/@ayyanar)

@ayyanar · India

  
[View profile](https://hashnode.com/@ayyanar)

Stay updated on the latest in AI/ML, Cloud, DevOps, MLOps, Generative AI and cutting-edge techniques with this continuous learning

I'm Ayyanar Jeyakrishnan ; aka AJ. With over 21 years in IT, I'm a passionate Multi-Cloud Architect specialising in crafting scalable and efficient cloud solutions. I've successfully designed and implemented multi-cloud architectures for diverse organisations, harnessing AWS, Azure, and GCP. My track record includes delivering Machine Learning and Data Platform projects with a focus on high availability, security, and scalability. I'm a proponent of DevOps and MLOps methodologies, accelerating development and deployment. I actively engage with the tech community, sharing knowledge in sessions, conferences, and mentoring programs. Constantly learning and pursuing certifications, I provide cutting-edge solutions to drive success in the evolving cloud and AI/ML landscape.

## Responses 1

Markdown supported · ⌘↵ to post

[Avinash Dalvi](https://hashnode.com/@AvinashDalvi)

[Jan 28, 2025](https://hashnode.com/posts/deepseek-r1-efficient-reinforcement-learning-with-grpo/6798621c024aa8f31b407afb/comment/6798670a5e344d91ba5031db)

Thanks for insightful blog. I am curious to know how come they give lower price compare to others model ?

[DataOps Labs](https://hashnode.com/@ayyanar)

[Jan 28, 2025](https://hashnode.com/posts/deepseek-r1-efficient-reinforcement-learning-with-grpo/6798621c024aa8f31b407afb/comment/6798670a5e344d91ba5031db#reply-679877cfa6400b1f7f36060f)

Efficient RL Framework: GRPO eliminates costly critic models in RL.

(A critic model in reinforcement learning is responsible for evaluating the actions taken by an agent (actor model) by estimating the expected rewards, also known as the value function. It helps guide the agent by providing feedback on how good a particular action or decision is. While effective, critic models are expensive because they often need to be as large and complex as the actor model, doubling the computational cost. Additionally, they require constant updates to align with the actor's learning, adding to memory and GPU/CPU usage. This makes training reinforcement learning systems with critic models resource-intensive, especially for large-scale models like language models.)

Floating Point 8 Precision: Reduces memory and compute needs during training and inference.

Mixture-of-Experts Design: Activates only a subset of parameters per query, optimizing performance-to-cost. (This is technique Pionered by Mistral)

DeepSeek models are trained on custom kernels for efficient GPU-to-GPU communication using NVLink and InfiniBand. Liger-Kernel also did somwhat similar Ref :[linkedin.com/blog/engineering/open-source/liger-k…](https://www.linkedin.com/blog/engineering/open-source/liger-kernel-open-source-ecosystem-for-efficient-llm-training?lipi=urn%3Ali%3Apage%3Ad_flagship3_detail_base%3B0zH7nA6VRl20GM0qdPMm3A%3D%3D)

[Avinash Dalvi](https://hashnode.com/@AvinashDalvi)

[Jan 28, 2025](https://hashnode.com/posts/deepseek-r1-efficient-reinforcement-learning-with-grpo/6798621c024aa8f31b407afb/comment/6798670a5e344d91ba5031db#reply-6798d55a10330a9b529e590b)

[DataOps Labs](https://hashnode.com/@ayyanar) Interesting read