---
title: "#llm-engineering-ai-infrastructure-python-gpu-optimization-performance-engineering-machine-learning-systems-devops-artificial-intelligence-software-engineering-programming-cloud-computing-deep-learning articles | Hashnode"
description: "Explore 3 articles tagged #llm-engineering-ai-infrastructure-python-gpu-optimization-performance-engineering-machine-learning-systems-devops-artificial-intelligence-software-engineering-programming-cloud-computing-deep-learning on Hashnode. Tutorials, guides, and how-tos from the community."
url: "https://hashnode.com/tag/llm-engineering-ai-infrastructure-python-gpu-optimization-performance-engineering-machine-learning-systems-devops-artificial-intelligence-software-engineering-programming-cloud-computing-deep-learning"
---

Tag

# #llm-engineering-ai-infrastructure-python-gpu-optimization-performance-engineering-machine-learning-systems-devops-artificial-intelligence-software-engineering-programming-cloud-computing-deep-learning

3 posts · 0 followers

[Theaicommit](https://hashnode.com/@theaicommit) · [Oct 22](https://hashnode.com/posts/why-speculative-decoding-slows-hybrid-mamba-models-at-large-batch-sizes/6adaa1e85e0fca8687b12dff)

### [Why Speculative Decoding Slows Hybrid Mamba Models at Large Batch Sizes](https://theaicommit.hashnode.dev/why-speculative-decoding-slows-hybrid-mamba-models-at-large-batch-sizes?utm_source=hashnode&utm_medium=feed)

On hybrid Mamba models, speculative decoding saves a full state copy per draft token. At large batch sizes that erases the speed-up. Here's the fix in vLLM

9 min read · [theaicommit.hashnode.dev](https://theaicommit.hashnode.dev/why-speculative-decoding-slows-hybrid-mamba-models-at-large-batch-sizes?utm_source=hashnode&utm_medium=feed)

[Theaicommit](https://hashnode.com/@theaicommit) · [Oct 22](https://hashnode.com/posts/how-sorting-your-prompts-speeds-up-a-mixture-of-experts-batch-job/6ad967b016d0cad75ff96324)

### [How Sorting Your Prompts Speeds Up a Mixture-of-Experts Batch Job](https://theaicommit.hashnode.dev/how-sorting-your-prompts-speeds-up-a-mixture-of-experts-batch-job?utm_source=hashnode&utm_medium=feed)

Similar prompts pick the same experts, so a sorted batch reads fewer weights per step. Hugging Face measured 5–17% more throughput on unchanged vLLM.

9 min read · [theaicommit.hashnode.dev](https://theaicommit.hashnode.dev/how-sorting-your-prompts-speeds-up-a-mixture-of-experts-batch-job?utm_source=hashnode&utm_medium=feed)

[Ahmed Adawy](https://hashnode.com/@ai-architecture) · [Oct 2](https://hashnode.com/posts/llm-inference-engineering-overcoming-the-kv-cache-bottleneck-and-maximizing-production-throughput/6abfff77746d63336e2beba1)

### [LLM Inference Engineering: Overcoming the KV-Cache Bottleneck and Maximizing Production Throughput](https://adawy.hashnode.dev/llm-inference-engineering-overcoming-the-kv-cache-bottleneck-and-maximizing-production-throughput?utm_source=hashnode&utm_medium=feed)

LLM Engineering AI Infrastructure Python GPU Optimization Performance Engineering Machine https://leanpub.com/b/thegenerativeaillmengineeringbundle

3 min read · [adawy.hashnode.dev](https://adawy.hashnode.dev/llm-inference-engineering-overcoming-the-kv-cache-bottleneck-and-maximizing-production-throughput?utm_source=hashnode&utm_medium=feed)
