Mar 6 · 5 min read · Over the past two years, more and more teams have integrated AI into their daily workflows. But soon, a practical issue emerged: The more the model is used, the faster Tokens are consumed, and both co
Join discussion
Mar 5 · 3 min read · After finishing my PFE (Final Year Project), I became obsessed with production environments. There is a massive, often painful difference between a model that runs in a Jupyter Notebook and one that a
Join discussion
Mar 4 · 3 min read · If you've been trying to run standard vLLM Docker images on the new NVIDIA DGX Spark, you’ve probably hit a wall. Between the ARM64 (Grace) architecture, the Blackwell (GB10) GPU, and the requirements
Join discussion
Feb 14 · 4 min read · A few months ago I started using my DGX Spark, and coming in with minimal experience of running local models, I wanted a simple and easy to understand inference engine (as I wrote in this article). Plus I could see the experts on the DGX Spark user g...
Join discussion
Feb 3 · 10 min read · Your GPU is sitting idle while your LLM inference requests queue up, one by one, painfully slow. You know there's a better way. You've heard about continuous batching, PagedAttention, and throughput numbers that seem too good to be true. Welcome to v...
Join discussion
Jan 26 · 7 min read · A comprehensive guide to running Large Language Models (LLMs) locally on your machine using various tools and platforms. 🎬 Video Demonstration 1. 🦙 Ollama - The Dominant Local LLM Ecosystem Ollama is the dominant ecosystem for running LLMs such a...
Join discussion
Jan 1 · 4 min read · When the MI300X was launched in December 2023, there was a lot of optimism. We were desperate for an alternative to nVidia’s expensive and scarce H100s, especially one that provided 192gb of VRAM per GPU compared to the H100’s 80gb. And the MI300X wa...
Join discussion
Dec 7, 2025 · 4 min read · Running vision-language models like Qwen3-VL with vLLM on high-end GPUs should be straightforward. Except when it's not. The Problem I was setting up Qwen3-VL-8B-Instruct on our H200 cluster (8x H200, 143GB VRAM each) when I hit this error: vllm ser...
Join discussionDec 4, 2025 · 11 min read · The Challenge: Trapped Between Cloud Power and Cloud Lock-In Organizations face an impossible choice. Hyperscaler clouds offer powerful data services, advanced catalogs, vector search, and high-performance analytics, but only within their walled gard...
Join discussionAug 31, 2025 · 4 min read · This document outlines the complete, step-by-step process for deploying the Qwen/Qwen3-Coder-30B-A3B-Instruct model on a DigitalOcean H100 droplet. The final setup uses vLLM for high-performance inference, Nginx as a secure reverse proxy for API key ...
Join discussion