---
title: "Discussion on \"One GPU, Two LLMs\" | Hashnode"
description: "Discussion on \"One GPU, Two LLMs\". Building a production-grade LLM inference platform on EKS with GPU time-slicing, KServe, Knative, and a custom gateway. Finally, debugging everything that broke."
url: "https://hashnode.com/posts/one-gpu-two-llms/6a5fb8e7acded6fb89b82065"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [One GPU, Two LLMs](https://orhunkupeli.hashnode.dev/one-gpu-two-llms?utm_source=hashnode&utm_medium=feed)

Building a production-grade LLM inference platform on EKS with GPU time-slicing, KServe, Knative, and a custom gateway. Finally, debugging everything that broke.

Jul 21 · 6 min read · orhunkupeli.hashnode.dev  
Read the article

-   [#kubernetes](https://hashnode.com/tag/kubernetes)
-   [#serverless](https://hashnode.com/tag/serverless)
-   [#knative](https://hashnode.com/tag/knative)
-   [#eks](https://hashnode.com/tag/eks)
-   [#mlops](https://hashnode.com/tag/mlops)
-   [#awq-quantization](https://hashnode.com/tag/awq-quantization)
-   [#terraform](https://hashnode.com/tag/terraform)
-   [#karpenter](https://hashnode.com/tag/karpenter)
-   [#vllm](https://hashnode.com/tag/vllm)
-   [#qwen](https://hashnode.com/tag/qwen)

## Written by

[Orhun Küpeli](https://hashnode.com/@orhunkupeli)

@orhunkupeli · Munich, Germany

  
[View profile](https://hashnode.com/@orhunkupeli)

## Responses

Markdown supported · ⌘↵ to post

No responses yet.