@alvinend

200台サーバーのローカルllmクラスターを構築

Jul 4, 2025 · 1 min read · Large-language-model（LLM）APIはとても強力ですが、コストが高いものです。一方、ローカルでLLMを推論させる場合、ハードウェアさえ手元にあればほぼ無料で動かせます。本記事では、オフィスで眠っていたApple Siliconラップトップを活用し、200台の推論クラスターを構築して本番トラフィックの25 %を処理するまでの道のりを紹介します。しかもデータセンター契約は一切ありません。ネタバレ: あるほこりだらけの会議室から始まり、最後は午前3時にオフィスのネットワークを総配...

1

Building a 200‑Server Local LLM Cluster

Jul 4, 2025 · 3 min read · Large‑language‑model (LLM) APIs are incredibly powerful—but they are also expensive. Local LLM inference, on the other hand, is almost free once the hardware is on your desk. This post walks through how we turned an ever‑growing pile of idle Apple si...

MMayank commented

Managing Snowflake's Procedure & UDF with Github

Jun 24, 2024 · 4 min read · Snowflake is an incredible data platform that my company and I have been leveraging for about a year and a half. It's robust, reliable, and feature-rich, with an intuitive UI that makes it easy to navigate. Notably, we haven't experienced any acciden...

Deploying Big Files with AWS Lambda and EFS Made Easy

Jun 11, 2024 · 8 min read · Background In some cases, we want to deploy our trained deep learning models or pre-trained models from platforms like Hugging Face to AWS Lambda for serverless inference. While the official service for such tasks is AWS Sagemaker, it can sometimes b...

Exploring AWS Aurora: MySQL vs PostgreSQL

Aug 7, 2023 · 4 min read · Last week, I created a blog post describing the differences between databases in AWS RDS. Now, it seems fitting to elucidate the differences between Aurora MySQL and PostgreSQL. Breakdown Security MySQLPostgreSQL Kerberos Auth✓✓ Aurora Postg...