
A deployment walkthrough covering PLE loading, FlashInfer B12x route fixes, shared workspaces, and concurrent inference memory tuning.
Lead developer at Neuroprem | LLM systems & inference
Software engineer and lead developer at Neuroprem, working on AI systems and LLM infrastructure. I write about quantization, distillation, efficient inference, retrieval and memory for agentic systems: technical notes, benchmarks, architecture decisions and lessons from failed experiments. Currently preparing a custom quantized GLiNER model for release and experimenting with Qwen inference and distillation. Less hype. More engineering. Engineering log: https://t.me/badbat4560 Models: https://huggingface.co/badbat4560 Code: https://github.com/badbat4560 DEV: https://dev.to/badbat4560 Neuroprem: https://neuroprem.com Instagram: https://www.instagram.com/badbat4560/
Markdown supported · ⌘↵ to post
No responses yet.