
Run llama3.2, phi3, mistral locally with Ollama. Benchmark P99 latency, tokens/sec, and memory. FastAPI wrapper with JSON validation included.
I explain how software and AI systems really work, then build them: DevOps, SRE, ML engineering, MLOps and LLMOps. ThinkWithOps on YouTube.
Markdown supported · ⌘↵ to post
No responses yet.