---
title: "Discussion on \"Local LLM's with .Net\" | Hashnode"
description: "Discussion on \"Local LLM's with .Net\". Phi3/Mistral/TinyLamma CPU inference with Llamasharp"
url: "https://hashnode.com/posts/local-llms-with-net/668cfc240ce234418bc89246"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Local LLM's with .Net](https://thecodesmith.hashnode.dev/local-llms-with-net?utm_source=hashnode&utm_medium=feed)

Phi3/Mistral/TinyLamma CPU inference with Llamasharp

Jul 9, 2024 · 3 min read · thecodesmith.hashnode.dev  
Read the article

-   [#llamasharp](https://hashnode.com/tag/llamasharp)
-   [#cpuinference](https://hashnode.com/tag/cpuinference)
-   [#thebloke](https://hashnode.com/tag/thebloke)
-   [#scisharp](https://hashnode.com/tag/scisharp)
-   [#ai](https://hashnode.com/tag/ai)
-   [#csharp](https://hashnode.com/tag/csharp)
-   [#phi-3](https://hashnode.com/tag/phi-3)
-   [#mistralai](https://hashnode.com/tag/mistralai)
-   [#inference](https://hashnode.com/tag/inference)
-   [#gguf](https://hashnode.com/tag/gguf)
-   [#huggingface](https://hashnode.com/tag/huggingface)
-   [#microsoft](https://hashnode.com/tag/microsoft)

## Written by

[Kevin Loggenberg](https://hashnode.com/@TheCodesmith)

@TheCodesmith · Johannesburg, ZA

  
[View profile](https://hashnode.com/@TheCodesmith)

Leading innovative software development to drive business intelligence and transformation

With an unwavering passion for metrics and an innate talent for problem-solving, I possess a wealth of experience as a seasoned full-stack developer. Over the course of my extensive tenure in software development, I have honed my skills and cultivated a deep understanding of the intricacies involved in solving complex challenges. Also, I like AI.

## Responses 1

Markdown supported · ⌘↵ to post

[Skyblade](https://hashnode.com/@skyblade)

[Nov 4, 2024](https://hashnode.com/posts/local-llms-with-net/668cfc240ce234418bc89246/comment/67291da24bb0e2b79c1f20d8)

How do I know the value for GpuLayerCount for a particular model? Are there any formulas or guidelines?

[Kevin Loggenberg](https://hashnode.com/@TheCodesmith)

[Jan 12, 2025](https://hashnode.com/posts/local-llms-with-net/668cfc240ce234418bc89246/comment/67291da24bb0e2b79c1f20d8#reply-6783fcebee32ab5b8a283be3)

Hi there.. Sorry for the late, somehow didn't get any notifications for this... this will be dependent on the graphics card you have.... the more vram and cpu power it has, will allow more GPU offloading from my understanding.... it probably could be an exact science if you take everything into consideration, but quickest for me is a bit of experimentation... start low and increase...