Discussion on "Run AI Locally on Your Mac with Ollama: Chat, Tools, RAG and a Custom Model (Real Numbers from a Mac mini M4)"

RainyTech · 2026-10-05T06:38:03.350Z

I wanted to know how far a plain Mac mini M4 with 16 GB can go with local AI. No cloud, no API key, nothing leaving the machine. So I installed Ollama and built five small projects with it: a chat app

Validation loss bottoming at step 100 (1.382) and climbing to 1.811 by step 300 is the overfitting I keep forgetting to watch for. You kept the step-100 checkpoint; I once shipped a later one because training loss was still falling while the held-out set had already turned. Did you settle on 300 total steps by watching that curve, or was it a budget set up front?

Honestly, it was a budget I set up front. The command was --iters 300 with an eval and a checkpoint saved every 100 steps. I only picked the step-100 checkpoint afterwards, once I saw val loss go 1.382 → 1.665 → 1.811 while train loss kept dropping to 0.046.

Looking back, 300 was far too many for the data. It's 108 training examples at batch size 4, so about 27 steps per epoch. 300 steps is roughly 11 passes over the same data, and step 100 is already about 3.7. The validation set is also small (only 12 examples), and I only checked it every 100 steps. So 100 is just the best of the three points I measured, not the actual lowest point. The real minimum could be anywhere from about step 50 to 150.

Next time I'll run eval every 25 steps and stop when val loss stops improving. Thanks for raising it. It's an easy one to miss when the training curve looks so good.

The tool loop is the part I would adjust before adding the second function from your examples. The displayed snippet calls get_disk_free() for every tool call, regardless of the requested function name. That works for the single-tool demonstration, but a request for system memory would need a different dispatch path.

A small function-name-to-handler mapping, with arguments validated for the selected handler, would make that extension explicit. I would also append response.message once before iterating over its tool calls, then append each tool result separately. Otherwise a response containing multiple calls repeats the same assistant message in the conversation history.

The updated snippet now has the dispatch boundary the article needed. For missing required arguments, inspect.signature(handler).bind(**arguments) can check both missing and unexpected names before the handler is invoked; a TypeError becomes a result for that particular tool call.

That is still argument binding, not type validation, so it should not claim to validate annotated value types. A tiny test handler with one required argument would expose the current missing-field gap even though get_disk_free and get_memory accept none. The two-call test can then assert that only the valid handler ran.

Thanks, signature(handler).bind(**arguments) is a neat way to catch both missing and unexpected arguments in one step. And fair point that it's argument binding, not type validation. A test handler with one required argument would expose the gap nicely.

The required-argument fixture can stay independent of the actual disk and memory tools. Give the fake handler a call counter, submit one missing-argument call followed by a valid call, and assert that only the latter increments it.

That checks a useful recovery property too: rejecting one tool call should not prevent valid sibling calls from producing their own results. Keep the call identifiers in those results so the model can associate the validation error and successful output with the right requests.

Search Hashnode

Run AI Locally on Your Mac with Ollama: Chat, Tools, RAG and a Custom Model (Real Numbers from a Mac mini M4)

Responses9