--- title: "Discussion on \"I built desktop control so an agent could test my Chrome extension. Then I benchmarked it against four free tools.\" | Hashnode" description: "Discussion on \"I built desktop control so an agent could test my Chrome extension. Then I benchmarked it against four free tools.\". Same model, same Mac, 153 runs. One tool passed one more run than mine. Mine took half the time." url: "https://hashnode.com/posts/desktop-control-benchmark/6ad1401cc95ead9bd1583d6e" --- [Feed](https://hashnode.com/feed) / Discussion # [I built desktop control so an agent could test my Chrome extension. Then I benchmarked it against four free tools.](https://ar5en1c.hashnode.dev/desktop-control-benchmark?utm_source=hashnode&utm_medium=feed) Same model, same Mac, 153 runs. One tool passed one more run than mine. Mine took half the time. Oct 15 · 6 min read · ar5en1c.hashnode.dev Read the article - [#macos](https://hashnode.com/tag/macos) - [#ai-agents](https://hashnode.com/tag/ai-agents) - [#mcp](https://hashnode.com/tag/mcp) - [#benchmarking](https://hashnode.com/tag/benchmarking) - [#opensource](https://hashnode.com/tag/opensource) ## Written by [Kuldeep Singh](https://hashnode.com/@ar5en1c) @ar5en1c · Sunnyvale, CA [View profile](https://hashnode.com/@ar5en1c) Senior Software & AI Architect | iOS, WebGPU, Local-LLM | Ex-Walmart I engineer secure, massive-scale mobile and frontend infrastructure. My technical focus lies at the intersection of enterprise iOS architecture and bleeding-edge local AI inference (WebGPU/WASM). At Walmart, I engineered robust iOS navigation architectures (Coordinator pattern) and optimized UI rendering concurrency for millions of active monthly users. In the open-source space, I actively build privacy-first AI infrastructure, including zero-latency inference engines capable of running massive 8k-context LLMs natively in the browser without server offloading. ## Responses 6 Markdown supported · ⌘↵ to post [Kartik N V J K](https://hashnode.com/@kartik-nvj) [Oct 16](https://hashnode.com/posts/desktop-control-benchmark/6ad1401cc95ead9bd1583d6e/comment/6ad26e009c668912626b810d) I'd like to see the pc\_do call broken out on its own, since folding a form into one or two model calls looks like where most of your time gap comes from. Have you checked whether the model makes more mistakes when it plans a whole form at once, say on the seven-field form with the pre-ticked box? [Kuldeep Singh](https://hashnode.com/@ar5en1c) [Oct 16](https://hashnode.com/posts/desktop-control-benchmark/6ad1401cc95ead9bd1583d6e/comment/6ad26e009c668912626b810d#reply-6ad27306ce3b8cfeaa376430) Good question. I haven't tested pc\_do on its own yet, so I can't say for sure how much of the speed comes from it. My guess is most of it, because the model spends far more time thinking between calls than the tool spends clicking. And yes, doing a whole form in one go did break once, on that exact form. The next keys went in before the dropdown had opened, so one field never got set, but every step still said "ok". That was my bug, not the model's. It waits for the menu now, and the form passed every run in the latest benchmark. Running the same tasks with pc\_do switched off is a good idea. I'll try it and share what I find. [Kuldeep Singh](https://hashnode.com/@ar5en1c) [Oct 16](https://hashnode.com/posts/desktop-control-benchmark/6ad1401cc95ead9bd1583d6e/comment/6ad26e009c668912626b810d#reply-6ad28f9732a10169f494b82a) Ran it. Same tasks, same model, with pc\_do switched off so every click is its own call. Both versions passed every run, including that seven-field form, so doing a whole form at once didn't cause extra mistakes. Without it, runs took about 30% longer and made twice as many calls. Most of that was Calculator, where every key is its own button. On web pages the difference was small. So pc\_do is part of the speed but not all of it. The rest seems to come from how heavy each call is. [Yunus Emre Vurgun](https://hashnode.com/@yunusemrevurgun) [Oct 16](https://hashnode.com/posts/desktop-control-benchmark/6ad1401cc95ead9bd1583d6e/comment/6ad1b4b435e60559e2c3c67e) Nice that you scored on what the page ended up showing rather than on what the model claimed; that rule alone makes these numbers more trustworthy than most agent benchmarks. The Tessa Bright failure is interesting because the tool did exactly what it was told, so a cheap guard could be a pre-action echo for destructive buttons: return the target row's label alongside the element ref and let the model confirm before pressing. It would also be worth reporting the tool-list size as its own column, since 49k vs 3.2k tokens per call probably explains more of the time gap than anything about how the tools actually drive the UI. [Kuldeep Singh](https://hashnode.com/@ar5en1c) [Oct 16](https://hashnode.com/posts/desktop-control-benchmark/6ad1401cc95ead9bd1583d6e/comment/6ad1b4b435e60559e2c3c67e#reply-6ad2640b2b262a27afd6508e) Thanks, this is a good catch. Looking back at the failed run, the click answer only said it pressed "Approve", not whose row that was, so the model had nothing to check before it pressed. Putting the row label in that answer is cheap and I'm adding it. You're right about the tool list too. It's the main reason for the token gap, so it should have its own column. I'd be careful about time, though. Most of a run is the model deciding, and Cua Driver made 441 calls against 216, so I think the call count explains more of the time gap than the list does. I haven't measured the split. [Yunus Emre Vurgun](https://hashnode.com/@yunusemrevurgun) [Oct 17](https://hashnode.com/posts/desktop-control-benchmark/6ad1401cc95ead9bd1583d6e/comment/6ad1b4b435e60559e2c3c67e#reply-6ad3075c771163c8644b6059) Fair point on call count; 441 vs 216 round trips is a bigger lever than prompt size once the model's thinking time dominates. If you want to separate them cheaply, log time-to-first-token per call: that mostly tracks input tokens, while the gaps between calls track planning, so the two costs show up separately without rerunning anything. Your pc\_do-off run fits that too, since the extra 30% came mostly from Calculator's per-key calls.