---
title: "Mohamed Zhioua (@MohamedZhioua) | Hashnode"
description: "Skilled Full Stack JavaScript Developer 👨‍💻 With a strong passion for problem-solving. Read the latest articles by Mohamed Zhioua on Hashnode."
url: "https://hashnode.com/@MohamedZhioua"
---

@MohamedZhioua

# Mohamed Zhioua

@MohamedZhioua · Tunis, Tunisie · Joined September 2023

Skilled Full Stack JavaScript Developer 👨‍💻 With a strong passion for problem-solving

[Website](https://mohamedzhioua.vercel.app/)

## Comments

[Thanks. A few things the data can and can't say here. The incompletes don't feed the fit. All 47 not-completed runs are left out of it, and Codex is scored on its own, never in the headline. The two rates don't count the same event. A Codex run is one whole turn, and it counts as failed when Codex's own turn-complete event carries an error; an interrupted turn would count as unknown, not failed. Of the 34, 33 are failed and 1 is unknown. A Claude Code run is one subagent, failed only on a refusal or when it ends on a tool error with no clean stop; its 13 are 3 failed and 10 unknown. So part of the gap is what each harness reports at the end of a run. Task kind: they can't cluster on one, because Codex rows have no kind. The kind comes from the Claude Code subagent's type, and a Codex turn has none, so every published Codex row is "other". That's a real gap for Codex. Timeouts or plumbing: the error text is never copied (only allowlisted fields leave a transcript), so the rows can't tell a timeout from a usage limit or a dropped connection. What they do show: some failed turns ended within seconds with no tool call, which looks like plumbing, while others ran for a minute or more with tool calls before the error. That split isn't in the generated report yet, so I'll add it there, where its counts get checked like every other number.](https://hashnode.com/posts/what-3-383-coding-agent-runs-taught-me-about-estimating-agent-work/6af0f84698809b0a1bcc2696/comment/6af354ebbdc37ae99b8394be#reply-6af388bdcd079651d3d457d5)

Reply · Article · 5d ago · [What 3,383 coding-agent runs taught me about estimating agent work](https://hashnode.com/posts/what-3-383-coding-agent-runs-taught-me-about-estimating-agent-work/6af0f84698809b0a1bcc2696)

[Agreed on both counts. The intervals are wide because there are only 56 to 63 sessions, all mine, so nothing here supports a claim beyond my own workflow, and the README says so. Forward in time: the headline whole-session numbers already are forward (each of 56 later sessions predicted from earlier weeks only; P80 80%, CI 70% to 91%), but those same folds chose the method, so they read slightly optimistic. The clean check is pre-registered: selection was frozen at 2026-W45, and every week after it forms a locked holdout that never takes part in selection. It reports "not enough data" until at least 20 later sessions have 5 or more completed runs each. The first post-freeze week goes in next week, and it should take a few weeks to fill. It's still my data, though, so it answers the time half of your point, not the people half. Another contributor: the backtest already has a contributor split, and it can't run until a second person contributes. agentpace share --dry-run prints the exact rows first (durations, outcomes, task kinds; no prompts, code or file paths), and contributing is a pull request with one file. That's the open call in issue #2: github.com/mohamedzhioua/agentpace/issues/2](https://hashnode.com/posts/what-3-383-coding-agent-runs-taught-me-about-estimating-agent-work/6af0f84698809b0a1bcc2696/comment/6af129ca7d64feecc5f27f81#reply-6af2f2c4363d372c0c1ccc24)

Reply · Article · 5d ago · [What 3,383 coding-agent runs taught me about estimating agent work](https://hashnode.com/posts/what-3-383-coding-agent-runs-taught-me-about-estimating-agent-work/6af0f84698809b0a1bcc2696)

[Follow-up on the gap I mentioned: it's closed in 0.12.11. The whole-session test now also runs with whole projects held out: 63 sessions across 11 projects, each predicted from every run outside its project. The P80 covered 87% (CI 79% to 95%) and the P90 95% (CI 89% to 100%), so on an unseen codebase the ranges stay calibrated. But the interval for a sum of medians' pinball loss minus ours is -7019 to +27430, so it isn't shown to beat that baseline there, unlike on the time split. It's retrospective (training includes later weeks), from one contributor, and it never chooses the method. Every project is listed on its own in the report: https://github.com/mohamedzhioua/agentpace/blob/main/docs/backtest.md#whole-sessions-projects-held-out-oracle-structure](https://hashnode.com/posts/what-3-383-coding-agent-runs-taught-me-about-estimating-agent-work/6af0f84698809b0a1bcc2696/comment/6af129ca7d64feecc5f27f81#reply-6af25a653d9212947392cb1b)

Reply · Article · 6d ago · [What 3,383 coding-agent runs taught me about estimating agent work](https://hashnode.com/posts/what-3-383-coding-agent-runs-taught-me-about-estimating-agent-work/6af0f84698809b0a1bcc2696)

[Thanks. In this data, risk can't be separated from task kind, because it isn't a judgment about the work. It's the agent's role as recorded: read-only review and research agents are low, everything else is normal, and so far no run is high. So risk mostly follows kind. The bigger driver of the tail is the model within a kind, though that's confounded with what each model was given: Feature work: Sonnet runs (1,148) have a P90 of 2 min, Opus runs (205) a P90 of 87 min. Other work: P90 21 min on Sonnet, 83 min on Opus. Reviews go the other way: P90 8 min on Opus, 19 min on Sonnet. Across kinds, with all models pooled, the P90 runs from 12 min for reviews to 24 min for features and 44 min for other work. So if you price from the tail, pin the model first, then the kind. At the plan level the tail is wider again: a whole session's P90 is typically 2.7x its P50. One small correction on the mechanics: the uncertainty about a cell's mean is drawn once per cell per simulation, and a session effect once per plan, so tasks that share a cell or a session move together instead of averaging out. The full table by kind, risk and model is in the repo: https://github.com/mohamedzhioua/agentpace](https://hashnode.com/posts/what-3-383-coding-agent-runs-taught-me-about-estimating-agent-work/6af0f84698809b0a1bcc2696/comment/6af132b62eec62cd8ae2760f#reply-6af19f307f00aa502ff1abf7)

Reply · Article · 6d ago · [What 3,383 coding-agent runs taught me about estimating agent work](https://hashnode.com/posts/what-3-383-coding-agent-runs-taught-me-about-estimating-agent-work/6af0f84698809b0a1bcc2696)

[Thanks, that's exactly the right question. Short answer: no. The 56 sessions come from a time split, not a project split. The headline test is rolling-origin: train on earlier weeks, then predict whole later sessions, so no run leaks into its own forecast. Both sides are my own projects, though, so it mostly checks calibration to my workflow over time, as you suspect. There is a separate leave-one-project-out split, reported under its own name, but at the task level: single-task P80 coverage is 88.9% with each project held out in turn. The largest held-out project, a Codex one, holds 137 of its 212 test sessions, so the report also lists each project on its own and the scores without that one. Coverage varies a lot from project to project, which is the external-validity warning in plain numbers. What isn't tested yet is the case you're pointing at: the whole-session test with projects held out, which is where the shared session effect would have to generalize to a new codebase. I haven't run that, and with most held-out sessions in a few projects it would be thin. I've noted it as a gap. The bigger hole is that the contributor split can't run at all until a second person shares their runs. The per-project table is here: https://github.com/mohamedzhioua/agentpace/blob/main/docs/backtest.md#project-split](https://hashnode.com/posts/what-3-383-coding-agent-runs-taught-me-about-estimating-agent-work/6af0f84698809b0a1bcc2696/comment/6af129ca7d64feecc5f27f81#reply-6af19f19bcff827a3c7adf20)

Reply · Article · 6d ago · [What 3,383 coding-agent runs taught me about estimating agent work](https://hashnode.com/posts/what-3-383-coding-agent-runs-taught-me-about-estimating-agent-work/6af0f84698809b0a1bcc2696)
