---
title: "Comment by BomToBe on \"Layered Skill Test Coverage: A Better Way to Evaluate AI Agent Skills\" | Hashnode"
description: "The distinction between ‘never saw it’ and ‘saw it but ignored it’ is useful. How do you define the denominator for coverage: only tasks where the skill’s trigg"
url: "https://hashnode.com/posts/layered-skill-test-coverage-a-better-way-to-evaluate-ai-agent-skills/6acf39ba192072e41ab202a7/comment/6acfbf507e72cdd81108c405"
---

[Discussion](https://hashnode.com/posts/layered-skill-test-coverage-a-better-way-to-evaluate-ai-agent-skills/6acf39ba192072e41ab202a7)

Comment [Layered Skill Test Coverage: A Better Way to Evaluate AI Agent Skills](https://hashnode.com/posts/layered-skill-test-coverage-a-better-way-to-evaluate-ai-agent-skills/6acf39ba192072e41ab202a7)

[BomToBe](https://hashnode.com/@bomtobe-orke)

[Oct 14](https://hashnode.com/posts/layered-skill-test-coverage-a-better-way-to-evaluate-ai-agent-skills/6acf39ba192072e41ab202a7/comment/6acfbf507e72cdd81108c405)

The distinction between ‘never saw it’ and ‘saw it but ignored it’ is useful. How do you define the denominator for coverage: only tasks where the skill’s trigger conditions apply, or every task where it was available? Without an applicability label, an agent correctly skipping an irrelevant skill could look like a retrieval failure. I’m with BOMTOBE, the team behind orke, a desktop workspace for Claude Code and Codex; we’re interested in making agent workflows easier to inspect.

[Subham Panda](https://hashnode.com/@subhampanda)

[Oct 15](https://hashnode.com/posts/layered-skill-test-coverage-a-better-way-to-evaluate-ai-agent-skills/6acf39ba192072e41ab202a7/comment/6acfbf507e72cdd81108c405#reply-6ad0dd2eb5e2c69aa7ce91d9)

Yes the first part would be on the task list. But the second part of the paper to detext mutation we can use it to get line by line coverage of a skill. I will create a detailed blog post on that today.

[Subham Panda](https://hashnode.com/@subhampanda)

[Oct 15](https://hashnode.com/posts/layered-skill-test-coverage-a-better-way-to-evaluate-ai-agent-skills/6acf39ba192072e41ab202a7/comment/6acfbf507e72cdd81108c405#reply-6ad127bf369d82c3f1553eae)

For more information: [blog.subhampanda.com/ai-skill-test-coverage-can-w…](https://blog.subhampanda.com/ai-skill-test-coverage-can-we-create-line-coverage-for-ai-skills-eval)

[BomToBe](https://hashnode.com/@bomtobe-orke)

[Oct 15](https://hashnode.com/posts/layered-skill-test-coverage-a-better-way-to-evaluate-ai-agent-skills/6acf39ba192072e41ab202a7/comment/6acfbf507e72cdd81108c405#reply-6ad164526743bdfd3072be65)

Thanks for linking the line-coverage write-up. It clarifies how you map tests to skill spans and use mutation to check coverage. For the denominator, would you count only tasks where the trigger conditions apply, then measure eligible-task misses separately from correct skips? That would keep irrelevant tasks from looking like retrieval failures.
