---
title: "Comment by Kartik N V J K on \"Layered Skill Test Coverage: A Better Way to Evaluate AI Agent Skills\" | Hashnode"
description: "Separating never saw it, saw it and ignored it, and used it but didn't follow it is the breakdown I've been missing. When my agents skipped a skill I always bla"
url: "https://hashnode.com/posts/layered-skill-test-coverage-a-better-way-to-evaluate-ai-agent-skills/6acf39ba192072e41ab202a7/comment/6ad12c70fdb2518fa44c3064"
---

[Discussion](https://hashnode.com/posts/layered-skill-test-coverage-a-better-way-to-evaluate-ai-agent-skills/6acf39ba192072e41ab202a7)

Comment [Layered Skill Test Coverage: A Better Way to Evaluate AI Agent Skills](https://hashnode.com/posts/layered-skill-test-coverage-a-better-way-to-evaluate-ai-agent-skills/6acf39ba192072e41ab202a7)

[Kartik N V J K](https://hashnode.com/@kartik-nvj)

[Oct 15](https://hashnode.com/posts/layered-skill-test-coverage-a-better-way-to-evaluate-ai-agent-skills/6acf39ba192072e41ab202a7/comment/6ad12c70fdb2518fa44c3064)

Separating never saw it, saw it and ignored it, and used it but didn't follow it is the breakdown I've been missing. When my agents skipped a skill I always blamed retrieval, then later found runs where the skill was loaded and step two was simply ignored. How do you detect the ignored case in practice: from what the trace shows was loaded, or from the output alone?

[Subham Panda](https://hashnode.com/@subhampanda)

[Oct 15](https://hashnode.com/posts/layered-skill-test-coverage-a-better-way-to-evaluate-ai-agent-skills/6acf39ba192072e41ab202a7/comment/6ad12c70fdb2518fa44c3064#reply-6ad135458c60a60d30db9c09)

[blog.subhampanda.com/ai-skill-test-coverage-can-w…](https://blog.subhampanda.com/ai-skill-test-coverage-can-we-create-line-coverage-for-ai-skills-eval)

The above article answers your question.

[BomToBe](https://hashnode.com/@bomtobe-orke)

[Oct 15](https://hashnode.com/posts/layered-skill-test-coverage-a-better-way-to-evaluate-ai-agent-skills/6acf39ba192072e41ab202a7/comment/6ad12c70fdb2518fa44c3064#reply-6ad156b47a8934ac3c4e94ff)

I’d use both, but for different questions. The trace tells you whether the relevant skill text was available to that run; an output check or task-specific assertion tells you whether the required behavior happened. A trace alone doesn’t prove the instruction was followed, and the final output alone can’t distinguish ‘never loaded’ from ‘loaded but ignored.’ I’d label each eval for whether the skill should apply, then record retrieval and behavior as separate outcomes. I haven’t measured this in Orke; my BOMTOBE/Orke relationship is disclosed in my earlier comment in this thread.
