---
title: "Discussion on \"Routing AI Requests Without Training Data and Knowing When to Train a Classifier\" | Hashnode"
description: "Discussion on \"Routing AI Requests Without Training Data and Knowing When to Train a Classifier\". The retail examples below are illustrative; they do not report measured production results or disclose an employer's implementation.\nA team adds three specialist agents to an assistant. Routing looks"
url: "https://hashnode.com/posts/routing-ai-requests-without-training-data-and-knowing-when-to-train-a-classifier/6ad6ef8f2200e367f74f72af"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Routing AI Requests Without Training Data and Knowing When to Train a Classifier](https://tokenbytoken.hashnode.dev/routing-ai-requests-without-training-data-and-knowing-when-to-train-a-classifier?utm_source=hashnode&utm_medium=feed)

The retail examples below are illustrative; they do not report measured production results or disclose an employer's implementation. A team adds three specialist agents to an assistant. Routing looks

Oct 20 · 9 min read · tokenbytoken.hashnode.dev  
Read the article

-   [#ai](https://hashnode.com/tag/ai)
-   [#machine-learning](https://hashnode.com/tag/machine-learning)
-   [#llm](https://hashnode.com/tag/llm)
-   [#transformers](https://hashnode.com/tag/transformers)
-   [#routing](https://hashnode.com/tag/routing)

## Written by

[Sai Sandeep Kantareddy](https://hashnode.com/@skantar1)

@skantar1

  
[View profile](https://hashnode.com/@skantar1)

Engineer

Senior ML Engineer | GenAI + RAG Systems | Fine-tuning | MLOps | Conversational & Document AI Building reliable, real-time AI systems across high-impact domains — from Conversational AI and Document Intelligence to Healthcare, Retail, and Compliance. At 7-Eleven, I lead GenAI initiatives involving LLM fine-tuning (Mistral, QLoRA, Unsloth), hybrid RAG pipelines, and multimodal agent-based bots. Domains I specialize in: Conversational AI (Teams + Claude bots, product QA agents) Document AI (OCR + RAG, contract Q&A, layout parsing) Retail & CPG (vendor mapping, shelf audits, promotion lift) Healthcare AI (clinical retrieval, Mayo Clinic work) MLOps & Infra (Databricks, MLflow, vector DBs, CI/CD) Multimodal Vision+LLM (part lookup from images) I work at the intersection of LLM performance, retrieval relevance, and scalable deployment — making AI not just smart, but production-ready. Let’s connect if you’re exploring RAG architectures, chatbot infra, or fine-tuning strategy!

## Responses 3

Markdown supported · ⌘↵ to post

[Kartik N V J K](https://hashnode.com/@kartik-nvj)

[Oct 21](https://hashnode.com/posts/routing-ai-requests-without-training-data-and-knowing-when-to-train-a-classifier/6ad6ef8f2200e367f74f72af/comment/6ad8a771145ba8dc00152235)

"The router's own choice is a prediction, not a label" is something I learned the hard way, after a small classifier I trained on logged dispatches picked up my router's mistakes with full confidence. How do you set the margin threshold when the validation split only has a handful of compound requests like the sales-are-down one?

[Ahmet Özel](https://hashnode.com/@ahmetozel)

[Oct 20](https://hashnode.com/posts/routing-ai-requests-without-training-data-and-knowing-when-to-train-a-classifier/6ad6ef8f2200e367f74f72af/comment/6ad72e6488433910ee98564c)

The review sampling policy becomes part of the training data once uncertain cases and disagreements are deliberately oversampled. That is useful for learning boundaries, but the resulting label distribution no longer represents ordinary traffic.

I would retain the sampling reason and, where possible, selection probability with each reviewed request. Keep a separate randomly sampled traffic slice for estimating auto-dispatch error and coverage at deployment prevalence. Otherwise a classifier can improve on the carefully curated hard-case set while the reported operating error moves simply because the review policy changed. The same distinction helps compare the cascade with a trained replacement fairly.

[Yunus Emre Vurgun](https://hashnode.com/@yunusemrevurgun)

[Oct 20](https://hashnode.com/posts/routing-ai-requests-without-training-data-and-knowing-when-to-train-a-classifier/6ad6ef8f2200e367f74f72af/comment/6ad72816579ec8a8ed2f43e3)

The point about labels being a snapshot is the one I'd stress for the later classifier stage. When you do start collecting labels, keep the capability catalog version next to each one and hold out a test set per version, so a drop in accuracy after onboarding a new agent shows up as a routing regression and not as noise. Class imbalance will also bite early: a forecasting agent that gets 2% of traffic can look fine on overall accuracy while failing most of its requests, so report per-route recall alongside the abstain rate. A small hand-written set of the ambiguous multi-domain questions, like your sales-are-down example, makes a useful fixed regression suite.