The 200-concurrent-query claim is the next place I would put measurement detail beside the 40% average latency reduction. Concurrent users, requests in flight and sustained completed requests per second describe different loads, especially with a hosted model behind the application.
A useful test would ramp query arrivals while ingestion is running, then report throughput, queue wait, tail latency and timed-out requests. Include both short definition questions and longer walkthroughs. That would show whether asynchronous ingestion protects query service under contention, rather than merely making the upload interface nonblocking.
Ahmet Özel
AI Engineer. Computer Vision, RAG and LLM agents.
The 200-concurrent-query claim is the next place I would put measurement detail beside the 40% average latency reduction. Concurrent users, requests in flight and sustained completed requests per second describe different loads, especially with a hosted model behind the application.
A useful test would ramp query arrivals while ingestion is running, then report throughput, queue wait, tail latency and timed-out requests. Include both short definition questions and longer walkthroughs. That would show whether asynchronous ingestion protects query service under contention, rather than merely making the upload interface nonblocking.