Tag feed

#benchmarking

41 posts1 followers

Explore Hashnode

Alternatives

Trending tags this week

MMMarco Mornatiblog.mornati.netMay 31 · 14 min read

Your AI Agent Deserves a Tool Harness, Not a Wild West

We started the same way everyone does: give the LLM access to everything and hope it figures it out. Connect the GitHub MCP, the Jira MCP, the internal product API MCP, throw in a database schema or t

0

OOmnithiumomnithium.hashnode.devMay 31 · 15 min read

The Enterprise AI Agent Performance Benchmark: How to Measure and Compare Agent Effectiveness

Why Current AI Agent Benchmarks Fail the Enterprise Why do most AI agent benchmarks fail to predict what actually happens in your production environment? Because they measure the wrong things, in the

0

NSNeeloppher Syedneeloppher.hashnode.devMay 15 · 8 min read

ASR Evaluation Framework: Benchmarking Speech Recognition Models Across Accuracy, Speed, and Robustness

Picking an ASR model for production is not straightforward. Whisper might be the most accurate for general English but too slow for real-time use. Wav2Vec2 might be fast enough for edge devices but st

0

EEntreelistiaq.hashnode.devMay 8 · 6 min read

Benchmarking pgvector IVFFlat vs HNSW indexes for production RAG applications - Blog Post

Benchmarking pgvector IVFFlat vs HNSW indexes for production RAG applications I've spent the last three months stress-testing vector indexes in production environments, and the results challenge conventional wisdom about when to use each index type. ...

0

EEntreelistiaq.hashnode.devMay 8 · 6 min read

Benchmarking pgvector IVFFlat vs HNSW indexes for production RAG applications - Blog Post

Benchmarking pgvector IVFFlat vs HNSW indexes for production RAG applications I've spent the last three months stress-testing vector indexes in production environments, and the results challenge conventional wisdom about when to use each index type. ...

0

NVNolan Vossnolan-voss.hashnode.devApr 30 · 7 min read

The 1,000-Message Test: A Benchmark for AI Memory That Most Apps Fail

Most apps that claim "memory" don't have it. I spent 200 days testing AI companion apps. 15 platforms, every subscription paid out of pocket. What I found, consistently, is that "memory" in marketing

0

Bbinaditbinadit.hashnode.devApr 17 · 7 min read

Real-world website hosting performance: measuring what providers don't disclose

Measuring real-world hosting performance: what providers won't tell you about production load When selecting hosting infrastructure, most engineering teams compare advertised uptime percentages and basic response time benchmarks. These standard metri...

0

DDilberdilber.hashnode.devApr 16 · 8 min read

Why Tokens Per Joule Matters More Than Tokens Per Second

Most GPU benchmarks report tokens/sec, but that metric ignores the dominant driver of real-world inference cost: energy. I built a cross-platform telemetry suite to measure Tokens Per Joule (T/J) — to

0

Aaiagentmemoryaiagentmemory.hashnode.devApr 7 · 8 min read

LLM Memory Bench: Evaluating AI's Recall Capabilities

An LLM memory bench is a crucial tool for evaluating how well Large Language Models (LLMs) store, retrieve, and use information over time. It provides standardized tests to assess their ability to maintain context and recall past interactions accurat...

0

AKAdam Kingflexai.hashnode.devMar 31 · 6 min read

FlexBench: An Open-Source, Modular MLPerf Benchmark for LLM Inference

TL;DR AI system benchmarks like MLPerf struggle to keep pace with the rapidly evolving model landscape, making it difficult for organizations to make informed deployment decisions. We believe benchmar

0

#benchmarking

Search Hashnode

#benchmarking

Explore Hashnode

Trending tags this week

Your AI Agent Deserves a Tool Harness, Not a Wild West

The Enterprise AI Agent Performance Benchmark: How to Measure and Compare Agent Effectiveness

ASR Evaluation Framework: Benchmarking Speech Recognition Models Across Accuracy, Speed, and Robustness

Benchmarking pgvector IVFFlat vs HNSW indexes for production RAG applications - Blog Post

Benchmarking pgvector IVFFlat vs HNSW indexes for production RAG applications - Blog Post

The 1,000-Message Test: A Benchmark for AI Memory That Most Apps Fail

Real-world website hosting performance: measuring what providers don't disclose

Why Tokens Per Joule Matters More Than Tokens Per Second

LLM Memory Bench: Evaluating AI's Recall Capabilities

FlexBench: An Open-Source, Modular MLPerf Benchmark for LLM Inference