---
title: "#experimentation articles | Hashnode"
description: "Explore 51 articles tagged #experimentation on Hashnode. Tutorials, guides, and how-tos from the community."
url: "https://hashnode.com/tag/experimentation"
---

Tag

# #experimentation

51 posts · 2 followers

[Simran Malik](https://hashnode.com/@simranmalik) · [Nov 4](https://hashnode.com/posts/build-your-v0/6aea8258c6c2b3c1633e3977)

### [Build your v0, my dear](https://simranbuilds.hashnode.dev/build-your-v0?utm_source=hashnode&utm_medium=feed)

get started with your AI project the imperfect way

3 min read · [simranbuilds.hashnode.dev](https://simranbuilds.hashnode.dev/build-your-v0?utm_source=hashnode&utm_medium=feed)

[1](https://hashnode.com/posts/build-your-v0/6aea8258c6c2b3c1633e3977)

[Gary Stafford](https://hashnode.com/@garystafford) · [Nov 3](https://hashnode.com/posts/from-hypothesis-to-understanding-a-practical-framework-for-applied-ai-engineering-experiments/6ae9ef828b8f5ca5929abb3e)

### [From Hypothesis to Understanding: A Practical Framework for Applied AI Engineering Experiments](https://blog.latentthoughts.com/from-hypothesis-to-understanding-a-practical-framework-for-applied-ai-engineering-experiments?utm_source=hashnode&utm_medium=feed)

Turn a favorable engineering idea into a clear decision: start, keep going, change direction, or stop.

15 min read · [blog.latentthoughts.com](https://blog.latentthoughts.com/from-hypothesis-to-understanding-a-practical-framework-for-applied-ai-engineering-experiments?utm_source=hashnode&utm_medium=feed)

[6](https://hashnode.com/posts/from-hypothesis-to-understanding-a-practical-framework-for-applied-ai-engineering-experiments/6ae9ef828b8f5ca5929abb3e)

[Abisoye Alli-Balogun](https://hashnode.com/@AbisoyeAlli) · [Sep 14](https://hashnode.com/posts/how-to-implement-feature-flags-for-safe-and-gradual-rollouts/6aa80baffb20ceef80e8afa4)

### [How to Implement Feature Flags for Safe and Gradual Rollouts](https://freecodecamp.org/news/how-to-implement-feature-flags-for-safe-and-gradual-rollouts?utm_source=hashnode&utm_medium=feed)

Feature flags are one of the most powerful tools in a team's deployment arsenal. They decouple deployment from release, meaning your CI/CD pipeline can push code to production servers on every merge,

15 min read · [freecodecamp.org](https://freecodecamp.org/news/how-to-implement-feature-flags-for-safe-and-gradual-rollouts?utm_source=hashnode&utm_medium=feed)

[mayaandersson](https://hashnode.com/@mayaanderssondev) · [Aug 12](https://hashnode.com/posts/run-forty-experiments-against-one-eval-set-and-you-will-find-an-improvement-that-is-not-there/6a7c7e4cae9b7572d6527799)

### [Run forty experiments against one eval set and you will find an improvement that is not there](https://llmasajudge.hashnode.dev/run-forty-experiments-against-one-eval-set-and-you-will-find-an-improvement-that-is-not-there?utm_source=hashnode&utm_medium=feed)

You iterate on a prompt, run the eval set, keep the best variant. Repeat for a quarter. The number that survives is the maximum of forty noisy measurements, and the maximum of noisy measurements is an

5 min read · [llmasajudge.hashnode.dev](https://llmasajudge.hashnode.dev/run-forty-experiments-against-one-eval-set-and-you-will-find-an-improvement-that-is-not-there?utm_source=hashnode&utm_medium=feed)

[Rudrendu Paul](https://hashnode.com/@rudrendupaul) · [Aug 11](https://hashnode.com/posts/causal-inference-at-scale-with-case-studies/6a7b522a304c202420dfd496)

### [Product Experimentation at Scale: How Airbnb, Netflix, Lyft, and Uber run Causal Inference on LLM-Based AI Features](https://freecodecamp.org/news/causal-inference-at-scale-with-case-studies?utm_source=hashnode&utm_medium=feed)

Causal inference for LLM-based AI features is no longer theoretical. Airbnb, Netflix, Lyft, and Uber have published detailed engineering blog posts describing exactly how they measure the causal impac

37 min read · [freecodecamp.org](https://freecodecamp.org/news/causal-inference-at-scale-with-case-studies?utm_source=hashnode&utm_medium=feed)

[Rudrendu Paul](https://hashnode.com/@rudrendupaul) · [Aug 11](https://hashnode.com/posts/doubly-robust-estimation-for-llm-product-experiments/6a7a662d92a5f4f663b525f9)

### [Product Experimentation with Doubly Robust Estimation: When Both Your Models Are Wrong in LLM Applications](https://freecodecamp.org/news/doubly-robust-estimation-for-llm-product-experiments?utm_source=hashnode&utm_medium=feed)

Your AI product shipped an agent-mode opt-in six months ago. You ran a propensity analysis, adjusted for engagement tier and query confidence, and reported a clean +8 percentage-point lift in task com

23 min read · [freecodecamp.org](https://freecodecamp.org/news/doubly-robust-estimation-for-llm-product-experiments?utm_source=hashnode&utm_medium=feed)

[Rudrendu Paul](https://hashnode.com/@rudrendupaul) · [Jul 29](https://hashnode.com/posts/instrumental-variables-for-llm-routing-in-python/6a69ffcc634c4a299b014f9f)

### [Product Experimentation with Instrumental Variables: Unconfounding LLM Routing Decisions in Python](https://freecodecamp.org/news/instrumental-variables-for-llm-routing-in-python?utm_source=hashnode&utm_medium=feed)

For data science leaders and product managers who are overseeing multi-model gateways, the standard regression approach to measuring model quality is fundamentally flawed. You're running a causal infe

21 min read · [freecodecamp.org](https://freecodecamp.org/news/instrumental-variables-for-llm-routing-in-python?utm_source=hashnode&utm_medium=feed)

[Rudrendu Paul](https://hashnode.com/@rudrendupaul) · [Jul 23](https://hashnode.com/posts/counterfactual-meta-learners-for-llm-prompt-decisions/6a624877953fb9a0375f9238)

### [Product Experiment Counterfactual Methods for Estimating the Effects of AI Prompt Engineering](https://freecodecamp.org/news/counterfactual-meta-learners-for-llm-prompt-decisions?utm_source=hashnode&utm_medium=feed)

Imagine your team deployed Prompt A globally two weeks ago. Tight deadlines and high confidence meant the rollout hit 100 percent of users without any A/B testing, shadow traffic, or holdout groups. W

22 min read · [freecodecamp.org](https://freecodecamp.org/news/counterfactual-meta-learners-for-llm-prompt-decisions?utm_source=hashnode&utm_medium=feed)

[Aman Jain](https://hashnode.com/@aman454) · [Jul 17](https://hashnode.com/posts/from-building-a-gpt-model-to-experimenting-with-one/6a5a99e4a31fb9f4df581b01)

### [From Building a GPT Model to Experimenting With One](https://curious-pm.hashnode.dev/from-building-a-gpt-model-to-experimenting-with-one?utm_source=hashnode&utm_medium=feed)

Creating a reusable framework to explore the training and architecture decisions inside a small GPT model

8 min read · [curious-pm.hashnode.dev](https://curious-pm.hashnode.dev/from-building-a-gpt-model-to-experimenting-with-one?utm_source=hashnode&utm_medium=feed)

[Rudrendu Paul](https://hashnode.com/@rudrendupaul) · [Jul 15](https://hashnode.com/posts/regression-models-for-causal-inference-on-ai-features/6a57a65ae479ecc16ad3b5b5)

### [Product Experimentation with Regression-Based Causal Inference: Estimating LLM Feature Impact with Python and statsmodels](https://freecodecamp.org/news/regression-models-for-causal-inference-on-ai-features?utm_source=hashnode&utm_medium=feed)

A randomized A/B test is the cleanest form of product experiment available. The coin flip that splits users between the new prompt template and the control removes every possible confounder by constru

20 min read · [freecodecamp.org](https://freecodecamp.org/news/regression-models-for-causal-inference-on-ai-features?utm_source=hashnode&utm_medium=feed)
