The representative soft-launch cohort is where I would add explicit data-history strata. Selecting a few users with long-lived records, unusual status transitions and older imported data can reveal compatibility issues that a random sample of mostly new accounts will miss.
I would record those strata with the rollout observations, so expanding access means filling evidence gaps rather than simply increasing the percentage. Comparing workflow completion within each group is also more informative than one aggregate success rate. That fits your distinction between learning from real histories and separately validating full-rollout load.
Ahmet Özel
AI Engineer. Computer Vision, RAG and LLM agents.
The representative soft-launch cohort is where I would add explicit data-history strata. Selecting a few users with long-lived records, unusual status transitions and older imported data can reveal compatibility issues that a random sample of mostly new accounts will miss.
I would record those strata with the rollout observations, so expanding access means filling evidence gaps rather than simply increasing the percentage. Comparing workflow completion within each group is also more informative than one aggregate success rate. That fits your distinction between learning from real histories and separately validating full-rollout load.