Comment by Kartik N V J K on ""It worked when I ran it" - the six lines that quietly break ML reproducibility"

Feed
Pro
Search

Sign in
FactoryKit - the AI software factory: tasks in, pull requests out Bug0 - The AI-native e2e QA regression testing The foreword by Hashnode - official blog from the Hashnode team Passmark - The open-source AI framework for regression testing Hashnode gql skill - let your AI agent publish to your Hashnode blog Hackathons Changelog Brand @hashnode on X Hashnode on LinkedIn Support - hello+support@hashnode.com Code of Conduct Terms Privacy Sitemap

Search Hashnode

Search posts, tags, users, and pages

Comment by Kartik N V J K on ""It worked when I ran it" - the six lines that quietly break ML reproducibility" | Hashnode

DiscussionPopular posts

Comment"It worked when I ran it" - the six lines that quietly break ML reproducibility

Kartik N V J K

AI Developer | Making AI reliable, trustworthy & accessible to everyone | Active community contributor

3d ago

The "I re-ran my own script and couldn't reproduce it either" moment is the one that converts people to pinning everything, because it proves the code was never the whole story. Usually it's the unlogged stuff: an unset seed, nondeterministic data ordering, or a library version that moved under you. Of your six lines, which bites most often in practice, the seed or the environment drift?

JAY TANK

Engineering insights on building and scaling modern software.

3d ago

Great point; it's always the unlogged layer. In my experience, the seed bites most often (first thing to break on a re-run, one-line fix), but environment drift is the more expensive one, silent and delayed, showing up on someone else's machine or next month's CI instead of your second run. That's exactly why the tool checks statically instead of trusting a re-run to reveal it. Do you pin the full transitive tree + base-image digest, or stop at top-level pins?