Home
Blogs
Bookmarks
Forums
Hackathons
Search

Author

Write
Drafts

New
Bug0 - The AI-native e2e QA regression testing Bug0 Browsers - Cloud Chromium on demand, per-minute, live preview Passmark - The open-source AI framework for regression testing Changelog Brand @hashnode on X Hashnode on LinkedIn Code of Conduct Support - hello+support@hashnode.com
Sign in
Terms Privacy Sitemap
© 2026 LinearBytes Inc.

Tag feed

#megatron

3 posts·0 followers

Articles Threads

Trending tags this week

Search Hashnode

Search posts, tags, users, and pages

Tag feed

#megatron

3 posts·0 followers

Articles Threads

LWLewis Wonincliolabs.hashnode.dev

From Loss=36 to Convergence: Integrating Whisper+Gemma2 into Megatron's TransformerEngine

1h ago · 9 min read · From Loss=36 to Convergence: Integrating Whisper+Gemma2 into Megatron's TransformerEngine When we started debugging our AudioLLM on the Megatron trainer, our loss started at 36. This did not make sens

Join discussion

LWLewis Wonincliolabs.hashnode.dev

The MDS Shim — Zero-Conversion Data Loading for 800+ Datasets

Mar 27 · 12 min read · We have about 800 datasets in Mosaic MDS format, with tens of millions of multimodal samples — each one an audio clip, an instruction, and a target response — spread across thousands of compressed sha

Join discussion

LWLewis Wonincliolabs.hashnode.dev

Why We Moved an AudioLLM to Megatron

Mar 20 · 11 min read · We trained our 10B-parameter AudioLLM — a Whisper speech encoder fused with a Gemma2 9B text decoder — using Megatron with Mosaic Streaming to handle training data. The wall The architecture is a Whis

Join discussion