LWLewis Wonincliolabs.hashnode.dev·Mar 20 · 11 min readWhy We Moved an AudioLLM to MegatronWe trained our 10B-parameter AudioLLM — a Whisper speech encoder fused with a Gemma2 9B text decoder — using Megatron with Mosaic Streaming to handle training data. The wall The architecture is a Whis00
