"attention is all you need" - - Decoding Jargons.exe

#.Tokenization its the process of converting text into a smaller pieces(token) which is then converted by a LLM to its familiar language which is numerical ID. a token can be : A complete word Part