Search posts, tags, users, and pages
Abstract Algorithms
Exploring the fascinating world of algorithms, data structures, and software engineering through clear explanations and practical examples.
TLDR: LLMs don't read words — they read tokens. A token is roughly 4 characters. Byte Pair Encoding (BPE) builds an efficient subword vocabulary by iteratively merging frequent character pairs. Tokeni
No responses yet.
