NVNaveenKumar VRinclaybrainer.com·13h ago · 8 min readAI - Fundamentals : Part 1 : LLM TokensThere has been a lot of buzz around AI recently. Everywhere you look, people are talking about it, building with it, or trying to figure out how to use it. And, like many of you, we all want to jump o20
TWThomas Weitzelinemria.com·Feb 27, 2025 · 9 min readConcept-Based Tokenization in NLPWhat Is Concept-Based Tokenization? Concept-based tokenization segments text into linguistically meaningful units – typically a root concept (the core meaning of a word) and modifiers (grammatical or semantic affixes). This contrasts with character-l...00
