What is TokeniZation?
Imagine you have a toy box full of letters. To play with them you give each letter a number.
like this:
A → 1
B → 2
C → 3
D → 4
…
So if you write HELLO, it turns into numbers.That’s called tokenization → changing text into numbers that a compu...
ashrafulmalik.hashnode.dev2 min read