Tokenization : for freshers

Think tokenization as chopping vegetables to cook them as without cutting or chopping, how we’ll proceed with cooking. It means converting big tasks into small ones for the ease of use. Definition could be converting sentences into small words or cha...