Speculative Decoding

Speculative Decoding is an inference optimization technique that pairs a large target model with a lightweight draft mechanism that quickly proposes several next tokens. The target model verifies thos