Search posts, tags, users, and pages
CoMem caches transformer mid-layer states to cut 128k-token memory from 89 GB to 18 GB, with 7.83× prefill speedup and RULER score 97.05.
Chaeyeon Mia Lee
@miainflorence · Seoul, South Korea
Markdown supported · ⌘↵ to post
No responses yet.