Good question! We actually use jemalloc at work, so the OS level fragmentation issues that Hongli pointed out don't really impact us. See this link:
joyfulbikeshedding.com/blog/2019-03-14-what-cause…
I do think that combining this with a compacting GC would be helpful, but it's hard for me to estimate the improvement since it's very application specific. The next thing I want to work on is implementing a "variable width allocator" for Ruby. Right now Ruby's GC can only allocate fixed width slots. That means that things like string are allocated via the OS allocator, and that's what causes the fragmentation. If Ruby's GC could allocate various sizes, then the GC could control the fragmentation as well as compaction at the same time. This would fix the fragmentation issue and also be allocator independent. IOW it wouldn't matter if you use jemalloc or just glibc.