Jan Vladimir Mostert

Idea Incubator

Oct 9, 2015

As far as I know, Redis allocates memory in blocks / pages, if only some of the keys are freed, the block of memory is still allocated in the OS. In order the free the whole block, all the keys needs to be freed. So what you're seeing is memory fragmentation, actual memory you're using according to the OS will be 90MB whereas data sitting in Redis memory will be 2.7MB.

Quote: Jemalloc (and several other libraries) allocate arenas of sizes larger than an original request, then uses that arena to satisfy similarly-sized requests until the arena is all used. As memory is freed, portions of the arena are re-used as requested. If an arena is 100% empty, the Jemalloc will free it to the operating system. If an arena is 1% used, Jemalloc cannot free it to the operating system.

From the official Redis Docs:

Redis will not always free up (return) memory to the OS when keys are removed. This is not something special about Redis, but it is how most malloc() implementations work. For example if you fill an instance with 5GB worth of data, and then remove the equivalent of 2GB of data, the Resident Set Size (also known as the RSS, which is the number of memory pages consumed by the process) will probably still be around 5GB, even if Redis will claim that the user memory is around 3GB. This happens because the underlying allocator can't easily release the memory. For example often most of the removed keys were allocated in the same pages as the other keys that still exist.

redis.io/topics/memory-optimization

90MB is quite small, but it would be interesting to see what solutions are available to fix Redis memory fragmentation other than restarting Redis. My guess is that most people would simply restart Redis if they hit fragmentation issues, but I'm sure there are better solutions, maybe do a bit of Googling and post here once you find something that looks promising.

Depending on what you're using Redis for, if you're only using it as a caching mechanism, you should be able to set the maxmemory-policy to evict old keys. If you look at the redis.conf, you'll see there are 5 options: raw.githubusercontent.com/antirez/redis/2.8/redis…

volatile-lru -> remove the key with an expire set using an LRU algorithm
allkeys-lru -> remove any key according to the LRU algorithm
volatile-random -> remove a random key with an expire set
allkeys-random -> remove a random key, any key
volatile-ttl -> remove the key with the nearest expire time (minor TTL)
noeviction -> don't expire at all, just return an error on write operations

volatile-lru ( Least Recently Used) seems like the logical choice. I'm not sure what the default is set to.

Either way, if maxmemory is set, then Redis won't consume more memory from the OS than what you've assigned to it in maxmemory and with a a maxmemory-policy, Redis will recycle old keys if it does hit the maxmemory limit.

So to answer your question, I'm not sure. If Redis has a default maxmemory set and a default maxmemory-policy set, then it should be fine, if it doesn't ... it would be fun to see if you can overload Redis and how it responds :-)

Search Hashnode