Redis optimization

The best performance requires configuration tuning, which will reduce the likelihood of unforeseen errors, limitations and peak loads. This applies to all parts of the system, database, and hardware.

Where to begin

Any optimization should start with the profiling and application testing under high loads. Also do not forget about optimizing server, proper Nginx configurations, and further optimization of the Ubuntu-server. This approach will save your nerves and simplify further tuning procedure.

When Redis is set up and ready to go, be sure to check it with the built-in test:

$ redis-benchmark -q -n 100000 -c 50 -P 12

# The output
PING_INLINE: 641025.62 requests per second                                                             
PING_BULK: 694444.50 requests per second                                                               
SET: 434782.59 requests per second                                                                     
GET: 520833.34 requests per second                                                                     
INCR: 456621.00 requests per second                                                                    
LPUSH: 462962.94 requests per second                                                                   
LPOP: 440528.62 requests per second                                                                    
SADD: 418410.06 requests per second                                                                    
SPOP: 549450.56 requests per second                                                                    
LPUSH (needed to benchmark LRANGE): 473933.66 requests per second                                      
LRANGE_100 (first 100 elements): 37397.16 requests per second                                          
LRANGE_300 (first 300 elements): 10351.97 requests per second                                          
LRANGE_500 (first 450 elements): 6939.14 requests per second                                           
LRANGE_600 (first 600 elements): 5518.76 requests per second                                           
MSET (10 keys): 88105.73 requests per second

# Executes 100 000 requests from 50 clients by 12 commands at the same time

Backups

Redis has two percistence modes: RDB and AOF. Redis persistence

With the RDB, system creates a compact data snapshots at specified intervals. This is a good approach for data recovery in case of failure. Backups are written in a parallel process with the command fork(), which in the case of a large database requires resources and time. In addition, in a case of unexpected server shutdown, all the data, which were not recorded or were in the process of creating a backup, will be lost.

AOF is a operations log, that clients perform. The method name stands for Append Only File, ie all new data is added to the existing data, and every second by default. So in a case of failure, the losses will be minimal. But the approach is a little slower than the RDB, the log file is substantially larger, and performance depends on fsync parameters.

To configure Redis, edit it's configuration file /etc/redis/redis.conf:

 # Default parameters
save 900 1
save 300 10
save 60 10000

dir /var/redis/	# Sorage directory
dbfilename dump.rdb	# Filename 
rdbcompression yes	# Compression

# RDB is enabled by default

If you want maximum performance — completely disable persistence. Or adjust the snapshots frequency. Default parameters are:

  • Duplicate if there was at least one change for 15 minutes (900 seconds);
  • Duplicate if there were at least 10 changes for 5 minutes (300 seconds);
  • Create a copy if there was 10 000 changes during one minute .

Another option is replication.

Additional parameters

To avoid bottlenecks, increase the number of socket listening connections:

tcp-backlog 65536

# Default is 511

Also pay attention to the allowed number of clients:

maxclients 10000

# If too low, may encounter 'max number of clients reached'

Redis uses a full available maximum of RAM. So it makes sense to limit the allowed amount, so the other server processes can also be performed:

maxmemory 1572864000

# In bytes

Finding problems with INFO

Redis statistics can help in finding errors and weak points. For example, it is possible to estimate the speed of the cache:

$ redis-cli INFO stats |grep keyspace
keyspace_hits:1920
keyspace_misses:930

# System is slow

This may help to track maxmemory limit of available memory through the eviction of keys:

$ redis-cli INFO stats |grep evicted_keys
evicted_keys:11582

# The system rests on the available memory

It is also necessary to choose the best policy key eviction policy:

maxmemory-policy allkeys-lru

# Specify in the Redis configuration file

There're six parameters:

  • noeviction — return an error;
  • allkeys-lru — evict least recently used (LRU) keys;
  • volatile-lru — evict least recently used keys with specified expiration;
  • allkeys-random — evict random keys;
  • volatile-random — evict random keys with specified expiration;
  • volatile-ttl — evict keys with smallest expiration;

The most important

To create a fault-tolerant system use replication. Disabling snapshots will give a significant performance boost. It is also necessary to choose the best possible eviction policy for your application.

материалы по теме

  • NoSQL database solutions
  • Topics

    # redis # tuning

    Подпишитесь на Хайлоад с помощью Google аккаунта
    или закройте эту хрень