主从复制概述
主从复制,是指将一台Redis服务器的数据,复制到其他的Redis服务器。前者称为主节点(master),后者称为从节点(slave);数据的复制是单向的,只能由主节点到从节点。Master以写为主,Slave已读为主 默认情况下,每台Redis服务器都是主节点;且一个主节点可以有多个从节点(或没有从节点),但一个从节点只能有一个主节点。
主从复制的作用 主从复制的作用主要包括: 数据冗余:主从复制实现了数据的热备份,是持久化之外的一种数据冗余方式。 故障恢复:当主节点出现问题时,可以由从节点提供服务,实现快速的故障恢复;实际上是一种服务的冗余。 负载均衡:在主从复制的基础上,配合读写分离,可以由主节点提供写服务,由从节点提供读服务(即写Redis数据时应用连接主节点,读Redis数据时应用连接从节点),分担服务器负载;尤其是在写少读多的场景下,通过多个从节点分担读负载,可以大大提高Redis服务器的并发量。 高可用基石:除了上述作用以外,主从复制还是哨兵和集群能够实施的基础,因此说主从复制是Redis高可用的基础。
环境配置
只配置从库,不用配置主库
127.0.0.1:6379> info replication | |
# Replication | |
role:master | |
connected_slaves:0 | |
master_replid:f47467f195c4ed51c50938a60c17de4b01bb0f72 | |
master_replid2:0000000000000000000000000000000000000000 | |
master_repl_offset:0 | |
second_repl_offset:-1 | |
repl_backlog_active:0 | |
repl_backlog_size:1048576 | |
repl_backlog_first_byte_offset:0 | |
repl_backlog_histlen:0 |
复制3个配置文件,然后修改对应的信息 1、端口 2、pid名字 3、log文件名字 4、 dump.rdp名字 一主二从 从机
127.0.0.1:6380> SLAVEOF 127.0.0.1 6379 #slaveof host 6379 | |
OK | |
127.0.0.1:6380> info replication | |
# Replication | |
role:slave | |
master_host:127.0.0.1 | |
master_port:6379 | |
master_link_status:up | |
master_last_io_seconds_ago:2 | |
master_sync_in_progress:0 | |
slave_repl_offset:42 | |
slave_priority:100 | |
slave_read_only:1 | |
connected_slaves:0 | |
master_replid:da314952dadd611849cb227b8706af7bcd8af486 | |
master_replid2:0000000000000000000000000000000000000000 | |
master_repl_offset:42 | |
second_repl_offset:-1 | |
repl_backlog_active:1 | |
repl_backlog_size:1048576 | |
repl_backlog_first_byte_offset:1 | |
repl_backlog_histlen:42 | |
127.0.0.1:6381> SLAVEOF 127.0.0.1 6379 #slaveof host 6379 | |
OK | |
127.0.0.1:6381> info replication | |
# Replication | |
role:slave | |
master_host:127.0.0.1 | |
master_port:6379 | |
master_link_status:up | |
master_last_io_seconds_ago:8 | |
master_sync_in_progress:0 | |
slave_repl_offset:224 | |
slave_priority:100 | |
slave_read_only:1 | |
connected_slaves:0 | |
master_replid:da314952dadd611849cb227b8706af7bcd8af486 | |
master_replid2:0000000000000000000000000000000000000000 | |
master_repl_offset:224 | |
second_repl_offset:-1 | |
repl_backlog_active:1 | |
repl_backlog_size:1048576 | |
repl_backlog_first_byte_offset:113 | |
repl_backlog_histlen:112 |
建议在配置文件中设置,用命令的话是暂时的 主机可写,从机可读
如果主机断开连接,从机依旧可以连接到主机,但是没有写的权限,一旦主机恢复,从机就可以再次读取主机信息。 如果是用命令行配置的从机,一旦断开连接就会变成主机,需要重新配置,但是在配置文件配置的就不会出现这种情况
主从复制(复制原理)
Slave启动成功连接到master后会发送一个sync同步命令 Master接到命令,启动后台的存盘进程,同时收集所有接收到的用于修改数据集命令,在后台进程执行完毕之后, master将传送整个数据文件到slave , 并完成一-次完全同步。 全量复制:而slave服务在接收到数据库文件数据后,将其存盘并加载到内存中。 增量复制: Master继续将新的所有收集到的修改命令依次传给slave ,完成同步 但是只要是重新连接master , 一次完全同步(全量复制)将被自动执行
层层链路
上一个M链接下一个S
这个时候如果主机(79)宕机,我们可以用SLAVEOF no one
80从机就会变成主机,如果79主机恢复,需要从新配置
哨兵模式
主从切换技术的方法是:当主服务器宕机后,需要手动把一台从服务器切换为主服务器,这就需要人工干预 ,费事费力,还会造成一段时间内服务不可用。这不是一种推荐的方式,更多时候,我们优先考虑哨兵模式。Redis从2.8开始正式提供 了Sentinel (哨兵)架构来解决这个问题。 谋朝篡位的自动版,能够后台监控主机是否故障,如果故障了根据投票数自动将从库转换为主库. 哨兵模式是一种特殊的模式,首先Redis提供了哨兵的命令,哨兵是一一个独立的进程 ,作为进程,它会独立运行。其原理是哨兵通过发送命令,等待Redis服务器响应,从而监控运行的多个Redis实例。
这里的哨兵有两个作用
- 通过发送命令,让Redis服务器返回监控其运行状态,包括主服务器和从服务器。
- 当哨兵监测到master宕机,会自动将slave切换成master ,然后通过发布订阅模式通知其他的从服务器,修改配置文件,让它们切换主机。 然而一个哨兵进程对Redis服务器进行监控,可能会出现问题,为此,我们可以使用多个哨兵进行监控。各个哨兵之间还会进行监控,这样就形成了多哨兵模式。
假设主服务器宕机,哨兵1先检测到这个结果,系统并不会马上进行failover过程,仅仅是哨兵1主观的认为主服务器不可用,这个现象成为主观下线。当后面的哨兵也检测到主服务器不可用,并且数量达到一定值时,那么哨兵之间就会进行一次投票 ,投票的结果由一个哨兵发起,进行failover[故障转移]操作。切换成功后,就会通过发布订阅模式,让各个哨兵把自己监控的从服务器实现切换主机,这个过程称为客观下线。
测试
配置哨兵配置文件 vim sentinel.conf
# 监听端口 | |
port 26379 | |
# 后台运行 | |
daemonize yes | |
#pid文件 | |
pidfile /var/run/redis-sentinel.pid | |
# 哨兵得日志文件 | |
logfile "sentinel.log" | |
# 日志文件存放路径 | |
dir /www/server/redis/ | |
# 设置初始master以及法定认为下线人数: | |
#sentinel monitor | |
sentinel monitor mymaster 192.168.0.10 6379 1 | |
# master主观下线时间,默认30秒,30秒内没有回复pong,则认为下线了 | |
#sentinel down-after-milliseconds | |
sentinel down-after-milliseconds mymaster 30000 | |
# 指定故障转移超时时间,默认为3分钟 | |
sentinel failover-timeout mymaster 180000 | |
# 设置通知脚本,发生故障转移可以向管理员发送通知(可选) | |
sentinel notification-script mymaster /www/server/redis/notify.sh | |
# 禁止修改脚本,避免脚本重置 | |
sentinel deny-scripts-reconfig yes |
启动哨兵模式
[root@cs bin]# redis-sentinel xhconfig/sentinel.conf
如果master节点断开了,这个时候就会从从机随机选一个从机当主机
如果主机回来了,只能归并到新的主机下,当作从机
优点: 1、哨兵集群,基于主从复制模式,所有的主从配置优点,它全有 2、主从可以切换,故障可以转移,系统的可用性就会更好 3、哨兵模式就是主从模式的升级,手动到自动,更加健壮! 缺点: 1、Redis 不好啊在线扩容的,集群容量一旦到达上限,在线扩容就十分麻烦! 2、实现哨兵模式的配置其实是很麻烦的,里面有很多选择!
哨兵全部配置
[root@cs bin]# cat /opt/redis-5.0.13/sentinel.conf | |
# Example sentinel.conf | |
# *** IMPORTANT *** | |
# | |
# By default Sentinel will not be reachable from interfaces different than | |
# localhost, either use the 'bind' directive to bind to a list of network | |
# interfaces, or disable protected mode with "protected-mode no" by | |
# adding it to this configuration file. | |
# | |
# Before doing that MAKE SURE the instance is protected from the outside | |
# world via firewalling or other means. | |
# | |
# For example you may use one of the following: | |
# | |
# bind 127.0.0.1 192.168.1.1 | |
# | |
# protected-mode no | |
# port | |
# The port that this sentinel instance will run on | |
port 26379 | |
# By default Redis Sentinel does not run as a daemon. Use 'yes' if you need it. | |
# Note that Redis will write a pid file in /var/run/redis-sentinel.pid when | |
# daemonized. | |
daemonize no | |
# When running daemonized, Redis Sentinel writes a pid file in | |
# /var/run/redis-sentinel.pid by default. You can specify a custom pid file | |
# location here. | |
pidfile /var/run/redis-sentinel.pid | |
# Specify the log file name. Also the empty string can be used to force | |
# Sentinel to log on the standard output. Note that if you use standard | |
# output for logging but daemonize, logs will be sent to /dev/null | |
logfile "" | |
# sentinel announce-ip | |
# sentinel announce-port | |
# | |
# The above two configuration directives are useful in environments where, | |
# because of NAT, Sentinel is reachable from outside via a non-local address. | |
# | |
# When announce-ip is provided, the Sentinel will claim the specified IP address | |
# in HELLO messages used to gossip its presence, instead of auto-detecting the | |
# local address as it usually does. | |
# | |
# Similarly when announce-port is provided and is valid and non-zero, Sentinel | |
# will announce the specified TCP port. | |
# | |
# The two options don't need to be used together, if only announce-ip is | |
# provided, the Sentinel will announce the specified IP and the server port | |
# as specified by the "port" option. If only announce-port is provided, the | |
# Sentinel will announce the auto-detected local IP and the specified port. | |
# | |
# Example: | |
# | |
# sentinel announce-ip 1.2.3.4 | |
# dir | |
# Every long running process should have a well-defined working directory. | |
# For Redis Sentinel to chdir to /tmp at startup is the simplest thing | |
# for the process to don't interfere with administrative tasks such as | |
# unmounting filesystems. | |
dir /tmp | |
# sentinel monitor | |
# | |
# Tells Sentinel to monitor this master, and to consider it in O_DOWN | |
# (Objectively Down) state only if at least sentinels agree. | |
# | |
# Note that whatever is the ODOWN quorum, a Sentinel will require to | |
# be elected by the majority of the known Sentinels in order to | |
# start a failover, so no failover can be performed in minority. | |
# | |
# Replicas are auto-discovered, so you don't need to specify replicas in | |
# any way. Sentinel itself will rewrite this configuration file adding | |
# the replicas using additional configuration options. | |
# Also note that the configuration file is rewritten when a | |
# replica is promoted to master. | |
# | |
# Note: master name should not include special characters or spaces. | |
# The valid charset is A-z 0-9 and the three characters ".-_". | |
sentinel monitor mymaster 127.0.0.1 6379 2 | |
# sentinel auth-pass | |
# | |
# Set the password to use to authenticate with the master and replicas. | |
# Useful if there is a password set in the Redis instances to monitor. | |
# | |
# Note that the master password is also used for replicas, so it is not | |
# possible to set a different password in masters and replicas instances | |
# if you want to be able to monitor these instances with Sentinel. | |
# | |
# However you can have Redis instances without the authentication enabled | |
# mixed with Redis instances requiring the authentication (as long as the | |
# password set is the same for all the instances requiring the password) as | |
# the AUTH command will have no effect in Redis instances with authentication | |
# switched off. | |
# | |
# Example: | |
# | |
# sentinel auth-pass mymaster MySUPER--secret-0123passw0rd | |
# sentinel down-after-milliseconds | |
# | |
# Number of milliseconds the master (or any attached replica or sentinel) should | |
# be unreachable (as in, not acceptable reply to PING, continuously, for the | |
# specified period) in order to consider it in S_DOWN state (Subjectively | |
# Down). | |
# | |
# Default is 30 seconds. | |
sentinel down-after-milliseconds mymaster 30000 | |
# sentinel parallel-syncs | |
# | |
# How many replicas we can reconfigure to point to the new replica simultaneously | |
# during the failover. Use a low number if you use the replicas to serve query | |
# to avoid that all the replicas will be unreachable at about the same | |
# time while performing the synchronization with the master. | |
sentinel parallel-syncs mymaster 1 | |
# sentinel failover-timeout | |
# | |
# Specifies the failover timeout in milliseconds. It is used in many ways: | |
# | |
# - The time needed to re-start a failover after a previous failover was | |
# already tried against the same master by a given Sentinel, is two | |
# times the failover timeout. | |
# | |
# - The time needed for a replica replicating to a wrong master according | |
# to a Sentinel current configuration, to be forced to replicate | |
# with the right master, is exactly the failover timeout (counting since | |
# the moment a Sentinel detected the misconfiguration). | |
# | |
# - The time needed to cancel a failover that is already in progress but | |
# did not produced any configuration change (SLAVEOF NO ONE yet not | |
# acknowledged by the promoted replica). | |
# | |
# - The maximum time a failover in progress waits for all the replicas to be | |
# reconfigured as replicas of the new master. However even after this time | |
# the replicas will be reconfigured by the Sentinels anyway, but not with | |
# the exact parallel-syncs progression as specified. | |
# | |
# Default is 3 minutes. | |
sentinel failover-timeout mymaster 180000 | |
# SCRIPTS EXECUTION | |
# | |
# sentinel notification-script and sentinel reconfig-script are used in order | |
# to configure scripts that are called to notify the system administrator | |
# or to reconfigure clients after a failover. The scripts are executed | |
# with the following rules for error handling: | |
# | |
# If script exits with "1" the execution is retried later (up to a maximum | |
# number of times currently set to 10). | |
# | |
# If script exits with "2" (or an higher value) the script execution is | |
# not retried. | |
# | |
# If script terminates because it receives a signal the behavior is the same | |
# as exit code 1. | |
# | |
# A script has a maximum running time of 60 seconds. After this limit is | |
# reached the script is terminated with a SIGKILL and the execution retried. | |
# NOTIFICATION SCRIPT | |
# | |
# sentinel notification-script | |
# | |
# Call the specified notification script for any sentinel event that is | |
# generated in the WARNING level (for instance -sdown, -odown, and so forth). | |
# This script should notify the system administrator via email, SMS, or any | |
# other messaging system, that there is something wrong with the monitored | |
# Redis systems. | |
# | |
# The script is called with just two arguments: the first is the event type | |
# and the second the event description. | |
# | |
# The script must exist and be executable in order for sentinel to start if | |
# this option is provided. | |
# | |
# Example: | |
# | |
# sentinel notification-script mymaster /var/redis/notify.sh | |
# CLIENTS RECONFIGURATION SCRIPT | |
# | |
# sentinel client-reconfig-script | |
# | |
# When the master changed because of a failover a script can be called in | |
# order to perform application-specific tasks to notify the clients that the | |
# configuration has changed and the master is at a different address. | |
# | |
# The following arguments are passed to the script: | |
# | |
# | |
# | |
# is currently always "failover" | |
# is either "leader" or "observer" | |
# | |
# The arguments from-ip, from-port, to-ip, to-port are used to communicate | |
# the old address of the master and the new address of the elected replica | |
# (now a master). | |
# | |
# This script should be resistant to multiple invocations. | |
# | |
# Example: | |
# | |
# sentinel client-reconfig-script mymaster /var/redis/reconfig.sh | |
# SECURITY | |
# | |
# By default SENTINEL SET will not be able to change the notification-script | |
# and client-reconfig-script at runtime. This avoids a trivial security issue | |
# where clients can set the script to anything and trigger a failover in order | |
# to get the program executed. | |
sentinel deny-scripts-reconfig yes | |
# REDIS COMMANDS RENAMING | |
# | |
# Sometimes the Redis server has certain commands, that are needed for Sentinel | |
# to work correctly, renamed to unguessable strings. This is often the case | |
# of CONFIG and SLAVEOF in the context of providers that provide Redis as | |
# a service, and don't want the customers to reconfigure the instances outside | |
# of the administration console. | |
# | |
# In such case it is possible to tell Sentinel to use different command names | |
# instead of the normal ones. For example if the master "mymaster", and the | |
# associated replicas, have "CONFIG" all renamed to "GUESSME", I could use: | |
# | |
# SENTINEL rename-command mymaster CONFIG GUESSME | |
# | |
# After such configuration is set, every time Sentinel would use CONFIG it will | |
# use GUESSME instead. Note that there is no actual need to respect the command | |
# case, so writing "config guessme" is the same in the example above. | |
# | |
# SENTINEL SET can also be used in order to perform this configuration at runtime. | |
# | |
# In order to set a command back to its original name (undo the renaming), it | |
# is possible to just rename a command to itsef: | |
# | |
# SENTINEL rename-command mymaster CONFIG CONFIG |