XtraBackup是由Percona提供的开源备份软件。它能在不关闭服务器的情况下复制普通文件。但为了避免不一致,它会使用redo日志文件。XtraBackup被许多公司广泛用做标准备份工具。与逻辑备份工具相比,其优势是备份速度非常快,恢复速度也非常快。
Percona XtraBackup的工作原理:
1.XtraBackup复制InnoDB数据文件,这会导致内部不一致的数据,但是它会对文件执行崩溃恢复,以使其再次成为一个一致的可用数据库
2.这样做的可行性是因为InnoDB维护一个REDO日志,也称为事务日志。REDO日志包含了InnoDB数据每次更改的记录。当InnoDB启动时,REDO日志会检查数据文件和事务日志,并执行两个步骤。它将已提交的事务日志条目应用于数据文件,并对任何修改了数据但未提交的事务执行undo操作
3.Percona XtraBackup会在启动时记住日志序列号(LSN),然后复制数据文件。这需要一些时间来完成,如果文件正在改变,那么它会在不同的时间点反映数据库的状态。同时,Percona XtraBackup运行一个后台进程,用于监视事务日志文件,并从中复制更改。Percona XtraBackup需要持续这样做,因为事务日志是以循环方式写入的,并且可以在一段时间后重新使用。Percona XtraBackup开始执行后,需要复制每次数据文件更改对应的事务日志记录。
安装
--下载rpm包
[root@mysql ~]#
wget https://www.percona.com/downloads/Percona-XtraBackup-LATEST/Percona-XtraBackup-8.0-6/binary/redhat/6/x86_64/percona-xtrabackup-80-8.0.6-1.el6.x86_64.rpm
--yum安装rpm包,提示缺少
libev.so.4()(64bit)
[root@mysql ~]#
yum localinstall percona-xtrabackup-80-8.0.6-1.el6.x86_64.rpm
Loaded plugins: fastestmirror, security
Setting up Local Package Process
Examining percona-xtrabackup-80-8.0.6-1.el6.x86_64.rpm: percona-xtrabackup-80-8.0.6-1.el6.x86_64
Marking percona-xtrabackup-80-8.0.6-1.el6.x86_64.rpm to be installed
....省略...
--> Finished Dependency Resolution
Error: Package: percona-xtrabackup-80-8.0.6-1.el6.x86_64 (/percona-xtrabackup-80-8.0.6-1.el6.x86_64)
Requires: libev.so.4()(64bit)
You could try using --skip-broken to work around the problem
** Found 3 pre-existing rpmdb problem(s), 'yum check' output follows:
2:postfix-2.6.6-2.2.el6_1.x86_64 has missing requires of libmysqlclient.so.16()(64bit)
2:postfix-2.6.6-2.2.el6_1.x86_64 has missing requires of libmysqlclient.so.16(libmysqlclient_16)(64bit)
2:postfix-2.6.6-2.2.el6_1.x86_64 has missing requires of mysql-libs
--下载安装
libev.so.4()(64bit)
去如下网站查找对应系统的包
http://rpmfind.net/linux/rpm2html/search.php?query=libev.so.4%28%29%2864bit%29&submit=Search+...&system=&arch=
--下载并rpm安装
[root@oracle ~]#
wget http://rpmfind.net/linux/epel/6/x86_64/Packages/l/libev-4.03-3.el6.x86_64.rpm
[root@oracle ~]#
rpm -ivh libev-4.03-3.el6.x86_64.rpm
Preparing... ########################################### [100%]
1:libev ########################################### [100%]
--重新安装
percona-xtrabackup
[root@oracle ~]# yum localinstall percona-xtrabackup-80-8.0.6-1.el6.x86_64.rpm
Loaded plugins: fastestmirror, security
Setting up Local Package Process
Examining percona-xtrabackup-80-8.0.6-1.el6.x86_64.rpm: percona-xtrabackup-80-8.0.6-1.el6.x86_64
....省略...
Complete!
权限
如果需要创建专门的用户去执行备份恢复操作,需要选项如下:
mysql> CREATE USER 'bkpuser'@'localhost' IDENTIFIED BY 's3cr%T';
mysql> GRANT
BACKUP_ADMIN, SUPER, PROCESS,
CREATE TABLESPACE, RELOAD, LOCK TABLES, REPLICATION CLIENT
ON *.* TO 'bkpuser'@'localhost';
mysql> GRANT
SELECT, INSERT, CREATE
ON performance_schema.* TO 'bkpuser'@'localhost';
mysql> FLUSH PRIVILEGES;
--如下是官网对每个参数含义的解释
•
RELOAD
and
LOCK TABLES
(unless the --no-lock option is specified) in order to run FLUSH TABLES WITH READ LOCK and FLUSH ENGINE LOGS prior to start copying the files, and requires this privilege when Backup Locks are used
•
BACKUP_ADMIN
privilege is needed to query the performance_schema.log_status table, and run LOCK INSTANCE FOR BACKUP, LOCK BINLOG FOR BACKUP, or LOCK TABLES FOR BACKUP
•
REPLICATION CLIENT
in order to obtain the binary log position
•
CREATE TABLESPACE
in order to import tables (see Restoring Individual Tables)
•
PROCESS
in order to run SHOW ENGINE INNODB STATUS (which is mandatory), and optionally to see all threads which are running on the server (see Handling FLUSH TABLES WITH READ LOCK)
•
SUPER
in order to start/stop the slave threads in a replication environment, use XtraDB Changed Page Tracking for Incremental Backups and for handling FLUSH TABLES WITH READ LOCK
•
CREATE
privilege in order to create the PERCONA_SCHEMA.xtrabackup_history database and table
•
INSERT
privilege in order to add history records to the PERCONA_SCHEMA.xtrabackup_history table
•
SELECT
privilege in order to use --incremental-history-name or --incremental-history-uuid in order for the feature to look up the innodb_to_lsn values in the PERCONA_SCHEMA.xtrabackup_history table
1.全量备份和恢复
全量备份
在XtraBackup8概述
移除了innobackupex命令。
由于新的MySQL重做日志和数据字典格式,8.0版本只支持mysql8.0和percona8.0。
早于mysql8.0的版本需要使用xtrabackup2.4备份和恢复
--备份命令
[root@oracle bin]#
xtrabackup --user=root --backup --socket=/tmp/mysql.sock --target-dir=/xtrabackup
...省略...
190802 15:52:55 All tables unlocked
190802 15:52:55 [00] Copying ib_buffer_pool to /xtrabackup/ib_buffer_pool
190802 15:52:55 [00] ...done
190802 15:52:55 Backup created in directory '/xtrabackup/'
MySQL binlog position: filename 'binlog.000011', position '155'
190802 15:52:55 [00] Writing /xtrabackup/backup-my.cnf
190802 15:52:55 [00] ...done
190802 15:52:55 [00] Writing /xtrabackup/xtrabackup_info
190802 15:52:55 [00] ...done
xtrabackup: Transaction log of lsn (20323274) to (20323294) was copied.
190802 15:52:55 completed OK!
--查看备份
[root@oracle bin]#
ll /xtrabackup/
total 56372
-rw-r----- 1 root root 476 Aug 2 15:52 backup-my.cnf
-rw-r----- 1 root root 155 Aug 2 15:52 binlog.000011
-rw-r----- 1 root root 16 Aug 2 15:52 binlog.index
drwxr-x--- 2 root root 4096 Aug 2 15:52 company
-rw-r----- 1 root root 3329 Aug 2 15:52 ib_buffer_pool
-rw-r----- 1 root root 12582912 Aug 2 15:52 ibdata1
drwxr-x--- 2 root root 4096 Aug 2 15:52 mysql
-rw-r----- 1 root root 24117248 Aug 2 15:52 mysql.ibd
drwxr-x--- 2 root root 4096 Aug 2 15:52 performance_schema
drwxr-x--- 2 root root 4096 Aug 2 15:52 sys
-rw-r----- 1 root root 10485760 Aug 2 15:52 undo_001
-rw-r----- 1 root root 10485760 Aug 2 15:52 undo_002
-rw-r----- 1 root root 18 Aug 2 15:52 xtrabackup_binlog_info
-rw-r----- 1 root root 95 Aug 2 15:52 xtrabackup_checkpoints
-rw-r----- 1 root root 479 Aug 2 15:52 xtrabackup_info
-rw-r----- 1 root root 2560 Aug 2 15:52 xtrabackup_logfile
-rw-r----- 1 root root 262 Aug 2 15:52 xtrabackup_tablespaces
备份时间长短根据数据库大小有关,在备份的期间可以随时取消,因为xtrabackup命令不会对数据库进行操作
全量恢复
--停止数据库
[root@oracle data]#
service mysqld stop
Shutting down MySQL.. SUCCESS!
--删除现有的data目录
[root@oracle data]# rm -rf /usr/local/mysql/data/*
--执行恢复命令
[root@oracle data]#
xtrabackup --prepare --target-dir=/xtrabackup/
[root@oracle data]#
xtrabackup --copy-back --target-dir=/xtrabackup/
xtrabackup: recognized server arguments: --datadir=/usr/local/mysql/data --server-id=1
xtrabackup: recognized client arguments: --copy-back=1 --target-dir=/xtrabackup/
xtrabackup version 8.0.6 based on MySQL server 8.0.14 Linux (x86_64) (revision id: c0a2d91)
190802 16:33:39 [01] Copying undo_001 to /usr/local/mysql/data/undo_001
...省略...
190802 16:33:41 [01] ...done
190802 16:33:41 completed OK!
--对文件授权
chwon -R mysql.mysql /usr/local/mysql/data/*
Tips:
--copy-back命令表示将备份
复制
到datadir目录下,如果不想保留备份,可以使用--move-back命令,直接将备份
移动
到datadir目录下
2.增量备份
在进行增量备份之前,通过先进行一次全量备份。XtraBackup通过二进制方式在备份目录下写入xtrabackup_checkpoints文件。该文件其中一行会显示to_lsn,
该参数记录了数据库备份完成的LSN。全量备份命令:
方式一:
基于base的备份
[root@oracle xtrabackup]# xtrabackup --user=root --backup --socket=/tmp/mysql.sock --target-dir=/xtrabackup/base
xtrabackup_checkpoints的内容:
[root@oracle xtrabackup]# cat /xtrabackup/base/xtrabackup_checkpoints
backup_type =
full-backuped
from_lsn = 0
to_lsn =
19957274
last_lsn = 19957284
flushed_lsn = 0
在进行了全量备份后
,我们可以通过
增量备份
的命令进行备份:
基于base的incr1备份
[root@oracle xtrabackup]# xtrabackup --user=root --backup --socket=/tmp/mysql.sock --target-dir=/xtrabackup/incr1 --incremental-basedir=
/xtrabackup/base
xtrabackup_checkpoints的内容:
[root@oracle xtrabackup]# cat /xtrabackup/incr1/xtrabackup_checkpoints
backup_type =
incremental
from_lsn =
19957274
to_lsn = 19957304
last_lsn = 19957314
flushed_lsn = 0
基于incr1的incr2备份
[root@oracle xtrabackup]# xtrabackup --user=root --backup --socket=/tmp/mysql.sock --target-dir=/xtrabackup/incr2 --incremental-basedir=
/xtrabackup/incr1/
xtrabackup_checkpoints的内容:
[root@oracle xtrabackup]# cat /xtrabackup/incr2/xtrabackup_checkpoints
backup_type =
incremental
from_lsn =
19957304
to_lsn = 19957364
last_lsn = 19957374
flushed_lsn = 0
结论:
假设周一是基于base的备份,周二是基于base的incr1备份,周三是基于incr1的incr2备份,在恢复数据库的时候,需要使用base,incr1,incr2三个备份都存在时,才能进行完整的恢复,每个备份的from_lsn都是基于上一个备份的to_lsn,所以缺一不可。
方式二:
基于base的备份
[root@oracle xtrabackup]# xtrabackup --user=root --backup --socket=/tmp/mysql.sock --target-dir=/xtrabackup/base
xtrabackup_checkpoints的内容:
[root@oracle xtrabackup]# cat /xtrabackup/base/xtrabackup_checkpoints
backup_type =
full-backuped
from_lsn = 0
to_lsn =
19957274
last_lsn = 19957284
flushed_lsn = 0
基于base的incr1备份
[root@oracle xtrabackup]# xtrabackup --user=root --backup --socket=/tmp/mysql.sock --target-dir=/xtrabackup/incr1 --incremental-basedir=
/xtrabackup/base
xtrabackup_checkpoints的内容:
[root@oracle xtrabackup]# cat /xtrabackup/incr1/xtrabackup_checkpoints
backup_type =
incremental
from_lsn =
19957274
to_lsn = 19957304
last_lsn = 19957314
flushed_lsn = 0
基于base的incr2备份
[root@oracle xtrabackup]# xtrabackup --user=root --backup --socket=/tmp/mysql.sock --target-dir=/xtrabackup/incr2 --incremental-basedir=
/xtrabackup/base/
xtrabackup_checkpoints的内容:
[root@oracle xtrabackup]# cat /xtrabackup/incr2/xtrabackup_checkpoints
backup_type =
incremental
from_lsn =
19957274
to_lsn = 19957394
last_lsn = 19957404
flushed_lsn = 0
结论:假设周一是基于base的备份,周二是基于base的incr1备份,周三是基于base的incr2备份,在恢复数据库的时候,需要使用base和incr1,incr2两个备份中的其中一个,才能进行完整的恢复,因为incr1和incr2的from_lsn都是基于base备份中的to_lsn,所以恢复数据库时,只需要base和任意一个基于base的增量备份。
3.增量备份恢复
增量备份和全量备份的--prepare执行的方式是不一样的。在全量备份中,可以通过两种操作保持数据库的一致性:已提交的事务将根据数据文件和日志文件进行重放操作,并回滚未提交的事务。在准备增量备份时,必须跳过未提交事务的回滚,因为在备份的过程中,可能存在进行中且未提交的事务,并且这些事务很可能在下一次的增量备份中才进行提交,所以必须使用--apply-log-only选项来防止回滚操作。
基于方式一的恢复:
3.1准备好备份片
[root@oracle xtrabackup]# ll
total 20
drwxr-x--- 6 root root 4096 Aug 5 08:30 base
drwxr-x--- 6 root root 4096 Aug 5 08:32 incr1
drwxr-x--- 6 root root 4096 Aug 5 08:52 incr2
3.2执行恢复命令
[root@oracle xtrabackup]# xtrabackup --prepare --apply-log-only --target-dir=/xtrabackup/base
[root@oracle xtrabackup]# xtrabackup --prepare --apply-log-only --target-dir=/xtrabackup/base --incremental-dir=/xtrabackup/incr1
[root@oracle xtrabackup]# xtrabackup --prepare --target-dir=/xtrabackup/base --incremental-dir=/xtrabackup/incr2
[root@oracle xtrabackup]# xtrabackup --copy-back --target-dir=/xtrabackup/base/
3.3对data目录授权
[root@oracle mysql] chown -R mysql.mysql data/
基于方式二的恢复:
3.1准备好备份片
[root@oracle xtrabackup]# ll
total 20
drwxr-x--- 6 root root 4096 Aug 5 08:30 base
drwxr-x--- 6 root root 4096 Aug 5 08:32 incr1
drwxr-x--- 6 root root 4096 Aug 5 08:52 incr2
3.2执行恢复命令
[root@oracle xtrabackup]# xtrabackup --prepare --apply-log-only --target-dir=/xtrabackup/base
[root@oracle xtrabackup]# xtrabackup --prepare --target-dir=/xtrabackup/base --incremental-dir=/xtrabackup/incr2
[root@oracle xtrabackup]# xtrabackup --copy-back --target-dir=/xtrabackup/base/
3.3对data目录授权
[root@oracle mysql] chown -R mysql.mysql data/
Tips:
--apply-log-only命令应该用在所有增量备份(除最后一次增量备份)
,这就是为什么恢复脚本中,最后一次的命令不包含--apply-log-only。即使--apply-log-only在最后一次增量备份时被使用,备份仍将是一致的,但在这种情况下,数据库会执行回滚的操作。
4.压缩备份
[root@oracle xtrabackup]# xtrabackup --user=root --backup
--compress
--socket=/tmp/mysql.sock --target-dir=/xtrabackup/compress
压缩和未压缩的对比
[root@oracle xtrabackup]# du -h --max-depth=1
57M ./base
3.3M ./compress
压缩后的文件列表
[root@oracle compress]# ll
total 2876
-rw-r----- 1 root root 453 Aug 5 09:02 backup-my.cnf.qp
-rw-r----- 1 root root 183 Aug 5 09:02 binlog.000018.qp
-rw-r----- 1 root root 93 Aug 5 09:02 binlog.index.qp
drwxr-x--- 2 root root 4096 Aug 5 09:02 company
-rw-r----- 1 root root 912 Aug 5 09:02 ib_buffer_pool.qp
-rw-r----- 1 root root 253886 Aug 5 09:02 ibdata1.qp
drwxr-x--- 2 root root 4096 Aug 5 09:02 mysql
-rw-r----- 1 root root 2191005 Aug 5 09:02 mysql.ibd.qp
drwxr-x--- 2 root root 4096 Aug 5 09:02 performance_schema
drwxr-x--- 2 root root 4096 Aug 5 09:02 sys
-rw-r----- 1 root root 224626 Aug 5 09:02 undo_001.qp
-rw-r----- 1 root root 220474 Aug 5 09:02 undo_002.qp
-rw-r----- 1 root root 105 Aug 5 09:02 xtrabackup_binlog_info.qp
-rw-r----- 1 root root 95 Aug 5 09:02 xtrabackup_checkpoints
-rw-r----- 1 root root 471 Aug 5 09:02 xtrabackup_info.qp
-rw-r----- 1 root root 333 Aug 5 09:02 xtrabackup_logfile.qp
-rw-r----- 1 root root 234 Aug 5 09:02 xtrabackup_tablespaces.qp
如果想要加速备份的速度,可以采用--compress-threads命令
[root@oracle xtrabackup]# xtrabackup --user=root --backup
--compress
--compress-threads=4
--socket=/tmp/mysql.sock --target-dir=/xtrabackup/compress
解压缩
[root@oracle xtrabackup]# xtrabackup
--decompress
--target-dir=/xtrabackup/compress
报错:
xtrabackup: recognized server arguments: --datadir=/usr/local/mysql/data --server-id=1
xtrabackup: recognized client arguments: --decompress=1 --target-dir=/xtrabackup/compress
xtrabackup version 8.0.6 based on MySQL server 8.0.14 Linux (x86_64) (revision id: c0a2d91)
190805 09:07:23 [01] decompressing ./ibdata1.qp
sh: qpress: command not found
cat: write error: Broken pipe
Error: decrypt and decompress thread 0 failed.
