OGG 12c mysql 复制到 oracle 部署方案
1、 环境 源端数据库: mysql 5.7.31 目标端数据库: oracle 11.2.0.4 源端 ogg 介质: 122022_ggs_Linux_x64_MySQL_64bit 目标端 ogg 介质: 122022_fbo_ggs_Linux_x64_shiphome 注意: ( 1 )这里使用 ogg 12.2.0.2 的版本,如果使用低版本,抽取进程启动会报 600 错误 ( 2 )两端使用的版本尽量一致,不然使用直接加载的初始化方式在启动的时候会报版本错误,且直接加载方式的初始化进程不支持 FORMAT RELEASE(LEVEL) 配置 Mysql 端 122022_ggs_Linux_x64_MySQL_64bit 解压后即可使用 Oracle 端 122022_fbo_ggs_Linux_x64_shiphome 修改模板,使用静默安装 ( 3 )虽然 mysql 端设置了表名大小写不敏感的参数 lower_case_table_names=1 ,但是 ogg 对表名大小写依然敏感。 2 、源端配置 2.1 Mysql 参数权限
与 ogg 相关的 Mysql 数据库参数配置
binlog-format=ROW
log-bin=mysql-bin
需要同步的表要先授权
>GRANT INSERT, UPDATE, DELETE,select ON `psd`.`test` TO 'ogg'@'%'; 2.2 管理进程: mgr
> view param mgr PORT 1739 DYNAMICPORTLIST 1740-1838 AUTORESTART EXTRACT *,RETRIES 999,WAITMINUTES 3 STARTUPVALIDATIONDELAY 5 PURGEOLDEXTRACTS ./dirdat/*, USECHECKPOINTS, MINKEEPDAYS 3 LAGREPORTHOURS 1 LAGINFOMINUTES 30 LAGCRITICALMINUTES 45
2.3 抽取进程: extpr
> view param extpr extract extpr setenv(MYSQL_HOME="/usr/") sourcedb ogg@196.14.1.1:13306,userid ogg,password ogg tranlogoptions altlogdest /mysql/mdata/mysql-bin.index discardfile ./dirrpt/extbpm.dsc,append,megabytes 1024 exttrail ./dirdat/bp gettruncates dynamicresolution table psd.test,keycols(ID);
MYSQL_HOME 可以通过执行 show variables like '%basedir%' 来确定
tranlogoptions altlogdest 配置的是 log-bin 的 index 文件,可以通过查看 my.cnf 配置文件确定,也可以执行 show variables like '%datadir%' 来确定
2.4 传输进程 :dpepr
> view param dpepr extract dpepr sourcedb ogg@196.14.1.1:13306,userid ogg,password ogg rmthost 196.14.1.2,mgrport 1639 rmttrail ./dirdat/bp gettruncates table psd.test,keycols(ID);
2.5 源端初始化进程 :extinit
> view param extinit extract extinit setenv (NLS_LANG=AMERICAN_AMERICA.AL32UTF8) sourcedb ogg@196.14.1.1:13306,userid ogg,password ogg rmthost 196.14.1.2,mgrport 1639 RMTTASK REPLICAT,GROUP repinit ---对应的目标端初始化进程名 table psd.test,keycols(ID);
2.6 生成 def 文件
在 ogg 主目录新建一个 repdef.prm 文件
>vi repdef.prm defsfile ./dirdef/repdef.def,purge sourcedb ogg@196.14.1.1:13306,userid ogg,password ogg table psd.test;
生成 def 文件
>./defgen paramfile repdef.prm 3 、目标端配置 3.1 静默安装 ogg 3.1.1 修改属主
> chown oracle:dba fbo_ggs_Linux_x64_shiphome/ -R 3.1.2 修改安装模板文件
fbo_ggs_Linux_x64_shiphome/Disk1/response/oggcore.rsp 中的以下两个参数,根据实际情况配置,其他的参数不要动。 INSTALL_OPTION=ORA11g
SOFTWARE_LOCATION=/ogg/dogg 3.1.3 执行安装
> ./runInstaller -silent -responseFile /dogg/fbo_ggs_Linux_x64_shiphome/Disk1/response/oggcore.rsp Starting Oracle Universal Installer... Checking Temp space: must be greater than 120 MB. Actual 40776 MB Passed Checking swap space: must be greater than 150 MB. Actual 20225 MB Passed Preparing to launch Oracle Universal Installer from /tmp/OraInstall2022-03-13_02-56-14PM. Please wait ...oracle@yxchdb03:/ogg/odpogg/fbo_ggs_Linux_x64_shiphome/Disk1$ [WARNING] [INS-75003] The specified directory /ogg/odpogg is not empty. CAUSE: The directory specified /ogg/odpogg contains files. ACTION: Clean up the specified directory or enter a new directory location. [WARNING] [INS-75017] Cluster detected: Unable to determine if the specified software location is shared. It is recommended to install Oracle GoldenGate entirely on shared storage. CAUSE: One or more cluster nodes may be unreachable or the specified software location may not be accessible from one or more nodes. ACTION: Specify the software location on a shared storage. You can find the log of this install session at: /u01/app/oraInventory/logs/installActions2022-03-13_02-56-14PM.log WARNING:OUI-10030:You have specified a non-empty directory to install this product. It is recommended to specify either an empty or a non-existent directory. You may, however, choose to ignore this message if the directory contains Operating System generated files or subdirectories like lost+found. Do you want to proceed with installation in this Oracle Home? The installation of Oracle GoldenGate Core was successful. Please check '/u01/app/oraInventory/logs/silentInstall2022-03-13_02-56-14PM.log' for more details. Successfully Setup Software.
3.1.4 创建目录
oracle@yxdb:/ogg/dogg$ ./ggsci Oracle GoldenGate Command Interpreter for Oracle Version 12.2.0.2.2 OGGCORE_12.2.0.2.0_PLATFORMS_170630.0419_FBO Linux, x64, 64bit (optimized), Oracle 11g on Jun 30 2017 14:42:26 Operating system character set identified as UTF-8. Copyright (C) 1995, 2017, Oracle and/or its affiliates. All rights reserved. > create subdirs Creating subdirectories under current directory /ogg/odpogg Parameter files /ogg/dogg/dirprm: created Report files /ogg/dogg/dirrpt: created Checkpoint files /ogg/dogg/dirchk: created Process status files /ogg/dogg/dirpcs: created SQL script files /ogg/dogg/dirsql: created Database definitions files /ogg/dogg/dirdef: created Extract data files /ogg/dogg/dirdat: created Temporary files /ogg/dogg/dirtmp: created Credential store files /ogg/dogg/dircrd: created Masterkey wallet files /ogg/dogg/dirwlt: created Dump files /ogg/dogg/dirdmp: created
3.1 入库进程 reppr
> view param reppr REPLICAT reppr SETENV(ORACLE_SID = "fjdb") SETENV (ORACLE_HOME = "/u01/app/oracle/product/11.2/db_1") setenv (NLS_LANG=AMERICAN_AMERICA.UTF8) USERID ogg, PASSWORD ogg REPERROR DEFAULT,DISCARD SOURCEDEFS /ogg/dirdef/repdef.def DISCARDFILE ./dirrpt/reppr.dsc, APPEND, MEGABYTES 1024 DYNAMICRESOLUTION map psd.test,target bpm.test,KEYCOLS(ID);
3.2 目标端初始化进程 repinit
> view param repinit REPLICAT repinit USERID ogg, PASSWORD ogg SOURCEDEFS ./dirdef/reppr.def REPERROR(-1,IGNORE) REPERROR DEFAULT,DISCARD map psd.test,target bpm.test,KEYCOLS(ID);
4、 添加并启动进程 4.1 添加进程 4.1.1 源端
GGSCI (yxdb) 16> add extract extpr,tranlog,begin now GGSCI (yxdb) 17> add exttrail ./dirdat/bp,extract extpr,MEGABYTES 512 GGSCI (yxdb) 18> add extract dpepr,exttrailsource ./dirdat/bp GGSCI (yxdb) 19> add rmttrail ./dirdat/bp,extract dpepr,MEGABYTES 512 GGSCI (yxdb) 20> add extract extinit,sourceistable EXTRACT added.
4.1.2 目标端
>add replicat reppr,exttrail ./dirdat/bp,checkpointtable ogg.checkpointtab >add replicat repinit,specialrun
4.2 启动进程 4.2.1 源端
启动初始化进程
GGSCI (yxdb) 16>start extinit
执行该命令后后,目标端会自动启动初始化复制进程repinit,初始化完成后,会自动停止,在目标端执行info *,task可以查看初始化进程信息。
源端日志: 2022-03-15 08:53:37 INFO OGG-02911 Oracle GoldenGate Capture for MySQL, extinit.prm: Processing table psd.tb1. 2022-03-15 08:53:41 INFO OGG-02911 Oracle GoldenGate Capture for MySQL, extinit.prm: Processing table psd.tb2. 2022-03-15 08:53:49 INFO OGG-02911 Oracle GoldenGate Capture for MySQL, extinit.prm: Processing table psd.tb3. 目标端日志: 2022-03-15 08:53:37 INFO OGG-00996 Oracle GoldenGate Delivery for Oracle, repinit.prm: REPLICAT REPINIT started. 2022-03-15 08:53:37 WARNING OGG-02761 Oracle GoldenGate Delivery for Oracle, repinit.prm: Source definitions file, ./dirdef/repbpm.def, is ignored because trail file contains table definitions. 2022-03-15 08:53:37 INFO OGG-06505 Oracle GoldenGate Delivery for Oracle, repinit.prm: MAP resolved (entry psd_bpm.bpm_worklist_cur): map "psd"."tb1",target bpm.tb1,KEYCOLS(ID). 2022-03-15 08:53:37 INFO OGG-02756 Oracle GoldenGate Delivery for Oracle, repinit.prm: The definition for table psd.tb1 is obtained from the trail file. 2022-03-15 08:53:37 INFO OGG-06511 Oracle GoldenGate Delivery for Oracle, repinit.prm: Using following columns in default map by name: INSTANCE_ID, INSTANCE_NAME, TASK_ID, FROM_TASK_ID, NODE_ID, NODE_NAME, OVERDUR_DATE, REMARK, OVER_REASON, HANDLE_CONTENT,TENANT_ID. 2022-03-15 08:53:37 INFO OGG-06510 Oracle GoldenGate Delivery for Oracle, repinit.prm: Using the following key columns for target table BPM.tb1: ID.
待初始化完成后,启动抽取和传输进程
GGSCI (yxdb) 16>start extract extpr
GGSCI (yxdb) 16>start extract dpepr 4.2.2 目标端
GGSCI (yxdb) 16>start replicat reppr
观察日志,数据验证 5 、问题解决
Q1 :启动抽取进程报 error 600 2022-03-11 10:39:01 ERROR OGG-00146 VAM function VAMRead returned unexpected result: error 600 - VAM Client Report <CAUSE OF FAILURE : Sanity Check Failed for events WHEN FAILED : While reading log event from binary log WHERE FAILED : MySQLBinLog Reader Module CONTEXT OF FAILURE : No Information Available!>. 解决方法: 源端 mysql 使用 OGG-11.2.0.1 ,启动抽取进程会报上面的错误,需要升级到 12.2.0.2 以上版本。
Q2 :入库进程启动报 ERROR OGG-05302 2022-03-14 08:29:56 INFO OGG-03506 Oracle GoldenGate Delivery for Oracle, repbpm.prm: The source database character set, as determined from the trail file, is UTF-8. 2022-03-14 08:29:56 WARNING OGG-01003 Oracle GoldenGate Delivery for Oracle, repbpm.prm: Repositioning to rba 1853080 in seqno 6. 2022-03-14 08:29:56 ERROR OGG-05302 Oracle GoldenGate Delivery for Oracle, repbpm.prm: An error occured while doing commit handling. 2022-03-14 08:29:56 ERROR OGG-01668 Oracle GoldenGate Delivery for Oracle, repbpm.prm: PROCESS ABENDING. 解决方法: 更新检查点信息,启动进程(建议重新初始化) >dblogin userid username,password pwd >upgrade checkpointtable ogg.checkpointtab
Q3 :抽取进程无数据生成 抽取进程启动后不报错, trail 文件大小不增长 解决方法: 表名大小写问题,修改进程中的表名大小写与数据库字典中的一致
Q4 :传输进程数据不传输 传输进程启动后不报错,目标端 trail 文件大小不增长 解决方法: 表名大小写问题,修改进程中的表名大小写与数据库字典中的一致
