1 rman报错信息 $ rman Recovery Manager: Release 10.2.0.1.0 - Production on Mon Jun 23 18:51:35 2014 Copyright (c) 1982, 2007, Oracle. All rights reserved. RMAN-00571: =========================================================== RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS =============== RMAN-00571: =========================================================== RMAN-00554: initialization of internal recovery manager package failed RMAN-03000: recovery manager compiler component initialization failed RMAN-06035: wrong version of recover.bsq, expecting 10.2.0.1, found 10.2.0.4 2 经过比对，发现两个节点的lib文件不一致，经咨询，应该是在几年前升级PSU导致的问题。 在 2009年12月28日做数据库升级，过程中有部分的功能升级失败，导致节点2的RMAN不能使用。 节点 1 -rwxr-x--- 1 oracle oinstall 36877360 Dec 28 2009 /oracle/oracle10g2r/dbs/lib/libclntsh.so.10.1 $ -r-xr-xr-x 1 oracle oinstall 5337112 Apr 22 2008 /oracle/oracle10g2r/dbs/lib/libnnz10.so $ -r-xr-xr-x 1 bin bin 85568 Feb 16 2007 /usr/lib/hpux64/librt.so.1 $ -r-xr-xr-x 1 bin bin 91184 Jun 4 2009 /usr/lib/hpux64/libnss_dns.so.1 $ -r-xr-xr-x 1 bin bin 78488 Jun 9 2009 /usr/lib/hpux64/libdl.so.1 $ -r-xr-xr-x 1 bin bin 6580584 Dec 4 2008 /usr/lib/hpux64/libm.so.1 $ -r-xr-xr-x 1 bin bin 1535512 Jun 6 2009 /usr/lib/hpux64/libpthread.so.1 $ -r-xr-xr-x 1 bin bin 635200 Dec 13 2008 /usr/lib/hpux64/libunwind.so.1 $ -r-xr-xr-x 1 bin bin 1510856 Jun 4 2009 /usr/lib/hpux64/libnsl.so.1 $ -r-xr-xr-x 1 bin bin 4898536 Apr 8 2009 /usr/lib/hpux64/libc.so.1 $ -r-xr-xr-x 1 bin bin 85568 Feb 16 2007 /usr/lib/hpux64/librt.so.1 $ -r-xr-xr-x 1 bin bin 91184 Jun 4 2009 /usr/lib/hpux64/libnss_dns.so.1 $ -r-xr-xr-x 1 bin bin 78488 Jun 9 2009 /usr/lib/hpux64/libdl.so.1 $ -r-xr-xr-x 1 bin bin 6580584 Dec 4 2008 /usr/lib/hpux64/libm.so.1 $ -r-xr-xr-x 1 bin bin 1535512 Jun 6 2009 /usr/lib/hpux64/libpthread.so.1 $ -r-xr-xr-x 1 bin bin 635200 Dec 13 2008 /usr/lib/hpux64/libunwind.so.1 $ -r-xr-xr-x 1 bin bin 1510856 Jun 4 2009 /usr/lib/hpux64/libnsl.so.1 $ -r-xr-xr-x 1 bin bin 78488 Jun 9 2009 /usr/lib/hpux64/libdl.so.1 $ -r-xr-xr-x 1 bin bin 85680 Feb 16 2007 /usr/lib/hpux64/libuca.so.1 节点 2 -rwxr-xr-x 1 oracle oinstall 36155448 Dec 26 2009 /oracle/oracle10g2r/dbs/lib/libclntsh.so.10.1 $ -r-xr-xr-x 1 oracle oinstall 5227072 Feb 4 2006 /oracle/oracle10g2r/dbs/lib/libnnz10.so $ -r-xr-xr-x 1 bin bin 85568 Feb 16 2007 /usr/lib/hpux64/librt.so.1 $ -r-xr-xr-x 1 bin bin 91184 Jun 4 2009 /usr/lib/hpux64/libnss_dns.so.1 $ -r-xr-xr-x 1 bin bin 78488 Jun 9 2009 /usr/lib/hpux64/libdl.so.1 $ -r-xr-xr-x 1 bin bin 6580584 Dec 4 2008 /usr/lib/hpux64/libm.so.1 $ -r-xr-xr-x 1 bin bin 1535512 Jun 6 2009 /usr/lib/hpux64/libpthread.so.1 $ -r-xr-xr-x 1 bin bin 635200 Dec 13 2008 /usr/lib/hpux64/libunwind.so.1 $ -r-xr-xr-x 1 bin bin 1510856 Jun 4 2009 /usr/lib/hpux64/libnsl.so.1 $ -r-xr-xr-x 1 bin bin 4898536 Apr 8 2009 /usr/lib/hpux64/libc.so.1 $ -r-xr-xr-x 1 bin bin 85568 Feb 16 2007 /usr/lib/hpux64/librt.so.1 $ -r-xr-xr-x 1 bin bin 91184 Jun 4 2009 /usr/lib/hpux64/libnss_dns.so.1 $ -r-xr-xr-x 1 bin bin 78488 Jun 9 2009 /usr/lib/hpux64/libdl.so.1 $ -r-xr-xr-x 1 bin bin 6580584 Dec 4 2008 /usr/lib/hpux64/libm.so.1 $ -r-xr-xr-x 1 bin bin 1535512 Jun 6 2009 /usr/lib/hpux64/libpthread.so.1 $ -r-xr-xr-x 1 bin bin 635200 Dec 13 2008 /usr/lib/hpux64/libunwind.so.1 $ -r-xr-xr-x 1 bin bin 1510856 Jun 4 2009 /usr/lib/hpux64/libnsl.so.1 $ -r-xr-xr-x 1 bin bin 78488 Jun 9 2009 /usr/lib/hpux64/libdl.so.1 $ -r-xr-xr-x 1 bin bin 85680 Feb 16 2007 /usr/lib/hpux64/libuca.so.1 发现节点 1和节点2上有部分的库文件不一致。
三 故障处理
1 关闭节点2数据库 2 在节点2上重命名文件： $ mv /oracle/oracle10g2r/dbs/lib/libclntsh.so.10.1 /oracle/oracle10g2r/dbs/lib/libclntsh.so.10.1.bak623 $ mv /oracle/oracle10g2r/dbs/lib/libnnz10.so /oracle/oracle10g2r/dbs/lib/libnnz10.so.bak623 $ ls -l /oracle/oracle10g2r/dbs/lib/libclntsh.so.10.1 /oracle/oracle10g2r/dbs/lib/libnnz10.so /oracle/oracle10g2r/dbs/lib/libclntsh.so.10.1 not found /oracle/oracle10g2r/dbs/lib/libnnz10.so not found $ ls -l /oracle/oracle10g2r/dbs/lib/libclntsh.so.10.1.bak2014623 /oracle/oracle10g2r/dbs/lib/libnnz10.so.bak2014623 /oracle/oracle10g2r/dbs/lib/libclntsh.so.10.1.bak2014623 not found /oracle/oracle10g2r/dbs/lib/libnnz10.so.bak2014623 not found $ ls -l /oracle/oracle10g2r/dbs/lib/libclntsh.so.10.1.bak623 /oracle/oracle10g2r/dbs/lib/libnnz10.so.bak623 -rwxr-xr-x 1 oracle oinstall 36155448 Dec 26 2009 /oracle/oracle10g2r/dbs/lib/libclntsh.so.10.1.bak623 -r-xr-xr-x 1 oracle oinstall 5227072 Feb 4 2006 /oracle/oracle10g2r/dbs/lib/libnnz10.so.bak623 3 登陆到节点1，拷贝节点1的文件到节点2 $ scp /oracleacle10g2r/dbs/lib/libclntsh.so.10.1 oracle@cwgkdb2:/oracle/oracle10g2r/dbs/lib Password: libclntsh.so.10.1 100% 35MB 35.2MB/s 29.1MB/s 00:01 $ scp /oracle/oracle10g2r/dbs/lib/libnnz10.so oracle@cwgkdb2:/oracle/oracle10g2r/dbs/lib Password: libnnz10.so 100% 5212KB 5.1MB/s 5.1MB/s 00:01 $ $ ls -l /oracle/oracle10g2r/dbs/lib/libclntsh.so.10.1 /oracle/oracle10g2r/dbs/lib/libnnz10.so -rwxr-x--- 1 oracle oinstall 36877360 Jun 23 19:55 /oracle/oracle10g2r/dbs/lib/libclntsh.so.10.1 -r-xr-xr-x 1 oracle oinstall 5337112 Jun 23 19:55 /oracle/oracle10g2r/dbs/lib/libnnz10.so 4 启动数据库 startup 5 rman 验证 $ sqlplus / as sysdba SQL*Plus: Release 10.2.0.4.0 - Production on Mon Jun 23 19:56:10 2014 Copyright (c) 1982, 2007, Oracle. All Rights Reserved. Connected to an idle instance. SQL> startup ORACLE instance started. Total System Global Area 3.2212E+10 bytes Fixed Size 2213440 bytes Variable Size 8561899968 bytes Database Buffers 2.3606E+10 bytes Redo Buffers 42598400 bytes Database mounted. Database opened. SQL> exit Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production With the Partitioning, Real Application Clusters, OLAP, Data Mining and Real Application Testing options $ rman Recovery Manager: Release 10.2.0.4.0 - Production on Mon Jun 23 19:56:59 2014 Copyright (c) 1982, 2007, Oracle. All rights reserved. RMAN> exit Recovery Manager complete. $ rman target / Recovery Manager: Release 10.2.0.4.0 - Production on Mon Jun 23 19:57:20 2014 Copyright (c) 1982, 2007, Oracle. All rights reserved. connected to target database: CTESTPD (DBID=543556404) RMAN> exit Recovery Manager complete.
