RAC 19.6 RU补丁实施手册

目录

背景

一、        准备

2.1 软件目录备份

2.2 检查数据字典信息

2.3 检查集群状态

2.4 检查可执行文件权限

2.5 验证 Oracle Inventory

2.6 unzip 补丁包

2.7 补丁冲突检测

二、        手动应用 RU 补丁

3.1 停止从 DB home 运行 CRS 管理的资源

3.2 运行 prepatch 脚本

3.3 Patch GI home

3.4 Patch DB home

3.5 运行 post 脚本

3.6 注意

3.7 启动以前从 DB home 运行的 CRS 管理的资源

3.8 运行 datapatch 脚本

3.9 运行 utlrp 脚本

三、        [ 补丁更新问题 ] - 在第二节点安装 oracle Update 报错

4.1 在手动执行步骤 3.4 遇到以下错误：

4.2 原因

4.3 解决办法

4.4 建议

背景

客户数据库出现BUG 导致实例重启，客户申请了我们工程师远程分析后，评估后，需要尽快实施补丁升级。

1 ）环境变量

2 ）需要安装的补丁

Patch 30923276

准备

1.1 软件目录备份

1 ）备份DB

# nohup tar -cpvf /soft/backup/`hostname`_db_`date +%y-%m-%d-%T`.tar /oracle/app/oracle/product/19.3.0/dbhome  > /soft/backup/`hostname`_db_`date +%y-%m-%d-%T`.log &

2 ）备份GI

# nohup tar -cpvf /soft/backup/`hostname`_grid_`date +%y-%m-%d-%T`.tar /oracle/app/19.3.0/grid  > /soft/backup/`hostname`_grid_`date +%y-%m-%d-%T`.log &

3 ）备份Inventory

# nohup tar -cpvf /soft/backup/`hostname`_inventory_`date +%y-%m-%d-%T`.tar /oracle/app/oraInventory > /soft/backup/`hostname`_inventory_`date +%y-%m-%d-%T`.log &

1.2 检查数据字典信息

set linesize 1000 pagesize 5000

col DESCRIPTION  for a70

col ACTION_TIME for a20

select to_char(ACTION_TIME,'yyyy-mm-dd hh24:mi:ss')ACTION_TIME,patch_id,VERSION,action,status,DESCRIPTION  

from dba_registry_sqlpatch order by ACTION_TIME;

col OWNER for a30

col object_type for a30

col object_name for a30

select owner,object_type,object_name from dba_objects where status<>'VALID';

select count(*) from dba_objects where status<>'VALID';

1.3 检查集群状态

$ crsctl stat res -t

1.4 检查可执行文件权限

在 oracle 和 grid 用户下执行：

$ ls -ld $ORACLE_HOME/bin/oracle

1.5 验证 Oracle Inventory

在 Oracle 和 grid 用户下执行：

$ <ORACLE_HOME>/OPatch/opatch lsinventory -detail -oh <ORACLE_HOME>

1.6 unzip 补丁包

用 grid 用户解压

$ unzip p30923276_190000_Linux-x86-64.zip [-d <unzip_directory>]

1.7 补丁冲突检测

For Grid Infrastructure Home, as home user:

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /soft/patch/30923276/30850056

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /soft/patch/30923276/30489632 $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /soft/patch/30923276/30797938 $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /soft/patch/30923276/30898856

For Database home, as home user:

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /soft/patch/30923276/30850056 $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /soft/patch/30923276/30797938

手动应用RU 补丁

依次在集群的每个节点上执行以下 1-7 步骤以应用补丁

2.1 停止从 DB home 运行 CRS 管理的资源

If this is a GI Home environment, as the database home owner execute:

$ <ORACLE_HOME>/bin/srvctl stop home -o <ORACLE_HOME> -s <status file location> -n <node name>

If this is an Oracle Restart Home environment, as the database home owner execute:

$ <ORACLE_HOME>/bin/srvctl stop home -o <ORACLE_HOME> -s <status file location>

示例：

$ srvctl stop home -o $ORACLE_HOME -s /home/oracle/HQsPSL-OEMN-R01.st -n HQsPSL-OEMN-R01

2.2 运行 prepatch 脚本

If this is a GI Home, as the root user execute:

# <GI_HOME>/crs/install/rootcrs.sh -prepatch

If this is an Oracle Restart Home, as the root user execute:

# <GI_HOME>/crs/install/roothas.sh -prepatch

示例：

# /oracle/app/19.3.0/grid/crs/install/rootcrs.sh -prepatch

2.3 Patch GI home

As the GI home owner execute:

$ <GI_HOME>/OPatch/opatch apply -oh <GI_HOME> -local <UNZIPPED_PATCH_LOCATION>/%BUGNO%/%OCW TRACKING BUG%

$ <GI_HOME>/OPatch/opatch apply -oh <GI_HOME> -local <UNZIPPED_PATCH_LOCATION>/%BUGNO%/%DB WLM TRACKING BUG%

$ <GI_HOME>/OPatch/opatch apply -oh <GI_HOME> -local <UNZIPPED_PATCH_LOCATION>/%BUGNO%/%DB RU TRACKING BUG%

$ <GI_HOME>/OPatch/opatch apply -oh <GI_HOME> -local <UNZIPPED_PATCH_LOCATION>/%BUGNO%/%TOMCAT RU TRACKING BUG%

示例 :

/oracle/app/19.3.0/grid/OPatch/opatch apply -oh /oracle/app/19.3.0/grid -local /soft/patch/30923276/30850056

/oracle/app/19.3.0/grid/OPatch/opatch apply -oh /oracle/app/19.3.0/grid -local /soft/patch/30923276/30489632

/oracle/app/19.3.0/grid/OPatch/opatch apply -oh /oracle/app/19.3.0/grid -local /soft/patch/30923276/30797938

/oracle/app/19.3.0/grid/OPatch/opatch apply -oh /oracle/app/19.3.0/grid -local /soft/patch/30923276/30898856

2.4 Patch DB home

As the database home owner execute:

$ <UNZIPPED_PATCH_LOCATION>/%BUGNO%/%OCW TRACKING BUG%/custom/scripts/prepatch.sh -dbhome <ORACLE_HOME>

$ <ORACLE_HOME>/OPatch/opatch apply -oh <ORACLE_HOME> -local <UNZIPPED_PATCH_LOCATION>/%BUGNO%/%OCW TRACKING BUG%

$ <ORACLE_HOME>/OPatch/opatch apply -oh <ORACLE_HOME> -local <UNZIPPED_PATCH_LOCATION>/%BUGNO%/%DB RU TRACKING BUG%

$ <UNZIPPED_PATCH_LOCATION>/%BUGNO%/%OCW TRACKING BUG%/custom/scripts/postpatch.sh -dbhome <ORACLE_HOME> 

示例：

/soft/patch/30923276/30850056/custom/scripts/prepatch.sh -dbhome $ORACLE_HOME

$ORACLE_HOME/OPatch/opatch apply -oh $ORACLE_HOME -local /soft/patch/30923276/30850056

$ORACLE_HOME/OPatch/opatch apply -oh $ORACLE_HOME -local /soft/patch/30923276/30797938

/soft/patch/30923276/30850056/custom/scripts/postpatch.sh -dbhome $ORACLE_HOME

注：在第二节点执行此步骤会遇到oui-patch.xml 权限问题，详细请参考< 补丁更新问题>

2.5 运行 post 脚本

As the root user execute:

# <GI_HOME>/rdbms/install/rootadd_rdbms.sh

If this is a GI Home, as the root user execute:

# <GI_HOME>/crs/install/rootcrs.sh -postpatch

If this is an Oracle Restart Home, as the root user execute:

# <GI_HOME>/crs/install/roothas.sh -postpatch

示例：

# /oracle/app/19.3.0/grid/rdbms/install/rootadd_rdbms.sh

# /oracle/app/19.3.0/grid/crs/install/rootcrs.sh -postpatch

2.6 注意

If the message, "A system reboot is recommended before using ACFS is shown, then a reboot must be issued before continuing. Failure to do so will result in running with an unpatched ACFS\ADVM\OKS driver.

2.7 启动以前从 DB home 运行的 CRS 管理的资源

If this is a GI Home environment, as the database home owner execute:

$ <ORACLE_HOME>/bin/srvctl start home -o <ORACLE_HOME> -s <status file location> -n <node name>

If this is an Oracle Restart Home environment, as the database home owner execute:

$ <ORACLE_HOME>/bin/srvctl start home -o <ORACLE_HOME> -s <status file location>

示例 :

$ srvctl start home -o $ORACLE_HOME -s /home/oracle/HQsPSL-OEMN-R01.st -n HQsPSL-OEMN-R01

2.8 运行 datapatch 脚本

对于运行Oracle home 上的每个数据库，执行以下步骤。

注意 ：

当前环境上存在二个数据库实例，所以需要在每个数据库上都执行一次。（对于 rac 只需要在一个节点的实例上运行）

2.9 运行 utlrp 脚本

If the OJVM RU or RUR is also installed, you may see invalid objects after execution of datapatch in the previous step. If this is the case, run utlrp.sql to revalidate these objects.

cd $ORACLE_HOME/rdbms/admin

sqlplus /nolog

SQL> CONNECT / AS SYSDBA

SQL> @utlrp.sql

[ 补丁更新问题] - 在第二节点安装oracle Update 报错

3.1 在手动执行步骤 3.4 遇到以下错误：

oracle@HQsPSL-OEMN-R02:~> $ORACLE_HOME/OPatch/opatch apply -oh $ORACLE_HOME -local /soft/patch/30923276/30850056

Oracle Interim Patch Installer version 12.2.0.1.19

Copyright (c) 2020, Oracle Corporation.  All rights reserved.

Oracle Home       : /oracle/app/oracle/product/19.3.0/dbhome

Central Inventory : /oracle/app/oraInventory

   from           : /oracle/app/oracle/product/19.3.0/dbhome/oraInst.loc

OPatch version    : 12.2.0.1.19

OUI version       : 12.2.0.7.0

Log file location : /oracle/app/oracle/product/19.3.0/dbhome/cfgtoollogs/opatch/opatch2020-05-06_10-43-09AM_1.log

Verifying environment and performing prerequisite checks...

OPatch continues with these patches:   30850056 

Do you want to proceed? [y|n]

y

User Responded with: Y

All checks passed.

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.

(Oracle Home = '/oracle/app/oracle/product/19.3.0/dbhome')

Is the local system ready for patching? [y|n]

y

User Responded with: Y

Backing up files...

Applying interim patch '30850056' to OH '/oracle/app/oracle/product/19.3.0/dbhome'

ApplySession: Optional component(s) [ oracle.has.crs, 19.0.0.0.0 ] , [ oracle.xag, 19.0.0.0.0 ] , [ oracle.has.crs.cvu, 19.0.0.0.0 ] , [ oracle.rhp.crs, 19.0.0.0.0 ] , [ oracle.has.cvu, 19.0.0.0.0 ]  not present in the Oracle Home or a higher version is found.

Patching component oracle.rdbms, 19.0.0.0.0...

Patching component oracle.has.rsf, 19.0.0.0.0...

Patching component oracle.rhp.db, 19.0.0.0.0...

Patching component oracle.rhp.common, 19.0.0.0.0...

Patching component oracle.has.db.cvu, 19.0.0.0.0...

Patching component oracle.has.db, 19.0.0.0.0...

Patching component oracle.has.common, 19.0.0.0.0...

Patching component oracle.has.common.cvu, 19.0.0.0.0...

ApplySession failed in system modification phase... 'ApplySession::apply failed: java.io.IOException: oracle.sysman.oui.patch.PatchException: java.io.FileNotFoundException: /oracle/app/oraInventory/ContentsXML/oui-patch.xml (Permission denied)'

Restoring "/oracle/app/oracle/product/19.3.0/dbhome" to the state prior to running NApply...

#### Stack trace of processes holding locks ####

Time: 2020-05-06_10-43-09AM

Command: oracle/opatch/OPatch apply -oh /oracle/app/oracle/product/19.3.0/dbhome -local /soft/patch/30923276/30850056 -invPtrLoc /oracle/app/oracle/product/19.3.0/dbhome/oraInst.loc

Lock File Name: /oracle/app/oraInventory/locks/_oracle_app_oracle_product_19.3.0_dbhome_writer.lock

StackTrace:

-----------

java.lang.Throwable

        at oracle.sysman.oii.oiit.OiitLockHeartbeat.writeStackTrace(OiitLockHeartbeat.java:193)

        at oracle.sysman.oii.oiit.OiitLockHeartbeat.<init>(OiitLockHeartbeat.java:173)

        at oracle.sysman.oii.oiit.OiitTargetLocker.getWriterLock(OiitTargetLocker.java:346)

        at oracle.sysman.oii.oiit.OiitTargetLocker.getWriterLock(OiitTargetLocker.java:238)

        at oracle.sysman.oii.oiic.OiicStandardInventorySession.acquireLocks(OiicStandardInventorySession.java:564)

        at oracle.sysman.oii.oiic.OiicStandardInventorySession.initAreaControl(OiicStandardInventorySession.java:359)

        at oracle.sysman.oii.oiic.OiicStandardInventorySession.initSession(OiicStandardInventorySession.java:332)

        at oracle.sysman.oii.oiic.OiicStandardInventorySession.initSession(OiicStandardInventorySession.java:294)

        at oracle.sysman.oii.oiic.OiicStandardInventorySession.initSession(OiicStandardInventorySession.java:243)

        at oracle.sysman.oui.patch.impl.HomeOperationsImpl.initialize(HomeOperationsImpl.java:107)

        at oracle.glcm.opatch.common.api.install.HomeOperationsShell.initialize(HomeOperationsShell.java:117)

        at oracle.opatch.ipm.IPMRWServices.addPatchCUP(IPMRWServices.java:134)

        at oracle.opatch.ipm.IPMRWServices.add(IPMRWServices.java:146)

        at oracle.opatch.ApplySession.apply(ApplySession.java:899)

        at oracle.opatch.ApplySession.processLocal(ApplySession.java:4098)

        at oracle.opatch.ApplySession.process(ApplySession.java:5080)

        at oracle.opatch.ApplySession.process(ApplySession.java:4942)

        at oracle.opatch.OPatchACL.processApply(OPatchACL.java:310)

        at oracle.opatch.opatchutil.NApply.legacy_process(NApply.java:1429)

        at oracle.opatch.opatchutil.NApply.legacy_process(NApply.java:372)

        at oracle.opatch.opatchutil.NApply.process(NApply.java:352)

        at oracle.opatch.opatchutil.OUSession.napply(OUSession.java:1123)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at oracle.opatch.UtilSession.process(UtilSession.java:355)

        at oracle.opatch.OPatchSession.process(OPatchSession.java:2660)

        at oracle.opatch.OPatch.process(OPatch.java:840)

        at oracle.opatch.OPatch.main(OPatch.java:897)

 ------------------------------------

OPatch failed to restore OH '/oracle/app/oracle/product/19.3.0/dbhome'. Consult OPatch document to restore the home manually before proceeding.

NApply was not able to restore the home.  Please invoke the following scripts:

  - restore.[sh,bat]

  - make.txt (Unix only)

to restore the ORACLE_HOME.  They are located under

"/oracle/app/oracle/product/19.3.0/dbhome/.patch_storage/NApply/2020-05-06_10-43-09AM"

UtilSession failed: ApplySession failed in system modification phase... 'ApplySession::apply failed: java.io.IOException: oracle.sysman.oui.patch.PatchException: java.io.FileNotFoundException: /oracle/app/oraInventory/ContentsXML/oui-patch.xml (Permission denied) '

Log file location: /oracle/app/oracle/product/19.3.0/dbhome/cfgtoollogs/opatch/opatch2020-05-06_10-43-09AM_1.log

OPatch failed with error code 73

3.2 原因

参考文档： 《 opatchauto apply Results java.io.FileNotFoundException: <oraInventory>/ContentsXML/oui-patch.xml (Permission denied) Error in Non-OUI Nodes (Doc ID 2582139.1)    》

The issue is analyzed and discussed in following internal / unpublished Bug

 BUG 29859410 - OPATCHAUTO_194:OPATCHAUTO FAILED DUE TO WRONG PERMISSION OF OUI-PATCH.XML IN DIFFERENT USER ENV

 Bug 30041091 : OPATCHAUTO_194:OPATCHAUTO FAILED DUE TO WRONG PERMISSION OF OUI-PATCH.XML IN DIFFERENT USER ENVIRONMENT

3.3 解决办法

1). 执行 restore.sh 脚本

2 ）从节点 1 复制 oui-patch.xml 文件到节点 2 上。

3 ）手动回滚补丁

$ORACLE_HOME/OPatch/opatch nrollback -local -id 30898856 -oh $ORACLE_HOME

这个时候会提示以下错误：

$ORACLE_HOME/inventory/oneoffs/30850056 is corrupted

从 1 节点 $ORACLE_HOME/inventory/oneoffs/ 拷贝过去即可 , 然后再进行回滚。

4) 再次进行补丁应用。

3.4 建议

建议在第二个节点打oracle 补丁时, 预先检查oui-patch.xml 文件权限，如果与节点1 权限比较有差异的话，按照节点1 的权限进行修改。