ORA-29770: global enqueue process LMON is hung

#背景： AIX Oracle11.2.0.4 RAC 节点2实例异常重启。 #数据库alert日志： Sun Jan 02 23:08:13 2022 LMON (ospid: 4981218) waits for event 'rdbms ipc message' for 90 secs. Sun Jan 02 23:08:28 2022 Errors in file /u01/app/oracle/diag/rdbms/bjsbjl/bjsbjl2/trace/bjsbjl2_lmhb_3736326.trc  (incident=336164): ORA-29770: global enqueue process LMON (OSID 4981218) is hung for more than 70 seconds Incident details in: /u01/app/oracle/diag/rdbms/bjsbjl/bjsbjl2/incident/incdir_336164/bjsbjl2_lmhb_3736326_i336164.trc Sun Jan 02 23:08:36 2022 Sweep [inc][336164]: completed Sweep [inc2][336164]: completed Sun Jan 02 23:08:40 2022 ERROR: Some process(s) is not making progress. LMHB (ospid: 3736326) is terminating the instance. Please check LMHB trace file for more details. Please also check the CPU load, I/O load and other system properties for anomalous behavior ERROR: Some process(s) is not making progress. LMHB (ospid: 3736326): terminating the instance due to error 29770 Sun Jan 02 23:08:40 2022 System state dump requested by (instance=2, osid=3736326 (LMHB)), summary=[abnormal instance termination]. System State dumped to trace file /u01/app/oracle/diag/rdbms/bjsbjl/bjsbjl2/trace/bjsbjl2_diag_6619226_20220102230840.trc Instance terminated by LMHB, pid = 3736326 #lmbh的trace文件： * 略 * Short stack dump: ORA-32516: cannot wait for process 'Unix process pid: 4981218, image: oracle@gshsqjdb02 (LMON)' to finish executing ORADEBUG command 'SHORT_STACK'; wait time exceeds 2495 ms * 略 * kjfmGCR_HBCheckAll: 1 process failed HBStatusCheck kjfmGCR_HBAnalyze: LMON has no heartbeat *** 2022-01-02 23:08:28.904 ============================== LMON (ospid: 4981218) has not moved for 113 sec (1641136108.1641135995) Incident 336164 created, dump file: /u01/app/oracle/diag/rdbms/bjsbjl/bjsbjl2/incident/incdir_336164/bjsbjl2_lmhb_3736326_i336164.trc ORA-29770: global enqueue process LMON (OSID 4981218) is hung for more than 70 seconds ORA-32515: cannot issue ORADEBUG command 'SHORT_STACK' to process 'Unix process pid: 4981218, image: oracle@gshsqjdb02 (LMON)'; prior command execution time exceeds 2486 ms * 略 * #MOS上有篇文档： OERR: ORA-29770 "global enqueue process %s (OSID %s) is hung for more than %s seconds" Reference Note (Doc ID 2010718.1) 其中找到29770关于LMON的一个bug： Bug Fixed Description 27254851 18.3, 19.1 INSTANCE CRASHED WITH ORA-29770 DUE TO LMON HAS NOT HEARTBEAT 该问题在18，19版本才修复。