在检查系统中，发现服务器到存储链路存在问题，对数据库的影响是造成数据库宕机，启动时报错。 具体信息如下 数据库启动时 ，报错4193错误。 ``` Total System Global Area 1235959808 bytes Fixed Size 2252784 bytes Variable Size 385875984 bytes Database Buffers 838860800 bytes Redo Buffers 8970240 bytes Database mounted. ORA-01092: ORACLE instance terminated. Disconnection forced ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], [] Process ID: 10817 Session ID: 1 Serial number: 5 ``` alert日志 ``` --alert 日志 Thread 1 opened at log sequence 2 Current log# 2 seq# 2 mem# 0: /u01/app/oracle/oradata/oratest/redo02.log Successful open of redo thread 1 MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set SMON: enabling cache recovery Errors in file /u01/app/oracle/diag/rdbms/oratest/oratest/trace/oratest_ora_10817.trc (incident=15753): ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], [] Incident details in: /u01/app/oracle/diag/rdbms/oratest/oratest/incident/incdir_15753/oratest_ora_10817_i15753.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Block recovery from logseq 2, block 3 to scn 1057010 Recovery of Online Redo Log: Thread 1 Group 2 Seq 2 Reading mem 0 Mem# 0: /u01/app/oracle/oradata/oratest/redo02.log Block recovery stopped at EOT rba 2.5.16 Block recovery completed at rba 2.5.16, scn 0.1057008 Block recovery from logseq 2, block 3 to scn 1057007 Recovery of Online Redo Log: Thread 1 Group 2 Seq 2 Reading mem 0 Mem# 0: /u01/app/oracle/oradata/oratest/redo02.log Block recovery completed at rba 2.5.16, scn 0.1057008 Errors in file /u01/app/oracle/diag/rdbms/oratest/oratest/trace/oratest_ora_10817.trc: ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], [] Errors in file /u01/app/oracle/diag/rdbms/oratest/oratest/trace/oratest_ora_10817.trc: ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], [] Error 600 happened during db open, shutting down database USER (ospid: 10817): terminating the instance due to error 600 Instance terminated by USER, pid = 10817 ORA-1092 signalled during: ALTER DATABASE OPEN... opiodr aborting process unknown ospid (10817) as a result of ORA-1092 Sun Jan 02 21:44:48 2022 ORA-1092 : opitsk aborting process ``` 通过跟踪open进程， 发现错误信息如下： ``` WAIT #140022227445520: nam='db file sequential read' ela= 19 file#=1 block#=544 blocks=1 obj#=0 tim=1641262059791045 Incident 6153 created, dump file: /u01/app/oracle/diag/rdbms/mydb11/mydb11/incident/incdir_6153/mydb11_ora_13573_i6153.trc ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], [] ``` 通过以上可以发现，在启动时，扫描到1号文件的 544块出现位图，检查trace文件， ``` shell #trace文件中报错，显示544文件损坏，类型是 UNDO BLOCK Block after image is corrupt: buffer tsn: 0 rdba: 0x00400220 (1/544) scn: 0x0000.000f2a5c seq: 0x0b flg: 0x04 tail: 0x2a5c020b frmt: 0x02 chkval: 0x461c type: 0x02=KTU UNDO BLOCK Hex dump of block: st=0, typ_found=1 Dump of memory from 0x0000000092E76000 to 0x0000000092E78000 #数据库尝试修复，但未成功，再继续看trace文件，下面记录了undo header信息 Doing block recovery for file 1 block 128 Block header before block recovery: buffer tsn: 0 rdba: 0x00400080 (1/128) scn: 0x0000.000f2a5d seq: 0x01 flg: 0x04 tail: 0x2a5d0e01 frmt: 0x02 chkval: 0xa4a2 type: 0x0e=KTU UNDO HEADER W/UNLIMITED EXTENTS Resuming block recovery (PMON) for file 1 block 128 Block recovery from logseq 6, block 26150 to scn 993894 Extent Control Header ----------------------------------------------------------------- Extent Header:: spare1: 0 spare2: 0 #extents: 6 #blocks: 47 last map 0x00000000 #maps: 0 offset: 4128 Highwater:: 0x00400220 ext#: 4 blk#: 0 ext size: 8 #blocks in seg. hdr's freelists: 0 #blocks below: 0 mapblk 0x00000000 offset: 4 Unlocked Map Header:: next 0x00000000 #extents: 6 obj#: 0 flag: 0x40000000 Extent Map ----------------------------------------------------------------- 0x00400081 length: 7 0x00400088 length: 8 0x00400210 length: 8 0x00400218 length: 8 0x00400220 length: 8 0x00400228 length: 8 TRN CTL:: seq: 0x001c chd: 0x0002 ctl: 0x0018 inc: 0x00000000 nfb: 0x0001 mgc: 0x8002 xts: 0x0068 flg: 0x0001 opt: 2147483646 (0x7ffffffe) uba: 0x00400220.001c.01 scn: 0x0000.000e184c Version: 0x01 FREE BLOCK POOL:: uba: 0x00400220.001c.0a ext: 0x4 spc: 0x17d2 uba: 0x00000000.001a.02 ext: 0x2 spc: 0x1f16 uba: 0x00000000.0018.3a ext: 0x0 spc: 0x80e uba: 0x00000000.0000.00 ext: 0x0 spc: 0x0 uba: 0x00000000.0000.00 ext: 0x0 spc: 0x0 ``` 通过以上信息，我们可以看到，损坏的544，在段头中有连接，也就是1号文件的128块上，搜索 '0x00400220'也可以找出。也就是在open过程中，进程通过128 信息找到544，发现544损坏后报错。那么我们可以通过清空dba地址，不让进程找到544块。 可以通过bbed检查并修改, ``` BBED> set file 1 block 128 FILE# 1 BLOCK# 128 BBED> p ktuxc struct ktuxc, 104 bytes @4148 struct ktuxcscn, 8 bytes @4148 ub4 kscnbas @4148 0x000e184c ub2 kscnwrp @4152 0x0000 struct ktuxcuba, 8 bytes @4156 ub4 kubadba @4156 0x00400220 ub2 kubaseq @4160 0x001c ub1 kubarec @4162 0x01 sb2 ktuxcflg @4164 1 (KTUXCFSK) ub2 ktuxcseq @4166 0x001c sb2 ktuxcnfb @4168 1 #有空闲块可使用 ub4 ktuxcinc @4172 0x00000000 sb2 ktuxcchd @4176 2 sb2 ktuxcctl @4178 24 ub2 ktuxcmgc @4180 0x8002 ub4 ktuxcopt @4188 0x7ffffffe struct ktuxcfbp[0], 12 bytes @4192 struct ktufbuba, 8 bytes @4192 ub4 kubadba @4192 0x00400220 # dba地址，跟trace文件一直 ub2 kubaseq @4196 0x001c ub1 kubarec @4198 0x0a sb2 ktufbext @4200 4 sb2 ktufbspc @4202 6098 ``` 可以通过清空该地址，启动数据库，参考命令如下： ``` #请谨慎操作 set file 1 block 128 modify /x 00 offset 4168 modify /x 0x00000000 offset 4192 sum apply ```