19c安装配置

NOTE: remote asm mode is remote (mode 0x2; from cluster type) NOTE: Cluster configuration type = CLUSTER [4] Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.3.0.0.0. ORACLE_HOME:    /u01/oracle/19.0.0 System name:    Linux Node name:      otest1 Release:        3.10.0-514.el7.x86_64 Version:        #1 SMP Tue Nov 22 16:42:41 UTC 2016 Machine:        x86_64 Using parameter settings in client-side pfile /u01/oracle/cfgtoollogs/dbca/orcl/initorclTempOMF.ora on machine otest1 System parameters with non-default values:   processes                = 1320   nls_language             = "AMERICAN"   nls_territory            = "AMERICA"   sga_target               = 2960M   control_files            = "/u01/oracle/cfgtoollogs/dbca/orcl/tempControl.ctl"   db_block_size            = 8192   compatible               = "19.0.0"   db_create_file_dest      = "+DATA"   undo_tablespace          = "UNDOTBS1"   instance_number          = 1   remote_login_passwordfile= "EXCLUSIVE"   dispatchers              = "(PROTOCOL=TCP) (SERVICE=orclXDB)"   local_listener           = "-oraagent-dummy-"   audit_file_dest          = "/u01/oracle/admin/orcl/adump"   audit_trail              = "DB"   db_name                  = "seeddata"   db_unique_name           = "orcl"   open_cursors             = 300   pga_aggregate_target     = 984M   _diag_hm_rc_enabled      = FALSE   diagnostic_dest          = "/u01/oracle"   enable_pluggable_database= TRUE 2020-05-04T17:22:44.559519+08:00 Cluster Communication is configured to use IPs from: GPnP IP: 169.254.2.52         Subnet: 169.254.0.0 KSIPC Loopback IP addresses(OSD): 127.0.0.1 KSIPC Available Transports: UDP:TCP Setting CPU count to 4 ORACLE_BASE from environment = /u01/oracle 2020-05-04T17:24:47.392340+08:00 Create controlfile reuse set database "orcl" MAXINSTANCES 32 MAXLOGHISTORY 1 MAXLOGFILES 192 MAXLOGMEMBERS 3 MAXDATAFILES 1024 Datafile '+DATA/ORCL/DATAFILE/sysaux.260.1039541037', '+DATA/ORCL/DATAFILE/system.259.1039540983', '+DATA/ORCL/DATAFILE/undotbs1.261.1039541073', '+DATA/ORCL/DATAFILE/users.262.1039541073' LOGFILE GROUP 1  SIZE 200M, GROUP 2  SIZE 200M RESETLOGS 2020-05-04T17:33:45.869478+08:00 ALTER SYSTEM SET _catalog_foreign_restore=FALSE SCOPE=MEMORY; 2020-05-04T17:33:45.870955+08:00 ALTER SYSTEM SET _restore_create_directory=FALSE SCOPE=MEMORY; CREATE PLUGGABLE DATABASE PDB$SEED AS CLONE  USING '/u01/oracle/19.0.0/assistants/dbca/templates/pdbseed.xml'  source_file_name_convert = ('/oradata/SEEDDATA/pdbseed/system01.dbf','+DATA/ORCL/86B637B62FE07A65E053F706E80A27CA/DATAFILE/system.267.1039541595', '/oradata/SEEDDATA/pdbseed/sysaux01.dbf','+DATA/ORCL/86B637B62FE07A65E053F706E80A27CA/DATAFILE/sysaux.268.1039541595', '/oradata/SEEDDATA/pdbseed/undotbs01.dbf','+DATA/ORCL/86B637B62FE07A65E053F706E80A27CA/DATAFILE/undotbs1.269.1039541595', '/oradata/SEEDDATA/pdbseed/temp01.dbf','+DATA') file_name_convert=NONE  NOCOPY  STORAGE ( MAXSIZE UNLIMITED MAX_SHARED_TEMP_SIZE UNLIMITED) 2020-05-04T17:33:46.047693+08:00 PDB$SEED(2):Endian type of dictionary set to little **************************************************************** Pluggable Database PDB$SEED with pdb id - 2 is created as UNUSABLE. If any errors are encountered before the pdb is marked as NEW, then the pdb must be dropped NOTE: Cluster configuration type = CLUSTER [4] Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.3.0.0.0. ORACLE_HOME:    /u01/oracle/19.0.0 System name:    Linux Node name:      otest1 Release:        3.10.0-514.el7.x86_64 Version:        #1 SMP Tue Nov 22 16:42:41 UTC 2016 Machine:        x86_64 Using parameter settings in server-side spfile +DATA/ORCL/PARAMETERFILE/spfile.274.1039542325 System parameters with non-default values:   processes                = 1320   nls_language             = "AMERICAN"   nls_territory            = "AMERICA"   sga_target               = 2960M   control_files            = "+DATA/ORCL/CONTROLFILE/current.263.1039541141"   db_block_size            = 8192   compatible               = "19.0.0"   cluster_database         = TRUE   db_create_file_dest      = "+DATA"   thread                   = 1   undo_tablespace          = "UNDOTBS1"   instance_number          = 1   remote_login_passwordfile= "EXCLUSIVE"   dispatchers              = "(PROTOCOL=TCP) (SERVICE=orclXDB)"   local_listener           = "-oraagent-dummy-"   audit_file_dest          = "/u01/oracle/admin/orcl/adump"   audit_trail              = "DB"   db_name                  = "orcl"   open_cursors             = 300   pga_aggregate_target     = 984M   diagnostic_dest          = "/u01/oracle"   enable_pluggable_database= TRUE 2020-05-04T17:50:41.309879+08:00 Cluster Communication is configured to use IPs from: GPnP IP: 169.254.2.52         Subnet: 169.254.0.0 KSIPC Loopback IP addresses(OSD): [root@otest1 19.0.0]# sh root.sh Performing root user operation. The following environment variables are set as:     ORACLE_OWNER= grid     ORACLE_HOME=  /s01/app/19.0.0 Enter the full pathname of the local bin directory: [/usr/local/bin]:    Copying dbhome to /usr/local/bin ...    Copying oraenv to /usr/local/bin ...    Copying coraenv to /usr/local/bin ... Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Relinking oracle with rac_on option Using configuration parameter file: /s01/app/19.0.0/crs/install/crsconfig_params The log of current session can be found at:   /s01/app/grid/crsdata/otest1/crsconfig/rootcrs_otest1_2020-05-04_04-26-25PM.log 2020/05/04 16:26:42 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'. 2020/05/04 16:26:42 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'. 2020/05/04 16:26:42 CLSRSC-363: User ignored prerequisites during installation 2020/05/04 16:26:42 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'. 2020/05/04 16:26:46 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'. 2020/05/04 16:26:48 CLSRSC-594: Executing installation step 5 of 19: 'SetupOSD'. 2020/05/04 16:26:48 CLSRSC-594: Executing installation step 6 of 19: 'CheckCRSConfig'. 2020/05/04 16:26:48 CLSRSC-594: Executing installation step 7 of 19: 'SetupLocalGPNP'. 2020/05/04 16:27:17 CLSRSC-594: Executing installation step 8 of 19: 'CreateRootCert'. 2020/05/04 16:27:21 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector. 2020/05/04 16:27:25 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'. 2020/05/04 16:27:39 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'. 2020/05/04 16:27:39 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'. 2020/05/04 16:27:47 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'. 2020/05/04 16:27:47 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service' 2020/05/04 16:28:16 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'. 2020/05/04 16:28:23 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'. 2020/05/04 16:28:32 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'. 2020/05/04 16:28:39 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'. ASM has been created and started successfully. [DBT-30001] Disk groups created successfully. Check /s01/app/grid/cfgtoollogs/asmca/asmca-200504PM042923.log for details. 2020/05/04 16:30:17 CLSRSC-482: Running command: '/s01/app/19.0.0/bin/ocrconfig -upgrade grid oinstall' CRS-4256: Updating the profile Successful addition of voting disk eeac1cc2fa144f7cbf2120a2efb7bb25. Successfully replaced voting disk group with +DATA. CRS-4256: Updating the profile CRS-4266: Voting file(s) successfully replaced ##  STATE    File Universal Id                File Name Disk group --  -----    -----------------                --------- ---------  1. ONLINE   eeac1cc2fa144f7cbf2120a2efb7bb25 (/dev/sdc) [DATA] Located 1 voting disk(s). 2020/05/04 16:31:52 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'. 2020/05/04 16:33:21 CLSRSC-343: Successfully started Oracle Clusterware stack 2020/05/04 16:33:21 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'. 2020/05/04 16:35:16 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'. 2020/05/04 16:35:55 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded [root@otest1 19.0.0]# [root@otest2 19.0.0]# sh root.sh Performing root user operation. The following environment variables are set as:     ORACLE_OWNER= grid     ORACLE_HOME=  /s01/app/19.0.0 Enter the full pathname of the local bin directory: [/usr/local/bin]:    Copying dbhome to /usr/local/bin ...    Copying oraenv to /usr/local/bin ...    Copying coraenv to /usr/local/bin ... Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Relinking oracle with rac_on option Using configuration parameter file: /s01/app/19.0.0/crs/install/crsconfig_params The log of current session can be found at:   /s01/app/grid/crsdata/otest2/crsconfig/rootcrs_otest2_2020-05-04_04-37-57PM.log 2020/05/04 16:38:07 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'. 2020/05/04 16:38:07 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'. 2020/05/04 16:38:07 CLSRSC-4004: Failed to install Oracle Trace File Analyzer (TFA) Collector. Grid Infrastructure operations will continue. 2020/05/04 16:38:07 CLSRSC-363: User ignored prerequisites during installation 2020/05/04 16:38:07 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'. 2020/05/04 16:38:10 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'. 2020/05/04 16:38:10 CLSRSC-594: Executing installation step 5 of 19: 'SetupOSD'. 2020/05/04 16:38:10 CLSRSC-594: Executing installation step 6 of 19: 'CheckCRSConfig'. 2020/05/04 16:38:11 CLSRSC-594: Executing installation step 7 of 19: 'SetupLocalGPNP'. 2020/05/04 16:38:15 CLSRSC-594: Executing installation step 8 of 19: 'CreateRootCert'. 2020/05/04 16:38:15 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'. 2020/05/04 16:38:21 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'. 2020/05/04 16:38:21 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'. 2020/05/04 16:38:24 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'. 2020/05/04 16:38:25 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service' 2020/05/04 16:38:55 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'. 2020/05/04 16:38:58 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'. 2020/05/04 16:39:01 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'. 2020/05/04 16:39:04 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'. 2020/05/04 16:39:19 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'. 2020/05/04 16:40:15 CLSRSC-343: Successfully started Oracle Clusterware stack 2020/05/04 16:40:15 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'. 2020/05/04 16:40:42 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'. 2020/05/04 16:40:52 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded [root@otest2 19.0.0]# Cause?-?The plug-in failed in its perform method  Action?-?Refer to the logs or contact Oracle Support Services.  Log File Location /tmp/GridSetupActions2020-05-04_04-01-08PM/gridSetupActions2020-05-04_04-01-08PM.log Copying files for 'Oracle Database Utilities 19.0.0.0.0 ' instantiating '/s01/app/19.0.0/bin/aqxmlctl.template'. deleting '/s01/app/19.0.0/bin/aqxmlctl.template' creating link 'libagtsh.so.1.0' for '/s01/app/19.0.0/lib/libagtsh.so' instantiating '/s01/app/19.0.0/rdbms/install/sbs/diagsetup.sbs'. changing permissions for '/s01/app/19.0.0/bin/diagsetup' Copying files for 'Oracle Notification Service 19.0.0.0.0 ' Copying files for 'SQL*Plus 19.0.0.0.0 ' Copying files for 'Oracle Advanced Security 19.0.0.0.0 ' moving '/s01/app/19.0.0/network/jlib/netentrust8.jar.dbl' to '/s01/app/19.0.0/network/jlib/netentrust8.jar' Copying files for 'Cluster Ready Services Files 19.0.0.0.0 ' instantiating '/s01/app/19.0.0/crs/install/crsconfig_params'. Copying files for 'Database Configuration and Upgrade Assistants 19.0.0.0.0 ' setting environment variable 'ORACLE_SID' to '' instantiating '/s01/app/19.0.0/assistants/dbua/dbua.sbs'. deleting '/s01/app/19.0.0/assistants/dbua/dbua.sbs' Copying files for 'Oracle JVM 19.0.0.0.0 ' instantiating '/s01/app/19.0.0/javavm/install/update_javavm_binaries.pl'. Copying files for 'Oracle Internet Directory Client 19.0.0.0.0 ' instantiating '/s01/app/19.0.0/ldap/install/odisrvreg.orc'. instantiating '/s01/app/19.0.0/ldap/install/schemasync.orc'. deleting '/s01/app/19.0.0/ldap/install/odisrvreg.orc' deleting '/s01/app/19.0.0/ldap/install/schemasync.orc' instantiating '/s01/app/19.0.0/ldap/install/oidca.orc'. changing permissions for '/s01/app/19.0.0/bin/oidca' Copying files for 'Oracle Net Listener 19.0.0.0.0 ' Copying files for 'Oracle Database 19c 19.0.0.0.0 ' instantiating '/s01/app/19.0.0/rdbms/install/sbs/trcsess.sbs'. instantiating '/s01/app/19.0.0/bin/oradnfs_run.sh'. changing permissions for '/s01/app/19.0.0/bin/trcsess' instantiating '/s01/app/19.0.0/rdbms/install/rdbms/config.orc'. instantiating '/s01/app/19.0.0/rdbms/install/sbs/extusrupgrade.sbs'. deleting '/s01/app/19.0.0/rdbms/install/sbs/extusrupgrade.sbs' instantiating '/s01/app/19.0.0/rdbms/install/rootadd_rdbms.sh'. Copying files for 'Oracle Grid Infrastructure 19c 19.0.0.0.0 ' instantiating '/s01/app/19.0.0/crs/sbs/rootcrs.sbs'. changing permissions for '/s01/app/19.0.0/crs/install/rootcrs.sh' instantiating '/s01/app/19.0.0/crs/sbs/roothas.sbs'. changing permissions for '/s01/app/19.0.0/crs/install/roothas.sh' touching '/s01/app/19.0.0/lib/ldflags' touching '/s01/app/19.0.0/lib/libagtsh.so' touching '/s01/app/19.0.0/lib/nnfgt.o' setting environment variable 'LD_LIBRARY_PATH' to '/s01/app/19.0.0/lib' setting environment variable 'LD_LIBRARY_PATH' to '/s01/app/19.0.0/network/lib' setting environment variable 'ORACLE_BASE' to '/s01/app/grid' Linking in progress Linking 'Java Development Kit 1.8.0.201.0 ' Linking 'oracle.swd.oui.core.min 12.2.0.7.0 ' Linking 'Installer SDK Component 12.2.0.7.0 ' Linking 'Oracle One-Off Patch Installer 12.2.0.1.15 ' Linking 'Oracle Universal Installer 12.2.0.7.0 ' Linking 'Tomcat Container 19.0.0.0.0 ' Linking 'Oracle Grid Infrastructure Bundled Agents 19.0.0.0.0 ' Linking 'Oracle Core Required Support Files for Core DB 19.0.0.0.0 ' Linking 'Oracle Grid Management Database 19.0.0.0.0 ' Linking 'Oracle USM Deconfiguration 19.0.0.0.0 ' Linking 'Oracle DBCA Deconfiguration 19.0.0.0.0 ' Linking 'Oracle Quality of Service Management (Server) 19.0.0.0.0 ' Linking 'Universal Storage Manager Files 19.0.0.0.0 ' Linking 'Oracle Text Required Support Files 19.0.0.0.0 ' Linking 'Oracle JDBC Server Support Package 19.0.0.0.0 ' Linking 'Database Workspace Manager 19.0.0.0.0 ' Linking 'Hadoopcore Component 19.0.0.0.0 ' Linking 'Automatic Storage Management Assistant 19.0.0.0.0 ' Linking 'Oracle Database 19c Multimedia Files 19.0.0.0.0 ' Linking 'Oracle Multimedia Java Advanced Imaging 19.0.0.0.0 ' Linking 'Oracle Globalization Support 19.0.0.0.0 ' Linking 'Oracle Multimedia Locator Java Required Support Files 19.0.0.0.0 ' Linking 'Oracle Multimedia Locator RDBMS Files 19.0.0.0.0 ' Linking 'Oracle Bali Share 11.1.1.6.0 ' Linking 'Oracle Globalization Support For Core 19.0.0.0.0 ' Linking 'Expat libraries 2.0.1.0.4 ' Linking 'BLASLAPACK Component 19.0.0.0.0 ' Linking 'DB TOOLS Listener 19.0.0.0.0 ' Linking 'Oracle Database Deconfiguration 19.0.0.0.0 ' Linking 'RHP Files for Common 19.0.0.0.0 ' Linking 'RHP Files for DB 19.0.0.0.0 ' Linking 'Perl Modules 5.28.1.0.0 ' Linking 'Oracle JVM For Core 19.0.0.0.0 ' Linking 'Perl Interpreter 5.28.1.0.0 ' Linking 'Secure Socket Layer 19.0.0.0.0 ' Linking 'Oracle Universal Connection Pool 19.0.0.0.0 ' Linking 'Oracle JDBC/OCI Instant Client 19.0.0.0.0 ' Linking 'Oracle Multimedia Client Option 19.0.0.0.0 ' Linking 'Oracle Java Client 19.0.0.0.0 ' Linking 'Database Migration Assistant for Unicode 19.0.0.0.0 ' Linking 'Oracle JFC Extended Windowing Toolkit 11.1.1.6.0 ' Linking 'OLAP SQL Scripts 19.0.0.0.0 ' Linking 'PL/SQL Embedded Gateway 19.0.0.0.0 ' Linking 'Database SQL Scripts 19.0.0.0.0 ' Linking 'SSL Required Support Files for InstantClient 19.0.0.0.0 ' Linking 'LDAP Required Support Files 19.0.0.0.0 ' Linking 'SQL*Plus Files for Instant Client 19.0.0.0.0 ' Linking 'Oracle Net Required Support Files 19.0.0.0.0 ' Linking 'Oracle Notification Service for Instant Client 19.0.0.0.0 ' Linking 'Deinstallation Tool 19.0.0.0.0 ' Linking 'Oracle Locale Builder 19.0.0.0.0 ' Linking 'XML Parser for Java 19.0.0.0.0 ' Linking 'Oracle Security Developer Tools 19.0.0.0.0 ' Linking 'Oracle Wallet Manager 19.0.0.0.0 ' Linking 'Installation Plugin Files 19.0.0.0.0 ' Linking 'Platform Required Support Files 19.0.0.0.0 ' Linking 'Oracle Help Share Library 11.1.1.7.0 ' Linking 'Oracle Help for Java 11.1.1.7.0 ' Linking 'RDBMS Required Support Files for Instant Client 19.0.0.0.0 ' Linking 'Oracle Extended Windowing Toolkit 11.1.1.6.0 ' Linking 'oracle.swd.commonlogging 13.3.0.0.0 ' Linking 'Tracle File Analyzer 19.0.0.0.0 ' Linking 'oracle.swd.opatchautodb 12.2.0.1.5 ' Linking 'Oracle LDAP administration 19.0.0.0.0 ' Linking 'RHP Files for CRS 19.0.0.0.0 ' Linking 'Cluster Verification Utility Files 19.0.0.0.0 ' Linking 'RDBMS Required Support Files 19.0.0.0.0 ' Linking 'Cluster Verification Utility CRS Files 19.0.0.0.0 ' Linking 'Cluster Verification Utility Common Files 19.0.0.0.0 ' Linking 'Oracle Clusterware RDBMS Files 19.0.0.0.0 ' Linking 'Oracle Ice Browser 11.1.1.7.0 ' Linking 'Oracle Globalization Support 19.0.0.0.0 ' Linking 'Buildtools Common Files 19.0.0.0.0 ' Linking 'Oracle RAC Required Support Files-HAS 19.0.0.0.0 ' Linking 'SQL*Plus Required Support Files 19.0.0.0.0 ' Linking 'XDK Required Support Files 19.0.0.0.0 ' Linking 'Parser Generator Required Support Files 19.0.0.0.0 ' Linking 'Precompiler Required Support Files 19.0.0.0.0 ' Linking 'Installation Common Files 19.0.0.0.0 ' Linking 'Oracle Core Required Support Files 19.0.0.0.0 ' Linking 'HAS Common Files 19.0.0.0.0 ' Linking 'Required Support Files 19.0.0.0.0 ' Linking 'Oracle JDBC/THIN Interfaces 19.0.0.0.0 ' Linking 'Oracle Multimedia Locator 19.0.0.0.0 ' Linking 'Assistant Common Files 19.0.0.0.0 ' Linking 'Oracle Multimedia 19.0.0.0.0 ' Linking 'Oracle Netca Client 19.0.0.0.0 ' Linking 'Oracle Net 19.0.0.0.0 ' Linking 'HAS Files for DB 19.0.0.0.0 ' Linking 'PL/SQL 19.0.0.0.0 ' Linking 'Oracle Recovery Manager 19.0.0.0.0 ' Linking 'Oracle Database Utilities 19.0.0.0.0 ' Linking 'Oracle Notification Service 19.0.0.0.0 ' Linking 'SQL*Plus 19.0.0.0.0 ' Linking 'Oracle Advanced Security 19.0.0.0.0 ' Linking 'Cluster Ready Services Files 19.0.0.0.0 ' Linking 'Database Configuration and Upgrade Assistants 19.0.0.0.0 ' Linking 'Oracle JVM 19.0.0.0.0 ' Linking 'Oracle Internet Directory Client 19.0.0.0.0 ' Linking 'Oracle Net Listener 19.0.0.0.0 ' Linking 'Oracle Database 19c 19.0.0.0.0 ' Linking 'Oracle Grid Infrastructure 19c 19.0.0.0.0 ' Linking in progress Preparing the home for Linking Linking ldflags Executables Building ASM Client Shared Libraries Linking Netclient Executables Linking RACG Executables Linking Netclient Executables Linking PLSQL Executables Linking RDBMS Executables Building Network Install Executables Linking UDP Linking SQLPLUS Executables Linking OID Client Linking Adapters Linking RDBMS Executables Linking Utilities Executables Linking Adapters Linking netclient Install Executables Linking Netclient Executables Linking RMAN Utility Setup in progress Setting up 'Java Development Kit 1.8.0.201.0 ' spawning '/bin/chmod -R u+w /s01/app/19.0.0/jdk' Setting up 'oracle.swd.oui.core.min 12.2.0.7.0 ' Setting up 'Installer SDK Component 12.2.0.7.0 ' deleting '/s01/app/19.0.0/oui/bin/platform/linux64/oraparam.ini' deleting '/s01/app/19.0.0/oui/bin/platform/linux64/clusterparam.ini' deleting '/s01/app/19.0.0/oui/bin/platform' Setting up 'Oracle One-Off Patch Installer 12.2.0.1.15 ' deleting '/s01/app/19.0.0/OPatch/opatch.bat' deleting '/s01/app/19.0.0/OPatch/scripts/opatch_jvm_discovery.bat' deleting '/s01/app/19.0.0/OPatch/scripts/opatch_wls.bat' deleting '/s01/app/19.0.0/OPatch/datapatch.bat' deleting '/s01/app/19.0.0/OPatch/operr.bat' deleting '/s01/app/19.0.0/OPatch/jlib/oracle.opatchcore.classpath.windows.jar' deleting '/s01/app/19.0.0/OPatch/jlib/oracle.opatch.classpath.windows.jar' Setting up 'Oracle Universal Installer 12.2.0.7.0 ' Setting up 'Tomcat Container 19.0.0.0.0 ' Setting up 'Oracle Grid Infrastructure Bundled Agents 19.0.0.0.0 ' Setting up 'Oracle Core Required Support Files for Core DB 19.0.0.0.0 ' Setting up 'Oracle Grid Management Database 19.0.0.0.0 ' Setting up 'Oracle USM Deconfiguration 19.0.0.0.0 ' Setting up 'Oracle DBCA Deconfiguration 19.0.0.0.0 ' Setting up 'Oracle Quality of Service Management (Server) 19.0.0.0.0 ' Setting up 'Universal Storage Manager Files 19.0.0.0.0 ' Setting up 'Oracle Text Required Support Files 19.0.0.0.0 ' Setting up 'Oracle JDBC Server Support Package 19.0.0.0.0 ' Setting up 'Database Workspace Manager 19.0.0.0.0 ' Setting up 'Hadoopcore Component 19.0.0.0.0 ' Setting up 'Automatic Storage Management Assistant 19.0.0.0.0 ' Setting up 'Oracle Database 19c Multimedia Files 19.0.0.0.0 ' Setting up 'Oracle Multimedia Java Advanced Imaging 19.0.0.0.0 ' Setting up 'Oracle Globalization Support 19.0.0.0.0 ' Setting up 'Oracle Multimedia Locator Java Required Support Files 19.0.0.0.0 ' Setting up 'Oracle Multimedia Locator RDBMS Files 19.0.0.0.0 ' Setting up 'Oracle Bali Share 11.1.1.6.0 ' Setting up 'Oracle Globalization Support For Core 19.0.0.0.0 ' Setting up 'Expat libraries 2.0.1.0.4 ' Setting up 'BLASLAPACK Component 19.0.0.0.0 ' Setting up 'DB TOOLS Listener 19.0.0.0.0 ' Setting up 'Oracle Database Deconfiguration 19.0.0.0.0 ' Setting up 'RHP Files for Common 19.0.0.0.0 ' Setting up 'RHP Files for DB 19.0.0.0.0 ' Setting up 'Perl Modules 5.28.1.0.0 ' instantiating '/s01/app/19.0.0/perl/lib/site_perl/5.28.1/x86_64-linux-thread-multi/auto/DBD/Oracle/mk.pm'. Setting up 'Oracle JVM For Core 19.0.0.0.0 ' Setting up 'Perl Interpreter 5.28.1.0.0 ' instantiating '/s01/app/19.0.0/perl/bin/corelist'. instantiating '/s01/app/19.0.0/perl/bin/cpan'. instantiating '/s01/app/19.0.0/perl/bin/dbilogstrip'. instantiating '/s01/app/19.0.0/perl/bin/dbiprof'. instantiating '/s01/app/19.0.0/perl/bin/dbiproxy'. instantiating '/s01/app/19.0.0/perl/bin/enc2xs'. instantiating '/s01/app/19.0.0/perl/bin/h2ph'. instantiating '/s01/app/19.0.0/perl/bin/h2xs'. instantiating '/s01/app/19.0.0/perl/bin/instmodsh'. instantiating '/s01/app/19.0.0/perl/bin/json_pp'. instantiating '/s01/app/19.0.0/perl/bin/libnetcfg'. instantiating '/s01/app/19.0.0/perl/bin/perlbug'. instantiating '/s01/app/19.0.0/perl/bin/perldoc'. instantiating '/s01/app/19.0.0/perl/bin/perlivp'. instantiating '/s01/app/19.0.0/perl/bin/perlthanks'. instantiating '/s01/app/19.0.0/perl/bin/piconv'. instantiating '/s01/app/19.0.0/perl/bin/pl2pm'. instantiating '/s01/app/19.0.0/perl/bin/pod2html'. instantiating '/s01/app/19.0.0/perl/bin/pod2man'. instantiating '/s01/app/19.0.0/perl/bin/pod2text'. instantiating '/s01/app/19.0.0/perl/bin/pod2usage'. instantiating '/s01/app/19.0.0/perl/bin/podchecker'. instantiating '/s01/app/19.0.0/perl/bin/podselect'. instantiating '/s01/app/19.0.0/perl/bin/prove'. instantiating '/s01/app/19.0.0/perl/bin/ptar'. instantiating '/s01/app/19.0.0/perl/bin/ptardiff'. instantiating '/s01/app/19.0.0/perl/bin/ptargrep'. instantiating '/s01/app/19.0.0/perl/bin/shasum'. instantiating '/s01/app/19.0.0/perl/bin/splain'. instantiating '/s01/app/19.0.0/perl/bin/xsubpp'. instantiating '/s01/app/19.0.0/perl/lib/5.28.1/x86_64-linux-thread-multi/Config.pm'. instantiating '/s01/app/19.0.0/perl/lib/5.28.1/x86_64-linux-thread-multi/Config_heavy.pl'. instantiating '/s01/app/19.0.0/perl/lib/5.28.1/x86_64-linux-thread-multi/CORE/config.h'. instantiating '/s01/app/19.0.0/perl/lib/5.28.1/x86_64-linux-thread-multi/perllocal.pod'. Setting up 'Secure Socket Layer 19.0.0.0.0 ' Setting up 'Oracle Universal Connection Pool 19.0.0.0.0 ' Setting up 'Oracle JDBC/OCI Instant Client 19.0.0.0.0 ' Setting up 'Oracle Multimedia Client Option 19.0.0.0.0 ' Setting up 'Oracle Java Client 19.0.0.0.0 ' Setting up 'Database Migration Assistant for Unicode 19.0.0.0.0 ' Setting up 'Oracle JFC Extended Windowing Toolkit 11.1.1.6.0 ' Setting up 'OLAP SQL Scripts 19.0.0.0.0 ' Setting up 'PL/SQL Embedded Gateway 19.0.0.0.0 ' Setting up 'Database SQL Scripts 19.0.0.0.0 ' instantiating '/s01/app/19.0.0/rdbms/install/sbs/initmeta.sbs'. Setting up 'SSL Required Support Files for InstantClient 19.0.0.0.0 ' Setting up 'LDAP Required Support Files 19.0.0.0.0 ' Setting up 'SQL*Plus Files for Instant Client 19.0.0.0.0 ' Setting up 'Oracle Net Required Support Files 19.0.0.0.0 ' Setting up 'Oracle Notification Service for Instant Client 19.0.0.0.0 ' Setting up 'Deinstallation Tool 19.0.0.0.0 ' Setting up 'Oracle Locale Builder 19.0.0.0.0 ' Setting up 'XML Parser for Java 19.0.0.0.0 ' Setting up 'Oracle Security Developer Tools 19.0.0.0.0 ' Setting up 'Oracle Wallet Manager 19.0.0.0.0 ' Setting up 'Installation Plugin Files 19.0.0.0.0 ' Setting up 'Platform Required Support Files 19.0.0.0.0 ' Setting up 'Oracle Help Share Library 11.1.1.7.0 ' Setting up 'Oracle Help for Java 11.1.1.7.0 ' Setting up 'RDBMS Required Support Files for Instant Client 19.0.0.0.0 ' Setting up 'Oracle Extended Windowing Toolkit 11.1.1.6.0 ' Setting up 'oracle.swd.commonlogging 13.3.0.0.0 ' Setting up 'Tracle File Analyzer 19.0.0.0.0 ' Setting up 'oracle.swd.opatchautodb 12.2.0.1.5 ' Setting up 'Oracle LDAP administration 19.0.0.0.0 ' Setting up 'RHP Files for CRS 19.0.0.0.0 ' Setting up 'Cluster Verification Utility Files 19.0.0.0.0 ' Setting up 'RDBMS Required Support Files 19.0.0.0.0 ' creating link 'libclntsh.so.19.1' for '/s01/app/19.0.0/lib/libclntsh.so' creating link 'libclntshcore.so.19.1' for '/s01/app/19.0.0/lib/libclntshcore.so' Setting up 'Cluster Verification Utility CRS Files 19.0.0.0.0 ' Setting up 'Cluster Verification Utility Common Files 19.0.0.0.0 ' Setting up 'Oracle Clusterware RDBMS Files 19.0.0.0.0 ' Setting up 'Oracle Ice Browser 11.1.1.7.0 ' Setting up 'Oracle Globalization Support 19.0.0.0.0 ' Setting up 'Buildtools Common Files 19.0.0.0.0 ' Setting up 'Oracle RAC Required Support Files-HAS 19.0.0.0.0 ' Setting up 'SQL*Plus Required Support Files 19.0.0.0.0 ' Setting up 'XDK Required Support Files 19.0.0.0.0 ' Setting up 'Parser Generator Required Support Files 19.0.0.0.0 ' Setting up 'Precompiler Required Support Files 19.0.0.0.0 ' Setting up 'Installation Common Files 19.0.0.0.0 ' instantiating '/s01/app/19.0.0/deinstall/deinstall'. spawning '/bin/chmod -R 755 /s01/app/19.0.0/jdk/jre' changing permissions for '/s01/app/19.0.0/root.sh' changing permissions for '/s01/app/19.0.0/rootupgrade.sh' Setting up 'Oracle Core Required Support Files 19.0.0.0.0 ' Setting up 'HAS Common Files 19.0.0.0.0 ' Setting up 'Required Support Files 19.0.0.0.0 ' Setting up 'Oracle JDBC/THIN Interfaces 19.0.0.0.0 ' Setting up 'Oracle Multimedia Locator 19.0.0.0.0 ' Setting up 'Assistant Common Files 19.0.0.0.0 ' Setting up 'Oracle Multimedia 19.0.0.0.0 ' instantiating '/s01/app/19.0.0/ord/im/admin/imimdloc.sql'. instantiating '/s01/app/19.0.0/ord/admin/ordisys.sql'. instantiating '/s01/app/19.0.0/ord/im/admin/ordlib.sql'. Setting up 'Oracle Netca Client 19.0.0.0.0 ' Setting up 'Oracle Net 19.0.0.0.0 ' Setting up 'HAS Files for DB 19.0.0.0.0 ' touching '/s01/app/19.0.0/bin/ldapbind' touching '/s01/app/19.0.0/lib/ldflags' touching '/s01/app/19.0.0/lib/nnfgt.o' touching '/s01/app/19.0.0/rdbms/lib/config.o' touching '/s01/app/19.0.0/lib/libclntsh.so' touching '/s01/app/19.0.0/lib/libclntshcore.so' Setting up 'PL/SQL 19.0.0.0.0 ' Setting up 'Oracle Recovery Manager 19.0.0.0.0 ' Setting up 'Oracle Database Utilities 19.0.0.0.0 ' Setting up 'Oracle Notification Service 19.0.0.0.0 ' updating '/s01/app/19.0.0/opmn/conf/ons.config. instantiating '/s01/app/19.0.0/opmn/bin/onsctli'. instantiating '/s01/app/19.0.0/bin/onsctl'. instantiating '/s01/app/19.0.0/opmn/bin/onsctl'. Setting up 'SQL*Plus 19.0.0.0.0 ' Setting up 'Oracle Advanced Security 19.0.0.0.0 ' Setting up 'Cluster Ready Services Files 19.0.0.0.0 ' Setting up 'Database Configuration and Upgrade Assistants 19.0.0.0.0 ' instantiating '/s01/app/19.0.0/assistants/dbca/dbca.sbs'. deleting '/s01/app/19.0.0/assistants/dbca/dbca.sbs' instantiating '/s01/app/19.0.0/assistants/dbca/emca.sbs'. deleting '/s01/app/19.0.0/assistants/dbca/emca.sbs' Setting up 'Oracle JVM 19.0.0.0.0 ' Setting up 'Oracle Internet Directory Client 19.0.0.0.0 ' Setting up 'Oracle Net Listener 19.0.0.0.0 ' instantiating '/s01/app/19.0.0/network/install/sqlnet/setowner.sh'. Setting up 'Oracle Database 19c 19.0.0.0.0 ' instantiating '/s01/app/19.0.0/rdbms/install/rootadd_filemap.sh'. updating '/s01/app/19.0.0/root.sh. updating '/s01/app/19.0.0/root.sh. updating '/s01/app/19.0.0/rootupgrade.sh. updating '/s01/app/19.0.0/rootupgrade.sh. instantiating '/s01/app/19.0.0/rdbms/install/config_filemap.sbs'. instantiating '/s01/app/19.0.0/demo/schema/mk_dir.sql.sbs'. Setting up 'Oracle Grid Infrastructure 19c 19.0.0.0.0 ' instantiating '/s01/app/19.0.0/crs/config/rootconfig.sbs'. updating '/s01/app/19.0.0/root.sh. updating '/s01/app/19.0.0/root.sh. updating '/s01/app/19.0.0/rootupgrade.sh. updating '/s01/app/19.0.0/rootupgrade.sh. updating '/s01/app/19.0.0/install/cluster.ini. Saving inventory Starting 'Prepare for configuration steps' Completed 'Prepare for configuration steps' Copying /s01/app/19.0.0 to remote nodes Loading products list. Please wait. Instantiating root scripts for add node Saving Cluster Inventory Starting 'Update Inventory' Completed 'Update Inventory' Starting 'Oracle Net Configuration Assistant' Parsing command line arguments:     Parameter "orahome" = /s01/app/19.0.0     Parameter "instype" = typical     Parameter "inscomp" = client,oraclenet,javavm,server,ano     Parameter "insprtcl" = tcp     Parameter "cfg" = local     Parameter "authadp" = NO_VALUE     Parameter "responsefile" = /s01/app/19.0.0/network/install/netca_typ.rsp     Parameter "silent" = true     Parameter "orahnam" = OraGI19Home1     Parameter "ouiinternal" = true Done parsing command line arguments. Oracle Net Services Configuration: Profile configuration complete. Profile configuration complete. Listener "LISTENER" already exists. Oracle Net Services configuration successful. The exit code is 0 Completed 'Oracle Net Configuration Assistant' Starting 'Automatic Storage Management Configuration Assistant' SYS_PASSWORD_PROMPT ASMSNMP_PASSWORD_PROMPT Post configuration completed successfully Completed 'Automatic Storage Management Configuration Assistant' Starting 'Oracle Cluster Verification Utility' Verifying Node Connectivity ...   Verifying Hosts File ...PASSED   Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED   Verifying subnet mask consistency for subnet "180.18.13.0" ...PASSED   Verifying subnet mask consistency for subnet "180.18.10.0" ...PASSED Verifying Node Connectivity ...PASSED Verifying Multicast or broadcast check ...PASSED Verifying Time zone consistency ...PASSED Verifying Cluster Manager Integrity ...PASSED Verifying User Mask ...PASSED Verifying Cluster Integrity ...PASSED Verifying OCR Integrity ...PASSED Verifying CRS Integrity ...   Verifying Clusterware Version Consistency ...PASSED Verifying CRS Integrity ...PASSED Verifying Node Application Existence ...PASSED Verifying Single Client Access Name (SCAN) ...   Verifying DNS/NIS name service 'otest-cluster-scan' ...     Verifying Name Service Switch Configuration File Integrity ...PASSED   Verifying DNS/NIS name service 'otest-cluster-scan' ...FAILED (PRVG-1101) Verifying Single Client Access Name (SCAN) ...FAILED (PRVG-11372, PRVG-1101) Verifying OLR Integrity ...PASSED Verifying Voting Disk ...PASSED Verifying ASM Integrity ...PASSED Verifying ASM disk group free space ...PASSED Verifying User Not In Group "root": grid ...PASSED Verifying Clock Synchronization ...PASSED Verifying VIP Subnet configuration check ...PASSED Verifying Network configuration consistency checks ...PASSED Verifying File system mount options for path GI_HOME ...PASSED Verifying Access control attributes for /var/tmp/.oracle ...PASSED Post-check for cluster services setup was unsuccessful.  Checks did not pass for the following nodes: otest2,otest1 Failures were encountered during execution of CVU verification request "stage -post crsinst". Verifying Single Client Access Name (SCAN) ...FAILED PRVG-11372 : Number of SCAN IP addresses that SCAN "otest-cluster-scan" resolved to did not match the number of SCAN VIP resources   Verifying DNS/NIS name service 'otest-cluster-scan' ...FAILED   PRVG-1101 : SCAN name "otest-cluster-scan" failed to resolve CVU operation performed:      stage -post crsinst Date:                         May 4, 2020 4:41:47 PM CVU home:                     /s01/app/19.0.0/ User:                         grid Setup completed with overall status as Failed Oracle Cluster Verification Utility failed. [grid@otest1 19.0.0]$ ./gridSetup.sh ERROR: Unable to verify the graphical display setup. This application requires X display. Make sure that xdpyinfo exist under PATH variable. Launching Oracle Grid Infrastructure Setup Wizard... The response file for this session can be found at:  /s01/app/19.0.0/install/response/grid_2020-05-04_04-01-08PM.rsp You can find the log of this install session at:  /tmp/GridSetupActions2020-05-04_04-01-08PM/gridSetupActions2020-05-04_04-01-08PM.log Moved the install session logs to:  /s01/app/oraInventory/logs/GridSetupActions2020-05-04_04-01-08PM [grid@otest1 19.0.0]$ [Clone API Thread] [ 2020-05-04 16:19:48.157 HKT ] [Version.isPre:757]  version to be checked 19.0.0.0.0 major version to check against 10 [Clone API Thread] [ 2020-05-04 16:19:48.158 HKT ] [Version.isPre:768]  isPre.java: Returning FALSE [Clone API Thread] [ 2020-05-04 16:19:48.158 HKT ] [UnixSystem.getCSSConfigType:3367]  configFile=/etc/oracle/ocr.loc [Clone API Thread] [ 2020-05-04 16:19:48.158 HKT ] [UnixSystem.getCSSConfigType:3411]  configType=null [Clone API Thread] [ 2020-05-04 16:19:48.160 HKT ] [QueryCluster.<init>:56]  No Cluster detected [Clone API Thread] [ 2020-05-04 16:19:48.160 HKT ] [QueryCluster.isCluster:65]  Cluster existence check = false [root@otest1 GridSetupActions2020-05-04_04-01-08PM]# tail -100 gridSetupActions2020-05-04_04-01-08PM.log INFO:  [May 4, 2020 4:41:36 PM] Started Plugin named: asmca INFO:  [May 4, 2020 4:41:36 PM] Found associated job INFO:  [May 4, 2020 4:41:36 PM] Starting 'Automatic Storage Management Configuration Assistant' INFO:  [May 4, 2020 4:41:36 PM] Starting 'Automatic Storage Management Configuration Assistant' INFO:  [May 4, 2020 4:41:36 PM] Executing ASMCA INFO:  [May 4, 2020 4:41:36 PM] Command /s01/app/19.0.0/bin/asmca -silent -postConfigureASM -oui_internal INFO:  [May 4, 2020 4:41:36 PM] ... GenericInternalPlugIn.handleProcess() entered. INFO:  [May 4, 2020 4:41:36 PM] ... GenericInternalPlugIn: getting configAssistantParmas. INFO:  [May 4, 2020 4:41:36 PM] ... GenericInternalPlugIn: checking secretArguments. INFO:  [May 4, 2020 4:41:36 PM] ... GenericInternalPlugIn: starting read loop. INFO:  [May 4, 2020 4:41:41 PM] SYS_PASSWORD_PROMPT INFO:  [May 4, 2020 4:41:41 PM] Processing: SYS_PASSWORD_PROMPT for argument tag -sysAsmPassword INFO:  [May 4, 2020 4:41:41 PM] Skipping line******* INFO:  [May 4, 2020 4:41:41 PM] ASMSNMP_PASSWORD_PROMPT INFO:  [May 4, 2020 4:41:41 PM] Processing: ASMSNMP_PASSWORD_PROMPT for argument tag -asmMonitorPassword INFO:  [May 4, 2020 4:41:41 PM] End of argument passing to stdin INFO:  [May 4, 2020 4:41:41 PM] Skipping line:********* INFO:  [May 4, 2020 4:41:44 PM] Skipping line: INFO:  [May 4, 2020 4:41:44 PM] Post configuration completed successfully INFO:  [May 4, 2020 4:41:44 PM] Skipping line: Post configuration completed successfully INFO:  [May 4, 2020 4:41:44 PM] Skipping line: INFO:  [May 4, 2020 4:41:45 PM] Completed Plugin named: asmca INFO:  [May 4, 2020 4:41:45 PM] ConfigClient.saveSession method called INFO:  [May 4, 2020 4:41:45 PM] Completed 'Automatic Storage Management Configuration Assistant' INFO:  [May 4, 2020 4:41:45 PM] Completed 'Automatic Storage Management Configuration Assistant' INFO:  [May 4, 2020 4:41:45 PM] Started Plugin named: cvu INFO:  [May 4, 2020 4:41:45 PM] Found associated job INFO:  [May 4, 2020 4:41:45 PM] Starting 'Oracle Cluster Verification Utility' INFO:  [May 4, 2020 4:41:45 PM] Starting 'Oracle Cluster Verification Utility' INFO:  [May 4, 2020 4:41:45 PM] Executing CLUVFY INFO:  [May 4, 2020 4:41:45 PM] Command /s01/app/19.0.0/bin/cluvfy stage -post crsinst -collect cluster -n all INFO:  [May 4, 2020 4:41:45 PM] ... GenericInternalPlugIn.handleProcess() entered. INFO:  [May 4, 2020 4:41:45 PM] ... GenericInternalPlugIn: getting configAssistantParmas. INFO:  [May 4, 2020 4:41:45 PM] ... GenericInternalPlugIn: checking secretArguments. INFO:  [May 4, 2020 4:41:45 PM] No arguments to pass to stdin INFO:  [May 4, 2020 4:41:45 PM] ... GenericInternalPlugIn: starting read loop. INFO:  [May 4, 2020 4:42:15 PM] Verifying Node Connectivity ... INFO:  [May 4, 2020 4:42:16 PM]   Verifying Hosts File ...PASSED INFO:  [May 4, 2020 4:42:18 PM]   Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED INFO:  [May 4, 2020 4:42:18 PM]   Verifying subnet mask consistency for subnet "180.18.13.0" ...PASSED INFO:  [May 4, 2020 4:42:18 PM]   Verifying subnet mask consistency for subnet "180.18.10.0" ...PASSED INFO:  [May 4, 2020 4:42:18 PM] Verifying Node Connectivity ...PASSED INFO:  [May 4, 2020 4:42:20 PM] Verifying Multicast or broadcast check ...PASSED INFO:  [May 4, 2020 4:42:20 PM] Verifying Time zone consistency ...PASSED INFO:  [May 4, 2020 4:42:21 PM] Verifying Cluster Manager Integrity ...PASSED INFO:  [May 4, 2020 4:42:21 PM] Verifying User Mask ...PASSED INFO:  [May 4, 2020 4:42:22 PM] Verifying Cluster Integrity ...PASSED INFO:  [May 4, 2020 4:42:28 PM] Verifying OCR Integrity ...PASSED INFO:  [May 4, 2020 4:42:29 PM] Verifying CRS Integrity ... INFO:  [May 4, 2020 4:42:29 PM]   Verifying Clusterware Version Consistency ...PASSED INFO:  [May 4, 2020 4:42:31 PM] Verifying CRS Integrity ...PASSED INFO:  [May 4, 2020 4:42:32 PM] Verifying Node Application Existence ...PASSED INFO:  [May 4, 2020 4:42:35 PM] Verifying Single Client Access Name (SCAN) ... INFO:  [May 4, 2020 4:42:35 PM]   Verifying DNS/NIS name service 'otest-cluster-scan' ... INFO:  [May 4, 2020 4:42:35 PM]     Verifying Name Service Switch Configuration File Integrity ...PASSED INFO:  [May 4, 2020 4:43:16 PM]   Verifying DNS/NIS name service 'otest-cluster-scan' ...FAILED (PRVG-1101) INFO:  [May 4, 2020 4:43:16 PM] Verifying Single Client Access Name (SCAN) ...FAILED (PRVG-11372, PRVG-1101) INFO:  [May 4, 2020 4:43:19 PM] Verifying OLR Integrity ...PASSED INFO:  [May 4, 2020 4:43:29 PM] Verifying Voting Disk ...PASSED INFO:  [May 4, 2020 4:43:37 PM] Verifying ASM Integrity ...PASSED INFO:  [May 4, 2020 4:43:41 PM] Verifying ASM disk group free space ...PASSED INFO:  [May 4, 2020 4:43:41 PM] Verifying User Not In Group "root": grid ...PASSED INFO:  [May 4, 2020 4:43:42 PM] Verifying Clock Synchronization ...PASSED INFO:  [May 4, 2020 4:43:45 PM] Verifying VIP Subnet configuration check ...PASSED INFO:  [May 4, 2020 4:43:45 PM] Verifying Network configuration consistency checks ...PASSED INFO:  [May 4, 2020 4:43:45 PM] Verifying File system mount options for path GI_HOME ...PASSED INFO:  [May 4, 2020 4:43:46 PM] Verifying Access control attributes for /var/tmp/.oracle ...PASSED INFO:  [May 4, 2020 4:43:46 PM] Post-check for cluster services setup was unsuccessful. INFO:  [May 4, 2020 4:43:46 PM] Checks did not pass for the following nodes: INFO:  [May 4, 2020 4:43:46 PM] otest2,otest1 INFO:  [May 4, 2020 4:43:46 PM] Failures were encountered during execution of CVU verification request "stage -post crsinst". INFO:  [May 4, 2020 4:43:46 PM] Verifying Single Client Access Name (SCAN) ...FAILED INFO:  [May 4, 2020 4:43:46 PM] PRVG-11372 : Number of SCAN IP addresses that SCAN "otest-cluster-scan" INFO:  [May 4, 2020 4:43:46 PM] resolved to did not match the number of SCAN VIP resources INFO:  [May 4, 2020 4:43:46 PM]   Verifying DNS/NIS name service 'otest-cluster-scan' ...FAILED INFO:  [May 4, 2020 4:43:46 PM]   PRVG-1101 : SCAN name "otest-cluster-scan" failed to resolve INFO:  [May 4, 2020 4:43:46 PM] CVU operation performed:      stage -post crsinst INFO:  [May 4, 2020 4:43:46 PM] Date:                         May 4, 2020 4:41:47 PM INFO:  [May 4, 2020 4:43:46 PM] CVU home:                     /s01/app/19.0.0/ INFO:  [May 4, 2020 4:43:46 PM] User:                         grid INFO:  [May 4, 2020 4:43:46 PM] Completed Plugin named: cvu INFO:  [May 4, 2020 4:43:46 PM] Setup completed with overall status as Failed INFO:  [May 4, 2020 4:45:02 PM] ConfigClient.saveSession method called INFO:  [May 4, 2020 4:45:02 PM] Oracle Cluster Verification Utility failed. INFO:  [May 4, 2020 4:45:02 PM] Oracle Cluster Verification Utility failed. INFO:  [May 4, 2020 4:45:02 PM] ConfigClient.executeSelectedToolsInAggregate action performed INFO:  [May 4, 2020 4:45:02 PM] Exiting ConfigClient.executeSelectedToolsInAggregate method INFO:  [May 4, 2020 4:45:02 PM] Adding ExitStatus SUCCESS_MINUS_RECTOOL to the exit status set INFO:  [May 4, 2020 4:45:02 PM] ConfigClient.saveSession method called INFO:  [May 4, 2020 4:45:02 PM] Calling event ConfigSessionEnding INFO:  [May 4, 2020 4:45:02 PM] ConfigClient.endSession method called INFO:  [May 4, 2020 4:45:02 PM] Completed Configuration INFO:  [May 4, 2020 4:45:02 PM] Adding ExitStatus FAILURE to the exit status set INFO:  [May 4, 2020 4:47:43 PM] Finding the most appropriate exit status for the current application INFO:  [May 4, 2020 4:47:43 PM] inventory location is/s01/app/oraInventory INFO:  [May 4, 2020 4:47:43 PM] Adding ExitStatus SUCCESS_WITH_WARNINGS to the exit status set INFO:  [May 4, 2020 4:47:43 PM] Finding the most appropriate exit status for the current application INFO:  [May 4, 2020 4:47:43 PM] Exit Status is -1 INFO:  [May 4, 2020 4:47:43 PM] Shutdown Oracle Grid Infrastructure 19c Installer [root@otest2 19.0.0]# ps -ef|grid -bash: grid: command not found [root@otest2 19.0.0]# ps -ef|grep grid grid      4264     1  0 15:55 ?        00:00:00 /usr/bin/ssh-agent -s grid     18211     1  0 16:39 ?        00:00:05 /s01/app/19.0.0/bin/oraagent.bin grid     18233     1  0 16:39 ?        00:00:02 /s01/app/19.0.0/bin/mdnsd.bin grid     18235     1  1 16:39 ?        00:00:06 /s01/app/19.0.0/bin/evmd.bin grid     18264     1  0 16:39 ?        00:00:03 /s01/app/19.0.0/bin/gpnpd.bin grid     18317 18235  0 16:39 ?        00:00:02 /s01/app/19.0.0/bin/evmlogger.bin -o /s01/app/19.0.0/log/[HOSTNAME]/evmd/evmlogger.info -l /s01/app/19.0.0/log/[HOSTNAME]/evmd/evmlogger.log grid     18334     1  1 16:39 ?        00:00:07 /s01/app/19.0.0/bin/gipcd.bin grid     18414     1  2 16:39 ?        00:00:12 /s01/app/19.0.0/bin/ocssd.bin  -S 2 grid     18781     1  2 16:40 ?        00:00:10 /s01/app/19.0.0/bin/oraagent.bin grid     18893     1  0 16:40 ?        00:00:04 /s01/app/19.0.0/jdk/bin/java -server -Xms30M -Xmx512M -Djava.awt.headless=true -Ddisable.checkForUpdate=true -DTRACING.ENABLED=false -XX:ParallelGCThreads=1 -cp /s01/app/19.0.0/jlib/cha.jar:/s01/app/19.0.0/jlib/chaconfig.jar:/s01/app/19.0.0/jlib/cha-diag-msg.jar:/s01/app/19.0.0/jlib/clsce.jar:/s01/app/19.0.0/jlib/srvm.jar:/s01/app/19.0.0/jlib/srvmhas.jar:/s01/app/19.0.0/jlib/srvmasm.jar:/s01/app/19.0.0/jlib/netcfg.jar:/s01/app/19.0.0/jdbc/lib/ojdbc8.jar:/s01/app/19.0.0/ucp/lib/ucp.jar:/s01/app/19.0.0/jlib/fte.jar:/s01/app/19.0.0/jlib/jai_core.jar:/s01/app/19.0.0/jlib/mlibwrapper_jai.jar:/s01/app/19.0.0/jlib/vecmath.jar:/s01/app/19.0.0/jlib/jai_codec.jar:/s01/app/19.0.0/jlib/jh.jar oracle.cha.server.CHADDriver grid     18895     1  0 16:40 ?        00:00:00 /s01/app/19.0.0/bin/tnslsnr ASMNET1LSNR_ASM -no_crs_notify -inherit grid     18927     1  0 16:40 ?        00:00:00 /s01/app/19.0.0/opmn/bin/ons -d grid     19020 18927  0 16:40 ?        00:00:00 /s01/app/19.0.0/opmn/bin/ons -d grid     19101     1  0 16:40 ?        00:00:00 asm_pmon_+ASM2 grid     19103     1  0 16:40 ?        00:00:00 asm_clmn_+ASM2 grid     19105     1  0 16:40 ?        00:00:00 asm_psp0_+ASM2 grid     19108     1  2 16:40 ?        00:00:10 asm_vktm_+ASM2 grid     19112     1  0 16:40 ?        00:00:00 asm_gen0_+ASM2 grid     19114     1  0 16:40 ?        00:00:00 asm_mman_+ASM2 grid     19118     1  0 16:40 ?        00:00:00 asm_gen1_+ASM2 grid     19121     1  0 16:40 ?        00:00:00 asm_diag_+ASM2 grid     19123     1  0 16:40 ?        00:00:00 asm_ping_+ASM2 grid     19125     1  0 16:40 ?        00:00:00 asm_pman_+ASM2 grid     19127     1  0 16:40 ?        00:00:03 asm_dia0_+ASM2 grid     19129     1  0 16:40 ?        00:00:01 asm_lmon_+ASM2 grid     19132     1  0 16:40 ?        00:00:01 asm_lmd0_+ASM2 grid     19135     1  0 16:40 ?        00:00:01 asm_lms0_+ASM2 grid     19137     1  0 16:40 ?        00:00:01 asm_lmhb_+ASM2 grid     19139     1  0 16:40 ?        00:00:00 asm_lck1_+ASM2 grid     19144     1  0 16:40 ?        00:00:00 asm_dbw0_+ASM2 grid     19146     1  0 16:40 ?        00:00:00 asm_lgwr_+ASM2 grid     19148     1  0 16:40 ?        00:00:00 asm_ckpt_+ASM2 grid     19150     1  0 16:40 ?        00:00:00 asm_smon_+ASM2 grid     19152     1  0 16:40 ?        00:00:00 asm_lreg_+ASM2 grid     19155     1  0 16:40 ?        00:00:00 asm_pxmn_+ASM2 grid     19157     1  0 16:40 ?        00:00:00 asm_rbal_+ASM2 grid     19159     1  0 16:40 ?        00:00:00 asm_gmon_+ASM2 grid     19161     1  0 16:40 ?        00:00:00 asm_mmon_+ASM2 grid     19163     1  0 16:40 ?        00:00:00 asm_mmnl_+ASM2 grid     19165     1  0 16:40 ?        00:00:01 asm_imr0_+ASM2 grid     19168     1  0 16:40 ?        00:00:00 asm_scm0_+ASM2 grid     19170     1  0 16:40 ?        00:00:02 asm_gcr0_+ASM2 grid     19172     1  0 16:40 ?        00:00:00 asm_lck0_+ASM2 grid     19189     1  0 16:40 ?        00:00:00 oracle+ASM2_ocr (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) grid     19193     1  0 16:40 ?        00:00:00 asm_asmb_+ASM2 grid     19196     1  0 16:40 ?        00:00:00 oracle+ASM2_asmb_+asm2 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) grid     19204     1  0 16:40 ?        00:00:01 asm_ppa7_+ASM2 grid     19283     1  0 16:40 ?        00:00:00 oracle+ASM2 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) grid     19443     1  0 16:40 ?        00:00:00 /s01/app/19.0.0/bin/tnslsnr LISTENER -no_crs_notify -inherit root     26412 15284  0 16:49 pts/0    00:00:00 grep --color=auto grid [root@otest2 19.0.0]# ssh otest1 The authenticity of host 'otest1 (180.18.13.241)' can't be established. ECDSA key fingerprint is d4:e5:0f:89:6d:77:a0:26:54:17:20:2a:0c:98:3c:48. Are you sure you want to continue connecting (yes/no)? ^C [root@otest2 19.0.0]# exit logout Connection to 180.18.13.243 closed. [root@unicomser-b ~]# ssh 180.18.13.241 root@180.18.13.241's password: Last login: Mon May  4 16:41:58 2020 [root@otest1 ~]# ps -ef|grep grid grid       649     1  0 16:35 ?        00:00:00 /s01/app/19.0.0/bin/tnslsnr LISTENER -no_crs_notify -inherit grid       712     1  0 16:35 ?        00:00:04 /s01/app/19.0.0/jdk/bin/java -Xms128M -Xmx384M -classpath /s01/app/19.0.0/jdk/jre/lib/rt.jar:/s01/app/19.0.0/jdk/jre/lib/i18n.jar:/s01/app/19.0.0/jlib/jagent.jar:/s01/app/19.0.0/jlib/srvm.jar: -Dora.jagent.conf=conf/orajagent.conf oracle.jagent.core.OraJavaAgentDriver grid      4832     1  0 15:55 ?        00:00:00 /usr/bin/ssh-agent -s root      5683  5668  0 16:00 pts/0    00:00:00 su - grid grid      5684  5683  0 16:00 pts/0    00:00:00 -bash grid      9476 30633  0 16:40 ?        00:00:00 /s01/app/19.0.0/opmn/bin/ons -d grid      9937     1  0 16:40 ?        00:00:01 asm_ppa7_+ASM1 grid     18595     1  0 16:27 ?        00:00:00 /bin/sh ./OSWatcher.sh 30 48 NONE /s01/app/grid/tfa/repository/suptools/otest1/oswbb/grid/archive grid     20138 18595  0 16:27 ?        00:00:00 /bin/sh ./OSWatcherFM.sh 48 /s01/app/grid/tfa/repository/suptools/otest1/oswbb/grid/archive grid     22151     1  0 16:45 ?        00:00:00 oracle+ASM1_crf (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) grid     25815 18595  0 16:49 ?        00:00:00 sleep 30 grid     25874 20138  0 16:49 ?        00:00:00 sleep 60 root     25951 25923  0 16:49 pts/4    00:00:00 grep --color=auto grid grid     28203     1  0 16:31 ?        00:00:08 /s01/app/19.0.0/bin/oraagent.bin grid     28225     1  0 16:31 ?        00:00:03 /s01/app/19.0.0/bin/mdnsd.bin grid     28227     1  0 16:31 ?        00:00:08 /s01/app/19.0.0/bin/evmd.bin grid     28256     1  0 16:31 ?        00:00:04 /s01/app/19.0.0/bin/gpnpd.bin grid     28353 28227  0 16:32 ?        00:00:03 /s01/app/19.0.0/bin/evmlogger.bin -o /s01/app/19.0.0/log/[HOSTNAME]/evmd/evmlogger.info -l /s01/app/19.0.0/log/[HOSTNAME]/evmd/evmlogger.log grid     28370     1  0 16:32 ?        00:00:08 /s01/app/19.0.0/bin/gipcd.bin grid     28472     1  1 16:32 ?        00:00:17 /s01/app/19.0.0/bin/ocssd.bin  -S 1 grid     28893     1  0 16:32 ?        00:00:00 asm_pmon_+ASM1 grid     28895     1  0 16:32 ?        00:00:00 asm_clmn_+ASM1 grid     28897     1  0 16:32 ?        00:00:00 asm_psp0_+ASM1 grid     28899     1  1 16:32 ?        00:00:16 asm_vktm_+ASM1 grid     28903     1  0 16:32 ?        00:00:00 asm_gen0_+ASM1 grid     28905     1  0 16:32 ?        00:00:00 asm_mman_+ASM1 grid     28909     1  0 16:32 ?        00:00:00 asm_gen1_+ASM1 grid     28912     1  0 16:32 ?        00:00:01 asm_diag_+ASM1 grid     28914     1  0 16:32 ?        00:00:00 asm_ping_+ASM1 grid     28916     1  0 16:32 ?        00:00:00 asm_pman_+ASM1 grid     28918     1  0 16:32 ?        00:00:06 asm_dia0_+ASM1 grid     28920     1  0 16:32 ?        00:00:01 asm_lmon_+ASM1 grid     28922     1  0 16:32 ?        00:00:02 asm_lmd0_+ASM1 grid     28924     1  0 16:32 ?        00:00:04 asm_lms0_+ASM1 grid     28927     1  0 16:32 ?        00:00:01 asm_lmhb_+ASM1 grid     28931     1  0 16:32 ?        00:00:00 asm_lck1_+ASM1 grid     28933     1  0 16:32 ?        00:00:00 asm_dbw0_+ASM1 grid     28935     1  0 16:32 ?        00:00:00 asm_lgwr_+ASM1 grid     28937     1  0 16:32 ?        00:00:00 asm_ckpt_+ASM1 grid     28939     1  0 16:32 ?        00:00:00 asm_smon_+ASM1 grid     28941     1  0 16:32 ?        00:00:00 asm_lreg_+ASM1 grid     28943     1  0 16:32 ?        00:00:00 asm_pxmn_+ASM1 grid     28945     1  0 16:32 ?        00:00:00 asm_rbal_+ASM1 grid     28947     1  0 16:32 ?        00:00:00 asm_gmon_+ASM1 grid     28949     1  0 16:32 ?        00:00:00 asm_mmon_+ASM1 grid     28951     1  0 16:32 ?        00:00:00 asm_mmnl_+ASM1 grid     28953     1  0 16:32 ?        00:00:02 asm_imr0_+ASM1 grid     28955     1  0 16:32 ?        00:00:00 asm_scm0_+ASM1 grid     28957     1  0 16:32 ?        00:00:00 asm_lck0_+ASM1 grid     28964     1  0 16:32 ?        00:00:05 asm_gcr0_+ASM1 grid     29007     1  0 16:32 ?        00:00:00 asm_asmb_+ASM1 grid     29009     1  0 16:32 ?        00:00:00 oracle+ASM1_asmb_+asm1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) grid     29036     1  0 16:32 ?        00:00:00 oracle+ASM1_ocr (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) grid     30285     1  1 16:34 ?        00:00:14 /s01/app/19.0.0/bin/oraagent.bin grid     30311     1  0 16:34 ?        00:00:00 /s01/app/19.0.0/bin/tnslsnr ASMNET1LSNR_ASM -no_crs_notify -inherit grid     30365     1  0 16:34 ?        00:00:00 oracle+ASM1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) grid     30633     1  0 16:34 ?        00:00:00 /s01/app/19.0.0/opmn/bin/ons -d grid     30922     1  0 16:34 ?        00:00:00 /s01/app/19.0.0/bin/tnslsnr LISTENER_SCAN1 -no_crs_notify -inherit grid     31139     1  0 16:35 ?        00:00:02 /s01/app/19.0.0/bin/scriptagent.bin grid     31188     1  1 16:35 ?        00:00:12 /s01/app/19.0.0/jdk/bin/java -server -Xms128M -Xmx384M -Djava.awt.headless=true -Ddisable.checkForUpdate=true -Djava.util.logging.config.file=/s01/app/grid/crsdata/otest1/qos/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -DTRACING.ENABLED=false -Djava.rmi.server.hostname=otest1 -Doracle.http.port=8888 -Doracle.jmx.port=8895 -Doracle.tls.enabled=false -Doracle.jwc.tls.http.enabled=false -Djava.security.manager -Djava.security.policy=/s01/app/grid/crsdata/otest1/qos/conf/catalina.policy -Djava.security.egd=file:/dev/urandom -Dcatalina.home=/s01/app/19.0.0/tomcat -Dcatalina.base=/s01/app/grid/crsdata/otest1/qos -Djava.io.tmpdir=/s01/app/grid/crsdata/otest1/qos/temp -Doracle.home=/s01/app/19.0.0 -Doracle.jwc.mode=CLUSTERWARE -classpath /s01/app/19.0.0/jlib/cryptoj.jar:/s01/app/19.0.0/jlib/oraclepki.jar:/s01/app/19.0.0/jlib/osdt_core.jar:/s01/app/19.0.0/jlib/osdt_cert.jar:/s01/app/19.0.0/tomcat/lib/tomcat-juli.jar:/s01/app/19.0.0/tomcat/lib/bootstrap.jar:/s01/app/19.0.0/jlib/jwc-logging.jar org.apache.catalina.startup.Bootstrap start grid     31608     1  0 16:35 ?        00:00:03 /s01/app/19.0.0/jdk/bin/java -server -Xms30M -Xmx512M -Djava.awt.headless=true -Ddisable.checkForUpdate=true -DTRACING.ENABLED=false -XX:ParallelGCThreads=1 -cp /s01/app/19.0.0/jlib/cha.jar:/s01/app/19.0.0/jlib/chaconfig.jar:/s01/app/19.0.0/jlib/cha-diag-msg.jar:/s01/app/19.0.0/jlib/clsce.jar:/s01/app/19.0.0/jlib/srvm.jar:/s01/app/19.0.0/jlib/srvmhas.jar:/s01/app/19.0.0/jlib/srvmasm.jar:/s01/app/19.0.0/jlib/netcfg.jar:/s01/app/19.0.0/jdbc/lib/ojdbc8.jar:/s01/app/19.0.0/ucp/lib/ucp.jar:/s01/app/19.0.0/jlib/fte.jar:/s01/app/19.0.0/jlib/jai_core.jar:/s01/app/19.0.0/jlib/mlibwrapper_jai.jar:/s01/app/19.0.0/jlib/vecmath.jar:/s01/app/19.0.0/jlib/jai_codec.jar:/s01/app/19.0.0/jlib/jh.jar oracle.cha.server.CHADDriver [root@otest1 ~]# [root@otest1 ~]# su  - oracle Last login: Mon May  4 20:04:02 HKT 2020 on pts/4 [oracle@otest1 ~]$ sqlplus  / as sysdba SQL*Plus: Release 19.0.0.0.0 - Production on Mon May 4 20:06:31 2020 Version 19.3.0.0.0 Copyright (c) 1982, 2019, Oracle.  All rights reserved. Connected to: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.3.0.0.0 SQL> select instance_name,status from gv$instance; INSTANCE_NAME    STATUS ---------------- ------------ orcl1            OPEN orcl2            OPEN SQL> select name,open_mode from v$database; NAME      OPEN_MODE --------- -------------------- ORCL      READ WRITE SQL> select file_name from dba_data_files; FILE_NAME -------------------------------------------------------------------------------- +DATA/ORCL/DATAFILE/system.259.1039540983 +DATA/ORCL/DATAFILE/sysaux.260.1039541037 +DATA/ORCL/DATAFILE/undotbs1.261.1039541073 +DATA/ORCL/DATAFILE/users.262.1039541073 +DATA/ORCL/DATAFILE/undotbs2.271.1039541913 SQL> [oracle@otest1 ~]$ cat /etc/hosts 180.18.13.241   otest1   otest1.oracle.com 180.18.13.242   otest1v 180.18.13.243   otest2   otest2.oracle.com 180.18.13.244   otest2v 180.18.13.245   otest-cluster otest-cluster-scan 180.18.10.241   otest1p 180.18.10.242   otest2p 127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6 [oracle@otest1 ~]$ ifconfig -a -bash: ifconfig: command not found [oracle@otest1 ~]$ exit logout [root@otest1 ~]# ifconfig -a ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500         inet 180.18.13.241  netmask 255.255.255.0  broadcast 180.18.13.255         inet6 fe80::de63:afb3:dc56:f628  prefixlen 64  scopeid 0x20<link>         inet6 fe80::b40:1bd7:85b3:e603  prefixlen 64  scopeid 0x20<link>         ether 00:50:56:a6:f6:08  txqueuelen 1000  (Ethernet)         RX packets 1662706  bytes 3213695265 (2.9 GiB)         RX errors 0  dropped 0  overruns 0  frame 0         TX packets 751321  bytes 14403060075 (13.4 GiB)         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0 ens160:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500         inet 180.18.13.242  netmask 255.255.255.0  broadcast 180.18.13.255         ether 00:50:56:a6:f6:08  txqueuelen 1000  (Ethernet) ens160:2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500         inet 180.18.13.245  netmask 255.255.255.0  broadcast 180.18.13.255         ether 00:50:56:a6:f6:08  txqueuelen 1000  (Ethernet) ens192: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500         inet 180.18.10.241  netmask 255.255.255.0  broadcast 180.18.10.255         inet6 fe80::225b:e49b:4398:310a  prefixlen 64  scopeid 0x20<link>         ether 00:50:56:a6:65:86  txqueuelen 1000  (Ethernet)         RX packets 1327323  bytes 1554905180 (1.4 GiB)         RX errors 0  dropped 10  overruns 0  frame 0         TX packets 1048252  bytes 1023200050 (975.7 MiB)         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0 ens192:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500         inet 169.254.2.52  netmask 255.255.224.0  broadcast 169.254.31.255         ether 00:50:56:a6:65:86  txqueuelen 1000  (Ethernet) lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536         inet 127.0.0.1  netmask 255.0.0.0         inet6 ::1  prefixlen 128  scopeid 0x10<host>         loop  txqueuelen 1  (Local Loopback)         RX packets 260659  bytes 593182661 (565.7 MiB)         RX errors 0  dropped 0  overruns 0  frame 0         TX packets 260659  bytes 593182661 (565.7 MiB)         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0 [root@otest1 ~]# [grid@otest1 19.0.0]$ ls acfsccm    assistants   crf        demo         gridSetup.sh   jdk      nls          ords      QOpatch      root.sh.old.1   sqlplus   welcome.html acfsccreg  auth         crs        diagnostics  has            jlib     ohasd        oss       qos          root.sh.old.2   srvm      wlm acfscm     bin          css        dmu          hs             ldap     ologgerd     osysmond  racg         root.sh.old.3   suptools  wwg acfsiob    cdata        ctss       env.ora      install        lib      OPatch       oui       rdbms        rootupgrade.sh  tfa       xag acfsrd     cdp          cv         evm          instantclient  log      opmn         owm       relnotes     runcluvfy.sh    tomcat    xdk acfsrm     cfgtoollogs  dbjava     gipc         inventory      md       oracore      perl      rhp          sdk             ucp addnode    cha          dbs        gnsd         javavm         mdns     oraInst.loc  plsql     root.sh      slax            usm advmccb    clone        deinstall  gpnp         jdbc           network  ord          precomp   root.sh.old  sqlpatch        utl [grid@otest1 19.0.0]$ ./runcluvfy.sh  stage -post  crsinst -n "otest1,otest2"  -verbose Verifying Node Connectivity ...   Verifying Hosts File ...   Node Name                             Status   ------------------------------------  ------------------------   otest1                                passed   otest2                                passed   Verifying Hosts File ...PASSED Interface information for node "otest1"  Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU  ------ --------------- --------------- --------------- --------------- ----------------- ------  ens160 180.18.13.241   180.18.13.0     0.0.0.0         180.18.10.254   00:50:56:A6:F6:08 1500  ens160 180.18.13.242   180.18.13.0     0.0.0.0         180.18.10.254   00:50:56:A6:F6:08 1500  ens160 180.18.13.245   180.18.13.0     0.0.0.0         180.18.10.254   00:50:56:A6:F6:08 1500  ens192 180.18.10.241   180.18.10.0     0.0.0.0         180.18.10.254   00:50:56:A6:65:86 1500 Interface information for node "otest2"  Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU  ------ --------------- --------------- --------------- --------------- ----------------- ------  ens160 180.18.13.243   180.18.13.0     0.0.0.0         180.18.10.254   00:50:56:A6:49:04 1500  ens160 180.18.13.244   180.18.13.0     0.0.0.0         180.18.10.254   00:50:56:A6:49:04 1500  ens192 180.18.10.242   180.18.10.0     0.0.0.0         180.18.10.254   00:50:56:A6:34:B4 1500 Check: MTU consistency on the private interfaces of subnet "180.18.10.0"   Node              Name          IP Address    Subnet        MTU   ----------------  ------------  ------------  ------------  ----------------   otest1            ens192        180.18.10.241  180.18.10.0   1500   otest2            ens192        180.18.10.242  180.18.10.0   1500 Check: MTU consistency of the subnet "180.18.13.0".   Node              Name          IP Address    Subnet        MTU   ----------------  ------------  ------------  ------------  ----------------   otest1            ens160        180.18.13.241  180.18.13.0   1500   otest1            ens160        180.18.13.242  180.18.13.0   1500   otest1            ens160        180.18.13.245  180.18.13.0   1500   otest2            ens160        180.18.13.243  180.18.13.0   1500   otest2            ens160        180.18.13.244  180.18.13.0   1500   Source                          Destination                     Connected?   ------------------------------  ------------------------------  ----------------   otest1[ens160:180.18.13.241]    otest1[ens160:180.18.13.242]    yes   otest1[ens160:180.18.13.241]    otest1[ens160:180.18.13.245]    yes   otest1[ens160:180.18.13.241]    otest2[ens160:180.18.13.243]    yes   otest1[ens160:180.18.13.241]    otest2[ens160:180.18.13.244]    yes   otest1[ens160:180.18.13.242]    otest1[ens160:180.18.13.245]    yes   otest1[ens160:180.18.13.242]    otest2[ens160:180.18.13.243]    yes   otest1[ens160:180.18.13.242]    otest2[ens160:180.18.13.244]    yes   otest1[ens160:180.18.13.245]    otest2[ens160:180.18.13.243]    yes   otest1[ens160:180.18.13.245]    otest2[ens160:180.18.13.244]    yes   otest2[ens160:180.18.13.243]    otest2[ens160:180.18.13.244]    yes   Source                          Destination                     Connected?   ------------------------------  ------------------------------  ----------------   otest1[ens192:180.18.10.241]    otest2[ens192:180.18.10.242]    yes   Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED   Verifying subnet mask consistency for subnet "180.18.13.0" ...PASSED   Verifying subnet mask consistency for subnet "180.18.10.0" ...PASSED Verifying Node Connectivity ...PASSED Verifying Multicast or broadcast check ... Checking subnet "180.18.10.0" for multicast communication with multicast group "224.0.0.251" Verifying Multicast or broadcast check ...PASSED Verifying Time zone consistency ...PASSED Verifying Cluster Manager Integrity ...   Node Name                             Status   ------------------------------------  ------------------------   otest1                                running   otest2                                running Verifying Cluster Manager Integrity ...PASSED Verifying User Mask ...   Node Name     Available                 Required                  Comment   ------------  ------------------------  ------------------------  ----------   otest2        0022                      0022                      passed   otest1        0022                      0022                      passed Verifying User Mask ...PASSED Verifying Cluster Integrity ...   Node Name   ------------------------------------   otest1   otest2 Verifying Cluster Integrity ...PASSED Verifying OCR Integrity ...PASSED Verifying CRS Integrity ...   Verifying Clusterware Version Consistency ...PASSED Verifying CRS Integrity ...PASSED Verifying Node Application Existence ... Checking existence of VIP node application (required)   Node Name     Required                  Running?                  Comment   ------------  ------------------------  ------------------------  ----------   otest1        yes                       yes                       passed   otest2        yes                       yes                       passed Checking existence of NETWORK node application (required)   Node Name     Required                  Running?                  Comment   ------------  ------------------------  ------------------------  ----------   otest1        yes                       yes                       passed   otest2        yes                       yes                       passed Checking existence of ONS node application (optional)   Node Name     Required                  Running?                  Comment   ------------  ------------------------  ------------------------  ----------   otest1        no                        yes                       passed   otest2        no                        yes                       passed Verifying Node Application Existence ...PASSED Verifying Single Client Access Name (SCAN) ...   SCAN Name         Node          Running?      ListenerName  Port          Running?   ----------------  ------------  ------------  ------------  ------------  ------------   otest-cluster-scan                true          LISTENER_SCAN1  1521          true Checking TCP connectivity to SCAN listeners...   Node          ListenerName              TCP connectivity?   ------------  ------------------------  ------------------------   otest1        LISTENER_SCAN1            yes   Verifying DNS/NIS name service 'otest-cluster-scan' ...     Verifying Name Service Switch Configuration File Integrity ...PASSED   Verifying DNS/NIS name service 'otest-cluster-scan' ...FAILED (PRVG-1101) Verifying Single Client Access Name (SCAN) ...FAILED (PRVG-11372, PRVG-1101) Verifying OLR Integrity ...PASSED Verifying Voting Disk ...PASSED Verifying ASM Integrity ...PASSED Verifying ASM disk group free space ...PASSED Verifying User Not In Group "root": grid ...   Node Name     Status                    Comment   ------------  ------------------------  ------------------------   otest2        passed                    does not exist   otest1        passed                    does not exist Verifying User Not In Group "root": grid ...PASSED Verifying Clock Synchronization ...   Node Name                             Status   ------------------------------------  ------------------------   otest1                                passed   otest2                                passed   Node Name                             State   ------------------------------------  ------------------------   otest2                                Active   otest1                                Active   Node Name     Time Offset               Status   ------------  ------------------------  ------------------------   otest2        0.0                       passed   otest1        0.0                       passed Verifying Clock Synchronization ...PASSED Verifying VIP Subnet configuration check ...PASSED Verifying Network configuration consistency checks ...PASSED Verifying File system mount options for path GI_HOME ...PASSED Verifying Access control attributes for /var/tmp/.oracle ...PASSED Verifying Access control attributes for %OCRCONFIGDIR%/maps ...PASSED Post-check for cluster services setup was unsuccessful. Checks did not pass for the following nodes:         otest2,otest1 Failures were encountered during execution of CVU verification request "stage -post crsinst". Verifying Single Client Access Name (SCAN) ...FAILED PRVG-11372 : Number of SCAN IP addresses that SCAN "otest-cluster-scan" resolved to did not match the number of SCAN VIP resources   Verifying DNS/NIS name service 'otest-cluster-scan' ...FAILED   PRVG-1101 : SCAN name "otest-cluster-scan" failed to resolve CVU operation performed:      stage -post crsinst Date:                         May 4, 2020 8:41:35 PM CVU home:                     /s01/app/19.0.0/ User:                         grid [grid@otest1 19.0.0]$ [grid@otest1 19.0.0]$ cd bin [grid@otest1 bin]$ ./crsctl status res -t -------------------------------------------------------------------------------- Name           Target  State        Server                   State details -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.LISTENER.lsnr                ONLINE  ONLINE       otest1                   STABLE                ONLINE  ONLINE       otest2                   STABLE ora.chad                ONLINE  ONLINE       otest1                   STABLE                ONLINE  ONLINE       otest2                   STABLE ora.net1.network                ONLINE  ONLINE       otest1                   STABLE                ONLINE  ONLINE       otest2                   STABLE ora.ons                ONLINE  ONLINE       otest1                   STABLE                ONLINE  ONLINE       otest2                   STABLE -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)       1        ONLINE  ONLINE       otest1                   STABLE       2        ONLINE  ONLINE       otest2                   STABLE       3        OFFLINE OFFLINE                               STABLE ora.DATA.dg(ora.asmgroup)       1        ONLINE  ONLINE       otest1                   STABLE       2        ONLINE  ONLINE       otest2                   STABLE       3        OFFLINE OFFLINE                               STABLE ora.LISTENER_SCAN1.lsnr       1        ONLINE  ONLINE       otest1                   STABLE ora.asm(ora.asmgroup)       1        ONLINE  ONLINE       otest1                   Started,STABLE       2        ONLINE  ONLINE       otest2                   Started,STABLE       3        OFFLINE OFFLINE                               STABLE ora.asmnet1.asmnetwork(ora.asmgroup)       1        ONLINE  ONLINE       otest1                   STABLE       2        ONLINE  ONLINE       otest2                   STABLE       3        OFFLINE OFFLINE                               STABLE ora.cvu       1        ONLINE  ONLINE       otest1                   STABLE ora.orcl.db       1        ONLINE  ONLINE       otest1                   Open,HOME=/u01/oracl                                                              e/19.0.0,STABLE       2        ONLINE  ONLINE       otest2                   Open,HOME=/u01/oracl                                                              e/19.0.0,STABLE ora.otest1.vip       1        ONLINE  ONLINE       otest1                   STABLE ora.otest2.vip       1        ONLINE  ONLINE       otest2                   STABLE ora.qosmserver       1        ONLINE  ONLINE       otest1                   STABLE ora.scan1.vip       1        ONLINE  ONLINE       otest1                   STABLE -------------------------------------------------------------------------------- [grid@otest1 bin]$ [grid@otest1 bin]$ ./crsctl status res -t -init -------------------------------------------------------------------------------- Name           Target  State        Server                   State details -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.asm       1        ONLINE  ONLINE       otest1                   Started,STABLE ora.cluster_interconnect.haip       1        ONLINE  ONLINE       otest1                   STABLE ora.crf       1        ONLINE  ONLINE       otest1                   STABLE ora.crsd       1        ONLINE  ONLINE       otest1                   STABLE ora.cssd       1        ONLINE  ONLINE       otest1                   STABLE ora.cssdmonitor       1        ONLINE  ONLINE       otest1                   STABLE ora.ctssd       1        ONLINE  ONLINE       otest1                   ACTIVE:0,STABLE ora.diskmon       1        OFFLINE OFFLINE                               STABLE ora.evmd       1        ONLINE  ONLINE       otest1                   STABLE ora.gipcd       1        ONLINE  ONLINE       otest1                   STABLE ora.gpnpd       1        ONLINE  ONLINE       otest1                   STABLE ora.mdnsd       1        ONLINE  ONLINE       otest1                   STABLE ora.storage       1        ONLINE  ONLINE       otest1                   STABLE -------------------------------------------------------------------------------- [grid@otest1 bin]$ SQL> show con_name CON_NAME ------------------------------ CDB$ROOT SQL> show pdbs     CON_ID CON_NAME                       OPEN MODE  RESTRICTED ---------- ------------------------------ ---------- ----------          2 PDB$SEED                       READ ONLY  NO          3 PDB                            READ WRITE NO SQL> [oracle@otest1 ~]$ lsnrctl status LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 04-MAY-2020 23:29:19 Copyright (c) 1991, 2019, Oracle.  All rights reserved. Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521)) STATUS of the LISTENER ------------------------ Alias                     LISTENER Version                   TNSLSNR for Linux: Version 19.0.0.0.0 - Production Start Date                04-MAY-2020 16:35:47 Uptime                    0 days 6 hr. 53 min. 32 sec Trace Level               off Security                  ON: Local OS Authentication SNMP                      OFF Listener Parameter File   /s01/app/19.0.0/network/admin/listener.ora Listener Log File         /s01/app/grid/diag/tnslsnr/otest1/listener/alert/log.xml Listening Endpoints Summary...   (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))   (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=180.18.13.241)(PORT=1521)))   (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=180.18.13.242)(PORT=1521))) Services Summary... Service "+ASM" has 1 instance(s).   Instance "+ASM1", status READY, has 1 handler(s) for this service... Service "+ASM_DATA" has 1 instance(s).   Instance "+ASM1", status READY, has 1 handler(s) for this service... Service "86b637b62fdf7a65e053f706e80a27ca" has 1 instance(s).   Instance "orcl1", status READY, has 1 handler(s) for this service... Service "a4d0792695aa3407e053f10d10acd468" has 1 instance(s).   Instance "orcl1", status READY, has 1 handler(s) for this service... Service "orcl" has 1 instance(s).   Instance "orcl1", status READY, has 1 handler(s) for this service... Service "orclXDB" has 1 instance(s).   Instance "orcl1", status READY, has 1 handler(s) for this service... Service "pdb" has 1 instance(s).   Instance "orcl1", status READY, has 1 handler(s) for this service... The command completed successfully [oracle@otest1 ~]$ [root@otest1 bin]# ./crsctl stop crs -f CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'otest1' CRS-2673: Attempting to stop 'ora.crsd' on 'otest1' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server 'otest1' CRS-2673: Attempting to stop 'ora.orcl.db' on 'otest1' CRS-2673: Attempting to stop 'ora.qosmserver' on 'otest1' CRS-2673: Attempting to stop 'ora.chad' on 'otest1' CRS-2677: Stop of 'ora.orcl.db' on 'otest1' succeeded CRS-33673: Attempting to stop resource group 'ora.asmgroup' on server 'otest1' CRS-2673: Attempting to stop 'ora.DATA.dg' on 'otest1' CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'otest1' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'otest1' CRS-2673: Attempting to stop 'ora.cvu' on 'otest1' CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'otest1' succeeded CRS-2673: Attempting to stop 'ora.otest1.vip' on 'otest1' CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'otest1' succeeded CRS-2673: Attempting to stop 'ora.scan1.vip' on 'otest1' CRS-2677: Stop of 'ora.DATA.dg' on 'otest1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'otest1' CRS-2677: Stop of 'ora.otest1.vip' on 'otest1' succeeded CRS-2677: Stop of 'ora.scan1.vip' on 'otest1' succeeded CRS-2677: Stop of 'ora.cvu' on 'otest1' succeeded CRS-2677: Stop of 'ora.asm' on 'otest1' succeeded CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'otest1' CRS-2677: Stop of 'ora.qosmserver' on 'otest1' succeeded CRS-2677: Stop of 'ora.chad' on 'otest1' succeeded CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'otest1' succeeded CRS-2673: Attempting to stop 'ora.asmnet1.asmnetwork' on 'otest1' CRS-2677: Stop of 'ora.asmnet1.asmnetwork' on 'otest1' succeeded CRS-33677: Stop of resource group 'ora.asmgroup' on server 'otest1' succeeded. CRS-2672: Attempting to start 'ora.qosmserver' on 'otest2' CRS-2672: Attempting to start 'ora.scan1.vip' on 'otest2' CRS-2672: Attempting to start 'ora.cvu' on 'otest2' CRS-2672: Attempting to start 'ora.otest1.vip' on 'otest2' CRS-2676: Start of 'ora.otest1.vip' on 'otest2' succeeded CRS-2676: Start of 'ora.cvu' on 'otest2' succeeded CRS-2676: Start of 'ora.scan1.vip' on 'otest2' succeeded CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'otest2' CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'otest2' succeeded CRS-2676: Start of 'ora.qosmserver' on 'otest2' succeeded CRS-2673: Attempting to stop 'ora.ons' on 'otest1' CRS-2677: Stop of 'ora.ons' on 'otest1' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'otest1' CRS-2677: Stop of 'ora.net1.network' on 'otest1' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'otest1' has completed CRS-2677: Stop of 'ora.crsd' on 'otest1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'otest1' CRS-2673: Attempting to stop 'ora.crf' on 'otest1' CRS-2673: Attempting to stop 'ora.mdnsd' on 'otest1' CRS-2677: Stop of 'ora.crf' on 'otest1' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'otest1' succeeded CRS-2677: Stop of 'ora.asm' on 'otest1' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'otest1' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'otest1' succeeded CRS-2673: Attempting to stop 'ora.ctssd' on 'otest1' CRS-2673: Attempting to stop 'ora.evmd' on 'otest1' CRS-2677: Stop of 'ora.evmd' on 'otest1' succeeded CRS-2677: Stop of 'ora.ctssd' on 'otest1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'otest1' CRS-2677: Stop of 'ora.cssd' on 'otest1' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'otest1' CRS-2673: Attempting to stop 'ora.gpnpd' on 'otest1' CRS-2677: Stop of 'ora.gpnpd' on 'otest1' succeeded CRS-2677: Stop of 'ora.gipcd' on 'otest1' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'otest1' has completed CRS-4133: Oracle High Availability Services has been stopped. [root@otest1 bin]# [root@otest2 bin]# ps -ef|grep ora root      2384     1  0 16:39 ?        00:00:13 /s01/app/19.0.0/bin/orarootagent.bin grid      2461     1  0 16:39 ?        00:00:14 /s01/app/19.0.0/bin/oraagent.bin root      2972     1  1 16:39 ?        00:00:31 /s01/app/19.0.0/bin/orarootagent.bin grid      3053     1  1 16:39 ?        00:00:33 /s01/app/19.0.0/bin/oraagent.bin grid      3089     1  2 16:39 ?        00:00:47 /s01/app/19.0.0/jdk/bin/java -server -Xms30M -Xmx512M -Djava.awt.headless=true -Ddisable.checkForUpdate=true -DTRACING.ENABLED=false -XX:ParallelGCThreads=1 -cp /s01/app/19.0.0/jlib/cha.jar:/s01/app/19.0.0/jlib/chaconfig.jar:/s01/app/19.0.0/jlib/cha-diag-msg.jar:/s01/app/19.0.0/jlib/clsce.jar:/s01/app/19.0.0/jlib/srvm.jar:/s01/app/19.0.0/jlib/srvmhas.jar:/s01/app/19.0.0/jlib/srvmasm.jar:/s01/app/19.0.0/jlib/netcfg.jar:/s01/app/19.0.0/jdbc/lib/ojdbc8.jar:/s01/app/19.0.0/ucp/lib/ucp.jar:/s01/app/19.0.0/jlib/fte.jar:/s01/app/19.0.0/jlib/jai_core.jar:/s01/app/19.0.0/jlib/mlibwrapper_jai.jar:/s01/app/19.0.0/jlib/vecmath.jar:/s01/app/19.0.0/jlib/jai_codec.jar:/s01/app/19.0.0/jlib/jh.jar oracle.cha.server.CHADDriver grid      3379     1  0 16:40 ?        00:00:00 oracle+ASM2_asmb_+asm2 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) grid      3394     1  0 16:40 ?        00:00:00 oracle+ASM2 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) oracle    3399     1  0 16:40 ?        00:00:08 /s01/app/19.0.0/bin/oraagent.bin oracle    3498     1  0 16:40 ?        00:00:00 ora_pmon_orcl2 oracle    3500     1  0 16:40 ?        00:00:00 ora_clmn_orcl2 oracle    3502     1  0 16:40 ?        00:00:00 ora_psp0_orcl2 oracle    3504     1  0 16:40 ?        00:00:05 ora_ipc0_orcl2 oracle    3506     1  1 16:40 ?        00:00:36 ora_vktm_orcl2 oracle    3510     1  0 16:40 ?        00:00:01 ora_gen0_orcl2 oracle    3512     1  0 16:40 ?        00:00:00 ora_mman_orcl2 oracle    3516     1  0 16:40 ?        00:00:01 ora_gen1_orcl2 oracle    3519     1  0 16:40 ?        00:00:01 ora_diag_orcl2 oracle    3521     1  0 16:40 ?        00:00:00 ora_ofsd_orcl2 oracle    3524     1  0 16:40 ?        00:00:02 ora_dbrm_orcl2 oracle    3527     1  0 16:40 ?        00:00:00 ora_vkrm_orcl2 oracle    3529     1  0 16:40 ?        00:00:00 ora_ping_orcl2 oracle    3531     1  0 16:40 ?        00:00:00 ora_svcb_orcl2 oracle    3533     1  0 16:40 ?        00:00:00 ora_acms_orcl2 oracle    3535     1  0 16:40 ?        00:00:00 ora_pman_orcl2 oracle    3537     1  0 16:40 ?        00:00:14 ora_dia0_orcl2 oracle    3539     1  0 16:40 ?        00:00:04 ora_lmon_orcl2 oracle    3541     1  0 16:40 ?        00:00:03 ora_lmd0_orcl2 oracle    3543     1  0 16:40 ?        00:00:07 ora_lms0_orcl2 oracle    3545     1  0 16:40 ?        00:00:07 ora_lms1_orcl2 oracle    3547     1  0 16:40 ?        00:00:03 ora_lmd1_orcl2 oracle    3555     1  0 16:40 ?        00:00:00 ora_rms0_orcl2 oracle    3557     1  0 16:40 ?        00:00:04 ora_lmhb_orcl2 oracle    3559     1  0 16:40 ?        00:00:00 ora_lck1_orcl2 oracle    3563     1  0 16:40 ?        00:00:00 ora_dbw0_orcl2 oracle    3565     1  0 16:40 ?        00:00:00 ora_lgwr_orcl2 oracle    3567     1  0 16:40 ?        00:00:01 ora_ckpt_orcl2 oracle    3569     1  0 16:40 ?        00:00:00 ora_lg00_orcl2 oracle    3571     1  0 16:40 ?        00:00:00 ora_smon_orcl2 oracle    3573     1  0 16:40 ?        00:00:00 ora_lg01_orcl2 oracle    3575     1  0 16:40 ?        00:00:00 ora_smco_orcl2 oracle    3577     1  0 16:40 ?        00:00:00 ora_reco_orcl2 oracle    3579     1  0 16:40 ?        00:00:00 ora_w000_orcl2 oracle    3581     1  0 16:40 ?        00:00:00 ora_lreg_orcl2 oracle    3583     1  0 16:40 ?        00:00:00 ora_w001_orcl2 oracle    3585     1  0 16:40 ?        00:00:00 ora_pxmn_orcl2 oracle    3587     1  0 16:40 ?        00:00:00 ora_rbal_orcl2 oracle    3589     1  0 16:40 ?        00:00:00 ora_asmb_orcl2 oracle    3591     1  0 16:40 ?        00:00:02 ora_fenc_orcl2 oracle    3593     1  0 16:40 ?        00:00:08 ora_mmon_orcl2 grid      3595     1  0 16:40 ?        00:00:00 oracle+ASM2_asmb_orcl2 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) oracle    3597     1  0 16:40 ?        00:00:02 ora_mmnl_orcl2 oracle    3599     1  0 16:40 ?        00:00:00 ora_d000_orcl2 oracle    3601     1  0 16:40 ?        00:00:00 ora_s000_orcl2 oracle    3603     1  0 16:40 ?        00:00:03 ora_imr0_orcl2 oracle    3605     1  0 16:40 ?        00:00:01 ora_scm0_orcl2 oracle    3607     1  0 16:40 ?        00:00:10 ora_gcr0_orcl2 oracle    3610     1  0 16:40 ?        00:00:01 ora_lck0_orcl2 oracle    3612     1  0 16:40 ?        00:00:00 ora_rsmn_orcl2 oracle    3614     1  0 16:40 ?        00:00:00 ora_tmon_orcl2 oracle    3616     1  0 16:40 ?        00:00:02 ora_m000_orcl2 oracle    3626     1  0 16:40 ?        00:00:00 ora_mark_orcl2 oracle    3640     1  0 16:40 ?        00:00:00 ora_tt00_orcl2 oracle    3642     1  0 16:40 ?        00:00:00 ora_tt01_orcl2 oracle    3644     1  0 16:40 ?        00:00:00 ora_tt02_orcl2 oracle    3654     1  0 16:40 ?        00:00:00 ora_w002_orcl2 oracle    3661     1  0 16:40 ?        00:00:00 ora_gtx0_orcl2 oracle    3663     1  0 16:40 ?        00:00:00 ora_rcbg_orcl2 oracle    3666     1  0 16:40 ?        00:00:00 ora_w003_orcl2 oracle    3668     1  0 16:40 ?        00:00:00 ora_aqpc_orcl2 oracle    3675     1  0 16:40 ?        00:00:00 ora_p000_orcl2 oracle    3677     1  0 16:40 ?        00:00:00 ora_p001_orcl2 oracle    3680     1  0 16:40 ?        00:00:00 ora_p002_orcl2 oracle    3682     1  0 16:40 ?        00:00:00 ora_p003_orcl2 oracle    3684     1  0 16:40 ?        00:00:00 ora_p004_orcl2 oracle    3686     1  0 16:40 ?        00:00:00 ora_p005_orcl2 oracle    3688     1  0 16:40 ?        00:00:00 ora_p006_orcl2 oracle    3690     1  0 16:40 ?        00:00:00 ora_p007_orcl2 oracle    3700     1  0 16:40 ?        00:00:00 ora_w004_orcl2 oracle    3703     1  0 16:40 ?        00:00:10 ora_cjq0_orcl2 oracle    3718     1  0 16:40 ?        00:00:00 oracleorcl2 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) oracle    3738     1  0 16:40 ?        00:00:00 oracleorcl2 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) oracle    3740     1  0 16:40 ?        00:00:00 ora_qm02_orcl2 oracle    3742     1  0 16:40 ?        00:00:01 ora_qm05_orcl2 oracle    3744     1  0 16:40 ?        00:00:00 ora_q002_orcl2 oracle    3932     1  0 16:41 ?        00:00:06 ora_m001_orcl2 oracle    3934     1  0 16:41 ?        00:00:02 ora_m002_orcl2 oracle    3936     1  0 16:41 ?        00:00:03 ora_m003_orcl2 oracle    3969     1  0 16:41 ?        00:00:08 ora_m004_orcl2 oracle    4024     1  0 16:41 ?        00:00:00 ora_q00n_orcl2 oracle    4468     1  0 16:50 ?        00:00:00 ora_w005_orcl2 oracle    4471     1  0 16:50 ?        00:00:00 ora_w006_orcl2 oracle    4477     1  0 16:50 ?        00:00:00 ora_w007_orcl2 oracle    4851     1  0 16:58 ?        00:00:00 ora_ppa7_orcl2 oracle    5247     1  0 17:09 ?        00:00:00 ora_q003_orcl2 oracle    5270     1  0 17:09 ?        00:00:00 ora_o000_orcl2 grid      5272     1  0 17:09 ?        00:00:00 oracle+ASM2_o000_orcl2 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) oracle    5284     1  0 17:09 ?        00:00:00 oracleorcl2 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) grid      5299     1  1 17:09 ?        00:00:01 /s01/app/19.0.0/jdk/bin/java -Xms128M -Xmx384M -classpath /s01/app/19.0.0/jdk/jre/lib/rt.jar:/s01/app/19.0.0/jdk/jre/lib/i18n.jar:/s01/app/19.0.0/jlib/jagent.jar:/s01/app/19.0.0/jlib/srvm.jar: -Dora.jagent.conf=conf/orajagent.conf oracle.jagent.core.OraJavaAgentDriver grid      5350     1 11 17:09 ?        00:00:13 /s01/app/19.0.0/jdk/bin/java -server -Xms128M -Xmx384M -Djava.awt.headless=true -Ddisable.checkForUpdate=true -Djava.util.logging.config.file=/s01/app/grid/crsdata/otest2/qos/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -DTRACING.ENABLED=false -Djava.rmi.server.hostname=otest2 -Doracle.http.port=8888 -Doracle.jmx.port=8895 -Doracle.tls.enabled=false -Doracle.jwc.tls.http.enabled=false -Djava.security.manager -Djava.security.policy=/s01/app/grid/crsdata/otest2/qos/conf/catalina.policy -Djava.security.egd=file:/dev/urandom -Dcatalina.home=/s01/app/19.0.0/tomcat -Dcatalina.base=/s01/app/grid/crsdata/otest2/qos -Djava.io.tmpdir=/s01/app/grid/crsdata/otest2/qos/temp -Doracle.home=/s01/app/19.0.0 -Doracle.jwc.mode=CLUSTERWARE -classpath /s01/app/19.0.0/jlib/cryptoj.jar:/s01/app/19.0.0/jlib/oraclepki.jar:/s01/app/19.0.0/jlib/osdt_core.jar:/s01/app/19.0.0/jlib/osdt_cert.jar:/s01/app/19.0.0/tomcat/lib/tomcat-juli.jar:/s01/app/19.0.0/tomcat/lib/bootstrap.jar:/s01/app/19.0.0/jlib/jwc-logging.jar org.apache.catalina.startup.Bootstrap start grid      5429  5375 33 17:09 ?        00:00:39 /s01/app/19.0.0/jdk/bin/java -d64 -classpath /s01/app/19.0.0/jdk/lib/rt.jar:/s01/app/19.0.0/jlib/cvu.jar:/s01/app/19.0.0/jlib/srvm.jar:/s01/app/19.0.0/oui/jlib/OraInstaller.jar:/s01/app/19.0.0/oui/jlib/jsch.jar:/s01/app/19.0.0/oui/jlib/OraPrereq.jar:/s01/app/19.0.0/oui/jlib/prov_fixup.jar:/s01/app/19.0.0/oui/jlib/xmlparserv2.jar:/s01/app/19.0.0/oui/jlib/share.jar:/s01/app/19.0.0/oui/jlib/orai18n-mapping.jar:/s01/app/19.0.0/jlib/srvmhas.jar:/s01/app/19.0.0/jdbc/lib/ojdbc8.jar:/s01/app/19.0.0/jlib/netcfg.jar:/s01/app/19.0.0/jlib/ldapjclnt19.jar -DCV_DESTLOC=/tmp -DCV_HOME=/s01/app/19.0.0 -DRUNNING.MODE=cvuresource -DCV_LOG_LOCATION=/s01/app/grid/crsdata/otest2/cvu/cvulog -DCV_PID_FILE=/s01/app/grid/crsdata/otest2/cvu/init/cvu.pid -DCV_TRACELOC=/s01/app/grid/crsdata/otest2/cvu/cvutrc -Xmx2048m oracle.ops.verification.client.CluvfyDriver comp healthcheck -mandatory -_format -_check_type partial grid      6112     1  0 17:09 ?        00:00:00 oracle+ASM2_ocr (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) grid      6116     1  0 17:09 ?        00:00:00 oracle+ASM2_o000_+asm2 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) grid      9168     1  0 17:10 ?        00:00:00 oracle+ASM2_mmnl_orcl2 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) root     11130 10303  0 17:11 pts/0    00:00:00 grep --color=auto ora [root@otest2 bin]# ./crsctl stop crs -f CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'otest2' CRS-2673: Attempting to stop 'ora.crsd' on 'otest2' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server 'otest2' CRS-2673: Attempting to stop 'ora.orcl.db' on 'otest2' CRS-2673: Attempting to stop 'ora.chad' on 'otest2' CRS-2673: Attempting to stop 'ora.qosmserver' on 'otest2' CRS-2677: Stop of 'ora.orcl.db' on 'otest2' succeeded CRS-33673: Attempting to stop resource group 'ora.asmgroup' on server 'otest2' CRS-2673: Attempting to stop 'ora.DATA.dg' on 'otest2' CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'otest2' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'otest2' CRS-2673: Attempting to stop 'ora.cvu' on 'otest2' CRS-2673: Attempting to stop 'ora.otest1.vip' on 'otest2' CRS-2677: Stop of 'ora.DATA.dg' on 'otest2' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'otest2' CRS-2677: Stop of 'ora.otest1.vip' on 'otest2' succeeded CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'otest2' succeeded CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'otest2' succeeded CRS-2673: Attempting to stop 'ora.scan1.vip' on 'otest2' CRS-2673: Attempting to stop 'ora.otest2.vip' on 'otest2' CRS-2677: Stop of 'ora.scan1.vip' on 'otest2' succeeded CRS-2677: Stop of 'ora.otest2.vip' on 'otest2' succeeded CRS-2677: Stop of 'ora.qosmserver' on 'otest2' succeeded CRS-2677: Stop of 'ora.asm' on 'otest2' succeeded CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'otest2' CRS-2677: Stop of 'ora.cvu' on 'otest2' succeeded CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'otest2' succeeded CRS-2673: Attempting to stop 'ora.asmnet1.asmnetwork' on 'otest2' CRS-2677: Stop of 'ora.asmnet1.asmnetwork' on 'otest2' succeeded CRS-33677: Stop of resource group 'ora.asmgroup' on server 'otest2' succeeded. CRS-2677: Stop of 'ora.chad' on 'otest2' succeeded CRS-2673: Attempting to stop 'ora.ons' on 'otest2' CRS-2677: Stop of 'ora.ons' on 'otest2' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'otest2' CRS-2677: Stop of 'ora.net1.network' on 'otest2' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'otest2' has completed CRS-2677: Stop of 'ora.crsd' on 'otest2' succeeded CRS-2673: Attempting to stop 'ora.storage' on 'otest2' CRS-2673: Attempting to stop 'ora.crf' on 'otest2' CRS-2673: Attempting to stop 'ora.mdnsd' on 'otest2' CRS-2677: Stop of 'ora.crf' on 'otest2' succeeded CRS-2677: Stop of 'ora.storage' on 'otest2' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'otest2' CRS-2677: Stop of 'ora.mdnsd' on 'otest2' succeeded CRS-2677: Stop of 'ora.asm' on 'otest2' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'otest2' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'otest2' succeeded CRS-2673: Attempting to stop 'ora.ctssd' on 'otest2' CRS-2673: Attempting to stop 'ora.evmd' on 'otest2' CRS-2677: Stop of 'ora.ctssd' on 'otest2' succeeded CRS-2677: Stop of 'ora.evmd' on 'otest2' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'otest2' CRS-2677: Stop of 'ora.cssd' on 'otest2' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'otest2' CRS-2673: Attempting to stop 'ora.gpnpd' on 'otest2' CRS-2677: Stop of 'ora.gpnpd' on 'otest2' succeeded CRS-2677: Stop of 'ora.gipcd' on 'otest2' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'otest2' has completed CRS-4133: Oracle High Availability Services has been stopped. [root@otest2 bin]#