Process m000 died, see its trace file

The database is a physical DG of 11g

Standby Database error Process m000 died, see its trace file.

The error that is now being searched online is that resource exhaustion cannot be connected, but the database is still alive.

This situation is resolved by adjusting the number of resources, such as processes reaching the upper limit

View Resource Usage

select * from v$resource_limit; resource_name                   current_utilization max_utilization initial_allocation   limit_ VALUE------------------------------ ------------------- --------------- --------------------  --------------------processes                                         50               54        150                   150sessions                                          64               68        252                   252enqueue_locks                                     35               39       3310                  3310enqueue_resources                                 17               17       1328             UNLIMITEDges_procs                                          0                0           0                     0ges_ress                                           0                0          0             UNLIMITEDges_locks                                          0                0           0            unlimitedges_cache_ ress                                    0                0           0            unlimitedges_ reg_msgs                                       0               0           0             UNLIMITEDges_big_msgs                                       0                0          0             UNLIMITEDges_rsv_msgs                                       0                0          0                      0gcs_resources                                      0               0           0                     0gcs_shadows                                        0                0           0                     0dml_locks                                          0                0       1108             UNLIMITEDtemporary_table_locks                              0               0   unlimited            unlimitedtransactions                                       5                5         277             unlimitedbranches                                           0               0         277             UNLIMITEDcmtcallbk                                          2                3        277             UNLIMITEDmax_rollback_segments                             11              11         277                 65535sort_segment_locks                                 0                1  UNLIMITED             unlimitedk2q_locks                                          0               0         504             UNLIMITEDmax_shared_servers                                 1               1   Unlimited            unlimitedparallel_max_servers                               0                0        135                  3600 

Increase processes

Alter system set PROCESSES=200 Scope=spfile;

Then restart the database to resolve.

But my problem cannot be solved by the above method.

Look again at the problem instance and find the following:

Login via Sqlplus, go in is the idle process, can only kill the OS process to restart.

Restarts and then shuts down automatically.

View Alert logs no information

Trace the Smon process display using the Strace command

[[email protected] ~]$ strace -p 7351process  7351 attachedgetrusage (rusage_self, {ru_utime={0, 21996}, ru_stime={0, 14997}, &NBSP, ...})  = 0getrusage (rusage_self, {ru_utime={0, 21996}, ru_stime={0, 14997},  ...})  = 0semtimedop (15794179, {{18, -1, 0}}, 1, {3, 0})  = -1  EAGAIN  (resource temporarily unavailable) Semtimedop (15794179, {{18, -1, 0} }, 1, {3, 0})  = -1 EAGAIN  (resource temporarily unavailable) Semtimedop (15794179, {{18, -1, 0}}, 1, {3, 0})  = -1 EAGAIN  ( resource temporarily unavailable) Semtimedop (15794179, {{18, -1, 0}}, 1, {3,  0})  = -1 EAGAIN  (resource temporarily unavailable) 

You can see that the resource is already unavailable and is suspected of being a memory problem.

Here, the server is used as a test, running several instances, a single node 10g,11g,12c,11g standby,11g RAC, and so on 5 instances.

Use Ipcs-m to view

 ipcs -m ------ Shared Memory Segments -------- key         shmid      owner       perms      bytes      nattch      status  0x1644d05c 28082176   oracle      600        945815552  28                  0xb4a8f6f0 28147713    oracle     660        4096        0                   0xb3928380 28475394   oracle      640        14680064   82                  0x00000000 28508163   oracle      640        868220928  82                  0xb2f44a00 41123844    oracle     660        4096        0                   0x27bbfbde 3309577    oracle      755        1079228    1                                  0x00000000 22708244   oracle      640        33554432   49                  0x00000000  22741013   oracle     640         2449473536 49                  0x0fc14ec0 22773782   oracle     640         2097152    49

You can see 6 of the above semaphore, excluding 0x00000000 (the semaphore is derived)

However, there are 5 instances, and the suspected standby memory segment is closed unexpectedly and the system is not cleaned out

And there is an Oracle segment with a permission of 755, which is generally 640, which is suspected to be the cause, and

To see which instance of Oracle each memory segment belongs to, you can use the Oracle command SYSRESV

[Email protected] ~]$ SYSRESVIPC Resources for Oracle_sid "ORCL": Gkfx memory:id KEY41385984 0XB2F4 4a00semaphores:id KEY16220162 0xb38b1d5coracle Instance alive for Sid "Orcl"

Instances can be specified with multiple instances of the same version

Sysresv-l SID

Delete a shared memory segment

[Email protected] trace]$ IPCRM--HLEPUSAGE:IPCRM [[-Q msqid] [-M shmid] [-S semid] [-Q Msgkey] [-M shmkey] [- S Semkey] ...]

or sysresv-f Sid deleted

Restart Stanby after deletion, everything is OK.

Process m000 died, see its trace file