Symptom:
Node 2 run: Select sum (bytes) from dba_segments long-term waiting: Waiting For 'gc Cr request'
Node 1: Select Sid, serial # From GV $ session where username = '****'
Analysis:
Internal wait relationship of the database:
Blocking blocker event
388 773,397 Enq: PV-syncstart
773 397 Enq: Pr-contention
397 Enq: Pr-contention OS thread startup: ora-445
-- 1
Node 1 run the select Sid statement, serial # From GV $ session where username = 'wanqiong ':
Cause of failure: Process 53/57 runs the statement for two sessions, process 20: ckpt block.
Process 20 because waiting for 'enq: Pr-contention 'name | mode = 0x50520006
0x50520006 owner: process 45: cjq0
Analysis by awk-F ass109.awk calldb1_ora_15639.trc:
Resource holder state
Enqueue PR-00000000-00000000 45: 0: Waiting For 'OS thread startup'
Enqueue PV-00000000-00000000 38: 38: Is waiting for 45:
Latch 380030bc0 ??? Blocker
Process 45: cjq0 processes are the culprit! The process cannot be started.
-- 2 node 2
Node 2 running statement: Select sum (bytes) from dba_segments, waiting time 0: Waiting For 'gc Cr request'
No obvious node communication error was found in cssd and crsd. It is suspected that node 1 will crash.
-- 3 Node 1 warning Log Analysis
Mon Aug 11 09:45:49 2014
Archived log entry 58557 added for thread 1 sequence 27907 ID 0x4bcd9425 DEST 1:
Mon Aug 11 16:08:44 2014
Errors in file/Oracle/diag/rdbms/calldb/calldb1/Trace/calldb1_cjq0_8914.trc (incident = 56362 ):
ORA-00445: background process "j000" did not start after 120 seconds
Incident Details in:/Oracle/diag/rdbms/calldb/calldb1/Incident/incdir_56133/calldb1_cjq0_8914_i56362.trc
Kkjcre1p: Unable to spawn jobq slave Process
Errors in file/Oracle/diag/rdbms/calldb/calldb1/Trace/calldb1_cjq0_8914.trc:
Mon Aug 11 16:12:25 2014
Errors in file/Oracle/diag/rdbms/calldb/calldb1/Trace/calldb1_mmon_8497.trc (incident = 56202 ):
ORA-00445: background process "m000" did not start after 120 seconds
Incident Details in:/Oracle/diag/rdbms/calldb/calldb1/Incident/incdir_56202/calldb1_mmon_8497_i56202.trc
Mon Aug 11 16:14:28 2014
AWR analysis memory 16 GB
Begin end
Host MEM (MB): 15,744.0 15,744.0
SGA use (MB): 4,608.0 4,608.0
PGA use (MB): 462.5 460.1
% Host mem used for SGA + PGA: 32.21 32.19
12 GB memory after restart
[[Email protected]/home/Oracle] $ prtconf | grep 'memory'
Memory size: 12288 megabytes
We suspect the problem lies in the memory:
1379200.1 according to the document, you can determine:
