SHUTDOWN: waiting for active callto complete processing

When a latch free problem occurs, kill the session at that time, report that the session sid does not exist, and kill it at the OS layer, however, the session Status shown in v $ session is still active!

This is a silent problem, but it is a test environment. So I think the database is stopped.

Shutting down instance (immediate)
License high water mark = 705
Thu May 30 22:03:39 2013
Active call for process 20102 user 'otfmsstg' program 'oracle @ z4ah8020'
Active call for process 20272 user 'otfmsstg' program 'oracle @ z4ah8020'
Active call for process 20387 user 'otfmsstg' program 'oracle @ z4ah8020'
SHUTDOWN: waiting for active cballs to complete.

Always stuck in this place ..

The content of 1039389.6 on support is as follows:

Checked for relevance on 11-Dec-2008

Problem Description:
================================
 
You are attempting to shut down the database and the database hangs.
Alert log contains the following message:
 
 
SHUTDOWN: Waiting for active cballs to complete
 
 
There are no other error messages in the alert log.

Solution Description:
==================================

--------------------------------------------------------
Caveat: This note is intended for Unix platforms primarily, but we shoshould
Address this issue on Windows platforms as well. On Windows, stopping
Service is a valid workaround for these errors.
--------------------------------------------------------

Locate and kill any client connections to the database at the Unix level,
Follows:
 
1. Locate any client connections to the database using ps, and grep for any
Processes belonging to this.

Example: ps-ef | grep V733
 
2. Look for processes that include a 'local = no' designation.
 
Example: osupport 6235 1 0 Nov 24 running lev733 (LOCAL = NO)
 
3. Kill the Unix process (es) with the 'Local = no' designation.
 
Example: Kill-9 6235
 
 
Explanation:
================

The database is waiting for pmon to clean up processes, but pmon is unable
Clean them. The client connections to the server are causing the shutdown
Immediate or normal to hang. Killing them allows pmon to clean up and release
The associated Oracle processes and resources.

What resources are we talking about?

1) Any non committed transactions must be rolled back
2) Any temporary space (sort segments/lobs/session temporary tables) must be freed
3) The session itself and any associated memory consumed by the session.
4) Internal locks/enqueues must be cleaned up

Often Oracle (SMON or PMON depending on whether Shared Server is used) will wait for the OS to terminate the process (es) associated with the session. I the OS never returns, or fails to terminate them, then the instance shutdown will hang with this message (Shutdown Waiting for Active callto Complete)

Other means exist to achieve a quick shutdown, as outlined inNote 386408.1-What Is The Fastest Way To Cleanly Shutdown An Oracle Database?

Note that in E-Business Environments, the same messages can be produced if the database shutdown process starts before it turns out that some sessions cannot be cleaned up by the pmon process and must be killed manually.

Z4ah8020: fmsstag> ps-ef | grep otfmsstg | grep LOCAL = NO
Otfmsstg 27651 1 0 Mar 17? 0: 03 oraclefmsstag (LOCAL = NO)
Otfmsstg 27523 1 0 Mar 17? 0: 02 oraclefmsstag (LOCAL = NO)
Otfmsstg 27603 1 0 Mar 17? 0: 02 oraclefmsstag (LOCAL = NO)
Otfmsstg 6931 1 0 21:35:06? 0: 00 oraclefmsstag (LOCAL = NO)
Otfmsstg 17581 1 0 Mar 18? 55: 52 oraclefmsstag (LOCAL = NO)
Otfmsstg 17553 1 0 Mar 18? Oraclefmsstag (LOCAL = NO)

After kill-9 27651 and other processes

Thu May 30 22:10:33 2013
ALTER DATABASE CLOSE NORMAL
Thu May 30 22:10:35 2013
SMON: disabling tx recovery
SMON: disabling cache recovery
Thu May 30 22:10:44 2013
Shutting down archive processes
Archiving is disabled
Archive process shutdown avoided: 0 active
Thread 1 closed at log sequence 19859
Successful close of redo thread 1
Thu May 30 22:10:45 2013
Completed: ALTER DATABASE CLOSE NORMAL
Thu May 30 22:10:45 2013
ALTER DATABASE DISMOUNT
Completed: ALTER DATABASE DISMOUNT
ARCH: Archiving is disabled
Shutting down archive processes
Archiving is disabled
Archive process shutdown avoided: 0 active
ARCH: Archiving is disabled
Shutting down archive processes
Archiving is disabled
Archive process shutdown avoided: 0 active
Thu May 30 22:11:19 2013

The database is shut down normally, and latch free disappears after restart ..

The latch free issue will be studied in the future. experienced experts will provide more guidance.

Thank you.