In this Document
Applies:
Oracle GoldenGate-Version 11.1.1.1.0 and later Information in this document applies to any platform. * ** Checked for relevance on 24-Mar-2013 *** SYMPTOMS
Extract or pump abends with "TCP/IP error 9 (Bad file number)" when starting. The exact error number may vary: E.g ., 1. Version 11.1.1.1 12771498 ERROR OGG-01224 TCP/IP error 9 (Bad file number ). ERROR OGG-01668 process abending.2. pre GA release version: v11_000000000000024 Ggs error 150 TCP/IP error 9 (Bad file number ). Ggs error 190 process abending.Let's say you only have 4 Extract pump processes communicating with this target system, and have specified 21 dynamic ports on the target system. Why can't your Extract connect to the remote system? CAUSE
The cause is that all the ports listed in DYNAMICPORTLIST in the manager parameter at downstream server are in use. This error message is usually due to port allocation failure or some orphan collector processes on target preventing the startup of new collectors. In the GoldenGate environment on the target system, check the ports that Manager is using with this command: GGSCI (ggdb1) 21> send mgr getportinfo detailSending GETPORTINFO, request to MANAGER... Dynamic Port List Starting Index 21 Reassign Delay 3 seconds Entry Port Error Process Assigned Program -------------------- 0 7810 98 1 7811 98 2 7812 0 3 7813 0 18713 20:39:12 Server 4 7814 0 5 7815 0 3662 22:11:02 Server 6 7816 0 27070 02:16:11 Server 7 7817 0 7789 16:56:10 Server 8 7818 0 14116 17:36:18 Server 9 7819 0 10900 17:59:59 Server 10 7820 0 28045 04:26:01 Server 11 7821 98 12 7822 0 31379 05:29:31 Server 13 7823 0 23538 10:21:54 Server 14 7824 0 23593 10:22:01 Server 15 7825 0 6687 07:17:50 Server 16 7826 0 17339 07:22:01 Server 17 7827 0 24905 08:51:51 Server 18 7828 98 19 7829 0 1881 08:55:45 Server 20 7830 0 5884 08:57:03 Server It is reporting error 98 on the open ports from 7810-7828. The error 98 is "address in use ". You can also check the Manager report file on the target system (dirrpt/MGR. rpt) for an error message. Ggs info 301 Command received ed from EXTRACT on host 10.30.113.28 (start server cpu-1 PRI-1 PARAMS ). Ggs info 302 No Dynamic Ports Available.This confirms that the Manager cannot allocate anymore dynamic ports. Since there shocould be plenty of ports available, this indicates that there may be "orphaned" server collector processes. SOLUTION
The workaround is to increase the port numbers in DYNAMICPORTLIST in mgr. prm. Bounce the manager afterwards.If the source Extract dies without communicating to the target server collector, that server will be orphaned and must be killed. development plans to improve this behavior in a future release (tracked via enhancement Bug 10430342), but until that time, they must be handled manually in this manner. You can determine which processes are orphans by stopping the upstream pumps and then seeing what Server processes are still running, and killing them. once these servers are killed, you should be able restart all the pumps. In ggsci on the target system: Send mgr childstatus debugThis will retrieve status information about processes started by Manager, and the corresponding port numbers that have been allocated by Manager. GGSCI (ggdb1) 23> send mgr childstatus debugSending CHILDSTATUS, request to MANAGER... Child Process Status-14 Entries ID Group Process Retry Time Start Time Port ----------------------- 0 PEESIS 27760 0 None 09:30:45 7843 1 PPESIS 27767 0 None 09:30:45 7845 2 SRESIP 31149 1 2011/08/04 09:44:32 2011/08/04 09:30:45 8003 3 PEASPIS 3177 2 2011/08/04 10:12:07 2011/08/04 09:30:45 8002 4 PPASPIS 27784 0 None 09:30:45 7854 5 SRASPIP 27792 0 None 2011/08/04 09:30:45 7860 6 PEIPATIS 27798 0 None 09:30:46 7861 8 SRIPATIP 27800 0 None 2011/08/04 09:30:46 8000 9 PEDTIS 28879 0 None 09:30:53 8001 13 PETIBCOS 28959 0 None 09:30:58 8004 19 SRDTIP 29051 0 None 09:31:07 8005 20 SREEXIP 29060 0 None 09:31:08 8006 21 SREXCIP 29097 0 None 09:31:09 8007 22 SRIPIP 29098 0 None 09:31:10 8008 You can also use this command to determine what server collector processes are running: Ps-ef | grep server{Ggate} sintegoradb1.aeso. ca:/usr/ggate/product/10.4.0/ggs> ps-ef | grep server Ggate 1881 7278 0? 00:00:02./server-p 7829-k-l/usr/ggate/product/10.4.0/ggs/ggserr. log Ggate 3556 1 0 Jul25? 00:01:57./server-p 7877-k-l/usr/ggate/product/10.4.0/ggs/ggserr. log Ggate 3558 1 0 Jul20? 00:03:51./server-p 7828-k-l/usr/ggate/product/10.4.0/ggs/ggserr. log Ggate 5884 7278 0? 00:00:02./server-p 7830-k-l/usr/ggate/product/10.4.0/ggs/ggserr. log Root 6312 1 0 Jul05? 00:00:00/usr/bin/hidd-server Ggate 6687 7278 0? 00:00:04./server-p 7825-k-l/usr/ggate/product/10.4.0/ggs/ggserr. log Ggate 9121 1 0 Jul06? 00:07:34./server-p 7953-k-l/usr/ggate/product/10.4.0/ggs/ggserr. log Root 10924 1 0 Jul05? 00:00:00/usr/libexec/gam_server Ggate 13060 1 0 Jul22? 00:02:35./server-p 7867-k-l/usr/ggate/product/10.4.0/ggs/ggserr. log Ggate 14345 1382 0 00:00:00 pts/4 grep server Ggate 17339 7278 0? 00:00:03./server-p 7826-k-l/usr/ggate/product/10.4.0/ggs/ggserr. log Ggate 18528 1 0 Jul06? 00:05:54./server-p 7967-k-l/usr/ggate/product/10.4.0/ggs/ggserr. log Ggate 23233 1 0 Jul06? 00:05:55./server-p 7965-k-l/usr/ggate/product/10.4.0/ggs/ggserr. log Ggate 23538 7278 0 Aug02? 00:00:14./server-p 7823-k-l/usr/ggate/product/10.4.0/ggs/ggserr. log Ggate 23593 7278 0 Aug02? 00:00:15./server-p 7824-k-l/usr/ggate/product/10.4.0/ggs/ggserr. log Ggate 24905 7278 0? 00:00:02./server-p 7827-k-l/usr/ggate/product/10.4.0/ggs/ggserr. log Ggate 31379 7278 0 Aug01? 00:00:28./server-p 7822-k-l/usr/ggate/product/10.4.0/ggs/ggserr. log In summary: 1) Stop the Manager on the target system, and all of the upstream GoldenGate pump Extract processes. 2) Kill any remaining server processes. 3) Restart the Manager, and then restart the Extracts. REFERENCES
|