Zabbix Login Problem: Cannot allocate shared memory for collector

Problem Description:
On a zabbix monitored server (64-bit centos6.8 system, 64G content) start zabbix_agent, the discovery process does not start, 10050 port is not up!

The boot zabbix_agent process did not get an error, but the 10050 port did not start properly.
[Email protected] ~]#/USR/LOCAL/ZABBIX/SBIN/ZABBIX_AGENTD
[Email protected] ~]# Ps-ef|grep zabbix_agent
Root 27506 27360 0 11:07 pts/5 00:00:00 grep--color Zabbix
[Email protected] etc]# lsof-i:10050

Check the/usr/local/zabbix/logs/zabbix_agentd.log log and find the following error:
................
27667:20161027:111554.851 cannot allocate shared memory of size 657056: [approx] No space left on device
27667:20161027:111554.851 cannot allocate shared memory for collector
..............

Cause Analysis:
This is due to the kernel's limitations on share memory.

Processing Process records:
[Email protected] logs]# ipcs-l

------Shared Memory Limits--------
Max number of segments = 4096
Max seg Size (Kbytes) = 1940588
Max Total Shared memory (Kbytes) = 8388608
Min seg Size (bytes) = 1

------Semaphore Limits--------
Max number of arrays = 128
Max semaphores per array = 250
Max semaphores system wide = 32000
Max Ops per semop call = 100
Semaphore max value = 32767

------Messages:limits--------
Max queues System wide = 32768
Max size of message (bytes) = 65536
Default max size of queue (bytes) = 65536

From the above command results can be seen:
The Max Total shared memory setting is the 2m,max seg size setting is 8M, which is obviously not enough allocate (allocation) zabbix_agent the memory used to start .

To view the current shared memory settings,
[Email protected] logs]# Sysctl-a|grep SHM
Kernel.shmmax = 1987162112
Kernel.shmall = 2097152
Kernel.shmmni = 4096
kernel.shm_rmid_forced = 0
Vm.hugetlb_shm_group = 0

Where Kernel.shmall represents the total amount of shared memory allocated, here is 2g,kernel.shmax for a single segment can allocate memory (in bytes), here is 2M, so there must be a problem!

Then view/etc/sysctl.conf
[Email protected] logs]# cat/etc/sysctl.conf
........
Kernel.shmall = 2097152
Kernel.shmmax = 1987162112

Obviously the values of the KERNEL.SHAMLL and Kernel.shmmax parameters set in the sysctl.conf file are small.

--------------------------------------------------------------
This machine is a 64-bit CentOS 6.8 system, 64G of memory, to view other monitored servers found in the same system:
[Email protected] ~]# cat/etc/sysctl.conf
........
Kernel.shmmax = 68719476736
Kernel.shmall = 4294967296

[Email protected] logs]# ipcs-l

------Shared Memory Limits--------
Max number of segments = 4096
Max seg Size (Kbytes) = 67108864
Max Total Shared memory (Kbytes) = 17179869184
Min seg Size (bytes) = 1

------Semaphore Limits--------
Max number of arrays = 128
Max semaphores per array = 250
Max semaphores system wide = 32000
Max Ops per semop call = 100
Semaphore max value = 32767

------Messages:limits--------
Max queues System wide = 32768
Max size of message (bytes) = 65536
Default max size of queue (bytes) = 65536

That is, the 64-bit CENTOS6 system (64G) The default value of the above two parameters is 64G and 4G, set is the system can recognize the maximum memory.
---------------------------------------------------------------

Now only need to adjust the value of the two parameters in this machine to solve the problem!
[Email protected] logs]# cat/etc/sysctl.conf
........
Kernel.shmmax = 68719476736
Kernel.shmall = 4294967296
KERNEL.MSGMNB = 65536
Kernel.msgmax = 65536

Execution Sysctl-p Effective
[Email protected] logs]# sysctl-p

See again the discovery has been modified successfully!
[Email protected] logs]# Sysctl-a|grep SHM
Kernel.shmmax = 68719476736
Kernel.shmall = 4294967296
Kernel.shmmni = 4096
kernel.shm_rmid_forced = 0
Vm.hugetlb_shm_group = 0
[Email protected] logs]# ipcs-l

------Shared Memory Limits--------
Max number of segments = 4096
Max seg Size (Kbytes) = 67108864
Max Total Shared memory (Kbytes) = 17179869184
Min seg Size (bytes) = 1

------Semaphore Limits--------
Max number of arrays = 128
Max semaphores per array = 250
Max semaphores system wide = 32000
Max Ops per semop call = 100
Semaphore max value = 32767

------Messages:limits--------
Max queues System wide = 32768
Max size of message (bytes) = 65536
Default max size of queue (bytes) = 65536

finally restart the Zabbix and discover that port 10050 has been successfully started:
[Email protected] ~]#/USR/LOCAL/ZABBIX/SBIN/ZABBIX_AGENTD
[Email protected] logs]# Ps-ef|grep Zabbix
Zabbix 27776 1 0 11:22? 00:00:00/usr/local/zabbix/sbin/zabbix_agentd
Zabbix 27777 27776 0 11:22? 00:00:00/usr/local/zabbix/sbin/zabbix_agentd:collector [Idle 1 sec]
Zabbix 27778 27776 0 11:22? 00:00:00/usr/local/zabbix/sbin/zabbix_agentd:listener #1 [Waiting for connection]
Zabbix 27779 27776 0 11:22? 00:00:00/usr/local/zabbix/sbin/zabbix_agentd:listener #2 [Waiting for connection]
Zabbix 27780 27776 0 11:22? 00:00:00/usr/local/zabbix/sbin/zabbix_agentd:listener #3 [Waiting for connection]
Zabbix 27781 27776 0 11:22? 00:00:00/usr/local/zabbix/sbin/zabbix_agentd:active checks #1 [idle 1 sec]
Root 28188 27360 0 11:48 pts/5 00:00:00 grep--color Zabbix
[Email protected] logs]# lsof-i:10050
COMMAND PID USER FD TYPE DEVICE size/off NODE NAME
Zabbix_ag 27776 Zabbix 4u IPv4 112357384 0t0 TCP *:zabbix-agent (LISTEN)
Zabbix_ag 27777 Zabbix 4u IPv4 112357384 0t0 TCP *:zabbix-agent (LISTEN)
Zabbix_ag 27778 Zabbix 4u IPv4 112357384 0t0 TCP *:zabbix-agent (LISTEN)
Zabbix_ag 27779 Zabbix 4u IPv4 112357384 0t0 TCP *:zabbix-agent (LISTEN)
Zabbix_ag 27780 Zabbix 4u IPv4 112357384 0t0 TCP *:zabbix-agent (LISTEN)
Zabbix_ag 27781 Zabbix 4u IPv4 112357384 0t0 TCP *:zabbix-agent (LISTEN)
[Email protected] logs]#

Summarize:
In fact, not only the Zabbix program startup will encounter this problem, many programs this error can also be used to solve the problem, because the kernel of the resource constraints.

Zabbix Login Problem: Cannot allocate shared memory for collector