Goal
(1) Understand the effect of Erlang concurrent process scheduling on CPU core load in multi-core CPU environment;
(2) The memory increment mechanism of erlang virtual machine;
(3) The operation of Erlang process;
(4) Monitoring CPU usage under Linux
Experimental environment
Lenovo Minicomputer: Operating system: RedHat Enterprise linuxserver release6.4 (Santiago)
Kernel version: Linux Server1 2.6.32-358.el6.x86_64#1 SMP
CPU Model: Intel (R) Xeon (r) CPU e7-4820 @2.00ghz;
4 CPUs with physical cores per CPU of 8, total physical cores of 32, logical cores of 64
Memory: 125G
Disk: 289G
Erlang OTP:ERLANG/OTP 17[erts-6.0][source][64bit][smp:64:64][async-thread:10]
The test process Erlang concurrency process
Test code
-module (Test).
-export ([START_PROC/1]).
Start_proc (Num)
Case Num =:= 0 of
True-OK;
False--spawn (fun (), loop () end), Start_proc (Num-1)
End.
Loop (), loop ().
CPU Health
Test:start_proc (1000000) After starting 1 million processes, execute Mpstat–pall 2 10
Concurrent IO Test Code
-module (Testio).
-export ([START_PROC/1]).
Name (),
{a,b,c} =erlang:now (),
Integer_to_list (A) ++integer_to_list (B) ++integer_to_list (C).
Io ()
{OK,FD} = File:open ("./data/" ++name (), [Write,raw,binary,append]),
File:write (Fd, "Hi Zcc,nice to Meet You"),
File:read (fd,23),
File:close (Fd).
Start_proc (Num)
io = Fun ()-io () end,
Case Num =:= 0of
True->ok;
False-> spawn (IO), Timer:sleep (1), Start_proc (Num-1)
End.
View kernel-related information under Linux
Vmstat–n 3 Refresh every 3s
Analysis of CPU utilization using SAR Sar–u 2 10
Profiling run Process Queue Length Sar–q 2 10
CPU%usr%nice%sys%iowait%irq%soft%steal%guest%idle
Mpstat–p 1: View information for CPU number 1th
%steal:% of Virtual CPU unconscious wait time
%nice:cpu time percentage in user mode with nice value
Press the F key after Top–m and press J, enter to see which core the process is running on
Conclusion
(1) in the default Erlang SMP scheduling process Schedualer is the CPU's core corresponding to the CPU, not the number of corresponding, as in this environment there are 4 cpu,32 physical cores, open hyper-thread 64 logical core, the number of scheduling process is 64;
(2) When Erlang concurrent multiple processes, through the load of the CPU can be seen, each process will be evenly distributed across the core to run, and not a nuclear load is too large, a nuclear load is too small to happen, this thing is the operating system to do, the programmer does not care;
(3) The memory of the Erlang virtual machine will automatically request an increase from the host memory as the number of processes increases, unlike the Java Virtual machine, jvm1.6 memory does not automatically increase memory, can only be manually configured to increase the JVM memory, from 1.7 to share the physical host memory; Erlang virtual The machine does not suppress the memory growth mechanism. The virtual machine constantly allocates memory, forcing the system to use swap zones until the virtual machine runs out of memory and becomes very dull. The ' private heap ' and queue-based programming models of Erlang virtual machines are both an advantage and a disadvantage when running Erlang VMs in a production environment, with a system-level detection to kill processes when Erlang memory usage soars.
(4) The Erlang Scheduler runs on an OS thread, and the OS determines whether it executes on a core, and generally the OS guarantees that the thread will run on a core during execution;
(5) Estimating the maximum number of processes in a system = Total memory/erlang:process_info (self (), memories), Wordsize=erlang:system_info (wordsize) of Erlang. A process in Erlang accounts for approximately 2667 bytes of memory.
(6) test results
boot process number |
virtual machine memory |
start time |
100,000 |
299m |
12.3s |
1 million |
2.7g |
6.8s |
10 million |
27.2g |
57.6s |
55164851 |
132g |
400s |