SMP test of Erlang in multi-core environment under Linux

Source: Internet
Author: User
Tags cpu usage

Goal

(1) Understand the effect of Erlang concurrent process scheduling on CPU core load in multi-core CPU environment;

(2) The memory addition mechanism of Erlang virtual machine;

(3) The operation of Erlang process;

(4) Monitoring CPU usage under Linux

Experimental environment

Lenovo Minicomputer: Operating system: RedHat Enterprise linuxserver release6.4 (Santiago)

Kernel version number: Linux Server1 2.6.32-358.el6.x86_64#1 SMP

CPU Model: Intel (R) Xeon (r) CPU e7-4820 @2.00ghz;

4 CPUs with physical cores per CPU of 8, total physical cores of 32, logical cores of 64

Memory: 125G

Disk: 289G

Erlang OTP:ERLANG/OTP 17[erts-6.0][source][64bit][smp:64:64][async-thread:10]

The test process Erlang concurrency process

Test code

-module (Test).

-export ([START_PROC/1]).

Start_proc (Num)

Case Num =:= 0 of

True-OK;

False--spawn (fun (), loop () end), Start_proc (Num-1)

End.

Loop (), loop ().

CPU Execution Status

Test:start_proc (1000000) After starting 1 million processes, run Mpstat–pall 2 10


Concurrent IO Test Code

-module (Testio).

-export ([START_PROC/1]).

Name (),

{a,b,c} =erlang:now (),

Integer_to_list (A) ++integer_to_list (B) ++integer_to_list (C).

Io ()

{OK,FD} = File:open ("./data/" ++name (), [Write,raw,binary,append]),

File:write (Fd, "Hi Zcc,nice to Meet You"),

File:read (fd,23),

File:close (Fd).

Start_proc (Num)

io = Fun ()-io () end,

Case Num =:= 0of

True->ok;

False-> spawn (IO), Timer:sleep (1), Start_proc (Num-1)

End.

View kernel-related information under Linux

Vmstat–n 3 Refresh every 3s

Analysis of CPU utilization using SAR Sar–u 2 10

Parsing execution Process Queue Length Sar–q 2 10

CPU%usr%nice%sys%iowait%irq%soft%steal%guest%idle

Mpstat–p 1: View information for CPU number 1th

%steal:% of Virtual CPU unconscious wait time

%nice:cpu time percentage in user mode with nice value

Press the F key after Top–m and press J, enter to see which core the process is executing on

Conclusion

(1) in the default Erlang SMP scheduling process Schedualer is corresponding to the CPU core, and not the number of CPUs corresponding, as in this environment there are 4 cpu,32 physical cores, open hyper-thread 64 logical core, the number of scheduling process is 64;

(2) When Erlang concurrent multiple processes, through the load of the CPU can be seen, each process will be evenly distributed across the core execution, and not a certain nuclear load too large, a nuclear load is too small, this thing is an operating system to do, the program apes do not care;

(3) Erlang virtual machine memory will be added as the number of processes, and actively request to add from the host memory, which is different from the Java Virtual machine, jvm1.6 once memory does not actively add memory, can only manually configure the addition of JVM memory, from 1.7 to share the physical host memory; ERL The ANG virtual machine does not suppress the memory growth mechanism. The virtual machine constantly allocates memory, forcing the system to use the swap area until the virtual machine runs out of memory and becomes dull. The ' private heap ' and queue-based programming models of Erlang virtual machines are both a strength and a disadvantage when performing Erlang VMs in a production environment, with a system-level check to kill processes when Erlang memory usage soars.

(4) The Erlang Scheduler executes on an OS thread, which is determined by the OS whether it executes on a core, and generally, the OS guarantees that the thread executes on one core during execution;

(5) Estimating the maximum number of processes in a system = Total memory/erlang:process_info (self (), memories), Wordsize=erlang:system_info (wordsize) of Erlang. A process in Erlang accounts for approximately 2667 bytes of memory.

(6) test results

Number of START processes

Virtual machine Memory

Start time

100,000

299M

12.3s

1 million

2.7G

6.8s

10 million

27.2G

57.6s

55164851

132G

400s

SMP test of Erlang in multi-core environment under Linux

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.