Optimized solution for memory crunch in Erlang server

Source: Internet
Author: User

  

Issue: Server 1 million people online, 16G memory is almost eaten up. Player process occupies high memory

Workaround:

The first step:
Erlang:system_info (Process_count). See if the number of processes is normal and exceeds the maximum number of processes for the Erlang virtual machine.
Step Two:
View the node's memory bottleneck location
> Erlang:memory ().
[{total,2099813400},
{processes,1985444264},
{processes_used,1985276128},
{system,114369136},
{atom,4479545},
{atom_used,4477777},
{binary,22756952},
{code,10486554},
{ets,47948808}]
The display memory is mostly consumed on the process, thus determining that the process is consuming a lot of memory

Step Three:
View the most memory-intensive processes

>spawn (Fun ()-Etop:start ([{Output, text}, {interval, 1}, {lines, +}, {sort, memory}]).
(Start etop in output text, with an interval of 1 seconds, an output line number of 20 rows, sorted in memory.) Here spawn a new process to output etop data without affecting Erlang shell input.)

Fourth step: View the state of the process that occupies the highest memory
>erlang:process_info (PID (0,12571,0)).
[{current_function,{mod_player,send_msg,2}},
{initial_call,{erlang,apply,2}},
{status,waiting},
{message_queue_len,0},
{messages,[]},
{links,[<0.12570.0>]},
{dictionary,[]},
{Trap_exit,false},
{Error_handler,error_handler},
{Priority,normal},
{Group_leader,<0.46.0>},
{total_heap_size,12538050},
{heap_size,12538050},
{stack_size,10122096},
{reductions,3795950},
{garbage_collection,[{min_bin_vheap_size,46368},
{min_heap_size,233},
{fullsweep_after,65535},
{minor_gcs,0}]},
{suspending,[]}]

where "{total_heap_size,12538050}," indicates a memory consumption of 12358050 words (32-bit system word size is 4, 64-bit system word size is 8, can be Erlang:system_info ( Wordsize), 64-bit system will be nearly 100M, too exaggerated!

Fifth Step:
Manual GC Recycle, hope the problem can be solved
> Erlang:garbage_collect (PID (0,12571,0)).
True
Look at the process memory again, find that there is no change!GC no back received any resources, so the memory is still playing a role, not recycled!

Sixth step:
Do not suspect the system, first of all to suspect their own code
Careful observation of the code, its approximate structure is as follows:
Send_msg (Socket, Pid)
Try
Receive
{Send, Bin}
...
{inet_reply, _sock, Result}
...
Catch
_:_->
Send_msg (Sock,pid)
End.
The purpose is to loop through the data and then send it, using the Try...catch to catch the exception.
This piece of code is not a tail recursion! Try...catch will save the appropriate information in the stack, the exception capture needs to be placed inside the function, so send_msg Last call is Try...catch, not itself, so not the tail recursion!
Can be verified by code:
Cat Test.erl
-module (Test).
-compile ([Export_all]).


T1 ()
Pid = Spawn (Fun (), do_t1 () end),
Send_msg (Pid, 100000).

T2 ()
Pid = Spawn (Fun (), do_t2 () end),
Send_msg (Pid, 100000).

Send_msg (_pid, 0)
Ok
Send_msg (Pid, N)
Pid!<<2: (N) >>
Timer:sleep (200),
Send_msg (Pid, N-1).

DO_T1 ()
Erlang:garbage_collect (self ()),
Result =erlang:process_info (self (), [Memory, Garbage_collection]),
Io:format ("~w~n", [Result]),
Io:format ("Backtrace:~w~n~n", [Erlang:process_display (Self (), backtrace)]),
Try
Receive
_->
DO_T1 ()
End
Catch
_:_
DO_T1 ()
End.

Do_t2 ()
Erlang:garbage_collect (self ()),
Result =erlang:process_info (self (), [Memory, Garbage_collection]),
Io:format ("~w~n", [Result]),
Io:format ("Backtrace:~w~n~n", [Erlang:process_display (Self (), backtrace)]),
Receive
_,
Do_t2 ()
End.

Version 1:erlctest.erl && erl-eval "test:t1 ()"
Version 2:erlctest.erl && erl-eval "test:t2 ()"
You'll see that the call stack for version 1 code is growing, memory is growing, and the version 2 function call address remains the same, and memory hasn't changed!

Summarize:
1, in the server programming, the cycle must ensure that the tail recursion;
2, use OTP as much as possible, and avoid this problem if you replace the handwritten loop with Gen_server.

Transferred from: http://www.2cto.com/os/201405/299927.html

Optimized solution for memory crunch in Erlang server

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.