. The course of TCP reverse proxy, Socket connection pool and packet parser in net development

TCP Reverse Proxy

General Web reverse Proxy is familiar to everyone, mainly by setting up a proxy server between the client and service side, forwarding the client's request to the service side or the database, and returning the result to the client.

The main features are:





1, cache Some database I/O is too heavy, but update infrequent data, or static data, such as files, pictures and so on.


2, the isolation Client (public network) and the service side (Windows Service, Web service, file service), only the reverse proxy server IP, name, host and port, etc. exposed to the public network.


3, based on the 2nd, it should be lightweight, can be restarted at any time, this is very useful when the server itself is in a higher cost or can not tolerate the restart of the service.





For example, the server itself needs to handle a lot of business logic, may involve recalculation (high CPU and memory requirements), heavy I/O (High disk and network requirements), or mixed types, the server's machine costs are high because of the need for more powerful CPUs, larger capacity memory, and faster disks and networks, If you also need to consider DDoS and CC defense capabilities, the server's machine costs will increase dramatically.





At this point, you can consider the reverse proxy technology, the choice with hard defense, the configuration is lower than the service end of a lot of cheap machines, as a reverse proxy server (such as a cloud of X cloud Host, with 5G hard, but its non-SSD cloud disk I/O ability is poor, at this time not as a Business Server host machine, but can be as a reverse proxy server), To form the reverse proxy distribution cluster.





DDoS attacks, traffic needs to converge to a peak, will kill with the machine, and according to the DDoS attacker's traffic to suppress the difference between the machine and network conditions, which usually takes a while, through the reverse proxy distribution cluster, a reverse proxy was killed, dead or black hole window usually within half an hour to a few hours, If you can guarantee a relatively abundant reverse proxy reserve, so that the entire cluster before the death of the machine can be jumped out of the black hole to rejoin the cluster of agents to provide services to clients, then can form a confrontation. Even if you don't have enough reserves, you can at least gain more time for the decisions that are being attacked.





To sum up, the reverse proxy technology by increasing the additional network transmission time, but obtains many client and the service side direct connection not to have the superiority.





Common Web application servers, such as Nginx, can provide reverse proxy capabilities.





However, the TCP-level reverse proxy is still relatively small.





The project at that time forced out such a demand, which used some basic components as follows.





Connection Pool





If the client uses a TCP protocol and a reverse proxy server for communication, such as a common desktop client, consider connecting to the proxy server in a single long connection + asynchronously.





And more than one proxy server and the real business services between the end, because the agent and service between the more synchronous communication, in order to efficiency, you can consider the use of multiple connections + pooling technology, so that the connection between long and short, the combination of the advantages of both.





The following is the actual use of the connection pool code in the project, which is referenced in the MongoDB C # driver section at that time:

 <summary>///  Connection pool///  features and updates:/// 1: From a single remove unavailable connection, to bulk removal/// 2: Remove connection no longer prevent, exposure to heavy storm risk///
    Purpose: As soon as possible to find the connection is not used to prevent the request failed///    consider: Generally only open 200 connections, no big problem.  5: Increase the number of queued threads and queue timeout, considering: network jitter and business layer slow operation/// 6: Increase the maximum connection survival and idle time, consider: network jitter and business layer slow operation/// 7: Maximize load request and mitigate   A moment   pass to the main number of requests and connections/// </summary> Public class sessionpool {    
Private object _poollock = new object ();     public int poolsize { get; set; }      public ilist<synctcpsession> avaliablesessions     {         get { return _avaliablesessions; }    
 }     public ILog Logger;
    private int _waitQueueSize;
    private bool _inMaintainPoolSize;    private bool _inEnsureMinConnectionPoolSizeWorkItem;     private IList<SyncTcpSession> _avaliableSessions = new 
List<synctcpsession> ();     public int maxwaitqueuesize { get; set; }      public int maxconnectionpoolsize { get; set; }      public int minconnectionpoolsize { get; set; }      public timespan waitqueuetimeout { get; set; }     /// 
<summary>     ///  Connection Maximum survival time (min)     /// </summary>
    public timespan maxconnectionlifetime { get; set; }     /// <summary>     ///  Connection Maximum idle time (sec)      ///&nbSp;</summary>     public TimeSpan MaxConnectionIdleTime { get;  set; }     public IPEndPoint RemoteAddr { get; set; &NBSP}     public sessionpool (Ilog log)     {  
      Logger = log;     }     /// <summary>     ///  Get Available Connections     /// </summary>     /// <returns></ Returns>     public synctcpsession getavaliablesession ()      {        lock  (_poollock)      
   {            //The thread that is waiting to get the connection has a serious backlog            &nbsp//indicates that the number of connections is insufficient to meet the business, or that the business layer handles the backlog             // Consider optimizing the business tier and database or increasing the timeout time             if  (_ Waitqueuesize >= maxwaitqueuesize)              {                 Var ex = new exception ("Too many threads waiting to get the connection!")
");                 logger.error ( Ex.
MESSAGE, EX);                 return 
Null             }       
      _waitQueueSize += 1;             try    &Nbsp;        {                 DateTime timeoutAt = DateTime.Now + 
Waitqueuetimeout;                 while  ( true)                 {                      //have available connections                      if  (_avaliablesessions.count > 0)                      {                           //first try to find a connection that has been opened                          for  (int i = _ avaliablesessions.count - 1; i >= 0; i--)       
                  {                              if  (_avaliablesessions[i). State == sessionstate.open)                              {                                  var connection = _avaliablesessions[i];                                  _
Avaliablesessions.removeat (i);                   
              return connection;                              }                          }                           //Otherwise remove the least recently used connection and return to the new connection                          avaliablesessions[0].
Close ();                   
      avaliablesessions.removeat (0);                   
      return new synctcpsession (this);                      }                      //no connection available, new connection                      if  (poolsize < maxconnectionpoolsize)                      {                      
   var connection = new synctcpsession (this);                   
      PoolSize += 1;                   
      return connection;                      }                 
    //cannot create a new connection and no connection is available, waiting for the connection to be reclaimed.                     var timeremaining = timeoutat - datetime.now;                      if  (Timeremaining > timespan.zero)                      {                        
 monitor.wait (_poollock, timeremaining);                      }                      else                      {                        //wait timeout to indicate that the number of connections is insufficient to meet the business or the business layer processes backlogs, consider optimizing business tiers and databases, or increasing timeout times                          var ex = new 
TimeoutException ("Wait synctcpsession has timed out.");                          logger.error (ex.
MESSAGE, EX);                   
  }                 }             }              finally              {  &Nbsp;             _waitqueuesize -= 
1;             }         &NBSP}     }     /// <summary>        Empty connection pool     /// </summary>     public void  Clear ()     {        lock  (_poollock)          {             foreach  (var connection in avaliablesessions)              {                 connection.
Close ();             }
            avaliablesessions.clear ();
            PoolSize = 0;
            monitor.pulse (_poolLock);
            logger.info ("Connection pool is empty.");         }          /// 
<summary>     ///  How many connections to maintain connection pools     /// </summary>     public void maintainpoolsize ()     {   
     if  (_inmaintainpoolsize)         {
            return;         }         _inmaintainpoolsize = true;         try         {             IList<SyncTcpSession> 
Connectionstoremove = new list<synctcpsession> ();             lock  (_poollock)              {        
        var now = DateTime.Now;                 // Changed to: Remove all unavailable connections, expose connection storm risk, but consider actual business connections rarely idle, connect storm risk less                  for  (int i = avaliablesessions.count - 1;  i >= 0; i--)                 {                     var connection =
 AvaliableSessions[i];                      if  (now > connection. Createdat + maxconnectionlifetime                          | |  now > connection. Lastusedat + maxconnectionidletime                          | |  connection. IsConnected ()  == false)                      {                         //is closed            after maximum life, idle time, or not connected               //Add Delete Collection                       
   connectionstoremove.add (connection);                          //removal of             from available connections                
Avaliablesessions.removeat (i);                      }                  }                  // }             &NBSP}             //removed from the lock              if  (Connectionstoremove.any ())              {         
       int i = 0;                 foreach  ( Var conntoremove in connectionstoremove)                  {                     i++;                   
  removeconnection (Conntoremove);                 }                  logger.infoformat ("
Bulk Remove Connection: quantity {0}. ",  i);             }              if  (poolsize < minconnectionpoolsize)              {                 threadpool.queueuserworkitem (Ensureminconnectionpoolsizeworkitem,
NULL);             }          }         finally         {       
     _inMaintainPoolSize = false;         }     }     private  void ensureminconnectionpoolsizeworkitem (object state)     {         if  (_inensureminconnectionpoolsizeworkitem)          {            return;          }         _
inensureminconnectionpoolsizeworkitem = true;         try         {             while  (True)              {                 lock  (_poollock)                  {                     if  (poolsize >= minconnectionpoolsize)   
                  {                   
      return;                   
  }                 }              &nbsP;  var connection = new synctcpsession (this);                 try                  {                     var
 added = false;                      lock  (_poollock)                      {                         if  (poolsize  < maxconnectionpoolsize)                           {                             
Avaliablesessions.add (connection);                   
          PoolSize++;                   
          added = true;                   
          monitor.pulse (_poollock);                          }                     }      
               if  (!added)                      {                         connection.
Close ();                   
  }                 }                 catch                  {              &nbSp;      thread.sleep (Timespan.fromseconds (1));                 }   
          }         }         catch         {        &NBSP}         finally          {            _
inensureminconnectionpoolsizeworkitem = false;         }          ///  <summary>     ///  Recycle connection     /// </summary>      /// <param name= "Connection" ></param>   &Nbsp; public void releaseconnection (synctcpsession connection)     {         //Close the connection each time, then retire to short connection        
 // removeconnection (connection);
        // return;         if  (connection == null)     
        return;         if  (connection. Sessionpool != this)         {             connection.
Close ();
            logger.info ("Connection does not belong to this connection pool.");        &NBSP}         if  ( Connection. State != sessionstate.open)         {            
Removeconnection (connection);
            logger.info ("Remove connection: Connection is closed.");
            return;        &NBSP}         if  ( Datetime.now - connection. Createdat > maxconnectionlifetime)         {  
          removeconnection (connection);
            logger.info ("Remove connection: exceeds maximum surviving time.");
            return;        &NBSP}         lock  (_ Poollock)         {            connection.
lastusedat = datetime.now;
            avaliablesessions.add (connection);
            monitor.pulse (_poolLock);         }          ///  <summary>     ///  Remove and close connection     /// </summary>      /// <param name= "Connection" ></param>     private  void removeconnection (synctcpsession connection)     {         lock  (_poollock)         {                  Avaliablesessions.remOve (connection);                
  PoolSize -= 1;                 monitor.pulse (
_poollock);        &NBSP}         connection.
Close ();     }

In the code above, the

uses the synchronous connection Synctcpsession, which is used primarily for synchronous communication between servers, and is guaranteed to have the following characteristics:

    associated with, and managed by, the connection pool.
    no status, network failure clean connection, timeout and retransmission is handled by client.
    provides synchronous communication capabilities. The
    network between deployment and business servers is intranet, simplifying packet parsing.
    do not use buffer pool, use only MemoryStream object, on-demand construction object, this consideration is based on the agent task is not very heavy, forwarding, cache supplemented, and there are more than one cluster of components, the overall memory relatively abundant (by contrast, The Business Server uses a buffer pool. The

code is as follows:

 <summary>///  Synchronization Session object/// </summary> public class synctcpsession { 
   private readonly object _instancelock = new object ();
    private readonly ILog _logger;
    private readonly IPEndPoint _remoteAddr;
    private readonly SessionPool _sessionPool;
    private Socket _socket;
    private SessionState _state = SessionState.Initial;     public sessionpool sessionpool { get { return _ SESSIONPOOL; }&NBSP}     public synctcpsession (Sessionpool pool)      {        _logger = pool.
Logger;
        _sessionPool = pool;         _remoteAddr = _sessionPool.RemoteAddr;
        CreatedAt = DateTime.Now;         _socket = new socket (AddressFamily.InterNetwork ,  sockettype.stream, protocoltype.tcp)                        {SendTimeout = 90000, 
receivetimeout = 180000};     }     public datetime createdat { get; set ;  }     public datetime lastusedat { get; set;       public sessionstate state { get { return _state; } &NBSP}     public void connect ()     {         _socket.
Connect (_REMOTEADDR);
        LastUsedAt = DateTime.Now;
        _state = SessionState.Open;    &NBSP}     public void close ()     {         lock  (_instancelock)          {            try              {                 if  (_socket == null | |  _state == sessionstate.closed)           
          return;                 if&nBSP; (_socket.
Connected)                 {                      try                      {                         _socket.
Shutdown (Socketshutdown.both);                      }                      finally                      {                          _socket.
Close ();                          _socket.
Dispose ();                   
      _socket = null;                      }                 
    _state = SessionState.Closed;                 }              }              catch         &NBsp;   {                        //  _logger.
Info ("session is closed");         }          /// 
<summary>     ///  Get current connection     /// </summary>     /// <returns></returns>     public bool  IsConnected ()     {        if (_state!= Sessionstate.open)         {       
     return false;         }         byte[] 
tmp = new byte[1];         try         {             _socket.
blocking = false;             _socket.
Send (tmp, 0, 0);             _socket.
blocking = true;
            return true;         }catch (Socketexception ex)          {            _logger.
Error ("[not connected]");
            return false;         }          public  Bool send (byte[] sentbytes)     {        lock  (_instancelock)         {   
         LastUsedAt = DateTime.Now;             if  (_state ==  sessionstate.initial)             {                 //fail to return if no connection succeeded                  if (! Open ())                 {                    
 _state = SessionState.Closed;                      return false;                 }              }              //failure to return if no connection is currently in              if (! IsConnected ())             {                 _state = 
sessionstate.closed;                 return 
False             }              //the connection may have been turned off at this time           
  int allLen = sentBytes.Length;             try              {                 while  (alllen > 0)                  {                     //sends data to a buffer, does not guarantee immediate delivery across the network, may send timeouts                      int sent = _ Socket.
Send (Sentbytes, 0, sentbytes.length, socketflags.none);                   
  allLen -= sent;                 }                  lastusedat = datetime.now;                 return 
True             }              catch  (Socketexception ex)              {                 //If an error occurs, return failure                  _logger. Error (String. Format (' [send failed] {0} ',  ex.
message),  EX);                 _state =
 SessionState.Closed;                 return false;             }         &NBSP}     }     public byte[] receive (Out bool  successful)     {        successful =
 false;
        const int headerLen = 8;
        byte[] ret = null;
        bool foundHeader = false;
        LastUsedAt = DateTime.Now;         if  (_socket == null | |  !_socket.
Connected)  return null;         lock  (_instancelock)          {            //  deployment environment, internal network is relatively stable, simplifying packet resolution       
      var buffer = new byte[16*1024];
            int remaining = -1;             using  (var ms =  New memorystream ())             {                 try                  {                     while  ( remaining != 0)                      {  &Nbsp;                      int alllen = _socket. Receive (Buffer, 0, buffer.
Length, socketflags.none);                          if  (!foundheader)                          {                              if  (Alllen >= headerlen)                               {                               
  foundHeader = true;                                  int bodylen =   (buffer[4] << 24)  +  (buffer[5] << 16)  +  (Buffer[6]  << 8)  +                                 
               buffer[7];                                  remaining =&nBSP; (int)   (headerlen + bodylen - ms.
Length);                              }                          }                           ms.
Write (Buffer, 0, alllen);                          if  (Foundheader)                          {                             remaining -= alllen;                          }                      }                      ret = new byte[ms.
Length];                      ms.
position = 0;                      ms. Read (Ret, 0, ret.
Length);                   
  LastUsedAt = DateTime.Now;                     successful =
 true;                   
  return ret;                 }                  catch  (Exception
 EX)                 {                   
  successful = false;                   
  _state = SessionState.Closed;                     _logger. Error (String. Format (' [recv failed] {0} ',  ex.
message),  EX);                 }   
          }         }
        return ret;    &NBSP}     public bool open ()     {         try         {  
          connect ();
            return true;        &NBSP}         catch  ( EXCEPTION EX)         {             _state = sessionstate.closed;             _logger. Error (String. Format (' [open failed] {0} ',  ex.
message),  EX);
            return false;         }          public  Bool sendrequest (byte[] package)     {       
Return send (package);    &NBSP}     public byte[] getbody (Byte[] data)      {        if  (data == null | |  data. length < 1)         {      
      return null;         }         if  (data. LENGTH < 8)         {             _logger.
Error ("Insufficient packet length received");
            return null;         }         int  bodylen =  (data[4] << 24)  +  (data[5] << 16)  +  ( DATA[6] << 8)  +             
          data[7];
        var body = new byte[bodyLen];         if  (Bodylen + 8 != data. Length)         {    &nbsP;       _logger. Errorformat ("Packet length error: Totallen: ({0}), bodylen:{1}",  data. Length, body.
Length);
            return null;         }         
Buffer.blockcopy (Data, 8, body, 0, bodylen);
        return body;    &NBSP}}

Packet resolution

to provide the ability to send data in a fragmented, and to avoid sticky problems, package resolution is required, and the complete version of the package parsing code provides the following characteristics:

     can distinguish whether the received byte stream contains a complete package, if it throws an event, if not, continue to receive the data. The
    can detect whether the header is complete. The
    can detect whether the package is intact.
    Network quality is poor (foreign exchange domestic, non-LAN, trans-Telecom, Unicom) using this parser, client and agent communication between the use of this parser. The

code is as follows:

 <summary>///  Package Parser/// </summary> Public class packageanalyzer {     public packageanalyzer (Int headerlen)     {   
     _headerLen = headerLen;
        _header = new byte[_headerLen];
   &NBSP}     /// <summary>     ///  Header length     /// </summary>     private readonly int _
Headerlen;     /// <summary>     ///  Baotou Buffer     /
 </summary>     private readonly byte[] _header;     /// <summary>     ///  How many bytes to make up a complete package      /// </summary>     private int _requireddatalength;     /// <summary>     ///  Header Protocol identifier bytes received in byte array      /// </summary>     private int _receivedheaderlength
;     /// <summary>     ///  Baotou Get status    
 /// </summary>     private Header _headerFlag;     /// <summary>     ///  Baotou Status     /  </summary>     private enum header     { 
       notfound, found, partialfound     }     /// <summary>     ///  header has been saved    
 /// </summary>     private bool _headerWritten;     /// <summary>     ///  Packet Memory     ///  </summary>     public BufferWriter Writer { get; set; &NBSP}     /// <summary>     ///  whether to allow variable length storage      /// </summary>     public bool EnabledVariant {  get; set; }     /// <summary>     ///  Processing delegates     /// </summary>     /// <param  when package analysis succeeds Name= "Requestinfo" ></param>     public delegate void 
Onanalyzesuccess (Binaryresponseinfo requestinfo);     /// <summary>     ///  analyze header from byte stream and get full package      /// </summary>     /// <param  name= "Data" > Byte stream </param>     /// <param name= "offset" > offset </param >     /// <param name= "Total" > Gross bytes </param>       <param name= "Onanalyzesuccesscallback" > Subcontracting Success Callback </param>     public  void tryprocess (byte[] data, int offset, int total, onanalyzesuccess  onanalyzesuccesscallback)     {        while   (total > 0)         {             //has not yet acquired the head              if  (_headerflag == header.notfound)              {                 //remaining                  if  (Total >= _headerlen)                  {                     //Get full head           
          _headerFlag = Header.Found;                   
  array.copy (Data, offset, _header, 0, _headerlen);                   
  offset += _headerLen;                      total -= _headerlen;                      //Get Data length                      _requiredDataLength =  (_header[4] << 24)  +  (
_HEADER[5] << 16)  +  (_header[6] << 8)  + _header[7];                   
  _receivedHeaderLength = 0;                      //Continue processing                 Insufficient  } //                 else                 {                     array.copy (Data, offset, _header,
 0, total);                   
  _receivedHeaderLength += total;                   
  _headerFlag = Header.PartialFound;                   
  break;                 }              }              //has acquired head             if  (_headerflag == header.found)              {                 //can get complete data                  if  (total >= _requireddatalength)                  {                     //Save Data                      //not yet written to head                      if   (!_headerwritten)                      {                         writer = new bufferwriter (
SYSTEM.TEXT.ENCODING.UTF8);                   
      _headerWritten = true;                   
      writer.write (_header, 0, _headerlen);                      }                 
    writer.write (data, offset, _requireddatalength);                     offset += _requireddatalength;                   
  total -= _requiredDataLength;                      //Reset All Status                
     _requiredDataLength = 0;                   
  _receivedHeaderLength = 0;                   
  _headerFlag = Header.NotFound;                   
  _headerWritten = false;                     //got the full package, Start reading Messages                      //////////////////////////////////////////////////////                     var reader =  new bufferreader (System.text.encoding.utf8, writer)  { EnabledVariant = 
enabledvariant };                   
  BinaryResponseInfo responseInfo;                   
  messageread (Reader, out responseinfo);                   & nbsp; if  (responseinfo != null)                          if  ( Onanalyzesuccesscallback != null)                              
Onanalyzesuccesscallback (Responseinfo);                      //////////////////////////////////////////////////////                 }                  //cannot get complete data                  else                 {                     //Save Data                      //not yet written to head       
              if  (!_headerwritten)                      {                         writer = new bufferwriter (
SYSTEM.TEXT.ENCODING.UTF8);                   
      _headerWritten = true;         &nbsP;               writer.write (_
Header, 0, _headerlen);                      }                      //Write Data              
       writer.write (data, offset, total);                   
  _requiredDataLength -= total;                   
  break;                 }             &NBSP}             //part    
         if  (_headerflag == header.partialfound)             {                 //cannot get full head                  if  (total + _receivedheaderlength  < _headerlen)                  {                     array.copy (data, offset, _header, _receivedheaderlength, 
Total);                      _receivedheaderlength += total;                   
  break;                 }                  //can get full head                  else                  {                     _headerflag =
 Header.Found;                   
  var delta = _headerLen - _receivedHeaderLength;              &Nbsp;      array.copy (data, offset, _header, _
Receivedheaderlength, delta);                   
  total -= delta;                   
  offset += delta;                      //Get Data length                      _requiredDataLength =  (_header[4] << 24)  +  (
_HEADER[5] << 16)  +  (_header[6] << 8)  + _header[7];                      _receivedheaderlength = 0;                      //Continue processing                  }                      &NBSP}     }     /// <summary>      ///  get data length from Baotou     /// </summary>     /// < Param name= "Buffer" > Baotou </param>     /// <param name= "offset" > Offset </param>     /// <param name= "Length" > Length </param>      /// <returns></returns>     public virtual int  Getdatalengthfromheader (byte[] buffer, int offset, int length)   &nbsp {        //5th, 6, 7, 8 byte identification length          return  (buffer[offset + 4] << 24)  +  (buffer[offset +  5] << 16)  +  (buffer[offset + 6] << 8)  +                  (buffer[offset +
 7]);     }     /// <summary>     ///  Get message entity     /// </summary>     /// <param name= " Reader "> Data Flow Reader </param>     /// <param name=" Requestinfo "> Request Information </ Param>     private void messageread (bufferreader reader, out  Binaryresponseinfo requestinfo)     {         var&Nbsp;package = reader.
Tobytes ();         //Service identification (which service is invoked on the business end)        
 var servicekey = system.text.encoding.utf8.getstring (PACKAGE, 0, 4);         var bodyLen =  (package[4] << 24 )  +  (package[5] << 16)  +  (package[6] << 8)  +  (
PACKAGE[7]);         //  system.diagnostics.debug.assert (bodyLen 
> 0,  "Packet data is empty");
        var body = new byte[bodyLen];         buffer.blockcopy (Package, _headerlen, body, 0,
 bodylen);         requestinfo = new binaryresponseinfo ()  {  key = servicekey, body = body };    &NBSP}}

Because of historical reasons, some naming is not very appropriate, the code structure is not very good, such as the above class, called parser may be more appropriate.