A small tool monitors the performance of memcached/Kt/MongoDB/redis

Although many open-source tools use plug-ins or other methods to monitor the internal performance provided by these open-source components (the so-called internal performance is the data obtained by the STAT command class provided by these open-source components ), however, both deployment and expansion are troublesome. In fact, it takes 1-2 days to implement such a tool, and it is also very convenient to scale.

For example, MongoDB:

Another example is redis:

This tool provides the following functions:

1) You only need simple configuration (in the demoCodeYou can configure it to monitor redis, MongoDB, KT, and memcached. The only reason is that we only use this, in fact, it is very convenient to expand, especially when there is.. Net client components.

2) You can select the monitoring granularity. Different monitoring granularities display different time periods on one page, which roughly ensures that about 100 points on one page.

3) for the sake of simplicity, the page is only pulled down. You can modify it to a drag or paging control.

 

The implementation principle of this tool is as follows:

1) MongoDB is used to save a large amount of data. In order to improve performance, the capped set of MongoDB is used, and the removal of historical data does not need to be considered.

2) The 10-second monitoring granularity is the granularity of the raw collected data. Data is collected regularly by the Collector and directly written into MongoDB.

3) Other monitoring granularity is based on the aggregation of raw data. The aggregation machine regularly aggregates data and uses the capped set, but the set size will be smaller.

4) The value type is as follows:

public enum ItemValueType
    {
        TextValue,
        StateValue,
        TotalValue,
        ExpressionValue,
    }
For StateValue, which is the state value, directly stored in the database, for TotalValue, which is the total value, the previous value will be saved in memory and then the next value will be subtracted to obtain the difference after the next value is collected. There is no processing for ExpressionValue, you can implement it yourself. For Mongodb, we choose to monitor these data:

    private static Dictionary <string, ComponentItem> items = new Dictionary <string, ComponentItem>
        {
            {"version", new ComponentItem ("Other", "Version Number", ItemValueType.TextValue)},
       
            {"mem.resident", new ComponentItem ("Memory", "Size of physical memory used", ItemValueType.StateValue)},
            {"mem.virtual", new ComponentItem ("Memory", "Virtual Memory Size Used", ItemValueType.StateValue)},
            {"mem.mapped", new ComponentItem ("memory", "mapped memory size", ItemValueType.StateValue)},
            {"mem.mappedWithJournal", new ComponentItem ("memory", "mapped memory size with journal", ItemValueType.StateValue)},
            {"extra_info.page_faults", new ComponentItem ("Memory", "Number of page faults when loading disk contents", ItemValueType.TotalValue)},

            {"connections.current", new ComponentItem ("connection", "current connections", ItemValueType.StateValue)},
            {"connections.available", new ComponentItem ("connection", "number of available connections", ItemValueType.StateValue)},

            {"network.bytesIn", new ComponentItem ("network", "network read bytes", ItemValueType.TotalValue)},
            {"network.bytesOut", new ComponentItem ("network", "number of bytes sent over the network", ItemValueType.TotalValue)},
            {"network.numRequests", new ComponentItem ("Network", "Network Requests", ItemValueType.TotalValue)},

            {"opcounters.insert", new ComponentItem ("operand", "insertnumber", ItemValueType.TotalValue)},
            {"opcounters.query", new ComponentItem ("operand", "querynumber", ItemValueType.TotalValue)},
            {"opcounters.update", new ComponentItem ("operand", "updatenumber", ItemValueType.TotalValue)},
            {"opcounters.delete", new ComponentItem ("operand", "delete number", ItemValueType.TotalValue)},
            {"opcounters.getmore", new ComponentItem ("operand", "cursor getmore number", ItemValueType.TotalValue)},
            {"opcounters.command", new ComponentItem ("operand", "other operand", ItemValueType.TotalValue)},

            {"indexCounters.btree.accesses", new ComponentItem ("Index", "Number of Access Indexes", ItemValueType.TotalValue)},
            {"indexCounters.btree.hits", new ComponentItem ("index", "number of memory hit indexes", ItemValueType.TotalValue)},
            {"indexCounters.btree.misses", new ComponentItem ("index", "memory miss index times", ItemValueType.TotalValue)},
            {"indexCounters.btree.resets", new ComponentItem ("Index", "Number of Index Counter Resets", ItemValueType.TotalValue)},
            {"indexCounters.btree.hits * 100 / indexCounters.btree.accesses", new ComponentItem ("Index", "hitsratio", ItemValueType.ExpressionValue)},
        };
5) The client is very simple, using highchart + ajax, and directly interact with the WCF server to obtain the jsonp data source.

  <system.serviceModel>
    <behaviors>
      <serviceBehaviors>
        <behavior name = "ServiceBehaviour">
          <serviceDebug includeExceptionDetailInFaults = "true" />
          <serviceMetadata />

        </ behavior>
      </ serviceBehaviors>
      <endpointBehaviors>
        <behavior name = "HttpBehaviour">
          <webHttp defaultBodyStyle = "Wrapped" defaultOutgoingResponseFormat = "Json" helpEnabled = "true" />
          <enableWebScript />
        </ behavior>
      </ endpointBehaviors>
    </ behaviors>
    <bindings>
      <webHttpBinding>
        <binding name = "webHttpBindingJsonP" crossDomainScriptAccessEnabled = "true" />
      </ webHttpBinding>
    </ bindings>
    <services>
      <service name = "Adhesive.ComponentPerformance.Core.Service" behaviorConfiguration = "ServiceBehaviour">
        <endpoint address = "http: // localhost: 8888 /"
                   binding = "webHttpBinding" bindingConfiguration = "webHttpBindingJsonP"
                  behaviorConfiguration = "HttpBehaviour"
                  contract = "Adhesive.ComponentPerformance.Core.Service" />
        <endpoint address = "http: // localhost: 8888 / mex" binding = "mexHttpBinding" contract = "IMetadataExchange" />

      </ service>
    </ services>
  </system.serviceModel>
Without further ado, click here to download the DEMO code. It is recommended that you modify the following places before use:

1) Improve the place to record logs

2) Change the configuration to your own configuration service

3) Improve other open source components that need to be monitored

4) Improve the paging and other functions of the website

 

Finally, I wish you all a Happy New Year!