Tracking OpenLDAP server performance

LDAP has become a standard Internet Directory Access Protocol and is used to access all files from the DNS region to user account information. As enterprises and software vendors rely more on LDAP directory servers, it is imperative to measure the server throughput and performance. This article describes tools that can be used to monitor the running status and Performance Optimization of the LDAP directory server, and explains how ORCA is increasingly applied to the performance monitoring of the Directory Server over time.

Logs

When diagnosing the performance of an LDAP server, the log file is usually viewed at the beginning. The OpenLDAP server provides a flexible log subsystem and defines several log levels to control the details of log files:

Level description

-1 start all debugging

0 no output debugging

1. Trace function call

2. debug data packet processing

4. Deep tracing debugging

8. Connection Management

16 print data packet sending and receiving

32 query filter processing

64. Configuration File Processing

128 process the access control list

256 collect log connection/Operation/result

512 send statistics log entries

1024 print the communication with the shell backend

2048 print entry parsing debugging

Each log level can be appended, and can be configured using the "loglevel" command of slapd. conf, or transmitted to slapd as the parameter "-d" option. The following example shows how to record the access control list and search filter processing details:

$ Slapd-4-f/etc/slapd. conf-u openldap-g openldap \

  -h "ldap://ldap.prefetch.net  ldaps://ldap.prefetch.net" -d 160

OpenLDAP records all information to the LOCAL4 Device of syslog by default. If you want to use a different device, you can configure the device name using the "-l" option of slapd.

Tracking operation time

When LDAP clients and servers are separated by routers and firewalls, occasional network problems (such as lost TCP segments or corrupted CRC) may cause unexpected behavior of applications. To help measure the latency between an LDAP client and the server, we developed a ldap-ping.pl. The ldap-ping.pl is written in Perl and depends on Time: HiRes, Getopt: Std, Net: LDAP and Net: LDAPS modules.

The ldap-ping.pl works by opening a TCP connection to the Directory Server, issuing anonymous bindings, searching RootDSE, and unbinding from the server. These operations are measured using Perl's high-resolution timer and displayed in ping format:

$ Ldap-ping.pl-s ldap.pre fetch.net-p 389-d 10

Querying LDAP server ldap.prefetch.net:389 every 10 seconds (Ctrl-C to stop):

Fri Nov 12 16:42:14 2004: new=0.025s, = bind=0.008s, search=0.067s, unbind=0.003s [local port=50377] [Normal Delay]

Fri Nov 12 16:42:25 2004: new=0.011s, = bind=0.001s, search=0.015s, unbind=0.001s [local port=50378] [Normal Delay]

Fri Nov 12 16:42:35 2004: new=0.010s, = bind=0.002s, search=0.015s, unbind=0.001s [local port=50379] [Normal Delay]

Fri Nov 12 16:42:45 2004: new=0.009s, = bind=0.002s, search=0.015s, unbind=0.001s [local port=50380] [Normal Delay]

The ldap-ping.pl script accepts three parameters; the "-s" option represents the Connected Server, and the "-p" option specifies the TCP port that the directory server listens, the "-d" option allows the Administrator to specify the latency between probes. If the binary pfiles file exists, the script prints the local port number.

Tracking Performance

The OpenLDAP server can be configured to provide real-time performance statistics through the monitor branch command. Currently available statistics include: the number of bytes sent, the entries returned to the client, the total number of connections to the server, the current active connections, read/write waits, and individual operations (for example, read, search, and modify) fault. The following example shows how to set the monitor branch command in slapd. conf and restrict access to the IP address 192.168.1.8:

Database monitor

access to dn="cn=monitor"

        by peername=192.168.1.8   read

        by * none

Once the monitor branch is configured, you can use the "ldapsearch" tool to view all available statistics:

$ Ldapsearch-x-B "cn = monitor"-H ldaps: // ldap.prefetch.net objectclass = *

We can also adjust the search base to obtain individual statistics:

$ Ldapsearch-LLL-x-B "cn = Current, cn = Connections, cn = Monitor"-H ldaps: // ldap.prefetch.net objectclass = *

dn: cn=Current,cn=Connections,cn=Monitor

objectClass: top

objectClass: monitor

objectClass: extensibleObject

cn: Current

description: 46

Collect performance data

The ldapsearch utility allows us to collect performance indicator snapshots at a certain point in time, but how can we use historical data? To solve this problem, I developed a ldap-gather.pl. The ldap-gather.pl script collects statistics from the monitor branch and writes it to the passed directory parameters:

$ Ldap-gather.pl-s ldap.prefetch.net-p 389-d/usr/local/orca/var/orca/ldapallator/ldap.prefetch.net: 389

An initial call to the ldap-gather.pl will generate a text file containing a line of descriptive data headers and a line of actual data:

TIMESTAMP TOTAL_CONNECTIONS BYTES_SENT COMPLETED_OPERATIONS REFERRALS_SENT ENTRIES_SENT BIND_OPERATIONS UNBIND_OPERATIONS ADD_OPERATIONS DELETE_OPERATIONS MODIFY_OPERATIONS COMPARE_OPERATIONS SEARCH_OPERATIONS

1100656501 118 649271 165 0 4620 24 24 0 0 0 0 117

If the data file does not exist, the ldap-gather.pl script creates a new data file and appends new data if the file exists. The file name contains the word "ldapallator" and a date stamp (for example, filename-YYYY-MO-DD-INDEX ):

$ Ls-la | tail-1

-rw-r--r--   1 orca     other       6424 Nov 29 17:25 ldapallator-2004-10-29-000

To automatically collect data at the same time interval, we can set a cron job to run ldap-gather. pl:

5, 15, 25, 35, 45, 55 *** sh-c "/usr/local/etc/ldap-gather.pl

                            -s ldap.prefetch.net 

                            -p 389 -d /usr/local/orca/var/orca/ldapallator/ldap.prefetch.net:389"

This will enable the ldap-gather.pl to collect one-time data every ten minutes. Once the data is captured, we can use ORCA to generate a chart performance report.

Generate charts based on LDAP performance data

This ORCA package contains a set of Perl scripts and configuration files to draw arbitrary data. ORCA uses RRD to store data and configures a configuration file. ORCA uses typical "configure", "make", and "make install" steps to build the software package. ORCA Perl scripts use the Data: Dumper, Digest: MD5, Math: IntervalSearch, RRD, and Storable modules, and use the makemodules_install option to integrate with existing Perl installation. Once the installation is complete, you can execute "orca" to ensure that the build process is successfully completed:

$/Usr/local/orca/bin/orca

/usr/local/orca/bin/orca: no configuration file specified

usage: /usr/local/orca/bin/orca [options] configuration_file

Options:

  -daemon           Run Orca in daemon mode

  -gifs             Output GIFs instead of PNGs

  -logfile filename Output all messages

  -no-html          Update RRD files and images but not HTML files

  -no-images        Update RRD files but not image and HTML files

  -once             Run only once and do not continue to monitor input files

  -verbose          Verbose; list multiple times for increased verbosity

Orca understands the first unique command line option, i.e. -d for -daemon.

If the Perl interpreter cannot find one of the required modules, you will see various console errors and the process will exit. Once the Perl interpreter runs normally, we can create an ORCA profile and start collecting ldap-gather.pl data for further analysis.

The ORCA configuration file contains instructions for finding and generating graphical data. The configuration file contains three main parts. The first part defines several variables (for example, base_dir, rrd_dir, html_dir) to control the storage location of RRD and image files. This section also contains several variables to define the webpage format generated by ORCA.

The second part contains a series of "group" entries that define the data of the generated chart. Sample group entries that match files collected by the ldap-gather.pl are shown in:

Group ldapallator {

find_files              /usr/local/orca/var/orca/ldapallator/(.*)/(?:ldapallator)-\d{4}-\d{2}-\d{2}(?:-\d{3,})?(?:\.(?:Z|gz

|bz2))?

column_description      first_line

date_source             column_name TIMESTAMP

interval                600

filename_compare        sub {

                          my ($ay, $am, $ad) = $a =~ /-(\d{4})-(\d\d)-(\d\d)/;

                          my ($by, $bm, $bd) = $b =~ /-(\d{4})-(\d\d)-(\d\d)/;

                          if (my $c = (( $ay       <=>  $by) ||

                                       ( $am       <=>  $bm) ||

                                       (($ad >> 3) <=> ($bd >> 3)))) {

                            return 2*$c;

                          }

                          $ad <=> $bd;

                        }

}

In this example, the keyword "find_files" tells ORCA what file is used as input (this file matches a regular expression ). The number of seconds defined by the "interval" keyword indicates the interval between updates to this set of data files, while "column_description" describes the position of the column description. "Column_description" value "first_line" indicates that the field description will be located in the first line of each file.

The third part contains a group of items indicating the chart. The following example shows the requirements ?? Command to draw the number of connections of the Directory Server:

Plot {

title                   %g Total Connections

source                  ldapallator

data                    TOTAL_CONNECTIONS

data_type               derive

line_type               line2

legend                  Connections

y_legend                Connections

data_min                0

}

The "plot" keyword defines a new graph with the specified title "title". The "source" keyword indicates the "group" entry of the graph source. Each graph uses the "data" keyword to reference a specific column of data by name (for example, TOTAL_CONNECTIONS is the column of each data file collected by the ldap-gather.pl ); the "data_type" keyword specifies whether the data will be drawn as an absolute value, a derivative of the previous value, or a counter. You can use "data_min" and "data_max" to control the value of the X axis of the curve. Finally, you can use a readable legend description in the PNG file created by ORCA.

You can call ORCA to process and exit data at one time, or set to run with a daemon to continuously check whether new data exists. The following example runs ORCA to process all data since the last call:

$/Bin/sh-c "cd/usr/local/orca &/usr/local/orca/bin/orca-once/usr/local/orca/lib/ldapallator. cfg"

If ORCA runs successfully, HTML and PNG files are generated in the "html_dir" directory. If you do not want to run ORCA as a daemon, you need to add a cron job to process the files cyclically at the same time interval:

0 0 ***/bin/sh-c "cd/usr/local/orca &/usr/local/orca/bin/orca-once

/Usr/local/orca/lib/ldapallator. cfg ">/dev/null 2> & 1

This command runs ORCA's daily jobs in the middle of the night and processes the data of the previous day. Figure 3 contains a complete ORCA configuration file, as shown in figure 4, Figure 5 and 6 charts that contain the total number of ORCA connections, total number of operations, and number of returned entries.

 

Figure 4 Total number of connections

 

Figure 5 total running count

 

Figure 6 Number of returned items

Conclusion

This article provides an overview of several LDAP server monitoring technologies. In our example, we use OpenLDAP servers, but these technologies are also applicable to commercial directory servers. For other information about ORCA, it can be used to plot arbitrary data and other aspects, see references. If you have any questions or comments about this article, please feel free to email the author.

Reference

This article references the following content:

• ORCA WebsiteOpenLDAP websiteldap-ping.pl Perl scriptldap-gather.pl Perl script Thank you

  Thanks to Clay mcluer for his original ldap-ping.pl work and to ORCA and OpenLDAP team members for their outstanding contributions!