Improve Linux kernel and scalability to adapt to Enterprise Environments

Improve Linux kernel and adapt scalability to the enterprise environment-general Linux technology-Linux programming and kernel information. The following is a detailed description. The first step to improve Linux performance is to quantify it. But how can we precisely quantify the performance of Linux or its equivalent system performance? In this article, members of the IBM Linux technology center described several benchmark program tests they performed on Linux 2.4 and 2.5 kernels at the end of last year. Based on these expert experiences, they are intended to serve readers.

Currently, the Linux operating system is one of the most successful open source projects. Linux, as a Web server operating system, demonstrates its high reliability. It occupies a large share in the Web server market. These Web servers are usually low-end and intermediate-grade systems with up to four symmetric multi-processor (SMP); enterprise-level systems have more complex requirements, for example, you need more processors, I/O configurations, and large memory and bandwidth. In order to make Linux ready to enter the enterprise environment and to enter the SMP market as a commercial application, compared with commercial UNIX systems, its SMP scalability, disk and network I/O performance, scheduler, and virtual memory manager must be improved.

Linux Scalability Effort (LSE) is an open source project that solves the Linux kernel problems for enterprise-level machines, the scalability of these machines is 8 or higher.

The Linux Performance Team of the IBM Linux Technology Center (LTC) actively participated in the LSE activity. In addition, they aim to improve Linux kernel performance (especially for SMP scalability.

This article describes the strategies and methods used by the Team to measure, analyze, and improve the Linux kernel performance and scalability while focusing on platform independence. The team used a set of benchmarking procedures to complete the task. These benchmarking programs take into account various workloads, including Web Services, databases, and file services. In addition, we will show you the kernel components (such as the disk I/O subsystem) that each benchmark program focuses on ).

　　 Analysis Method

Here we will discuss how to quantify the Linux performance of SMP scalability. If you want to, you can directly jump to the benchmark test program result section.

Policies we use to improve Linux performance and scalability include running several industry-accepted and component-level benchmarking programs, selecting suitable hardware and software, and developing program running rules for Benchmark Testing, set performance and scalability goals, and measure, analyze, and improve performance and scalability. These processes are described in detail in this section.

Performance is defined as the raw throughput on a single processor (UP) or SMP. We separate SMP Scalability (CPU) from resource Scalability (for example, the number of network connections.

　　 Hardware and software

Most of this work uses the IA-32 (x86) architecture, from one to eight processors. We also studied issues related to the IA-32 and NUMA IA-64 architecture for future use of inconsistent memory access (NUMA. The hardware selection is usually based on the benchmark test program and related workloads. Software selection should be combined with IBM's Linux middleware policies and/or open source middleware. For example:

　　 Database

We use the database benchmark test program, while on the hardware, we use eight SMP systems with large disk configurations. The database software uses IBM DB2 for Linux, and the SCSI controller is IBM ServeRAID 4 H. This database is for 8-way SMP.

　　 SMB File Service

The benchmark test program is NetBench, and the hardware is a 4-way SMP system. Up to 48 clients can drive the SMP server. The middleware is Samba (Open Source Code ). SMB file service is for 4-way SMP.

　　 Web Services

The benchmark test program is SPECweb99, the hardware is 8 SMP, and has a large memory configuration, up to 32 clients. This benchmark is only used for research purposes and is not used. (For more information about this, see the benchmarking procedure section ). The Web server is Apache, which is the foundation of the ibm http server. We chose 8-way SMP to study scalability, but Apache because it supports the measurement and analysis of the next-generation posix thread (NGPT) (see references ). In addition, it is open source code and the most popular Web server.