The open-source community is challenging the extreme speed of Linux kernel development

The open-source community is adding new features to the Linux kernel at an extremely fast speed, while working hard to fix bugs. Slowing down development seems unlikely. & hellip; the most

The open-source community is adding new features to the Linux kernel at an extremely fast speed, while working hard to fix bugs. It seems impossible to slow down development ......

The latest version of Linux kernel was released this month. it shows a series of dazzling changes. 2.6.23 kernel is released only three months away from the last kernel update. It adds new features suitable for commercial use, including better support for virtualization technology and sched) and new device drivers and bug patches.

Every two or three months, we can see a lot of changes in the "code tree" maintained by Linux Torvalds, which shows that the development speed of the Linux kernel is accelerating. So far, this development process has produced undeniable high-quality and reliable code.

But you may not have imagined that Torvalds is pushing the speed of open-source development to a new limit. As the size and complexity of the Linux kernel increase, this fast iterative update puts a lot of pressure on the volunteer community responsible for testing and modifying bugs.

Torvalds cannot slow down because of two reasons. First of all, Linux cannot lag behind in technology, otherwise it will lose business users with increasingly demanding requirements. Therefore, hooks are added to the new kernel to take advantage of the latest virtualization capabilities embedded in Intel and AMD processors. The second reason is that Linux needs to promote the development of the developer community. The continuous addition of new features can free developers from getting bored. Otherwise, they may move to other projects and attract new talents when existing developers get older or quit.

The informal and unpredictable new feature roadmap of Linux is derived from the above pressure, which is also a continuous driving force for Linux to ensure system quality and stability while adding new features. Can a 16-year-old open-source project continue to develop in the next 16 years? "No other open-source projects can be as large and fast as Linux," said Dan Frye, vice president of IBM who has been paying attention to the development progress of Linux kernel. "This is a developer community that dares to be the first in the world ."

Commercial users rely on this hasty development process to improve many aspects of Linux, including not only virtualization technology, but also power management and security. These rapid changes in the kernel may take up to two years to be reflected in the Red Hat and Novell commercial Linux releases. Most users use the Linux systems launched by these companies, so they get a buffer zone that is crazy about the development of Linux kernel. Even so, the development of the kernel will determine the future of Linux.

On average, Linux adds 2000 lines of code per day, even though Trovalds aims to limit the number of codes added to the kernel to make it as efficient as possible. The Linux modular kernel is the core of the operating system and is responsible for processing all common tasks, such as memory management, CPU scheduling requests, and input/output. It is surrounded by hundreds of additional software packages for more specific tasks, such as transferring files between Linux and Windows systems, or configuring files for display on Apache Web servers. However, the kernel must be constantly developed to handle more features and hardware and meet the needs of more users. When Linux was born in 1991, it was only 10250 lines of code written by operating system enthusiasts. Currently, more than 8 million lines of code are available.

Some people think that the average hourly increase of 86 lines of code in the Linux kernel is beyond the limit of software development. Alan Cox, a key maintainer of Linux, has warned that some device driver modifications should be tested before being added to the kernel. Andrew Morton is an outstanding Linux developer and named "kernel Colonel" after being appointed as the Linux kernel owner by Torvalds. He also expressed his frank views on the unmodified bug of Linux. "I want developers to spend more time fixing bugs and less time adding new features," Morton said. "This is my personal opinion ".

However, at a recent Linux Kernel Summit in Cambridge, England, Torvalds said he had made an overly cautious mistake in the past. He believes that the slow release of the kernel will lead to the development of Linux, because various new features are waiting to be added to the kernel. If feedback from kernel maintainers and trusted developers is not received in a timely manner, community contributors will lose interest in continuing to submit code. (Torvalds did not respond to our interview request .)

Torvalds has done some operations to speed up the development of the Linux kernel. He relies on a basic open-source idea: compared with more organized software testing, a large number of users are more likely to detect software defects by testing frequently released code. New code added to the kernel may cause Linux errors once it fails to run on some hardware or conflicts with other programs, whether in or out of the kernel. When a new code has a bug, people expect the developers who submitted the code to solve the bug, but they usually do not solve it.

Jonathan Corbet, a kernel developer, told us in a document released by the Linux Foundation that at the Cambridge summit, morton said that he really wanted to appoint a "annoying guy" as the Linux kernel bug Administrator. he hoped this person could find the source of the bug, then, "the developers who do not want to modify the bug will be flushed ". Natalie Protasevich is the currently appointed bug administrator. In Morton's view, Protasevich has brought more discipline to the Linux kernel bug cleanup, despite her lack of a tough style she prefers. The Linux kernel's Bugzilla database records more than 1500 bugs at one time, and has now reduced to 1400.

"An extremely subtle balance is being formed between a fast development process and complete code reviews," said Dirk Hohndel, head of Linux and Open Source Technology at Intel. Even in the current rapid development progress, developers want to add or users have the required features, not everything can be added to the kernel.

Linux business users may be frustrated with this process. For European travel service provider Amadeus, Linux is the key to its strategy to reduce IT infrastructure costs. Fred Bessis, vice president of the company's technology and strategic planning, said the company reduced device spending by about 10% by eliminating mainframe systems and then running Linux on "cheap hardware. The company has more than ten years of experience using Linux. it understands the current situation of Linux and has witnessed a long process of potentially useful new features leading to the Linux commercial version.

Holger Weisbrodt, a senior system programmer at Amadeus, said that new hardware and drivers will soon be available in the kernel, however, the new bug diagnosis and debugging tool "takes a long time to be added to the kernel ". He hopes that Linux kernel developers will pay more attention to common bug debugging tools.

The development process of the new Linux kernel shows this unpredictability, embodied in the two new features it contains: The new scheduler and improved virtualization support. These two features are completely different from the kernel development path, and both have their own risks and complexity.

Be able to endure criticism

To adopt a new feature, it may go through a long and frustrating process. The Linux kernel's most important recently-the new scheduler-is one of the examples. The kernel scheduler combines the fairness and time-sharing features of Unix systems with the preemption and fast interruption features of real-time operating systems. The former can handle more tasks and users, the latter can quickly respond to emergencies. In commercial operating systems, these two features are often implemented as different functions, while Linux tries to implement them simultaneously.

Kernel code contributors have spent many years developing schedulers, but one of them, Australian doctor Con Kolivas, set off a wave of waves throughout the open-source community this summer, he published a heated discussion article in The Australian PC magazine about why he was frustrated and exited Linux development.

He previously compiled code called "-ck patchset" for kernel 2.6.23, which is reviewed by Ingo Molnar. Molnar, a developer of RedHat, has contributed to the development of multiple schedulers and has become one of the trusted experts in the scheduler field. He found that the code submitted by Koliva has defects in the real-time scheduling, so he compiled his own multi-purpose scheduler version based on this code. This practice of modifying and publishing code to others is advocated by the general public protocol (GPL), and the kernel maintainer will try its best to pay tribute to the initial contributor of the code. But Kolivas had previously learned that his code was not accepted, so he was a little angry about it.

Kolivas encountered a problem that caused obstacles to every developer. His design idea is to use different schedulers according to different types of tasks. Torvalds and its partners believe that they can do things well once through basic skills, and oppose using multiple alternative methods to do the same thing. They believe that this will make software maintenance easier and make interaction between different subsystems in the kernel easier to predict. Tornalds uses its own influence to promote this architecture principle, and most participants believe in this principle in the mailing list discussing new code. In the discussion in this list, the code written by Kolivas experienced a rare fiasco. "On the Linux kernel mailing list, the comments made by the author to the code written by other developers may be harsh. if someone says the same in a business company, he may be fired, "Joel Berman, product management director of RedHat, who has been subscribing to this mailing list, said.

The Scheduler included in the kernel version 2.6.23 is called the Completely Fair sched. it is a bit ironic to consider the compromises that occur during the Scheduler development process. Not only are Kolivas angry, but there are other people who want the scheduler to achieve better real-time performance. they hope to add these improvements to the kernel next year.

Rapid development of virtualization technology

Compared with the long-standing debate on schedulers, Avi Kivity is much simpler. The Israeli developer submitted a virtualization engine called KVM, which contains a huge 12000-line code package. When the code contributor submits a patch, it is more advantageous for kernel developers and maintainers to know their names. But "The arrival of KVM was very sudden," Morton said. "I have never heard of Kivity myself or his Qumranet company before ".

Kivity describes itself as a "long-term diver" on the kernel mailing list. he enjoys reading the content and is familiar with attacks and debates among experts, but he has not submitted much code. He designed KVM according to the Linux kernel standards he had learned, so that Linux experts familiar with kernel file systems could keep abreast of the latest code and respond to the questions and comments raised by kernel maintainers in a timely manner. KVM meets an important requirement of Linux, because with the increasing attention of virtualization technology, the Linux kernel finally has the corresponding functions, which can use the virtualization interfaces provided by Intel and AMD chips. KVM also cleverly utilizes the kernel scheduler and memory manager, which has almost no impact on other modules in the operating system. As a result, KVM was added to the Linux kernel in less than three months after it was submitted last fall.

Morton said that if the added code comes from an unknown developer and a newly created company, it will bring some risks because both of them may disappear, as a result, no one is very familiar with the code. Considering the independence of these codes, other developers can easily remove them from the kernel even if their development is terminated.

Even if codes like KVM are put into the kernel, it will take a year or two to add them to the two major enterprise Linux versions, Red Hat Enterprise Linux and SUSE Enterprise Server. (For the Red Hat Fedora and Novell OpenSUSE community release versions, the kernel is updated faster .) This is mainly for extensive testing and preparation for technical support. Many enterprise users are satisfied with the stability of the Enterprise Edition Linux, rather than willing to use the latest kernel with the most features.

In any case, Linux is at the forefront in the competition for various operating systems, because a large number of developers are adding various new features to the kernel. These code contributors may be for fame or curiosity, and sometimes for salary. In the last 28 months, a total of 11 new kernels were released, with the number of personally identifiable contributors increased from 479 to 838. According to general estimates, every developer with a name in the code receives an average of about three to four people for help. This means that about three thousand people are involved in each iteration of the Linux kernel.

The development of Linux depends on this volunteer community. although kernel maintainers, who lead the development of Linux subsystems, are paid by some companies, such as Google, HP, IBM, Novell, and RedHat. Because of this community, Morton will say there is no "absolute compromise" between development speed and reliability, because only new features can be added to the kernel as soon as possible, in order to be better tested before being added to the commercial release.

However, such development still has defects compared with code from commercial companies. "I don't want to call this situation 'unpredictable ', but it does not guarantee a delivery date," said Hohndel of Intel, "Because once the Linux code is compiled, it will be submitted."

In the next two to three months, Torvalds will release the 2.6.24 kernel. Many of the new features included in this version were developed and tested by hundreds of new faces who did not participate in the development of the 2.6.23 kernel released this month. It is unpredictable how many functions of the new kernel will be added to the commercial tested Linux release. This is really not the product "road map" that people usually refer ". But at least till now, this development method has not yet moved in the wrong direction.