Interview: Put Samba above POSIX

Interview: Let Samba go beyond POSIX-Linux Enterprise Application-Linux server application information. The following is a detailed description. DeveloperWorks talked to Tridge about his work at IBM, Samba 3.0 status, and his views on smbd rewriting.

If you have the honor to attend the Samba eXPerience conference in G (o) ttingen, Germany, you may have heard of Samba founder Andrew Tridgell's speech on "Exotic Filesystem Backend" (see references to get a link to the Samba experience homepage ).

What exactly does Tridge have to say about the backend of an external file system? The actual situation is that since January this year I was employed by the IBM armaden Research Center (Almaden Research Center, see references for links to the Almaden lab page, the Australian hacker has been committed to bringing Samba beyond the POSIX field, and is also committed to identifying what needs to be done to enable Samba to support new file systems such as XFS, ext3, and Storage Tank. The conclusion is to completely rewrite Samba's smbd code, which has become his "new favorite" project.

DeveloperWorks: What kind of work do you do in IBM?

Tridgell: I am not only employed by IBM AS A Samba technician. Actually, I joined Almaden Research Labs, which is doing a slightly more proactive job than the upcoming product. My work is not limited to Samba. Although I am currently engaged in a specific research project that is a storage project connected to a network, in this project, we are studying next-generation NAS solutions and are integrating the project with some external file systems being developed by IBM. This is a very interesting job, and it gives me the opportunity to work for a long period of time. When I was working in Quantum and VA [Editor of the two companies Tridgell previously worked.], I often go beyond the limits of Samba's core structure. To deliver products within three or four months, we do have to bypass these core limits, which means we have to make a rough compromise, especially in the Access Control List (ACL) and lack of support for transactions such as file streams, or make some trade-offs in user and group management. These are only effective trade-offs, because we do not have the strength to really say: "Well, let's rewrite the entire Samba ".

In research institutions, I have a longer perspective. I can study what to do in the next year or two, and point out: "Where do we want to take the next generation of Samba?" And "What does it look like if we want to solve these problems correctly ?". I can solve these problems seriously. In fact, many of my current work has been inspired by the problems I encountered while developing products for VA and Quantum. Therefore, other companies that produce NAS products will eventually benefit from this work.

DW: are you discussing the Samba 4.0 type problem?

Tridgell: temporarily, may include 4.0. It may also be 3.1 or 3.5. This does depend on the progress. At present, this work is indeed in its early stages. So far, it has taken me about a month to rewrite the kernel. Compilation has not been performed yet. There is still a long way to go before compilation. I hope that, at the Samba experience conference in Germany, these core changes can be compiled in practice so that I can start to let other developers study them. So far, all I have done is to start the core structure change. I have demonstrated these changes within the Samba team and have been approved by other Samba core developers who say they appreciate these ideas. So far, everyone is very enthusiastic about them.

DW: Can you tell me the changes?

Tridgell: Historically, Samba is a gateway to POSIX. It provides access to POSIX-like systems. This is useful at 01:10, many people install POSIX-like systems on their servers-such as Linux, Solaris, AIX, and other systems-to provide these POSIX servers with high-quality CIFS [public Internet file systems, common Internet File System] access is very useful. POSIX has some limitations. It is difficult to map some things from the Windows world to the POSIX world. A typical example is file stream. In the Windows world, files can have multiple associated datasets. However, in the POSIX world, a file has only one data stream associated with it. You can edit the bytes in the file from 0 to N, and allow two-dimensional access in Windows.

Now, we haven't supported file streams in Samba for a long time. We are lucky to do this without making any mistakes, this is basically because almost all applications on Windows do support servers without file streams, because legacy Windows servers (such as Windows 9x) do not support file streams. Therefore, applications often support systems with or without file streams. Microsoft Office stores files on Samba servers in a slightly different way than on Windows 2000 servers. However, this will change over time. For example, I would not be surprised if a file stream is required for future Office versions. I wouldn't be surprised if more applications are dependent on such features.

Therefore, we need to be able to terminate close relationships with POSIX and support more advanced file systems available on Linux, such as JFS and XFS, in Samba. There are many interesting file systems that are being developed for Linux. Their functions far exceed what is defined in the POSIX specification of the core file system. We hope to use these file systems within Samba.

DW: So what file systems are you focusing on?

Tridgell: Storage Tank File System (see references for links) is a file system I am focusing on, a SAN-based file system.

However, the initial work I am currently working on is not specific to Storage Tank. It is a very universal task. My first file system backend is an extraordinary file system: it is a CIFS backend. Therefore, you have a Samba Server that communicates with another SMB [Server Message Block, Server Message Block] Server (such as another Samba or Windows Server, samba uses remote file access to another SMB server as its backend. We never expected anyone to do this in the production system. It is hard to imagine a situation where you want to have the "CIFS gateway on CIFS ", however, the CIFS backend does provide an ideal experimental bed for developing core functions of external file system backend in Samba, because Windows 2000 backend has all the functions we might want, because it has all the functions supported by CIFS, It is CIFS. Therefore, for example, it supports native nt acl because it is a native NT.

Therefore, by using this backend in Samba as the backend of the experimental file system, we can develop the core structures required to support all these extensions. Once it is provided to a file system that is not too "external", such as ext3, XFS, or Storage Tank, they will have a subset of available functions, however, we know that the core structure will run normally. If you want, we will have a perfect backend.

DW: Microsoft has been talking about making some fundamental changes to its File System-for example, creating a database-oriented File System, such as an Object File System ). If Microsoft has completely changed its file system, do you expect the Samba project to be developed to support this change?

Tridgell: As a plug-in for the Samba project, I hope Samba will eventually disappear because we hope it will no longer be needed. But strictly speaking, this is really just a joke. For a long time, Microsoft has been switching to a different Network File System Protocol. For example, some people think that WebDAV may become this protocol, and Web will become a file system. This seems to no longer exist as a possibility; it does not have the rich features that Microsoft applications want.

I have not really participated in discussions about database alternatives for this file system. At this time, it seems extremely risky. I am not surprised if Microsoft changes its core file system protocols at some point, because these protocols, along with CIFS/SMB protocols, also include a lot of legacy "burdens ". There is a large amount of content that is meaningless to modern platforms, but they have to be carried on as part of their core protocols inherited from themselves or elsewhere. Therefore, starting from scratch is certainly very attractive, but again, this will be a huge project for them. Moreover, we have not seen any signs that the job is about to begin. But again, we may not see it. We do not discuss the CC list in Microsoft.

DW: Do you think that some of your ongoing research will have applications that exceed CIFS?

Tridgell: Samba itself is slightly out of the CIFS range. When people do not want the file and print services at all, they install the Samba component. For example, we can see that the company installed the Winbind daemon, which is a newer Samba component that provides domain integration-allows your Linux desktop to be integrated with existing Microsoft domains, it does not provide any file or print services at all. This is an evolving Samba usage. Therefore, the Samba role is not limited to file and print.

If a new Protocol appears, will the Samba team start to work on the protocol? Well, it does depend on how closely the agreement is associated with our existing work and whether it is appropriate for us to do the work. For example, when Active Directory appears, it is very closely associated with the existing MS-RPC protocol and the existing domain infrastructure. Therefore, it is clear that this problem should be handled by the Samba team, and we have handled this issue in the past 18 months and have worked very well.

Microsoft is also involved in the NFSv4 field. Although NFSv4 shares many common concepts with CIFS and SMB, it does not share enough common technical details and does not have any public code, therefore, the Samba team will not participate in any of these activities unless they are interested.

DW: You said: you learned a lot during your previous years in the NAS project. What are the biggest problems you are working on in Samba?

Tridgell: Samba is trying to solve the general problem of semantic ing. It tries to map the POSIX semantics to the CIFS protocol. Therefore, Samba consists of many widgets that provide this ing. A typical example is to map case sensitivity to case insensitive. Windows expects a server that is case-insensitive, while POSIX systems are case-sensitive.

From case-sensitive ing to case-insensitive ing, it seems very insignificant, but in fact it is difficult to effectively perform this ing. Given a file name, it is difficult to prove that the file does not exist. If the file exists, you can guess exactly what the case-sensitive name is. In fact, most of the time, it is just the original string sent from the client through the line. However, it is very expensive to prove on POSIX that a file is definitely not in any combination of upper and lower cases. Now, the POSIX system does not have any attractive method to say: "Please execute this open in case-insensitive mode." In this way, in Samba, we have a variety of basic structures to provide ing from case-insensitive to case-sensitive data. However, this infrastructure is deeply embedded into all parts of Samba code.

These days, we have seen a file system with non-POSIX semantic capabilities on Linux and other UNIX systems. For example, they may have some methods that may require the file system: "Please operate in case-insensitive mode ". We need to instruct Samba: It's not necessary to do all this work now, because the file system will do this-and the efficiency is much higher than what Samba may do in the user space. However, it is very important to remove all the code from Samba. Therefore, I realized that to do this, we need to rebuild the Samba structure to make it more modular, this structure allows you to use a backend that is completely isolated from POSIX without paying the POSIX ing cost. This is indeed the core of my current attempt to rewrite it.

DW: It sounds like a completely new Samba architecture.

Tridgell: This is a major change. It does only change the core smbd function. For example, it does not change the domain functionality in MS-RPC code, nor change our user and group ing code. Therefore, it only affects the part that is always at the Samba core, but as I guess, this Part may currently account for 30% of Samba code. However, the rewriting made in the 30% code is very meaningful.

DW: I also want to ask you about the latest situation of Samba 3.0. I think it has been in alpha testing since November 2001.

Tridgell: it has been in alpha testing for a long time. In fact, in a sense, alpha is not appropriate because many companies are actually using it for production. I know that at least two of the vendors that are using their tools include Samba 3.0. It is being deployed widely. We originally hoped that the final release was completed by the Samba XP conference this year, but it was not actually completed. There are still some documents to be compiled and some QA work to be completed, but the code status is indeed very good. For some purposes, it is much better than our stable release because it can get a lot of QA from these companies in production. Especially when I was working at Quantum, they had a very good QA Team who invested a lot of work to make the 3.0 code library stable as a file server. This is especially true for large domains or under high loads. This work means that Samba 3.0 is actually much more stable than people may expect from alpha testing code.

DW: So, large domains and high loads are what Samba 3.0 is most interested in?

Tridgell: Right. For example, if you want to support the Active Directory. Many companies need active directories because their infrastructure is built around the Active Directory model. In this case, 3.0 is your only choice. It is the only backend that supports Kerberos Authentication and LDAP (such as LDAP) with active directory servers. It is also a more flexible authentication system. If you need a complex authentication backend, you do need Samba 3.0, and 3.0 has much better character set processing capabilities for internationalization. Now, it uses Unicode instead of Multi-byte encoding to communicate with Microsoft clients online. This allows it to support non-ASCII character sets in a much clearer and much better way.

DW: So, what are the behaviors of 3.0 really like alpha testing code? Who doesn't want to use it?

Tridgell: the document preparation has not been completed yet. Management tools are also rough. Some similar things are not completely completed. Assume that I have published it as 3.0. A few years ago, when I was a release manager, my publishing work was often much faster than they were. Each release manager often has his own style... Jeremy [Allison] is the former release manager, and Jerry [Carter] just took over a few months ago. Jerry and Jeremy tend to be more conservative, which may be a good thing.