Postfix local delivery and POP/IMAP

The SMTP protocol requires that the MTA that receives the mail be responsible for sending the mail to the final destination, which may be the mailbox of the local system or other hosts on the same network, this process is called delivery ).
This chapter discusses the process of posting a local email to a mailbox using Postfix and how POP/IMAP Serer can access the mailbox. Many users often mistakenly think that the same set of software is used for receiving and sending emails. The IMAP and pop protocols allow users to remove emails from their mailboxes, while Postfix is responsible for placing received emails into their mailboxes. That is to say, the POP/IMAP service is provided by software other than the Postfix. There are many software that provides POP/IMAP services, including Popper and Wu IMAP kit.
Before discussing the delivery process of local emails, let's clarify the definitions of "local", "foreign", and "virtual" emails and the related MDA.

Postfix shipping agent

The Postfix determines whether to accept the email Based on the email address and how to select the appropriate MDA for subsequent shipping tasks. Postfix receives emails from three domains: local, relay, and virtual. Their definitions and related MDA are as follows:

Local email
If the mail terminal is one of the domains listed by the mydestination parameter, the Postfix regards it as a local mail, and the local MDA (or other programs you specify) executes the shipping task. The recipient of a local email must have the user account of the local system (the host where the Postfix server is located) or its name is defined in the alias file (traditionally/etc/aliases ). The local mail will be delivered to the system's mail storage directory (usually/var/spool/mail/), or the mail file in the personal home directory (~ /Mail /).

Forward email
If the mail terminal is one of the domains listed by the relay_domains parameter, the Postfix regards it as the forwarded mail, and the relay MDA executes the shipping task. Generally, a postfix can be used only when the Postfix is used as the email gateway of the local area network and there are other domain email servers on the same network. That is to say, the so-called "forwarding" usually refers to other hosts on the same local area network, while relay is actually a copy of smtp mda, it was deliberately designed to be especially suitable for sending messages to hosts on the local network.

Virtual domain email
An email server generally serves only one canonical domain. To serve multiple domains at the same time, an additional domain is called a virtual domain ). Virtual domains are delivered by virtual MDA. Based on whether the user has a system account on the server, the email in the virtual domain can also be divided into "virtual mailbox" and "virtual alias. The recipient of a virtual mailbox does not have a system account, and each virtual mailbox domain has its own mailbox directory (Mail Spool). All virtual mailbox domains must be listed in the virtual_mailbox_domains parameter. On the other hand, the recipient of a virtual alias domain can have a local or non-local SYSTEM account, and the Postfix will rewrite the Receiving address of this type of mail and deliver it to smtp mda (if the new address is not a local domain ), or return to the receiving Queue (if the new address is a local domain ).

Email format

When the Postfix is delivered to a local email, the email content is sent to the appropriate mailbox on the Postfix system. The two most common mailbox formats are the traditional mbox and the newer maildir. Both use common files to store mail content. The difference is that the internal organization of files is different. In Postfix, when you set any mail file or directory parameters, if you add a/symbol at the end of the path, it means you want to use a mailbox in maildir format.

Mbox format

Traditionally, the UNIX system stores all emails of the same user in the same file. A mailbox format like this is usually called mbox. The first five characters of each email in the email file must be "from ". Traditionally, for convenience, we usually write it as "from _", the following underlined characters emphasize the existence of spaces. Do not confuse the "from" used to separate emails in the mbox file with the "from:" field in the mail header. The last line of the email in the mbox file must be blank. Therefore, a line of space followed by a from _ can be considered as the beginning of the next letter.
Before Postfix writes an email to the mbox file, the from _ text line is created using the sender address on the envelope and the date at that time, and the string of the line is written to the end of the mbox file, then enter the email content. If the Postfix finds that the mail content itself has any line of text starting with "from", it will add a> symbol at the beginning of the line, avoid the text line being mistaken for the beginning of the next letter.
When the POP/IMAP Server reads emails in the mbox file, the first step is to scan the file content and find the from text line that indicates the beginning of the mail. When reading the mail content, if you encounter the next from _ text line (or the end of the file), you can conclude that the current mail has been read. Some pop/IMAP servers will take the initiative to restore the original state of "> from", but some will not.
Since Postfix and POP/IMAP server may access the same mbox file at the same time, all of them must use the File Locking Mechanism to ensure access. Before a local email is delivered to a local email, you must lock the file before writing the email content to the mbox file. Postfix supports multiple locking mechanisms, depending on the system platform. Use the postconf-l command to check which locking mechanisms your system provides for Postfix:
Postconf-l
If you want to know the details of the various locking mechanisms listed by Postfix on your system, please tell man the name of the locking mechanism:
Man folck
If your system platform supports flock and fcntl, you can find their online instruction files, because both are functions provided by the operating system or function library, the dotlock mechanism supported by any system platform may not be able to find the instruction file, because dotlock is only an unwritten protocol between programs and does not require additional function libraries. The principle of dotlock is very simple. You can give an example to illustrate it clearly. Assume that the Postfix needs to access the user1 email file. It must first check whether there is one in the same directory of the file. user1.lock file. If it exists, it indicates that the user1 email file is currently occupied by another process; if. if the user1.lock file step exists, the Postfix will generate one to let other processes know that the user1 file is currently in use. After the Postfix closes the user1 file, you must actively remove the. user1.lock file so that other processes can use the user1 email file. The disadvantage of the dotlock locking mechanism is that it is not mandatory (any process can not check whether user1.lock exists and directly access the user1 file), and the efficiency is not good.
Generally, you do not have to worry about the details of the locking mechanism or the types of locking mechanisms supported by the system, because postfix can automatically make the best choice.

Maildir format

The maildir mailbox format is different from mbox in that it uses a directory structure to store emails. Maildir is designed to solve the possibility of mbox format and file lock problems. For example, if the system crashes before the email content is fully written to the mbox file, only part of the content may be in the mailbox. When the system resumes operation, when MDA writes emails to the mailbox, the new content will be followed by the previous incomplete content, resulting in problems.
Another drawback of mbox format is that when POP/IMAP server and SMTP server attempt to enable the same mailbox at the same time. If the two parties do not use the same locking mechanism, the mailbox file may be damaged. As mentioned earlier, there are several file locking mechanisms, but not all mail programs use locking mechanisms. However, if the maildir format is used, the file protection lock is not used, because each mail is stored in a separate file. Therefore, you cannot access the same file at the same time by using an unused email program.
A maildir directory contains three subdirectories: TMP/, new/, and cur /. These subdirectories and their upper directories must be located in the same file system. Normally, they should be placed in the mail directory of the user's main directory.
The email file in the new/directory is a message that the MDA has been delivered but has not been read by the user. The modification time of the file itself is the time when the email is received. The email file usually contains emails in RFC 2822 format and does not require "from _".
After you read the email, the email file will be transferred to the cur/directory. The tmp/directory allows the MDA to store the mail content as a file. After all the files are written to the file, the mail file will be moved to the new/directory.

Should I select mbox or maildir?

There is no simple answer to this question. Which mailbox format is the most suitable for you depends on many factors. The advantage of mbox format is that it is supported almost all over the world, but it is precisely because it has a file lock problem that leads to the emergence of maildir format. The maildir format is also quite questionable in terms of scalability, because some file systems may not be able to cope with too many Mail Files. In terms of efficiency, the two formats have their own advantages and disadvantages: When searching, accessing, and deleting a specific email, maildir is faster; but in terms of the delivery efficiency of MDA, it may be faster to directly put the email content into text (in mbox format. In fact, your choice may depend on the POP/IMAP server you use. If the POP/IMAP server you set up only supports the maildir format, you obviously have no choice. Postfix supports both formats, so you only need to consider other factors. If your environment makes you feel embarrassed, we recommend that you test the two formats, try to experiment with the actual running environment and workload as close as possible, and make a choice based on the experiment results.

Body injection for local emails

If the domain part of the recipient address is one of all the domains listed in the mydestination parameter, the Postfix will send it as a local email to the local MDA for delivery. You can list multiple domains in mydestination at will, but some local users will receive emails from all domains. For example, if oreilly.com, ora.com, and oreillymedia.com are listed in the mydestination parameter at the same time, the mail is sent to the kdent@ora.com, kdent@oreilly.com, or kdent@oreillymedia.com, and finally to the same local mailbox. To avoid accepting emails from unknown users, the names of all local recipients must be listed in the table indicated by the local_recipient_maps parameter. The default value of this parameter is to point to the password file and alias table of the UNIX system, so you usually do not need to modify it.
When you check the personal name of an email address, the Postfix first checks the alias table. If a matched alias is found, the new recipient is assigned the corresponding name of the alias and the email is re-submitted as a new email. Otherwise, the email is sent to the user on the system. The Postfix first checks whether the local user has set his/her own. forward file. If yes, It resends the email Based on the set content. If no cleavage exists, it is placed in the user's mailbox.

. Forward file

The. forward file allows you to set your own Alias .. The forward file is in the same format as the RHS-VALUE part of the alias file, or even loose. For example, the RHS-value of an alias file can have multiple values separated by semicolons. The. forward file also follows the same conventions, but it also allows you to write values in different rows.
. Forward file ownership must be the recipient's system account, which is usually placed in the user's main directory. You can use the forward_path parameter to change the storage path of the. forward file. Postfix is provided for the following eight variables to indicate the storage path of the. forward file. The actual values of these variables are determined by the system environment during delivery:
$ User: the recipient's account name (Source:/etc/passwd)
$ Home: the recipient's home directory (Source:/etc/passwd)
$ Shell recipient Shell
$ Complete email address for the recipient
$ Extension: the recipient's name must be extended (not necessarily) and separated by a separator such as "+" and "person's name. Take user1 + labs@example.com as an example, $ extension equals user1.
The Domain Section of the $ domain Receiving address.
$ Local: the complete name of the recipient address (including the extension part-if any ). Take user1 + labs@example.com as an example, $ local equals user1 + labs.
$ Recipient delimiter the delimiter (usually +) between the person's name and the extended part of the recipient address. If you have added support for a non-standard. forward file, you can set it as follows;
Forward_path =/home/$ user/. forward/home/$ user/other_forward

Alias shipping operation

When a command or file is specified for an alias file, the Postfix must first change the identity you have to execute to the alias file owner, and then execute the command with the permissions you have for this identity, or write the email content to a file. The only exception is that when the alias file owner is root, the Postfix uses the account specified by the default_privs parameter (default value: Nobody)

Mailbox shipping operation

When Postfix delivers a mail to a local user, it must write the mail content to the user's mailbox on the system. The default Postfix mailbox format is mbox. When you install Postfix, it determines the location of the mail storage directory based on the UNIX platform type you use. The mail_spool_directory parameter can be used to specify a directory other than the default directory. The specified directory path affects the mail format selected by pstfix. For example, if you set it as follows:
Mail_spool_directory =/var/spool/mail
This indicates that the Postfix should use the mbox format to store the email in the/var/spool/mail directory. If you want to use maildir format, you must add a/symbol after the Directory Name:
Mail_spool_directory =/var/spool/mail/
You can also require Postfix to put the email in the user's home directory. Set a relative path to the home_mailbox parameter to indicate which file you want to use as the email address:
Home_mailbox = mbox
Append a/symbol after the path name, indicating that the Postfix should use the maildir format shipping program:
Home_mailbox = maildir/
This causes the Postfix to deliver the mail to the maildir/subdirectory in the user's main directory.
Note: When the maildir format is used, Postfix usually automatically creates necessary subdirectories and files-if the user's identity has sufficient permissions. However, for security reasons, if the permission mode of the upper-level directory is 775, the local MDA will not create any additional files or directories.

Pop and IMAP

After you enter the mail content in the Postfix, you still need a channel to read the mail. Many sites provide POP/IMAP services so that users can obtain their emails over the Internet. In most cases, postfix can work with POP/IMAP servers without any special settings on both sides. Of course, the premise is that both sides must use the same email format.

Comparison of pop and IMAP

For users who cannot always maintain network connections, the POP protocol is ideal because it allows users to connect to the mail server, take away all emails, and then cut off the network connection. You can read emails offline because all emails are already on your computer. Most pop client software allows you to choose whether to clear the mailbox on the server after removing the mail. If the mailbox is not cleared for a period of time, emails will gradually accumulate and occupy more and more server disk space. The pop software is quite easy to design, but the biggest problem is that your emails are not necessarily placed in the places you need (imagine if you have more than one computer ). In addition, Pop does not support multiple mailboxes, and requires you to download the complete email before reading the email. If you want to first read the subject of all emails and then decide which emails to download, pop is powerless.
The IMAP protocol is designed to overcome the disadvantages of pop. IMAP keeps all emails on the server. You must first connect to the server before reading the email. After going online, you can remotely perform any control and management actions, just as if all emails are stored locally. Because all the actual actions occur on the server, whether the user has a computer at home, a workstation in the office, or a notebook on the computer during a business trip, you can see the same content. In terms of functionality, IMAP is more powerful and more flexible than pop. You can also read the emails offline, have multiple mailboxes, or even download only the correct answers to the emails before deciding whether to get the rest of the emails. Therefore, if you find that an email contains a large file that you are not interested in, you must wait for a long download process.

Postfix and POP/IMAP servers

Cooperation between Postfix and POP/IMAP server is quite simple. Each time the Postfix receives a local email, it is stored in the mailbox. When the POP/IMAP server receives the user's request, it only needs to retrieve the email from the same mailbox. Both Postfix and POP/IMAP server must agree to use the same mailbox format and the same locking mechanism. Postfix can be used with any standard POP/IMAP server that uses the traditional email format. You may want to adjust the mail_spool_directory parameter, but for most pop/IMAP servers, you only need to install and start the server according to the standard installation instructions. For POP/IMAP servers that do not support traditional mail formats, you can use the "local mail Transmission Protocol" to deliver mails, And the POP/IMAP Server saves the mails to the mailbox.

Local Email Transfer Protocol (LMTP)

Some pop/IMAP servers use non-standard mailbox formats. Obviously, it is unreasonable to require MTA such as Postfix to recognize multiple exclusive formats. Therefore, we need a channel that is irrelevant to the mail format, so that the mail can be transferred from a mail program to another mail program on the same machine, this channel is LMTP. LMTP is short version of SMTP. The LMTP server also has the right to decide whether to accept or reject emails. However, the LMTP server is not responsible for handling emails that cannot be delivered immediately.
When the MTA sends a multi-recipient email to the SMTP server, if some recipients cannot accept the email for some reasons, the SMTP server is responsible for putting the email into the queue for the next transmission, the LMTP client is declared that the shipping task has been successful. However, the LMTP server is not responsible for this, that is, the delivery status of each recipient must be individually returned to the LMTP client. For undeliverable recipients, their emails are placed in the LMTP client queue, and the LMTP client is responsible for subsequent processing.
LMTP conversations may occur between different email subsystems on the same machine or between different machines on the same LAN. However, if there is a wide area network between the two parties, the LMTP is not guaranteed to be reliable, because the protocol determines whether the email is delivered smoothly based on the response speed. SMTP has been found to have a synchronization problem between its receiving and sending systems, occasionally leading to repeated delivery of mail. The two sides of the Langu LMTP dialogue are located at both ends of the Wide Area Network. I believe the problem will be more serious.
Note: In addition to sending the message to a non-standard email address, LMTP enables the mail manager to build an easy-to-expand and reliable email system. For example, one or more Postfix servers can be set up for websites with a large number of mails to receive mails from the Internet and then deliver the mails to multiple LMTP backend systems. When the number of emails increases, you only need to set up several front-end or back-end systems.

Postfix and Cyrus IMAP

Cyrus IMAP is a server dedicated to providing POP/IMAP services. You do not need a system account. If you want to set up an email server for existing users in the system, consider other simple POP/IMAP solutions, such as Qualcomm's Qpopper (only pop function ), or IMAP toolkit of the University of Washington, neither of which requires any special settings in Postfix.
Cyrus IMAP provides two LMTP delivery channels: Unix-domain socket and TCP socket. You must know which channel Cyrus IMAP uses to set the Postfix. If Unix-domain socket is used, both Postfix and Cyrus IMAP server must be on the same machine. If TCP socket is used, Postfix and Cyrus IMAP server can be on the same machine, it can also be separated from different hosts on the local network. The LMTP shipping parameter of Postfix is defined in the transort_maps parameter in the main. cf configuration file.
To enable Postfix to receive the email sent to the local Cyrus IMAP server, the domain name of the recipient address must be listed in the mydestination parameter. Then, you must request the Postfix to send the email to Cyrus IMAP. Use the mailbox_transport, local_transport, or fallback_transport parameters to let the Postfix know how many local shipping procedures are required before the mail is sent to Cyrus. If you use local_transport or fallback_transport, write all Cyrus user names in a query table referred to by the local_recipient_maps parameter to avoid Postfix rejection of Cyrus users' emails.
Mailbox_transport
The email is first sent to the local MDA. The local checks the alias file and the. forward file, and expands the recipient's email address. The processed mail is forwarded to the LMTP client of the Postfix, which is responsible for the delivery to the LMTP server.
Local_transport
When LMTP is used for local transmission, the email will be directly sent to the Postfix LMTP client. The local MDA originally responsible for processing the local email will not intervene at all. Therefore, both the alias file and the. forward file have no function.
Fallback_transport
When LMTP is used for alternate transmission, the Postfix will first send the mail to the local MDA for processing, and execute the expansion of the alias file and the. forward file. If the recipient has a normal system account, the mail will be delivered to the appropriate mailbox on the system; if the recipient does not have a system account, the mail will be handed to postfix LMTP clenet, it is responsible for handing over the email to the LMTP server. When your mail server serves a group of users with its own system account and receiving emails from another group of Cyrus IMAP Server users at the same time, the LMTP shipping operation must be performed using the fallback_transport mechanism.
Use the following format to set the shipping mechanism you selected:
Xxx_transport = service: socket_type [:/path/to/socket]
For LMTP shipping operations, the service must be LMTP (representing/etc/Postfix/master. CF) The LMTP service defined by socket_type must be either Unix (Unix-domain socket) or iNet (TCP socket. The default value is Inet, which indicates that the LMTP server set up by Postfix uses TCP socket to receive messages. Therefore, if the default value meets your needs, you only need to set it as follows:
Local_transport = LMTP
If local_transport and Unix-domain socket are used, the following settings should be made in/etc/Postfix/Main. Cf:
Local_transport = LMTP: Unix:/var/IMAP/sockte/LMTP

Example of combination of Postfix and Cyrus IMAP

Cyrus IMAP uses the spring rain SASL function library to verify the user's identity. Therefore, you must first build and install the Cyrus SASL function library before you can successfully build the Cyrus IMAP server. In addition, Cyrus software requires at least versions of Berkeley DB 3 and later. If your system
The Berkeley dB on is older than the third edition. You may need to update the entire system. Mixing different versions of Berkeley dB on the same system may cause hard-to-trace problems. If you must upgrade the database function library, we recommend that you rebuild all the Berkeley dB software you need to use so that all the software on the system can use the same version of the function library.
For example, we have set up a postfix server that can receive example.com domain emails and run Cyrus IMAP server on the same machine. Assume that the accounts of all email users are already built in the Cyrus SASL database, but there are still a few shell accounts on this machine. Although these accounts will not be used to receive emails, we hope they can still be used. the foward file and alias file can be forwarded to the correct place for the system to generate emails to the root account and emails sent to the postmaster alias.
Obviously, we cannot directly deliver all emails to the Cyrus IMAP Server (so we should exclude local_transport), because we need the alias file and. expand the forward file (only mailbox_transport and fallback_transport can be selected ). However, because the Shell account is not used to receive emails, the ideal choice is mailbox_transport. We should point this parameter to lmtp mda and make sure that the LMTP service in master. Cf can deliver the mail to Cyrus IMAP server. Follow these steps:
1. Install Cyrus IMAP on the system. Check the Cyrus configuration file (usually/etc/Cyrus. conf), determine its service channel is Unix-domain socket, and write down the location of the socket file. You should see something like the following:
Services {
# Add or remove Based on preferences
IMAP cmd = "imapd" Listen = "IMAP" prefork = 0
POP3 cmd = "PoP3D" Listen = "POP3" prefork = 0
# LMTP is required for delivery
Lmtpunix cmd = "lmtpd" Listen = "/var/IMAP/socket/LMTP" prefork = 0
}
The Line "lmtpunix" indicates the correct position of the socket file.
2. Follow the instructions in the accompanying Cyrus file to set up a system account on the server for the Cyrus IMAP server.
3. Check the content of/etc/Postfix/master. CF and confirm that the LMTP service configuration conforms to your system environment. Generally, you should set it like this:
LMTP Unix--N--LMTP
If your Postfix is installed in the default mode, the above line should already appear in your/etc/Postfix/master. cf configuration file. The fifth column indicates whether the lmtp mda is in the chroot environment. In this example, this column is set to N because it must be able to read the socket file of the Cyrus IMAP server.
4. Check main. CF and make sure that the domain listed in the mydestination parameter contains the email receiving domain you want. You can list them directly.
5. mailbox_transport is required to use the LMTP service specified by Master. CF and direct it to the Cyrus IMAP socket file you noted down earlier.
6. Reload Postfix